If the Temple Keeper leaves, gets kicked by the leader, or the Alliance Leader picks a different player for this role, the Temple is removed from their city and is in "Limbo", preventing any Temple upgrades, spell charging or spell casting. The new Temple Keeper can rebuild the Temple in their city straight on the same previously achieved level (upgrades are not lost in this event). While rebuilding, spells cannot be charged or cast, as a cooldown for moving the Temple. Gold cannot be used to speed up this process.
The level of the Temple is gated by the level of the Temple Keeper. If a level 30 Temple is put into limbo then the new Temple Keeper needs to be level 30 or greater to build it.