As time passes some buildings will move around toward the center of the island. Build mode will be enabled/disabled every time a player is eliminated, so keep an eye out for the message that let's you know what's going on.