I always leave a governor, even if its one of the recruited generals (let the settlement have a school and it should work, also sometimes the recruited general is better suited for administration than to put forth a killing machine).
Another thing, its a question of balance, basically I measure how much growth I do need VS the speed of building being built, to not get into a revolt. I have found that up to 5% growth is still desirable. But the main idea is that in the end I have at least aqueducts and all the farming buildings, thus I usually have a pop of 30-35k in a settlement.
Think that more pop = more money, thus sometimes you avoid to build temples and economic buildings to invest on health buildings. But as a rule I destroy the pop growth buildings, when I reach huge city (game mechanics issue), and build more advanced farms (thus reaching a good pop, with squalor around 8%) . The growth measures I use to boost quickly to raise higher level troops, and to gain more taxation money. Not to lead the settlement to a continuous and ludicrous pop of 90k (game mechanics again, not thats unrealistic, or a negative point against the mod)