
Originally Posted by
Red Harvest
It is way simpler than all of the above: Until you can build the next level of settlement, you can't get rid of all of the culture penalty. When you conquer the culture penalty starts at 50%. When you upgrade the settlement to the next level most or all of the culture penalty should disappear (some left over building might still give you a pip or two.)
In the interim, you can reduce down the penalty by destroying and/or replacing any buildings made by other cultures, but some things can't be destroyed. So until the settlement is upgraded you will be stuck with a minimum of ~25% culture penalty.