While I am quite enjoying Kingdoms for the time being, on a fundamental level I will have to agree with the OP.
The AI really needs an overhaul, and the interesting bit is that the shortcomings are well known and discussed in the community on a regular basis (all artillery stacks, lack of generals, reshuffling troops that finally rebel, suicidally charging generals etc.).
The main problem is that the AI does not really know how to play the game efficiently. While I'd hesitate to compare Total War with Civilization I have to admit that the latter have difficulty levels where I am basically toast. In MTW2 we have to come up with complicated house rules to keep ourselves from steamrolling the AI.
I can see the economical sense in basing an expansion almost solely on new content, and I believe a lot of people around here, like me, bought Kingdoms with future mods in mind and view the four campaigns as enjoyable distraction for a little while. I'm just waiting for LTC for Kingdoms and then I am back playing the GC again.
At the end of the day, MTW2 is one of the finest strategy games ever made. It's just that out of the box it's got some really fundamental AI flaws that go unnoticed with the vast majority of players, but really screws it up for the hardcore fans that keep playing the game over and over again.
But I don't regret buying Kingdoms, and for any normal player out there it's one meaty expansion, isn't it? And again, I'm still playing the damn game all the time. ain't I.
/Sibben