I read that spearmen have an edge defending/fighting on walls in BC. That is might be that issue. I am testing and fixing how units behaive in open field, where prior spearmen did defeated swordsmen and axemen units.
From what I read, historically taking a castle or high walled city required at least 3 times the troops of defender ( without cases when gates where opened by spies or walls taking down by trebueshets or other advanced siege equipment).
It didn't guaranteed a victory though, generals prefered to have at least 5 to 1 advantage in order to take a highly fortified castle.
Personally, It alsways bothered me how easy is to take highly fortified settlements in vanilla, in TW or SS.
There is might be some problem with spearmen on walls in BC, I haven't look at that closely yet.
But defending walls units should definately have a huge advantage especially vs coming up the ladders units.
Using commong sence, if a man, who just climbed up to a wall, and what he sees is 5-6 spearpoints atacking him from different directions, that man will be in rough shape. I am not talking about defenders just pusing him off the walls with spears or other such weapons. In my latest Seljuk campaign, I'd always use trebuchets to make gaps in wall, because climbing up the ladders is very costly ( the way it should be in my opinion)