Germanicus5 has been working on this problem.
you might like to try out his BAI.
http://www.twcenter.net/forums/showthread.php?t=253132
pathfinding in settlements is very difficult to get right.
Part of problem resides in size of units - The bigger they get the worse the problem.
also the settlements themselves sometimes have paths defined that don't quite match the buildings.
with ladders there's a problem with throughput - getting a flow of units up ladders and onto walls
without them getting in a tangle.
Probably best if you're having problems to put only a single unit up a ladder at a time.
don't order another one up until the ladder is clear.