This is the biggest annoyance for me on the vanilla game, though I've reduced it quite a bit. Mainly, there is a lot of lag and AI units standing around and doing nothing, crowding about, even when the FPS remains high. The main culprit seems to be ladders (more specifically, units crowding at the bottom of ladders and at the top of the walls after climbing up and not doing anything), but there might be other factors at play as well.