Run away from them? When you see them, hit spacebar to pause and then change direction. If you are slower than them, you may be carrying too much in your inventory, or you may have too many foot troops.
What's in your party? A good way to get money would be to win tournaments, but that's chancy in the later rounds (you can end up with a weapon you have a low proficiency in -eg bow for a melee character). Killing and looting is best, but looters are pretty bad.
What's your character build and all that? You should be able to hammer deserters generally (unless it's around 12 Swadian Men at Arms in early game
). Sea raiders are all right if you have a decent party size.
What difficulty (damage you take, damage allies take, ai difficulty) are you using?