They eventually go away, after 10 turns I believe. There's nothing to do, as far as I know.