I object to you calling Lord Elgin a "filthy brigand". The marbles were not stolen, Lord Elgin obtained permission to move them. As others have pointed out, in the context of the time his actions may well have saved the marbles from further destruction, since the Turks and Venetians had already caused a lot of damage and there was no reason to think things would improve.
Overall I think it's fair that the Elgin marbles should be returned to Greece, now that the country is able to protect them. They are clearly a Greek treasure and it would be a nice gesture if the British Museum would return them. Perhaps in exchange, a deal could be made between the BM and museums in Athens. Museums around the world often send items to each other for temporary exhibitions. The items are then sent back when the exhibition finishes, usually after a month or so. If the British Museum were to return the Elgin marbles, it might be a fair compromise that in exchange a Greek museum could send a few other items for a temporary exhibit, and maybe the British Museum could also send other items to Greece. It would be a kind of ongoing deal, basically involving lending each other items for temporary exhibitions in both countries.
For example the Chinese sent part of the Teraccotta army to London a few years back for a temporary exhibition at the British Museum. After a couple of months, the exhibition finished and the items were returned to China. Maybe a similar kind of deal could be done with Greece. The Elgin marbles would stay in Greece, but other items could be (temporarily) loaned for special exhibitions.