Van Rompuy told a Brussels conference that if Greece were to leave the euro zone, that would also have geopolitical repercussions in the current standoff with Russia over Ukraine, emboldening Moscow to see Europe as weak.
"Everybody has to realize that once you go out, I would not say it is forever ... but it is for a very, very long time," said the former chairman of EU summits who stepped down last October.
"That decision... for leaving is in some way for generations and generations."
The biggest victims of such an outcome would be the Greek people, he said, but the euro area would also be damaged because doubts would arise if another crisis arose in future elsewhere in the currency zone.
Van Rompuy urged all sides to consider the political and geopolitical implications of such a step and not just the economic and financial arguments.
"I hope we will never have to answer the Grexit question," he added.
