Yes when i was a kid, friend of the family was moving fund (before reporting) to brazil to start a life after retirement.
Divorced the Wife, left her the house and skipped to brazil.
couple of years later HMRC came knocking, dragged his a*s back to the UK, didn't imprison him (this was around 2010) hit him with something like 600,000 pounds in fines, destroyed his relationship with the chick x/x% his age, and since then he's been in the UK repaying this...
Ex wife got a house out of it, she did well, he did well in respect he was banging a 18yr old at 55.
In my home country, it's common to get a partial audit request, ie. Show records for a particular category of expenses for a particular year. That happened to me, but in the end, when my documentation didn't exactly align, it was a simple matter of paying what I couldn't verify... turns out only to be a few hundred dollars lol. No fines, no threats, just accounting. It caused me a lot of stress, but most of it was in my own mind. Keep in mind, I'm talking 100-250k/yr net earnings bracket.
To be fair if someone is making good money, and its not essential they remain in a country (say the UK as an example), then they are an absolute fool for remaining there.
The UK has one of the fairest tax setups for non-residents in the Western World, in addition there's tonnes of countries that are of good standard of living without massive financial outlay where one can reside and pretty much be tax exempt based on legalised structuring (1yr deferred matured, corporate, etc).
For the EU i can't really give any input, I know in Switzerland I pay a handsome sum, and haven't even stepped foot into the country since Covid (non resident-non-citizen- wealth tax), and they literally were about to take me to court for a unpaid tax worth less than the piece of paper that finally found its way to a friend who immediately settled (been bouncing around the world for a year apparently).
It's not so much a fool for not harvesting a tax structure where one can legally avoid exposure to tax, its the stupidity of residing in a place where one has to go through mass accounting and reporting annually.
It's the same with Companies - BVI for example now has to do tax accounting and reporting - it costs more to do than the actually expenditure of holding company (IP) in a year.
** For example, a Belgium neighbour has to do accounting and submissions annually in Belgium, he hasn't lived there for 20 odd years **
Australia now introducing a 45day residency tax retirement -> the accounting for that is going to mind-boggling.