In many jurisdictions there are zero overheads required operate a business as a sole-trader, no business license or company registration necessary, banking is free in many cases (even without requiring minimum account balances) or has low per-item fees, accounting software is available for free or you can simply do 'cash basis accounting' in a notebook, tax filing can be done for free.
Even with fairly straightforward businesses you can easily end up with $2000 per year of overheads if you're not careful, including: software licenses, domain registration, hosting & email, monthly catch-up with an accountant, statutorily required insurance, company registration and licenses, bank application fees or monthly account fees, yearly filing costs etc. And then out of the blue something could come along and doubles that number, like needing a lawyer.