If you haven't filed tax returns, you may think the IRS is going to become hyper-aggressive. While it is possible, it is more likely that the IRS will file the returns for you, inflating a tax bill. Also, you could actually be missing out on refunds you would have been due.
The Three-Year rule on claiming a tax refund
It turns out that the IRS is only required to pay you "back refunds" for three years. But, if you don't file taxes the IRS has forever to assess you a tax bill. It's done through something known as a Substitute Filed Return (SFR), sent by way of a CP3219N notice.
It's called the "Three-Year Rule", and here's how it applies to you:
The Internal Revenue Code says a claim for a refund has to be filed within three years from the date the return was filed. But what if there isn't a return filed? Unfortunately that doesn't absolve you from the requirement to file your return. The IRS has forever to assess taxes when you don't file, yet you have a limited time to claim a refund.
In order to comply you must file the late returns and incur any expenses necessary to do so, and you will not have the benefit of your lost refund dollars to help. You may still have penalties assessed against you for failing to file, and those will need to be addressed.
Miss the Three-year rule?
You may not be totally out of luck. There is another rule known as the "Two-Year Rule". It works like this: If you paid any taxes in the last two years that you are seeking a refund on now, you may. For example, if you received an SFR for 2007 and paid it off in full a few months ago, you could file a tax return now through something that is technically an audit reconsideration.
UPDATE March 2017: The IRS claims that it has a billion dollars awaiting to be refunded for people who have not filed 2013 tax returns. If you have not filed your 2013 retrun, if you do so by the three-year anniversary of the day your return was due (typically April 15th for US filers, June 15th for expats, or Octber 15th if on extension you can get your refund. Miss these dates and whatever is not refunded, the IRS retains.
Are you worried about criminal charges if you haven't filed taxes?
Is that the reason why you've been delaying? Well don't. The IRS offers something called the domestic Voluntary Disclosure Program. In this program, as long as you aren't currently under audit or criminal investigation and none of your income is from illegal activities such as drug trafficking or money laundering, you can make a disclosure and file your missing tax returns. Then, you can make arrangements to pay (although you don't have to full pay — just make arrangements to settle your tax debt) and you will not have to worry about criminal charges due to unfiled taxes.
The chances of you needing a voluntary disclosure are slim. This is one of those things you should get a legal opinion on.