As a taxpayer, you have the right to challenge the IRS and have your position listened to. This is the fourth item in the Tax payer Bill of Rights. The IRS wants you to know all your rights when you need to work directly with the government agency on a tax issue. You can object to an IRS decision, you can provide more documentation to an open issue, the IRS has to respond to your questions quickly, and they have to look over your documents in a timely manner.
More about the your right to challenge the IRS:
When you get that letter the mail that your return had a mathematical error and there has been a change in your return amount, you have 60 days to contest the change. You should provide records that back-up you position. And you have a right to call the IRS with the number listed on the letter you received.
If the IRS does not agree with your position, they need to notify you promptly. The information will come in the mail. You can then open a case with the US Tax Court within 90 days to continue your objection to any adjustment. If you are outside the US, you have an additional 60 days.
You can also raise objections during an audit of your return. If the conclusion of the IRS does not go your way, you have the right to petition the US Tax Court once again.
If the IRS is going to take more serious action against you, like going after old debt, or taking a lien against property, then you can have your case heard before an Office of Appeals, an independent body.