Would you believe that the Fifth Amendment's right against self-incrimination stems from some rather intense church – state intermingling? Not only that but a church – state mingling so intense that it's one of the primary reasons the United States exists? Yes, it's true!
The story, as best I can determine, goes something like this: the Anglicans, who were part of the official church of England, didn't want the Puritans around anymore. They wanted to give them the boot. The problem was that the only way to know if someone was a Puritan was to examine their consciousness. Due to it being the 1500's and a suspicious lack of Magnetic resonance imaging (MRI) systems, they were plumb out of luck for mind reading. That being said, even if the MRI did exist and you could see inside someone's brain, there's no way to determine what their beliefs are (pro-tip: This is still true to this day).
In order to get the information they wanted, the English government had to get crafty. So what did they do? They made everyone take an oath to support Anglicism. If someone refused to take the oath, they were put in jail with all the other Puritan scum. Genius, right?
Well, not so quick. There was a massive uproar that led to some serious changes for English law. Many of Puritans left England to build a new colony that would eventually become the United States of America, complete with a Constitution guaranteeing certain inalienable rights. One such right was put in place by the Fifth Amendment.
The privilege against self-incrimination thus became part of the English law for most Englishmen in 1641. The restatement of it in the Bill of Rights of 1689 added nothing new. From this source, it entered into the Virginia Bill of Rights and went on to become part of the Constitution of the United States:
There's a lesson to this story: the government can't require someone to take an oath or refuse to take an oath if refusal of which allows for criminal punishment. If the government could require oaths subject to criminal penalties, it would invariably lead to an out-of-control government that operates above the law — the very type of over-reaching English interference that the United States rebelled against.
But wait…the Bank Secrecy Act of 1970 requires you to take an oath
Regardless of historical events, the Bank Secrecy Act of 1970 requires any foreign bank account holders to take an oath if they have foreign financial accounts exceeding $10,000 in value. Failing to take this oath, via the FBAR Form (now FinCEN Form 114), is subject to criminal penalties.
Today, the Third Circuit Court of Appeals told New Jersey couple, Eli and Renee Chabot, that foreign bank account statements were not protected by the Fifth Amendment, and are forcing them to testify against their own interests by handing over any self-incriminating documents to FinCEN (the Financial Crimes Enforcement Network).
Here's what the court said:
"The Chabots contend that the Financial Crimes Enforcement Network’s administration of § 1010.420 evidences the regulation’s essentially criminal purpose. The Chabots attempt to bolster their argument by highlighting the fact that the records that §1010.420 requires account holders to keep are also useful for potential criminal proceedings against these individuals. As the government asserted at oral argument, “bank records can be very important for a lot of things you might want to investigate about a person.”
It's only logical for us to see that the court will agree that there are criminal penalties that the oath is subject to (not permissible under the Fifth), and yes it also noted the alarming invasion of privacy at stake with this statement: “bank records can be very important for a lot of things you might want to investigate about a person," so clearly there must be some 4th Amendment violations as well. Therefore, any sane person could weigh the evidence and come to the conclusion that the court will rule that FBAR penalties violate the Fifth Amendment (not to mention other parts of the Constitution).
The next sentence is, unfortunately, not surprising to those of us who have witnessed the federal court eviscerate civil right protections:
"Just because some of these things have criminal aspects does not mean that § 1010.420’s purpose is essentially criminal."

This is why we can't have nice things.