Congress cuts IRS power as oligarchs hide taxes

Congress cuts – From OffshoreAlert conferences where IRS and Justice officials rub shoulders with offshore tax strategists to Congress’s dramatic cuts to enforcement funding, the story of U.S. tax enforcement is increasingly shaped by a single reality: the people accused of e

By the time the IRS can get a case moving, the clock is already working against it.

That sense of futility hangs over OffshoreAlert conferences—gatherings that for a quarter-century have pulled together investigative journalists. IRS special agents. representatives from the Department of Justice. officials from the Securities and Exchange Commission. CIA and FBI agents. white-collar defense lawyers and accountants from the Wealth Defense Industry. private detectives hired by ex-spouses and business partners trying to recover offshore assets. cryptocurrency players. whistleblowers. bankruptcy attorneys. judges and prosecutors. and officials from Caribbean tax havens.

A few oligarchs show up, too. The networking at cocktail parties matters as much as the formal sessions. And there is always risk. At one event. OffshoreAlert organizer David Marchant told the crowd that a speaker from the previous year had been shot in the head in the Bahamas during an assets dispute. Drucker later wrote that “at least two speakers at Mr. Marchant’s conference have been served with subpoenas on the dais.”.

Marchant’s own life has carried that threat. When he sat down for an interview at his 2023 conference at Miami’s Ritz-Carlton. he said he had been “threatened by Russians more than once. ” to the point where he sometimes got so spooked he would go sleep at a friend’s house. “It was affecting me mentally. ” he admitted. describing a habit as simple as inserting his key into a car while expecting—quietly. insistently—that the vehicle might explode.

He said he isn’t doing this out of ideology. “Fraud pisses me off,” Marchant said. “The day that it stops pissing me off, I’ll do something else.”

His investigations have helped drive criminal indictments of more than a dozen people, and “half of whom went to prison,” which he described as “very satisfying.”

Yet the conferences also offer a glimpse of how enforcement can stall—especially when the targets have money, time, and lawyers stacked with ex-IRS and ex-prosecutors.

That tension is built into the tax story, and it traces back to enforcement leaders who ran into walls once powerful interests decided they’d had enough.

More than half a century ago. the saga of Dick Jaffe—a renowned former special agent who worked in the IRS’s Intelligence Division—helped set the tone for today’s struggles. In 1973, Jaffe and one informant were investigating money laundering by organized crime figures under an IRS effort dubbed Operation Tradewinds. During that work, they obtained a client list connected to an American-owned, Bahamas-based entity called Castle Bank.

Cross-referenced with other materials, the list provided what Jaffe’s team believed was a smoking gun implicating a “who’s who” of rich and powerful Americans hiding money from the taxman.

At first, Jaffe was celebrated inside the IRS and given additional resources to build individual cases—until the people he had exposed began calling in favors.

President Richard Nixon promptly installed Donald Alexander. a tax attorney who represented ultrawealthy clients and was openly hostile to enforcement. as the new IRS commissioner. Alexander. Blum later recalled. proceeded to dismantle what Jaffe had accomplished. crush the Castle Bank investigation. and even target the agents who took part in it.

Blum described the episode as an “in-your-face flex of oligarchic wealth power.” And the fallout mattered. “What that story did,” Blum told me, “was make it impossible for IRS to deal with the problem of tax evasion by the rich for a full 20 years, if not longer.”

Blum’s own experience in the 1990s echoed that message. During that decade. a bank client in the Cayman Islands gave him reams of information about the activities of 20 very rich Americans cheating the IRS out of millions every year. The informant urged him to deliver the materials to the IRS in Miami, which he did.

The special agent he met with had worked with Jaffe nearly two decades earlier. “He looked at all the documents and information I had, and said to me, ‘Get out of here!’” Blum recalled. Blum wasn’t sure he’d heard right, expecting excitement or gratitude. He asked what the problem was.

The agent told him, “This whole thing is a career killer.”

That snubbing unfolded during what former longtime IRS special agent Ralph Gay described as the Criminal Investigation Division’s “financial heyday.” Gay said the Department of Justice kept “throwing roadblocks up” after a Colombian money-laundering investigation he ran implicated a politically connected American bank owner. and that banker was never indicted.

In the mid-1990s. Gay said CID investigators were able to secure ample funding from Congress—but only because the unit shifted focus to narcotics and foreign money laundering rather than U.S. oligarchs defrauding the treasury. When super-rich targets felt threatened, Gay said, investigators were “reined in.”.

Even today, the IRS says its criminal investigations unit has “about 2,100” armed special agents. That number is less than two-thirds the 1996 peak and roughly the same as the number the agency had during the 1970s—while the number of taxpayers is vastly larger and the ultra-rich ranks are “more bloated. ” as the article describes. alongside a mature “Wealth Defense Industry.”.

The enforcement imbalance isn’t just about manpower. It’s also about what lawmakers choose to fund.

In 2022, Congress approved $46.5 billion over a decade for IRS enforcement. By March 2025, Republicans had slashed it to just under $4 billion—a 91 percent decrease.

And even when the agency appears to have momentum, budget cuts can undo it quickly. Under President Joe Biden. Congress approved an $80 billion increase in IRS funding over a decade. with more than half of it slated to restore the agency’s ability to go after the super-rich. By March 2025, that $45.6 billion enforcement budget was slashed to just under $4 billion—again a 91 percent decrease.

A month later, the IRS was ordered to abandon rules requiring that certain lucrative tax shelters be reported, making the work of overwhelmed auditors harder. The result, the article states, was another $100 billion America’s wealthy “would never pay.”

By early 2026, the IRS had become, in the account, a “more debilitated institution than at any time in over a century.”

Inside that slowdown, the stakes become easier to understand. The IRS publishes data each year on the tax gap—the difference between what’s estimated due and what’s collected. For 2022. the most recent year available. the IRS estimated the gap at $696 billion. which the article describes as roughly the combined 2024 budgets for the departments of Agriculture. Education. Homeland Security. State. and Treasury.

But the article emphasizes that this is a gross underestimate because “in the realm of offshore tax havens, the IRS is flying blind,” with IRS projections saying they do not fully represent noncompliance because data are lacking.

Charles P. Rettig, commissioner in 2021, put the tax gap at about $1 trillion per year—significantly greater than America’s bloated defense budget. The 1975 tax gap, by contrast, was only $40 billion.

The picture becomes starker when you remember who can realistically cheat. Most Americans can’t, the article notes: employers report earnings, withhold taxes, and forward the money to Washington. Only 1 percent of wage earners misreport income because they will be caught; audits of ordinary workers are computerized. creating “limited ability for you not to do the right thing when you have a W-2.”.

Meanwhile. “the lion’s share of the tax gap” is driven by Americans who make money from money—where there is no third-party reporting. IRS data show misreporting and noncompliance in that category shoots up to 55 percent. and the agency estimates three-quarters of the tax gap is due to individuals gaming the system rather than corporations.

Even when agents find problems, their ability to finish cases runs into legal and institutional friction. Congress imposes a three-year statute of limitations on tax enforcement, and complicated audits and investigations take time. Agents don’t want those cases. a former IRS executive told me in the article—“because they won’t be fast. they are difficult. and it takes a lot of time and energy”—and after a year and a half. “the agent gets tired of it. and they’re like. oh. I just want this to be over with.”.

The IRS criminal investigation unit, in this portrayal, is left chasing a scenario where the targets don’t just have lawyers—they have time to turn disputes into delay.

Blum offered a metaphor: “It’s not good enough to have a bicycle chasing a Ferrari. You’ve got to deflate the bicycle’s tires to cripple it.”

Once the super-rich get pushed on taxes. the article describes a playbook: billionaires can hire top-tier tax attorneys. then barrage the IRS with issues and objections until the clock runs out. A retired special agent said those disputes often end in settlements where the accused pay less than they would if fully pursued.

Defense lawyers, the account says, bring in legions of former IRS agents and federal prosecutors. “There’s a revolving door,” Blum confirmed. “That’s where the money is.”

Jeffrey Marcus, a former federal prosecutor who switched sides, joked at the 2017 OffshoreAlert conference: “as a defense lawyer, I just want to note that one man’s crook is another man’s client.”

The result is that the IRS often reaches a settlement rather than a public outcome that could reshape behavior across the wealthy.

Even in criminal cases, the article describes a system that can be difficult to carry. In the rare criminal tax prosecution. the argument stresses. jurors are asked to follow a legal narrative so convoluted that even assistant U.S. attorneys and expert witnesses struggle to understand the details. A former IRS special agent told me: “We teach our agents to treat a jury—and I know this sounds terrible—but you treat a jury like eighth graders. ” and added that he’d heard prosecutors lowered it to sixth grade.

Prosecutors also face public resentment of the IRS, with a former special agent in the account saying jurors who resent the taxman “may pull the trigger for the taxpayer and vote for acquittal.”

All of it combines into what the article calls a near-impenetrable shield for the richest clients.

The book’s examples sharpen how that plays out when the government actually goes after wealthy figures—and how quickly outcomes can end up far short of deterrence.

In 2015. the IRS announced it would seek $3.2 billion in back taxes. interest. and penalties from Sam Wyly and the estate of his brother. who died in 2011 after his Porsche collided with an SUV. The brothers. the article says. evaded taxes for decades using offshore trusts and also used their wealth to influence elections. including bankrolling the “Swift Boat Veterans for Truth” ad in 2004 that helped derail the presidential campaign of Sen. John Kerry.

Wyly’s lawyers whittled his $2 billion IRS tab to $300 million. Sam Wyly responded with an emphatic “no” when asked if he had regrets or would do anything differently.

After a four-year civil trial, the article says his $2 billion share was reduced to $1.1 billion. Then. doubts followed the numbers: the article says a skeptical financial analyst wrote. “Let’s see how much of the $1.1 billion tax bill winds up getting paid.” In the end. the IRS settled with Wyly for only $300 million—an 85 percent discount on the original amount. He was never charged with a crime or faced the threat of incarceration.

The article says the Wylys’ wrongdoing was exposed not by the IRS but by “a group of computer wizards combing through data at a third-party financial institution,” linked to what the government called a sprawling “scheme of secrecy.”

Wyly also argued he relied on the advice of his lawyers and accountants—an argument the account describes as a so-called reliance defense. One court ruling ran to 459 pages. When asked again whether he had regrets or would do anything differently. the article says Wyly replied with an emphatic “no.” Even then. his reputation was shielded in part by privacy provisions that bar the government from commenting on individual taxpayers.

Then there is the case of Robert F. Smith and his mentor Robert Brockman.

In 2020, the Department of Justice issued a criminal indictment of Brockman. The government alleged Brockman engaged for decades in a tax evasion scheme that concealed approximately $2 billion in income from the IRS in secret accounts and structures in Bermuda, Switzerland, and elsewhere.

Smith. described in the account as being deeply embroiled. secured a nonprosecution agreement—called “unheard of” in such cases by a special agent with extensive knowledge of the situation in the article—after agreeing to cooperate against Brockman. seen as the bigger fish. Brockman, who pleaded not guilty on all counts, died in 2022 at age 81 before the government could put him on trial.

After the investigation became clear. the account says Smith applied for amnesty through the Offshore Voluntary Disclosure Program. an escape valve set up by the IRS in 2009 after Blum flushed out the names of nearly 4. 500 wealthy Americans evading federal taxes through the Swiss bank UBS. Because Smith and Brockman were already under investigation, the application was rejected.

Smith and his advisers then pushed a strategy tied to high-profile philanthropy. The idea. the article says. originated with Evatt Tamine. a Bermuda-based attorney implicated in the evasion scheme and also said to have cut a nonprosecution deal. Tamine counseled that charitable visibility could deflect legal consequences. and described the idea as “a strong barrier against an attack from the IRS.”.

The article says Smith agreed to “abandon” $182 million in tax deductions he had claimed when he used money withheld from the treasury for the philanthropy binge.

The case becomes even more stark in the details the account provides. Before Brockman’s death. he gave $25 million to Baylor College of Medicine and millions more to Centre College and Rice University. Smith launched a more aggressive campaign in 2016, four years before DOJ indicted Brockman and publicized its arrangement with Smith.

The account says Smith admitted to lying to the IRS and using offshore accounts to evade taxes for 15 years. It also says he used the untaxed proceeds of his evasion scheme to set up a charitable foundation and claimed a tax deduction for doing so. He donated $50 million to Cornell. his alma mater. followed by $30 million more. with Cornell naming its school of chemical and biomolecular engineering in his honor. It says his extensive biography on the school’s website makes no mention of his offenses even though half of the donation came after the donations became public.

The article adds that Smith donated $20 million to the Smithsonian’s National Museum of African American History and Culture. It notes that in 2019. while giving the commencement address at Morehouse College in Atlanta. he told the 400 graduates that he would pay off their student debt—$34 million in all. It also says he gave tens of millions to Carnegie Hall, the Susan G. Komen breast cancer research foundation, and the National Park Foundation.

The account says that if the money had simply been paid in taxes, the government would have hundreds of billions more each year for public goods, while the tax gap provides lawmakers a rationale, including soaring deficits, to slash spending.

The broader point is driven home through how the Brockman–Smith case was described in the article: as a public relations disaster from “almost every angle.” Brockman never saw justice. and the account describes U.S. attorney David Anderson being left with excuses—Anderson’s press release quoted him: “It is never too late to do the right thing.” The article says Smith committed serious crimes but agreed to cooperate.

And the penalty for Smith. in the account. amounted to paying back $139 million in back taxes and penalties while abandoning $182 million in deductions. It adds that Smith agreed to abandon the $182 million in deductions when he used money withheld from the treasury for philanthropy. It also notes Smith had a $10 billion fortune and used some of his gains to buy two personal ski properties. luxurious homes in California and Colorado. and a commercial property in France.

What remains is a sense—felt in the conference rooms, in the lost cases, in the budget cuts, and in the settlements—that the government’s ability to deter major tax evasion has been systematically weakened.

The article closes by describing what this looks like in real life: one special agent who worked such a case said he was glad to have been involved but described the emotional roller coaster of “soaring hopes and gut-wrenching disappointments” and said. “If you asked me would I do it all over again. ” the answer would not just be no. but “hell no!”.

In an environment where the IRS is stretched thin—where the enforcement funding that Congress approved is repeatedly slashed, where reporting rules are abandoned, and where cases can be delayed into settlements—the question stops being abstract. It becomes practical.

Who gets targeted. Who gets deterred. And who gets to keep believing the system will not catch up.

IRS tax gap offshore tax evasion Congress enforcement funding OffshoreAlert David Marchant Dick Jaffe Offshore Voluntary Disclosure Program Robert Smith Robert Brockman Sam Wyly