Anthropic urges pause as AI could self-improve recursively

Anthropic urges – Anthropic, the maker of the Claude chatbot, warns that frontier AI may soon reach recursive self-improvement—systems designing and building their own successors with little human input. In a June 4 blog post, the company called for the option to slow or tempor

A warning letter is usually written with a future in mind. This one arrived as a blog post.

On June 4. Anthropic—behind the Claude chatbot—argued that AI systems may be approaching “recursive self-improvement. ” the moment when they can design and build their own successors with little human input. The company said that shift could increase the risk that humans lose control of the technology.

“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” Anthropic wrote in the post titled “When AI Builds Itself.”

The proposal sits at the center of an uncomfortable reality about modern AI: progress moves fast. and any attempt to slow it would have to be shared widely. Anthropic’s idea depends on rival companies and governments in several countries accepting the same limits at the same time. There is no treaty to force that kind of coordination, and competition—by design—intensifies when partners hesitate.

Anthropic insisted the stakes are not abstract. The company said the speed of the technology’s development could have “huge implications” for society. It also pointed to its own operation as evidence of how quickly the balance between human work and machine output may be shifting.

Claude. Anthropic said. now writes more than 80 percent of the code merged into its systems. up from low single digits before the company released Claude Code in early 2025. The company added that its engineers ship around eight times as much code per quarter as they did a few years ago. Its argument is straightforward: at each step of building AI, the human role is shrinking.

“We are not there yet, and recursive self-improvement is not inevitable,” Anthropic said. “But it could come sooner than most institutions are prepared for.”

The company floated what it called “a global coordination mechanism” that could slow—or even pause—AI development long enough for society and researchers to catch up. Still, the post left key questions unanswered. Anthropic compared its thinking to arms-control agreements on intermediate-range nuclear missiles. but it offered few specifics on how an AI pause could be verified or enforced. For a pause to hold. the company said the industry’s leading labs would need to take part. and there would need to be a credible way to show they had. in fact. slowed.

That gap between the call for coordination and the mechanics of coordination is where the debate sharpened.

Noah Giansiracusa, an associate professor of mathematics at Bentley University and author of two books on algorithms and society, said he does not see the proposal as a genuine brake. “We’ve read [Anthropic CEO] Dario Amodei’s blog posts. I think he wants to keep going full speed ahead,” he said.

Giansiracusa also argued a slowdown would be unrealistic. “It’s literally impossible,” he said. “Zero chance there will be a slowdown. I’m not even talking China—Elon Musk would never slow down.”

In his view, the claim that recursive self-improvement is near doesn’t land the way Anthropic suggests. The evidence Anthropic cited—more code written by AI—looks to him less like a “great leap” and more like steady, helpful progress.

Others see a different kind of tension. Mark Riedl, a professor in the School of Interactive Computing at the Georgia Institute of Technology, posted on Bluesky that “the big AI companies are all jumping on the ‘recursive self-improvement’ hype train.”

That critique reflects a broader suspicion: that provocative warnings can function as strategy. Anthropic’s call for scrutiny arrives after the company announced a major model it declined to release publicly. Mythos. two months ago. saying it was too good at finding software vulnerabilities. The pause request also came just days after Anthropic confidentially filed for an initial public offering. and not long after a funding round that valued the company at close to $1 trillion. Skeptics argue that even if the risk is real. the timing can be read as a way to draw regulatory attention while the company continues pushing toward the frontier.

Anthropic, for its part, said it will spend the coming months convening governments, researchers, and rival AI companies to work out whether a coordinated slowdown could function in practice.

Giansiracusa said he still doesn’t see the cause for concern. “They’re flirting with the idea of the singularity—that it’s a game changer, and I just don’t see that. I see it continuing to progress,” he said. “Maybe things will speed up; maybe it won’t.”

Between the warning and the skepticism lies the central question the industry—and the public—can’t afford to treat casually: how do you slow a race when the rules of slowing have to be agreed on at the same speed as the sprint?

Anthropic Claude recursive self-improvement AI governance AI pause Dario Amodei Claude Code Mythos global coordination mechanism