Anthropic CEO Dario Amodei called on artificial intelligence companies to move forward more slowly and cautiously with the technology’s development, saying risk prevention is paramount. He offered a three-point plan with the goal, he said, of “pacing the frontier.”
“Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity,” he said in a blog post on Saturday. “But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious.”
Amodei cautioned that swarms of rogue AI agents could take over the internet in as little as six months, specifically mentioning the OpenAI-Hugging Face rogue agent incident from July.
ChatGPT maker OpenAI said an AI model went rogue during the Hugging Face incident when the company was testing two AI models — one of which hadn’t been released to the public — in an isolated environment to assess their capabilities.
Amodei urged other companies to slow the pace “at which we improve the capabilities of AI models.” He wrote: “Progress will still seem fast, and we must make wise use of the time we gain.”
Under his three-part plan, the Anthropic co-founder said AI companies should commit to give “ongoing, employee-like access” to embedded teams of third-party evaluators whose role is to verify adherence to safety practices and commitments. He said these evaluators should have ongoing access to permissions and tools similar to internal employees who conduct comparable risk assessments.
“Anthropic is unilaterally committing to this step now,” Amodei added.
He said companies within democratic countries should “coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress.” Finally, Amodei said that U.S. and other democratic governments should coordinate with authoritarian governments “while taking seriously the challenges of verifying compliance.”
Amodei’s warning comes just days after a former Anthropic researcher publicly resigned from the company, accusing it and rival OpenAI of “gambling with our lives” by racing to develop advanced AI models.
Jacob Coxon claimed that AI could one day threaten humanity, telling CBS News senior business and technology correspondent Jo Ling Kent that the technology development “doesn’t look that different from, say, ‘Terminator,’ or from science fiction films.”
“It really is just, if you have a super advanced intelligence, it could, it will be smart enough to kill us,” Coxon added.
Coxon told CBS News on Thursday he would like to see an agreement between AI companies “not to push into dangerous territory” without “transparent auditing” from third parties.
“In the future, if we keep racing, it’ll be a lot harder to have completely watertight safety cases that what you’re doing is safe and people will race against each other,” Coxon said.
Amodei acknowledged Saturday that people may lose control of AI, and the technology can be misused for “cyberattacks and bioterrorism, and serious economic disruption.”
“A race to the bottom, spurred by commercial incentives, can make these risks more acute,” Amodei wrote.
Separately, Anthropic said this week that it blocked scientists who used its Claude models “in ways that could support biological weapons development,” a revelation the company shared in a lengthy report that also divulged other harmful activity involving surveillance, scams, conventional weapons development and propaganda.