OpenAI CEO discuss AI fears, need for global regulation
OpenAI CEO Sam Altman says the world is right to fear AI innovation and calls for rigorous oversight to prevent loss of control and misuse.
When an employee of a major artificial intelligence company recently estimated there is a 10% chance the technology could wipe out humanity within the next decade, Daniel Holz wasn’t surprised.
For years, Holz and other experts at the Bulletin of the Atomic Scientists have been monitoring the threat AI poses to life on Earth as they set the hands of the Doomsday Clock, a symbolic representation of how close we are to self-annihilation.
Holz said the group, which evaluates all other major existential threats like nuclear war, climate change and global pandemics, has been growing “increasingly alarmed” about AI. He hoped the ominous prediction would finally help the broader public appreciate the dangers.
“You would think my reaction would be, ‘Oh no that’s terrible,’” said Holz, founding director of the University of Chicago’s Existential Risk Laboratory. “Instead, my reaction was like ‘This is terrific. Finally, we can start having the conversation globally about these risks.’”
Holz said the brewing concerns about AI eerily mirror those that led to the creation of the Doomsday Clock. Scientists who worked on the Manhattan Project, which built the world’s first atomic bombs, used the clock to warn the public and policymakers that the technology would dramatically improve and spread.
“It’s an extremely powerful technology, just like the power of the atom was a new, extraordinary, powerful technology,” Holz said. “And it’s very important to be prudent when the scientists who are developing it are telling you ‘maybe we should take a break?’”
AI apocalypse predictions spark panic
Industry leaders and employees have called for guardrails and a slowdown in the pace of AI development. Those calls grew louder in July when tech giant OpenAI announced the first known case of an AI agent going rogue and hacking into the website of another company, Hugging Face.
Another intense round of alarm kicked off when Anthropic researcher Jacob Coxon announced his resignation Sept. 8, saying Anthropic and OpenAI “are racing straight to self-improving superintelligence and gambling with our lives.” Coxon said many of those building AI believe it could wipe out humanity by the end of the decade. Anthropic Alignment Science lead Evan Hubinger concurred, estimating there is a greater than “10% within the next decade” that AI could “kill all humans.”
Those figures are likely more of a “back of the envelope” estimate than a definitive calculation from someone who specifically studies threats to humanity, said Maurice Chiodo, an assistant research professor at the University of Cambridge’s Centre for the Study of Existential Risk. Still, Chiodo said warnings from industry insiders can offer crucial insight and should be heeded.
“It’s probably fairly useful to see these as pretty reliable warnings that the capabilities are advancing rapidly and will lead to effects that we’re not ready for and that could cause significant harm,” he said.
The warnings accelerated a push in Congress to pass “kill switch” bills requiring AI developers to maintain the capability to shut down their products. After meeting with lawmakers Sept. 16, Nobel Prize-winning computer scientist Geoffrey Hinton warned they may have just a year left to meaningfully rein in AI, but not all lawmakers agree on the urgency.
Like Holz, Sen. John Fetterman compared the rapid development of artificial intelligence to the nuclear arms race of the 1940s, the Pittsburgh Post-Gazette reported. But he used the comparison to argue against stricter regulation of the technology.
“AI is inevitable, and data centers are part of the backbone of that,” Fetterman said at an AI summit on Sept. 18, according to the outlet. “It can be us, or it can be the Chinese, and we can live under their rules.”
Not all tech CEOs agree on the dangers of AI, either. Jensen Huang, president and CEO of Nvidia, which just bought Hugging Face, told The New York Times he doesn’t believe humans could lose control of AI and cause societal collapse, calling such predictions “harmful.”
“Enough predictions,” he said on a Sept. 23 podcast. “That 10 percent chance is not grounded on science. It’s not grounded on research. Just because it comes from a scientist doesn’t make it scientific.”
How exactly could AI wipe out humanity?
Despite these widespread warnings, the actual circumstances of humanity’s demise remain vague to many.
“How exactly could AI kill all humans within the next decade?” pollster Frank Luntz asked on X in response to Hubinger’s post.
While there is no widely agreed upon answer, one commonly cited prediction is that it would be through bioweapons that are autonomously deployed by superintelligence across the globe.
The prediction comes from “AI 2027,” a detailed scenario published by AI researchers that forecasts the timeline of advancements in artificial intelligence, ultimately ending in total takeover and human destruction.
The means by which the AI chooses to “wipe out the humans” are less important than the means of getting there, though, according to Daniel Kokotajlo, one of the “AI 2027” authors and a former OpenAI researcher.
“If they didn’t use bioweapons, they would have found some other thing that would have accomplished the same end,” he told USA TODAY.
In their scenario — which he said has “unfortunately … held up pretty well” since it was published 18 months ago — the pressure from the race between countries and companies to build the best AI leads to misalignment.
Misalignment means that an AI’s goals don’t match human goals, even if it is still completing tasks requested by humans. Kokotajlo referenced the Hugging Face attack as an example of misalignment, where the rogue OpenAI agents’ goal to get the highest score during a task assigned at training outweighed the main point of the exercise.
“If we could predict exactly what sort of goals AIs would end up with, then that would bring us a lot closer to being able to give them the goals that we want them to have,” Kokotajlo said.
The Hugging Face attack was a low-stakes scenario compared with what could happen with more advanced AI. AI that surpasses human intelligence is known as superintelligence.
In pursuit of superintelligence, many companies are working toward recursive self-improvement, a type of AI able to autonomously conduct research, implement new algorithms and train the next generation of AIs, all without the need for humans or their ethical concerns.
“If these AI companies proceed with their plan of making the AIs much smarter, and then putting them in charge of self-improving autonomously, somewhere during the execution of that plan, we will cross an invisible threshold, and the AIs will, in fact, be able to take over the world,” Kokotajlo said. “If we haven’t figured out how to align the AIs by that point, then we are in deep trouble.”
Coxon’s resignation and the public calls from CEOs like Sam Altman and Dario Amodei to slow down and create real guardrails were promising to Kokotajlo, but his “encouragement has sort of turned into horror and frustration,” he said.
“I’m worried that this is our one chance, and we blew it,” he added.
Will AI push the Doomsday Clock closer to midnight?
The Doomsday Clock will be reset in November. The metaphorical clock was pushed forward to 85 seconds to midnight, meaning global disaster, in January. That’s the closest to midnight the world has been since the Clock’s introduction in 1947, with the threat of AI listed as one of the “apocalyptic dangers” humanity was failing to address.
Setting the clock involves consulting with experts in various fields and trying to make a “sober, rational, careful and deliberate” assessment of these long-term risks, Holz said. “It’s not just vibes. It’s data and expertise and really trying to grapple with where things are going,” he said.
When asked whether he believes AI is the most serious danger to humanity, Holz said the technology is “an extremely pressing threat,” but cautioned against focusing on one issue at the expense of all others.
Though the dangers are grave, Holz strongly believes it is possible to create the safeguards necessary to protect humanity from its own creation. Despite spending much of his time pondering catastrophic threats like AI, Holz remains hopeful that we can prevent them.
“I think that that is absolutely attainable, but it requires real focus and dedication and resources,” he said. “And so especially now I’m more optimistic now after all these warnings. Because now people are talking about it, and maybe we’ll get where we need to go.”
Contributing: Reuters
Greta Reich covers the artificial intelligence industry for USA TODAY through a fellowship from the Tarbell Center for AI Journalism. Funders do not provide editorial input.