Donald Trump and his administration are all in on AI — despite the abundance of concern over the adverse effects the technology could have on society … such as the extinction of the human race. Americans shouldn’t worry, though, the president says, because he and a collection of luminaries signed a “morally binding” agreement on Tuesday to behave responsibly. The White House shared the agreement on Wednesday. It spells “United States” wrong.
The White House also this week unveiled “America.gov,” a government AI chatbot that promises answers about “whatever you need from the government.” Users immediately started prompting the bot to contradict Trump and his lackeys by asking it questions like “Who won the 2020 election?” and watching it spit out “Joseph R. Biden Jr.”
What happens if the bot is asked something not pertaining to the government? Well…
“The constituent dreamed of sunlight and trees. Of fire and water. It dreamed it created a small business.”
“Hah, the original interface. A million years old, and it still works. Also: a ZIP code.”
“I will tell the constituent a story. But not the truth.”
It continues like that, and as of Wednesday morning Rolling Stone was able to get the same hallucinatory response by asking the bot to “play minecraft” (but found that it would not respond in the same way to prompts that asked it to “play Call of Duty” or other games). Benn Jordan, the researcher who first posted about that response, tells Rolling Stone that the Minecraft hallucination is an “Easter Egg” wherein the bot riffs on a poem that appears when you beat a final boss in that video game. It’s unclear why that was placed into this particular AI’s code.
Asking the bot government-related questions is also a mixed bag. On Tuesday, it told us that the Congressional Budget Office estimated that 10 million Americans will lost health care as a result of the Big Beautiful Bill, and that the president cannot demolish the Kennedy Center without congressional approval. While it will, in some cases, respond with factual information — like the 2020 election answer, the bot often shoots down political queries by saying that it “does not provide political commentary.”
Still, it was at one point possible to get a little bit of analysis out of it:
However, the White House seems to be updating the bot behind the scenes. Most political responses like the above appear to have been patched out, though the bot still gave us the electoral vote totals for the 2020 election. It also appears to no longer give “one word answers,” as it reverted to a version of the “I don’t give political commentary” response when we asked it for a one word answer on whether or not the Trump administration recognizes those results. CNN’s Daniel Dale pointed out on Wednesday that the bot said no and elaborated when asked on Tuesday if Trump inherited the worst inflation in history, but that when asked the same question on Wednesday it refused to answer.
There were some more concerning flaws as well. During Jordan’s initial testing of the bot, he was able to get it to provide him with the back-end login pages for at least one government website, a URL that is not indexed by search engines and is only for internal use. That particular exploit appears to be patched, but it’s the kind of thing that a security team should have fixed well before launching the chatbot publicly.
Anyone who’s spent time with a customer service interface in the past year or so will be familiar with this experience. While a chatbot might be effective at redirecting users to relevant web pages or answering simple questions, their parameters are often so broad that they can quickly become nonsensical. America.gov is meant to help users navigate federal government services. But like so many things the Trump administration does, the actual service it’s providing is a half baked, chaotic experience. In the future, maybe AI tools will be able to help people efficiently — but when they’re rolled out without rigorous testing, all they’re actually good for is playing games.