Pangram flags – A forum analysis suggests parts of Pope Leo XIV’s latest encyclical on AI may have been generated with AI tools, with multiple Pangram estimates flagging large portions while other sections appear human-written. The Vatican has not yet responded to a request f
By the time the question landed online—whether the Pope used AI to write about the dangers of AI—the encyclical itself had already done its quiet work. Pope Leo XIV’s latest letter, focused on AI’s impact on humanity, had been published in the Vatican tradition. But the language is now being argued over by strangers who treat text like evidence.
An analysis by Linch Zhang. posted on the forum LessWrong. says certain paragraphs in Magnifica Humanitas may have been written with AI assistance. Zhang’s reading points to results from the popular AI detector Pangram. which Zhang says found parts of the document to be between 40 percent and 100 percent written by AI. The finding hinges on linguistic fingerprints Zhang believes match AI-generated writing.
One repeated tell is the frequency of the word “genuinely.” Zhang argues that the word shows up in ways associated with AI writing produced by Anthropic’s Claude—more than it appears in earlier encyclicals. Another step came from a separate test: someone ran the document through Pangram section by section. finding that 62 percent of the first chapter was flagged as AI generated. When The Verge ran roughly 2,000 words of the encyclical through Pangram, it estimated that 46 percent was AI-written.
Not everything, though, falls neatly into the “AI wrote it” box. Zhang says Pangram flagged some sections as “essentially 0% AI.” He also points to a comparison that the detector treated as human: the first 20 paragraphs of the last four encyclicals. when run through Pangram. had a 100 percent confidence of being human written. A transcript of Pope Leo’s speech, also run through Pangram, received a 100 percent human rating.
The text, in other words, comes with a split verdict—uncertainty built right into the measurements. AI detection isn’t foolproof, and different detectors can disagree. Even when there appears to be consensus, there’s no guarantee the results are correct. Pangram. however. is described as generally respected among AI researchers. and in March 2025 it said it estimated a false positive rate for reporting human-written work as AI-generated of “approximately 1 in 10. 000.”.
Encyclicals are lengthy letters published by the pope, meant to impart teachings addressing moral and social challenges of the time. Magnifica Humanitas is the pope’s first encyclical. The previous one was written by Pope Francis in October 2024.
The current encyclical is also the first to focus on AI and its wide-ranging influences. Pope Leo has presented it alongside Christopher Olah, a co-founder of Anthropic. The Vatican did not immediately reply to a request for comment.
Still, the controversy is now living in the space between faith and forensics: an AI-focused message, judged by tools designed to infer whether a message was touched by AI at all—while the institution at the center of the text has not yet answered.
Pope Leo XIV Magnifica Humanitas AI encyclical AI detectors Pangram Linch Zhang LessWrong Anthropic Claude Christopher Olah Vatican
Pope using AI to say AI is dangerous… seems ironic lol.
They’re really trying to “detect” the Pope like it’s a homework plagiarism thing? If detectors flag it, that doesn’t mean it’s true. I don’t even trust that word frequency stuff.
Wait so Pangram is detecting AI by counting words like “genuinely”? So if he uses “genuinely” a lot then it’s AI?? That’s kinda dumb logic. Also if it says 0% AI in some parts then maybe it’s just edited by a human with weird writing habits.
The Vatican tradition but “quiet work” like the article says… I feel like this is all just people on forums trying to start a fight. If The Verge got 46 percent, ok? That still means more than half could be human, so why is everyone acting like the whole thing is fake? Also 2000 words?? Kinda sounds like they chopped it up and guessed. If the Pope didn’t answer yet, people should chill.