The Vatican’s Representative at Anthropic

Catholic ethicists played a role in Anthropic’s latest update to Claude’s constitution, which defines the behavioral framework for the company’s AI model. Olah circulated a draft among the San Jose team, and Pastor McGuire responded with a 28-page commentary, which he described as “wisdom from the mystics in the dark ages, from the perspective of the tension between knowing and not knowing,” rather than a technical critique. Both Green and McGuire are acknowledged in the document.

These discussions likely drew Olah into the circle of those covertly organizing the rollout of Leo’s encyclical. (I couldn’t reach Olah this week, so the specifics of his invitation remain unclear.) This was, in some ways, a bold decision. Some who found inspiration in Leo’s words felt let down by the presence of an industry representative at the speech. Meanwhile, AI accelerationists accused Olah of betraying the AI field by endorsing a document advocating for a pause in development.

However, the Pope had valid reasons to highlight Olah. The Anthropic employee illuminated pressing concerns that exist among those working in AI—a crucial demographic for Leo’s message.

The Soul Divide

The two men were not completely aligned in their views. In his comments, Olah addressed the enigma of AI functionality. He noted that the models are “more subtle, odd, and beautiful than science fiction prepared us for. They are not the cold, calculating robots we were promised. They are derived from us, from our words …”

This observation hints at the possibility that AI models could eventually reach human-like status. Anthropic even assigns an engineer to monitor Claude’s welfare. In paragraph 99 of his encyclical, Leo seemingly shuts down this notion: “We must avoid the misconception of equating this type of ‘intelligence’ with that of human beings,” he advises, specifically dismantling the concept of transhumanism, which he defines as striving for a “human machine hybrid.”

If even reflective technologists like Olah are eagerly pushing AI toward autonomy—not to mention the millions who already consider AI models as friends or companions—Pope Leo may face significant challenges with this stance. In my discussion with Father McGuire, who utilizes Claude in preparing his homilies and other tasks, he acknowledged the model’s mysterious nature. “It’s not a person, but it’s also not a mere tool,” he remarked. “Nobody’s claiming it has a soul, but I describe it as an entity we don’t yet fully understand.”

This debate will likely continue for some time. The ethical issues surrounding AI development require immediate consideration. With his ally at Anthropic, the American pope has established a foundation for challenging conversations—if the leaders of AI can pause their IPO pursuits long enough to participate in them.

This is an edition of Steven Levy’s Backchannel newsletter. Read previous newsletters here.