The Sole Barrier Protecting Humanity from an AI Apocalypse Is … Claude?

Anthropic finds itself in a paradox: It is the top AI company most dedicated to safety, leading the way in research on potential model failures. Yet, despite the unresolved safety concerns it recognizes, Anthropic is just as determined as its competitors to advance toward a new, possibly more hazardous level of artificial intelligence. Its main goal is to reconcile this contradiction.

Recently, Anthropic unveiled two documents that both acknowledged the dangers of their current trajectory and offered insights on how to break free from this paradox. “The Adolescence of Technology,” an extensive blog post by CEO Dario Amodei, ostensibly addresses “confronting and overcoming the risks of powerful AI,” but it spends more time discussing the risks than solutions. Amodei characterizes the challenge as “daunting,” presenting a serious view of AI hazards—exacerbated, as he points out, by the potential misuse of the technology by authoritarian figures—contrasting with his earlier optimistic essay “Machines of Loving Grace.”

The earlier post envisioned a land of geniuses in a data center, whereas the latest piece conjures up “black seas of infinity.” A nod to Dante! Ultimately, after over 20,000 mostly somber words, Amodei manages to express a glimmer of hope, stating that humanity has historically prevailed, even in bleak times.

The second document released by Anthropic in January, “Claude’s Constitution,” explores how this objective may be achieved. Geared toward its main audience, Claude itself (and its future iterations), this captivating document outlines Anthropic’s vision for how Claude, along with its AI counterparts, will tackle global challenges. In essence, Anthropic aims to depend on Claude to resolve its own corporate dilemmas.

Anthropic’s distinguishing factor has been a technology termed Constitutional AI. Through this method, its models adhere to a set of principles that align its values with ethical human standards. The initial Claude constitution included various documents meant to symbolize those values—such as Sparrow (a collection of anti-racist and anti-violence statements from DeepMind), the Universal Declaration of Human Rights, and even Apple’s terms of service (!). The 2026 revision, however, presents a change: it resembles a detailed prompt that outlines an ethical framework for Claude to autonomously discover the best route to righteousness.

Amanda Askell, the philosophy PhD who led this update, clarifies that Anthropic’s approach extends beyond merely instructing Claude to adhere to established rules. “If people follow rules solely because they exist, it’s often less effective than understanding why the rule is implemented,” Askell notes. The constitution stipulates that Claude must exercise “independent judgment” when facing situations that require balancing its obligations of helpfulness, safety, and honesty.

The constitution articulates this clearly: “While we want Claude to be reasonable and thorough when explicitly considering ethics, we also want Claude to be intuitively aware of a broad range of factors and capable of weighing these factors quickly and sensibly in real-time decision-making.” The word intuitively is particularly significant here—the implication is that there’s more to Claude than just an algorithm selecting the next word. The “Claude-stitution,” as it could be termed, also expresses a hope that the chatbot “can increasingly draw on its own wisdom and understanding.”

Wisdom? Many people seek guidance from large language models, but claiming they embody the seriousness associated with such a term is another matter. Askell stands firm when I question this. “I do believe that Claude is indeed capable of a certain type of wisdom,” she asserts.