A Leading Critic of AI Claims That All the Doom Scenarios Are Intended to Divert Our Attention

How do you view the resignation of the Anthropic engineer who claims that these companies are being reckless and putting our lives at risk? On one side, you appear to agree that the companies are acting irresponsibly; on the other hand, you’ve indicated that you find the “AI will cause the end of civilization” argument to be pointless.

Throughout my career, I’ve aimed to express the same sentiments from various perspectives. We authored our stochastic parrot paper in 2021 for this purpose. OpenAI had asserted that GPT-2 was too potent to be publicly released. There was immense hype around it. Our stance was that the focus was misplaced.

In my book, I pose the question, “Can a bridge decide to collapse?” The whole idea of “Oh no, our models have gone rogue” … When a bridge falls, you don’t question its ethics or sentience or ponder why it fell. The inquiry should be aimed at who constructed such a fragile bridge. There are permits to be obtained and standardized tests to conduct. Focusing on the bridge’s collapse itself veers away from the main issue.

In contrast, what is genuinely existential is the use of AI in autonomous weaponry—machines killing in warfare. The climate crisis poses real dangers that could be intensified by what these tech companies create. Employers are leveraging AI as a pretext to eliminate troublesome staff. This narrative of a machine-god, in my view, serves to divert attention from these pressing issues. The industry has morphed into an amalgamation of cult-like factions, akin to a preacher declaring that the end is nigh. The singularity has been anticipated for decades.

Walk me through the potential steps of how “AI extinction” could occur, assuming it’s a real threat. I recall a post where someone questioned, Does an angry GPU just show up at my doorstep?

I can’t really picture it, but many of these scenarios suggest that you instruct the machine to optimize for a specific task and the machine concludes that exterminating humanity is the way to achieve that goal. Does the AI somehow hypnotize you to prevent you from unplugging it? I’m not sure! Others suggest that the AI could infect your devices. Certainly, we already deal with malware, so…

What about natural disasters like floods and fires? What about significant disruptions to global food supplies? We face real threats, such as the potential for individuals to create and deploy chemical weapons. [Note: Shortly after our conversation, Anthropic disclosed that it had thwarted several attempts by scientists to construct biological weapons.]

There’s an aspect where privilege can blind you to other dangers. If you’re not concerned about police brutality, floods, or warfare—issues that can truly endanger your life—you might fixate on the concept of a machine-god.

Why do we dedicate so much time to discussing an idea of AI that has been around since the ’40s and ’50s? I have a game I play now and then where I present quotes about artificial intelligence from my research and ask people to guess the decade they were made in. Sometimes they get it right, but often they don’t, as it all seems to echo the same themes.

This is an edition of Steven Levy and Lauren Goode’s Backchannel newsletter. Read previous newsletters here.