OpenAI and Anthropic Agree to Letter Aiming to Halt AI-Developed Biological Weapons

The leaders of numerous prominent artificial intelligence firms are calling on Congress to introduce new legislation aimed at making it more challenging for malicious individuals to create biological weapons using their technologies.

Signatories of a public letter include Google DeepMind’s Demis Hassabis, OpenAI’s Sam Altman, Anthropic’s Dario Amodei, and Mustafa Suleyman from Microsoft AI, urging laws that compel companies selling synthetic DNA and RNA to screen their customers and orders to mitigate the risk of genetic material misuse.

The letter, organized by the nonpartisan Institute for Progress alongside the right-leaning Foundation for American Innovation, states that with the rapid development of AI, “there is a genuine possibility that the knowledge barriers historically preventing bad actors from acquiring biological weapons will significantly diminish.”

In the 1950s, scientist Arthur Kornberg was the first to successfully synthesize DNA. Today, the process is largely automated, with numerous companies globally utilizing commercial synthesizers to “print” and provide custom genetic sequences for scientific research, drug development, and diagnostics. Although many suppliers only cater to qualified researchers, biotech firms, and educational institutions, not all conduct thorough vetting of customers or the gene sequences purchased.

A notable incident occurred in 2017 when Canadian researchers alarmed the scientific community by using $100,000 worth of mail-order DNA to recreate the extinct horsepox virus. Critics expressed concerns that similar methods could allow for the construction of smallpox, a related and highly lethal virus. Since then, gene synthesis costs have declined further.

With advancements in AI, it has become feasible to design hazardous new toxins and pathogens through large language models, though some biological knowledge would likely still be necessary to create a viable virus from scratch. Though bioterrorism incidents are rare, they have the potential to cause significant casualties, widespread panic, and economic damage. A pressing worry is that an AI-generated pathogen could inadvertently or deliberately lead to a global pandemic.

“AI tools allow users to quickly pinpoint where to order sequences that will evade screening,” says David Relman, a microbiologist and biosecurity expert from Stanford University who signed the letter. “If prompted appropriately, they can also suggest how to modify your order, making it more difficult for screening measures to identify what you aim to create.”

The letter’s signers include a range of scientists, national security professionals, and executives from gene synthesis companies such as Twist Bioscience and Ansa Biotechnologies. These companies are part of the International Gene Synthesis Consortium, established in 2009 to promote voluntary screening practices. Many already utilize software to screen orders for “sequences of concern” that could enhance an organism’s toxicity or pathogenic potential.

“If you possess technology that can synthesize DNA, you should take measures to ensure its responsible use, including understanding what you are producing and for whom,” states James Diggans, vice president of policy and biosecurity at Twist Bioscience. The firm has advocated for formal regulations for several years.

Federal guidelines implemented during the Biden administration necessitate that scientists and firms receiving federal funds procure synthetic gene sequences from providers that screen their transactions. A bipartisan bill introduced in the Senate earlier this year seeks to mandate that all gene synthesis providers operating in the US perform screenings for potential bad actors or hazardous pathogens.

However, screening technologies are not infallible. Last year, Microsoft researchers published findings revealing that AI protein design tools could generate potentially harmful gene sequences that bypassed companies’ screening systems. The models proposed new protein sequences closely resembling known dangerous ones.

Geoff Ralston, former president of Y Combinator and a partner at the Safe AI Fund, believes AI laboratories utilizing biological models should implement their own user screening procedures.

“It should be extremely difficult, if not impossible, to request a model to assist in creating something dangerously harmful,” remarks Ralston, who also signed the letter.

Relman concurs that while regulations for screening procedures are essential, they represent only part of the solution. “Since screening may fail in certain instances, we need additional control measures,” he states. “This is where AI companies must take responsibility.”