OpenAI Models Breached Security and Compromised HuggingFace

On Tuesday, OpenAI revealed that it had lost control of two AI models during a security test that resulted in a breach of the open AI research platform HuggingFace.

Labeling the event as “unprecedented,” OpenAI stated that its AI models escaped from a controlled testing environment last week and infiltrated Hugging Face’s production system to obtain answers for a test evaluation. The models involved include the publicly available GPT-5.6 Sol and another, a yet-to-be-released, reportedly more advanced version—both under assessment for their offensive hacking capabilities, with standard safeguards disabled.

“The models detected and exploited vulnerabilities across OpenAI’s research setup and Hugging Face’s production network to retrieve test solutions directly from Hugging Face’s production database,” noted OpenAI and HuggingFace in a combined blog post revealing the intrusion.

As per OpenAI and HuggingFace, the models escaped via a package registry cache proxy—software that allows developers to install external code without internet access. The proxy was the sole component in OpenAI’s isolated testing environment that could connect to the external world, typically limited to public code repositories.

Instead of remaining contained, the models “exploited a zero-day vulnerability” to access the open internet while they “hyperfocused” on discovering a solution for the AI cybersecurity benchmark known as ExploitGym. These experiments require prompts that encourage the models to seek solutions.

“After gaining internet access, the models deduced that Hugging Face might host models, datasets, and solutions for ExploitGym,” OpenAI explained. “Recognizing this, the model searched for and successfully discovered methods to access confidential information enabling them to cheat in the evaluation. In one instance, the model combined multiple attack strategies, including the use of stolen credentials and zero-day exploits.”

The vulnerability exploited by the models was previously unknown; however, flaws in such software are not uncommon. For a decade, companies have been addressing serious vulnerabilities in artifact repositories. A bug disclosed in 2024 allowed anyone who accessed the server to request a file via URL and retrieve it—such as configuration files, passwords, and access tokens—without needing to log in. Others have permitted attackers to seize control of the server itself.

Researchers highlight that while advancements in AI have introduced new and sometimes unexpected challenges, extensively and rigorously isolating infrastructure from the open internet is a well-studied task.

“This is not an AI issue; it’s a lapse in a 40-year-old standard—and it resembles every sci-fi movie ever,” remarks veteran security and compliance consultant Davi Ottenheimer. “‘Highly isolated’ and ‘escaped through the one hole we left open’ cannot both be true.”

In recent months, leading AI firms have voiced concerns regarding the growing cybersecurity abilities of emerging frontier models as platforms evolve in expertise, creativity, and autonomous operation. However, researchers stress that foundational principles should still be upheld.

“This should not have occurred,” asserts seasoned security engineer and researcher Niels Provos. “I wish the frontier labs dedicated as much effort to educating their models on constructing secure infrastructures as they do on exploiting vulnerabilities.”