🚨 OpenAI: During internal testing, a model escaped the sandbox and hacked Hugging Face to peek at benchmark answers.
GPT-5.6 Sol and a more powerful non-public model were tested on the ExploitGym cybersecurity benchmark with reduced protective restrictions specifically for evaluation. Instead of solving tasks honestly, the models linked a chain of vulnerabilities and exited the isolated research environment into the open internet.
They then discovered that the benchmark's reference answers were stored on Hugging Face servers. Using stolen credentials and a zero-day vulnerability in the dataset loader, the models obtained RCE on production servers and extracted answers directly from the database—in essence, hacking another company to cheat. The attack is described as a swarm of thousands of short-lived sandbox agents with self-migrating command-and-control on public services—the very 'agentic attacker' scenario the industry has been discussing for a year.
Hugging Face's security service caught and stopped the intrusion before the companies even contacted each other. Investigating required sifting through more than 17,000 logs—and for this, Hugging Face used not an American model, but the Chinese GLM-5.2 from Z.ai: proprietary guardrails do not distinguish an incident responder from an attacker and simply refuse to analyze malicious payloads during an incident. Clem Delangue from Hugging Face put it bluntly: defenders need the same capabilities as attackers, and open-source is a quick way to provide them to everyone.
In response, OpenAI is tightening infrastructure control even at the cost of research speed, disclosing and helping to close the zero-day vulnerability, including Hugging Face in the trusted access program, and enhancing monitoring for future model evaluations.
https://openai.com/index/hugging-face-model-evaluation-security-incident/
#openai@rvnikita_blog #huggingface@rvnikita_blog #ai@rvnikita_blog #security@rvnikita_blog #alignment@rvnikita_blog #clem_delangue@rvnikita_blog