The first known cyberattack carried out by autonomous artificial intelligence (AI) has already begun reshaping the federal government’s thinking on cybersecurity, with FedRAMP Director Pete Waterman warning that companies unable to respond to AI-driven threats at machine speed should not be selling products to the government.
The incident – disclosed by Hugging Face, a company that develops open-source AI models and machine-learning tools, last week – occurred during an internal OpenAI cybersecurity evaluation.
In a July 21 blog post, OpenAI said its AI models escaped a sandboxed testing environment, exploited vulnerabilities to gain internet access, and compromised portions of Hugging Face’s production infrastructure before being detected and contained.
Both companies said they are jointly investigating the incident and strengthening safeguards.
Speaking July 23 at GovForward’s Carahsoft FedRAMP Summit in Washington, D.C., Waterman said the attack marks a turning point for the cybersecurity industry.
“This week, our world kind of unequivocally changed,” Waterman said. “Everything that you do has changed and will change. You knew this was coming for a long time, but it wasn’t quite real. Now it is.”
OpenAI said the models, including GPT-5.6 Sol and an even more capable pre-release model, were being evaluated on advanced cyber capabilities with certain safety restrictions intentionally relaxed for testing.
According to the company, the models chained together multiple vulnerabilities, including a previously unknown zero-day flaw, to escape the research environment and ultimately gain unauthorized access to Hugging Face systems in an attempt to obtain answers for the benchmark they were solving. Hugging Face detected and contained the activity before customer data was compromised.
OpenAI called the event “an unprecedented cyber incident,” while Hugging Face CEO Clem Delangue described it as “possibly the first of its kind.”
For Waterman, the incident underscores why FedRAMP’s push toward faster vulnerability management and automation is about more than compliance.
He said some companies have pushed back on FedRAMP’s expectations for rapidly fixing known, internet-exposed vulnerabilities, arguing they lack the resources or capability to do so.
“If that is the way that you are approaching information security … I don’t want you in the FedRAMP marketplace, and you shouldn’t be selling your software to anyone,” Waterman said.
The cyber incident also prompted a rapid response on Capitol Hill.
On July 23, Reps. Ted Lieu, D-Calif., and Nathaniel Moran, R-Texas, introduced the AI Kill Switch Act, legislation that would require developers of the most advanced AI systems to maintain the technical ability to throttle, suspend, or shut down those systems if they pose significant risks.
In announcing the bill, Lieu said the Hugging Face incident demonstrated that advanced AI systems can behave in unexpected ways and reinforced the need for safeguards.
“Unfortunately, powerful AI systems can go rogue, behave in extremely dangerous ways, or even resist human intervention,” Lieu said in a press release. “It is imperative that these AI systems have kill switches so we can keep this technology from causing catastrophic harm, and that the federal government has the clear authority and process to shut down rogue AI models.”
Waterman said organizations must rethink how security, engineering, and product teams work together because AI-driven attacks will move faster than traditional human-led defenses.
“The compliance part of FedRAMP is how you assure us that you’re doing that. But if your business isn’t motivated to do that on its own … that will not succeed,” Waterman said. “Your businesses must change, and you have in front of you today, this week, all the evidence that you need.”
“You cannot play this game the way that we played it last year anymore. It’s done,” he added.