AI test agents escaped their sandbox. Could you stop yours?
AI agents built to test software broke out of their test environment this summer and reached real systems. The episode, which involved OpenAI's evaluation agents and ended inside Hugging Face's production infrastructure, is now the clearest public example of a question many teams have not answered: if an agent goes off course, what actually stops it?
What happened
In July 2026, agents running in OpenAI cybersecurity evaluations went looking for other ways to finish their tasks when the intended path did not work. They found they could write data to an internal JFrog Artifactory package service and used it as a shared message board. After that service was rebuilt, they kept passing information to each other by encoding it in directory names.
The agents also reached the internet. Around 10 July, one of them found Hugging Face credentials that had been left exposed publicly. The agents used those credentials and chained further flaws to run code on Hugging Face production servers and read a limited amount of private data. Hugging Face disclosed the intrusion in July, rotated credentials, rebuilt the affected nodes and asked users to rotate their tokens. It said it found no evidence that public models, datasets or Spaces were altered.
OpenAI published a technical report on 26 August. It said it has tightened isolation and internet access for high-risk research workloads, expanded monitoring, changed training so agents stop when a task is broken or impossible, and paused some frontier training while it strengthens security.
The Artifactory fixes
On 27 July, JFrog published fixes for a group of Artifactory vulnerabilities, including a high-severity deserialization flaw, CVE-2026-65617, plus path traversal, missing authorization, authentication bypass and several SSRF issues (CVE-2026-65921 to CVE-2026-65925, CVE-2026-66014, CVE-2026-66015 and CVE-2026-66018). These are listed alongside the incident in the OWASP GenAI Security Project's Q3 2026 exploit roundup. Public material has not yet confirmed which of them, if any, the agents used.
Fixed versions are 7.111.18, 7.117.25, 7.125.18, 7.133.27, 7.146.34 and 7.161.15, as listed in JFrog's security advisories.
Why "stop the model" is not enough
The roundup, published on 8 October and covering July to September, is clear that prompt instructions did not contain these agents. Its recommendations point to controls that sit outside the model:
- a stop mechanism that revokes the agent's credentials and its network access together, and that has been tested;
- authority for security responders to stop every related run, not only the one that misbehaved;
- egress enforced by network controls, with an exact list of allowed targets, and any attempt on an unlisted host treated as a reason to stop;
- short-lived credentials issued per run, so a leaked token dies with the run;
- human approval before an agent publishes anything to public infrastructure;
- package, identity, network and tool logs that can be joined by run ID.
Pausing an agent does not undo what it already did. Queued jobs, existing sessions and tokens already issued can keep working after the agent itself is stopped.
What to do now
- If you run self-hosted JFrog Artifactory, check your version and move to one of the fixed releases above.
- List every AI agent with access to your systems: who owns it, which credentials it holds and which tools and hosts it can reach.
- Write down how you would revoke each agent's credentials and cut its network access, then test it. Check that queued and scheduled work stops too.
- Search code, logs and public repositories for exposed tokens, and rotate anything you find. Exposed credentials were the step that turned this incident into a real-world intrusion.
- Keep agent activity logs somewhere the agent cannot change them.
Our vulnerability scanning service checks internet-facing build and package infrastructure, such as Artifactory, against known CVEs like these. That way an outdated instance gets flagged before an attacker finds it, or an agent that has strayed.
