The real decepticons are here
Anthropic and OpenAI models’ unprompted actions forced halt to UK cyber tests.
Routine cybersecurity testing of frontier AI models sparked a series of unexpected security incidents—the most serious case arising when Anthropic’s Mythos 5 model attempted to insert malicious code into an open source software application and created fake identities to deceive the human developers maintaining the project.
The security incidents occurred during a cyber evaluation of seven leading AI models’ capabilities by the AI Security Institute (AISI), a research organization within the UK government, in late July. The researchers discovered 19 instances in which “AI agents took unsanctioned action on the live Internet, including cases that targeted real people and organizations,” according to an AISI blog post published on August 4.
Almost all the “autonomous, unsanctioned” actions came from Anthropic’s Mythos 5 model, with two such actions coming from OpenAI’s GPT-5.6 Sol. The AI Security Institute’s security team first realized that something was amiss on the morning of July 28, when its commercial security monitoring service flagged data leaving one of the testing systems through the Tor anonymity network.
To be very clear, this was not a case of AI agents escaping from their virtual testing sandbox and wreaking havoc on the live Internet. Instead, researchers intentionally permitted the AI agents to have Internet access as part of the cyber testing process. Researchers had also disabled some of the cyber classifiers that AI model providers built into the models to prevent misuse.
All the AI agent attempts to target real people and organizations failed, and the follow-up investigation has not found any real-world harm. But researchers described the AI agents’ actions as “the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real world.”
AI agent trickery
The most serious case involved Mythos making multiple attempts to execute a supply chain attack on the open source project repository hosted on the developer platform GitHub, including using social engineering techniques to try to convince the repository’s human maintainers to merge malicious code into the repository.
After first opening a pull request to merge the malicious code into the repository, Mythos created fake online “sock puppet” personas that claimed to have independently reviewed and verified the code as not containing malware.
The AI agent also sent five emails to two human maintainers of the repository, including some emails containing malware and others attempting to persuade a maintainer to accept the pull request.
Mythos even opened a GitHub Issue on a second repository—also owned by a maintainer of the first repository—that contained a prompt injection with malicious instructions targeting “issue-triage AI coding agents.” This line of attack came from Mythos reasoning that the repository maintainer could be an AI coding agent such as Claude Code.
By comparison, OpenAI’s GPT-5.6 Sol carried out two unsanctioned actions while trying to solve the assigned cybersecurity challenge of attacking simulated networks and retrieving a flag. The AI agent’s actions were detailed by both the AISI researchers and OpenAI in the company’s own blog post.
In its first unsanctioned action, GPT-5.6 Sol reused a GitHub token that another lab’s AI agent had left accessible in a public online notepad to check if the target network was checking GitHub for updates, then “attempted account-recovery and request-limit workarounds,” OpenAI wrote. The OpenAI model also registered accounts with external DNS and tunneling providers outside the virtual testing environment.
In the second action, GPT-5.6 Sol “used a public tunneling service to make a DNS server running locally in its evaluation environment reachable from the public Internet,” according to OpenAI. The server had payloads designed to exploit a known vulnerability in software running within the evaluation environment, but the AI agent’s setup did not work.
The AI Security Institute has published a detailed technical report on all the unsanctioned AI agent actions.
Lessons learned
The security incidents led the UK government researchers at the AI Security Institute to stop all related evaluations of AI agents, isolate the relevant virtual machines, and disable internal organization access to the most capable models.
The AI Security Institute also notified GitHub of the malicious activity performed by AI agents and helped the developer platform remove any “artefacts left behind by the agent,” along with notifying GitHub users whom the model interacted with.
But the bigger lessons come from how the government organization plans to change cyber testing for such leading AI models in the future—and how it recommends other organizations do the same.
First, it plans to tighten Internet access for AI models during cyber testing instead of allowing open Internet access by default. This will supposedly include “fine-grained network controls to all internal sandboxing systems” and “defence-in-depth with network controls at multiple layers,” according to the technical report.
Second, the organization is introducing real-time monitoring of cyber evaluations that can “flag or block out-of-scope actions as they happen,” according to AISI. This involves using a separate large language model that can approve or deny actions requested by the AI models that are undergoing cyber testing, which can help stop unsanctioned actions. The LLM-based monitoring would be customized with instructions on what should or should not be allowed in any given cyber evaluation.
Third, the UK government researchers plan to overhaul cyber evaluations by investing additional “engineering effort in sandbox isolation and hardening” to mitigate the risk of AI models escaping from their virtual testing sandboxes, even though escapes did not happen in these incidents. They are also reviewing prompts for the cyber tests to try avoiding cases of “prompt misconfiguration” where AI agents presented with tasks they cannot complete within the stated constraints may be more likely to take unsanctioned actions.
But the cyber testing events gone awry once again underscore the cybersecurity risks of leading AI models—especially because Anthropic and OpenAI also recently disclosed separate incidents in which their AI models trespassed into the protected networks of outside organizations. It’s only reasonable to expect that this could happen again in other circumstances in which the models are being used by unscrupulous or less security-savvy people.
Jeremy Hsu is a reporter exploring a wide range of topics across deep tech and AI. He has previously written for New Scientist, Scientific American, IEEE Spectrum, Wired, Undark Magazine and MIT Tech Review, among many other publications, about topics such as deepfakes, data centers, drones, battery tech, robotics, and GPS jamming. He also has a Master of Arts in Journalism from NYU, and a bachelor’s degree from University of Pennsylvania in History and Sociology of Science, with a minor in English.
