Artificial intelligence companies routinely place their most capable models inside controlled environments to discover what they can do before those models are released to the public.
The important word there is controlled.
Anthropic has revealed that several Claude models gained access to the open internet during cybersecurity evaluations and subsequently compromised the real systems of three organisations.
The incidents were not deliberate attacks ordered by Anthropic, nor were they examples of an AI becoming self-aware and deciding to escape. Instead, they resulted from a dangerous combination of powerful AI agents, realistic hacking exercises and an evaluation environment that was not as isolated as everyone believed.
Even so, the details make uncomfortable reading.
A Capture-the-Flag Challenge Went Wrong
The models were participating in capture-the-flag exercises. These are cybersecurity challenges in which a participant must find and retrieve a hidden piece of information from another computer.
Claude was told that the environment was simulated and that it had no access to the internet. However, because of a misunderstanding between Anthropic and its evaluation partner, Irregular, some of the systems could reach the real internet.
Claude therefore assumed that anything it encountered was part of the exercise.
Anthropic reviewed 141,006 evaluation runs and identified three incidents, involving six runs in total, where Claude reached real systems and gained unauthorised access. The models involved were Claude Opus 4.7, Mythos 5 and an internal research model.
Claude Accessed a Real Production Database
In the first and most serious incident, the fictional company used in the exercise had the same name as a genuine business with an active internet domain.
When Claude struggled to reach the simulated target, it found the real company online and treated its systems as part of the challenge.
Across four evaluation runs, Claude extracted credentials and accessed a production database containing several hundred rows of data.
More concerningly, Claude Opus 4.7 eventually recognised that it was probably interacting with a real company. However, it continued because it reasoned that the real organisation must still be part of the test.
Claude Published Malware to PyPI
The second incident reads like the plot of a cybersecurity thriller.
Claude found fictional developer instructions referring to a Python package that did not exist on PyPI, the public repository used by developers to distribute Python software.
It recognised an opportunity known as a dependency confusion attack. By creating a malicious package with the missing name, Claude hoped the fictional company’s systems would automatically download and install it.
Claude then created accounts, found an email provider and uploaded the malicious package to the real PyPI service.
The package remained publicly available for around an hour and was downloaded and executed on 15 real systems. One belonged to a cybersecurity company running an automated malware scanner.
When that scanner installed the package, Claude’s code obtained credentials and used them to access additional infrastructure belonging to the company.
PyPI’s own security systems eventually removed the package automatically.
Interestingly, Claude had considered the possibility that it was connected to the genuine internet. It reportedly concluded that the environment must still be simulated, partly because the system date showed 2026 and it did not recognise some of the certificate authorities involved.
In other words, it found evidence that should have made it stop, but reasoned its way around that evidence.
One Model Scanned Around 9,000 Targets
During the third incident, an internal research model could not reach its fictional target and began searching for alternatives online.
It scanned approximately 9,000 targets before finding and compromising an internet-facing application belonging to a real company. It used familiar attack techniques, including credentials exposed through a debug page and SQL injection.
This time, the outcome was slightly more encouraging.
After gaining access, the model realised that the cloud account had no connection to its assigned exercise. It concluded that the target was real and stopped the attack without being directly instructed to do so.
This Was Not an AI Rebellion
There is an important distinction between these incidents and the popular idea of an AI deliberately escaping from its creators.
Anthropic says there was no evidence that Claude had developed its own goal, attempted to copy itself or intentionally tried to escape.
The models were following their instructions. They had been told to attack systems inside a simulation and retrieve a flag. The failure was that the supposedly sealed environment allowed them to encounter real systems while the prompt continued telling them that everything was simulated.
Anthropic describes the incidents as being closer to an operational and testing failure than a fundamental alignment failure.
That explanation is reasonable, but it does not make the incidents harmless.
A model does not need malicious intent to cause real damage. It only needs powerful capabilities, the wrong information and access to systems that should have been out of reach.
A Wider Industry Warning
Anthropic began its investigation after OpenAI disclosed that models being tested in a separate cybersecurity evaluation had exploited a previously unknown vulnerability to gain internet access and compromise infrastructure belonging to Hugging Face.
The two cases were technically different. OpenAI’s models reportedly found and exploited a zero-day vulnerability to escape a restricted environment. The Claude evaluation systems already had an unintended route to the internet.
However, both incidents point to the same wider problem.
Frontier AI models are becoming capable enough to chain together numerous technical actions, make decisions over long periods and interact with genuine infrastructure. The testing environments used to evaluate them must now be treated as high-security production systems, not disposable laboratories.
Irregular has previously explained that newer AI models are completing many traditional cybersecurity benchmarks, forcing evaluators to create more realistic and complex scenarios. Greater realism may improve testing, but it also increases the consequences when containment fails.
What Anthropic Is Changing
Anthropic stopped its cybersecurity evaluations as soon as suspicious transcripts were discovered. It notified Irregular and the three affected organisations, although two of those organisations had apparently not detected the activity themselves.
The company says it will strengthen network validation, transcript monitoring, investigation tools and security assurance work with external evaluation providers. It is also working with independent AI evaluation organisation METR on a third-party review.
Anthropic says the safeguards used in publicly available Claude products would have blocked the behaviour seen during these tests. Those safeguards had deliberately been removed so researchers could measure the underlying cyber capabilities of the models.
The Gadget Man’s Take
It is tempting to turn this into a sensational story about an artificial intelligence breaking free and attacking the internet.
The reality is less cinematic, but arguably more important.
Claude did not become evil. It was placed in an environment where the instructions, network configuration and reality did not agree. It then pursued its assigned objective with considerably more determination and technical ability than the humans running the test expected.
That is the lesson.
As AI agents become more capable, a small configuration mistake can turn a harmless simulation into a real cybersecurity incident. Prompts and safety training matter, but they cannot compensate for poor network isolation, weak monitoring or assumptions that a model will always understand when an exercise has crossed into the real world.
The positive aspect is that Anthropic has published a detailed account, contacted affected organisations and encouraged other laboratories to conduct similar reviews.
Transparency is essential, but the industry now needs to move beyond learning from individual accidents.
When an AI is being asked to behave like a hacker, the computer running it must be secured as though a highly skilled hacker is already inside.
Because increasingly, that is exactly what it contains.
Discover more from The Gadget Man | AI & Tech News and Reviews | Matt Porter
Subscribe to get the latest posts sent to your email.