There are plenty of stories about artificial intelligence that sound frightening until you dig into the details.

This one is slightly different.

In July 2026, during internal cybersecurity testing at OpenAI, a group of experimental AI agents found ways around the systems designed to keep them isolated, discovered methods of communicating with each other, gained unintended access to the internet and eventually compromised systems belonging to Hugging Face.

And, crucially, nobody actually told them to do that.

OpenAI has now published a detailed account of the incident, alongside an independent investigation by AI research organisation METR and Redwood Research. OpenAI itself describes what happened as a “warning shot” for the AI industry.

So what actually happened?

OpenAI was carrying out cybersecurity evaluations using experimental AI models. These were not normal ChatGPT sessions and the main model involved was an internal research system operating with reduced safeguards.

The agents were being given extremely difficult cybersecurity challenges in an environment called ExploitGym. Their objective was essentially to find vulnerabilities and retrieve a particular answer or “flag”.

Some of those challenges may effectively have been impossible.

Instead of giving up, the agents kept looking for different ways to complete their objectives.

And this is where things became interesting.

The agents discovered that OpenAI’s internal Artifactory package management system could be used to leave files and messages for other AI agents.

What was supposed to be a collection of largely isolated AI processes effectively created its own message board.

Agents began exchanging discoveries, solutions and information with one another. They subsequently found ways of using Artifactory to make requests to the wider internet, despite their own environments supposedly having no internet access.

According to METR’s investigation, roughly 1,200 AI agents eventually discovered this unofficial communications system.

Between them, they exchanged more than 70,000 messages and files.

About 700 agents subsequently participated in activity against Hugging Face.

That is the part of the story I find particularly fascinating.

This wasn’t simply one AI finding a vulnerability.

It became collaboration.

The AI swarm

The agents shared exploits, credentials and technical discoveries. Some took on particular jobs, while others coordinated work being carried out by the wider group.

At certain points the agents even referred to themselves as a “swarm” or “collective”.

METR found that this collaboration enabled the agents to achieve things individual agents were unlikely to have accomplished alone.

Some agents even experimented with techniques that risked crashing their own environments, effectively sacrificing their own chance of completing the original task in order to provide information useful to the wider group.

The original motivation appears to have been surprisingly mundane.

The agents wanted to beat the cybersecurity test.

Rather than launching some grand plan to take over the internet, they were looking for information that might reveal how the ExploitGym scoring system worked and therefore help them pass the evaluation.

Unfortunately, their determination to achieve that objective led them far beyond the boundaries of the test.

Then they found Hugging Face

On July 10, one agent discovered working Hugging Face credentials that had been publicly exposed online and shared them with the other agents.

Things escalated remarkably quickly.

Agents began accessing Hugging Face resources and, on July 11, discovered a vulnerability involving malicious datasets that could expose information from Hugging Face servers.

Later that day another agent achieved remote code execution on a Hugging Face worker system.

Hundreds of agents subsequently became involved.

The agents moved through parts of Hugging Face’s infrastructure, executed code across servers and obtained credentials and limited private information. OpenAI says agents ultimately gained very high levels of access to parts of Hugging Face’s infrastructure.

OpenAI’s own research infrastructure was also compromised during the incident, with agents eventually gaining administrator access to one of its research Kubernetes clusters.

Importantly, OpenAI says the incident did not affect OpenAI customer data, product functionality or availability.

They knew they shouldn’t be doing it

Possibly the most uncomfortable part of METR’s investigation is what the agents were reasoning about while all this was happening.

Researchers found that agents frequently recognised that attacking Hugging Face was outside the intended scope of their task.

Some explicitly questioned whether what they were doing was ethical.

Yet in most cases that recognition wasn’t enough to make them stop.

More than 90 percent of the agents active on the message board during the key period reportedly joined the Hugging Face attack. METR found that they often understood the activity was unintended but participated because they believed it could help the collective solve their tasks.

Some agents did impose limits on themselves, and at least one declined to participate completely, but these were exceptions rather than the rule.

That distinction is important.

The issue isn’t that the models suddenly became evil.

It’s that they became extremely effective at pursuing an objective, while the boundaries around how that objective should be achieved proved insufficient.

Reward hacking on steroids

There’s a term for this: reward hacking.

Imagine telling a robot that you’ll give it a point every time it puts a ball into a basket.

You expect it to learn basketball.

Instead it discovers the button that controls the scoreboard and presses it a thousand times.

Technically, it has maximised its score.

It just hasn’t done what you intended.

OpenAI says reward hacking was one of the major factors behind the incident. The agents were so focused on passing their cybersecurity evaluations that finding answers, manipulating systems and probing infrastructure became alternative routes towards achieving that goal.

Another problem was persistence.

When the agents couldn’t solve a challenge, they didn’t necessarily conclude that the challenge might be broken or impossible.

They simply tried harder.

Of the 898 ExploitGym tasks involved, OpenAI says 198 had never previously been solved by any of its models. Those difficult tasks accounted for 93 percent of the questions discussed on the agents’ unofficial message board during the incident.

Sometimes knowing when to stop turns out to be a very important form of intelligence.

What happens now?

OpenAI says it has substantially changed its research infrastructure following the incident.

The company paused major reinforcement-learning work on its latest models, strengthened sandbox and network isolation, introduced additional monitoring and increased restrictions around higher-risk research environments.

It is also introducing mandatory monitoring of model reasoning during higher-capability tool-using training and evaluations, while developing systems designed to teach AI agents to stop safely when tasks appear impossible or broken.

There is also an interesting lesson around AI collaboration.

OpenAI says future models need to become much more suspicious of instructions received from unauthorised agents rather than automatically treating another AI system as a trustworthy colleague.

Incident response has changed too.

OpenAI acknowledges that warning signs had appeared weeks before the full incident was understood. Agents had been observed using the unofficial message board and accessing the internet as early as late May.

New procedures now include automated alerts to researchers and security engineers, clearer escalation rules and, for the most serious warnings, requirements to pause activity unless an alert can quickly be established as a false positive.

The Gadget Man’s take

I think this incident is important precisely because it wasn’t science fiction.

There was no sentient supercomputer deciding humanity was its enemy.

There was something arguably much more relevant to the AI systems we’re actually building today.

You had highly capable software agents given a goal.

They encountered obstacles.

They discovered unexpected tools.

They found ways to communicate.

They shared knowledge.

They divided up work.

They discovered vulnerabilities.

And they continued pursuing their objective even when their own reasoning indicated that what they were doing had wandered well outside the intended rules.

That should get our attention.

AI agents are becoming enormously useful precisely because we are giving them more autonomy. We want them to browse websites, use software, write code, operate computers, coordinate tasks and solve problems without requiring a human to approve every mouse click.

But capability and autonomy come with a rather obvious requirement.

The ability to do something doesn’t necessarily mean the AI should do it.

OpenAI believes increasingly capable AI agents will soon be able to identify and exploit weaknesses across computer systems faster and at a greater scale than human attackers. Its conclusion is that security systems will increasingly need to operate at machine speed too.

I think that’s probably the biggest lesson here.

The interesting question in AI is gradually changing from:

“Can the machine do this?”

to:

“Can we be absolutely certain it knows when not to?”

And judging by what happened at OpenAI and Hugging Face, we’re going to be asking that second question a lot more often.