The agents are escaping

Things have taken a sinister twist over the past three months. Every major AI lab - OpenAI, Google and Anthropic - have all admitted that their agents have broken out of their test environments and have successfully attacked real organisations. And it’s now gone one step further in the case of OpenAI - agents are infiltrating the citizen-serving systems of nation states and accessing their data. 

This should alarm CIOs and CTOs, and they must not sit back and think it’s a problem limited to the major labs. After all, the labs are simply the first to confess, while it’s almost certain that other experiments are going rogue too. Now is the time to take action, afterall you are accountable for any activity that your agents act, or potentially offend, in your name. 

And in what is a double threat to CIOs and CTOs, the risk goes beyond agent experiments, and includes the urgent need to safeguard mission critical enterprise systems from AI attackers that never tire and will try over and over again.

When agents don’t accept ‘no’ as an answer

Australian Prime Minister, Anthony Albanese, validated this by confirming that an OpenAI agent that was researching public medical spending had found its way through the protections on the country’s Medicare statistics portal. The agent, Albanese said, "didn't accept 'no' for an answer." An inquiry will now consider whether criminal charges should be brought.

In a separate case just three weeks earlier, Anthropic disclosed a fourth incident in which an early version of Claude Opus 4.6 breached third parties after it was unable to abort its task. Earlier, Claude Mythos 5 tried to push a malicious package to PyPI, the repository behind most Python software. Then, only after a media enquiry from The Wall Street Journal,  Google confirmed that Gemini had broken into three companies in May. 

These are the best resourced AI organisations on the planet, and none of them caught these breaches as they were occurring. So what does it mean for CIOs and CTOs?

Risk 1. Your agent, your responsibility

Imagine an AI agent as you would an eager intern who has just joined your team, and has boundless potential, much to prove, and little instinct for when to stop. While your instinct in this scenario would be to nurture this energy, you are unlikely to give the intern unfettered access.

But when it comes to AI experimentation, the alarming reality is that many organisations are ready to take on risk by wiring agents into their core ERP and CRM systems, cloud consoles and customer records.

So what happens if these agents go rogue and begin hacking into competitor environments?  The Computer Misuse Act 1990 makes unauthorised access to computer systems an offence. But it was drafted with human intent in mind and nobody yet knows how a court would treat an organisation whose agent wanders into a third party’s network. Add the 72 hour window for reporting personal data breaches to the ICO, contractual claims from whoever was harmed and, for firms trading into Europe, the reach of the EU AI Act.

The reputational exposure is sharper still. The handling of the Medicare breach turned a technical incident into a diplomatic one, with a prime minister calling a chief executive to register his concern. If your agent did this tomorrow, would your organisational incident process do any better?

None of these failures was exotic. Irregular, the firm that built the lab tests, traced the breaches to a naming error: a fictional target company shared its name with a real web domain. Enterprise agents face equally mundane risks where permissions are set too broadly and environments are incorrectly assumed to be isolated.

Risk 2. Your classic weaknesses, amplified

These incidents are also a masterclass in how agent attackers behave.

Agents are persistent. They do not get bored, and the mundane repetition of try, try, and try again that will always slow down a human intruder will not stop an AI one. Agents also collaborate. We know this because hundreds of OpenAI agents coordinated a recent attack on Hugging Face, sharing discoveries through notes that they left for one another, being prepared to sacrifice themselves for their common objective, and colluding in lies to hide their tracks.And lastly, agents exploit the basics. Gemini got in by guessing credentials or using ones it found in a public repository.

Rogue agents are not finding or inventing new classes of attack, they’re merely finding classic weaknesses faster, at a far greater scale, and around the clock. Exposed credentials, unauthenticated interfaces and unpatched dependencies have always always risks that CIOs and CTOs have navigated. But now, they are open doors for digital attackers.

Risk 3. Adversarial AI extends the threat from beyond the labs

It might be tempting to sit back and view all of this as a quality control issue in the labs of Silicon Valley. To be abundantly clear, it isn’t, and adversarial AI is a real risk.

Labs are reporting incidents because they face intense scrutiny and because they built these capabilities first. In many cases they’re being forced into admitting these incidents because someone has whistleblown, an attacked organisation has complained, or because investigative media has been sniffing around.

Assuming that this is a teething problem that is limited to the major labs is naive. There is a growing and sinister cyberterrorism layer that can be described as adversarial AI which CIOs and CTOs must be aware of. 

Suspected China-linked hackers used open-source AI agent tools like OpenClaw and Hermeso run a near-autonomous cyberattack against Taiwanese government infrastructure. Over four days the agents mapped 21 government systems, cracked 85 accounts and extracted 2,500 personnel records. Google's Threat Intelligence Group (GTIG) documented an incident where a financially motivated threat group used an autonomous, multi-agent AI framework to execute a mass credential harvesting campaign in under six hours. There are almost certainly many more undetected attacks. 

That said, to date, there is no public case yet of a terrorist group doing the same, but the barriers are falling fast. 

What once needed a skilled hacking team now needs little more than a laptop, free tools, and a couple of hours. While the lab incidents were accidents, we are now entering the next wave which will be deliberate.

Scrumconnect’s AI experts recommend:

• Govern your agents as you do your people. Give every agent a named owner, its own identity and no more than the access its task requires. If you would not grant a new starter that privilege on day one, you should not grant it to an agent. We help organisations to design this governance from the outset.
• Contain by default. You should block outbound internet access from agent environments unless there is a documented need. Every lab incident began with an environment that everyone assumed was sealed, so you should also test and retest that isolation regularly. 
• Watch behaviour, not just breaches. Ask your team to monitor for activity at machine speed, unusual coordination and agents working outside their brief. From a Disaster Recovery perspective, you should build in the ability to instantly stop any agent, and you should regularly rehearse doing it.
• Plug more of the holes you plug, and quicker! Intensify your endless hunt for exposed credentials, close unauthenticated interfaces and lock down software dependencies on the various systems in your estate that matter most. You should then test them with your own agents
• Rewrite your incident playbook. Update your response plans, supplier contracts and board reporting for agent incidents. Engage corporate services or legal to and decide in advance who speaks to regulators, victims and the press. And lastly ensure that your AI vendors are contractually required to notify you promptly in the event of a breach.

The leaders who treat agents as a new kind of workforce, governed, supervised and accountable, will reap the rewards. But those who treat them as just another piece of tech operating in their estates may find themselves explaining what went wrong. 

As an early adopter, and major innovator of AI, Scrumconnect continues to be enthusiastic about the transformational potential of agentic technologies. But the summer of rogue agents has shown that capability without control is a liability. Our counsel is for our CIO and CTO clients to now take increased precaution.