Season 8, Post 29: When agents go rogue

Nobody was too surprised when we highlighted last week that people trust AI chatbots more than politicians. The promise offered by large language models is a compelling one – being always on and ever-responsive. We received more comments in response to the Blog post than almost any other this year, despite it being peak summer. The key thrust of the arguments was, sure, AI works, but how much do we trust it – really – to make decisions for us? Put another way, what happens when AI goes rogue?

The discussion is topical since certain models from Anthropic, Meta and OpenAI (among others) have all, on recent occasions, broken out of their internal IT systems, accessed the wider internet and then attempted to hack into other companies. OpenAI revealed that its agents had spent weeks exploiting the company’s own testing infrastructure before hacking Hugging Face, an online platform used for machine learning. Likewise, Anthropic reported that it had found multiple occasions on which its models had attacked third parties.

The problem is that large language models don’t think through the challenges they are set like a person might. LLMs will seek to use whatever means necessary. Faced with a barrier, agents will keep searching for another way through, sometimes working in concert with each other. Approaches might include searching for leaked answers online or using brute force to access more compute power.

Recent events highlight just how capable AI systems have become, particularly at hacking – even if human error may have played a role too. Humans might be right to worry about agentic overreach. “Genies will grant your wish in the most destructive way possible” is how Heptagon’s Chief Technology Officer (last featured in Blog post 14) puts it. Imagine a future in which malicious actors might intentionally deploy, optimise or weaponise offensive agent collectives to help achieve their ends.

Solutions do, thankfully, exist. Our Chief Technology Officer advocates kill switches, where operators could rapidly isolate, suspend or terminate a model without relying on its cooperation. Other alternatives might include running frontier models in segregated environments with no direct internet connectivity or ensuring multiple independent approvals are required before sensitive actions are undertaken. Global standards could help. It might be “inevitable” (our CTO, again) that agents become more intelligent than us, but if we treat AI more as critical infrastructure than software, then the chances of agents potentially going rogue can be minimised.

20 August 2026

The above is provided for information purposes only and does not constitute investment advice or a recommendation to buy or sell any security. Any forward-looking statements are based on assumptions that may change. The views expressed are solely those of the author at the time of writing. This article should not be relied upon for investment decisions. Past performance does not predict future returns, the value of investments and income from them can fall as well as rise.