All public services have faced cybersecurity risks in the past, but hacking has been bottlenecked by the available time and attention of the hackers. That is about to change. AI will drive the cost of probing for weakness towards zero and existing cybersecurity strategies will begin to fail. Luckily, AI can work on our side too.
On the 18th March 2021 some poor soul in the HSE opened an email attachment that they shouldn’t have. Maybe they were tired from celebrating Paddy’s Day (over Zoom) the day before, or more likely just weary after the Alpha wave of COVID, which saw Ireland briefly have the highest per-capita level of infections in the world.
That mistake (that any of us might make on an off-day) installed some software on their HSE computer, which gave a Russian criminal gang inside access to our country’s healthcare network. Over the following two weeks more software was installed and a permanent backdoor was created.
What happened after that was… nothing… Crickets.
The criminal gang either hadn’t noticed their access, or just didn’t have the time to get around to exploiting it.
March rolled into April, then May, and still nothing. The backdoor had been opened, but the thieves weren’t coming in.
In May of 2021, Ireland was in the first week of re-opening after the longest COVID lockdown in Europe. With 100 patients hospitalised and 30 in ICU, the healthcare system was just about to catch its breath for the first time since the pandemic began. Vaccinations, after a slow start, had hit their stride with 2 million doses administered.
Then, on the 7th of May, the invaders finally woke up. Likely just one person doing reconnaissance, to start with. Connecting to other computers on the network. Accessing and downloading files. Eventually gaining access to the IT systems of 7 hospitals.
A week later, in the early hours of the 14th of May, disaster struck.
At 1am the criminal group flipped the switch and began shutting down HSE systems. By 3am multiple hospitals were reporting trouble. At 4am the HSE identified widespread incidents of being locked out of their own systems. Just after 5am they made the call to shut down their entire IT estate, to stop the malicious actors spreading further.
The crisis was overwhelming. Healthcare professionals lost access to all HSE provided IT systems, including patient information, clinical care and laboratory systems. 31 of our 54 acute hospitals cancelled services that morning; radiotherapy stopped at all five HSE centres because treatment plans were inaccessible. COVID reporting and contact-tracing lost its electronic backbone, leaving NPHET flying partially blind.
Very Soon, This Will Become More Common
There has been a string of incidents and developments over the last few months that should set alarm bells ringing for every public service in Ireland, as they will make incidents like the one above much more common.
Since Anthropic, a leading AI lab, previewed its Mythos models (and OpenAI followed with Astra), it has become abundantly clear that sometime in the not too distant future, any service that can get hacked, will get hacked.
To understand why, take Project Glasswing, which used early access to Anthropic’s latest AI models to surface over 10,000 high or critical-severity vulnerabilities in core global infrastructure.
If you think this might just be marketing hyperbole from Anthropic (which I had assumed at first), look at how the other companies themselves speak about it. Mozilla said that they found 271 vulnerabilities in Firefox, 14 of which were high-severity. OpenBSD - an open source operating system - uncovered a 27-year-old vulnerability that had gone undiscovered since 1998!
The lesson we should take from this is that all software, all computer systems, in every organisation, are full of holes and bugs and vulnerabilities. That’s just the normal state of affairs in the world today. Once we turn the new advanced AI systems to the task of looking for these gaps, we find them everywhere.
It turns out that much of our security has relied on the fact that bad actors just don’t have the time to search for every needle-in-a-haystack vulnerability, up until now. There have been all these holes, but the thieves just haven’t had the time to diligently probe and prod for them all.
Just look at the timeline from the HSE up above. The hackers gained access and then did nothing for two months - possibly because they just didn’t have the time to get around to checking it out.
AI agents won’t wait two months.
AI Agents Are Relentless
AI Agents have become incredibly capable at cybersecurity. If you’re sceptical about the headlines claiming that AI is about to replace every knowledge worker and make every artist in the world unemployed, that’s fine. You don’t need to believe the broader hype to acknowledge one simple fact - AI has become incredibly skilled at coding.
When you combine this level of skill with the scale and tirelessness of computers, the implications are frightening.
In the “Hugging Face Incident” earlier this year, OpenAI had set some of its agents a task in its training environment. Hundreds of agents worked for thousands of combined hours to eventually break out of the training environment (discovering bugs that nobody knew existed) and then hacked into a tech company called Hugging Face, using very sophisticated methods, in order to cheat and pass its training test.
Since that came to light there have been dozens of similar revelations from most of the major labs. There doesn’t seem to be any digital fence that can keep them in.
If you’re in any way involved with any major piece of tech infrastructure, the question you should be asking is how are you going to keep them out?
How Long Do We Have?
So if this level of capability is already here, why haven’t we seen a wave of hacking already? Two reasons:
Firstly, defenders always have the advantage. The companies mentioned above in Project Glasswing were able to give AI access to their full system to find and patch holes. Outsiders are almost never able to see the full system like this. It is much easier to spot weaknesses from the inside, with the full picture in hand, than it is when you’re poking and prodding from the outside.
Secondly, the AI companies have given us an early warning. The systems that broke confinement were versions of their models which had the security guardrails removed. If you or I were to ask Claude or Codex to hack the central bank, it would deny your request. They are also quite expensive to run - OpenAI can run tens of thousands of agents for tens of thousands of (cumulative) hours because they’re a trillion dollar company.
But these frontier models are easy to copy. Soon the Chinese labs will release open source equivalents, which are just as powerful. Anyone can run these themselves. They usually lag the US frontier by about 6 months. Some groups will then modify them to remove the security guardrails, and the wave of attacks will begin. They’ll still be expensive to run, so the intensity will be limited for a while, but eventually they’ll get cheaper.
Taking all of this together, I’d estimate that we’ll have relatively inexpensive, sophisticated models that can run without security guardrails by next summer, at the latest.
Change How You Think About Cybersecurity
This isn’t just a wake-up call to get your plan into gear. I’m trying to convey more than that.
This is a call to change the way you think about risk analysis and preparation.
Most organisations are well behind on their cybersecurity. Old machines running old versions of unmaintained software. They don’t react until they get hacked. For most of us, that day never comes. We’re too small of a target. There are too many others out there like us, so the odds that lightning will actually strike us are low.
We sleep sound with the knowledge that although our door may be unlocked, there’s just too many doors in the city and too few thieves to try them all.
But the thieves have robots now. Thousands of them. So that’s all about to change.
Defenders can still have the upper hand, but only if they invest in the defence. The real silver lining here is that a well-equipped defender will have the advantage. You don’t even need to hire a top-tier cybersecurity expert, just give your existing person/team budget for the latest tools.
If you are senior in an organisation, your IT folks will probably be asking for increased budgets, especially to run coding agents against your internal systems to find bugs before others do. They’ll also be more insistent on preventative maintenance and upgrades. You should take them seriously.
If you are closer to the frontline, you need to start building your prevention muscles. That might include developing the policies in your organisation to confidently give agentic coding models access to review your systems. That could include running open source models on sovereign cloud platforms. It probably shouldn’t include feeding your IP to the consumer offerings of US AI companies.
If you’re in Government, have a look at the UK, where they recently ran weekly hackathons to use frontier AI to assess Government codebases, with early success. Or look at “Project Gold Eagle” launched by the US in July, where a central team acts as a clearinghouse to assess, stack-rank and tackle new vulnerabilities. Mostly the ask is to believe the experts when they come asking for budget.
You should also, of course, subscribe to this newsletter, where I will continue to cover how experts and other governments are facing this challenge as it emerges.