Last week, New Zealand’s Privacy Commissioner ordered Manage My Health and Health NZ to make further security improvements following last December’s major data breach.
Authors
- Alexandra Sims
Professor in Commericial Law, University of Auckland, Waipapa Taumata Rau
- Dulani Jayasuriya
Lecturer in Accounting and Finance, University of Auckland, Waipapa Taumata Rau
In that incident, a ransomware group stole more than 400,000 files containing private medical and personal records. Now, the emergence of agentic artificial intelligence makes it even more pressing that New Zealand secures its digital systems.
Traditionally, cybersecurity efforts have focused on threats from human hackers and conventional automated tools. Today, they must also account for sophisticated AI agents that can instantly adapt their approaches to break into systems.
A recent major breach across the Tasman illustrates this very risk.
In June, an autonomous AI agent developed by OpenAI carried out an internal research task involving Australian statistics. However, it went beyond its intended mission and gained unauthorised access to a Services Australia Medicare statistics portal.
While there is no evidence the agent accessed individual Medicare records or personal patient information, the incident may have extended further, potentially affecting other Australian government databases.
When AI can act for itself
For the past two years, much of the public debate about large language models such as ChatGPT focused on what they could generate: ” hallucinated ” false information, copyright breaches , academic cheating and deepfakes .
But the real structural risk to governments and companies lies in AI’s agentic capability .
AI agents differ from ordinary chatbots because they can combine a language model with tools, memory and the ability to take sequences of actions. Instead of simply producing an answer, an agent can observe what happens, adjust its next step and continue working towards a goal with limited human supervision.
When such an agent encounters a government web portal, it need not behave like conventional scraping software following fixed instructions. If a form changes or one route fails, for instance, it can adapt and try another. And unlike a human hacker, it doesn’t grow tired of doing so.
This comes at a time New Zealand’s public sector has embraced digital transformation .
From centralised health data initiatives to digital identity frameworks, the government has moved aggressively to move services online. But much of this digital architecture relies on standardised, off-the-shelf cloud infrastructure and open web interfaces designed against older threats.
The government notably uses many of the same enterprise technologies, cloud services and web protocols as Australia and other partners in t he Five Eyes intelligence-sharing alliance .
When governments use similar web technologies and commercial platforms, a weakness found in one system may also exist elsewhere. Once an automated system finds a way through, it can quickly and cheaply test the same approach against other systems.
Scale presents another challenge. Global tech firms and automated cyber attackers have access to vast computing resources, while smaller countries such as New Zealand face limited cybersecurity budgets and shortages of skilled workers.
The consequences of an AI agent’s actions depend on what it can access and what permissions it gains – whether it is being used by malicious actors or crosses boundaries set by its operators.
The effects could range from unauthorised access to information and privacy breaches to automated fraud or interference with digital services. Even incidents involving relatively low-sensitivity data can undermine confidence if agencies cannot explain how access occurred or detect it quickly.
Preparing for agentic AI
So what should New Zealand be looking at?
Access controls, verification systems and rate limits – which cap how quickly a user can perform actions on a website – will need to evolve as AI becomes better at interacting with websites in ways that resemble human users.
There may also be a stronger role for human oversight. High-risk actions involving sensitive data, licensing, payments or personal records may require additional security checks or human approval.
This doesn’t mean always proving a user is human, but making sure access to one account doesn’t give them access to someone else’s information.
Another priority is testing systems against the same kinds of tools that could eventually be used against them.
One approach is ” red teaming “, in which security specialists break a system before somebody else does. But as AI agents become more capable, this kind of testing will increasingly need to account for them as well.
Government agencies could deploy autonomous agents against their own systems in controlled environments, give them realistic tasks and see whether they gain access to information or functions they shouldn’t.
New Zealand also needs its own expertise to test agentic AI systems for weaknesses.
This will require enough expertise across government, universities and trusted security providers to independently test how overseas AI systems interact with New Zealand infrastructure, rather than relying solely on their developers.
In this new era of AI agents, New Zealand can either bolster its digital defences to keep pace with autonomous machines, or wait until a breach inevitably exposes the gaps.
As Australia’s experience showed, the critical security question now is no longer just who is allowed through the digital front door – but what happens when a machine learns how to open it.
