AI is changing the terms of cyber defence

AI can help defenders analyse threats and respond faster. But there is a basic problem: a defender’s actions can cause harm too.

The UK National Cyber Security Centre made this point in “One does not simply defend agentically”. Attackers can try different approaches and move on when one fails. Defenders have to consider what else might break when they act. Isolating a device might stop an intrusion. It might also interrupt a service that people rely on.

That difference is easy to lose in the enthusiasm for AI-powered cyber defence. If attackers are using AI, the argument goes, defenders need AI as well. There is truth in that. But it is not enough to ask whether a system can respond quickly. We also need to ask whether it understands the consequences of its response, and who is answerable if it gets things wrong.

The defender’s burden

A cyber attacker may need to find one weakness. A defender has to make sense of an entire environment: networks, suppliers, staff accounts, legacy systems and all the connections between them.

AI could help security teams work through large volumes of information and identify patterns that are difficult to spot manually. The NCSC has also warned that more capable AI could speed up some cyber tasks and leave defenders with less time to respond.

But a fast response is not necessarily a good response. An AI system might correctly identify suspicious activity and still recommend an action that disrupts an essential service. The technical assessment and the operational decision are related, but they are not the same thing.

This is especially important for organisations responsible for services people cannot simply do without. In Australia, a response affecting a hospital, government service or energy network could reach well beyond the security team that initiated it.

More than a technical decision

Agentic AI brings the issue into sharper focus because these systems can do more than generate advice. Depending on how they are designed, they may use tools, access systems and take actions.

There is a significant difference between asking an AI to summarise an alert and giving it the authority to disable accounts or change network settings. The more it can do, the more important it becomes to understand what it can access, what it is permitted to change and how its actions can be stopped or reversed.

ASD’s Australian Cyber Security Centre sees opportunities for AI to strengthen cyber defence, while also stressing the need to manage the risks of adopting it. Its guidance on agentic AI highlights the security issues that arise when systems can act within connected environments.

This is governance in its most practical form. It is about deciding who, or what, has authority to act, and who carries responsibility when that action has consequences. An organisation cannot hand that responsibility to a system simply because the system acted quickly.

The systems underneath

There is another problem: AI can only work with the environment it is given. If an organisation does not know what its systems connect to, which services depend on them or how to restore them, an AI agent will not fill in those gaps. It may act on an incomplete picture.

That is why the basics still matter. Data governance fundamentals such as asset knowledge, along with reliable logging, clear ownership and tested recovery processes may seem less exciting than autonomous defence. They are also what make it possible to use automation without losing control of the environment.

NIST’s Cyber AI Profile puts the issue in a wider frame: organisations need to secure AI systems, consider how AI is used for cyber defence, and prepare for AI-enabled attacks. These are connected problems. AI is not just another tool for the security team. It is also part of the environment that needs protection. NIST urges a rethink of cybersecurity for the age of AI.

Who gets to act?

We are likely to see more AI used in cyber security. The question is not whether to use it, but where to draw the line between assistance and authority.

An AI system may help a defender see more, process information faster and spot activity that would otherwise be missed. That is useful. But in critical systems, being able to act is not the same as having the right to act.

The test should not be how much work can be handed to an agent. It should be whether the organisation can understand its actions, contain their effects and recover if they go wrong.

Attackers may gain speed from AI. Defenders need to gain capability without giving up control.