Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
Agents need a control plane outside their reach¶
Google DeepMind has published an Artificial Intelligence Control Roadmap for securing internal systems as agents become more capable and operate with greater autonomy. The work considers how organizations can manage systems that may be useful, imperfectly aligned, and able to interact with valuable digital resources.
One principle deserves broad adoption: the mechanisms that observe and constrain an agent should not depend entirely on the agent's own willingness or ability to comply.
Oversight inside the same failure boundary is fragile¶
An artificial intelligence (AI) agent may plan, write code, call tools, inspect systems, and evaluate its own output. Those abilities can improve performance. They should not become the only basis for trust.
Asking the same system to report whether it has behaved safely creates a common-mode risk. A failure in interpretation, a compromised context, or an adversarial input may affect both action and self-assessment. The audit trail can become another generated artifact rather than independent evidence.
Safety-critical engineering addresses similar problems through independence and defense in depth. A protection mechanism is stronger when it does not share every assumption, component, and failure mode with the process it protects.
Control is more than a refusal policy¶
A robust agent architecture can separate a work plane from a control plane. The work plane pursues the assigned objective. The control plane establishes identity, scopes permissions, monitors actions, enforces resource limits, records evidence, and interrupts behavior when necessary.
Some controls belong below the model: sandboxing, network boundaries, tool allowlists, rate limits, transactional approvals, immutable logs, and credentials that expire with the task. Others require an independent model, human review, or specialized detector. No single layer is sufficient.
The National Institute of Standards and Technology's (NIST) zero-trust architecture provides a useful analogy. Trust is not granted permanently because an actor entered the environment successfully. Access decisions consider identity, resource, policy, and current conditions. Agent authority should be equally specific.
Control needs a concept of mission¶
Technical restrictions can prevent a tool call while missing an operationally harmful outcome. An agent may remain inside its permissions and still pursue the wrong objective, use stale information, or optimize a metric at the expense of the mission.
That is why the control plane needs context supplied by the organization:
- the intended outcome and prohibited side effects;
- the resources and people potentially affected;
- the actions that require human judgment;
- the evidence needed before an irreversible step;
- and the conditions that require the task to stop.
These constraints should be testable. “Use good judgment” is not a control. “Do not send an external message without named approval and a preview of recipients and attachments” can be implemented and verified.
Rehearse loss of control¶
Organizations should test more than whether the agent completes the happy path. Exercises can include misleading tool output, corrupted memory, permission escalation, conflicting instructions, unavailable services, hidden prompt injection, and attempts to alter logs or monitoring.
The exercise should ask whether the organization can detect the condition, contain the agent, reconstruct the sequence, restore affected systems, and update controls. That turns a technical test into a resilience test.
Woods' work on resilience engineering distinguishes absorbing disruption from adapting to it. An effective control system supports both: it limits immediate harm and creates evidence for improvement.
Capable agents will be valuable because they can act across boundaries and sustain complex work. Those same properties make independent control essential. The goal is not to surround every action with friction. It is to establish a trustworthy outer structure that remains available when the agent's own reasoning is the thing in question.
Sources and research trail¶
- Google DeepMind, “Securing the Future of AI Agents” (June 18, 2026).
- National Institute of Standards and Technology, Zero Trust Architecture (2020).
- National Institute of Standards and Technology, Cybersecurity Framework 2.0 (2024).
- Woods, “Four Concepts for Resilience and the Implications for the Future of Resilience Engineering” (2015).
- Leveson, Engineering a Safer World (2011).