Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
Tool-using AI needs explicit authority¶
OpenAI has begun introducing plugins that let ChatGPT retrieve current information, run computations, and interact with external services. The early examples include browsing, code execution, travel, shopping, and other application connections.
This changes the nature of the system. A model that produces text can mislead. A model connected to tools can also act.
Capability is not authority¶
A plugin gives an artificial intelligence (AI) system a new capability: call an application programming interface (API), retrieve a record, reserve something, calculate a result, or initiate a transaction. It does not follow that the system should exercise that capability whenever its language model infers that doing so would help.
Organizations already understand this distinction for people and software services. An employee may know how to approve an expense but lack the authority to do so. A service account may be technically able to reach a database but should receive only the permissions needed for its function.
Tool-using AI needs the same discipline, made more explicit because natural-language instructions are ambiguous and external content can influence model behavior.
Least privilege must extend into the conversation¶
Saltzer and Schroeder's principle of least privilege remains the right starting point: every component should receive only the access necessary for the task and for no longer than necessary. For plugins, that means more than limiting the underlying credential.
The user experience should also express scope. A system asked to “help plan a trip” should not have to infer whether it may search, reserve, purchase, or disclose traveler information. Those are different authorities.
A useful action model separates:
- Read: retrieve information without changing external state.
- Draft: prepare a proposed action for human review.
- Commit: execute a reversible external action.
- Irreversible commit: take an action that is difficult or costly to undo.
Each level should have different permissions, confirmations, and evidence. The model should not climb the ladder simply because the conversation became more persuasive.
Confirmation must support a real decision¶
A generic “Are you sure?” prompt is weak oversight. Users habituate to it, especially when the system has already done most of the work. A meaningful confirmation should show the proposed action, target, data disclosed, cost, assumptions, and reversal path.
Human–automation research warns that people can become passive monitors of systems they usually trust. Bainbridge's ironies of automation are especially relevant here: the human may be asked to catch the rare, consequential mistake after the machine has made routine action effortless.
For higher-risk actions, the interface should introduce useful friction. Require the user to resolve an ambiguity, choose among options, or inspect the source that supports the action. The goal is not to annoy the user. It is to restore enough situation awareness for consent to mean something.
Design for recovery before autonomy¶
Before connecting a model to a production tool, teams should answer:
- Can every action be attributed to a user, model, plugin, and configuration?
- Are external instructions treated as untrusted data rather than higher-priority commands?
- What prevents the model from chaining several individually permitted actions into an impermissible outcome?
- Can the user preview, cancel, reverse, and report an action?
- What happens when the plugin returns incomplete or malicious content?
- Which actions always require a second person or separate service?
The National Institute of Standards and Technology Zero Trust Architecture offers the right posture: do not grant broad trust based on location or prior success; evaluate access explicitly and continuously.
Plugins make conversational systems more useful because they connect language to the world. They also collapse the distance between an incorrect interpretation and a real consequence. The design principle should be simple: give the system capability in small increments, express authority unambiguously, and make recovery part of the feature before action becomes routine.
Sources and research trail¶
- OpenAI, “ChatGPT Plugins” (March 23, 2023).
- Saltzer and Schroeder, “The Protection of Information in Computer Systems” (1975).
- National Institute of Standards and Technology, Zero Trust Architecture (2020).
- Bainbridge, “Ironies of Automation” (1983).
- Parasuraman, Sheridan, and Wickens, “A Model for Types and Levels of Human Interaction with Automation” (2000).