Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
Model behavior is an organizational decision¶
OpenAI has published a useful account of a difficult problem: how should a conversational artificial intelligence system behave, and who should decide? The company describes tensions among default behavior, user customization, safety boundaries, and the wide range of values held by people who use the system.
The question is often framed as model alignment. For organizations deploying these systems, it is also product governance. Every default encodes a decision about authority, acceptable variation, and whose judgment applies when values conflict.
There is no neutral default¶
Artificial intelligence (AI) systems do not enter a workflow without a point of view. Their training, fine-tuning, instructions, filters, and interface shape which requests they accept, how confidently they answer, and when they refuse. A default may be broadly useful without being universally appropriate.
That matters because defaults are sticky. Users often interpret them as recommendations, and product teams may inherit them without examining whether they fit the local mission. In a low-consequence creative task, a flexible default may be desirable. In a regulated or safety-critical process, the same flexibility may produce ambiguity about which rules govern the output.
The organizational mistake is to assume that the provider's model behavior resolves the customer's policy problem. It cannot. The provider can establish a general safety boundary. The deploying organization must still decide what the system is authorized to do in a particular setting.
Customization creates a governance surface¶
Customization is attractive because it can adapt a general model to different users and contexts. It can also fragment behavior. If every team creates its own instructions, examples, and exceptions, the organization may end up operating many distinct systems while believing it has deployed one.
Those configurations deserve the same discipline applied to other consequential software artifacts. They need owners, version control, testing, approval, and retirement. A changed system prompt can alter behavior as meaningfully as changed application code.
The National Institute of Standards and Technology Privacy Framework provides a useful analogy: organizational values become actionable through profiles that adapt common outcomes to a specific context. AI behavior can be managed similarly. Enterprise principles establish the common boundary; use-case profiles specify local behavior, evidence, and escalation.
Participation needs a decision architecture¶
OpenAI is right to ask who should influence model behavior. Affected people, domain experts, developers, operators, leaders, and the public may each hold relevant knowledge. But participation does not eliminate the need for decision rights. An organization still must explain who listens, who decides, which constraints are non-negotiable, and how disagreement is recorded.
Research on value-sensitive design treats values as part of the design process rather than an after-the-fact check. The practical implication is to examine stakeholders and value tensions while requirements are still being shaped. For an AI assistant, that includes questions such as:
- Which behavior should remain consistent for every user?
- Which behavior may vary by role, task, or organizational unit?
- Which configurations require independent review?
- How will users recognize the active rules and limitations?
- How will feedback lead to a documented change rather than an invisible patch?
Govern the configuration, not only the model¶
Most organizations will not train the underlying model. They will still shape behavior through instructions, retrieval sources, tools, permissions, user-interface choices, and human review. That surrounding configuration is where much of their responsibility lives.
A sensible first control is a behavior specification for each consequential use: intended role, prohibited actions, expected refusals, uncertainty language, escalation route, and representative tests. The specification should be understandable to domain owners, not only engineers.
Model behavior will never be perfectly uniform or value-free. The credible goal is not to hide those choices inside a technical system. It is to make them governable: visible enough to debate, specific enough to test, and owned by people with authority to change them.
Sources and research trail¶
- OpenAI, “How Should AI Systems Behave, and Who Should Decide?” (February 16, 2023).
- Friedman, Kahn, and Borning, “Value Sensitive Design and Information Systems” (2002).
- National Institute of Standards and Technology, NIST Privacy Framework 1.0 (2020).
- Shneiderman, “Human-Centered Artificial Intelligence” (2020).