Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
The AI product is more than the model¶
Microsoft has spent the spring adding generative artificial intelligence capabilities across Azure, developer tools, and business applications. In this month's platform update, one sentence deserves more attention than another feature list: generative AI does not absolve builders from thinking about what good product-making looks like.
That should be the operating principle for the current wave of experimentation.
Capability is not a user need¶
A powerful model can summarize, classify, draft, translate, and generate code. Those verbs are capabilities, not products. A product begins with a person trying to accomplish something under real constraints.
Artificial intelligence (AI) creates value only when it improves that work without creating a larger burden somewhere else. A faster draft that takes longer to verify is not necessarily an improvement. A chatbot that answers routine questions but escalates difficult cases without context may shift cost rather than reduce it. A coding assistant that increases output while increasing insecure dependencies may optimize the wrong outcome.
Product teams should describe the changed workflow before selecting the model.
The surrounding system determines usefulness¶
The model sits inside a larger system of data, retrieval, prompts, permissions, interface, monitoring, and human responsibility. Much of the product quality comes from those layers.
The machine-learning technical-debt literature has warned for years that the model can be a small part of a production system. Generative models make the lesson easy to forget because their open-ended output is so visible. The demonstration centers the model; the operational product depends on everything around it.
A useful product brief should state:
- the user's task and current pain;
- the decision or action the output supports;
- source data and its quality;
- acceptable and unacceptable failure;
- how the user verifies or corrects output;
- what happens when the model cannot help;
- operational owner and support path; and
- evidence that the whole workflow improved.
Design for the exception¶
Demonstrations tend to show a clean request and a helpful answer. Real work contains ambiguous inputs, missing records, conflicting policies, access restrictions, and users under pressure.
The exception path is often the real product. The system should recognize when it lacks required information, preserve context during escalation, and make recovery easier. A refusal without a next step is not a complete safety feature. A human handoff without provenance is not a complete workflow.
Human-centered design research emphasizes setting expectations, supporting efficient correction, and making clear why a system acted as it did. Amershi and colleagues' human–AI interaction guidelines are especially useful now because they focus on the experience over time, not just the first successful interaction.
Measure the work after the novelty fades¶
Initial adoption is a weak measure. People will try a novel capability because it is interesting. The stronger evidence appears after routine use:
- completion time including verification and rework;
- downstream error and escalation rates;
- quality differences across user groups and cases;
- whether users retain necessary expertise;
- support and operating cost; and
- whether the process remains resilient during model or service changes.
Collect qualitative evidence too. Watch people perform the work. Ask what they stopped checking, which cases they avoid, and where they maintain unofficial notes. Those behaviors reveal gaps that a usage dashboard cannot.
Product management is the integration discipline¶
Generative AI gives teams an unusual amount of capability before they have a clear product. Good product management supplies the missing discipline: choose a meaningful problem, understand the work, define evidence, manage tradeoffs, and own the result after release.
The models will improve and the platform announcements will continue. Organizations should welcome both without confusing momentum in the technology with progress in the mission. The product is the complete system by which a person achieves an outcome. The model is one powerful component—and it should be judged by what happens to the work around it.
Sources and research trail¶
- Microsoft Azure, “What's New in Azure Data & AI: How Customers Realize Tangible ROI” (June 27, 2023).
- Sculley et al., “Hidden Technical Debt in Machine Learning Systems” (2015).
- Amershi et al., “Guidelines for Human-AI Interaction” (2019).
- Shneiderman, “Human-Centered Artificial Intelligence” (2020).
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (2023).