Knowledge Strategy¶

October 15, 2023
in Human-Centered AI, Decision Support, Knowledge Strategy
3 min read

Expert judgment belongs in the system design

A new study from International Business Machines (IBM) researchers examines how machine-learning predictions might be adjusted when a domain expert's judgment conflicts with the model, particularly when a case is poorly represented in the training data. The work addresses a practical reality: experts and models often disagree for reasons that neither an accuracy score nor an appeal to experience can settle alone.

The disagreement should be treated as information.

August 22, 2023
in Knowledge Strategy, AI Engineering, AI Governance
3 min read

Retrieval is a knowledge-governance problem

International Business Machines (IBM) Research has published a clear explanation of retrieval-augmented generation, an approach that gives a large language model access to external sources at the time of a request. Instead of relying only on patterns encoded during training, the system retrieves relevant material and uses it to generate a more current, domain-specific, and potentially verifiable answer.

Retrieval-augmented generation (RAG) is a promising architecture. It is not a substitute for governing the knowledge being retrieved.

July 11, 2023
in Human-Centered AI, Knowledge Strategy, Product Management
3 min read

Long context changes the knowledge-work interface

Anthropic has released Claude 2 with a context window that can accept roughly 100,000 tokens—enough for hundreds of pages of material in one prompt. The immediate attraction is obvious: a user can bring a long report, technical documentation, or even a book into a conversation without dividing it into tiny fragments.

More context changes what a language model can see. It does not guarantee that the model will attend to the right thing.

March 30, 2023
in AI Governance, Knowledge Strategy, Organizational Learning
3 min read

A framework needs a community of practice

The National Institute of Standards and Technology (NIST) has launched its Trustworthy and Responsible Artificial Intelligence Resource Center, known as the Artificial Intelligence Resource Center (AIRC). The new site gathers the Artificial Intelligence Risk Management Framework (AI RMF), its playbook, crosswalks, and implementation resources in one place.

Central access is useful. The larger opportunity is to create a place where organizations learn how the framework behaves in practice.

March 6, 2023
in Human-Centered AI, Knowledge Strategy, Operating Models
3 min read

Copilots will rewire the handoff

Microsoft has introduced generative artificial intelligence capabilities across Dynamics 365, bringing “copilot” functions into sales, customer service, marketing, and supply-chain work. The examples emphasize drafting emails, summarizing interactions, creating content, and surfacing information inside the applications where people already work.

The most important design question is not how much text a copilot can generate. It is what happens to the handoff.

February 23, 2023
in Cybersecurity, Digital Engineering, Knowledge Strategy
3 min read

A digital twin can preserve operational judgment

The National Institute of Standards and Technology (NIST) is exploring how digital twins could help manufacturers detect cyberattacks. By comparing a physical process with its virtual representation, a team may recognize changes that ordinary information-technology monitoring misses: a machine behaving differently, a process drifting, or a control command producing an unexpected physical result.

The cybersecurity potential is important. So is the knowledge-management lesson. A useful digital twin does not merely mirror equipment. It preserves an organization's understanding of what normal operation means.

February 7, 2023
in Human-Centered AI, Knowledge Strategy, Decision Support
3 min read

Search has become a knowledge-verification problem

Microsoft has introduced a new version of Bing that combines search with a conversational artificial intelligence system. Instead of returning only a ranked list of links, it can synthesize an answer, respond to follow-up questions, and show sources alongside the conversation.

This interface is convenient because it compresses the distance between a question and a usable explanation. It is risky for exactly the same reason.

January 11, 2023
in AI Governance, Operating Models, Knowledge Strategy
3 min read

Responsible AI needs an operating system

Google has opened the year by publishing its most detailed account yet of how it puts artificial intelligence principles into practice. The report covers governance reviews, technical tools, education, and work across product teams. Its value is not that Google has found a universal formula. It is that the report makes a less glamorous truth visible: responsible artificial intelligence (AI) is an operating problem.

Principles matter. They tell an organization what it is trying to protect. But principles do not decide whether a particular use should proceed, determine which test is sufficient, or preserve the evidence behind a difficult exception.