Civil-Rights Risk Lives in the Decision System, Not Only the Model
The Department of Justice’s January 2024 interagency convening on artificial intelligence and civil rights reflected an important federal position: existing civil-rights and consumer-protection laws continue to apply when decisions are mediated by algorithms.
That proposition is necessary. Its technical consequence deserves equal emphasis.
Discrimination rarely resides in a single model parameter or fairness metric. It can enter through the choice of problem, the collection of data, a proxy variable, the design of an interface, unequal access to a digital service, the discretion granted to staff, or the absence of a practical appeal. An AI system can satisfy a narrow statistical test and still participate in an unlawful or inequitable decision process.
Civil-rights analysis must therefore treat the whole decision system as the unit of accountability.