Inioluwa Deborah Raji and colleagues, 2020

“We introduce a framework for algorithmic auditing that supports artificial intelligence system development end-to-end.”

Inioluwa Deborah Raji and colleagues proposed an internal algorithmic-auditing framework in 2020 after examining accountability practices and organizational barriers. Their approach treats auditing as a lifecycle rather than a final inspection. It includes scoping, mapping, artifact collection, testing, reflection, and post-audit decisions. The emphasis is on creating records and responsibilities that can survive beyond one conscientious employee. What sounds simple in isolation was supported by experiments, design choices, and limits that are essential to understanding the claim responsibly.

End-to-end auditing recognizes that harms can enter long before a model is trained. Product goals, data collection, labeling, procurement, interface design, deployment incentives, and monitoring all shape outcomes. A late technical test may identify symptoms without revealing who had the authority to prevent them. The framework therefore connects evidence to governance: which decision was made, by whom, on what basis, and with what route for escalation. That distinction matters whenever a memorable result travels beyond the laboratory. Capability describes what a system can do under stated conditions; wisdom concerns when, why, and for whom it should be used.

Organizations can apply the approach through model cards, data documentation, risk registers, launch reviews, incident logs, and periodic re-evaluation. The artifacts should support real decisions rather than becoming paperwork generated for compliance theater. Independent perspectives, including legal, security, accessibility, domain, and affected-community expertise, can reveal risks that a model team does not see from inside its workflow. Applied responsibly, the idea supports staged testing, comparison with simpler approaches, and direct feedback from people affected by the system. Those practices make impressive demonstrations easier to evaluate.

An internal audit can fail if leadership lacks incentives to act or if findings remain confidential when people face harm. Audit teams need access, independence, resources, and escalation paths. Claims should be tied to evidence, and unresolved risks should have named owners and deadlines. External oversight may still be necessary, especially where organizations judge systems from which they profit. Trust should follow evidence rather than surface fluency or impressive scale. When stakes rise, testing, transparency, and the ability to stop the system must rise with them. Raji’s framework makes accountability operational. Responsible AI is not a single fairness score; it is a chain of documented decisions with people empowered to change the outcome. Seen this way, the quotation belongs to an ongoing inquiry. Its insight becomes stronger when later evidence, affected communities, and competing approaches are allowed to test its boundaries. The work continues with care.