“Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.”
Cynthia Rudin used this sentence as the title of a 2019 Nature Machine Intelligence commentary. Her argument targets decisions where errors can alter health, liberty, credit, or safety. In such settings, organizations often deploy a complex model and then attach a separate explanation method. Rudin asks why the decision model itself cannot be built to be understandable when interpretable alternatives can achieve comparable performance. What sounds simple in isolation was supported by experiments, design choices, and limits that are essential to understanding the claim responsibly.
A post-hoc explanation is an approximation of another system’s behavior. It may highlight plausible features without faithfully representing the actual decision process. An interpretable model, by contrast, exposes its reasoning structure directly through a rule list, scoring system, sparse model, or other form people can inspect. Interpretability is not merely aesthetic; it can support error detection, scientific insight, contestability, and informed professional judgment. That distinction matters whenever a memorable result travels beyond the laboratory. Capability describes what a system can do under stated conditions; wisdom concerns when, why, and for whom it should be used.
Hospitals, courts, banks, insurers, and public agencies increasingly face demands to explain automated decisions. Rudin’s challenge suggests a procurement question: require evidence that opacity is necessary before accepting it. Teams can benchmark interpretable and black-box approaches on the same data, include domain experts, and measure more than predictive accuracy. Usability, stability, subgroup performance, and the ability to identify data problems also matter. Applied responsibly, the idea supports staged testing, comparison with simpler approaches, and direct feedback from people affected by the system. Those practices make impressive demonstrations easier to evaluate.
An interpretable model can still be unjust if its target, data, or policy is unjust. Transparency does not automatically make a decision legitimate, nor does complexity always make it wrong. The relevant standard is whether affected people and responsible professionals can understand and challenge the reasoning at the level required by the stakes. Institutions must pair model transparency with due process, oversight, and authority to correct harmful outcomes. Trust should follow evidence rather than surface fluency or impressive scale. When stakes rise, testing, transparency, and the ability to stop the system must rise with them. Rudin’s imperative shifts the burden of proof. In high-stakes settings, opacity should be justified, not assumed, and explanation should be designed into the decision process from the beginning. Seen this way, the quotation belongs to an ongoing inquiry. Its insight becomes stronger when later evidence, affected communities, and competing approaches are allowed to test its boundaries. The work continues with care.
A post-hoc explanation approximates an opaque model, while an interpretable model exposes a decision structure directly.
The distinction matters because a plausible explanation may not faithfully describe why the black box produced its result.
Organizations can compare transparent and opaque models on accuracy, stability, subgroup outcomes, and operational usefulness.
High stakes justify asking for evidence that extra complexity provides a real benefit that simpler approaches cannot match.
A clear model may still optimize an unfair target, rely on biased data, or implement a harmful policy.
Interpretability should be combined with due process, participation, monitoring, and authority to correct the underlying decision.
Explore more "Quotes of The Day"
Continue exploring how writers, thinkers, artists, and advocates have described the choices that shape an everyday life.