“Algorithms must not create a new black box.”
European Commission President Ursula von der Leyen offered this warning during her 2020 State of the Union address. In the same passage, she praised artificial intelligence for possible gains in precision farming, health care, early diagnosis, and autonomous driving. Her concern was not that complex computation should be forbidden. It was that technological sophistication could become an excuse for decisions no one could explain, contest, or repair. The phrase “new black box” captures a political problem as much as a technical one: institutions cannot legitimately exercise power if their reasons disappear behind a system’s output.
Transparency in AI has several layers. A model may be technically interpretable, yet the organization using it may hide its purpose, training sources, performance limits, or error rates. Conversely, a complex model may resist a simple account of every internal step while still being governed through documentation, controlled tests, audit trails, and clear explanations of how its recommendation is used. The meaningful question is what people need to know to evaluate the decision. A loan applicant, clinician, regulator, and model engineer each require different forms of evidence and access.
Black-box concerns grow with the stakes. A music recommendation can be imperfect without threatening a person’s basic rights. A system used in hiring, education, policing, border control, or medical care can change life chances. In those settings, opacity can conceal bad data, proxy discrimination, unsafe automation, or an objective that was never publicly justified. Explanation alone is not a cure: a polished account may rationalize a harmful process. Governance also needs performance requirements, human review with real authority, monitoring after deployment, and remedies that correct both individual outcomes and systemic patterns.
Von der Leyen’s line anticipated the European effort to regulate AI according to risk. Its lasting value is broader than any single law. It asks designers to treat visibility as part of system quality, not as a public-relations feature added later. It asks leaders to know where a model operates, what evidence supports it, and who can stop its use. Most importantly, it centers the person receiving the decision. Trust does not come from being told that an algorithm is advanced. It comes from rules and procedures that remain intelligible when something goes wrong. The practical goal is not a perfect explanation of every parameter. It is enough visibility to test a claim, challenge an outcome, trace responsibility, and choose a remedy. A system becomes governable when the people affected can see more than its final answer.
Von der Leyen praised practical AI applications before warning that algorithms must not become a new black box.
The contrast made accountability a condition of innovation rather than its enemy.
Useful transparency includes purpose, data governance, tests, limits, oversight, and the role of the output in a decision.
Different audiences need different explanations, but none should be left with an unexplained exercise of power.
High-impact decisions can magnify hidden bias or error when affected people cannot challenge the process.
Auditability, human authority, monitoring, and remedies turn visibility into actual accountability.
Explore more "Quotes of The Day"
Continue exploring how writers, thinkers, artists, and advocates have described the choices that shape an everyday life.