Cathy O’Neil, 2017

Cathy O’Neil in a realistic editorial setting — AI Quote of the Day

“Algorithms are opinions embedded in code.”

Cathy O’Neil’s phrase “Algorithms are opinions embedded in code” cuts through a common myth about automated decision making. An algorithm may use mathematics, but mathematics does not choose the question, define success, select the data, or decide which errors are acceptable. People do. A credit model reflects an opinion about what predicts repayment. A hiring system reflects an opinion about what makes a promising employee. A recommendation engine reflects an opinion—often an economic one—about which behavior should be encouraged. Coding can make those judgments consistent and scalable without making them neutral.

The quote does not mean every algorithm is arbitrary. Some opinions are grounded in strong evidence, constrained by professional standards, and tested against alternatives. The important point is that the judgment should be visible enough to examine. What outcome is the model optimizing? Which groups are represented in the data? Are labels based on a fair process or on historical practices that already excluded people? Does the system treat a convenient proxy as if it were the thing that truly matters? Technical documentation and independent audits turn those questions into part of engineering rather than an afterthought.

Scale increases the stakes. A biased human decision may harm one person; a biased model can repeat the same pattern across thousands of applicants before anyone recognizes it. Automation can also create a false sense of inevitability. Workers may defer to a score because it looks scientific, even when the model was never tested for the current population. O’Neil’s wording restores agency. If an algorithm contains opinions, then those opinions can be challenged, compared with public values, and changed. People affected by the system have knowledge about its consequences that performance metrics alone may never reveal.

Good algorithmic governance therefore asks for more than accuracy. It requires a clear purpose, proportional use, subgroup testing, security, privacy, monitoring, and a route for appeal. In high-impact settings, an organization should be able to explain why automation is appropriate at all and who remains accountable for the result. O’Neil’s quote is powerful because it makes the invisible legible. Code may execute a rule at machine speed, but the rule still expresses a human view of the world. Responsible AI begins when builders and institutions are willing to name that view and answer for what it does. Once those questions are ordinary, algorithms can be discussed like other consequential public and commercial systems: with evidence, disagreement, revision, and enforceable limits. The aim is not to remove judgment from technology, but to make judgment explicit enough for democratic and professional accountability.

Human judgment appears in the chosen target, training data, labels, features, thresholds, and definition of acceptable error.

Those decisions can be reasonable, but they should be documented rather than hidden behind technical complexity.

Automated systems can repeat one flawed assumption across thousands of people and institutions.

Consistency is valuable only when the underlying rule is appropriate and failures are detected before they spread.

Audits, impact assessments, subgroup testing, and appeal processes make algorithmic judgments open to evidence and challenge.

The organization deploying a model remains responsible; a score cannot absorb moral or legal accountability.