What Should You Never Hand to a Machine?
The useful way to think about artificial intelligence is not as intelligence at all. Three economists, Ajay Agrawal, Joshua Gans, and Avi Goldfarb, argue in “Prediction Machines” that what this technology has actually made cheap is prediction, and that when the price of something falls, the things that go with it become more valuable. Prediction's partner is judgment. So the cheaper prediction becomes, the more valuable judgment becomes, and the more expensive it is to give judgment away by accident. The AI Alignment Filter™ ends on exactly that gate: is this tool replacing judgment, or enhancing it? Four kinds of decisions should never cross the line. Decisions that cannot be undone. Decisions where you carry the liability but cannot explain the reasoning. Decisions where being seen by a person is the point. And decisions about situations the machine has never encountered.
Introduction
There is a question people ask about artificial intelligence that cannot be answered, and a question underneath it that can.
The unanswerable one is "what can AI do?" The answer changes every few months; it varies by vendor, and anyone who tells you confidently is either selling something or has not been paying attention.
The answerable one is: what in my work is judgment, and am I protecting it or giving it away without noticing?
That question has a stable answer. It was true before this technology arrived, and it will be true after the current generation of it is obsolete. And unlike the first question, you are the world's leading expert on it, because it is about your work.
This week the Filter runs at full strength. Not "should I be using AI," which is not a decision. Does this particular tool belong in this particular workflow, at this scale, in this Business or practice?
And the gate that decides it is the fourth step.
Key Takeaways
• What this technology made cheap is prediction, not intelligence. Those are different things, and the difference is the whole argument.
• When prediction gets cheaper, judgment gets more valuable, because judgment is what turns a prediction into a decision.
• A decision is a prediction plus a judgment plus an action. Machines have taken one of the three.
• Four categories should never cross the line: irreversible decisions, decisions you are liable for but cannot explain, decisions where being seen by a person is the value, and decisions about situations with no precedent.
• Confident wrong answers are the characteristic failure of these systems, not obviously wrong ones.
• Step four of the Filter is a hard gate. A tool that replaces judgment fails it regardless of everything before.
What Gets Cheap, and What Gets Valuable
In Prediction Machines, the economists Ajay Agrawal, Joshua Gans, and Avi Goldfarb make an argument that sounds deflationary and turns out to be the most useful frame available.
Their claim is that the current wave of artificial intelligence has not delivered intelligence. It has delivered one component of it, and that component is prediction. Filling in missing information. Given what we know, what is the most likely value of the thing we do not know?
Once you see it that way, the economics become ordinary, and ordinary is good, because we know how to reason about ordinary economics.
When the price of something falls, two things happen. You use more of it, in places you previously would not have bothered. And the things that complement it become more valuable, because there is now more demand for whatever finishes the job.
Cheap prediction is used everywhere. And the complement to prediction, the thing that finishes the job, is judgment.
This is why the anxious framing gets the conclusion backward. Cheap prediction does not make judgment less relevant. It makes judgment the scarce input, and therefore the expensive one.
Which means that giving judgment away, casually, because a tool arrived that seemed to include it, is a considerably worse trade than it looks on the day.
The Knowledge
Agrawal, Gans and Goldfarb use an example I have not stopped thinking about since I read it.
For decades, a London taxi driver had to pass an examination on the city's streets so demanding that it took years to prepare for. Drivers could predict the best route given the hour, the weather, and the traffic, and that prediction was the skill. It was the whole basis of the job's value.
Then satellite navigation made that prediction close to free, and the value of the skill collapsed. Not slowly. Not partially.
Here is the part worth sitting with. The thing that made a good cabbie good was never only the routing. It was reading a passenger who was late and anxious. Knowing when a rule should be broken. Deciding that a particular street, tonight, was a bad idea for reasons no dataset contains.
None of that transferred to the machine. But because the prediction was the visible, testable, examinable part, it was the part the job had been organized around, and when it went, the rest went with it.
That is the real risk, and it is not that a machine takes your judgment. It is that your work gets reorganized around the part a machine can do, and the judgment quietly stops being the thing you are paid for or the thing you practice.
Prediction Is Not Decision
Here is the distinction to carry into every evaluation you run.
A decision has three parts. A prediction about what will happen. A judgment about what matters, given what is at stake, who bears it. And an action.
Machines have taken the first part and have made it very cheap. They have not taken the second, and the authors are explicit that so far these systems make only routine decisions, which is a narrower claim than most vendors imply.
The people who turn predictions into good decisions become more valuable, not less.
So when a tool is presented to you, separate the parts before you evaluate it. Which of the three is it actually doing?
A tool that produces a better prediction and hands it to a person who exercises judgment is enhancing judgment. That is the good case, and it is most of the useful products in this market.
A tool that produces a prediction and then acts on it, with judgment either absent or nominal, has taken over a decision. Sometimes that is appropriate, for small and reversible things. Often it is appropriate, and nobody checked whether it was.
The Four That Never Cross the Line
Step four of the Filter asks whether a tool replaces judgment or enhances it. In practice, four categories of decisions answer that question before you get to the tool.
Decisions that cannot be undone
The test is the cost of being wrong, not the probability of being wrong. A system that is right ninety-nine times out of a hundred is excellent, and if the hundredth outcome is permanent, excellent is not the relevant measure. Reversibility is what buys you the right to be wrong occasionally.
Decisions where you carry the liability but cannot explain the reasoning
For the most capable systems, how the machine reached its answer is genuinely opaque, including to the people who built it. Eric Topol names this directly in Deep Medicine and notes that it has not been solved. If you are accountable for an outcome and cannot reconstruct why the recommendation was made, you have accepted liability for a process you do not have access to.
Decisions where being seen by a person is the value
A complaint, a difficult performance conversation, a conflict between two people who have to keep working together. In these, the content of the decision is not the point. Being heard by someone with standing to act is the point, and a well-worded automated response is not a cheaper version of that. It is a different thing wearing its clothes.
Decisions about situations with no precedent
These systems learn from history. Asked about something genuinely novel, they do not report uncertainty. They pattern-match to the nearest familiar thing and answer with the same fluency as always, which is the most dangerous property they have.
The Confident Wrong Answer
That last point deserves its own paragraph, because it is the failure mode people are least prepared for.
Agrawal, Gans and Goldfarb warn that these systems can behave as though they know something they do not. The wrong answers do not arrive flagged. They arrive in the same tone, the same format, and the same confident register as the right ones.
Every other tool in your practice fails legibly. A handpiece that stops working has stopped working. A spreadsheet with a broken formula shows an error. You find out.
This class of tool fails silently and plausibly, which means the only detection mechanism is a person who knows enough to notice, reading the output with genuine attention rather than scanning it for approval.
That is what we mean by keeping judgment in the loop. Not a checkbox that says a human reviewed it. A person who actually would have caught it.
Discernment lives in the edit.
Running Step Four Honestly
There is a self-deception available at step four, and it is worth naming, because it is the most common way a good framework produces a bad decision.
A tool is described as enhancing judgment because a human technically remains in the workflow. The human clicks approve. The human could, in principle, override it.
Ask instead: in the last month, how many times did a person actually change the output? If the answer is never, the human is not exercising judgment. The human is providing legal cover for a process that runs without them, and everyone involved has quietly agreed not to notice.
That is not an argument against the tool. It might be an excellent tool doing a task that genuinely does not need judgment, in which case stop pretending there is a review step and redesign the workflow honestly.
What it cannot be is both. Either the judgment is real, and someone is exercising it, or it is not, and the workflow should say so.
Frequently Asked Questions
Is this an argument against adopting AI?
No. The authors of Prediction Machines are enthusiasts, and so am I for the right applications. The argument is about where the line sits, and drawing it precisely is what lets you adopt confidently everywhere else.
How do I tell prediction from judgment in my own work?
Ask what would change if you had perfect information. The parts that would become trivial were prediction. The parts that would still be hard, because they involve weighing things that matter against each other, are judgment.
What about tools that claim to include judgment?
Ask what the tool does when it is uncertain. Systems with real judgment about their own limits will decline, escalate, or flag. Systems without it answer everything at the same confidence, and that is observable in about ten minutes of testing.
Our team already uses AI informally. Is that a problem?
It is the most common situation and the risk is unevenness rather than the use itself. One person drafts with it, another summarizes with it, nobody has discussed it, and the standard varies invisibly. A four-line written policy fixes most of it: where it is encouraged, where it is limited, what must always be reviewed, and who holds final responsibility.
Does judgment actually get weaker if it is not used?
That is the concern clinicians themselves raise most often about this technology, and it is a reasonable one to take seriously without overstating the evidence. The practical answer does not depend on settling it. Keep people doing the reasoning often enough that they would notice if the machine were wrong, because that is the only thing standing between a confident wrong answer and a bad outcome.
Final Thoughts
The question this week is deliberately narrow. Not what can AI do. What should never leave your hands.
The economists give the reason it matters. Prediction got cheap, so judgment got expensive, and anything expensive that you give away without noticing is a bad trade, whatever the invoice says.
The Filter gives the mechanism. Four steps, in order, and the fourth is a gate rather than a consideration.
And the fifty years give me the last part, which is that every technology I have watched arrive in this profession was adopted fastest by the people who understood their own work least well. Not because they were foolish. Because if you cannot name what the judgment in your job is, you cannot tell when you have handed it over.
A co-pilot, not an autopilot. That has been the position here since the beginning, and this is the year it gets tested properly.
Sources & Further Reading
Agrawal, Ajay, Joshua Gans and Avi Goldfarb. Prediction Machines: The Simple Economics of Artificial Intelligence. Harvard Business Review Press, 2018. The prediction-and-judgment argument, the London taxi example, and the caution about confident wrong answers are theirs.
Topol, Eric. Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again. Basic Books, 2019. On the unresolved opacity of the most capable systems, and what that means where a clinician carries liability.