Skip to content
mentua.org

A Good Decision Can End Badly: How to Judge Choices When Luck Is Involved

Line drawing of a filled point from which five curved paths branch upward, four dotted and ending in small points, one solid and ending in an open circleLine drawing of a filled point from which five curved paths branch upward, four dotted and ending in small points, one solid and ending in an open circle

6 min read≈ 1,220 words

What this is about

When a choice turns out badly, most people decide that it was a bad choice, even when the risk was small and known in advance. You learn what a famous 1988 study found about this habit, why luck makes results a poor guide to single decisions, and what to judge instead. You also learn when the result should count after all.

In 1988 the researchers Jonathan Baron and John Hershey of the University of Pennsylvania asked students to judge a short medical case. A man with heart trouble could have a bypass operation that would relieve his pain and raise his life expectancy from 65 to 70, but 8 percent of patients die from the operation itself. When the students read that the patient died, they rated the decision to operate as worse than when they read that he lived, although the facts, the risk and the choice were exactly the same. If the quality of a decision can change after the fact, through an event that nobody controlled, how should you judge your own choices?

Why do we judge decisions by their results?

Psychologists call this habit outcome bias, which means judging a decision by how it turned out instead of by what was known when it was made. Baron and Hershey found it in five studies, and it appeared even among students who said that the result should not matter. A replication with 692 online participants, published in 2023, found the effect again. The poker player Annie Duke opens her 2018 book Thinking in Bets with the same problem, and she uses the word poker players have for it, “resulting”.

The reason is that the result is visible and certain, while the decision is a set of facts, guesses and chances that has to be rebuilt from memory. Once you know that the patient died, the 8 percent stops feeling like a small risk and starts to look like a warning that the doctor ignored. Daniel Kahneman describes this together with hindsight bias, the feeling afterward that the result was predictable all along, in “Thinking, Fast and Slow” by Daniel Kahneman: Summary and Review.

Where does luck hide inside a result?

Every result has two sources, the quality of the choice and chance. If you combine a good or bad decision with a good or bad result, you get four cases. Two of them are fair, because a careful choice ends well or a careless one ends badly. The third case is the bypass patient who died after a sound decision. The fourth case does the most harm, for example when a person drives home after drinking and arrives safely, because the habit is rewarded and nothing is learned.

Our attention makes this worse, because we mostly notice extreme results, and extreme results usually contain a large share of luck. The investor with last year’s best return was probably skilled and lucky at once, so whoever copies her copies luck that will not repeat. What Is Regression to the Mean? explains why an extreme result is usually followed by a more ordinary one.

There is also a problem of timing, because a result is not fixed. The same event can look like a disaster after one month and like a lucky turn after two years, as What Is the Parable of the Chinese Farmer? shows. Whoever judges by results must therefore also choose when the result counts, and that choice is often arbitrary.

What if the best strategy usually fails?

Some of the best rules we know fail most of the time. In the secretary problem, you see options one at a time and cannot go back, and under its strict conditions the best possible rule finds the very best option only about 37 percent of the time. Someone who judged the rule by its results would drop it after two or three misses and switch to a worse rule that happened to work once. What Is the Secretary Problem? works through the rule with ten flats.

The bypass case works the same way. If a hundred doctors make the same sound decision, about eight of their patients will die, and none of those eight decisions was wrong. A good decision is a good bet, not a promise, so it can only be judged against the other choices that were open at the time.

How should you judge a decision instead?

I think three questions do most of the work.

  1. What did I know when I decided? This includes the facts, the risks and the chances as they looked at that moment, not as they look now.
  2. What could I reasonably have found out? A decision is not good if a cheap phone call would have changed it, and it is not bad because a fact was hidden or would have taken months to find.
  3. How did I decide? Did I compare real options, think about how likely each result was, and ask what I could lose?

These questions have one weakness: after the result, memory is not honest. If things went badly, you are likely to remember being more doubtful than you were, and if they went well, more confident.

The remedy is a short decision record, written before the result is known. It needs only a few lines: the date, the choice, the options you rejected, what you expect to happen and how sure you are, and what would show that you were wrong. Later you compare the record with what happened, which is the first question of an after-action review, “What was supposed to happen?” What Is an After-Action Review? shows how to run the rest of that comparison.

When should the result count after all?

The rule “judge the process, not the result” has two limits.

The first limit is that the rule can become an excuse. A person who calls every failure bad luck never learns anything. The investment strategist Michael Mauboussin argues in The Success Equation (2012) that one result says little, and that the more luck an activity involves, the more results you need before they show skill. If a surgeon loses 30 of 100 patients in an operation with a known risk of 8 percent, that is very unlikely to be bad luck. The results show that the process, or the 8 percent itself, is wrong. So treat one bad result as a reason to check the process, and a series of bad results as evidence against it.

The second limit concerns results that cannot be undone, such as a death, financial ruin or a broken trust. In those cases even a small chance of the bad result deserves more weight than an average suggests, because you cannot repeat the bet and let luck even out. A good process for such decisions asks what could go wrong for good and avoids it, which is the method described in What Is Inversion? Thinking Backward to Avoid Mistakes. After an irreversible loss, it is fair to ask whether the person who decided gave that risk enough weight, even if the chance was small. Real decisions of this kind always carry uncertainty, while moral thought experiments usually remove it, which is one reason What Is the Trolley Problem? warns against reading their answers as a guide to real choices.

What to take away

Judge a single decision by what was known and how it was made, and judge a decision maker, yourself included, by many results over time. Write your record before you know how things end, because afterward you will not remember honestly what you knew. Then a bad result can teach you something true about how you decide.

Think about it

There is no right answer here. Take a minute before you move on.

  1. 1.Think of a decision of yours that turned out badly. If the result had been good, would you still call it a mistake, and what exactly would you change about how you decided?
  2. 2.Which of your habits has been rewarded so far mainly because nothing has gone wrong yet?

The takeaway

Outcome bias means judging a decision by how it turned out instead of by what was known when it was made, and a 1988 study by Baron and Hershey showed it even in people who said results should not matter. Because results mix skill with luck, a single result says little about a single decision. Judge a decision by what you knew, what you could have found out and how you weighed the chances, and write this down before the result is known. Let results count when there are many of them, and give risks that cannot be undone extra weight.

outcome-biasdecision-makingjudging-decisionsdecision-qualityluck-and-skillresultingannie-dukethinking-in-betshindsight-biasregression-to-the-meandecision-journalriskuncertaintyirreversible-decisionscritical-thinkingessay

What Is Regression to the Mean?

Why an extreme result is usually followed by a more ordinary one, how Galton found it in family heights, and how it makes us credit the wrong cause.

5 min read≈ 1,160 words3 questions

What Is the Secretary Problem?

How to pick the best option when you see them one at a time and cannot go back. The 37 percent rule, worked through with ten flats, and its limits.

7 min read≈ 1,630 words3 questions