Bayes' Theorem, Explained

A woman goes for a routine breast cancer screening, and the result of her mammogram is positive. The test finds 90 percent of all breast cancers. She asks her doctor: “Does that mean I have cancer? How certain is it?” Most people would guess that the chance is high, perhaps 90 percent. In a study with 160 gynecologists, many of the doctors guessed the same. Their answer was about ten times too high.

How likely is something after you see new evidence?

Bayes’ theorem is the rule that answers this question. It tells you how to change a belief when new evidence arrives. The rule combines two things. The first is how common the thing was before you saw the evidence. The second is how much more often the evidence appears when the thing is true than when it is false.

The number in the first part is called the base rate. It is the share of cases in a group before you know anything else about one person. In our example, the base rate is the share of women in the screening group who have breast cancer. Many people forget this number completely, and that is where the doctors went wrong.

Here are the numbers the gynecologists were given in 2007, at a training course run by the psychologist Gerd Gigerenzer:

• 1 percent of the women in the screening group have breast cancer.
• If a woman has breast cancer, the test is positive in 90 percent of cases.
• If a woman does not have breast cancer, the test is still positive in 9 percent of cases. This is called a false positive.

What is the chance that a woman with a positive result has cancer? Percentages make this hard to see. Gigerenzer suggested a better way, which he calls natural frequencies. Instead of percentages, you imagine a concrete group of people and count them. We will do this with 1,000 women.

1. Start with the base rate. Of 1,000 women, 1 percent have cancer. That is 10 women. The other 990 do not.
2. Follow the women with cancer. The test finds 90 percent of them. So 9 of the 10 women test positive.
3. Follow the women without cancer. The test is wrongly positive for 9 percent of them. That is about 89 of the 990 women.
4. Count all positive results. 9 + 89 = 98 women have a positive test.
5. Ask the question again. Of these 98 women, how many have cancer? Only 9. That is about 9 percent, or roughly 1 in 10.

So a positive test raises the chance from 1 in 100 to about 1 in 10. That is a big change, but it is far from 90 percent. Why is it so low? Because healthy women are so many more than sick women. A small error rate in a large group produces more false alarms than a high hit rate in a small group produces true ones.

What about the doctors? Before the training, only 21 percent of them chose the right answer. Most of them, 60 percent, said the chance was 90 or 81 percent. After they learned to translate the numbers into natural frequencies, 87 percent answered correctly. The doctors were not bad at medicine. The way the numbers were written made the answer hard to see.

The same calculation can be written as a formula. A conditional probability is the chance of one thing, given that another thing is true. People write it as P(A | B), which you read as “the probability of A, given B”. Bayes’ theorem says:

P(cancer | positive) = P(positive | cancer) × P(cancer) / P(positive)

With our numbers, that is 0.9 × 0.01 / 0.098, which is about 0.09. The top of the fraction is the 9 true positives. The bottom is all 98 positives. The formula and the counting give the same answer, because they are the same idea.

The most common mistake is to swap the two conditional probabilities. “The test is positive in 90 percent of women with cancer” is P(positive | cancer). The woman wants to know P(cancer | positive). The two can be very different. Bayes’ theorem is the rule that gets you from the first number to the second.

Bayes’ theorem also works more than once. Suppose the woman takes a second, independent test with the same accuracy, and it is positive again. Her starting chance is now about 9 percent, not 1 percent. Counting again with the 98 women, about 8 of the 9 sick women and about 8 of the 89 healthy women test positive a second time. Her chance rises to about one in two. Each result turns the old estimate into the starting point for the next one.

What Bayes’ theorem teaches you

Bayes’ theorem teaches you to judge new evidence against what you knew before. When you see evidence, ask two questions: “How common was this before?” and “How often would I see this evidence if it were not true?” Most bad judgments about evidence leave out one of these.

• Health. A positive result from a screening test is a reason for a second test, not a reason to panic. Ask your doctor how many people with a positive result actually have the disease. The answer depends on how common the disease is in people like you.
• The news and the courtroom. “The evidence fits the crime” is not the same as “the person is guilty”. If many innocent people would also show the same evidence, it proves little. Lawyers call the swap of the two probabilities the prosecutor’s fallacy.
• Surprising claims. A single study with an amazing result should move your belief only a little if the claim was unlikely to begin with. This is one reason why many surprising results in science do not hold up when other teams repeat the study.
• People. A colleague is late once. Before you decide that they are unreliable, think about how often they were on time before. One event changes the picture a little, not completely.

In psychology, the habit of ignoring the base rate is called base rate neglect. Daniel Kahneman describes it as one of the typical errors of fast, intuitive thinking in “Thinking, Fast and Slow” by Daniel Kahneman: Summary and Review. Gigerenzer’s work adds a hopeful point. The error is much smaller when people see counts instead of percentages, so the way numbers are shown matters.

Professionals use the theorem in many places. Doctors use it to read test results. Spam filters use it to estimate how likely an email is to be spam, given the words in it. Many machine learning systems update their estimates in the same way. Some scientists build a whole method of statistics on it, called Bayesian statistics.

Where does Bayes’ theorem need care?

The mathematics of the theorem is not in doubt. It follows in a few lines from the basic rules of probability. The difficulties come from the numbers you put into it.

The first difficulty is the starting estimate. In the mammogram example, the base rate is a measured share of a real group. For many other questions, such as whether a new theory is true, nobody can measure it. Then two people can start from different estimates and reach different answers from the same evidence. This is the strongest objection to Bayesian methods, and it has divided statisticians for a long time. The usual reply is that the starting estimate matters less and less as evidence builds up. With enough good evidence, people who started far apart end up close together.

The second difficulty is choosing the right group. A base rate always belongs to a group of people, and you may not be typical of that group. A woman with a lump in her breast is not part of a routine screening group. Her starting chance is higher, so the same positive test means much more for her. Using the wrong base rate gives a wrong answer, however careful the calculation is.

The third difficulty is hidden assumptions. The second-test example assumes that the two tests make their errors independently. If both tests are fooled by the same thing, for example dense breast tissue, a second positive result adds less than the calculation suggests.

A common misconception runs the other way. Some people learn about false positives and decide that tests are useless. That is also wrong. In the example, the test raised the chance from 1 in 100 to about 1 in 10. That is good evidence. It only becomes misleading when people forget where they started.

Learn more

Bayes’ theorem is named after Thomas Bayes, an English minister in Tunbridge Wells, a spa town south of London, who died in 1761. He never published this result himself. His friend Richard Price found an essay on the question among his papers, edited it and presented it to the Royal Society in London in December 1763. It was printed in 1764. The historian of statistics Stephen Stigler has argued that Price saw it as a weapon against the philosopher David Hume. Hume had argued that reports of miracles can never be strong enough evidence to believe them.

In 1774 the French mathematician Pierre-Simon Laplace published the same idea, apparently without knowing Bayes’s work. Laplace stated it in its general form and used it far more widely. For this reason, some historians say he deserves much of the credit.

Gigerenzer’s book Calculated Risks: How to Know When Numbers Deceive You (Affiliate Link) is the best next step if you want to read health and risk numbers with confidence. It explains natural frequencies with many cases from medicine and the law. In the United Kingdom it was published as Reckoning with Risk. For a different kind of probability problem, where you must decide when to stop searching, read What Is the Secretary Problem?.