If a medical test is 99% accurate and you test positive, what are the chances you actually have the disease?
Say the test correctly flags 99% of sick people and correctly clears 99% of healthy people, and 1 in 1,000 people has the disease. The answer is nowhere near 99%.
The base rate fallacy
Test 100,000 people
100 have the disease; 99,900 don't. Here is everyone who tests positive.
The same 99% test, for diseases of different rarity
Chance that a positive result is real
A 99% accurate test does not mean a positive result is 99% likely to be correct. When only 1 in 1,000 people has the disease, the healthy crowd is so large that even a 1% error rate produces ten false alarms for every true positive. Researchers Gigerenzer and Hoffrage found that even practicing physicians routinely get this calculation wrong.
The same arithmetic applies whenever you screen for something rare: fraud alerts, airport security flags, spam filters. The rarer the thing you're looking for, the more of your alarms will be false, however accurate the test.
Before trusting a test result, how common was the thing it tested for?
Why does 99% accuracy become about 9%?
Imagine 1,000 people. About one actually has the disease, and a 99% sensitive test will usually find that person. But about 999 people are healthy, and a 1% false-positive rate incorrectly flags roughly ten of them. That leaves about one true positive among roughly eleven positive results.
It's the same trap as the Monty Hall problem and the birthday paradox: a probability result that runs against gut instinct until you work through the actual mechanism.
Related exhibits
Sources: Bayes' theorem (classical probability theory). Base-rate neglect as a documented cognitive bias: Kahneman & Tversky (1974); in medical testing, Gigerenzer & Hoffrage (1995) found that even practicing physicians routinely get this calculation wrong. The examples assume equal sensitivity and specificity (99%).