Survivorship bias
The bullet holes were on the planes that survived. The planes hit elsewhere never returned to be counted.
Try it
Study the damage on returning planes, then see who never came home.
Press the first button: 200 simulated planes each take 4 hits.
Simulated. Every plane takes 4 hits at random across its whole body. A hit to the engine or cockpit is fatal 50% of the time, any other hit never is.
What is going on
When a sample is filtered by success, the evidence you can see is a biased picture of the whole. The missing cases carry the information you actually need.
The same trap hides behind “every great company did X”, “my grandfather smoked and lived to 95” and any list of winners that leaves out the many who did the same and lost.
How to spot it
- A study or list that only includes successes, survivors or current members.
- Words like “always” and “every” applied to winners.
- Ask: who would be in this sample if they had failed, and why are they not here?
Abraham Wald and the returning bombers
During the Second World War the Statistical Research Group at Columbia University was asked how to armour bombers. The obvious answer was to reinforce the places where returning aircraft were most damaged. Abraham Wald’s insight was that the returning planes showed where a plane could be hit and still fly home, so the armour belonged where the survivors were not hit.
Mangel and Samaniego’s 1984 paper describes Wald’s actual, more technical work for the group. The planes in the demo are simulated, not Wald’s data; the aim is to show the mechanism.