Unstatistical

Regression to the mean

The best performers last time tend to do worse next time, and the worst tend to improve. Nobody needs to have done anything.

Try it

Pick the top scorers from round one and watch them fall back in round two.

Simulated. Each person has a fixed skill (average 100, spread 10). Each test score is skill plus fresh random luck. The top 10% from test 1 are highlighted. Nothing is done to them between tests.

What is going on

Any result is part skill and part luck. Pick the people at the extremes and you pick those with the luckiest or unluckiest draws as well. Luck does not repeat, so the second measurement drifts back towards the average.

This is why a punishment seems to work after a bad result and praise seems to backfire after a good one, and why a treatment given at someone’s worst moment looks effective even if it does nothing.

How to spot it

  • A “turnaround” after a record-bad or record-good period.
  • An intervention chosen because of an extreme reading.
  • Ask: what would we expect to happen with no intervention at all, and is there a comparison group?

Galton’s tall fathers, Kahneman’s flight instructors

Francis Galton noticed in 1886 that the children of unusually tall parents were on average shorter than their parents, and those of short parents taller, and called it “regression towards mediocrity”. Daniel Kahneman tells of Israeli flight instructors convinced that shouting at cadets after a poor landing worked, and praise after a good one did not; both are what regression predicts.

The scores in the demo are simulated as skill plus luck.

Sources