Rare Events Need Base Rates

The concept

To interpret a signal, consider both its accuracy and how common the target event is before the signal arrives. False alarms can outnumber true detections when the target is rare. Natural frequencies make this easier to see: start with a population, separate actual cases from noncases, and then count the signal in each group.

Worked example

A fictional archive scans 10,000 images for a rare cataloging issue present in 1 percent: 100 images. A detector finds 90 of those. It also wrongly flags 1 percent of the 9,900 unaffected images, producing 99 false alarms. Of 189 total flags, only 90 are genuine, about 48 percent. The detector is useful for prioritizing review, but a flag is far from a definitive finding.

Limits and judgment

The numbers are illustrative assumptions, not measured performance claims. Actual prevalence and error rates may vary between collections. Do not transfer a model's advertised accuracy into a new setting without checking definitions. When consequences are significant, a follow-up verification step can be more appropriate than treating every positive signal as an established fact.

RETRIEVE. REASON. REMEMBER.

Knowledge check.

Choose one answer for each question, then check your reasoning.