Choosing Which Errors Matter

The concept

A classifier makes both false alarms and missed detections. Precision asks what fraction of positive predictions were correct. Recall asks what fraction of actual positive cases were found. Neither number alone establishes usefulness. Choosing a threshold requires an explicit account of the costs of each error and the capacity available to handle alerts.

Worked example

A fictional archive has 100 fragile documents among 1,000 items. A screening tool flags 80 items, including 60 fragile documents. Precision is 60 divided by 80, or 75 percent; recall is 60 divided by 100, or 60 percent. Forty fragile documents were missed. If lowering the threshold finds 20 more fragile documents but adds 150 false alarms, the conservation team must consider whether it has enough staff to examine the larger queue.

Limits and judgment

Accuracy can conceal this tradeoff when positive cases are rare. Predicting that every document is sturdy gives 90 percent accuracy here but finds none of the fragile items. Metrics should be calculated on representative cases and examined alongside a practical workflow, not used as decorative percentages.

RETRIEVE. REASON. REMEMBER.

Knowledge check.

Choose one answer for each question, then check your reasoning.