Reading focus
Syllabus: Why Are Suicidal Thoughts and Actions Difficult to Explain and Predict?
Keep the denominator of each performance measure visible.
Why you’re reading it. Learn why usefulness requires a specified comparison and evidence about consequences.
A question to consider. In an invented dataset with 100 events among 10,000 observations, a model identifies 80 events but also flags 990 non-events. What fraction of its positive predictions is correct, and what does that number leave unanswered?
Why this reading
A model can rank people reasonably well yet estimate their probabilities poorly. Whiting and Fazel challenge judgments based on a single performance measure and ask what comparison would establish practical value.
Before you read
Discrimination concerns ranking; calibration concerns agreement between predicted probabilities and observed frequencies. Positive predictive value is the proportion of positive predictions followed by the outcome. It depends partly on the outcome’s base rate.
Read for
- Separate statistical performance from evidence that using a model improves outcomes.
- Examine which comparisons are missing and how the consequences of errors enter the authors’ argument.
Consider afterwards
- Why could a high negative predictive value arise even from an uninformative rule when an outcome is rare?
- What would a fair evaluation of a model added to existing practice compare?
Editorial context
This is a methodological critique, not proof of clinical benefit. Fazel discloses involvement in developing a prediction tool. Its policy references describe 2019 practice. The later systematic appraisal extends the evaluation questions while retaining substantial uncertainty.