Reading focus
Syllabus: How Do We Know What a Psychological Test Measures?
Apply the distinctions developed throughout the syllabus to both the audited reports and the audit’s own design.
Why you’re reading it. Finish with a practical appraisal of measurement evidence and the limits of conclusions drawn from reporting.
A question to consider. A paper cites an established scale, deletes three items, and reports alpha. Write the three most consequential questions you would ask before accepting its interpretation.
Why this reading
This audit turns construct validation into a question about research practice: what measurement evidence do published studies report? It provides a concrete opportunity to evaluate the distance between a recommended standard and the information readers actually receive.
Before you read
An audit samples and codes published reports. Missing reporting is not automatically evidence that an analysis was never performed. A previously published scale may have been altered or used in a different population.
Read for
- Follow the article-selection and coding rules before interpreting the findings.
- Distinguish scale citations, reliability coefficients, and evidence about structure.
- Ask how restricting parts of the coding to Methods sections affects the conclusions.
Consider afterwards
- Design a short measurement-evidence audit of one paper. How would you distinguish absent evidence, evidence reported elsewhere, and evidence that does not support the intended claim?
Editorial context
The audit concerns a bounded sample from one journal, not a census of psychology or proof that every reviewed measure is invalid. Its coding choices limit what it can detect. The later Flake and Fried tutorial offers complementary questions for improving transparency and justification.