Reading focus
Syllabus: How Can We Compare Psychological Processes Across Cultures?
Distinguish the article’s counts of publications, studies and samples, and connect missing information with limits on interpretation.
Why you’re reading it. Translate the foundation’s sampling concern into a practical appraisal of research claims.
A question to consider. Your responsibility study recruits university students online in two countries. Which populations and settings could its conclusions describe, and what recruitment changes would address the most important gap?
Why this reading
Rad and colleagues turn a broad concern about sampling into an audit of published research. They examine both who participated and how authors described the reach of their conclusions.
Before you read
The study examines one journal in 2014 and a smaller follow-up period in 2017. An article, a study and a sample are different units of counting. Missing demographic information limits what an audit can establish.
Read for
- Track the denominators and coding decisions behind the reported proportions.
- Separate evidence about publication practices from the authors’ proposals for changing incentives and reporting.
Consider afterwards
- How would you rewrite a claim about “people” to identify the population actually sampled?
- What further evidence would be needed to evaluate whether a proposed editorial policy improved generalization?
Editorial context
This dated audit concerns Psychological Science, not every journal or current research practice. Broader participation and more explicit population claims are distinct objectives. The proposed reforms are arguments informed by the audit; the paper does not experimentally evaluate their effects.