Psychology Reader

What Would Justify a Causal Claim?

From a counterfactual question to a defensible comparison

Level
Intermediate
Reading load
9 core readings in 6 units
Prerequisites
How to Read Psychological Science, or equivalent familiarity with experiments, measurement and scientific uncertainty. Comfort with basic algebra and the idea of statistical adjustment is helpful. The guides introduce potential outcomes, causal diagrams and the main design terms.
Last reviewed
September 2026

Introduction

Students who attend a study group may do better on an examination. The observation is useful, but a further question remains: what would happen if more students attended? A causal claim crosses from the pattern observed to the consequences of a change. To justify that crossing, we need a specified alternative and a comparison that separates the change from other influences.

This syllabus begins by distinguishing scientific questions and defining the unobserved alternative outcome. It then examines what assigning people to different conditions contributes, when adjustment is useful, and how an observational study can make its intended comparison explicit. Studies of changes not assigned by the researchers connect those arguments with development in an actual community. The final units distinguish effects from mechanisms and ask how approaches with different weaknesses can strengthen an inference together.

By the end, you should be able to state the alternatives a causal claim compares, explain which assumptions a design needs, identify a plausible alternative explanation and propose evidence that would bear on it. The aim is a defensible argument about a particular question, with its limits visible. The name of a method cannot supply that argument on its own.

Unit 1 of 6

What change are we asking about?

Begin with the scientific task. Predicting an outcome and predicting what would happen under an intervention require different justifications. The potential-outcome framework then makes the missing comparison explicit: a person cannot simultaneously experience both alternatives at the same time.

  1. Reading 1 of 9

    A Second Chance to Get Causal Inference Right

    Hernán, M. A., Hsu, J., & Healy, B. (2019). A second chance to get causal inference right: A classification of data science tasks. CHANCE, 32(1), 42–49. https://doi.org/10.1080/09332480.2019.1579578

    Methodological contribution · Foundational argument · Moderate · Full work: 8 pages

    Compare the descriptive, predictive and causal questions that could be asked of one dataset.

    Why you’re reading it. Separate a useful forecast from a claim about changing an outcome.

    A question to consider. For the association between study-group attendance and exam performance, write one question of each kind.

  2. Reading 2 of 9

    Estimating Causal Effects of Treatments

    Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701. https://doi.org/10.1037/h0037350

    Foundational argument · Methodological contribution · Challenging · Full work: 14 pages

    Translate the potential-outcome notation into a specific person, intervention, comparison and time.

    Why you’re reading it. Define the effect before deciding how to estimate it.

    A question to consider. For a study-group intervention, what is the missing outcome for a student who attends, and what could another student legitimately tell us about it?

Unit 2 of 6

What does assignment contribute?

Random assignment addresses selection into treatment, but its actual implementation and the observations available afterwards still matter. The training study compares assigned and chosen conditions while holding much of the setting constant. Examine why carefully measured preexisting differences help here and what would change elsewhere.

  1. Reading 3 of 9

    Can Nonrandomized Experiments Yield Accurate Answers?

    Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random and nonrandom assignments. Journal of the American Statistical Association, 103(484), 1334–1344. https://doi.org/10.1198/016214508000000733

    Methodological contribution · Challenging · Full work: 11 pages

    Track both stages of allocation and compare what the different sets of adjustment variables accomplish.

    Why you’re reading it. Evaluate assignment and adjustment in an actual methodological experiment.

    A question to consider. If the same training comparison had substantial dropout and no pretest of relevant skills, which part of its causal argument would become weaker?

Unit 3 of 6

What makes an observational comparison informative?

A causal diagram states assumptions about relationships; it does not discover them by itself. Use that language to assess adjustment, then make an observational study’s intended experiment explicit. Eligibility, treatment strategies, starting times and outcomes must describe a comparison the available data can support.

  1. Reading 4 of 9

    Thinking Clearly About Correlations and Causation: Graphical Causal Models for Observational Data

    Rohrer, J. M. (2018). Thinking clearly about correlations and causation: Graphical causal models for observational data. Advances in Methods and Practices in Psychological Science, 1(1), 27–42. https://doi.org/10.1177/2515245917745629

    Methodological contribution · Moderate · Full work: 16 pages

    Compare plausible diagrams before deciding which variables to adjust for.

    Why you’re reading it. Explain why controlling more variables can either help or damage the intended causal comparison.

    A question to consider. Would adjusting for motivation measured after joining a study group help estimate the total effect of attendance? Draw two plausible stories before answering.

  2. Reading 5 of 9

    Using Big Data to Emulate a Target Trial

    Hernán, M. A., & Robins, J. M. (2016). Using big data to emulate a target trial when a randomized trial is not available. American Journal of Epidemiology, 183(8), 758–764. https://doi.org/10.1093/aje/kwv254

    Methodological contribution · Challenging · Full work: 7 pages

    Connect each protocol decision to the corresponding information in an observational study.

    Why you’re reading it. Turn causal assumptions into a concrete design whose gaps can be inspected.

    A question to consider. Using records of students who choose to attend a study group, specify eligibility, comparison strategies, starting time and an outcome; what important information might still be missing?

Unit 4 of 6

When can circumstances supply a comparison?

An external change may make some alternative explanations less plausible without removing all of them. Rutter organizes designs around the threats they address. Apply that reasoning to an income change during an ongoing study, separating receipt of a payment, crossing a poverty threshold and other changes in the community.

  1. Reading 6 of 9

    Proceeding from Observed Correlation to Causal Inference: The Use of Natural Experiments

    Rutter, M. (2007). Proceeding from observed correlation to causal inference: The use of natural experiments. Perspectives on Psychological Science, 2(4), 377–395. https://doi.org/10.1111/j.1745-6916.2007.00050.x

    Methodological contribution · Major synthesis · Moderate · Full work: 19 pages

    Match each design to the alternative explanation it helps address and state what remains assumed.

    Why you’re reading it. Broaden the available comparisons when investigator assignment is unavailable or inappropriate.

    A question to consider. What would a change in income policy help establish about children’s symptoms that an ordinary comparison of richer and poorer families would leave unresolved?

  2. Reading 7 of 9

    Relationships Between Poverty and Psychopathology: A Natural Experiment

    Costello, E. J., Compton, S. N., Keeler, G., & Angold, A. (2003). Relationships between poverty and psychopathology: A natural experiment. JAMA, 290(15), 2023–2029. https://doi.org/10.1001/jama.290.15.2023

    Landmark empirical study · Methodological contribution · Moderate · Full work: 7 pages

    Reconstruct the before-and-after and between-group comparisons, distinguishing behavioral and emotional outcomes.

    Why you’re reading it. Evaluate a concrete natural experiment and identify the boundary between an effect and its proposed explanation.

    A question to consider. What evidence would help separate the income payment’s influence from other changes accompanying the casino opening?

Unit 5 of 6

Does finding an effect explain how it happens?

An intervention can change an outcome while several mechanisms remain possible. A measured intermediate variable is not automatically an identified pathway. Bring that distinction to the parental-supervision interpretation in the income study and to the demands of a new mechanism experiment.

  1. Reading 8 of 9

    Yes, but What’s the Mechanism?

    Bullock, J. G., Green, D. P., & Ha, S. E. (2010). Yes, but what’s the mechanism? (Don’t expect an easy answer). Journal of Personality and Social Psychology, 98(4), 550–558. https://doi.org/10.1037/a0018933

    Critical reappraisal · Methodological contribution · Challenging · Full work: 9 pages

    Separate the assumptions needed for a total effect from those needed for an indirect effect.

    Why you’re reading it. Keep a plausible developmental explanation distinct from evidence that identifies its pathway.

    A question to consider. If income payments change both parental supervision and children’s symptoms, what additional design could test the supervision explanation, and what new assumption would that design introduce?

Unit 6 of 6

What can different approaches establish together?

Several similar estimates may share one source of error. Conversely, different estimates may concern different exposure durations or populations. End by comparing the questions, assumptions and expected biases before deciding whether a set of results strengthens a common causal claim.

  1. Reading 9 of 9

    Triangulation in Aetiological Epidemiology

    Lawlor, D. A., Tilling, K., & Davey Smith, G. (2016). Triangulation in aetiological epidemiology. International Journal of Epidemiology, 45(6), 1866–1886. https://doi.org/10.1093/ije/dyw314

    Methodological contribution · Challenging · Full work: 21 pages

    Use the criteria for triangulation to compare causal questions and likely sources of bias across designs.

    Why you’re reading it. Build a cumulative argument without counting significant findings or assuming different methods have independent errors.

    A question to consider. For the effect of family income on children’s symptoms, propose two approaches with meaningfully different weaknesses; what agreement or disagreement would change your judgment?

Scope and omissions

The nine core articles span six units and 112 full-work pages. This is an intermediate conceptual path through causal identification, with formal arguments, diagrams and design comparisons. Understanding what the calculations assume matters more here than being able to implement every estimator. Manuscript and bundled access copies have different page counts, explained in their access notes.

The examples connect education, development, psychological mechanisms and epidemiology. Each setting has its own populations, measures, timing and ethical constraints. Income support in one rural community does not stand for every change in poverty, and health-method examples do not supply treatment advice. Consent, feasibility, burdens and potential harms constrain which interventions can be studied.

This is not exhaustive training in regression discontinuity, instrumental variables, interrupted time series, longitudinal estimation, sensitivity analysis or causal discovery. Optional works extend some of those questions. Identifying an effect in one study remains distinct from explaining every part of a process or generalizing it to new people and settings.

Synthesis

Causal reasoning asks what would differ under alternative conditions, for particular people over a particular period. Since we cannot observe both alternatives for the same case at the same time, the design must justify the comparison we use. Random assignment, adjustment, and natural experiments offer different routes, each with conditions and vulnerabilities that belong in the argument.

Keep three further distinctions visible. A well-identified average effect need not describe every individual. Evidence that an intervention works does not automatically identify its mechanism. Agreement among studies is most useful when their questions are comparable and their important sources of bias differ. Disagreement can expose either an inferential weakness or a genuine difference in the effects being estimated.

Return to the study-group example. Specify the intervention, comparator, population, outcome and time. Sketch an assignment-based design and an observational alternative. For each, name a serious threat, the evidence that could reduce it, and one assumption that would remain. Then propose a mechanism and a further comparison capable of testing it. Explain what you would learn from agreement across the approaches and what disagreement would lead you to investigate.

To examine how these design arguments connect to psychological explanation, explore How Do We Build and Test Psychological Theories?. For environmental change across development, continue with How Do Contexts Shape Developmental Pathways?.

For effect magnitude, uncertainty and the assumptions behind statistical summaries, continue with What Can Statistical Evidence Tell Us?.

Further reading

Optional. Not part of the core sequence.

  • Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960. Paper. An extended comparison of the potential-outcome framework with other accounts of causation. The selected article is pp. 945–960; separately published discussion has its own pagination.
  • Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. Paper. Develops propensity-score adjustment. Consider what balance on observed covariates can establish and which assumptions about unobserved causes remain.
  • Cook, T. D., Shadish, W. R., & Wong, V. C. (2008). Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons. Journal of Policy Analysis and Management, 27(4), 724–750. Paper. Extends the design comparison to a set of studies and examines conditions under which their estimates agree. Its findings are conditional on the comparisons examined.
  • Bullock, J. G., & Green, D. P. (2021). The failings of conventional mediation analysis and a design-based alternative. Advances in Methods and Practices in Psychological Science, 4(4), Article 25152459211047227. Paper. A later argument for designing experimental contrasts around a proposed mechanism. Examine the identification assumptions of the alternative as carefully as those of conventional mediation.
  • Treur, J. L., Lukas, E., Sallis, H. M., & Wootton, R. E. (2024). A guide for planning triangulation studies to investigate complex causal questions in behavioural and psychiatric research. Epidemiology and Psychiatric Sciences, 33, Article e61. Paper. Turns triangulation into a prospective planning process, including the question, proposed designs, biases and expected results. Public full text.
  • Hernán, M. A., Dahabreh, I. J., Dickerman, B. A., & Swanson, S. A. (2025). The target trial framework for causal inference from observational data: Why and when is it helpful? Annals of Internal Medicine, 178(3), 402–407. Paper. Clarifies the difference between problems created by design choices and limitations of available observational data. Public author manuscript.