Counterfactual Thinking

A causal effect compares the same unit under alternatives. For student $i$, write $Y_i(1)$ for the final score if assigned Study Sprint access and $Y_i(0)$ for the score if assigned usual resources. The individual effect is $Y_i(1)-Y_i(0)$.

Prerequisites

Complete Prediction, Description, and Causation. Means, proportions, and differences are sufficient for the worked example.

Learning goals

• Express an individual causal effect with two potential outcomes.
• Explain why one potential outcome is unobserved for each unit.
• Separate an observed group difference from a treatment effect.

The fundamental missing outcome

Only one assignment is realized for a student, so only one of $Y_i(1)$ and $Y_i(0)$ is observed. The other is counterfactual. This is not ordinary missing data that can be recovered from a second column; it is the central design problem of causal inference.

The observed outcome can be written as

$Y_i=A_iY_i(1)+(1-A_i)Y_i(0).$

Comparing different students substitutes one group for each group's unobserved alternative. That substitution is credible only under design or modeling assumptions.

A deterministic teaching population

This table is fictional and deliberately reveals both potential outcomes. Real studies never observe both for the same student.

students = [
    # name, assigned access, score without access, score with access
    ("A", 0, 50, 58),
    ("B", 0, 55, 62),
    ("C", 0, 60, 66),
    ("D", 1, 65, 70),
    ("E", 1, 75, 78),
    ("F", 1, 80, 82),
]

observed_treated = [y1 for _, a, _, y1 in students if a == 1]
observed_control = [y0 for _, a, y0, _ in students if a == 0]
observed_difference = (
    sum(observed_treated) / len(observed_treated)
    - sum(observed_control) / len(observed_control)
)
finite_population_ate = sum(y1 - y0 for _, _, y0, y1 in students) / len(students)

print(f"observed difference: {observed_difference:.1f}")
print(f"finite-population ATE: {finite_population_ate:.1f}")

The observed difference is 21.7 points, while the finite-population average effect is 5.2. In the constructed schedule, more-prepared students received access. The large observed difference mixes baseline readiness with treatment effects. The code illustrates a logical distinction; access to simulated truth does not prove that an estimand is identified in real data.

Make the alternatives precise

“No program” could mean no coaching, another course, or delayed access. “Program” could vary by tutor, dose, format, and compliance. Counterfactual notation is meaningful only after these versions are bounded. The follow-up clock must also start at a common eligibility and assignment time.

The comparison is asymmetric in practice: an effect of assigning access versus usual resources need not answer the effect of compulsory attendance versus self-study. Changing the intervention changes the potential outcomes and therefore the estimand.

Assumptions and limits

The individual effect is generally unobservable. Average effects can be identified when a comparison group represents the missing potential outcomes under explicit assumptions. Randomization supports that exchange; observational comparisons require additional causal knowledge and adjustment. No algorithm reconstructs counterfactuals from associations alone.

Common mistakes

• Treating a student's earlier score as their no-treatment potential outcome.
• Defining treatment after seeing post-treatment behavior.
• Comparing different eligibility times or follow-up windows.
• Saying a counterfactual is “what would have happened” without naming the alternative intervention.
• Reading the fictional full potential-outcome table as a data format available in practice.

Practice

For a reminder email, define $Y(1)$ and $Y(0)$ including recipient population, send time, message version, and follow-up. List one treatment variation that would make your definition ambiguous.

Next steps

Causal Graphs and DAGs represents assumptions about how assignment and outcomes are generated.

Primary sources

Checked 2026-10-15: Rubin, “Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies” and Hernán and Robins, Causal Inference: What If.