Skip to main content

Difference-in-Differences

Difference-in-differences (DiD) compares an outcome change for an exposed group with the contemporaneous change for a comparison group. Subtracting baseline levels is not the identifying trick; the design relies on the comparison group's change representing the treated group's untreated counterfactual trend.

Prerequisites​

Complete Doubly Robust Estimation. Distinguish a treatment effect from a time trend and a group difference.

Learning goals​

  • derive a two-group, two-period ATT from four means.
  • state conditional or unconditional parallel trends, no anticipation, treatment timing, and interference assumptions.
  • use pre-period placebos and alternative comparisons as falsification evidence without calling non-rejection proof.

Assignment mechanism and target​

Suppose one campus adopts Study Sprint in term 0 while another does not. The target is the ATT for eligible students at the adopting campus after adoption. Treatment is a campus-time policy, not an individual choice.

Identification requires that, absent adoption, the adopting campus's mean outcome would have changed like the comparison campus's mean outcome. Also require no anticipation before term 0, no other campus-specific shock at term 0, stable measurement and composition, a clear treatment version, and no relevant spillovers between campuses. With repeated cross-sections, population composition must also be comparable over time.

Worked contrast and placebo failure​

This deterministic simulation contains campus-period means. The first comparison has a ten-point DiD. An earlier pseudo-treatment contrast shows the treated campus was already improving three points faster, undermining the parallel-trends story.

treated = {"early": 42, "pre": 50, "post": 65}
control = {"early": 50, "pre": 55, "post": 60}
did = (treated["post"] - treated["pre"]) - (control["post"] - control["pre"])
placebo = (treated["pre"] - treated["early"]) - (control["pre"] - control["early"])
before_after = treated["post"] - treated["pre"]
print(f"treated before-after change={before_after:.1f}")
print(f"difference-in-differences ATT={did:.1f}")
print(f"pre-period placebo DiD={placebo:.1f}")
print("design verdict=parallel trends is contradicted in this fixture")

The 15-point before-after change confounds treatment with shared time movement. DiD removes the comparison campus's five-point movement, but the three-point pre-period placebo remains. One could model a defensible conditional trend, find a better comparison, or gather more periods; simply adding a linear trend until the desired answer appears is not repair.

Diagnostics and falsification​

Plot raw outcomes and group composition across many pre-periods. Estimate pre-period placebo effects with intervals, but judge their magnitude and power: a wide interval containing zero can also contain important violations. Test pseudo-adoption dates, outcomes the program cannot affect, unaffected groups, and alternative comparison sets. Investigate simultaneous policies and measurement changes.

Pre-trend non-rejection does not prove parallel counterfactual trends after treatment. Conversely, rejecting exact equality may matter little if the deviation is too small to change the decision. Report how large a trend violation would overturn the conclusion.

With staggered adoption and heterogeneous effects, a generic two-way fixed-effects coefficient can combine inappropriate comparisons and opaque weights. Use cohort-time estimands with not-yet-treated or never-treated controls and explain their aggregation.

Failure modes​

  • Treated units act before official adoption, violating no anticipation.
  • A contemporaneous campus policy changes only the treated campus.
  • Student composition or outcome definitions change at adoption.
  • Treated students share the program with comparison students.
  • Treatment timing varies, but already-treated units are used as controls.

Common mistakes​

  • Calling a before-after change a DiD effect.
  • treating baseline level similarity as parallel trends.
  • using a non-significant lead as proof of identification.
  • ignoring dependence within the policy-assignment unit.
  • interpreting a staggered two-way fixed-effects coefficient without its comparisons.

Practice​

Write the four means and the two subtractions for a policy example. Then name one anticipation channel, one concurrent shock, one negative-control outcome, and the cluster level for inference.

Next steps​

Event Studies and Parallel Trends expands DiD across relative time and makes dynamic effects, anticipation, and misleading lead coefficients visible.

Primary sources​

Checked 2026-10-15: Callaway and Sant'Anna, multiple-period DiD, Goodman-Bacon, differences-in-differences with variation in treatment timing, and Roth et al., “What's Trending in Difference-in-Differences?”.