Skip to main content

Synthetic Controls

Synthetic control estimates what would have happened to one treated aggregate unit by weighting untreated donor units to reproduce its pre-treatment trajectory and predictors. Its credibility comes from design logic, pre-treatment fit, a defensible donor pool, and placebo evidence—not from the optimizer alone.

Prerequisites​

Complete Instrumental Variables. Be comfortable distinguishing unit-level effects from an effect on a city, campus, region, or other aggregate unit.

Learning goals​

  • define the treated unit, intervention time, donor pool, predictors, and post-treatment gap before fitting.
  • inspect pre-treatment fit and donor weights without treating either as proof.
  • run in-space, in-time, leave-one-donor-out, and contamination checks.

Assignment mechanism and target​

Suppose one campus adopts Study Sprint campus-wide in term 0. The estimand is the effect on that campus's average score in each post-adoption term. Donor campuses must not adopt the program, anticipate it, receive spillovers, or experience incompatible concurrent shocks.

The design assumes a weighted donor combination can approximate the treated campus's untreated outcome path after matching relevant pre-treatment outcomes and predictors. Long pre-period fit helps assess this claim but cannot observe the post-treatment counterfactual. Measurement and composition must stay comparable, and the intervention must be well defined.

Worked synthetic path and contaminated donor​

This deterministic simulation uses fixed nonnegative weights summing to one. Donors A and B exactly reproduce the treated pre-period path. The valid post-period gap is four points. If donor B receives a concurrent improvement, the synthetic comparison moves and the apparent effect falls to one.

treated = [10, 12, 14, 20]
donor_a = [8, 10, 12, 14]
donor_b = [12, 14, 16, 18]
weights = [0.5, 0.5]
synthetic = [weights[0] * a + weights[1] * b for a, b in zip(donor_a, donor_b)]
pre_rmspe = (sum((a - b) ** 2 for a, b in zip(treated[:3], synthetic[:3])) / 3) ** 0.5
effect = treated[3] - synthetic[3]
contaminated_b_post = 24
contaminated_post = weights[0] * donor_a[3] + weights[1] * contaminated_b_post
print(f"donor weights=A:{weights[0]:.1f}, B:{weights[1]:.1f}")
print(f"pre-treatment RMSPE={pre_rmspe:.1f}")
print(f"post-treatment effect={effect:.1f}")
print(f"contaminated-donor apparent effect={treated[3] - contaminated_post:.1f}")

Perfect pre-fit in a tiny constructed fixture is not empirical validation. It may reflect overfitting when there are many donors and few pre-periods. The donor shock counterexample shows why pool construction is a substantive design decision.

Diagnostics and falsification​

Plot treated and synthetic outcomes over the full period and the gap separately. Report predictor balance, pre-treatment RMSPE, every donor weight, and whether the treated unit lies in the donors' convex hull. Compare alternative pre-periods and pre-declared predictor sets.

Run in-space placebos by assigning treatment to donors and comparing post/pre fit ratios, while excluding placebo units with incomparable pre-fit transparently. Run in-time placebos before treatment, leave-one-positive-weight-donor-out analyses, and checks for spillovers or concurrent donor shocks. A large treated gap relative to placebos is evidence under the design, not a randomization p-value unless a corresponding assignment mechanism is justified.

Poor pre-fit means the donor pool does not reproduce the treated unit; adding extrapolative negative weights changes the method and assumptions. With one treated unit, uncertainty cannot be rescued by counting individual observations inside it as independent policy assignments.

Failure modes​

  • The treated unit is unlike every convex combination of donors.
  • A high-weight donor is affected by the intervention or another shock.
  • Treatment was anticipated during the fitting period.
  • Composition or outcome definitions change only in the treated unit.
  • Predictors are chosen after inspecting post-treatment gaps.

Common mistakes​

  • Treating zero pre-RMSPE as proof of an unbiased counterfactual.
  • hiding donor identities or weights.
  • using affected donors because they improve pre-fit.
  • claiming individual-level sample size for an aggregate intervention.
  • reporting only the largest post-treatment gap without placebos.

Practice​

Design a donor-pool audit for a campus policy. For each donor, record exposure, spillover paths, measurement changes, concurrent shocks, and reasons for inclusion made before fitting.

Next steps​

Sensitivity Analysis and Honest Conclusions combines diagnostics across methods and makes “not identified” an explicit, useful endpoint.

Primary sources​

Checked 2026-10-15: Abadie, Diamond, and Hainmueller, synthetic controls, Abadie, “Using Synthetic Controls”, and Abadie, Diamond, and Hainmueller, comparative politics.