Propensity Scores and Weighting

The propensity score is the probability of treatment given measured pre-treatment covariates. Weighting can create a pseudo-population whose measured covariate distribution targets an ATE, ATT, or overlap population. The score model is an assignment model, not an outcome forecast and not a test for hidden confounding.

Prerequisites

Complete Matching and Balance. Be able to distinguish the study population from the population retained after an overlap restriction.

Learning goals

• map each weighting formula to its estimand.
• diagnose positivity problems through score distributions, weights, effective sample size, and covariate balance.
• explain why a well-fitted propensity model or balanced weighted sample does not prove causal validity.

Assignment mechanism and estimand

For voluntary Study Sprint enrollment, estimate each student's probability of enrollment from baseline causes selected before outcomes are inspected. ATE inverse-probability weights are $1/e(X)$ for treated units and $1/(1-e(X))$ for controls. ATT weights are 1 for treated units and $e(X)/(1-e(X))$ for controls. These answer different questions.

Identification requires conditional exchangeability given the measured covariates, positivity in the target population, consistency, a defensible interference boundary, and valid measurement. Flexible prediction cannot repair an omitted common cause.

These are causal assumptions supplied by the design and subject-matter argument, not outputs of the propensity model.

Worked weighting and an overlap failure

This deterministic simulation supplies known scores so the arithmetic is visible. In a real analysis, scores are estimated without using post-treatment variables, and uncertainty must account for estimation and the sampling design.

import statistics

rows = [
    (1, 0.20, 58), (0, 0.20, 50),
    (1, 0.40, 58), (0, 0.40, 50),
    (1, 0.60, 58), (0, 0.60, 50),
    (1, 0.80, 58), (0, 0.80, 50),
]
weights = [1 / score if treated else 1 / (1 - score) for treated, score, _ in rows]
treated_mean = sum(w * y for w, (a, _, y) in zip(weights, rows) if a) / sum(
    w for w, (a, _, _) in zip(weights, rows) if a
)
control_mean = sum(w * y for w, (a, _, y) in zip(weights, rows) if not a) / sum(
    w for w, (a, _, _) in zip(weights, rows) if not a
)
effective_n = sum(weights) ** 2 / sum(w * w for w in weights)
failure_scores = [(1, 0.02), (0, 0.98)]
failure_max = max(1 / score if treated else 1 / (1 - score)
                  for treated, score in failure_scores)
print(f"weighted ATE estimate={treated_mean - control_mean:.2f}")
print(f"maximum teaching weight={max(weights):.2f}")
print(f"effective sample size={effective_n:.2f} of {len(rows)}")
print(f"positivity-failure maximum weight={failure_max:.1f}")

The regular fixture yields finite weights but still needs balance checks. The counterexample assigns treated and untreated units scores near the wrong extremes, producing weights of 50. A few observations then dominate the estimate. Truncating weights may stabilize variance but changes bias and sometimes the effective target; report the rule and compare conclusions.

Diagnostics and falsification

Plot score distributions by treatment, identify unsupported regions, report weight quantiles and maximums, and calculate effective sample size. After weighting, compare means, variances, interactions, and distributional features of baseline covariates. Refit the assignment model to improve balance, not merely discrimination or likelihood.

Use negative-control outcomes and exposures when scientifically credible. Repeat estimates across pre-declared overlap rules and weighting targets. Inspect whether conclusions depend on a handful of influential units. None of these checks establishes no unmeasured confounding.

When positivity fails structurally—certain students always or never enroll—an ATE for everyone is not recovered by extrapolating extreme weights. Narrow the population and rename the estimand, change the intervention, collect new comparisons, or say “not identified from these data.”

Failure modes

• Important assignment causes are unmeasured or measured after treatment.
• Estimated probabilities approach zero or one in the target population.
• A misspecified score model balances its own score but not important covariates.
• Peer spillovers violate the unit-level treatment definition.
• Weight trimming is selected after seeing which estimate looks favorable.

Common mistakes

• Maximizing treatment-prediction accuracy instead of achieving design balance.
• Reporting a propensity-model p-value as an identification test.
• Mixing ATE, ATT, and overlap weights while keeping one causal label.
• Hiding effective sample-size collapse behind the original row count.
• Declaring balance proof that hidden bias is absent.

Practice

For an ATE and an ATT, write the weight applied to the same untreated student with score 0.8. Explain the different target populations and state what you would do if no untreated students resemble high-propensity treated students.

Next steps

Doubly Robust Estimation combines an assignment model with an outcome model, while preserving the same identification requirements.

Primary sources

Checked 2026-10-15: Rosenbaum and Rubin, propensity scores, Crump et al., limited overlap, and Austin and Stuart, IPTW practice.