Preregistration and Analysis Plans

Preregistration creates a time-stamped record of what was decided before outcomes could influence those decisions. A pre-analysis plan makes the record executable enough that another analyst can identify the primary result, reproduce it, and distinguish planned analysis from exploration.

Preregistration does not make a weak design strong, guarantee faithful implementation, or ban learning. Its value comes from specificity, timing, version history, and honest labeling of deviations.

Prerequisites

Complete Peeking, Stopping, and Sequential Designs and the preceding design lessons.

Learning goals

• Produce a defensible experiment plan with assignment, power assumptions, outcomes, guardrails, and stopping rules.
• Separate protocol fields, statistical analysis, operational monitoring, and exploratory work.
• Version and report changes without pretending they were planned earlier.

A decision-ready plan

Before assignment, preregister these fields:

1. Decision and governance: owner, affected stakeholders, adoption rule, review process, conflicts, and responsible person for harm decisions.
2. Population and timing: eligibility, exclusions, recruitment, consent, assignment time zero, outcome horizon, and transport target.
3. Units and interference: assignment unit, observation unit, unit of analysis, cluster or network boundaries, and spillover assumptions.
4. Treatment and comparator: reproducible versions, delivery checks, allowed co-interventions, exposure measures, and noncompliance definition.
5. Estimands: primary intention-to-treat contrast, outcome scale, population weighting, intercurrent-event strategy, and any secondary or heterogeneous effects.
6. Assignment: mechanism, probabilities, ratio, blocks or pairs, concealment, seed custody or allocation service, and audit fields.
7. Outcomes: one primary outcome with source, scale, aggregation, timing, validity rules, plus secondary outcomes and guardrail metrics.
8. Power: smallest worthwhile effect, minimum detectable effect, alpha, target power, variance source, clustering, attrition, noncompliance, multiplicity, allocation, sensitivity grid, and rounding.
9. Analysis: estimator, covariates, design terms, standard errors, missing-data strategy, multiplicity family, interval, diagnostic checks, and software environment.
10. Stopping and monitoring: fixed endpoint or sequential looks, efficacy and futility boundaries, stopping for harm, delayed outcomes, access roles, and post-stop follow-up.
11. Ethics and data stewardship: informed consent or justified review outcome, exposure minimization, privacy, retention, access, fairness, complaints, and local oversight.
12. Reporting: participant flow, implementation failures, exclusions, deviations, all planned outcomes, uncertainty, limitations, and repository/archive location.

This plan should point to frozen code or formulas where practical. A vague promise to “use regression if needed” gives the analyst too many outcome-informed choices.

Freeze, amend, and disclose

Register before assignment or, at minimum, before anyone with analysis influence can inspect outcomes. Preserve the original record. If circumstances require a change, create a dated amendment stating what changed, why, which data were visible, who decided, and how interpretation changes. Never silently overwrite the old plan.

Classify results as preregistered confirmatory, amended before outcome access, or exploratory after outcome access. Exploration is valuable for generating future questions, but its uncertainty and multiplicity differ from a frozen confirmatory test. A registry entry without a detailed analysis plan may establish that a study existed while leaving analytical flexibility unresolved.

Study Sprint worked plan

The decision is to offer the six-week package next term. Eligible students are frozen before 1:1 individual allocation, stratified by prior-score band. The primary estimand is the six-week intention-to-treat mean score effect; 4 points is the independently justified worthwhile effect. Planning assumes an SD of 10, two-sided alpha 0.05, 80% power, and 99 students per arm before justified attrition allowance. The primary model includes treatment and block indicators.

The guardrails are withdrawal, reported burden, access failures, privacy incidents, and prespecified subgroup deterioration. The fixed design is analyzed after the scheduled exam; operations can stop exposure for a credible harm or privacy threshold through the named governance path. Missing exams, delivery failures, and every deviation are reported by arm. No decision is based solely on whether the p-value is below 0.05.

Final review checklist

Ask an independent reviewer to trace each decision backward: does the stopping rule match the analysis, does the analysis match assignment, does assignment match the interference model, do outcomes match the estimand, and does the estimand answer the original decision? Recalculate the power table and run the allocation code before launch. Confirm that consent and review are appropriate for the actual exposure and jurisdiction.

Common mistakes

• Registering after examining outcome patterns.
• Listing hypotheses without operational definitions or an analysis plan.
• Hiding deviations by editing the original record.
• Treating every secondary metric as confirmatory without a multiplicity strategy.
• Using preregistration as a badge while omitting harms, implementation failures, or null results.

Practice

Fill all twelve headings for Study Sprint in one page, link the allocation and power scripts, and ask a colleague to produce the primary result using only the plan. Any question they must ask reveals an unresolved degree of freedom to fix or disclose.

Next steps

The next batch begins with difference in means and uncertainty, then adds regression adjustment, metric-specific methods, multiplicity, missingness, noncompliance, variance reduction, and heterogeneous effects. Carry this frozen plan into every analysis lesson.

Primary sources

Checked 2026-10-15: AEA RCT Registry policy, AEA RCT Registry documentation, CONSORT 2010 statement, and FDA, Adaptive Design Clinical Trials guidance.