Skip to main content

Causal Questions from Domain Knowledge

Causal analysis begins before a dataset or model. Domain experts know which interventions are feasible, why people receive them, which outcomes matter, how measurements are produced, and where spillovers or harms can occur. Analysts translate that knowledge into a target question and expose the remaining assumptions.

Prerequisites​

Complete Randomized vs. Observational Evidence and the preceding foundation lessons.

Learning goals​

  • Turn a stakeholder decision into a seven-part causal protocol.
  • Use domain knowledge to define interventions, graphs, measurements, and limits.
  • Write a conclusion that carries its identifying and transport assumptions.

A seven-part question protocol​

For Study Sprint, document:

  1. Decision and stakeholders: the college must decide whether to offer access next term; students bear time costs and may benefit or be harmed.
  2. Eligible population: students registered before term start who have not already taken the final.
  3. Intervention and comparator: assignment at term start to one defined six-week access package versus stated usual resources.
  4. Outcome and horizon: the scheduled final-exam score six weeks later, plus separately monitored participation burden; the score is not complete learning or welfare.
  5. Estimand: ATE as a mean score difference in next term's eligible population. Prespecified preparedness CATEs may support equitable implementation but are not individual effects.
  6. Causal structure and data process: preparedness and motivation may affect voluntary enrollment and score; assignment may affect study hours; peers may share materials; missing exams and surveys create selection paths.
  7. Identification and falsification plan: prefer randomized assignment if ethical and feasible. Otherwise emulate the target trial, justify the adjustment set, inspect overlap and missingness, use negative or placebo checks where meaningful, and perform sensitivity analyses. Such checks can reveal problems but cannot prove the identifying assumptions.

This protocol forces different questions to produce different requirements. Predicting low scores needs representative labels and validation. Describing last term's participants needs accurate records. Estimating an offer effect needs treatment alternatives, a target population, exchangeable comparisons, positivity, consistency, an interference definition, and valid measurement.

Pair every conclusion with its boundary​

A responsible result template is:

For [population], assigning [intervention] rather than [comparator] changed [outcome and horizon] by [estimate and uncertainty], if [identification, measurement, interference, and transport assumptions] hold. The study does not establish [important excluded interpretation].

For observational Study Sprint records, the excluded interpretation might be an effect of compulsory attendance, an individual benefit prediction, a long-term learning effect, or an effect in another college.

When not to estimate​

Stop or redesign when the intervention is too vague, the outcome does not represent the decision, treatment timing is unavailable, important common causes were never measured, relevant groups have no overlap, interference dominates the unit-level question, or the decision would require transporting far beyond the evidence. “Not identified with these data” is a useful finding.

Stakeholder knowledge is evidence about mechanisms and measurement, not permission to choose a favorable graph. Record competing explanations, the source of each arrow, disagreements, and what observations or designs could challenge them.

Common mistakes​

  • Starting with available columns and inventing the decision afterward.
  • Letting the estimator define the estimand.
  • Treating a domain expert's preferred story as verified fact.
  • Omitting burdens, harms, or outcomes important to affected people.
  • Reporting caveats separately from the causal claim so the headline loses its conditions.

Practice​

Create a one-page protocol for a real decision using the seven headings. Ask one domain expert to challenge the intervention versions and another to challenge measurement and selection. Revise the DAG and assumption ledger before requesting an estimate.

Next steps​

Continue to the next batch on experiment design, where decisions become testable hypotheses, units, allocation mechanisms, power calculations, and analysis plans. Until then, use this page as a pre-analysis checklist.

Primary sources​

Checked 2026-10-15: Hernán and Robins, “Using Big Data to Emulate a Target Trial”, Greenland, Pearl, and Robins, “Causal Diagrams for Epidemiologic Research”, and ICH E9(R1), estimand framework.