Project: Turn Event Data into a Product Review
This capstone turns a repository-owned event fixture and review contract into two local artifacts: machine-readable JSON evidence and a semantic HTML review. It validates before computing, preserves first close and restatement, and never upgrades an observed movement into a causal claim.
Prerequisites
Complete writing evidence-based decision memos and the preceding 22 Product Analytics & Metric Systems lessons.
Learning goals
- Run one offline command with no credentials or downloads.
- Validate schema, consent, IDs, duplicate neutrality, event time, and receive time.
- Compute metric version
1.1.0with explicit close states and unit-economic assumptions. - Inspect accessible evidence that separates facts, anomalies, hypotheses, limits, guardrails, recommendations, owners, and follow-up.
Decision contract
The 20-row fixture contains 19 unique event IDs. review-013 is a harmless producer retry: deduplication removes it and distinct-actor adoption is unchanged. review-017 is valid behavior received after first close, so current adoption restates from 3/4 to 4/4. Baseline adoption is 2/4. These are real properties of the synthetic fixture, not evidence about a production population.
The financial contract declares EUR, a one-day horizon, recognized revenue, refunds, three allocated costs, three missing-cost categories, uncertainty, and non-financial guardrails. The decision owner is core-product; data quality, economics, and experiment follow-up have different owners.
Run the public command with an explicit local directory:
npm run project:product-review -- --output-dir /tmp/product-review
It creates product-review.json and product-review.html. The HTML uses a language declaration, one main landmark, ordered headings, a captioned table with scoped headers, and text labels that do not rely on color.
Worked example
import json
import subprocess
import sys
import tempfile
from pathlib import Path
bundles = []
for _ in range(2):
directory = tempfile.TemporaryDirectory()
subprocess.run([sys.executable, "scripts/run-product-review.py", "--output-dir", directory.name], check=True, capture_output=True, text=True)
root = Path(directory.name)
bundles.append((directory, (root / "product-review.json").read_bytes(), (root / "product-review.html").read_bytes()))
evidence = json.loads(bundles[0][1])
html = bundles[0][2].decode()
print(f"byte_stable={bundles[0][1:] == bundles[1][1:]} facts={len(evidence['verified_facts'])} anomalies={len(evidence['anomalies'])} recommendations={len(evidence['recommendations'])}")
print(f"html_accessible={all(token in html for token in ['<html lang=\"en\">', '<main>', '<caption>', '<th scope=\"col\">'])} metric_version={evidence['metric_version']}")
for directory, _, _ in bundles:
directory.cleanup()
Expected output:
byte_stable=True facts=6 anomalies=2 recommendations=2
html_accessible=True metric_version=1.1.0
Run the complete batch contract with:
npm run test:product-analytics-decisions-project
The verifier executes all eight lesson examples, runs the capstone twice, compares both artifacts byte for byte, checks accessible structure, audits all 24 ordered adjacent lessons, and verifies ledger and source evidence.
Limitations and common mistakes
- The fixture is tiny, synthetic, and deliberately legible; it is not a benchmark or forecast.
- Exact reproduction cannot establish construct validity, external validity, or causal impact.
- The HTML structure is statically checked, not a substitute for assistive-technology and user testing.
- The economic result excludes declared costs and carries no uncertainty interval.
- A passing guardrail means only that the encoded check passed; accessibility remains explicitly unmeasured.
Practice
Add a second delayed event and predict which first-close and restated fields should change. Add one invalid event and require the command to fail closed rather than silently exclude it. Then draft an experiment estimand without changing the observational evidence.
Next steps
You have completed all 24 Product Analytics & Metric Systems lessons. Revisit product decisions and measurement questions and audit how the capstone carries the original decision through metric contracts, behavioral analysis, economics, causal limits, and accountable follow-up.
Primary sources
Checked 2026-10-19: OpenTelemetry, Semantic conventions for events; W3C WAI, Tables Tutorial; Hernán and Robins, Causal Inference: What If; HM Treasury, The Green Book 2026.