Skip to main content

Metric Trees

A metric tree decomposes a decision-relevant outcome into components that teams can inspect. Some edges are arithmetic identities; others are hypotheses about behavior. Marking the edge type prevents a tidy diagram from being mistaken for a causal model.

Prerequisites​

Complete North Star, input, and guardrail metrics. Understand ratios and mutually exclusive versus overlapping groups.

Learning goals​

  • Build an arithmetic decomposition at compatible grains.
  • Label definitional, observational, and causal-hypothesis edges.
  • Keep identity, window, and version consistent across a tree.
  • Use residuals to detect missing or overlapping branches.

Three kinds of edge​

Consider daily eligible active actors and valid feature users. The relationship

adoption rate=valid feature userseligible active actors\text{adoption rate} = \frac{\text{valid feature users}}{\text{eligible active actors}}

is definitional once both sets, their window, and identity snapshot are fixed. In the fixture it is 2/4=0.52/4=0.5. This does not imply that increasing activation will cause more report exports.

An additive tree requires non-overlapping, exhaustive branches. If active actors are split by authenticated status, each actor must use one declared status at the tree's grain; counting event-level states can place one actor in both branches. If branches overlap, show them as separate diagnostics rather than claiming they sum to the parent.

A multiplicative business identity can also be exact, for example revenue = paying accounts × average revenue per paying account for one currency and period. Yet “improve onboarding → raise activation → raise feature value” is a chain of hypotheses. Historical association can prioritize investigation, but an intervention effect needs an experimental or causal design.

Every node should reuse a semantic contract. A tree that combines local-week revenue, UTC-day actors, current identity links, and last month's identity snapshot is not auditable. Freeze:

  • metric and schema versions;
  • unit of analysis and eligible population;
  • event-time window, timezone, and reporting cutoff;
  • duplicate, late-data, and identity policies;
  • ownership and guardrails.

Worked example​

import json
from pathlib import Path

root = Path("tests/fixtures/product-analytics")
rows = [json.loads(line) for line in (root / "events-v1.jsonl").read_text().splitlines()]
unique = {}
for row in rows:
unique.setdefault(row["event_id"], row)
plan = json.loads((root / "tracking-plan-v1.json").read_text())
links = plan["identity_snapshot"]["links"]
actor = lambda row: row["user_id"] or links.get(row["anonymous_id"], row["anonymous_id"])
active = {actor(row) for row in unique.values()
if row["event_name"] in {"page_view", "feature_used"}
and row["analytics_consent"] and row["purpose"] == "product_measurement"
and (row["event_name"] != "feature_used" or isinstance(row["properties"].get("feature"), str))}
feature = {actor(row) for row in unique.values()
if row["event_name"] == "feature_used" and row["analytics_consent"]
and isinstance(row["properties"].get("feature"), str)}
print(f"valid_feature_users={len(feature)} eligible_active_actors={len(active)} adoption_rate={len(feature)/len(active):.3f}")
print(f"identity_snapshot={plan['identity_snapshot']['as_of']} metric_tree_is_causal=False")

Expected output:

valid_feature_users=2 eligible_active_actors=4 adoption_rate=0.500
identity_snapshot=2026-10-02 metric_tree_is_causal=False

The tree is hand-checkable because both sets use one fixture and identity snapshot. It remains descriptive.

Limitations and common mistakes​

  • Mixing grains produces ratios whose numerator and denominator do not refer to compatible units.
  • Overlapping branches make an additive decomposition exceed its parent.
  • A stable arithmetic identity can contain unstable or invalid measurements.
  • Choosing only controllable branches can hide external constraints and harms.
  • Arrows imply causality to many readers; label hypothesis edges explicitly.

Practice​

Decompose eligible_active_actors by known and unresolved identity at the actor-day grain. State how an actor with anonymous and authenticated events is assigned, then calculate a residual check.

Next steps​

Test whether the inputs deserve to enter the tree in instrumentation validation and data quality.

Primary sources​

Checked 2026-10-19: Rodden, Hutchinson, and Fu, Goals-Signals-Metrics process; NIST, Engineering Statistics Handbook: Process Modeling; Hernán and Robins, Causal Inference: What If.