Retention and Churn
Retention is observed qualifying behavior after an anchor; churn is a declared absence or terminal event. They are not automatically complements. A person who has not returned by the data close may return later, be unobserved, withdraw consent, switch identity, or experience a competing event. Calling all of those people churned turns incomplete follow-up into a false fact.
Prerequisites
Complete cohort analysis. You should be comfortable with half-open calendar intervals and cohort maturity.
Learning goals
- Define a risk set, return event, churn event, grace period, and competing events.
- Distinguish observed churn from right censoring.
- Recognize when retention cannot be estimated from the available window.
- Keep account state and behavioral inactivity conceptually separate.
Risk time and outcomes
For a subscription, an explicit cancellation may be an observed churn event while plan migration or account deletion may be competing events. For an ad-supported product, “no activity for 28 days” is a retrospective operational rule, not an event at the last click. The actor must first accumulate 28 observable days at risk.
The fixture covers only one event day. At a close immediately after that day, none of its four eligible active actors has a complete next-day return interval. The honest result is four right-censored observations. There is no account_cancelled event in the versioned taxonomy, so no competing event is observed—not that competing events are impossible.
Metric contract
- Population: actors with at least one valid, consented
page_vieworfeature_usedon 2026-10-01 UTC. - Unit: resolved actor anchored at their first eligible activity.
- Observation window: anchor day plus the complete next UTC day; the fixture closes before that follow-up is observable.
- Denominator: actors whose complete return window is observable; censored actors remain in the risk accounting but not a naive matured-rate denominator.
- Missingness: absent future rows are right-censored; consent withdrawal, deletion, and collection outages require explicit handling rather than silent churn labels.
- Identity: fixed 2026-10-02 snapshot; unresolved identifiers may split one person, while shared accounts may combine people.
Worked example
import json
from datetime import datetime, timedelta
from pathlib import Path
root = Path("tests/fixtures/product-analytics")
rows = [json.loads(line) for line in (root / "events-v1.jsonl").read_text().splitlines()]
plan = json.loads((root / "tracking-plan-v1.json").read_text())
parse = lambda value: datetime.fromisoformat(value.replace("Z", "+00:00"))
deduped = {}
for row in sorted(rows, key=lambda item: parse(item["received_at"])):
deduped.setdefault(row["event_id"], row)
def valid(row):
required = plan["events"][row["event_name"]]["required_properties"]
kinds = {"string": str, "boolean": bool}
return all(key in row["properties"] and isinstance(row["properties"][key], kinds[kind]) for key, kind in required.items())
def actor(row):
return row["user_id"] or plan["identity_snapshot"]["links"].get(row["anonymous_id"], row["anonymous_id"])
active = {actor(row) for row in deduped.values() if row["analytics_consent"] and row["purpose"] == "product_measurement" and valid(row) and row["event_name"] in {"page_view", "feature_used"}}
data_end = datetime.fromisoformat("2026-10-02T00:00:00+00:00")
required_end = datetime.fromisoformat("2026-10-02T00:00:00+00:00") + timedelta(days=1)
censored = len(active) if data_end < required_end else 0
print(f"eligible_actors={len(active)} observed_next_day=0 right_censored={censored}")
print("retention=not_estimable churn=not_observed competing_events=none_recorded")
Expected output:
eligible_actors=4 observed_next_day=0 right_censored=4
retention=not_estimable churn=not_observed competing_events=none_recorded
Limitations and common mistakes
- Defining churn as
1 - retentionwhen censoring or competing events exist. - Moving the inactivity threshold until the result looks useful.
- Including actors who have not had enough time to demonstrate return.
- Treating deletion or denied measurement as proof of product abandonment.
- Comparing retention curves without aligned eligibility, identity, and observation support.
Practice
Extend the taxonomy on paper with subscription_cancelled and account_deleted. Define which is the event of interest, which is competing, and which is a privacy-driven observation boundary. Then state a 28-day inactivity rule without pretending the churn time equals the last event time.
Next steps
Activation and feature adoption defines milestone behavior and tests how conclusions change across thresholds.
Primary sources
Checked 2026-10-19: NIST on censoring and multiple failure modes, NIST reliability and survival functions, and Google Analytics cohort calculation modes.