Skip to main content

North Star, Input, and Guardrail Metrics

A metric portfolio makes tradeoffs visible. A candidate North Star summarizes a form of value delivery, input metrics describe necessary behaviors or capabilities, and guardrails reveal harms or constraints that the main number can hide. These roles are context-specific; “choose one North Star” is not a universal law.

Prerequisites​

Complete metric definitions and semantic contracts. Be able to distinguish a count from a rate and a descriptive association from an intervention effect.

Learning goals​

  • Evaluate whether a candidate metric represents product value for a declared decision.
  • Pair an outcome candidate with actionable inputs and non-negotiable guardrails.
  • Preserve each metric's full semantic contract.
  • Explain why optimizing a proxy can damage the underlying goal.

A portfolio, not a scoreboard​

For the tiny reporting-product fixture, valid_feature_users is a possible value signal: two eligible actors successfully used report export. It is not automatically a North Star. Export frequency may reward accidental repetitions, say nothing about report usefulness, omit collaborators, or conflict with accessibility and privacy. The owner must connect the proxy to a product-specific value hypothesis and revisit that connection.

verified_activation_rate is an input candidate because people generally need accounts before recurring authenticated use. It remains a descriptive rate, not proof that raising it will raise valuable use. A consent-boundary count is a guardrail because product measurement must not grow by accepting collection that the declared purpose does not permit.

Useful guardrails can cover reliability, latency, accessibility, complaints, deletion completion, fairness, safety, and cost. A guardrail needs an owner and action threshold; displaying it without a response policy is weak governance. Privacy is not merely a metric to trade against growth: lawful basis, purpose limitation, consent, and rights can impose hard constraints.

Before adopting a portfolio, ask:

  1. What decision will the portfolio inform?
  2. Whose value and whose possible harm are represented?
  3. What behavior could improve the proxy while worsening the goal?
  4. Which leading inputs are actually actionable?
  5. Which constraints stop optimization?
  6. What experimental or causal design would be needed for an intervention claim?

Worked example​

import json
from datetime import datetime
from pathlib import Path

root = Path("tests/fixtures/product-analytics")
rows = [json.loads(line) for line in (root / "events-v1.jsonl").read_text().splitlines()]
unique = {}
for row in rows:
unique.setdefault(row["event_id"], row)
links = json.loads((root / "tracking-plan-v1.json").read_text())["identity_snapshot"]["links"]
actor = lambda row: row["user_id"] or links.get(row["anonymous_id"], row["anonymous_id"])
valid_features = {actor(row) for row in unique.values()
if row["event_name"] == "feature_used" and row["analytics_consent"]
and isinstance(row["properties"].get("feature"), str)}
starters = {actor(row) for row in unique.values() if row["event_name"] == "signup_started"}
close = datetime.fromisoformat("2026-10-03T00:00:00+00:00")
activated = {actor(row) for row in unique.values() if row["event_name"] == "account_created"
and datetime.fromisoformat(row["received_at"].replace("Z", "+00:00")) <= close}
boundary = [row for row in unique.values()
if row["purpose"] == "product_measurement" and not row["analytics_consent"]]
print(f"candidate=valid_feature_users value={len(valid_features)}")
print(f"input=verified_activation_rate value={len(activated)/len(starters):.3f} guardrail=consent_boundary_rows value={len(boundary)}")

Expected output:

candidate=valid_feature_users value=2
input=verified_activation_rate value=0.500 guardrail=consent_boundary_rows value=1

These values test the measurement contract. They are not targets and have no external validity beyond the synthetic fixture.

Limitations and common mistakes​

  • A single North Star can erase heterogeneous users, harms, or business constraints.
  • Calling a metric “input” does not show that changing it causes the outcome.
  • Targets invite gaming when teams can change the proxy without improving value.
  • Averaging guardrails can hide severe failures for a small group.
  • More collection does not repair a weak value model and may violate data minimization.

Practice​

Propose a different candidate North Star for a collaborative reporting product. Name one beneficial behavior it misses, one way it can be gamed, two input metrics, and three guardrails including deletion completion.

Next steps​

Make the assumed relationships explicit in metric trees.

Primary sources​

Checked 2026-10-19: Rodden, Hutchinson, and Fu, HEART and Goals-Signals-Metrics; UK Government Analysis Function, Performance measurement; W3C, Privacy Principles.