Skip to main content

Re-ranking with Business Constraints

A relevance score is not a complete serving decision. The final stage must enforce eligibility and may trade some predicted relevance for diversity, freshness, risk control, inventory health, or other declared product goals.

Prerequisites​

Understand learning to rank, candidate provenance, and beyond-accuracy metrics.

Learning goals​

  • Separate hard constraints from soft preferences.
  • Make safety, availability, freshness, diversity, and policy rules explicit.
  • Record pre-policy and served orders plus rejection reasons.
  • Measure the relevance and coverage cost of re-ranking.

Constraint contract​

The local policy requires safe and available items, at most one result per provider, and at least one result published on or after 2026-10-01 in the top three. Safety and availability are hard filters. Provider diversity and freshness are explicit list constraints. A production owner must define precedence and fallback when the candidate pool cannot satisfy every rule.

Do not hide policy inside a single blended score. Hard rules need auditable pass/fail reasons. Soft preferences need weights, units, and a recorded trade-off curve. Keep the original score order so relevance loss, constraint repair, and displaced items remain observable.

Worked example​

The learned order begins with an unsafe item and contains two items from the same provider. A deterministic greedy pass applies the checked-in policy.

import json
import runpy

module = runpy.run_path("scripts/run-recommender-search-evaluation.py")
catalog = module["read_csv"]("search-catalog.csv")
items = {item["item_id"]: item for item in catalog}
with open("tests/fixtures/recommender-systems/search-policy.json", encoding="utf-8") as handle:
policy = json.load(handle)

ranked = ["s5", "s4", "s3", "s7", "s1", "s8", "s2", "s6"]
served = module["rerank"](ranked, items, policy)
print(f"pre_policy_top3={ranked[:3]} violations={module['violations'](ranked[:3], items, policy)}")
print(f"served={served} violations={module['violations'](served, items, policy)}")

Expected output:

pre_policy_top3=['s5', 's4', 's3'] violations=2
served=['s4', 's7', 's1'] violations=0

Report NDCG, catalog coverage, category diversity, and violations for baseline, ranked, and served stages. A zero violation count does not prove that the policy is fair, complete, or harmless. It only proves that this implementation satisfied the encoded rules on this fixture.

Limitations​

  • Greedy selection can miss a feasible or higher-utility list that a constrained optimizer would find.
  • Metadata can be stale, missing, or strategically manipulated.
  • Provider diversity is not a substitute for user-group fairness or viewpoint quality.
  • Freshness can crowd out durable content; safety classifiers can produce harmful false positives and negatives.
  • Policy changes need versioning, review, rollback, and monitoring.

Practice​

Construct a candidate pool in which the top-three cutoff and provider cap are infeasible together. Define whether to return fewer results, relax a soft rule, or invoke a safe fallback, and record who owns that decision.

Next steps​

Online evaluation and feedback loops explains why served behavior still requires bias-aware logging and controlled experiments.

Primary sources​

Checked 2026-10-17: Google for Developers, re-ranking and recommendation-system stages; Carbonell and Goldstein, The Use of MMR, Diversity-Based Reranking for Reordering Documents.