Matrix Factorization
Matrix factorization learns a short vector for each known user and item. A dot product provides a score, reducing a sparse user-item matrix to dense latent state. The dimensions are predictive coordinates, not automatically human-readable topics.
Prerequisites
You should understand dot products, matrix shapes, train/test separation, and the sparse local evidence in user and item neighborhoods.
Learning goals
- State the user-factor and item-factor shapes.
- Separate the training objective from the inference dot product.
- Explain regularization, biases, non-uniqueness, and cold-start limits.
- Evaluate factorization under the same candidates and split as simpler models.
Model contract
| Part | This lesson's contract |
|---|---|
| Objective | Learn two-dimensional representations that score observed training preference higher while controlling complexity. |
| Input representation | A sparse user-item table built before the shared fixture's cutoff; explicit values or an explicitly chosen implicit objective. |
| Learned state | User factors P, item factors Q, optional user/item/global biases, and training hyperparameters. |
| Inference path | Look up one user row and candidate item rows, compute dot products plus biases, filter, and rank. |
| Evaluation boundary | Fit and tune on training-only data; score the frozen test candidates, including separate warm and cold cohorts. |
| Principal failure modes | Non-identifiable factors, overfitting, popularity effects, objective mismatch, stale embeddings, and missing factors for cold entities. |
Worked example: inspect shapes and scores
The following vectors are a frozen learned-state artifact for arithmetic inspection. They are deliberately supplied rather than fitted on the tiny fixture: fitting many parameters to a few rows would make a meaningless performance demonstration.
user = (0.8, 0.2)
item_factors = {
"i3": (0.8, 0.1),
"i4": (0.4, 0.1),
"i5": (0.9, 0.1),
"i6": (0.3, 0.1),
"i7": (0.2, 0.1),
}
ranking = sorted(
((item, sum(a * b for a, b in zip(user, vector))) for item, vector in item_factors.items()),
key=lambda pair: (-pair[1], pair[0]),
)
print(f"user_shape={(len(user),)} item_matrix_shape={(len(item_factors), len(user))}")
print("ranking=[" + ", ".join(f"('{item}', {score:.3f})" for item, score in ranking) + "]")
Output:
user_shape=(2,) item_matrix_shape=(5, 2)
ranking=[('i5', 0.740), ('i3', 0.660), ('i4', 0.340), ('i6', 0.260), ('i7', 0.180)]
For users, items, and latent dimensions, the learned matrices have shapes and . The score is , optionally plus biases. Rotating both factor spaces can preserve every dot product, so labeling a dimension “statistics interest” without separate evidence is unjustified.
Limitations and common mistakes
- Ordinary squared error over observed ratings is not the same objective as top-k ranking from implicit data.
- Filling every missing cell with zero turns absence into strong negative preference.
- Factor dimension, regularization, epochs, and sampling are tuning choices and belong inside training validation.
- A known ID lookup cannot score a cold user or cold item without side features or a fallback.
- Lower reconstruction error on this fixture would not establish ranking quality or product impact.
Practice
Add an item bias of 0.5 to i4 and recompute. Explain why the new order says nothing about the semantic meaning of either latent dimension.
Next steps
Learning from implicit feedback makes preference, confidence, and sampled comparisons explicit instead of silently filling the matrix.
Primary sources
Checked 2026-10-17: Koren, Bell, and Volinsky, “Matrix Factorization Techniques for Recommender Systems”, and Google, collaborative-filtering basics.