Skip to main content

Matrix Factorization

Matrix factorization learns a short vector for each known user and item. A dot product provides a score, reducing a sparse user-item matrix to dense latent state. The dimensions are predictive coordinates, not automatically human-readable topics.

Prerequisites​

You should understand dot products, matrix shapes, train/test separation, and the sparse local evidence in user and item neighborhoods.

Learning goals​

  • State the user-factor and item-factor shapes.
  • Separate the training objective from the inference dot product.
  • Explain regularization, biases, non-uniqueness, and cold-start limits.
  • Evaluate factorization under the same candidates and split as simpler models.

Model contract​

PartThis lesson's contract
ObjectiveLearn two-dimensional representations that score observed training preference higher while controlling complexity.
Input representationA sparse user-item table built before the shared fixture's cutoff; explicit values or an explicitly chosen implicit objective.
Learned stateUser factors P, item factors Q, optional user/item/global biases, and training hyperparameters.
Inference pathLook up one user row and candidate item rows, compute dot products plus biases, filter, and rank.
Evaluation boundaryFit and tune on training-only data; score the frozen test candidates, including separate warm and cold cohorts.
Principal failure modesNon-identifiable factors, overfitting, popularity effects, objective mismatch, stale embeddings, and missing factors for cold entities.

Worked example: inspect shapes and scores​

The following vectors are a frozen learned-state artifact for arithmetic inspection. They are deliberately supplied rather than fitted on the tiny fixture: fitting many parameters to a few rows would make a meaningless performance demonstration.

user = (0.8, 0.2)
item_factors = {
"i3": (0.8, 0.1),
"i4": (0.4, 0.1),
"i5": (0.9, 0.1),
"i6": (0.3, 0.1),
"i7": (0.2, 0.1),
}

ranking = sorted(
((item, sum(a * b for a, b in zip(user, vector))) for item, vector in item_factors.items()),
key=lambda pair: (-pair[1], pair[0]),
)
print(f"user_shape={(len(user),)} item_matrix_shape={(len(item_factors), len(user))}")
print("ranking=[" + ", ".join(f"('{item}', {score:.3f})" for item, score in ranking) + "]")

Output:

user_shape=(2,) item_matrix_shape=(5, 2)
ranking=[('i5', 0.740), ('i3', 0.660), ('i4', 0.340), ('i6', 0.260), ('i7', 0.180)]

For mm users, nn items, and dd latent dimensions, the learned matrices have shapes P∈Rm×dP\in\mathbb{R}^{m\times d} and Q∈Rn×dQ\in\mathbb{R}^{n\times d}. The score is r^ui=pu⊤qi\hat r_{ui}=p_u^\top q_i, optionally plus biases. Rotating both factor spaces can preserve every dot product, so labeling a dimension “statistics interest” without separate evidence is unjustified.

Limitations and common mistakes​

  • Ordinary squared error over observed ratings is not the same objective as top-k ranking from implicit data.
  • Filling every missing cell with zero turns absence into strong negative preference.
  • Factor dimension, regularization, epochs, and sampling are tuning choices and belong inside training validation.
  • A known ID lookup cannot score a cold user or cold item without side features or a fallback.
  • Lower reconstruction error on this fixture would not establish ranking quality or product impact.

Practice​

Add an item bias of 0.5 to i4 and recompute. Explain why the new order says nothing about the semantic meaning of either latent dimension.

Next steps​

Learning from implicit feedback makes preference, confidence, and sampled comparisons explicit instead of silently filling the matrix.

Primary sources​

Checked 2026-10-17: Koren, Bell, and Volinsky, “Matrix Factorization Techniques for Recommender Systems”, and Google, collaborative-filtering basics.