Applied ML systems
Learning to rank basics
Learning to rank trains a model to order a list. Pointwise, pairwise and listwise methods differ in what the loss compares, not in how you serve the sorted list.
After candidate generation hands you a few hundred items, something has to decide their order. That is the ranking problem. Learning to rank (LTR) trains a model to produce that order. Search engines, feeds, ads and RAG re-rankers all face the same problem: given a query and a set of candidates, put the most relevant first.
Each training example is a query (a search, or one feed request) and the documents for it. Every document gets a relevance label. The label is either binary (relevant or not) or graded, such as 0 to 4.
Concept
The three families of LTR differ in what the loss compares. Pointwise scores each document alone. Pairwise checks that a more relevant document scores above a less relevant one. Listwise looks at the whole ordered list at once.
The three approaches
| Approach | What the model learns | Typical loss | When it shines |
|---|---|---|---|
| Pointwise | Score each document independently as regression or classification | MSE or logistic | Simple CTR prediction with lots of labels |
| Pairwise | Prefer the correct order of each pair | RankNet-style pairwise logistic | When relative order matters more than absolute score |
| Listwise | Optimize the list metric directly, or a surrogate of it | Softmax list loss, LambdaMART | When NDCG at k is the product metric |
All three produce a score, and you sort by that score at serve time. Only the training signal changes.
Walking through one query
A query has three documents with grades A = 2, B = 0 and C = 1.
- Pointwise asks the model to output 2, 0 and 1 for the three documents, one at a time.
- Pairwise looks at the pairs with different grades:
(A, B),(A, C)and(C, B). It wantss(A) > s(B),s(A) > s(C)ands(C) > s(B). A pair with equal grades gives no signal. - Listwise scores the whole order. It asks how good the ranking is as a list, with a metric such as NDCG.
Suppose the model outputs s(A) = 0.9, s(B) = 0.2 and s(C) = 0.6. Sorting gives A, C, B, which is grades 2, 1, 0. That is the ideal order, so every pair is correct and the list is perfect.
Common mistakes
- Using a ranker's output as a probability. Ranking scores only order items. If you need calibrated probabilities, a pointwise classifier is the right tool. See calibration and multi-objective ranking.
- Splitting a query across batches or folds. Documents from one query must stay together, or the pairwise and listwise losses compare the wrong things.
- Evaluating with one global number across queries. Measure per query, then average.
Check yourself
- Why does a pair with equal grades teach the pairwise model nothing? Answer: there is no correct order to prefer, so the loss has nothing to push.
- Which approach optimizes the whole list at once? Answer: listwise.
Key takeaways
- Learning to rank trains a model to order the candidates for each query.
- Pointwise, pairwise and listwise differ in the loss. Serving is the same: sort by score.
- Keep each query's documents together during training and evaluation.
- Scores from a ranker are for ordering, not for probabilities.
Further learning
Next, learn how the ranking is measured in ranking metrics.
Related
Ranking metrics
NDCG, MRR, MAP and recall at k measure how good an order is. Work one NDCG example by hand and the formula stops being abstract.
LambdaMART
LambdaMART combines boosted trees with pairwise gradients scaled by how much a swap would change NDCG. It is the workhorse of listwise ranking.
Ranking and learning to rank
Second-stage ranking for search, feeds and ads. Learning to rank, ranking metrics, LambdaMART, CTR models, position bias and calibration. Start here for the map.