LambdaMART

LambdaMART is the listwise ranker most often used in industry. It is built from two parts. MART is gradient-boosted regression trees, which add many small trees one at a time. LambdaRank is the training signal: a pairwise gradient scaled by how much swapping that pair would change NDCG.

The combined idea is simple. If swapping two documents barely changes NDCG, the model should spend little effort on that pair. If swapping would move a highly relevant document from position 10 to position 1, the model should push hard.

Walking through a swap

Take two documents for one query. One has grade 2 and the other grade 0.

If the grade-0 document is ranked first, the DCG is 0 + 3 / log2(3) = 1.893. The ideal DCG is 3 / log2(2) = 3. So NDCG is 1.893 / 3 = 0.631.

Swapping them gives DCG 3, and NDCG 1. The swap changes NDCG by 0.369, so the model pushes hard.

Now take two grade-0 documents. Swapping them changes nothing, so the change in NDCG is 0, and LambdaRank gives no gradient for that pair.

A swap near the top moves more NDCG than the same swap lower down, because the discount log2(i + 1) is smallest at the top.

Training it

In practice you use XGBoost with rank:ndcg or LightGBM with lambdarank. Each query's rows must stay together, which is what the qid argument does.

import numpy as np
import xgboost as xgb

rng = np.random.default_rng(0)
n_queries, docs_per_query = 50, 10
X = rng.normal(size=(n_queries * docs_per_query, 3))
qid = np.repeat(np.arange(n_queries), docs_per_query)   # rows grouped by query
y = np.clip(np.round(X[:, 0] + 0.5 * rng.normal(size=len(X)) + 1), 0, 3).astype(int)  # grades 0-3

ranker = xgb.XGBRanker(
    objective="rank:ndcg",
    tree_method="hist",
    n_estimators=100,
    learning_rate=0.05,
)
ranker.fit(X, y, qid=qid)

scores = ranker.predict(X[:docs_per_query])        # score the first query's documents
order = np.argsort(-scores)
print("grades in predicted order:", y[:docs_per_query][order])

Output:

grades in predicted order: [3 2 2 2 1 2 0 0 0 0]

The data here is synthetic. The grades are high at the top and zero at the bottom, so the ranker learned the order. The code needs xgboost and scikit-learn installed.

Neural rankers exist, but boosted trees still win on many tabular feature sets with graded labels.

Common mistakes

• Splitting a query's rows. The pairwise gradients then compare documents from different queries, and the signal is meaningless.
• Expecting probabilities. LambdaMART scores only order documents.
• Optimizing NDCG and reporting AUC. Report the metric you optimize, at the cutoff the UI shows.

Check yourself

1. Why does a swap at rank 1 matter more than the same swap at rank 10? Answer: the discount log2(i + 1) is smaller at the top, so the same gain counts for more there.
2. What does qid do? Answer: it marks which rows belong to the same query, so pairs are only compared within a query.

Key takeaways

• LambdaMART is boosted trees trained with pairwise gradients scaled by the NDCG change of each swap.
• Pairs that do not change NDCG get no gradient. Swaps near the top get the most.
• Keep each query's rows together with qid or the equivalent.
• Scores order the list. They are not calibrated probabilities.

Further learning

Next, see how click-through models estimate click probabilities for ads and feeds in CTR prediction.