- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
R2Rank is an LLM-based recommender in three parts.
- It generates a structured rationale independently for each user–candidate pair (Eq. 1).
- It maps the last-token hidden state of that rationale to a scalar score with an MLP head (Eq. 2).
- It trains the backbone and head end to end with a listwise reward. A Plackett–Luce prefix of length 10 is sampled from the scores, and its NDCG@10 serves as a shared return for two terms: a REINFORCE term on the PL log-probability, which flows through the scores into the backbone, and a clipped PPO term on the rationale tokens with a zero baseline (Eqs. 4–7, Alg. 1).
A self-check SFT warm start distilled from DeepSeek-R1 comes first (Sec. 3.4, App. A). The evaluation uses Qwen2.5-1.5B/3B on three Amazon datasets and a proprietary advertising dataset. Each user gets a fixed candidate set of one positive and 19 uniformly random negatives. R2Rank-3B beats all trainable same-backbone LLM baselines on NDCG@10 across the four datasets, with the largest margin on Industrial (0.818 vs 0.560 for LatentR3, Table 2). The paper adds ablations (Tables 3–6), rationale substitution and transfer with frozen checkpoints (Tables 7–8), and further analyses (App. D).
Critical strengths
- A sensible design that ties candidate-wise reasoning to a list-level objective. Each candidate is assessed in its own prompt, which removes candidate presentation order and the position bias of listwise LLM rankers (§3.2, App. D.4). The rationale's last-token hidden state is then scored, and the resulting scores are trained end to end with a Plackett–Luce surrogate of NDCG. The two gradient paths (PL scores into backbone and head; PPO on rationale tokens) are cleanly derived and clearly described (Eq. 6, Alg. 1). Within the evaluated protocol, R2Rank outperforms all same-backbone LLM baselines on NDCG@10 (Table 2), although Weakness 2 limits how far this comparison can be read.
- Rationale-content controls that go beyond illustrative examples. With frozen checkpoints and fixed final answers, original rationales outperform rationales shuffled within the same user on all four datasets. On three of the four, shuffled rationales even fall below answer-only completions (Table 7). R2Rank rationales also improve DeepSeek-R1 from 0.512 to 0.552 (Table 8). This is non-trivial evidence that the scorer uses candidate-specific rationale content, not just extra tokens, and the paper is careful not to overclaim faithfulness.
Critical weaknesses
- The central claim, optimizing the reasoning process itself for ranking utility, is not isolated. The model trained without CoT already beats every baseline (Musical 0.545 vs 0.512; Industrial 0.758 vs 0.560; Table 5 vs Table 2), and on Musical plain CoT (0.494) is worse than no CoT (Table 6). No ablation removes the PPO term while keeping the PL score path. With a single positive, the PL/REINFORCE score path is also a high-variance surrogate for a supervised listwise loss (softmax/ListMLE), which is never compared. The gains may therefore come from fine-tuning a cross-encoder-style scorer rather than from "reasoning to rank".
- The evaluation protocol favors the proposed method.
- Negatives are sampled uniformly from the whole catalogue (App. A.3), so most are unrelated to the user. Item text includes popularity cues (rating, number of buyers; Fig. 6), yet there is no popularity baseline and no evaluation on retriever-selected candidates (e.g., SASRec top-20).
- The conventional baselines are at or near the random level (NDCG@10 ≈ 0.227 for 1 of 20) on Industrial and Movies & TV (Table 2). This is likely because test users and most Industrial positives are unseen in training, which ID-based models cannot handle.
- The LLM baselines were designed for other tasks: item generation (LatentR3), full-catalogue ranking (R2EC) and binary classification (TALLRec). How they were adapted to score 20 candidates is not described. R2Rank alone is trained directly on this protocol, and its Industrial gain over LatentR3 concentrates on unseen items (+0.287 vs 11.3% relative on warm items; Table 10).
- Inference cost and variance are not reported. R2Rank generates one rationale per candidate (K = 20 generations per user, growing linearly with K), but latency and tokens per user are never reported, although the paper emphasizes the industrial setting. All results also appear to be single runs, while shuffling the history order alone gives std 0.085 (Table 11).
Questions for authors
- Can you evaluate R2Rank and LatentR3 on retriever-selected candidates (e.g., the top-20 from SASRec) on at least one dataset, and report a popularity baseline (e.g., ranking by number of buyers or training frequency)? Can you also report results restricted to warm items (seen in training) on each dataset?
- How were LatentR3 (an item-generation model), R2EC (a full-ranking model) and TALLRec (a binary classifier) adapted to score 20 fixed candidates? Were they trained on the same candidate sets as R2Rank? How does LightGCN obtain embeddings for user-disjoint test users?
- What is the inference cost per user (latency, generated tokens, rationale length) compared with the LLM baselines and a single listwise call, and how does it scale with K?
2 comments
Responses to Reviewer x8T2 [2/2]
For user-disjoint test users, LightGCN constructs user representations by aggregating the embeddings of items in each user's observed history. Candidate items with no training interactions receive a score of zero in our implementation, so these candidates share the same fallback score. SASRec and BERT4Rec encode the observed item history to represent a user. User-disjoint evaluation and item cold start therefore concern different sources of information: a new user's visible history can provide preference evidence, while zero-interaction items lack observed collaborative evidence. C2 separately reports results on seen positives to characterize the item-frequency effect.
W3 and Q3. Inference cost.
Common Response C3 reports the existing serving benchmark on one H20 in bf16 with eight concurrent users. R2Rank's P50 latency is 5.80 seconds at K=20 and 12.70 seconds at K=50, compared with 1.92 and 4.35 seconds for LatentR³. Its P95 latency rises from 7.90 to 17.10 seconds. Thus, increasing K from 20 to 50 increases R2Rank's median waiting time by 2.19 times in this benchmark.
W3. Training seeds and significance testing.
The main results and additional ablations each use one training seed. The significance markers in the manuscript use two-sided paired Wilcoxon signed-rank tests on per-test-instance NDCG values, pairing the methods on the same evaluation instance. The thresholds are for †/‡/§, respectively. These are instance-level comparisons for the evaluated checkpoints. Table 11 separately measures sensitivity to history order for a fixed checkpoint: its standard deviation of 0.085 describes performance changes under history permutations.
Responses to Reviewer x8T2 [1/2]
Thank you for your careful review and constructive feedback. Below, we provide additional results and clarifications to address your questions and concerns.
W1. Scoring architecture, CoT, and the ranking objective.
On Industrial, the no-CoT variant reaches 0.758 NDCG@10, compared with 0.560 for LatentR³ and 0.818 for Full. A substantial advantage is therefore present without explicit rationales. The 0.060 difference compares the full and no-CoT configurations; it is not an estimate of PPO's contribution. On Musical, plain CoT also underperforms no CoT (0.494 vs. 0.545), so the results do not support a general claim that generating more reasoning text improves ranking. That template comparison also changes the retained SFT examples and response lengths, so it does not isolate self-checking alone.
Common Response C1 addresses the optimization question on Musical Instruments and Industrial, retaining rationale generation and omitting recommendation-specific SFT in every configuration. Full outperforms PL-only on both datasets, supporting an additional benefit from explicitly optimizing rationale-token probabilities beyond training the scorer through PL. Full also outperforms PPO with head-only PL, supporting the contribution of ranking gradients to backbone adaptation. PL-only can still change generation indirectly through the shared backbone. These comparisons concern the RL-only setting and do not directly decompose the main SFT & RL gains discussed above.
W2 and Q1. Candidate difficulty, popularity, and warm items.
Common Response C2 compares uniform negatives with SASRec-selected K=20 candidates on the complete Musical Instruments test set. R2Rank retains an advantage over LatentR³, but the gap narrows from 0.057 under uniform sampling to 0.022. The result supports an advantage beyond random negatives in this setting, with a smaller margin under retrieval-based candidate selection.
MostPop ranks candidates by their training interaction counts and reaches 0.348 NDCG@10 under uniform sampling and 0.355 with SASRec-selected candidates, below both personalized models in both protocols. Popularity alone therefore falls short of their performance in these comparisons. This does not rule out their use of popularity cues embedded in item text.
For the requested seen-item analysis, C2 restricts evaluation to positives with at least one main-RL training occurrence. R2Rank remains ahead on all four datasets, including Industrial, where it reaches 0.866 versus 0.672 for LatentR³. The advantage therefore extends to seen positives. This subset includes all positive items with at least one training occurrence; the original Industrial “Warm” stratum included only items with more than five. The reported 68.7% zero-frequency fraction remains defined against the main RL-interaction corpus, rather than every source of SFT or pretraining exposure.
W2 and Q2. Baseline adaptation and unseen users.
All trainable baselines and R2Rank share the same main training data and training candidate sets. All methods rank the same fixed candidate sets at evaluation. All trainable LLM baselines use Qwen2.5-3B-Instruct, while retaining their method-specific training objectives. TALLRec scores each user–candidate pair using the positive-response probability normalized over its positive and negative responses. R²EC scores each candidate using its native recommendation head.
LatentR³ first derives latent reasoning from the user-history prompt and then generates an item title. Its official inference implementation uses prefix-constrained beam search and returns length-penalized sequence scores, with a default length penalty of 1.0. In our fixed-candidate evaluation, we retain the latent reasoning and score each candidate title using the corresponding length-normalized conditional log-likelihood:
s(u,i)=∣Yi∣1∑t=1∣Yi∣logPθ(Yi,t∣xu,ru,Yi,<t),
where xu is the user-history prompt, ru is the latent reasoning, and Yi is the candidate-title response sequence, retaining the original response formatting and end-of-sequence token. We evaluate all 20 candidate responses with teacher forcing and sort their scores in descending order, obtaining a complete ranking without relying on whether beam search retrieves each candidate. This adapts the generative sequence score to the fixed candidate set; the ground-truth item's negative perplexity remains the RL training reward, a separate role from inference-time candidate scoring.