- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies why chain-of-thought (CoT) supervision can hurt sequential recommendation. It attributes the degradation to disagreement between the item favored by a teacher rationale and the logged next item, which induces conflict between reasoning and ranking gradients. The proposed remedies include annealing the reasoning loss, projecting conflicting reasoning gradients, and outcome-based preference optimization. Experiments across recommendation datasets and model backbones provide evidence of gradient conflict and show that some interventions improve ranking performance. The research question is relevant, and the empirical diagnosis is potentially useful. However, several central theoretical claims are not established by the presented analysis, and an inconsistency in the evaluation protocol makes the main results difficult to interpret.
Critical strengths
- Relevant research question. The paper usefully distinguishes the potential benefit of reasoning from the benefit of imitating teacher-generated rationales. Understanding this distinction matters for applying language models to recommendation.
- Multiple diagnostic perspectives. The experiments examine gradient alignment over training, layerwise conflict, and answer decodability, rather than relying exclusively on final ranking metrics. Replication across datasets and backbone families strengthens the empirical observation that rationale supervision can be detrimental.
- Practical interventions with promising results. The proposed loss modifications are straightforward, and some improvements are substantial. For example, CARE-GPROJ improves Qwen-3B's ML-1M NDCG@10 from .296 to .427 in Table 1.
- Controlled candidate sets. Using identical candidate sets across methods helps isolate reranking behavior. The oracle-complete setting is useful for diagnosis, although its conclusions should remain scoped to that setting.
Critical weaknesses
- The agreement rate alone does not determine the sign of gradient conflict. Equation 4 is valid under the shared-logit reduction, but when , its sign still depends on the student distribution:
For the same disagreeing targets , this quantity is positive for and negative for . Equation 5 likewise contains student-dependent terms beyond the agreement probability. Consequently, the claim that a single quantity measurable before student training predicts the sign requires additional conditions. Sharing an unembedding matrix also does not, by itself, justify treating reasoning and answer positions as sharing one logit vector.
-
The full Transformer extension contains an invalid bound. In Appendix H.6, the proof bounds the distribution overlap by . Consider , , , and , with and . These satisfy the stated support-mass and cross-token conditions, but the overlap is .18, exceeding the claimed bound of .02. The resulting bound on also fails. This does not rule out a corrected conditional sign result, but the current proof does not establish the claimed extension.
-
The outcome-reward alignment guarantee is unsupported. Proposition 18 asserts that optimizing the ranking reward guarantees nonnegative alignment with the answer-loss gradient. NDCG and gold-label negative log-likelihood are not monotonically equivalent: increasing the gold-label probability can still worsen its rank if another candidate's probability increases further. Therefore, their use as measures of ranking quality does not establish gradient alignment, particularly for offline DPO on sampled preference pairs. Removing explicit teacher imitation also does not establish that the previously defined tends to zero. These statements should be proved under explicit assumptions or presented as empirical hypotheses.
Questions for authors
-
What additional conditions make rationale–log agreement sufficient to predict the sign of conflict, independently of the student distribution? Can the paper's central predictive claim be revised accordingly?
-
Can you correct the overlap bound in Appendix H.6 and report whether the assumptions required by the corrected result hold empirically?
-
How can Lemma 2 be reconciled with the fact that the defined NO-COT model predicts from , whereas the common risk in the lemma conditions on ?
-
Do the Table 1 CoT results use generated rationales at inference? Why does Table 15 describe a different protocol while reporting the same ranking values?
-
Can you provide matched-budget NO-COT outcome training and rationale removal/replacement controls? These would help distinguish improvements due to useful reasoning from improvements due to the training objective or initialization.
We thank the reviewer for the precise comments. We address each point below.
1. Agreement, the student, and Theorem 16 (W1, Q1)
With q~=q, Eq. 5 equals ∥p−q∥2≥0, which is consistent with our claim: conflict requires the teacher to disagree with the log. In general, at a fixed prompt, Ey,m⟨ga,gr⟩=⟨p−q,p−q~⟩=p−2q+q~2−41∥q−q~∥2, so conflict occurs exactly when the student lies in the ball with diameter [q,q~], which is empty when q~=q. The condition that removes the student dependence is training itself. Under Assumption 3, the CoT loss −logpy−λlogpm is minimized at p⋆=1+λq+λq~, inside that ball, where the inner product is −(1+λ)2λ∥q−q~∥2<0 for any q~=q. At a diffuse initialization the sign is that of Pr(y=m)−1/K, positive here, as the epoch-0 cosines show. The deciding quantity is the discrepancy E∥q−q~∥2. Before training it is bounded below by 1/K+Pr(m=m′)−2Pr(y=m), which is 0.35–0.44 on our four datasets (Table 12). Prop. 11 states this explicitly. For H.6, a bound that uses only (A1) and (A3) gives Ft≤−pa,zt(1−pr,t,zt)−pr,t,y(1−pa,y)+2η+c0, so the sign holds when cross-vocabulary mass exceeds 2η+c0. Its size is second order in that mass, so the readout term is not the dominant one.
2. The role of BLIND and HINDSIGHT (W2, Q2)
We do not compare BLIND with HINDSIGHT, and no claim in the paper rests on the difference between them. Our claim is that imitating a teacher's CoT hurts whichever trace is used, and that each trace fails for a different, identifiable reason. Theorem 6 says CoT beats direct prediction only if Icompπ>ByδKL/2. The two teachers sit at the two ends of the agreement axis, and each violates this condition through a different term:
3. Reasoning or additional training? (W3)
As stated in the paper (Section 5, App. J), CARE adds no training beyond SFT-CoT: it uses the same teacher traces, data, epochs and optimizer, and changes only the loss. ANNEAL changes a scalar weight after the switch epoch and is computationally free; CARE and SFT-CoT are therefore matched in training, and CARE-GProj improves over both SFT-CoT and direct supervision on all four datasets at 3B. Latent reasoning methods are the opposite case. CoCoNuT, for example, trains on the same teacher traces and then adds a multi-stage curriculum on top, so it trains more than SFT-CoT, yet it only tracks direct supervision. Additional training is therefore neither necessary nor sufficient for the gain: the gain appears without it (CARE) and does not appear with it (CoCoNuT). For outcome-reward training, which does add rollouts, the matched controls are in point 4.