acceptodds
Under review as a conference paper at ICLR 2027

When the Rationale Disagrees with the Log: Diagnosing and Repairing Chain-of-Thought for Sequential Recommendation

Abstract

Chain-of-thought reliably helps in mathematics, code and logic yet is repeatedly reported to be neutral or harmful in recommendation. We show that which of the two occurs is decided by a single quantity measurable before any reasoning supervision is trained, and we derive it in closed form. Because a decoder-only model emits the rationale and the answer through the same readout, the reasoning and ranking gradients satisfy an exact identity whose sign is governed by one indicator: whether the item the rationale argues for is the item that was logged. In mathematics the indicator equals one by construction and the two gradients reinforce. In non-verifiable, multi-modal prediction, where many continuations are legitimate but only one is recorded, it collapses to the overlap between the true continuation law and the teacher's; the gradients then provably conflict. Three further measurements confirm the mechanism: the reasoning-ranking cosine turns negative exactly when predicted, deepens toward the layers at which the model commits to an answer, and leaves reasoning states nearly answer-agnostic under a linear probe. The identity is also constructive, naming token-level rationale imitation as the sole negative driver and predicting that removing it reverses the sign. It does: conflict-aware training (Care) and outcome-reward reasoning restore answer decodability and turn chain-of-thought from a penalty into a gain over direct supervised fine-tuning. A corollary for practice is that the prevailing remedy, replacing discrete traces with latent ones, relocates the imitation target rather than removing it, which is why a change confined to the loss outperforms it. Reasoning is not the wrong tool for recommendation; imitating rationales is.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Recomendation Systems, Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_bKu7OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies why chain-of-thought (CoT) supervision can hurt sequential recommendation. It attributes the degradation to disagreement between the item favored by a teacher rationale and the logged next item, which induces conflict between reasoning and ranking gradients. The proposed remedies include annealing the reasoning loss, projecting conflicting reasoning gradients, and outcome-based preference optimization. Experiments across recommendation datasets and model backbones provide evidence of gradient conflict and show that some interventions improve ranking performance. The research question is relevant, and the empirical diagnosis is potentially useful. However, several central theoretical claims are not established by the presented analysis, and an inconsistency in the evaluation protocol makes the main results difficult to interpret.

Critical strengths
  1. Relevant research question. The paper usefully distinguishes the potential benefit of reasoning from the benefit of imitating teacher-generated rationales. Understanding this distinction matters for applying language models to recommendation.
  2. Multiple diagnostic perspectives. The experiments examine gradient alignment over training, layerwise conflict, and answer decodability, rather than relying exclusively on final ranking metrics. Replication across datasets and backbone families strengthens the empirical observation that rationale supervision can be detrimental.
  3. Practical interventions with promising results. The proposed loss modifications are straightforward, and some improvements are substantial. For example, CARE-GPROJ improves Qwen-3B's ML-1M NDCG@10 from .296 to .427 in Table 1.
  4. Controlled candidate sets. Using identical candidate sets across methods helps isolate reranking behavior. The oracle-complete setting is useful for diagnosis, although its conclusions should remain scoped to that setting.
Critical weaknesses
  1. The agreement rate alone does not determine the sign of gradient conflict. Equation 4 is valid under the shared-logit reduction, but when y≠my\neq m, its sign still depends on the student distribution:
⟨ga,gr⟩=∥p∥2−py−pm.\langle g_a,g_r\rangle=\|p\|^2-p_y-p_m.

For the same disagreeing targets y=1,m=2y=1,m=2, this quantity is positive for p=(0.1,0.1,0.8)p=(0.1,0.1,0.8) and negative for p=(0.45,0.45,0.1)p=(0.45,0.45,0.1). Equation 5 likewise contains student-dependent terms beyond the agreement probability. Consequently, the claim that a single quantity measurable before student training predicts the sign requires additional conditions. Sharing an unembedding matrix also does not, by itself, justify treating reasoning and answer positions as sharing one logit vector.

  1. The full Transformer extension contains an invalid bound. In Appendix H.6, the proof bounds the distribution overlap by 2δ2+δc02\delta^2+\delta c_0. Consider VA={y}V_A=\{y\}, VR={z}V_R=\{z\}, pa=(0.9,0.1)p_a=(0.9,0.1), and pr=(0.1,0.9)p_r=(0.1,0.9), with δ=0.1\delta=0.1 and η=0\eta=0. These satisfy the stated support-mass and cross-token conditions, but the overlap is .18, exceeding the claimed bound of .02. The resulting bound on FtF_t also fails. This does not rule out a corrected conditional sign result, but the current proof does not establish the claimed extension.

  2. The outcome-reward alignment guarantee is unsupported. Proposition 18 asserts that optimizing the ranking reward guarantees nonnegative alignment with the answer-loss gradient. NDCG and gold-label negative log-likelihood are not monotonically equivalent: increasing the gold-label probability can still worsen its rank if another candidate's probability increases further. Therefore, their use as measures of ranking quality does not establish gradient alignment, particularly for offline DPO on sampled preference pairs. Removing explicit teacher imitation also does not establish that the previously defined δKL\delta_{\mathrm{KL}} tends to zero. These statements should be proved under explicit assumptions or presented as empirical hypotheses.

Questions for authors
  1. What additional conditions make rationale–log agreement sufficient to predict the sign of conflict, independently of the student distribution? Can the paper's central predictive claim be revised accordingly?

  2. Can you correct the overlap bound in Appendix H.6 and report whether the assumptions required by the corrected result hold empirically?

  3. How can Lemma 2 be reconciled with the fact that the defined NO-COT model predicts from xx, whereas the common risk in the lemma conditions on (x,z)(x,z)?

  4. Do the Table 1 CoT results use generated rationales at inference? Why does Table 15 describe a different protocol while reporting the same ranking values?

  5. Can you provide matched-budget NO-COT outcome training and rationale removal/replacement controls? These would help distinguish improvements due to useful reasoning from improvements due to the training objective or initialization.

Reviewer_Df2WOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors investigate why CoT supervision can hurt sequential recommendation. Their proposed explanation is that a teacher generated rationale may favor a different item from the logged next interaction, and so creating conflict between two gradients. They derive a gradients and examine possible gradient alignment during training, its distribution across layers, and answer decodability from reasoning steps. They propose method CARE, which either anneals the rationale loss or projects out its conflicting gradient component, and also evaluate outcome-based reasoning supervision. Experiments on 4 recommendation datasets show consistent top 1 degradation from standard CoT supervision and improvements from the proposed methods.

Critical strengths

The paper investigates a useful distinction between learning to imitate a rationale and learning to predict the next interaction. The gradient measurements and representation probes provide concrete evidence for the proposed explanation.

There are useful experimental controls: direct prediction, standard CoT supervision, alternative reasoning modalities, and different initializations for outcome-based training. Experiments include Qwen and Llama models making observed penalty more substantive.

The proposed interventions are relatively simple and produce meaningful improvements in several setups: in particular, CARE-GPROJ improves NDCG@10 over direct supervision on all four datasets with Qwen2.5-3B (Table 1).

Critical weaknesses

The main theoretical claim is not completely supported. Equation 5 shows that gradient conflict depends not on whether the teacher agrees with the logged item but also on the students predictions. For example if the teacher follows the next-item distribution the expected gradient inner product is nonnegative even when many next items are possible. There also seems to be an incorrect bound in the proof of Theorem 16 (Appendix H.6).

Also BLIND generates long reasoning traces using the candidate list while HINDSIGHT generates short preference profiles without it. Therefore the observed differences could also come from the length, format or content of the reasoning. So the comparison between BLIND and HINDSIGHT does not fully isolate the effect of agreement with the answer. And also high teacher self-consistency does not necessarily mean that multiple recommendations are valid since the teacher could just be consistently wrong.

The authors say that reasoning improves recommendation when rationale imitation is removed. The experiments do not fully separate the benefits of reasoning from those of additional training. Appendix A talks about controls for training, inference computation and reward-only effects but these results are not clearly shown in Table 1. A comparison, with a direct-answer model trained using the outcome reward would help back up this claim.

Questions for authors
  1. Can the overlap bound and resulting sign guarantee in Theorem 16 be corrected? Under what additional conditions does rationale–log agreement predict conflict, given the nonnegative expectation when (\tilde q=q)?
  2. Does the BLIND–HINDSIGHT difference persist when candidate information, rationale format, and length are matched? And what distinguishes intrinsic ambiguity from a consistently mistaken teacher?
  3. Where are the additional controls described in Appendix A? How do the proposed methods compare with outcome-trained direct prediction, and how much of the improvement remains when generated rationales are removed or replaced?
3 comments

​We thank the reviewer for the precise comments. We address each point below.

1. Agreement, the student, and Theorem 16 (W1, Q1)

​ With q~=q\tilde q=q, Eq. 5 equals ∥p−q∥2≥0\|p-q\|^2\ge0, which is consistent with our claim: conflict requires the teacher to disagree with the log. In general, at a fixed prompt, ​ Ey,m⟨ga,gr⟩=⟨p−q,  p−q~⟩=∥p−q+q~2∥2−14∥q−q~∥2,\mathbb E_{y,m}\langle g_a,g_r\rangle=\langle p-q,\;p-\tilde q\rangle=\Big\|p-\tfrac{q+\tilde q}{2}\Big\|^2-\tfrac14\|q-\tilde q\|^2, ​ so conflict occurs exactly when the student lies in the ball with diameter [q,q~][q,\tilde q], which is empty when q~=q\tilde q=q. The condition that removes the student dependence is training itself. Under Assumption 3, the CoT loss −log⁡py−λlog⁡pm-\log p_y-\lambda\log p_m is minimized at p⋆=q+λq~1+λp^\star=\frac{q+\lambda\tilde q}{1+\lambda}, inside that ball, where the inner product is −λ(1+λ)2∥q−q~∥2<0-\frac{\lambda}{(1+\lambda)^2}\|q-\tilde q\|^2<0 for any q~≠q\tilde q\neq q. At a diffuse initialization the sign is that of Pr⁡(y=m)−1/K\Pr(y=m)-1/K, positive here, as the epoch-0 cosines show. The deciding quantity is the discrepancy E∥q−q~∥2\mathbb E\|q-\tilde q\|^2. Before training it is bounded below by 1/K+Pr⁡(m=m′)−2Pr⁡(y=m)1/K+\Pr(m=m')-2\Pr(y=m), which is 0.35–0.44 on our four datasets (Table 12). Prop. 11 states this explicitly. ​ For H.6, a bound that uses only (A1) and (A3) gives Ft≤−pa,zt(1−pr,t,zt)−pr,t,y(1−pa,y)+2η+c0F_t\le-p_{a,z_t}(1-p_{r,t,z_t})-p_{r,t,y}(1-p_{a,y})+2\eta+c_0, so the sign holds when cross-vocabulary mass exceeds 2η+c02\eta+c_0. Its size is second order in that mass, so the readout term is not the dominant one. ​

2. The role of BLIND and HINDSIGHT (W2, Q2)

We do not compare BLIND with HINDSIGHT, and no claim in the paper rests on the difference between them. Our claim is that imitating a teacher's CoT hurts whichever trace is used, and that each trace fails for a different, identifiable reason. Theorem 6 says CoT beats direct prediction only if Icompπ>ByδKL/2I^\pi_{\text{comp}}>B_y\sqrt{\delta_{\mathrm{KL}}/2}. The two teachers sit at the two ends of the agreement axis, and each violates this condition through a different term:

  • BLIND fails through IcompI_{\text{comp}}. Its trace is written from the history and the candidate list only, the same input the student has at test time, with no leakage of the answer, so it is the learnable trace: its δKL\delta_{\mathrm{KL}} is the smaller of the two. But its rationale agrees with the logged item only 7–13% of the time, so imitating it conflicts with ranking: the gradient cosine is negative in all eight cells, and by Lemma 2 Icompπ<0I^\pi_{\text{comp}}<0. The condition fails on its left-hand side.
  • HINDSIGHT fails through δKL\delta_{\mathrm{KL}}. Its trace is written with the answer in view, so it points to the logged item (Pr⁡(y=m)→1\Pr(y=m)\to1) and imitating it no longer conflicts with ranking (cosine near zero). But in reranking the label is never available at test time, so the student cannot learn to produce such a trace from its input: δKL\delta_{\mathrm{KL}} is very large. The condition fails on its right-hand side. ​ Length, format and content therefore do not threaten the claim: we never attribute the gap between the two regimes to agreement. What the pair shows is that neither choice of teacher escapes Theorem 6, which is why every later experiment uses BLIND, the only deployable regime. Where agreement itself must be isolated, the right comparison is within BLIND, with teacher, prompt, format and length fixed: training on equal-size subsets with m=ym=y and m≠ym\neq y gives a cosine of [X] vs [X] and NDCG@10 of [X] vs [X]. ​ On ambiguity: the result does not need the alternatives to be valid. The conflict is set by ∥q−q~∥\|q-\tilde q\| whatever its cause, valid alternative continuations or a consistently mistaken teacher; self-consistency was reported only to show that the teacher is not random. To separate the two causes, we measure how often mm appears among the user's later interactions, against a random candidate: a mistaken teacher's mm should be at chance, a valid alternative above it: [X] vs [X]. ​

3. Reasoning or additional training? (W3)

​

As stated in the paper (Section 5, App. J), CARE adds no training beyond SFT-CoT: it uses the same teacher traces, data, epochs and optimizer, and changes only the loss. ANNEAL changes a scalar weight after the switch epoch and is computationally free; CARE and SFT-CoT are therefore matched in training, and CARE-GProj improves over both SFT-CoT and direct supervision on all four datasets at 3B. Latent reasoning methods are the opposite case. CoCoNuT, for example, trains on the same teacher traces and then adds a multi-stage curriculum on top, so it trains more than SFT-CoT, yet it only tracks direct supervision. Additional training is therefore neither necessary nor sufficient for the gain: the gain appears without it (CARE) and does not appear with it (CoCoNuT). For outcome-reward training, which does add rollouts, the matched controls are in point 4. ​

4. Controls for training and reward effects (Q3)

​

The controls address three alternative explanations. We answer each below; the camera-ready will report all of them in Table 1 and App. A, at both backbones. ​

  • The reward alone. Table 1 already contains this control: the same DPO, reward and rollout budget, started from SFT-CoT instead of CARE, loses 4.8–10.8 NDCG@10 in all eight cells. The reward is identical and only the trace it acts on differs, so the reward does not produce the gain. ​
  • Outcome-trained direct prediction. In our pipeline the reward can only act through the trace. Outcome-DPO compares sampled rationales: each completion ends at ###, and the answer is never generated but read from the 20 letter logits at that position (App. D). A direct-answer model emits ### immediately, so its six rollouts are identical, every preference pair is empty, and the DPO gradient is exactly zero: outcome-DPO started from NO-COT returns NO-COT. This is the paper's point rather than a gap in it. An outcome reward needs a trace to shape, and the question is which trace, which the control above answers. For a direct predictor, the outcome reward and the SFT loss also push on the same quantity, the score of the logged item, so SFT NO-COT is already trained on the outcome. In the camera-ready we will report the pair statistics of outcome-DPO from NO-COT and, as a non-degenerate variant, NO-COT trained with the same reward applied to its letter distribution. ​
  • Extra test-time computation. Pause tokens (Goyal et al., 2024) give the NO-COT model six learnable positions to compute on before answering, with no written trace. App. S.4 (Table 17) reports them at 3B; we have now run all eight cells (NDCG@10): ​ | | ML-1M | Beauty | Toys | Sports | |---|---|---|---|---| | Qwen 3B SFT NO-COT | .296 | .369 | .373 | .366 | | Qwen 3B + pause tokens | .359 | .384 | .380 | .370 | | Qwen 3B Outcome-DPO (init: CARE-GProj) | .415 | .442 | .445 | .449 | | 7Qwen B SFT NO-COT | .494 | .440 | .433 | .426 | | Qwen 7B + pause tokens | .498 | .437 | .425 | .420 | |Qwen 7B Outcome-DPO (init: CARE-Anneal) | .500 | .470 | .447 | .476 | ​

At 7B, pause tokens stay within 0.8 of NO-COT. At 3B they help (+6.3 on ML-1M), and Outcome-DPO adds another 5.6–7.9 on top in every cell. The only cell where Outcome-DPO does not clear pause tokens clearly is 7B ML-1M, where no method gains more than 0.7 over NO-COT. Extra computation without a trace does not explain the gain.

Thank you for the detailed response. I appreciate the effort, and I think the training-control concern is now much better addressed: CARE uses the same traces, data, epochs and optimizer as SFT-CoT and changes only the loss, and the pause-token control is a useful check that the gain is not just extra computation. The explanation of why outcome-DPO from NO-COT is degenerate is also reasonable.

I still have two reservations. First, the correction to the Theorem 16 bound and the resulting sign guarantee is described but the actual bound is not visible in the response, so I cannot verify that H.6 is fixed. Second, the matched comparison within BLIND that would isolate agreement is promised but the numbers are shown as placeholders, and the same holds for separating intrinsic ambiguity from a consistently mistaken teacher. These were the core of my soundness concern.

Overall, the response is helpful and moves in the right direction, but I do not think the theoretical correction and the agreement-isolation results are yet demonstrated. I therefore keep my rating at 2 (weak rejection), while encouraging the authors to include the corrected bound, the matched-agreement numbers, and the non-degenerate outcome-trained direct-prediction control in the revision. If these are provided and hold up, I would be open to reconsidering.

Reviewer_zhC7OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the problem that CoT supervision may degrade the performance of LLMs in sequential recommendation tasks. The authors analyze the gradient relationship between the CoT reasoning loss and the answer-ranking loss from an optimization perspective. They argue that disagreement between the recommendation implied by teacher-generated reasoning traces and the ground-truth interaction logs is a major source of gradient conflict. Based on this analysis, the authors formulate three testable predictions and empirically investigate when gradient conflicts emerge during training, how they are distributed across network layers, and how much answer information is encoded in reasoning representations.

To mitigate these issues, the authors propose the CARE framework, which consists of two training strategies: ANNEAL and GPROJ. ANNEAL gradually reduces the weight of the CoT loss, allowing training to focus more on the ranking objective in later stages. GPROJ removes the component of the CoT loss gradient that conflicts with the ranking objective through gradient projection. Additionally, the authors introduce Outcome-Reward Reasoning, which constructs preference data using ranking-quality rewards and performs DPO, thereby avoiding direct imitation of teacher-generated reasoning traces during this stage.

The authors conduct experiments on four recommendation datasets using multiple LLM backbones. The results show that conventional CoT supervision generally underperforms direct answer supervision, while CARE and Outcome-DPO improve recommendation performance across multiple experimental settings and outperform direct supervision baselines in several cases.

Critical strengths

An interesting research problem and a potentially useful analytical perspective. The paper investigates the performance degradation associated with CoT supervision in sequential recommendation, which is a relevant research problem. By examining the gradient relationship between reasoning and ranking objectives, the authors provide a potentially useful perspective on this phenomenon.

Critical weaknesses
  1. Insufficient comparison with related work and unclear necessity of explicit CoT. The paper lacks sufficient discussion of existing LLM-based recommendation methods and experimental comparisons with representative approaches in this field. For example, prior studies such as LatentR³ (ICLR 2026) have achieved promising recommendation performance without relying on explicit textual CoT. Therefore, the paper does not sufficiently establish the advantages of the proposed methods over existing approaches. Furthermore, given that the paper identifies significant conflicts between teacher-generated CoT supervision and ground-truth ranking supervision, is explicit CoT still necessary for this task? The authors need to provide stronger evidence demonstrating the additional value of explicit CoT compared to alternative approaches.

  2. Limited methodological novelty. The proposed CARE method mitigates gradient conflicts between CoT supervision and ranking supervision through loss-weight annealing and gradient projection. However, relatively mature solutions already exist for addressing gradient conflicts in multi-objective optimization. In addition, Outcome-DPO primarily employs the existing DPO algorithm for optimization. Therefore, the proposed training strategies are largely based on established techniques and offer limited methodological novelty.

Questions for authors
  1. Necessity of explicit CoT and comparison with related work. Could the authors provide experimental comparisons with related methods such as LatentR³, along with appropriate ablation studies, to demonstrate the performance advantages of the proposed approach and the additional value of explicit CoT in sequential recommendation tasks?
1 comment

We thank the reviewer for the comments.

1. LatentR³ and the value of explicit CoT (W1, Q1)

LatentR³ solves a different task. It generates the next item's title over the full catalog, is rewarded by that title's perplexity, and never sees a candidate list. We rerank K=20 candidates that always include the ground truth among plausible near-misses, a harder setting where reasoning must separate items that are all reasonable continuations. We did not expect it to help: (i) much of its benefit is steering generation through a large catalog, which the shortlist already provides; (ii) its latent tokens are trained only from the logged item, the signal direct SFT already uses, so they escape the penalty but add no information for separating candidates; (iii) its reward is shaped for generating a title, not for choosing one label.

Our claim is about supervision, not the reasoning space. We do not argue that textual reasoning beats latent reasoning. Lemma 4 gives an exact identity, ⟨g_a, g_r⟩ = ‖p‖² − p_y − p_m + 1{y=m}, whose negative driver is the token-level imitation term; nothing in it privileges discrete tokens. Prediction (Table 5): methods that change the space but keep an imitation curriculum should sit near direct supervision. CoCoNuT (continuous space, imitated curriculum) tracks NO-COT rather than beating it (7B N@10: Beauty .439 vs .440; ML-1M .501 vs .494); Soft Thinking is far below (Beauty .264).

As requested, we added LatentR³ and pause tokens (Goyal et al., 2024), a CoT-free compute control (Appendix S.4, Table 17). NDCG@10, Qwen2.5-3B:

ML-1MBeautyToysSports
SFT NO-COT.296.369.373.366
Pause tokens.359.384.380.370
LatentR³.231.314.226.319
CARE-GProj (ours).427.442.443.446

Is explicit CoT necessary? We claim something narrower: the repeated finding that CoT fails in recommendation is an artifact of how the trace is supervised, not evidence that deliberation is useless. We train CoT exactly as for mathematics (imitate a written rationale, then answer); the same objective conflicts here because the item the rationale argues for is often not the logged one (Lemma 4). Acting on the prevailing inference, "reasoning does not transfer to recommendation", would abandon a mechanism that works once supervised correctly. The explicit trace adds:

  • Measurement. The conflict is observable only because the trace is readable: agreement is 0.07–0.13 vs 0.05 chance. Probing CoCoNuT's latent thought recovers only 0.05–0.14 of the committed item, vs a 0.67–0.76 ceiling (Table 13).
  • Auditability. Without the imitation target, the answer becomes decodable from the trace (0.41–0.69, up from 0.11–0.19), an inspectable explanation latent methods cannot offer.
  • Empirics. CARE beats direct supervision and SASRec, HSTU and AlphaRec in NDCG@1 on all four datasets, and beats pause tokens, so the gain is not extra compute. On ML-1M, CoCoNuT at 7B is within noise of our best method.
  • Cost. CoCoNuT starts from CoT supervision and adds a curriculum costing about as much as our whole method; LaDiR (Kang et al., 2026) did not improve over it. CARE costs the same as SFT-CoT with zero added inference cost (ANNEAL adds no training cost at all).

Table 1 compares 13 baselines on identical shortlists. The camera-ready will add a related-work section on LLM rerankers and latent-reasoning recommenders, and run LatentR³ and pause tokens on all backbones.

2. Methodological novelty (W2)

Our main contribution is theoretical, and to our knowledge this analysis has not been done before for CoT in recommendation. First, Theorem 6 gives a sufficient condition for CoT to beat direct prediction, I_comp > B_y·√(KL/2), and separates what a trace can bring: true informativeness (information about the answer absent from the input) and computational informativeness (help reaching an answer the input already determines); a trace generated from the prompt alone has zero true informativeness (Lemma 1), so any gain must be computational. Second, this explains the gap across domains: mathematics and code have a determinate answer and checkable steps, giving the trace a canonical target; recommendation has neither, so every trace is teacher-synthesized. Third, we show theoretically why CoT fails in recommendation: by the exact identity of Lemma 4, the rationale and answer gradients conflict precisely when the item the rationale argues for differs from the logged item, an event that is rare in verifiable tasks but frequent here (agreement 0.07–0.13 vs 0.05 chance). Given this diagnosis, we deliberately chose simple remedies (annealing, gradient projection, standard DPO) to avoid needless complexity, so that the gain can be attributed to the term the analysis removes. The same DPO started from SFT-CoT loses 4.8–10.8 NDCG@10 in all eight cells, so DPO is not the source.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.