acceptodds
Under review as a conference paper at ICLR 2027

SCOReD: Student-Aware CoT Optimization for Recommendation Distillation

Abstract

Chain-of-thought (CoT) distillation in the recommendation domain is a necessary precursor to RL training, but raw teacher traces are ill-suited to this task. Large teachers approach the recommendation task with unusually high reasoning uncertainty, repeatedly rechecking their answers without revising them; supervised fine-tuning on such traces produces verbose students that never revise their initial guess. Furthermore, due to the novelty of the recommendation domain, the teacher’s reasoning traces are highly out-of-distribution for the small student LLM. We propose Student-Aware CoT Optimization for Recommendation Distillation (SCOReD), a CoT optimization framework tailored to recommendation that first parses each teacher trace into typed segments and uses the student LLM’s attention to score the importance of each segment. Then SCOReD dynamically selects a per-segment edit (KEEP / REWRITE / FUSE / PRUNE) based on the output length and comparative log probability lift of the answer given the edit as per the student. Therefore, SCOReD prunes redundant sections of the reasoning trace while preserving information-dense sections and adapts raw teacher traces to the student’s output distribution. Training on SCOReD-optimized CoTs provides a cleaner learning signal to the student model and improves over baseline SFT by 2.88% NDCG and 2.96% Recall@3, while producing shorter reasoning traces.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Recommendation, Chain-of-Thought, Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_K6drOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies chain-of-thought distillation for LLM-based candidate reranking (Amazon Reviews 2023; 10 candidates with 3 positives and 7 uniformly sampled negatives). It observes that Gemma-4-26B teacher traces are dominated by verification and intermediate-ranking loops that rarely change the answer (§6.1, Tables 3–4). It proposes SCOReD, which works in three steps. (i) An LLM segments each trace into six typed stages. (ii) Segments are bucketed by attention from the student's </think> token. (iii) Each segment is edited with KEEP/REWRITE/FUSE/PRUNE, choosing the edit by a reward that combines the student's log-probability of the teacher's answer with length and perplexity penalties (Eq. 1–2). Qwen3 students (0.6B/1.7B/4B) are then fine-tuned on the edited traces. On Beauty, NDCG rises over raw-trace SFT (0.773→0.801 for 0.6B, 0.782→0.802 for 1.7B). Traces are about 42–44% shorter and parse errors drop (Table 1). Gains over raw SFT hold on three other domains (Table 2) and shrink with student size (Table 8). DAPO and OPSD after SFT bring no further gains (§6.3).

Critical strengths
  1. A well-motivated idea: edit away what the student does not need before distilling. §6.1 shows why raw teacher traces are poor distillation targets in recommendation. Verification and intermediate-ranking segments make up 63.3% of segments, and 92.89% of refinement loops leave the ranking unchanged (Tables 3–4). Turning this diagnosis into segment-level editing (KEEP/REWRITE/FUSE/PRUNE) rather than token pruning or whole-trace summarization is a sensible and reusable idea for CoT distillation.
  2. Consistent practical gains over raw-trace SFT. SCOReD improves NDCG over raw-trace SFT in all eight domain × size settings (Table 2) and on 4B (Table 8). It also shortens generated traces by 42–50% and lowers invalid-output rates (e.g., 3.51% → 0.40% for 0.6B, Table 1). For latency-constrained recommenders, a shorter and more format-reliable student is a meaningful outcome.
Critical weaknesses
  1. The contribution of the two student-aware components, attention scoring and the reward, is not established. The only ablation sweeps τ_low (Tables 5/7), which leaves both components in place throughout.
  • Attention scoring. Its role is unclear. Attention from the pre-SFT student's </think> token only sorts segments into buckets. It is justified by an observation borrowed from Lan et al. (2026), with no analysis of what it captures, and the layers and heads used are not specified. With τ_high = 0.05, only the top 5% of segments can be kept, and MEDIUM segments are always rewritten. No variant replaces it with random or teacher-based bucketing.
  • Reward. α = 0.005 and β = 0.1 are fixed. No result removes the length penalty, the perplexity penalty or the log P term, so their individual contributions are unknown. At τ_low = 0, the reward only chooses KEEP vs REWRITE for the top 5%. Yet this setting already moves NDCG from 0.773 to 0.793 (Table 5). Much of the gain may therefore come from having Gemma rewrite nearly every segment, not from student-aware selection.
  • Baselines. A*-Thought matches SCOReD's NDCG (0.799 vs 0.801; 0.802 vs 0.802), has higher Recall@3 and produces shorter traces (0.85K vs 1.33K tokens; Table 1). If invalid rankings score zero NDCG, the NDCG on valid outputs only is nearly identical for raw-trace SFT and SCOReD (about 0.801 vs 0.804 for 0.6B, and 0.805 vs 0.805 for 1.7B, from Table 1). Much of the gain may then reflect output validity rather than better ranking.
  1. The evidence for better reasoning is weak, and several reported numbers do not reconcile.
  • LAS. The claim that SCOReD keeps a clearer link between reasoning and ranking (§5) rests mainly on LAS. But over 90% of student CoTs already state the final ranking (App. G, Table 10), so recovering the ranking from those traces is close to trivial. LAS corrects for this only by weighting the small non-leaking subset at 50%. That subset differs in size by method (45 for Baseline SFT, 204 for SCOReD, 248 for A*-Thought), so the methods are scored on different examples, and no uncertainty is reported. More fundamentally, LAS measures consistency between the explanation and the decision, not whether the reasoning caused the decision. Even so, SCOReD's LAS is not higher than raw-trace SFT for 0.6B (0.656 vs 0.660), and Compress-Distill has the highest LAS for 1.7B (0.690).
  • Reporting. The motivating statistics in §1 (9.35 rechecks per trace; 79% unchanged) differ from §6.1 (8.65 verification segments; 92.89% of loops unchanged; 47.15% of traces changed). The abstract's "2.88% NDCG / 2.96% Recall@3" cannot be derived from Tables 1, 2 or 8. Table 1's 3.51% baseline error rate for 0.6B conflicts with Table 6 (126/4,474 = 2.82%). The post-SFT experiments start from the τ_low = 0.00 student, not the selected 0.50 model (App. E–F).
Questions for authors
  1. With τ_low = 0.5 fixed on Beauty/0.6B, could you report ablations that (a) replace </think>-attention bucketing with random or teacher-attention bucketing, (b) rewrite every segment without reward-based selection, and (c) set α = 0 and β = 0 separately, and drop the log P term? Which layers and heads are averaged for the attention score? These would show what each component contributes.
  2. How are invalid rankings scored in Table 1? Please report NDCG on examples that are valid for both methods, using your existing outputs. Please also reconcile the 3.51% vs 2.82% baseline error rates (Tables 1 and 6).
  3. Does SCOReD's LAS advantage over A*-Thought persist when the explicit final-ranking statement is masked from the student CoTs, or when LAS is computed on a common set of examples? Please also give bootstrap confidence intervals for the LAS differences.
  4. How do the §1 statistics (9.35 rechecks; 79%) relate to those in §6.1? How is the abstract's 2.88% / 2.96% computed? Could you report means and standard deviations over a few SFT seeds for Table 1?
  5. Why were DAPO and OPSD started from the τ_low = 0.00 student, and does the negative result hold from the τ_low = 0.50 model?
1 comment
AuthorsOpenReview ↗

Response to Reviewer K6dr

We thank the reviewer for the constructive feedback and for recognizing SCOReD's motivation, practical improvements, and consistent gains across recommendation domains and student sizes. We address each question below.

Q1. Component Ablations and Attention Scoring

We conduct a new ablation without student-attention importance scores or reward-based action selection: redundant intermediate-ranking stages are pruned using only the semantic stage labels, reducing teacher CoT length by 44.1%. This stage-based variant improves over Baseline SFT (NDCG 0.7818 vs. 0.773; error rate 3.51% → 2.46%), and adding student-aware selection raises NDCG further to 0.801. Compared with A*-Thought, which is also student-aware but removes sentences independently, SCOReD achieves comparable ranking metrics with higher LAS and a lower error rate. Please see Common Response 4 for details. We acknowledge that this experiment removes multiple components simultaneously. We will report additional component-level ablations and results across additional SFT seeds in the final manuscript.

Importance scoring with </think> token attention. The importance of a CoT token is computed from the attention of the </think> token to that token, averaged across all attention heads and layers. Segment importance is then computed by averaging the importance scores of its constituent tokens.

Q2. Invalid Ranking Scoring and Error Rate Discrepancy

Erroneous rankings receive a score of zero for NDCG, MAP, and Recall@3. As detailed in Common Response 1, the overwhelming majority of these errors originate in the reasoning process rather than in final-answer formatting.

We report ranking metrics on the common error-free subsets (3,926 examples for 0.6B and 3,966 for 1.7B) in Common Response 2. We find no statistically significant differences in ranking metrics between methods, including the Gemma-4-26B teacher.

Regarding the discrepancy between Tables 1 and 6, we mistakenly reported the 1.7B Baseline SFT error rate in Table 6. The 3.51% error rate in Table 1 is the correct value for the 0.6B Baseline SFT model. We will correct this in the final manuscript.

Q3. LAS and Explicit Ranking Leakage

We conduct an additional evaluation where we remove explicit complete rankings from the CoTs before LAS evaluation. SCOReD continues to outperform A*-Thought on both student sizes (0.2222 vs. 0.1987 for 0.6B; 0.2239 vs. 0.2046 for 1.7B), indicating that its relative advantage persists after removing explicit ranking statements. Please see Common Response 3 for details, including bootstrap CIs for the LAS differences. We additionally report LAS with 95% CIs on the common error-free subsets in Common Response 2, where SCOReD also exceeds A*-Thought (0.661 vs. 0.593 for 0.6B; 0.636 vs. 0.613 for 1.7B).

Q4. Clarification Regarding Reported Metrics

The difference comes from how refinement loops are counted. The average of 9.35 includes all R → ... → R/F patterns of all 8 rollouts of the Gemma teacher, while 8.65 counts only verification patterns (R → V → R and R → V → C → R) of the highest-NDCG rollout. Similarly, 79% of all refinement loops leave the ranking unchanged, compared to 92.89% of verification loops. The highest-NDCG rollouts usually start out with a better intermediate ranking and do not change their ranking often. We will clarify this distinction in the manuscript.

Based on Table 1, the 0.6B student's relative improvements are +3.62% in NDCG and +3.39% in Recall@3. The abstract's 2.88% NDCG and 2.96% Recall@3 improvements were computed under an earlier evaluation protocol in which incomplete rankings were scored as-is, while other erroneous rankings received zero. However, incomplete rankings can artificially inflate ranking metrics by omitting negative candidates ranked above a ground-truth item. We therefore adopt the consistent scoring convention described in Common Response 1, assigning zero to all erroneous rankings. We will update the abstract accordingly.

Q5. DAPO and OPSD Initialization

We selected the $\tau_{\mathrm{low}}=0.00$ checkpoint for the reported post-SFT experiments because its initial NDCG of 0.793 leaves greater room for potential improvement through DAPO or OPSD. In comparison, the $\tau_{\mathrm{low}}=0.50$ student already achieves 0.801 NDCG, approaching the teacher's 0.803.
We additionally conduct pilot experiments initialized from the $\tau_{\mathrm{low}}=0.50$ checkpoint:

MethodTraining StepsNDCGError Rate
SCOReD τ_low=0.50 (initialization)00.8010.40%
+ DAPO1,0000.7980.80%
+ OPSD1000.7910.72%

Both pilot experiments show a slight performance regression relative to the SCOReD initialization, consistent with the trends observed in our reported $\tau_{\mathrm{low}}=0.00$ experiments.

Reviewer_gyy4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper targets chain-of-thought distillation for LLM-based recommendation. It first documents a domain-specific pathology: a Gemma-4-26B teacher doing candidate ranking rechecks its initial answer 9.35 times per trace on average, 79% of rechecks leave the answer unchanged, and 92.89% of refinement loops leave the ranking unchanged. Naive SFT on such traces teaches students to be verbose and never revise. SCOReD parses each trace into six recommendation-specific stages, scores segment importance by the average attention each segment gets from the </think> delimiter under the pre-SFT student, buckets segments into high, medium and low importance, and picks a per-segment edit (KEEP, REWRITE, FUSE or PRUNE) that maximises a reward combining the student's log-probability of the target answer against length and perplexity penalties. Students trained on the edited traces improve NDCG from 0.773 to 0.801 (Qwen3-0.6B) and 0.782 to 0.802 (1.7B) over SFT on raw traces, with roughly 44% shorter traces and error rates dropping from 3.51% to 0.40%. Post-SFT DAPO and OPSD are reported not to help further.

Critical strengths
  1. The teacher-pathology analysis is the most valuable part of the paper, and it is quantified rather than asserted. The six-stage taxonomy, the adjacent-transition matrix exposing R→V→R as the dominant edge, and the finding that the large majority of refinement leaves the ranking unchanged together explain why recommendation CoT is a poor SFT target. This would be worth reporting even without the proposed method.

  2. Reporting that both DAPO and OPSD fail to improve on the SFT initialisation is an honest negative result that most submissions would have dropped, and it is useful to anyone planning an RL stage on top of recommendation distillation.

Critical weaknesses
  1. Against the strongest baseline the method is a wash, and the framing hides it. In Table 1, A*-Thought gets 0.799/0.675/0.551 at 0.6B and 0.802/0.680/0.556 at 1.7B, so tied or better than SCOReD on all three ranking metrics at 1.7B, while producing traces 56% and 71% shorter (0.85K vs 1.33K, 0.86K vs 1.47K). The abstract and introduction benchmark against "baseline SFT", so the headline claim is established against a baseline the paper's own table shows is not the relevant one. The real wins over A*-Thought are error rate and LAS, which are secondary to the metrics the paper leads with.

  2. There are no component ablations. The method has at least six hyperparameters (τlow, τhigh, τsim, α, β, Krollout) and three separate LLM passes, and only τlow is swept. Nothing isolates attention-based importance against random or positional bucketing, the three reward terms against each other, REWRITE against prune-only, or the six-stage recommendation segmentation against generic blank-line splitting. On top of that, REWRITE and FUSE are executed by the 26B teacher itself, so the "compressed" traces contain freshly generated teacher text. That makes this partly re-generation rather than compression, and the confound is never separated from the student-aware selection the paper credits.

  3. No seeds or variance, on a task that looks close to saturated. The 0.6B student reaches 0.801 NDCG against the 26B teacher's 0.803, and gains fall to +0.6% at 4B. Differences of 0.002 to 0.02 NDCG across Tables 1, 2 and 5 are reported with no dispersion estimate, which makes the monotonic trends (the τlow sweep in particular) hard to rely on.

Questions for authors
  • Since A*-Thought matches on ranking metrics at roughly half the trace length, what is the intended practical case for SCOReD? Is it the error-rate and LAS advantage, and is that worth the extra tokens?
  • How much of the gain comes from REWRITE specifically? A prune-only variant of SCOReD would isolate it directly against A*-Thought.
  • Is </think> attention actually better than simpler importance signals such as segment position or stage label, given that the stage taxonomy already flags Verification segments as the redundant ones?
1 comment
AuthorsOpenReview ↗

Response to Reviewer gyy4

We thank the reviewer for recognizing the value of our teacher-trace analysis and the practical relevance of our findings on post-SFT optimization. We address the questions below.

W1 and Q1. Comparison with A*-Thought and intended practical case for SCOReD

SCOReD achieves competitive ranking performance while providing higher reasoning–ranking consistency and lower error rates than A*-Thought. We believe these advantages are worth the additional tokens in settings where the reasoning itself is used, such as monitoring and further training.

Higher LAS is valuable for monitorability, error analysis, and model improvement. In particular, reasoning that leads to rankings preferred by users can be reinforced during further training, provided the reasoning is consistent with the final ranking. SCOReD achieves higher LAS than A*-Thought: 0.656 vs. 0.592 for the 0.6B student and 0.640 vs. 0.607 for the 1.7B student. SCOReD also approximately halves the error rate relative to A*-Thought (0.40% vs. 0.76% for 0.6B; 0.36% vs. 0.72% for 1.7B), improving reliability.

W2 and Q2. How much of the gain comes from REWRITE specifically? Would a prune-only SCOReD variant isolate its contribution against A*-Thought?

We ablate REWRITE with a prune-only variant of SCOReD, in which segments are kept or pruned but never rewritten. Please see Common Response 5 for the results and discussion.

W2 and Q3. Is </think> attention better than simpler importance signals such as segment position or stage label, given that the stage taxonomy already identifies redundant verification segments?

We conduct a new ablation that uses stage labels instead of student attention: redundant intermediate-ranking stages are identified from the semantic stage labels and pruned without student-attention importance scores or reward-based action selection, reducing teacher CoT length by 44.1%. This stage-based variant improves over Baseline SFT (NDCG 0.7818 vs. 0.773) but lags behind A*-Thought (0.799) and SCOReD (0.801), both of which use student-derived importance scores. Please see Common Response 4 for details.

W3. Dispersion Estimates for Results

We report 95% confidence intervals and paired comparisons for the $\tau_{\mathrm{low}}$ ablation in Table 5. All CIs are bootstrap percentile intervals over the 4,474 test examples (10,000 resamples; paired differences resample the same examples for both settings), so they reflect test-set sampling variability rather than variability across training seeds.

Sizeτ_lowAvg. tokensError rateNDCG [95% CI]
0.6B0.001,823.01.30% (58)0.7932 [0.7875, 0.7988]
0.301,738.70.72% (32)0.7986 [0.7933, 0.8039]
0.501,333.30.40% (18)0.8010 [0.7957, 0.8061]
1.7B0.001,895.70.87% (39)0.7969 [0.7914, 0.8023]
0.301,914.70.58% (26)0.8016 [0.7963, 0.8068]
0.501,467.00.36% (16)0.8023 [0.7971, 0.8073]

We additionally report paired differences in NDCG and error rate, with 95% confidence intervals.

SizeComparisonΔNDCG [95% CI]ΔError rate, pp [95% CI]
0.6B0.30 − 0.00**+0.0053
+0.0012,+0.0095+0.0012, +0.0095

** | −0.58 [−1.01, −0.18] | | | 0.50 − 0.00 | **+0.0077

+0.0038,+0.0118+0.0038, +0.0118

** | −0.89 [−1.27, −0.51] | | | 0.50 − 0.30 | +0.0024

−0.0011,+0.0059−0.0011, +0.0059

| −0.31

−0.63,+0.00−0.63, +0.00

| | 1.7B | 0.30 − 0.00 | **+0.0047

+0.0010,+0.0086+0.0010, +0.0086

** | −0.29

−0.65,+0.04−0.65, +0.04

| | | 0.50 − 0.00 | **+0.0054

+0.0018,+0.0091+0.0018, +0.0091

** | −0.51 [−0.85, −0.20] | | | 0.50 − 0.30 | +0.0007

−0.0026,+0.0040−0.0026, +0.0040

| −0.22

−0.51,+0.04−0.51, +0.04

|

For both student sizes, $\tau_{\mathrm{low}}=0.30$ and $0.50$ achieve statistically significant NDCG improvements over $\tau_{\mathrm{low}}=0.00$. While the NDCG differences between 0.30 and 0.50 are not statistically significant, $\tau_{\mathrm{low}}=0.50$ produces approximately 23% shorter reasoning traces for both student sizes, alongside lower error rates.

We will extend these confidence interval and significance analyses to Tables 1 and 2 and report results across multiple SFT training seeds in the final manuscript.

Reviewer_fXZJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to distill lengthy teacher reasoning traces into smaller models for recommendation reranking. SCOReD segments teacher traces into six semantic categories, estimates segment importance using the student's attention, and selects keep, rewrite, fuse, or prune operations using a reward combining the student's answer log probability, segment length, and perplexity. The edited traces are then used for supervised fine-tuning. Experiments cover four Amazon Reviews categories and three Qwen3 student sizes, with the main baseline comparison conducted on Beauty. Relative to SFT on unedited traces, NDCG improves for the 0.6B/1.7B students, while generation length decreases by approximately 44%/42% and invalid rankings become substantially less frequent. Additional DAPO and OPSD experiments do not provide consistent further improvements.

Critical strengths
  1. The method improves generation cost and output usability together. In Table 1, SCOReD reduces invalid outputs to 0.40%/0.36% for the students while substantially shortening generations. Shorter rankings with fewer omitted candidates have practical value even if some of the NDCG improvement comes from better format adherence. Table 2 and Appendix C also show that improvements over unedited-trace SFT extend beyond one domain or student size.

  2. The editing granularity is motivated by a concrete diagnosis of the traces. Section 6.1 analyzes reasoning stages and repeated ranking–verification loops, motivating operations that rewrite segments as well as remove them. The complete prompts in Appendix H and annotated examples in Appendix I make the design inspectable and provide a useful starting point for studying recommendation-specific trace adaptation.

Critical weaknesses
  1. The evidence does not clearly separate improved recommendation ranking from improved output validity, and the advantage over the strongest compression baseline is limited. In Table 1, A*-Thought achieves 0.799 NDCG versus SCOReD's 0.801 for the 0.6B student, while producing shorter outputs and higher Recall@3. For the 1.7B student, both achieve 0.802 NDCG, while A*-Thought has higher MAP and Recall@3 and shorter generations. Furthermore, if invalid rankings receive zero score, the rounded Table 1 values imply valid-output NDCGs of approximately 0.8011/0.8042 for baseline/SCOReD at 0.6B and 0.8047/0.8049 at 1.7B. This is a conditional diagnostic using each method's own valid subset, not a substitute for evaluation on common examples. Nevertheless, it suggests that reduced invalidity could explain much of the aggregate improvement. The scoring convention and this distinction need to be explicit before drawing stronger conclusions about ranking quality or superiority to existing student-aware compression.

  2. The necessity of the proposed components and the claimed reasoning-quality benefit remain insufficiently supported. The main ablation changes the low-importance threshold without isolating semantic segmentation, attention scoring, rewriting, or the reward components. Comparisons at matched generation length are also missing. LAS measures simulatability and does not establish that a trace causally supports the student's decision; the authors report that over 90% of traces reveal the final ranking. SCOReD's LAS does not consistently exceed unedited SFT or Compress-Distill, and the parsed-example counts in Appendix G need reconciliation with the valid-output counts in Table 1. Thus, the evidence for more coherent or faithful reasoning does not yet convincingly justify the additional segmentation, rewriting, and per-action student scoring.

Questions for authors

The following are clarification and validation requests that could affect my assessment, rather than general suggestions for future work.

  1. Scoring and the source of the gains: How are invalid rankings scored in NDCG, MAP, and Recall? Please reconcile Tables 1 and 6 and report results on the common subset where all methods produce valid rankings. How much improvement remains after applying the same format constraints or deterministic repair to all methods?
  2. Component attribution: With the same teacher traces, tuning budget, and approximately matched generation length, how do random or position-based scores, removal of semantic labels, and rewrite-only or prune-only variants perform? Are repeated training runs or paired confidence intervals available?
  3. Reasoning evidence: Does the simulatability advantage over A*-Thought persist after removing explicit complete rankings from the traces? What exactly does Parsed denote in Appendix G, and why do those counts differ from the numbers of valid final rankings implied by the main table?
  4. Reproducibility and cost: What are the number of teacher rollouts per example, the filtering procedure connecting 13,417 training examples to 11,350 traces, and the attention aggregation across layers and heads? Please quantify teacher calls, student scoring work, and total preprocessing cost per training example.
1 comment
AuthorsOpenReview ↗

Response to Reviewer fXZJ

We thank the reviewer for recognizing the practical benefits of SCOReD's shorter generations, improved ranking reliability, and interpretable trace-editing framework. We address the questions below.

W1 and Q1. Scoring and Source of Ranking Gains

Erroneous rankings receive a score of zero for NDCG, MAP, and Recall@3. As detailed in Common Response 1, most of these errors originate in the reasoning process rather than in final-answer formatting.

We report ranking metrics on the common error-free subsets (3,926 examples for 0.6B and 3,966 for 1.7B) in Common Response 2. We find no statistically significant differences in ranking metrics between methods, including the Gemma-4-26B teacher. However, this understates the underlying reasoning errors made by the student models that led to the invalid rankings.

Paired t-tests on per-example NDCG over the full 4,474-example test set give p-values for SCOReD against Baseline SFT, Compress-Distill, and A*-Thought of 9×10−199\times10^{-19}, <10−68<10^{-68}, and 0.320.32, respectively.

Regarding the discrepancy between Tables 1 and 6, we mistakenly reported the 1.7B Baseline SFT error rate in Table 6. The 3.51% error rate in Table 1 is the correct value for the 0.6B Baseline SFT model. We will correct this in the final manuscript.

Q2. Component Attribution

We conduct a new ablation without student-attention importance scores or reward-based action selection: redundant intermediate-ranking stages are pruned using only the semantic stage labels, reducing teacher CoT length by 44.1%. This stage-based variant improves over Baseline SFT (NDCG 0.7818 vs. 0.773) but lags behind A*-Thought (0.799) and SCOReD (0.801), both of which are student-aware. Please see Common Response 4 for details.

We also evaluate a prune-only variant of SCOReD in Common Response 5, and report paired confidence intervals for the $\tau_{\mathrm{low}}$ ablation in our response to Reviewer gyy4 (W3). We will report additional component-level ablations and results across additional SFT seeds in the final manuscript.

Q3 and W2. LAS Evaluation with Explicit Ranking Removed

We conduct an additional evaluation where we remove explicit complete rankings from the CoTs before LAS evaluation. SCOReD continues to outperform A*-Thought on both student sizes (0.2222 vs. 0.1987 for 0.6B; 0.2239 vs. 0.2046 for 1.7B), indicating that its relative advantage persists after removing explicit ranking statements. Please see Common Response 3 for details.

Clarification for |Parsed| in Table 10: |Parsed| denotes examples for which all four evaluations required by LAS, namely $f_{\mathrm{sim}}(e)$, $f_{\mathrm{sim}}(x,e)$, $f_{\mathrm{sim}}(x)$, and $f_{\mathrm{SFT}}(x)$, produce parseable ranking lists.

For example, for the 0.6B Baseline SFT evaluation, $f_{\mathrm{SFT}}(x)$ failed to produce a parseable final list in 7 of the 4,474 examples, and $f_{\mathrm{sim}}(x)$ failed in 104 additional examples, leaving 4,363 (the other two evaluations always produced parseable rankings). Note that parseable does not imply valid: LAS measures whether the simulator reproduces the student's own output, including erroneous rankings (e.g., ones missing a candidate), so such examples are retained. This is why |Parsed| (4,363) exceeds the number of valid final rankings implied by Table 1 (4,317). We will clarify this distinction in the final manuscript.

Q4. Reproducibility and Preprocessing Cost

Teacher rollouts and filtering. For each of the 13,417 training examples, we generate $k_{\mathrm{rollout}}=8$ teacher rollouts, totaling 107,336 rollouts. We exclude 998 examples for which all eight rollouts exceed 8,192 tokens, leaving 12,419 examples. LLM-based semantic segmentation fails on a further 1,069 examples, resulting in 11,350 training traces.

Attention aggregation. Token importance is computed using attention from the </think> token to each preceding CoT token, averaged across all attention heads and layers. Segment importance is then computed as the mean importance of the tokens within that segment.

Preprocessing cost. Semantic segmentation uses a single LLM call per training example (approximately 13.4K calls). The REWRITE/FUSE LLM is called 18.4 times per retained training example on average, with approximately 245 input tokens and 97 output tokens per call. Across 11,350 training examples, this corresponds to approximately 209K calls, 51.3M input tokens, and 20.2M output tokens. The student (Qwen3-0.6B) performs 106.4 forward passes per training example on average. One pass computes segment importance, while the remaining 105.4 score candidate editing actions (about 78 evaluate answer log-probabilities and about 27 compute perplexity). Across the training set, this corresponds to approximately 1.21M student forward passes. The student performs scoring only and generates no tokens during this stage.

Comments on the paper

6 comments
AuthorsOpenReview ↗

[fXZJ, K6dr] Common Response 1: Relation Between Error Rate and Ranking Metrics

We thank the reviewers for raising the possibility that ranking gains reflect fewer erroneous outputs rather than improved ranking quality.

Scoring convention. Erroneous rankings receive zero NDCG, MAP, and Recall@3. This avoids artificially inflated scores: duplicate indices can count relevant candidates multiple times, while omitted negative candidates can inflate NDCG/MAP.

Error attribution. We note that erroneous rankings are not merely formatting failures but frequently originate from errors within the reasoning process itself. To demonstrate this, we employ an LLM-as-judge to categorize erroneous outputs according to the following taxonomy:

  • Reasoning errors:
    • R1 (Lost): a previously discussed candidate disappears from a later ranking;
    • R2 (Never considered): a candidate is never mentioned;
    • R3 (Duplicate): a candidate is ranked twice during reasoning;
    • R4 (Wrong ID): a nonexistent candidate ID is used during reasoning;
    • R5 (Degenerate): reasoning derails without reaching a valid ranking.
  • Answer errors:
    • A1 (Copy slip): the final answer incorrectly copies an otherwise valid reasoning ranking;
    • A2 (Format): the answer is not a numerical list.
Method#ErrorR1R2R3R4R5A1A2
Baseline SFT157 (3.51%)121 (77%)16 (10%)4 (3%)9 (6%)7 (4%)00
Compress-Distill391 (8.74%)129 (33%)227 (58%)11 (3%)9 (2%)11 (3%)3 (1%)1 (0.3%)
A*-Thought34 (0.76%)17 (50%)8 (24%)3 (9%)4 (12%)1 (3%)1 (3%)0
SCOReD18 (0.40%)12 (67%)04 (22%)1 (6%)1 (6%)00

Of 600 erroneous outputs, 595 (99.2%) involve reasoning errors rather than final-answer errors. The judge also identifies where each locatable error first occurs (start/middle/end):

MethodR1R3R4R5
Baseline SFT1 / 117 / 31 / 3 / 02 / 7 / 06 / 1 / 0
Compress-Distill1 / 107 / 210 / 5 / 65 / 3 / 17 / 1 / 3
A*-Thought0 / 14 / 30 / 3 / 00 / 4 / 00 / 1 / 0
SCOReD0 / 12 / 00 / 3 / 10 / 1 / 00 / 0 / 1

R1 errors arise predominantly mid-reasoning, when candidates are repeatedly grouped and reordered. This supports reducing unnecessary intermediate ranking and verification. It also explains why deterministic output repair cannot replace improved reasoning: adding omitted IDs or removing duplicates may require decisions unsupported by the chain of thought, undermining CoT–answer consistency.

Example: Baseline SFT R1 (index 154)

The graded answer is [4, 6, 1, 10, 8, 9, 3, 2, 5] (9 entries). Missing candidates: [7].

[start: candidate 7 is listed]  
   *   Candidate 6: Hair Chignon Donut Bun Maker (Hair styling tool).  
   *   Candidate 7: St. Ives Body Lotion (Body skincare/Lotion).  
   *   Candidate 8: NaturOli Soap Nut Shampoo (Haircare/Shampoo).  
....  
[mid: grouping; candidate 7 is in no tier]  
   *   **Tier 1: Direct Matches (Cosmetics/Brand affinity)**  
       *   Candidate 4: OPI Skyfall Bond-ettes (Matches Brand 'OPI' and Category 'Nail Polish').  
   *   **Tier 2: Related Beauty Products (Makeup/Hair Styling/Skin/Tanning)**  
       *   Candidate 6: Hair Chignon Donut Bun Maker (Related category: Beauty/Styling tools).  
       *   Candidate 1: RoC Lift Serum (Skin care/Beauty).  
       *   Candidate 10: L'Oreal Conditioner (Hair care/Beauty).  
       *   Candidate 8: NaturOli Shampoo (Hair care/Beauty).  
       *   Candidate 2: Xen-Tan Lotion (Skin care/Beauty).  
       *   Candidate 9: Viva La Juicy Perfume (Fragrance/Beauty).  
       *   Candidate 3: Kiss Her By Kiss Perfume (Fragrance/Beauty).  
   *   **Tier 3: Less related/Different categories (Men's perfume vs Woman-centric)**  
       *   Candidate 5: YSL Men's Perfume (Gender mismatch potential/category shift).  
....  
[end: the 9-item ranking is carried to the answer]  
   Final sequence construction:  
   4 -> 6 -> 1 -> 10 -> 8 -> 9 -> 3 -> 2 -> 5.  
   ...  
   Everything seems consistent.  

Example: Compress-Distill R4 (index 3886)

The graded answer is [5, 4, 3, 9, 8, 6, 10, 26, 2, 1, 7] (11 entries). Out-of-range numbers: [26].

[start: the derma roller is a purchase; the 10 candidates are listed correctly]  
   ...  
   *   Purchase 26: Derma Roller (Micro-needling).  
   ...  
....  
[mid: grouping; purchase 26 appears as a candidate]  
*Tier 3: Less Relevant (Other beauty categories like nails/perfume/hair/tools):*  
   *   Candidate 1 (Nail Polish)  
   *   Candidate 7 (Nail Art Wipes)  
   *   Candidate 10 (Perfume Oil)  
   *   Candidate 2 (Perfume)  
   *   Candidate 26 (Derma Roller - mentions hair loss/wrinkles/acne/scars)  
....  
[end: candidate 26 is ranked and carried to the answer]  
Final Ordered List:  
[5, 4, 3, 9, 8, 6, 10, 26, 2, 1, 7]
AuthorsOpenReview ↗

[fXZJ, K6dr] Common Response 2: Evaluation on a Common Error-Free Subset and LAS Confidence Intervals

To separate ranking quality from output validity, we additionally evaluate all methods on the examples where every method produces an error-free ranking.

Model (N samples)MethodNDCGMAPRecall@3LASToken length
Qwen3-0.6B (3,926)Baseline SFT0.80240.67630.55130.654 [0.568, 0.745]2,438
Compress-Distill0.80340.67740.55290.705 [0.672, 0.737]1,104
A*-Thought0.80560.68080.55730.593 [0.563, 0.623]815
SCOReD0.80460.67870.55370.661 [0.625, 0.698]1,291
Gemma-4-26B (Ref.)0.80390.67900.5547—3,126
Qwen3-1.7B (3,966)Baseline SFT0.80330.67830.55440.690 [0.601, 0.780]2,528
Compress-Distill0.80140.67540.55050.720 [0.693, 0.749]1,116
A*-Thought0.80730.68380.55930.613 [0.584, 0.642]820
SCOReD0.80440.67940.55190.636 [0.603, 0.669]1,426
Gemma-4-26B (Ref.)0.80210.67680.5525—3,140

On this common subset, there are no statistically significant differences in ranking metrics between any of the methods, including the Gemma-4-26B teacher. However, as discussed in Common Response 1, filtering out erroneous rankings also removes notable reasoning errors made by Baseline SFT and Compress-Distill.

AuthorsOpenReview ↗

[fXZJ, K6dr] Common Response 3: LAS after Removing Explicit Ranking

To investigate whether LAS is driven by explicit ranking leakage, we conduct an additional experiment in which an LLM removes explicit complete rankings from the reasoning traces before LAS evaluation.

| Model | Method | |Parsed| | |f_sim(e)=y| | |f_sim(e)≠y| | E

Δ∥fsim(e)=yΔ \| f_sim(e)=y

| E

Δ∥fsim(e)≠yΔ \| f_sim(e)≠y

| LAS | | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | | 0.6B | Baseline SFT | 4,347 | 1,633 | 2,714 | +0.5236 | +0.1315 | 0.3276 | | | Compress-Distill | 4,336 | 286 | 4,050 | +0.3147 | +0.0304 | 0.1725 | | | A*-Thought | 4,352 | 406 | 3,946 | +0.3645 | +0.0329 | 0.1987 | | | SCOReD | 4,334 | 530 | 3,804 | +0.3962 | +0.0481 | 0.2222 | | 1.7B | Baseline SFT | 4,350 | 1,743 | 2,607 | +0.5222 | +0.1401 | 0.3312 | | | Compress-Distill | 4,351 | 335 | 4,016 | +0.3015 | +0.0441 | 0.1728 | | | A*-Thought | 4,343 | 453 | 3,890 | +0.3775 | +0.0316 | 0.2046 | | | SCOReD | 4,337 | 656 | 3,681 | +0.3948 | +0.0530 | 0.2239 |

After removing explicit complete rankings, SCOReD's LAS remains higher than A*-Thought's at both student sizes.

On the original (unredacted) traces, a paired bootstrap over test examples (2,000 resamples) gives a SCOReD − A*-Thought LAS difference of +0.064 (95% CI

+0.023,+0.107+0.023, +0.107

) for 0.6B and +0.032 (95% CI

−0.009,+0.071−0.009, +0.071

) for 1.7B; the advantage is statistically significant for 0.6B and remains positive for 1.7B.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.