acceptodds
Under review as a conference paper at ICLR 2027

EKLAV: Elicited Key CoT Learning via Answer only superVision

Abstract

Reasoning distillation trains a student to reproduce both the teacher’s chain-of- thought trace and its final answer. We ask whether the trace is better used as privileged training context than as an imitation target. EKLAV keeps the teacher’s reasoning in the input sequence but supervises only the final task answer, applying a task-dependent placement that moves part or all of the trace into the prompt. On BRIGHT passage reranking, EKLAV improves average nDCG@10 at all four tested checkpoints across two model families, winning 41 of 48 model-domain comparisons while reducing training compute by up to 31.6% and inference time by up to 32.7%. On four held-out table-reranking benchmarks, it is the best of four conditions on three, exceeding full-trace distillation by 3.0 points and answer- only training without reasoning by 5.5 points on average. Analysis of generated traces reveals that EKLAV students defer their first verbalized verdict later (0.76 vs. 0.44 relative position), reverse fewer decisions (5% vs. 16%), and produce reasoning more predictive of their own rankings than STD-COT’s, though less predictive than the base model’s. The recipe reuses a fixed teacher corpus, requires no student rollouts or iterative teacher feedback, and applies to both pointwise and listwise formulations. Code and Data are at https://anonymous.4open.science/r/Eklav-C1AE/README.md.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Passage Reranking, Table Reranking, Knowledge Distillation, Reasoning Distillation, Large Language Models, Chain-of-Thought

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pqZHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper questions a fundamental assumption in standard chain-of-thought (CoT) distillation: does the student need to imitate the teacher's reasoning to benefit from it?

In standard CoT distillation (STD-COT), the student is trained to reproduce both the teacher's reasoning trace and final answer. The authors argue that teacher reasoning contains two components: useful information and a particular way of verbalizing that information. They hypothesize that forcing the student to imitate the entire reasoning trace may be unnecessary and can hurt generalization.

EKLAV instead treats teacher reasoning as privileged information. The teacher's reasoning is provided during training, but its tokens receive no direct loss; the student is supervised only on the final answer. This allows the student to benefit from the information in the teacher's reasoning without being forced to reproduce the teacher's reasoning artifact.

The teacher reasoning is available only during training and is removed at deployment. At inference, the student receives the original input and generates its own reasoning before producing the final answer.

Importantly, EKLAV uses task-dependent placement rather than always putting the entire teacher CoT into the prompt. For pointwise passage reranking, the teacher reasoning is split approximately in half: part is placed in the prompt as a hint, while the remaining reasoning stays on the assistant side but is not supervised. For listwise table reranking, the entire teacher reasoning is placed in the prompt, and the student is supervised only on the final ranking output.

Critical strengths
  • Interesting reframing of reasoning distillation. The paper challenges the common assumption that teacher reasoning must be explicitly imitated by the student. The distinction between reasoning as an imitation target and reasoning as privileged information is conceptually interesting and provides a simple alternative training paradigm. This is consequential because the approach can potentially retain useful information from teacher reasoning while avoiding the cost and potential overfitting associated with reproducing the teacher's particular reasoning style.
Critical weaknesses
  • The mechanism is not sufficiently isolated. EKLAV changes multiple factors simultaneously: the reasoning loss is removed, reasoning is moved into the prompt, lexical filtering is applied, sequence length changes, and the overall context layout differs from STD-COT. The MASK-ONLY ablation is useful, but it does not fully isolate these components. For example, it would be important to evaluate what happens when the same filtered reasoning hint is placed in the prompt while the remaining reasoning is still supervised. Similarly, a systematic study of different prompt/response splits (e.g., 25/75, 50/50, 75/25, and 100/0) would help clarify which design choice is responsible for the gains.

  • There is a substantial training–inference distribution shift. During training, the student predicts the answer conditioned on teacher-generated reasoning, whereas at inference it receives no teacher reasoning and must generate its own reasoning before producing the answer. This is a substantial reasoning-context shift. The fact that EKLAV performs well despite this mismatch is interesting, but the paper provides limited analysis of why the learned behavior transfers successfully from teacher-generated to student-generated reasoning.

  • The evaluation is limited in task diversity. The strongest experiments focus on passage reranking and table reranking, which are useful but relatively specific settings. Moreover, the authors report that exploratory mathematical-reasoning experiments show the opposite pattern. This makes it difficult to conclude that EKLAV is generally superior for reasoning distillation. The evidence more directly supports the narrower claim that EKLAV is effective in the evaluated reranking settings. Broader evaluation across substantially different reasoning tasks would strengthen the paper's generality.

  • The evaluated model scale is relatively small. The experiments use models ranging from 4B to 14B parameters. While this provides a reasonable range for studying the method, it leaves open whether the observed behavior persists at larger scales and in more realistic teacher–student configurations. For example, it would be valuable to see whether the same conclusions hold for substantially larger teachers and students, such as a 70B teacher distilling into a 7B or 14B student. The interaction between model scale, teacher quality, and the usefulness of privileged reasoning is therefore still unclear.

Questions for authors

Please address the above concerns (Weaknesses section).

Reviewer_Py2iOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

During training, EKLAV uses the teacher model's reasoning text as context and computes the loss only on the final answer. For passage reranking, a filtered reasoning prefix is placed in the prompt, while the remaining teacher reasoning is retained in the response but excluded from the loss; for table reranking, the full reasoning is placed in the prompt. The teacher context is removed at test time. The paper evaluates this training scheme on BRIGHT and several table reranking datasets, reporting higher ranking scores, fewer invalid outputs, and shorter generations than standard CoT distillation.

Critical strengths
  1. The improvements on BRIGHT are fairly consistent across the evaluated models. Table 2 covers four model checkpoints from two model families, with gains of 2.2–3.1 points in average nDCG@10 over STD-COT. The comparisons use identical candidate sets and matched training settings, and no teacher context is provided at test time. These results support a practical conclusion: under the reported settings, the complete training scheme improves passage reranking performance.
  2. Under the same generation budget, shorter outputs and more valid rankings are practically useful. EKLAV substantially reduces output failures on several table datasets (Table 4). The reported generation lengths, together with the inference timings for passage reranking, also support its inference efficiency advantage (Table 6 and Appendix B.6). Training uses fixed teacher data and does not require additional sampling of student trajectories or repeated requests for teacher feedback.
Critical weaknesses
  1. The current ablation studies do not clearly establish the contribution of each design choice. CasCoD [1] already separates reasoning learning from answer learning and computes the answer loss conditioned on the question and teacher reasoning. EKLAV removes the reasoning loss while changing how the reasoning text is filtered and where it is placed; the effects of these changes need to be tested directly. In Table 3, MASK-ONLY scores 30.33, below STD-COT at 31.51 and EKLAV at 34.24. However, MASK-ONLY retains the complete, unfiltered reasoning and places it in the assistant context. Thus, the comparison with EKLAV changes both the text content and its placement. It demonstrates the effectiveness of the combined approach but does not isolate the contribution of each change. More convincing controls would hold the filtered text fixed and change only its placement, and then hold EKLAV's text layout fixed while retaining a reasoning loss with a nonzero weight. In addition, the BRIGHT results table does not include Base or NAIVE-SFT, so the value of adding teacher context over ordinary answer supervision remains unclear for passage reranking.
  2. The out-of-domain table experiments do not yet distinguish ranking quality from output validity. The evaluation assigns a score of zero to invalid outputs. On OTT-QA (small), the score increases from 83.20 to 90.27 while the failure rate falls from 14.5% to 0.1%; on FeTaQA, the failure rate also drops from 8.3% to zero (Table 4). Reducing failures is valuable in itself, but aggregate scores alone cannot establish how much the model's ability to judge relevance in new domains has improved. The experiments with a larger generation budget in Table 5 cover only in-domain datasets. Reporting results on the same subset of queries for which both models produce valid rankings, together with out-of-domain comparisons under different generation budgets, would help clarify the source of the generalization gains. The shared-valid-query analysis should accompany the full-set results because it conditions on successful outputs. The authors have acknowledged the issue of output validity, but the current analysis does not resolve it. 3.The gains on BRIGHT and the more reliable outputs give this work practical value. My main reservation is that the paper does not yet provide sufficiently clear experimental support for its incremental contribution relative to existing training objectives. The evidence most likely to change my assessment would be controlled comparisons with text content and layout held fixed, comparisons of reasoning-loss weights, and further analysis of out-of-domain output validity. [1] Chengwei Dai, Kun Li, Wei Zhou, and Songlin Hu. Improve Student’s Reasoning Generalizability through Cascading Decomposed CoTs Distillation. EMNLP 2024, pp. 15623–15643, Section 3.3. https://aclanthology.org/2024.emnlp-main.875/
Questions for authors
  1. Decision-relevant ablation request. For passage reranking, what happens if exactly the same filtered teacher text is retained entirely in the assistant context, with no loss computed on it, instead of moving the prefix into the prompt? With EKLAV's text layout held fixed, does assigning a nonzero loss weight to the teacher reasoning retained in the response improve or hurt performance?
  2. Decision-relevant evaluation request. On the out-of-domain table datasets, how do the two methods compare on the queries for which both produce valid rankings? How many failures are caused by exhausting the generation budget? Do the conclusions change when STD-COT's output limit is increased to 16,384 tokens?
  3. Baseline clarification. What are the BRIGHT scores of Base and NAIVE-SFT under the same evaluation protocol?
1 comment

Thank you again for the decision-relevant questions. Two new results add to our first reply (tables in Response 7).

Q3, value of teacher context over answer-only training. We completed the hint-size sweep. With a shorter filtered hint, EKLAV's lead over NAIVE-SFT on BRIGHT grows: 34.98 at 10% and 35.50 at 25%, against 33.39 for NAIVE-SFT (+1.59 and +2.11 points, compared with +0.85 at the paper's 50%; Tables 3 and 25). The two models share the training data and recipe and differ only in the teacher hint, so the gain comes from adding a short filtered prefix of the teacher's reasoning to answer-only training.

Generalization. We evaluated the Qwen3-8B passage checkpoints, without retraining, on seven held-out BEIR collections. EKLAV improves on the base model on all seven, and with a 25% hint it is ahead of STD-CoT on all seven (+3.62 points; Table 24), with the largest gains on finance QA and entity retrieval.

If these address your concerns, we would be grateful if you would consider revisiting your assessment.

Reviewer_SXgEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes EKLAV, a reasoning distillation approach that uses teacher CoT as training context while supervising only the final answer. Experiments on passage and table reranking show consistent improvements over standard CoT distillation, along with shorter generations and lower inference costs.

Critical strengths
  1. The paper explores a meaningful alternative to directly imitating teacher reasoning, with a simple and practical training recipe.
  2. The empirical results are encouraging, showing consistent improvements across model scales and substantial efficiency gains.
Critical weaknesses

1.During training, the student predicts answers conditioned on teacher-provided CoT, which may already reveal the answer. At inference, this context is removed and the student must generate its own reasoning, despite receiving no direct supervision for reasoning generation. Although the paper acknowledges this context shift, it remains unclear how answer-only supervision leads to the observed improvements in autonomous reasoning, rather than simply improving answer prediction or formatting. 2.Unparseable outputs receive zero nDCG, and EKLAV has substantially fewer parsing failures. Moreover, extending STD-CoT's decoding budget reverses some in-domain gains. 3.EKLAV jointly changes reasoning placement, loss masking, and sequence length. The MASK-ONLY ablation is useful but insufficient to isolate these effects. Different hint ratios and length-matched controls are missing.

Questions for authors
  1. Can the authors further explain how answer-only supervision conditioned on teacher CoT improves the student's self-generated reasoning at inference, given the reasoning context shift? What distinguishes this effect from improved answer prediction or output formatting?
  2. Can the authors report OOD results on jointly valid outputs and under an extended STD-CoT decoding budget?
  3. How sensitive is EKLAV to the prompt-side reasoning ratio and sequence length?
  4. Can the authors provide detailed mathematical-reasoning results and clarify the method's task-dependent limitations?

Comments on the paper

1 comment
Area_Chair_T3LBOpenReview ↗

Reminder: Reviewer Engagement During the Discussion Period

Dear Reviewers,

As the discussion period is drawing to a close, could you please review the authors' rebuttal and provide your feedback at your earliest convenience?

Active engagement in discussions with the authors is an essential part of the peer-review process and plays an important role in ensuring the success of the conference. Your participation would be greatly appreciated.

Thank you for your time and contributions.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.