acceptodds
Under review as a conference paper at ICLR 2027

Recursive Self-Improvement via On-Policy Distillation for Reasoning

28 views

Abstract

On-policy distillation (OPD) trains a student model by having it generate trajectories, then matching its next-token predictions with an external teacher's next-token predictions. This provides dense, token-level supervision to the student. On-policy self-distillation (OPSD) eliminates the need for the external teacher. Specifically, a second frozen copy of the student model, now given the ground truth in its context, serves as the teacher. The student model only receives the problem and learns to mimic the privileged teacher model, while the teacher remains frozen throughout training. Previous work showed that freezing the teacher is useful for training stability, but we argue that this can prevent the teacher from incorporating the improvements learned by the student during training. Our primary contribution is to address this limitation with a recursive framework built around two complementary components. First, we let the privileged teacher co-evolve with the student so that revision learned in one round can guide the next, a process we refer to as Dynamic Co-Evolution (DCE). Second, because stronger revision can also make responses too verbose and self-critical, we additionally train on shorter, verified rewrites of the model's own on-policy responses. We call this complementary objective Self-Refined Concise Learning (SRCL). Overall, our comprehensive evaluations show that DCE+SRCL outperforms OPSD across multiple model scales and four competition-level mathematics benchmarks. Specifically, on Qwen3-8B, DCE+SRCL reaches 65.97% Average@12, outperforming OPSD by 35.62 percentage points while reducing mean output length by 7.80% relative to DCE alone.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Recursive Self-Improvement, On-Policy Distilation, Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QZQPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper extends on-policy self-distillation with a dynamically updated teacher and an auxiliary objective that learns from concise, verified rewrites.

Critical strengths

The method is well motivated, and the experiments include useful ablations on teacher updates, reference-solution placement, and generation budgets.

Critical weaknesses

See Questions For Authors

Questions for authors

Could the authors train and evaluate OPSD and DCE+SRCL in thinking mode, comparing them with the original thinking-mode model under matched generation budgets?

2 comments
AuthorsOpenReview ↗

Response to Reviewer QZQP

We thank you for your thoughtful feedback and for recognizing the motivation of our method and the usefulness of the ablations on teacher updates, reference-solution placement, and generation budgets. We address your concern below.


W1: Training and Evaluation in Thinking Mode

We reran the experiments in thinking mode, training and evaluating OPSD and DCE+SRCL on Qwen3-4B and Qwen3-8B and re-evaluating the original models under the same protocol. DCE+SRCL outperforms OPSD by 3.68 and 8.38 percentage points at 4B and 8B, respectively.

Qwen3-4B

BaselineAverage@12 (%)
Original thinking model59.72
OPSD56.13
DCE+SRCL59.81

Qwen3-8B

BaselineAverage@12 (%)
Original thinking model61.53
OPSD55.86
DCE+SRCL64.24

The gains over the original models are smaller than in our main non-thinking experiments: +0.09 points at 4B and +2.71 points at 8B. We hypothesize that thinking mode already elicits stronger reflection from the student, leaving less room for improvement through teacher co-evolution. This interpretation is consistent with our non-thinking training curves, where most gains emerge by around step 50 and subsequent improvements are smaller (Figure 3, page 8).


If our responses have addressed your concerns, we would appreciate it if you could reconsider your score. We are happy to provide any further clarification if needed.

Thank you for running the thinking-mode experiments. While DCE+SRCL improves over OPSD, its gains over the original thinking models are small, particularly at 4B. I therefore maintain my original score.

Reviewer_nWUkOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces two changes to the OPSD pipeline: (a) the sharing of weights between the teacher and the student (i.e., the teacher’s weights also get updated over training as the student learns), called “DCE”, and (b) there is an additional SFT term in the objective that maximizes the likelihood of rephrasals of correct responses to be shorter and skip back-tracking/reflection. Results on non-thinking modes of Qwen3-1.7B/4B/8B show improvements over OPSD and GRPO. Ablations are given for freezing the teacher model, the placement of the reference response, and shorter token budgets.

Critical strengths

I am unable to think of any critically strong reasons to accept this paper.

Critical weaknesses

I give the following reasons that push me toward rejection.

  • Neither contribution here is novel, and in fact both have been explored before by [1]. In [1] the authors rephrase correct responses and fix incorrect responses as an SFT objective, and then perform distillation with self-feedback, including experiments where the teacher is also trained (by proxy of sharing weights with the student). [1] has been on arXiv since April 2026 and is not concurrent work.
  • Both in [1] and the current paper, only non-thinking results are presented. Whereas in [1] the authors at least conducted experiments on top of Qwen3 and Olmo3 instruct models, which have already been trained to respond without reasoning, the current paper uses Qwen3-1.7B/4B/8B (thinking models) without reasoning which provides an unfair disadvantage to the baseline numbers.
  • Training a model to not reflect on its reasoning and to compress its response is an undesirable trait, and I am suspicious of it as a method even if it can provide limited gains in specific settings.
  • The most important reason is that the paper is quite poorly written. For example, Figure 2 has text and shapes that are entirely misaligned and not fully rendered, and it focuses on trivialities like the measurement of logits. It appears that this figure was generated in an automated fashion or at least without review. Moreover, throughout the paper, there is an inexplicable focus on superfluous details - one example being “top-k = -1” in Section 4, whereas I would expect a normal phrasing to either omit top-k or mention “no top-k” in the experimental setup. I can only imagine that this number was copied directly from a hyperparameter table. Therefore, this paper needs substantial revision in writing and presentation in its current form.

[1] Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision, He et. al., arXiv 2026

Questions for authors

I do not have any questions that would materially affect my assesment of this submission.

6 comments
AuthorsOpenReview ↗

Response to Reviewer nWUk (Page 1)

We thank you for your time and feedback. We address your concerns about novelty, the evaluation setting, concise learning, and presentation below.


W1: Motivation and Technical Distinctions from SD-Zero

We thank the reviewer for bringing SD-Zero [1] to our attention. We would like to emphasize that both DCE and SRCL introduce novel contributions beyond SD-Zero, with distinct motivations, supervision settings, and training procedures, as detailed below.

SD-Zero explicitly trains the model's generation and revision capabilities with SFT. In its main setup, the resulting reviser then serves as a frozen teacher for on-policy self-distillation. Section 2.2 (page 4) explicitly states that “The reviser (teacher) remains frozen”. Section 3.4 (page 7) reiterates: “In SD-ZERO setup, the teacher is fixed as the SRT model throughout Phase 2.” [1]

For DCE, the key distinction is whether the teacher incorporates reflection capabilities that develop during on-policy training. DCE addresses the mismatch that arises when a frozen, gold-conditioned teacher favors termination of incorrect reasoning while the student's reflection capability strengthens (Figure 1, page 2). DCE uses the updated student parameters as the gold-conditioned teacher at every optimizer step, allowing these emerging capabilities to inform subsequent supervision as shown in Figure 4 (page 8).

For SRCL, the key distinction is when rewriting is optimized and how it controls reasoning cost. SD-Zero learns generation and revision through offline SFT and subsequently distills the reviser [1]. Its Section 3.3 reports that SRT leads to longer generated responses, whereas subsequent self-distillation reduces their length. A limitation of SD-Zero's offline SFT stage is that its rewrite targets are fixed before distillation and are not refreshed to address new detours and repeated reasoning as the policy evolves. SRCL instead constructs targets from current on-policy responses, retains only shorter, answer-verified rewrites, and jointly optimizes their cross-entropy loss with DCE (Equation 6, page 6). This makes concise learning an online auxiliary objective that adapts to the evolving policy, keeping explicit, answer-verified length control coupled to reasoning optimization throughout training.

[1] He et al. Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision. arXiv:2604.12002, 2026.


W2: Fairness of the Non-Thinking Evaluation

We respectfully disagree with the characterization that our paper uses Qwen3-1.7B/4B/8B as "thinking models" "without reasoning," thereby unfairly disadvantaging the baselines. Our experiments use the officially supported non-thinking mode of Qwen3. These checkpoints are hybrid models post-trained to support both thinking and non-thinking modes. Non-thinking mode does not mean "without reasoning". It disables Qwen3's separate <think>...</think> segment before the answer, while the model can still perform multi-step reasoning and reflection within its response.

The official model card states: “We provide a hard switch to strictly disable the model's thinking behavior, aligning its functionality with the previous Qwen2.5-Instruct models.” [2] Figure 4 (page 8) analyzes the reflection behavior that develops under this non-thinking setting.

Qwen3's non-thinking mode has been adopted in multiple studies of reasoning and self-distillation [3–6]. We follow the non-thinking setup released with OPSD (ICML 2026) [3]. Our goal is to improve reasoning quality while controlling token cost through better on-policy supervision and concise learning. Our results show that post-training in non-thinking mode can yield a better accuracy–token trade-off than using the original checkpoint with thinking enabled. For example, at 8B, DCE+SRCL achieves 65.97% accuracy with 17,561 mean output tokens, compared with 62.08% and 18,120 tokens for Base (thinking) (Table 1, page 7). As specified in Section 4 (page 6, lines 283–284) and Table 26 (page 26), our method and the training baselines are compared under the same non-thinking setting.

We also observe improvements on Gemma-4-12B-IT (Appendix A.2, Table 5, page 15), showing that the gains extend beyond the Qwen3 family.

AuthorsOpenReview ↗

Response to Reviewer nWUk (Page 2)

[2] Qwen3 official model card: thinking and non-thinking modes.

[3] Zhao et al. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. ICML 2026. Official GitHub: Non-Thinking Mode.

[4] Pei et al. Negative Self-Distillation: Learning to Reason by Avoiding Flaws. arXiv:2609.11699, 2026.

[5] Li et al. On-Policy Self-Distillation without Any Supervision. arXiv:2608.06296, 2026.

[6] Chen et al. DualOPSD: Closing the Feedback Loop in On-Policy Self-Distillation. arXiv:2608.26019, 2026.


W3: Concerns About Reflection and Response Compression

SRCL targets redundancy such as failed branches, repeated calculations, and continued checking after a correct answer is reached (Section 3.2, page 5). A concise solution can retain the successful derivation without replaying the entire search history. SRCL uses the completed rollout as context to construct this more direct solution target.

Only shorter, self-contained, answer-verified rewrites are used for SRCL training. The reflection-language constraint applies to these auxiliary targets. Overall, our framework encourages useful reflection while reducing redundant reasoning, rather than training the model “to not reflect on its reasoning.”

Adding SRCL reduces generated tokens by 7.80% at 8B while maintaining accuracy (65.76% to 65.97%), and by 10.82% at 4B while improving accuracy (60.00% to 61.88%) (Table 1, page 7). On Qwen3-8B, SRCL's accuracy gains over DCE are more pronounced under tighter 8K and 16K reasoning budgets, with fewer generated tokens under matched decoding settings (Table 3, page 9).


W4: Writing and Presentation

We noticed that Reviewers QZQP and iEtz both recognize the strong motivation of our work. Reviewer iEtz further describes our ablations as "well written and easy to follow."

We rechecked Figure 2 (page 4) and could not reproduce the reported misalignment or incomplete rendering. We suggest viewing the original PDF in a viewer such as the one built into Chrome. If the issue persists, we would appreciate it if the reviewer could let us know which elements appear misaligned or incomplete so that we can investigate further.

Regarding the comment that “it focuses on trivialities like the measurement of logits,” the logits and KL objective in Figure 2 are essential components of our training mechanism. The privileged teacher's logits define the next-token target distribution along the student's on-policy trajectory, and the KL loss uses this distribution to supervise the student.

We have revised the experimental setup to state explicitly that top-k filtering is disabled. This describes the decoding setting used in our experiments, corresponding to top_k = -1 in vLLM [7].

[7] vLLM v0.8.5 documentation: Inference Parameters, SamplingParams.top_k.


If our responses have addressed your concerns, we would appreciate it if you could reconsider your score. We are happy to provide any further clarification if needed.

Reviewer_nWUkOpenReview ↗

Thank you for the rebuttal

Thank you for your rebuttal.

  • I find this discussion on "officially supported mode" meaningless. Whether it is supported or not, the reason many prior OPD methods use non-thinking models is that OPD tends to destroy thinking abilities like backtracking. Your method also uses non-thinking models, and hence my concern is not resolved here.
  • On the comparison to SD-Zero: Your claim that the teacher is fixed throughout training for SD-Zero is only true of their default setting. In [1], see Section 3.4 for an evolving teacher. For your claimed distinction from SD-Zero, there does not really seem to be a clear distinction you claim. For instance: "This makes concise learning an online auxiliary objective that adapts to the evolving policy, keeping explicit, answer-verified length control coupled to reasoning optimization throughout training." - this sentence is just a sequence of words, and I am not able to infer what "online auxiliary objective that adpats..." or keeping "explicit, answer-verified length control coupled to reasoning optimization" mean. These appear to be heavily metaphorized or obfuscated concepts without meaning.
  • "SRCL targets redundancy such as failed branches, repeated calculations, and continued checking after a correct answer is reached". This is exactly my point. Failed calculations (and subsequent backtracking) and checking of the answer are usually useful components of a thinking trace. Removing them while useful on simpler questions, is simply not a desirable trait for the reasoning of an LLM, because they restrict the reasoning ability of the LLM.
  • On the writing and presentation: I am surprised that you did not find any issues with Figure 2. Let me name a few to be precise here:
    • The two "Privileged Teacher" texts are not centered in their bubbles. The second one is placed directly over a second misaligned bubble and is clearly an LM artifact.
    • In the middle box roughly a third of the space is devoted to showing how more tokens can be generated from a rollout prefix and their logits taken, hence enabling calculation of KL divergence. This is something perfectly obvious to any human and a human making a diagram would not spend any real estate on this, instead focusing on the components of the proposed method.
    • The 'repeat' arrow is just dangling in the air, another clear artifact of LM generation.

Due to the above points, I find the paper and, unfortunately, also the text of the rebuttal extremely unreadable. I thus elect to keep my score.

[1] "Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision", He et. al., arXiv 2026

Reviewer_iEtzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces a recursive self-improvement framework based on on-policy self-distillation, a relatively straightforward method in which a copy of the model given the ground-truth solution supervises the student. The paper first introduces Dynamic Co-Evolution (DCE), which updates this privileged teacher from the current model at every step. It then introduces Self-Refined Concise Learning (SRCL), which trains on shorter, answer-verified self-rewrites to reduce the response length that DCE induces. Experiments show that the proposed method is relatively strong, with results supported by ablations on teacher updates, solution placement, divergence choice and loss weights.

Critical strengths

The idea and motivation are strong, where the paper clearly shows that a frozen privileged teacher favours termination over reflection, a convincing reason to update it during training. DCE addresses this by refreshing the teacher from the current model at every step while SRCL complements this by training on shorter, answer-verified self-rewrites. The performance is reasonable, with gains over OPSD and GRPO across model sizes and families. Ablations are conducted, well written and easy to follow.

Critical weaknesses
  1. My main concern is Table 1, which is unconvincing. Base (thinking) is excluded from the main ranking, yet in several settings (notably at 1.7B) it outperforms the proposed method with higher accuracy and fewer tokens. The method largely appears to recover the model’s existing thinking behaviour, so the headline gain over non-thinking OPSD overstates the contribution. This also makes the method less convincing. Clarifying this would be helpful.

  2. Given the issue raised in point 1, and since AIME 2024, AIME 2025, AIME 2026 and HMMT 2025 each contain only 30 questions, reporting standard deviations would be very helpful.

  3. The claim of conciseness is overstated. DCE+SRCL generates about 4.5× more tokens than the base model (17,561 vs 3,922 at 8B). Moreover, at 1.7B it generates more tokens than Base (thinking) while achieving lower accuracy (19,504 vs 17,983 tokens; 26.88% vs 36.88%). This suggests overclaiming and weak length control. I would suggest revising the writing to avoid overclaiming.

  4. Training is unstable. At 1.7B, both variants collapse to 1.53% accuracy with outputs near the 32K cap, and 4B DCE falls from 60.00% to 49.58% by step 200. The reported numbers therefore depend on early stopping chosen on the test set. Clarifying this instability would be helpful.

  5. A sensitivity analysis has been conducted for λSλ_{S}, and early stopping has been applied. Is there a principled or theoretical way to choose λSλ_{S}?

Last but not least, there are multiple instances of overclaiming in the paper, and some tables are not reported clearly. I would recommend toning down these claims.

Questions for authors

See above. I will re-evaluate my score once these concerns have been addressed.

3 comments
AuthorsOpenReview ↗

Response to Reviewer iEtz (Page 1)

We thank you for your thoughtful feedback and for recognizing the motivation of DCE, the complementary role of SRCL, and the clarity of our ablations. We address your concerns about the comparison setting, uncertainty, conciseness, stability, and loss weights below.


W1: Non-Thinking Setting and the Table 1 Comparison

We would like to clarify that we train and evaluate Qwen3-1.7B/4B/8B in their officially supported non-thinking mode, with thinking disabled for both the student and the privileged teacher (Section 4, page 6). These checkpoints are hybrid models post-trained to support both thinking and non-thinking modes [1,2]. The official model card describes disabling thinking as “aligning its functionality with the previous Qwen2.5-Instruct models” [1].

Non-thinking mode is also used for main Qwen3 experiments in prior studies [4–6]. We follow the non-thinking setup released with OPSD (ICML 2026) [3]. Our goal is to improve reasoning within this officially supported mode through better on-policy supervision.

Thinking and non-thinking are distinct inference settings of the same hybrid checkpoint. Our main ranking therefore compares Base, SFT, GRPO, OPSD, and our method under the same non-thinking setting. We additionally include Base (thinking) in Table 1 (page 7) as a reference across modes, showing the performance of the original checkpoints with thinking enabled. Accordingly, the reported 35.62-point gain over frozen OPSD measures improvement between post-training methods under the same non-thinking protocol.

Even at 1.7B, DCE+SRCL substantially improves over both Base and OPSD, reaching 26.88% Average@12 compared with 9.17% and 10.35%, respectively. These are gains of 17.71 and 16.53 percentage points, demonstrating the effectiveness of our method at the smallest tested scale, although the original thinking reference remains stronger at this size.

As model size increases from 1.7B to 4B and 8B, DCE+SRCL accuracy rises from 26.88% to 61.88% and 65.97%. At 4B and 8B, it also exceeds the reported accuracy of the corresponding thinking references, which achieve 60.00% and 62.08%, respectively. This trend provides empirical support for the scalability of our approach across the tested model sizes, with stronger results at the larger scales even relative to the original thinking references.

[1] Qwen3 official model card.

[2] Qwen Team. Qwen3 Technical Report, Section 4: Post-training. arXiv:2505.09388, 2025.

[3] Zhao et al. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. ICML 2026. Official GitHub: Non-Thinking Mode.

[4] Pei et al. Negative Self-Distillation: Learning to Reason by Avoiding Flaws. arXiv:2609.11699, 2026.

[5] Li et al. On-Policy Self-Distillation without Any Supervision. arXiv:2608.06296, 2026.

[6] Chen et al. DualOPSD: Closing the Feedback Loop in On-Policy Self-Distillation. arXiv:2608.26019, 2026.


W2: Evaluation Protocol and Uncertainty

We report Average@12 ± SD below, with detailed results for each benchmark.

Qwen3-4B

BaselineAIME 2024AIME 2025AIME 2026HMMT 2025Avg.
Base21.39 ± 7.4520.56 ± 4.7817.22 ± 2.5911.11 ± 3.4817.57 ± 2.46
OPSD30.83 ± 3.8822.50 ± 3.4324.17 ± 5.3413.89 ± 4.0522.85 ± 2.12
DCE68.06 ± 6.0558.89 ± 4.0569.17 ± 5.9443.89 ± 4.9960.00 ± 2.66
DCE+SRCL74.17 ± 4.2259.44 ± 5.2269.72 ± 2.2344.17 ± 5.0961.88 ± 2.18

Qwen3-8B

BaselineAIME 2024AIME 2025AIME 2026HMMT 2025Avg.
Base28.89 ± 5.1221.39 ± 4.6817.78 ± 4.6110.00 ± 4.5219.51 ± 2.37
OPSD46.11 ± 5.4728.61 ± 4.6829.72 ± 4.9916.94 ± 4.1330.35 ± 2.42
DCE74.72 ± 3.8868.61 ± 6.8872.78 ± 6.2246.94 ± 5.8665.76 ± 2.91
DCE+SRCL73.89 ± 6.4971.94 ± 5.0971.94 ± 5.0946.11 ± 5.2165.97 ± 2.75

AuthorsOpenReview ↗

Response to Reviewer iEtz (Page 2)

W3: SRCL's Conciseness Relative to DCE

As clarified in W1, Base (thinking) serves as a reference across modes. SRCL's conciseness claim is relative to DCE alone, as explicitly stated in the abstract. Its purpose is to improve the balance between accuracy and token usage. Comparing the full DCE+SRCL framework with Base combines the effects of both components: at 8B, accuracy increases from 19.51% to 65.97%, alongside longer responses. Comparing DCE with DCE+SRCL directly assesses SRCL's contribution.

Table 1 (page 7) shows that adding SRCL at 8B reduces mean output length from 19,046 to 17,561 tokens (7.80%) while maintaining comparable accuracy (65.76% to 65.97%). At 4B, mean output length decreases from 19,360 to 17,265 tokens (10.82%), while accuracy improves from 60.00% to 61.88%.

SRCL also improves the accuracy–length trade-off under tighter 8K and 16K budgets (Table 3, page 9). On Qwen3-8B under matched Cue settings, adding SRCL raises accuracy from 31.04% to 35.07% at 8K and from 55.76% to 57.64% at 16K, while reducing mean generated tokens from 7,746 to 7,500 and from 13,171 to 12,407, respectively. Higher accuracy with fewer tokens is also observed under matched Truncate settings.


W4: Training Stability and Checkpoint Selection

We would like to note that training instability and performance degradation have also been observed in prior on-policy self-distillation studies [3,7,8].

Table 1 reports peak benchmark performance, consistent with OPSD's best-checkpoint reporting convention [3]. OPSD and our methods are evaluated at the same checkpoint steps up to 200 (Table 26, page 26). We also report the complete training trajectories, allowing the comparison to be assessed beyond the selected checkpoints.

The advantage persists at the common final step 200: DCE+SRCL reaches 64.58% versus 28.06% for OPSD at 8B, and 61.88% versus 22.50% at 4B (Tables 11–12, page 18). Although DCE alone declines to 49.58% at 4B, DCE+SRCL remains near 61% from steps 100 to 200. These results show that the gains at 4B and 8B are not confined to the peak checkpoints.

Much of the improvement already appears within the first 50 steps: DCE+SRCL reaches 61.32% at 8B and 57.99% at 4B (Figure 3, page 8). Figure 4 (page 8) also shows the early development of reflection preferences in both roles. On the fixed AIME26 trajectories, the probabilities assigned to observed reflection cues reach 80.99% for the student and 81.33% for the teacher by step 50 (Table 21, page 23).

The 1.7B collapse remains a limitation (Table 20, page 23). We hypothesize that the smaller model has greater difficulty sustaining reliable generation and privileged supervision as both roles evolve. Because the roles share updated parameters, a degraded checkpoint may also weaken subsequent teacher guidance and reinforce deterioration. We welcome future work on improving the stability of teacher and student co-evolution, particularly for smaller models.

[7] Shen et al. Purified OPSD: On-Policy Self-Distillation Without Losing How to Think. arXiv:2607.02234, 2026.

[8] Li et al. Diagnosing On-Policy Self-Distillation for Reasoning Language Models. arXiv:2609.39118, 2026.


W5: Choosing the Loss Weights

We would like to clarify that we use a weighted multi-task objective to jointly optimize DCE and SRCL, with the loss weights selected through hyperparameter search (Equation 6, page 6). The corresponding sensitivity analyses are reported in Tables 8 and 27 (pages 16 and 27). We welcome future work on more principled or adaptive loss weighting, as optimal loss balancing is not the primary focus of this paper.


If our responses have addressed your concerns, we would appreciate it if you could reconsider your score. We are happy to provide any further clarification if needed.

I thank the authors for their effort in running additional experiments and providing clarifications.

Unfortunately, I remain unconvinced. I understand that Qwen natively supports a thinking mode, but the concerns still remain on the table. After training, the proposed method appears to use more tokens than the original thinking mode, with achieving lower performance. I also noticed that the thinking-mode results are not bolded or treated as a comparison point, even though they are higher. It would be helpful if the authors could clarify why the table is presented this way or to modify it.

For these reasons, I will keep my current score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.