- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduced ReTeach, a multi-round reflection-and-retry process for self-teacher construction. Unlike related works that discard wrong trails or stop at a retrospective diagnosis, it asks the model to keep generating new responses until it reaches the correct answer or cost limit. This enables the student model to learn from reflective contexts without access to reference answers or solutions.
Critical strengths
- The paper is well-written and uses a straightforward way to explain its methods and analysis. Figure 1 and 2 are clear in showing the related work comparison and its own method.
- The motivation sounds interesting, combining self-evolution and self-distillation. One big problem of OPSD is deciding what to give as the hint in the teacher prompt, which normally tends to be gold answer or solution. ReTeach uses student model's evolving path to replace it and provide reflection signals.
- The experiments are sound. Results cover mathematics, science QA, and tool use, comparing against GRPO, OPSD, RLSD, RESD, and CEPO.
Critical weaknesses
- ReTeach requires multiple rounds of teacher generations and explicit reflection. However, the paper does not report training time, GPU-hours, or token consumption against the baselines. This analysis will make it easier to tell whether the improvements justify the additional cost.
- More analysis about reflection versus additional responses would be great. For example, what if we only ask the teacher model to generate more inferences without any reflection? Now it is unclear how much improvement comes from reflection rather than additional inference opportunities.
- I understand that designing ablation studies that change Wa, Wb, Wc in equation 11 at the same time would be difficult. And Figure 3 already shows the results when fixing Wa, Wb and changing Wc. However, it'd be great to see more discussion about them. For example, why fix both Wa, Wb=0.5? Is there any limitation on the relationship between these 3 weights, as the sum should be <= 1? Similarly, now λ in equation 9 also only covers 0.9999 (default), 0 (tracking), 1(frozen). It would be interesting to see, like λ=0.5 curves in Figure 4.
Questions for authors
- How much of the improvement comes from the teacher's original thinking mode, rather than reflection?
- If the work claims it doesn't need access to the gold answer, how can the verifier check whether the current teacher-generated reflection response is correct or not?
5 comments
W3: I understand that designing ablation studies that change Wa, Wb, Wc in equation 11 at the same time would be difficult. And Figure 3 already shows the results when fixing Wa, Wb and changing Wc. However, it’d be great to see more discussion about them. For example, why fix both Wa, Wb=0.5? Is there any limitation on the relationship between these 3 weights, as the sum should be <= 1? Similarly, now λ in equation 9 also only covers 0.9999 (default), 0 (tracking), 1(frozen). It would be interesting to see, like λ=0.5 curves in Figure 4.
3. [W3]: Hyperparameter Analysis
We thank the reviewer for these suggestions regarding our hyperparameter settings, which help us make the analysis more rigorous. Below, we discuss the two types of parameters in turn and will incorporate the corresponding discussion into the revised manuscript.
3.1 Sample weights.
Equation 10 first averages the loss within each category, and Equation 11 combines the category-level losses as
= w_A\mathcal{L}_A + w_B\mathcal{L}_B + w_C\mathcal{L}_C.$$ The weights $w_A$, $w_B$, and $w_C$ are nonnegative loss coefficients, not category probabilities. Their sum therefore need not equal 1 or be bounded by 1. Our default choice of $w_A=w_B=0.5$ assigns equal category-level weight to preserving initially correct behavior and learning from successfully corrected attempts. Choosing $w_B \geq w_A$ is a reasonable option when prioritizing the supervision obtained through teacher reflection. #### 3.2 EMA update coefficient. We respectfully clarify that $\lambda=0.5$ assigns half of the weight to the current student at every update, providing limited temporal smoothing for the teacher. Following the reviewer’s suggestion, and given the time available for additional training, we report Avg@16 on SciKnowEval-Physics for $\lambda=0.95$, averaged over two random seeds. We extend the analysis in Figure 4 with the results below. | $\lambda$ Value | Step 100 | Step 200 | Step 300 | Step 400 | | --- | :---: | :---: | :---: | :---: | | $\lambda=0$ (Tracking Teacher) | $58.80 \pm 1.67$ | $59.27 \pm 0.73$ | $56.56 \pm 1.05$ | $55.83 \pm 0.63$ | | $\lambda=1$ (Frozen Teacher) | $60.94 \pm 0.68$ | $62.71 \pm 0.27$ | $63.02 \pm 0.44$ | $63.31 \pm 0.23$ | | $\lambda=0.95$ | $60.24 \pm 0.22$ | $60.55 \pm 1.22$ | $60.82 \pm 0.72$ | $61.73 \pm 0.21$ | | $\lambda=0.9999$ (Default Setting) | $62.00 \pm 0.74$ | $63.12 \pm 0.64$ | $63.39 \pm 1.23$ | $64.24 \pm 1.36$ | With $\lambda=0.95$, ReTeach reaches 61.73, improving on the Qwen3-8B base model’s 59.84 by 1.89 percentage points. It shows sustained improvement across the evaluated training steps compared with the tracking teacher. Our default setting further increases $\lambda$ to balance the stability of the teacher’s reflection with gradual adaptation to student updates, achieving the best final performance of 64.24. We will extend the evaluation to additional random seeds and update Figure 4 in the revised manuscript.Q1: How much of the improvement comes from the teacher’s original thinking mode, rather than reflection?
4. [Q1]: Improvement by Thinking Mode in Teacher
We thank the reviewer for the opportunity to clarify our ablation setup. In the w/o Reflection experiment, we retain the teacher’s native thinking mode and iterative retries based on previous failed trajectories. We remove only the explicit <reflect>...</reflect> block and its enclosed content. The teacher reasons over the history in thinking mode, and then outputs the solution and final answer without a separate explicit reflection block. The table below reports Avg@16 (%) on SciKnowEval-Physics.
| Dataset | Qwen3-8B | GRPO | RLSD | RESD | CEPO | ReTeach w/o Reflection | ReTeach |
|---|---|---|---|---|---|---|---|
| SciKnowEval-Physics |
With native thinking and history-based retries, ReTeach w/o Reflection already achieves an Avg@16 of 62.91, outperforming GRPO, RLSD, and RESD. Its improvement over the initial model suggests that multi-round teacher reasoning can provide useful supervision for improving single-attempt responses. Adding the explicit <reflect>...</reflect> block further structures this process by asking the teacher to identify the causes of previous errors and articulate how to correct them before producing a solution. The resulting teaching context further improves performance to 64.24, a gain of 1.33 percentage points.
Q2: If the work claims it doesn’t need access to the gold answer, how can the verifier check whether the current teacher-generated reflection response is correct or not?
5. [Q2]: The Verifier Design
We thank the reviewer for the opportunity to clarify the verifier design. Our claim is about what the model observes during training. In our experiments, the verifier compares every generated response, from the initial student responses to each teacher reflection-and-retry response, against the reference answer and returns a binary judgment (correct or incorrect). The student and the teacher receive only this correctness signal, and the reference answers remain private to the verifier throughout training.
For comparison, in RESD the reflector receives the failed response together with feedback that explicitly contains the correct answer, which lets the model work backward from the given answer to build the solution steps. ReTeach keeps the reference answer private to the verifier. The teacher therefore has to reason through each correction on its own, guided only by whether each attempt is judged correct.
Our verifier design offers two benefits. First, the student learns from corrections developed through the teacher’s own problem-solving process without reference-answer leaking. Second, ReTeach is compatible with any black-box verifier that returns binary judgments, including rule-based answer matching, unit tests, and external grading services. Training can therefore proceed while reference answers remain confidential. We will clarify this distinction in the revised manuscript.
We sincerely thank Reviewer hNhG for the detailed feedback and for recognizing our paper “the motivation sounds interesting” and “the experiments are sound”. We also appreciate the reviewer’s acknowledgment that "the paper is well written". For weaknesses and questions, we address each point below.
1. [W1]: Computational Cost of ReTeach
We thank the reviewer for the opportunity to clarify the computational cost of ReTeach and its relationship to the performance gains. Following each method’s original setup, the methods are implemented using different frameworks. For a fair comparison, we evaluate all methods on the SciKnowEval-Physics dataset and report the actual generated tokens and rollouts per training step. For ReTeach, the token count includes the initial student responses, the teacher’s native thinking, explicit reflection, and all retries, including unsuccessful ones. The GRPO-based baselines use eight student rollouts per question with a global batch size of 32.
The results demonstrate that ReTeach achieves the highest Avg@16 with substantially fewer rollouts and a comparable overall generation budget.
2. [W2]: Improvement from Explicit Reflections
We thank the reviewer for the opportunity to clarify the contribution of explicit reflection. We respectfully clarify that our "w/o Reflection" ablation exactly implements the setting suggested by the reviewer, retaining additional teacher retries in thinking mode while removing the explicit
<reflect>...</reflect>tags and their enclosed reflection content. On the SciKnowEval-Physics dataset, the w/o Reflection configuration in Figure 4 achieves an Avg@16 of 62.91±0.67. Adding explicit reflection improves performance to 64.24, an increase of 1.33 percentage points. The results suggest that explicit reflection provides more useful teaching context by locating errors and reasoning toward a correct solution, helping improve the model’s single-attempt accuracy through self-distillation.