- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes BoundaryEvo, a distillation framework that combines knowledge distillation with curriculum learning for long-chain reasoning. Specifically, it studies how to iteratively adapt training data and distill teacher trajectories according to the evolving capability boundary of the student model, enabling the student to better acquire long-chain reasoning abilities. Using only 1,000 training examples, the method achieves strong improvements across 11 reasoning benchmarks—for example, improving Qwen3.5-4B from 79.8% to 92.5% on AIME25, from 62.9% to 75.3% on IMO-AnswerBench, and from 26.8% to 46.0% on AMO-Bench.
Critical strengths
Clear and well-motivated research direction. Existing distillation approaches typically rely on static data selection and lack a student-aware mechanism. This paper effectively captures this limitation by dynamically adapting the training data to the evolving capability boundary of the student model.
Critical weaknesses
1.Limited technical novelty and heavy engineering. Although the paper is packaged around the concept of the "capability boundary" (which has already been widely used to frame many existing works), the actual method is quite engineering-heavy. At its core, it essentially boils down to using an LLM-as-a-judge for data filtering and rejection-sampling SFT, combined with standard engineering tricks such as experience replay.
2.Lack of important baselines. The baseline comparisons are not comprehensive enough. In particular, the paper is missing comparisons against classic difficulty-based curriculum learning methods and standard pass@k-based rejection-sampling baselines, which are closely related to the proposed approach.
Questions for authors
Although the proposed pipeline is quite engineering-heavy, I still find this research direction meaningful. However, since the experiments in the paper rely on LoRA—which may introduce optimization bias—could the authors provide results with full-parameter fine-tuning to verify whether the conclusions still hold? In addition, could the authors compare against stronger baselines, such as a better-designed rejection-sampling SFT approach, and whether the multi-stage iterative training genuinely brings significant benefits compared to simply pooling all the selected data together and training in a single pass?
6 comments
Follow-up
Thank you for the detailed response and the additional experiments. The stronger baselines and full-parameter fine-tuning results address several of my concerns and strengthen the empirical support for the proposed framework.
I would appreciate further clarification on the comparison between iterative and pooled training. Given that both settings use the same training records and teacher trajectories, the substantial performance gap is particularly noteworthy. Could you clarify what “matched training exposure” entails in practice? Specifically, are the total optimization steps and training-token budgets matched, and how are the learning-rate schedule, warmup, and optimizer states handled across the iterative rounds versus the pooled run?
These details would help distinguish the benefit of the round-wise training procedure from potential differences in optimization. Since the pooled control already uses data acquired by BoundaryEvo, I would also encourage a more precise interpretation of what this comparison establishes about iterative training versus dynamic data selection.
Replying to Follow-up Comment by Reviewer hyUo
Dear Reviewer hyUo,
Thank you for your thoughtful follow-up and for acknowledging that the stronger baselines and full-parameter fine-tuning results address several of your concerns. We appreciate your careful examination of the iterative-versus-pooled comparison. You are right that the optimization details are essential to interpreting this result. We have checked the saved training configurations and logs and recomputed the supervised-token counts. We clarify the comparison below.
- Q1: What does “matched training exposure” mean, and are optimization steps and training tokens matched?
- A1:
“Matched training exposure” refers to the same training-record multiset, teacher trajectories, and cumulative supervised-token exposure, all of which are identical between the two settings. Both settings start from the same Stage 1 merged checkpoint and process each of the 900 Stage 2 record occurrences for five epochs, preserving replay multiplicities.
Table F1. Actual configurations of the iterative and pooled Stage 2 runs.
| Setting | Iterative training | Pooled training |
|---|---|---|
| Training records | 300 per round × 3 rounds | The same 900 records pooled together |
| Epochs | 5 per round | 5 over the pooled dataset |
| GPUs / per-device batch / gradient accumulation | 1 / 1 / 8 per round | 4 / 1 / 2 |
| Effective batch size | 8 | 8 |
| Optimization steps | 190 per round; 570 total | 565 total |
| Warmup steps | 19 per round; restarted each round | 57 once at the beginning |
| Peak learning rate | 1e-4 | 1e-4 |
| Learning-rate schedule | Three separate warmup + cosine schedules | One continuous warmup + cosine schedule |
| Optimizer state | Reinitialized each round | Preserved throughout the pooled run |
| LoRA adapter | Merged after each round; a fresh adapter is initialized for the next round | Initialized once and trained continuously |
| Reconstructed supervised response tokens | 105,064,215 | 105,064,215 |
Both settings use LoRA rank 64 and alpha 128, a maximum sequence length of 40,960 tokens, bf16, and DeepSpeed ZeRO-3. The original iterative runs use replay/novel allocations of 150/150 in Turn 2 and 162/138 in Turn 3; these repeated records retain their multiplicities in the pooled dataset.
The five-step difference arises from the final incomplete batch of each epoch: iterative training uses 3 × 5 × ceil(300/8) = 570 updates, whereas pooled training uses 5 × ceil(900/8) = 565. Specifically, iterative training has 555 full batches and 15 four-record tail batches; pooled training has 560 full batches and five four-record tail batches.
The token counts were reconstructed from the saved datasets using the corresponding tokenizer, the qwen3 template, and the 40,960-token truncation rule. They count non-masked response-label tokens, including reasoning, final answers, and end-of-response markers, and exclude prompt and padding tokens. The three rounds contribute 37,568,965, 33,702,775, and 33,792,475 supervised tokens, respectively. None of the 900 records was truncated in this reconstruction. These are reconstructed supervision budgets, not measurements from a runtime per-batch token counter; Stage 1 is excluded because both runs share its checkpoint.
- Q2: How are the learning-rate schedule, warmup, and optimizer states handled?
- A2:
In iterative training, each round loads the preceding merged model weights and creates a new LoRA adapter and trainer, with resume_from_checkpoint: null. Previous LoRA updates are therefore retained in the model weights, but the adapter, optimizer moments, optimizer step counter, and learning-rate scheduler are reinitialized. Each round performs 19 warmup steps toward 1e-4, followed by cosine decay toward zero over its 190-step schedule.
The pooled run initializes its adapter and optimizer once, performs 57 warmup steps, and then follows a single cosine schedule over 565 updates, retaining optimizer states throughout. Consequently, the iterative setting returns to a higher learning-rate regime in later rounds, while the pooled learning rate continues to decay. We agree that these differences can affect the results even when cumulative supervised-token exposure and peak learning rate are matched.
Official Comment by Authors
Dear Reviewer hyUo,
We sincerely thank you for the insightful comments and constructive suggestions. We appreciate your recognition of our clear research motivation and the meaningful direction of adapting distillation data to the student's evolving capabilities. In response to your suggestions, we added difficulty-based curriculum learning, stronger rejection-sampling SFT, a same-data pooled control, and a first-update full-parameter comparison.
We agree that LLM judging, rejection sampling, SFT, and replay are established components. Our contribution concerns which questions should receive a limited training budget as the student changes. Teacher verification determines whether a target solution is reliable; student-aware acquisition determines whether the corresponding question should be selected for the current student. BoundaryEvo couples repeated student–expert assessment with verified distillation and replay. We do not claim a new optimization objective or novelty for each component separately.
To test the value of this acquisition procedure, we added two stronger baselines. Difficulty curriculum shares the candidate pool, quality filtering, and teacher verification; it estimates difficulty from the initial student's empirical success rates and fixes an easy-to-hard schedule. Strong RS-SFT uses the same teacher, per-question sampling cap, and verifier to obtain correct trajectories for quality-qualified questions, without student–expert gap selection. These controls test whether ordinary difficulty ordering or obtaining reliable targets alone suffices.
Table R1. Qwen3.5-4B LoRA results for difficulty curriculum, strong RS-SFT, same-data pooled training, and BoundaryEvo (scores in %).
BoundaryEvo has higher reported scores on all five tasks, by 3.02–8.33 percentage points over the curriculum baseline and 0.83–5.03 over strong RS-SFT. In particular, the AIME25 / GPQA-Diamond scores are 92.50 / 81.30 for BoundaryEvo, versus 84.17 / 78.28 for curriculum and 87.50 / 79.29 for RS-SFT. These results support an empirical contribution beyond applying the individual filtering and training components.
For the pooled control, we collect the same 900 Stage 2 records acquired by BoundaryEvo across its three rounds, retaining replay multiplicities and the corresponding teacher trajectories. We then shuffle and train on this pooled multiset from the same Stage 1 checkpoint, with matched training exposure.
The pooled results are included in the table above. Iterative BoundaryEvo is higher on all five tasks, with differences of 12.50, 7.47, 14.17, 5.78, and 4.41 percentage points. This shows that simply pooling the acquired data does not match iterative training under the evaluated setup. These results indicate that multi-round training guided by student capability feedback provides gains over a single pooled training stage.
We compared BoundaryEvo with both RS-SFT and the LLM2LLM baseline under full-parameter fine-tuning, sharing the primed initialization and tuning budget. The following results are for one Stage 2 update:
Table R2. Full-parameter fine-tuning on Qwen3.5-4B: comparison after one Stage 2 update (scores in %).
Within full-parameter fine-tuning, BoundaryEvo exceeds RS-SFT by 2.50, 5.00, 4.17, 3.03, and 2.52 percentage points, and also exceeds the LLM2LLM baseline on all five tasks. This supports a conclusion with a defined scope: in the tested first-update setting, the observed acquisition advantage is not confined to LoRA.