- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes Curriculum-OPD, a framework for efficient multi-teacher, multi-domain OPD. Starting with the observation that the relationship between teacher-student alignment and downstream performance, the authors propose Curriculum-OPD which consists of three part: (1) sample selection based on contextual feedback estimated from short student responses, (2) question-level supervision selection among external teachers and reference-conditioned self-distillation, and (3) domain-balanced update projection to coordinate joint training. Experiments on seven benchmarks coving different domains, and the results show improvements over standard OPD and ExOPD.
Critical strengths
1: The paper addresses a meaningful problem in LLM post-training. Improving the efficiency of multi-teacher OPD is an important and timely problem to explore.
2: Direct motivation and interesting method design. The distinction between feedback with and without the input question provides an interesting perspective on teacher-student disagreement. The proposed framework connects sample selection, supervision assignment, and multi-domain optimization in a singe training pipeline, which is a great attempt from my perspective.
Critical weaknesses
1: The paper uses a 16 token preview for sample selection. However, for complex tasks, such as mathematical reasoning, the main difficulties may arise much later in the response. It is unclear whether such a short preview can reliably reflect the learning value of a question. I would encourage the authors to compare different preview lengths, such as 8, 16, 32, 64, and longer sequences, and investigate how the preview length affects both sample selection quality and downstream performance.
2: Computational efficiency claim remains a major concern for me. The paper primarily demonstrates efficiency in terms of training updates rather than total computational cost. According to Appendix C, the proposed method screens 56,608 mathematical questions and 25,276 code questions, each requiring student generation and four log-probability scoring views. In addition, OPHD requires generating a full initial-student response for each selected question and scoring it with multiple eligible supervision sources before training. Domain-balanced update projection also introduces additional gradient computations during training. These costs are particularly important given that the main experiments involve only 7 or 14 training updates. Although Figure 3 shows faster improvement in terms of updates, it remains unclear whether the savings outweigh the additional preprocessing and optimization costs. I would encourage the authors to report end-to-end GPU hours, including all preprocessing stages, and compare performance under matched computational budgets.
3: The interpretation of the early training dynamics in Figure 1 lacks detailed analysis. Figure 1(b) shows substantial changes in AIME24 accuracy within the first few training updates. However, Appendix D reports considerable response truncation during this period. For the 4B specialist, 59 and 35 out of 480 responses are truncated at updates 2 and 3, respectively. For the 30B teacher, the numbers are 74, 94, and 130 at updates 2, 3, and 4. These observations raise the question of whether changes in response length and truncation contribute to the observed performance fluctuations, rather than the proposed feedback dynamics alone. Notably, the accuracy decline of the 4B specialist at update 4 occurs without truncated responses, so truncation alone cannot explain all the fluctuations. See problem 3 for more details.
Questions for authors
1: Could you add a comparison with different token preview length, and provide results on how the preview length affect the performance?
2: Could you report the end-to-end GPU hours?
3: Could the authors provide a more detailed analysis of Figure 1(b), particularly why AIME24 accuracy increases from around 20% to 70% within only two training updates? According to the appendix, both training and evaluation use non-thinking mode. This raises the question of whether the rapid improvement comes partly from changes in the student's response length or its ability to generate longer reasoning steps, rather than solely from acquiring new reasoning capabilities. Moreover, Appendix D reports substantial response truncation during the early updates, suggesting that response length may also contribute to the observed performance fluctuations. It would be helpful to report the average response length, truncation rate, and accuracy across training updates to better understand these dynamics. Additionally, Figure 1 uses a 32,768-token generation limit, whereas the main experiments use 16,384 tokens. Could the authors clarify whether this difference affects the reported results? The reason I raise this is that both Qwen3 4B and 30B Instruct are models that produces long reasoning chains even in non-thinking mode, considering the length exploration recorded in the appendix, I do think it worthing taking this into consideration.