- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
the work proposed a new loss design for OPD, demonstrated that stage-aware reweighting outperforms fixed or uniform weighting. The experiments on distillation for training of student models, authors also conducted systematic ablation studies. The improvement was demonstrated on math benchmarks, with 0.11 points stronger than the best baseline.
Critical strengths
The stage-aware design of the training strategy is reasonable and the authors conducted careful research, carried out detailed experiments, clearly demonstrating the performance improvement.
Critical weaknesses
the improvement over baseline is very limited. The authors are encouraged to demonstrate other advantages of this specially designed loss, or explain why the improvement is so small, whether this is due to some choices in this work, such as models? If this level of improvement is hitting the ceiling, then does it worthwhile to further pursue this approach?
Questions for authors
- How to define the boundaries of the trajectories and why four stages.
- Whether the chosen models are suitable for this study, are the improvements in performance depending on the choice of the models?
- What are the computing costs? Additional evaluation of the weighting parameters at token level will add to the total cost of the computing? By how much?
Thanks for authors' response which partially addressed my questions. However, I still have remaining concerns: 1 about the novelty: why or whether it is significantly different from predefined reweighing methods? 2 the quality of writing and figure presentation, 3 and the marginal performance improvement over prior art.
Overall, I intend to maintain my rating.