- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes OSOL, which selects a focus domain during each mixed-domain update and adjusts GRPO coefficients based on the token log-probability drift between consecutive checkpoints. The experiments span four domains and two backbones, and the authors further investigate cross-step backtracking through theoretical analysis and controlled experiments.
Critical strengths
- The method constructs control signals using output changes from existing checkpoints, eliminating the need for explicit Hessian computation.
- Matched ablation experiments, which maintain a consistent focus structure, demonstrate that history residuals provide additional benefits beyond domain weighting.
- Diagnostic experiments employing disjoint probes, bootstrap intervals, and leave-one-out analysis enhance the credibility of the correlation results.
Critical weaknesses
- The main results lack multiple training seeds and uncertainty estimates; notably, the macro-average for the 8B model exceeds the strongest baseline by only 0.0035.
- Study I defines task damage as the increase in NLL on fixed responses, without verifying whether this change corresponds to an actual decline in accuracy or reward.
- The backtracking diagnostic utilizes information from after the second update, whereas gradient diagnostics use pre-update information; consequently, the higher correlation observed does not directly demonstrate superior predictive capability.
- The ablation regarding equal task weights does not directly compare single-focus versus multi-focus correction, and is thus insufficient to establish the necessity of selecting only one focus domain per step.
Questions for authors
NaN