- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes TrajectoryForge, a gradient-free method for improving a released model using two compatible checkpoints. It computes their displacement, decomposes it into weight-matrix components, and samples candidate models by extrapolating each component with an independent nonnegative half-normal coefficient. Candidates are ranked using benchmark performance, and the selected candidate is deployed as a single model without additional inference-time overhead.
Critical strengths
- The method addresses a useful and realistic deployment setting.
- The paper provides evidence beyond reporting a single favorable maximum. The positive candidate means for Qwen-Coder-1.5B and DeepSeek-Coder-6.7B are more informative than best-of- improvements alone.
Critical weaknesses
- The comparison does not clearly establish the benefit of the training direction. Using the same perturbation scale does not necessarily give directed and isotropic search comparable perturbation magnitudes. The methods also explore different search spaces, making it difficult to identify what causes the reported advantage. Furthermore, the scale-dependent angles in Table 12 appear inconsistent with the stated sampling procedure, weakening the explanation of an angular sweet spot.
- The generalization evidence is limited, and some reported statistics are inconsistent. Most improvements rely on selecting the best candidate using the evaluation benchmark itself. Only two model pairs improve on average, and cross-benchmark selection is demonstrated on one pair. Additionally, the confidence intervals for the mathematical models in Table 11 do not agree with the reported baselines and means. These inconsistencies need clarification before the empirical conclusions can be assessed reliably.
Questions for authors
Please see the critical weaknesses.