- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how relative factor scaling in LoRA affects training dynamics during online GRPO and SFT. Multiplying one low-rank factor by a positive constant and dividing the other by the same constant leaves the initial adapter weight and policy outputs unchanged, yet scales the two factor gradients in opposite directions. Under AdamW with a shared learning rate, coordinate-wise normalization turns this factor imbalance into skewed updates to the effective adapter weight, causing functionally identical initializations to follow different optimization trajectories. Using exact power-of-two rescalings, frozen-rollout GRPO, and teacher-forced SFT on Qwen3 (1.7B-8B) and Llama-3.2-3B, the authors show that scaling the two factor learning rates inversely with the gauge constant largely restores consistent training dynamics.
Critical strengths
- The experimental setup is well thought out: using exact power-of-two BF16 rescalings (so initial logits match to machine precision in Table 7) together with frozen-rollout one-step GRPO (Table 6) makes it easy to see that the divergence comes from AdamW's coordinate step rather than rollout noise.
- I appreciate the candor in Appendices O.1-O.2, where the authors report that native PiSSA is already balanced and none of the 19 real checkpoints they checked follow a global scalar gauge.
Critical weaknesses
- Scalar rescaling (Eq. 2) is just the 1D subcase of the symmetry solved by LoRA-RITE (Yen et al., 2025; Table 19), and Prop. 1 is a direct substitution. Crucially, Apps. O.1-O.2 show native PiSSA is balanced () and real adapters fit a global scalar gauge, so Eq. (6) mostly fixes a synthetic perturbation.
- In 200-step runs, weight decay is not rescaled with . Under AdamW decay , setting (Eq. 6) with shared (Table 4) shrinks by and by , as Eq. (8) () is only used in Table 6. Over 200 steps, this asymmetry confounds Fig. 5(c)'s undershoot.
- The 200-step Qwen3 runs (Figs. 4-5, Table 3) omit the shared-LR baseline, and step-0 accuracy on identical models varies by 3.91 pp in Table 17 (App. D.2), exceeding Table 3's AUC gaps.
Questions for authors
- Why was decay rescaling (Eq. 8) omitted in 200-step runs (Table 4), and does shrinkage on cause Fig. 5(c)'s undershoot?
- Why do shared-LR runs at beat at step 10 in Table 17 (69.53% vs. 58.59%), and how do they do at 200 steps?
Above all, my major concern is the synthetic scalar-gauge scope, it's my pleasure to discuss.