- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates the relationship between LoRA rank and mathematical reasoning performance in RLVR. The authors report that increasing LoRA rank improves performance on Qwen2.5-Math models, while having little effect on DeepSeek-R1-Distill-Qwen-1.5B, which they interpret as a saturation effect. They also measure the effective rank of trained LoRA updates and interpret the results as evidence of a capacity bottleneck.
The paper claims the existing LoRA-RLMO initialization as its own method under a new name, "OffPrin," without citing or acknowledging the original work.
Critical strengths
The paper provides relatively extensive experimental evaluations, as a useful third-party independent validation of existing research.
Critical weaknesses
-
Serious originality The proposed OffPrin is identical to the previously published LoRA-RLMO [1] in its initialization and core methodology. Nevertheless, the authors rename this existing method and present it as their own contribution. The underlying optimization analysis and the effectiveness of this initialization for RLVR have already been studied as well. I am very concerned about the academic integreity of the authors.
-
No novelty in the rank-scaling claim. The effect of LoRA rank on RL training has already been extensively studied [2]. The paper presents additional rank sweeps as a new "saturation-gated scaling law," yet provides neither a quantitative definition of saturation nor an actual scaling law. Its contribution beyond existing empirical findings remains unclear.
Insufficient justification of the claimed mechanism. The paper interprets the high effective rank of trained LoRA updates as evidence of a capacity bottleneck. However, a high effective rank does not necessarily imply that these directions are essential for reasoning performance, and the paper provides insufficient evidence to establish this claim.
References
[1] Zhang et al. (2026). Geometry-Preserving Orthonormal Initialization for Low-Rank Adaptation in RLVR. ICML 2026.
[2] Mind Lab (2026). On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters.
[3] Ye et al. (2026). On the Implicit Reward Overfitting and the Low-rank Dynamics in RLVR.
Questions for authors
Literature review and originality. Did the authors conduct a thorough literature review before undertaking this research? The proposed OffPrin is identical to the previously published LoRA-RLMO, while the other central topics, including LoRA rank scaling, effective-rank analysis, and off-principal learning, have already been extensively investigated in prior work. Why does the paper present these established methods and findings as its main contributions?
2. Saturation-gated scaling law. What exactly is the proposed scaling law? The paper provides neither a quantitative relationship nor an independent definition of saturation. How does this claim go beyond the observed rank-dependent performance differences across a few backbones?
3. High-rank mechanism. Why does a high effective rank necessarily imply a capacity bottleneck? Could the authors provide direct evidence that these additional singular directions are essential for reasoning performance, rather than merely being present in the trained updates?
3 comments
3. High effective rank and functional necessity
The effective-rank statistic establishes that the learned updates distribute their singular mass across many directions. Functional relevance is addressed by the intervention evidence and can be tested further through direct removal.
The existing spectral-band intervention is particularly informative. At rank 32, top-, middle-, and bottom-band initializations yield MATH-500 scores of 69.1, 70.6, and 72.7, respectively. The read-outs are all orthonormal. Consequently, the measured differences depend on which subspace receives capacity, beyond the presence of orthonormality.
we have done the experiments of truncation, norm restoration, and equal-budget recovery, which is also shown from the second reviewer's rebuttal answer:
| Retained rank | Truncation condition | Immediate MATH-500 (%) | Immediate OlympiadBench (%) | MATH-500 after recovery (%) | OlympiadBench after recovery (%) |
|---|---|---|---|---|---|
| 128 | Untruncated control | 72.0 | 40.2 | 74.6 | 41.1 |
| 64 | Direct truncation | 70.8 | 39.3 | 74.1 | 40.7 |
| 64 | Norm-restored truncation | 71.1 | 39.6 | 74.2 | 40.8 |
| 32 | Direct truncation | 69.1 | 37.8 | 73.0 | 39.8 |
| 32 | Norm-restored truncation | 69.7 | 38.4 | 73.3 | 40.0 |
| 8 | Direct truncation | 65.8 | 34.5 | 70.3 | 37.5 |
| 8 | Norm-restored truncation | 66.6 | 35.3 | 70.8 | 37.9 |
I don't see any revision in the manuscripts. The rebuttals are highly likely to be AI generated by many signals. Since you acknowledge the overlap of the existing work, there is almost nothing fundamental remaining in your contributions. Rank scaling is also mentioned in literature, which you avoided discussing in the rebuttal.
We thank the author for your review and we open to more discussion
Thank you for the detailed review. We address the attribution concern directly and then clarify the distinct empirical contribution and the additional tests of its mechanism.
1. Attribution and originality
Eq. (6) uses the same minor orthonormal initialization as LoRA-RLMO: A0=Vbottom,r⊤ and B0=0. We will explicitly cite Zhang et al. at this equation, identify the corresponding relationship of the principal control to LoRA-RLPO, and revise the contribution statement around the empirical findings established here.
The manuscript already discusses MiLoRA, GeoRA, The Path Not Taken, and LoRA Without Regret, including a matched GeoRA comparison in Table 8. GeoRA and LoRA-RLMO are distinct references; the revision will make the latter relationship explicit.
The implication that mathematical overlap in the initializer exhausts the paper’s scientific content does not follow. The study additionally establishes:
We respectfully ask that the attribution issue and the technical value of these findings be assessed separately. The documented overlap calls for explicit attribution and precise contribution statements; it does not establish intent or determine the originality of every experiment and finding in the study.
2. What does the rank-scaling result establish?
The empirical relationship is concrete. In Table 14, standard rank 1 versus rank 128 produces:
These comparisons establish that rank-one adequacy is strongly configuration-dependent under the tested training budget. Prior investigations of rank scaling make this question important; they do not make these controlled outcomes redundant.
We formalize the saturation interpretation with an independently measured continuation headroom,
HT(θ)=Jdev(UTref(θ))−Jdev(θ),
using a fixed full-parameter continuation probe. We then measure rank sensitivity across successive checkpoints from the same model lineage, with identical starting weights for each continuation arm.
A candidate quantitative relationship to test is
JT(r,θ)=JT(rmin,θ)+a(HT(θ))[1−(rrmin)β],β>0,
where the saturation hypothesis predicts that the rank-dependent amplitude a(H) approaches zero as measured headroom approaches zero.
We evaluate ranks {8, 16, 32, 64, 128} at four starting checkpoints across five independent training lineages. Leave-one-lineage-out evaluation hold out all checkpoints and rank outcomes from one lineage together. Prediction errors are computed on the held-out gains at ranks above 8.
Measured gains and model predictions for r128 − r8 :
The headroom-dependent model reduces held-out RMSE by 58% (1.55 → 0.65 pp). The headroom-independent model cannot capture the measured trend: it underpredicts the gain at high headroom by 2.6 pp and overpredicts it near saturation by 1.9 pp. The linear form a(H)=γH underpredicts intermediate checkpoints (residual +1.4 pp at HT=7.5), which suggests that the amplitude saturates sublinearly in headroom. We therefore treat the linear form as a first-order approximation.
This supplies an explicit, falsifiable formulation: independently measured headroom should predict the magnitude of the rank benefit within a model lineage.