- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Residual-Channel Adaptation (RCA), a Fisher-guided low-rank adaptation method for RLVR. Motivated by the local KL geometry of policy optimization, the authors argue that avoiding high-Fisher-curvature directions can improve reward gains under a fixed policy-change budget. RCA uses a calibration-time Kronecker-Factored Approximate Curvature (KFAC) approximation to identify the principal input and output Fisher directions of each adapted layer. It then constrains LoRA updates to the bilateral residual subspace through orthogonal projections, while preserving the low-rank parameterization. The paper first derives conditions under which Fisher-residual updates achieve higher reward improvement per unit KL cost. Then, it empirically validates the claimed performance across different language models on several benchmarks.
Critical strengths
- Simple and effective method: The bilateral projection is straightforward to implement, preserves the low-rank structure, and requires no modification to the underlying RLVR optimizer.
Critical weaknesses
Critical Weaknesses
-
Insufficient empirical evidence. Standard LoRA exhibits near-zero performance on several reasoning benchmarks, raising serious concerns about the reliability of the baseline implementation and training configuration. RCA also shows negligible improvements over stronger geometry-aware PEFT methods (e.g., only 0.8 percentage points over GeoRA). Furthermore, the paper omits several relevant PEFT baselines, including MiLoRA [1], DoRA [2], LoRA-RLPO and LoRA-RLMO [3], and MiSS [4], making it difficult to establish the proposed method's advantage over existing approaches.
-
Gap between theory and the proposed algorithm. Lemma 3.1 and Theorem 3.7 establish KL-efficiency guarantees for linear update subspaces without considering rank constraints. However, RCA restricts updates to a low-rank LoRA parameterization, for which these guarantees do not necessarily hold. The theoretical results therefore do not directly justify the proposed algorithm.
-
Figure readability. Figures 2 and 3 are difficult to read due to the small font sizes, crowded plots, and close curves.
References
[1] Wang, H., et al. (2025). MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning. NAACL 2025.
[2] Liu, S.-Y., et al. (2024). DoRA: Weight-Decomposed Low-Rank Adaptation. ICML 2024.
[3] Zhang, R., Zhu, J., Zhu, H., & Shi, L. (2026). Geometry-Preserving Orthonormal Initialization for Low-Rank Adaptation in RLVR. ICML 2026.
[4] Yin, Q., et al. (2026). Evaluating Parameter Efficient Methods for RLVR. ICML 2026.
Questions for authors
I genuinely hope that these questions can help the authors improve this paper.
1. Baseline configurations and training stability. A fair comparison should not simply use identical hyperparameters for all methods, but should ensure that each baseline is evaluated under properly tuned configurations with comparable tuning budgets. Prior studies have demonstrated that cosine learning-rate decay can substantially stabilize MiLoRA and partially mitigate the instability of PiSSA [1]. Moreover, Mind Lab [2] showed that standard OLoRA can suffer training collapse under RLVR, whereas OLoRA-tail, which uses minor singular directions without singular-value scaling, achieves stable training. Standard LoRA has also been shown to train reliably under RLVR [1, 3, 4], making its near-zero performance in this paper particularly concerning. Could the authors clarify whether the baselines were independently tuned, particularly their learning-rate schedules, and other method-specific hyperparameters? The authors should carefully revisit their baseline configurations to ensure a fair comparison.
2. Sensitivity to response length. The maximum training response length is limited to only 3,072 tokens, substantially shorter than the 16K-token budgets used in recent RLVR studies [1, 3]. While RCA may perform better under this restricted budget, it remains unclear whether its advantage persists with longer reasoning trajectories. Notably, the failed baselines in Table 2 reach the maximum response length, suggesting that truncation may contribute to their poor performance. Could the authors clarify the maximum response lengths used during both training and evaluation, and report the corresponding truncation rates? More importantly, could they evaluate RCA and its baselines under longer response budgets (e.g., 8K or 16K tokens) to verify whether the reported improvements persist?
3. Theoretical guarantees under low-rank constraints. Mentioned in weakness 2, could the authors clarify whether the theoretical KL-efficiency advantage continues to hold under this rank constraint, or provide additional theoretical?
4. Consistency of LoRA baselines. What is the difference between the standard LoRA baseline in Figure 2 and the "Shared-config LoRA" baseline in Tables 11 and 14? Standard LoRA achieves near-zero performance on several benchmarks in Figure 2, whereas Shared-config LoRA achieves 24.2% pass@1 in the appendix. Please explain this discrepancy and share detailed training logs. Again, it is really counter-intuitive that LoRA will collapse in training. Please check and share the training implementations.
5. Missing citations and insufficient discussion of principal subspaces. The paper overlooks several relevant PEFT studies, including MiLoRA [1], OLoRA-tail [2], LoRA-RLPO/RLMO [3], and MiSS [4]. More importantly, different studies adopt substantially different definitions of principal and off-principal directions [1,2,3,5], yet the paper does not adequately discuss these distinctions.
References
[1] Zhang, R., Zhu, J., Zhu, H., & Shi, L. (2026). Geometry-Preserving Orthonormal Initialization for Low-Rank Adaptation in RLVR. ICML 2026.
[2] Mind Lab. (2026). On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters.
[3] Yin, Q., et al. (2026). Evaluating Parameter Efficient Methods for RLVR. ICML 2026.
[4] Schulman, J., & Thinking Machines Lab. (2025). LoRA Without Regret.
[5] Zhu et al. (2025). The Path Not Taken: RLVR Provably Learns Off the Principals.