- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies how perturbations to a reference weight change the gradient-to-update map of spectrally initialized low-rank adapters. It decomposes perturbations into selected, complementary, and cross-subspace components, and argues that preserving selected subspaces need not preserve local update behavior.
Critical strengths
The paper raises a potentially useful distinction between preserving a selected subspace and preserving the resulting update behavior.
Critical weaknesses
The paper is hard to follow, and this is a substantive obstacle to evaluating its contribution.
-
The abstract introduces “spectrum-weighted map,” “spectral allocation,” “subspace–update gap,” “paired upstream updates,” and “natural-magnitude component interventions” in rapid succession, without first giving a concrete example of the question being tested. Section 3 then adds “SplitProbe” and several response measures. These labels often replace a plain explanation of what changes, what is held fixed, and what each experiment is intended to show. I had to reconstruct that logic from the equations and appendix.
-
The abstract blurs two distinct results. It highlights the – update angles from the factor-redistribution experiment (Section 4.6). The naturally delivered perturbations in Section 4.5 produce much smaller ten-step separations: 0.0076%–0.0609% of the reference update for DeBERTa and 0.0054%–0.251% for RoBERTa. The paper should distinguish these settings and their implications more clearly.
-
Numerical precision is inconsistent and excessive. Section 4.4 reports errors as 0.00014%–0.220%, while Appendix Table 11 includes values such as 0.0001378% and 0.220200%. Such precision for means over ten seed aggregates, without uncertainty in the same table, is hard to justify. Consistent rounding would make the results easier to interpret.
Overall, the paper may contain a good mathematical observation, but its terminology, organization, and numerical presentation make it unnecessarily hard to tell which conclusions are proved, which are observed under controlled probes, and which remain proposed applications.
Questions for authors
I encourage the authors to substantially improve the academic writing, especially by defining terms only when needed, explaining each experiment’s purpose before presenting its metrics, and clearly separating results from different experimental settings.