- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the convergence theory of LoRA (low-rank adaptation, ΔW = BA) under generalized smoothness, removing two assumptions common in prior work: bounded LoRA factor norms and global L-smoothness of the weight-space objective F. The latter is motivated empirically by the observation that Hessian norms tend to grow with gradient norms during neural network training. The analysis must therefore control the weight-gradient norm and the factor norms jointly (gradient–factor coupling), and, in the stochastic case, handle factor-amplified noise: a bounded-variance weight-gradient oracle induces factor-gradient noise with conditional second moment up to σ²R_t, where R_t is the factor scale.
Critical strengths
The theoretical analysis is sound and well-founded, and the extensive experiments are conducted with rigor.
Critical weaknesses
None
Questions for authors
None
We sincerely thank the reviewer for the careful assessment and positive evaluation of our work. We appreciate your recognition of the soundness of the theoretical analysis and the rigor of the experiments. We also acknowledge your assessment of the presentation and will further improve the explanation of the main proof ideas and the theoretical results. Thank you again for your thoughtful review. We would be happy to address any additional questions or provide further clarification.