- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This study analyzes LoRA performance in a simplified high-dimensional attention model. They consider two stages learning, pretraining followed by LoRA fine-tuning, in a teacher-student setting. They focus on what happens when pretraining is insufficient. Their analysis shows that insufficient pretraining can be represented as effective noise. When pretraining and LoRA training use the same input data, high test performance does not necessarily mean accurate parameter prediction. Based on these findings, they also propose practical heuristics to improve performance.
Critical strengths
- Given that it is certainly a non-trivial question how performance changes through post-training when pre-training is insufficient, the results of this study make a certain contribution to the success background of LoRA. In particular, it is interesting that, given the same inputs, high parameter prediction performance can sometimes coincide with low generalization performance.
- Furthermore, it is interesting that the impact of insufficient pre-training is concentrated into effective noise, and that heuristics to reduce this in downstream tasks (such as selecting data where upstream task impacts do not appear, or optimizing the scale) are shown to be effective. I feel this provides suggestive insights for algorithmic design.
Critical weaknesses
- The main concern with this study is the model setting. The study considers a teacher-student model, specifically assuming that the parameters to be estimated in post-training are truly low-rank, and that the learner knows which subspace to learn. However, the fundamental question regarding practical LoRA seems to lie one step prior to this. That is, why LLMs pre-trained on various tasks are able to generalize to downstream tasks (whose relation to those tasks is non-trivial) through only low-dimensional updates, or under what conditions they fail to do so. From that perspective, the model seems to fail to capture the essential aspects of LoRA. Furthermore, while this study considers the case where the inputs for pre-training and the LoRA task match as one of the scenarios, it was unclear whether investigating this is practically useful.
- I feel there is a lack of experiments. In this work, a significant model simplification has been made where the key and query are the same matrix and the value is not learned, but the justification for this is weak. For example, I would like to check whether qualitatively similar trends can be obtained even when training with full parameters, and whether the heuristics remain effective.
Questions for authors
It is not clear how the results differ from existing transfer learning theory, except for the use of an attention model. For example, the setting where parameters learned in an upstream task are frozen and only a small number of parameters are trained in the downstream task has also been studied in transfer learning for high-dimensional linear regression[1]. Are the phenomena demonstrated here, such as the effect of effective noise and the divergence between parameter reconstruction accuracy and generalization performance caused by input data reuse, specific to Attention or LoRA? Or do they also appear in general two-stage transfer learning?
[1] Dar, Y., & Baraniuk, R. G. (2022). Double double descent: On generalization errors in transfer learning between linear regression tasks.
4 comments
Thank you for the careful response and the additional experiments. The new experiments are meaningful and support the main claims of the paper.
However, I still have concerns about the practical motivation for the input-reuse setting. In my original comment, I asked how studying this setting helps us understand LoRA in practice. I do not think the response fully answers this question.
I think this issue is important because input reuse is behind some of the most interesting results in the paper. In particular, Section 4.2 show a non-trivial phenomenon. Overfitting in pretraining improves parameter recovery on the downstream task, but it makes the test error worse. This gap between test error and parameter recovery is one of the main contributions the paper highlights. But this result depends on the special input reuse setting, and it is not clear what real LoRA situation this setting corresponds to.
Of course, it is natural for theoretical work to simplify real training. I do not think the theoretical model has to match real LLM settings exactly. But for such a simplification to be meaningful, the model should capture the key features of the real problem. At least, the paper should give a convincing reason why studying these features matters.
For example, there may be prior practical work showing that input reuse is hard to avoid in real LoRA, and that it causes problems for training or evaluation. If so, that would strongly support this setting. Practical work showing that the gap between parameter recovery and test error is a real problem would also be good motivation.
For now, I will keep my score. If the authors give a convincing answer on this point, I will raise my score.
Thank you for clarifying the remaining concern, and for recognizing the value of the additional experiments. We would like to further clarify the practical motivation for studying input reuse and its relevance to multi-stage learning and LoRA fine-tuning.
A concrete motivation is task-adaptive pretraining (TAPT). Gururangan et al. [1] continues language-model pretraining on the downstream task's unlabeled training texts before supervised task learning. Thus, the same underlying inputs deliberately participate in two stages with different objectives. LoRA is also used in such workflows: Gu et al. [2] introduce LoRA-based TAPT followed by task classification. These methods are not identical to our model, but demonstrate that recurring task inputs across representation learning and adaptation is a practical design choice, not merely an artificial possibility. Our setting isolates this form of dependence: downstream training inputs have already been encountered during pretraining, whereas evaluation uses unseen inputs. Importantly, multi-stage training does not always improve downstream performance. Shi and Lipani [3] show that conventional TAPT can even be detrimental to subsequent fine-tuning in some settings.
Regarding the gap between parameter recovery and test error, related empirical studies highlight a broader distinction between training objectives and downstream quality. Liu et al. [4] find that language models with comparable pretraining losses can exhibit different downstream performance. They show that to continue pre-training after convergence makes the loss landscape flatter and is beneficial for downstream tasks, which is similar to our overfitting case, with our inverse-curvature order parameter going to infinity. Isik et al. [5] identify machine-translation settings where downstream cross-entropy improves while translation-quality metrics worsen.
We believe our theory provides a possible mechanistic explanation for this broader class of discrepancies: the quality of the learned task-specific component and the test error of the complete predictor need not improve together, because the latter also depends on residual errors inherited from pretraining. Section 4.2 characterizes this separation in a controlled setting with recurring training inputs, where improved recovery of the downstream signal can coexist with worse predictions on unseen data.
We do not claim that the cited empirical findings necessarily arise from our specific mechanism, or that all practical LoRA pipelines exhibit this inversion. Rather, these studies motivate understanding why different measures of learning and downstream performance can disagree. We will clarify this connection and the scope of our model in the paper, including the distinction between input reuse and the additional shared-noise assumption addressed by our independent-noise analysis.
[1] Gururangan et al., Don't Stop Pretraining: Adapt Language Models to Domains and Tasks, ACL 2020.
[2] Gu et al, NLoPT: N-gram Enhanced Low-Rank Task Adaptive Pre-training for Efficient Language Model Adaption, LREC-COLING 2024.
[3] Shi and Lipani., Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner, NeurIPS 2023.
[4] Liu et al., Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models., ICML 2023.
[5] Isik et al., Scaling Laws for Downstream Task Performance in Machine Translation, ICLR 2025.
Thank you for your response. I think these are very important points, and I have raised my score on the premise that they will be reflected in the paper.
We thank the reviewer for the comments and suggestions. We would like to address the weaknesses and questions below.
W1, Q1 : We agree that understanding why natural downstream tasks admit low-rank adaptation is an important question. Our paper addresses a complementary one: given a task-specific low-rank change, how does imperfect pre-training affect the amount of data needed to learn it, its prediction error, and recovery of the underlying task direction?
Importantly, in our setting, the learner is not given the downstream subspace. It optimizes over all w∈RD and must discover the unknown direction; in the untied extension, both factor subspaces are learned. The shared extensive-rank teacher represents transferable structure, while the additional spike represents the task-specific change. The two stages have different statistical scales, N=Θ(D2) and N′=Θ(D), yet both components contribute at order one to the attention scores. The construction therefore isolates how residual uncertainty in a frozen representation interferes with learning new structure from limited data, rather than assuming that the relevant direction is already known. Compared to [1], the key difference of our analysis relies in the different scales between pre-training and fine-tuning stages. This is specific to quadratic attention and LoRA. This allows the decoupling of the two learning problems in high-dimensions, and the subsequent effective noise mechanism and mismatch between parameter reconstruction accuracy and generalization performance with input reuse.
W2 : Our theory and our heuristics are valid in broader regimes. In the revised article we added several extensions.
Following the reviewer's suggestion, we added an experiment with distinct, jointly learned query, key and value matrices, as described in the new Appendix "Extension to complete self-attention". It shows that our effective-noise description holds in this broader setting. We compare a fully pre-trained attention to its effective low-dimensional Gaussian equivalent, that we derive in the Appendix, on both the test error and the test error of a subsequent fine-tuning. The errors of the two models are equal up to the numerical errors. This supports our effective-noise reduction holds for a general attention, beyond tied queries/keys and fixed values.
Following reviewer voya's suggestion, we show that our theory can straightforwardly be applied to standard LoRA: in the revised Appendix A8 "Extension to a generic untied low-rank update" we consider an attention with a general low-rank update uw⊤ (not necessarily rank one), as well as fine-tuning data with non-symmetric spikes u⋆(w⋆)⊤. We extend all our theoretical results of part 3 to this broader setting. The effect of the pre-training is caught by the same effective noise mechanism.
Last, we performed an experiment on non-Gaussian data from cifar10. It is described in the revised Appendix "Supplementary figures", paragraph "Numerical extension to non-Gaussian data". We are able to qualitatively reproduce the phenomenology we observe when reusing sequences Fig. 3, i.e. to observe a mismatch between test error and reconstruction. This suggests that our theory and interpretation can extend to more realistic settings.