acceptodds
Withdrawn from ICLR 2027

What Is Reusable Across LoRA Runs? The Base-Conditioned Core as a Relational Unit

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

PEFT makes task adaptation modular: a small LoRA update can be trained once, stored in an adapter library, and transferred or composed without retraining the full model. This workflow raises a practical question: when independently trained updates move to a new run, what preserves their function? We answer that reuse is carried by a base-conditioned core–base relation jointly determined by the donor–target frame relation, the learned core, and the target base. We introduce Bilateral Target-Frame Functional Registration (BTF-FR), which expresses a donor core in the target frame and evaluates the resulting dense intervention in the target model. Functional controls show that registered interventions recover same-task behavior, while raw splicing and spectrum- or norm-matched controls reveal the specific contribution of registration; related task and interface conditions produce an ordered response. BTF-FR therefore provides a functional and auditable criterion for organizing cross-run adapter reuse around donor–target relations and target bases.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
PEFT, LoRA, adapter reuse, cross-run transfer, functional registration, target-frame alignment, gauge invariance, Grassmann geometry, model merging, base-conditioned core

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_CDYhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates what functional information can be reused across independently trained LoRA runs. The authors introduce Bilateral Target-Frame Functional Registration (BTF-FR) with orthogonal Procrustes mapping to transport a donor core into target frames. This method also validates different LLM models and demonstrates compelling performance. However, I find this paper's entire scope and novelty are highly limited, and I don't think I am convinced by the experiments from this paper's experiments part, and further clarification and explanation are highly demanded. The main construction is understandable, but the distinction between pretrained model weights, update subspaces, and the target base needs greater precision. Several important experimental conditions remain operationally underspecified.

Critical strengths
  1. Using orthogonal is a simple but efficient choice for investigating LoRA's update despite it having been widely used for all kinds of LoRA research, especially for parameter-efficient continual learning and model merging.
  2. The design for this paper is understandable and easy to follow.
Critical weaknesses
  1. There are many papers about this LoRA updating, and how does your performance compare to those models? KnOTS [1] already transforms independently trained LoRA updates into an aligned space through joint SVD. GAM [2] explicitly aligns both left and right singular bases using orthogonal Procrustes transformations, making it particularly relevant to the bilateral registration in BTF-FR. TSPA [3] further studies low-rank rotation and scaling with separate functional and parameter alignment objectives. Additionally, W2T [4] uses similar core-based merging. I have no idea how your method is any different compared to these, and this paper seems incremental.

[1] Stoica, George, et al. “Model Merging with SVD to Tie the Knots.” ICLR, 2025. [2] Sun, Qi, et al. “From ‘Weak’ Signals to Strong Models: Preference Delta Aggregation with LoRA Merging.” arXiv, 2026, arXiv:2606.00357. [3] Li, Zijian, et al. “Two-Stage Parameter Alignment for Multi-LoRA Merging in Large Language Models.” ACL. [4] Han, Xiaolong, et al. “W2T: LoRA Weights Already Know What They Can Do.” arXiv, 2026, arXiv:2603.15990.

Questions for authors
  1. There are many papers about this LoRA updating, and how does your performance compare to those models?
  2. What is the scalability of your method?
  3. What's the benefit of your method compared with other merging methods?
  4. Have you tried QR decomposition in your updating? If not, how's its core matrix different than your core-based combination?
Reviewer_UvzdOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I believe the motive behind this paper is to pursue a line of interpretable, explainable AI in terms of low-rank adapters. However, this was not done adequately.

Critical strengths

None

Critical weaknesses

This is, unfortunately, not a well written work. For starters, the abstract makes no sense - abbreviations like PEFT and LoRA are used outright without even an effort to explain what each of them represent. It's not understandable what workflow is being referred to right in the abstract. What is meant by a new run? What is the function being referred to here? This sentence from the abstract is unintelligible - "We answer that reuse is carried by a core base relation jointly determined by the donor-target-frame relation, the learned core, and the target base." The next few sentences are completely unintelligible as well. I would recommend the authors, at the very least, show it to an advisor or somebody who has published in this area, and at least have them read the abstract and introduction.

The Figure 1 caption is again completely incorrigible. It's not understandable what runs mean here. What does conditioning on update location mean? What does Grassmann update location space even refer to without any mathematical or intuitive understanding? What does reusable object even mean?

The explanation of the problem statement is utterly confusing. From line 54 "the practical promise is substantial: a capability learned once could be moved across runs and tasks, reducing retraining cost and enabling modular reuse and composition. We test this promise directly. Raw splicing can fail to recover stable task selectivity even when donor and target adapters were trained on the same task, suggesting that functional reuse cannot be reduced to copying an isolated update and instead depends on its relation to the target model..." This makes little to no sense.

The paragraph following is even more depressingly unintelligible - "This leads to a direct research question: What is reusable? Specifically, we ask whether the reusable object across runs should be defined as a raw adapter, a normalized core, or a functional relation conditioned on the target base. This question yields a falsifiable prediction: after target-frame registration, same-task relations should recover the most target functionality; relations with the same output interface but different tasks should retain an intermediate response; and relations with different tasks and interfaces should approach Zero-Core. The prediction turns “what is reusable?” into a measurable donor–target compatibility relation rather than a similarity between isolated parameters."

The appendix is an LLM-written mess, which is not human-readable. It largely consists of training logs and stuff, which is unclear why they are in this paper.

It is unclear what exactly the problem statement is that the authors are confusedly trying to pursue in this paper. Are they trying to figure out what parts of low-rank adapters can be reused, or whether the low-rank modules themselves can be reused? In both cases, we have some sort of prior answers. For the first one, we have the paper and research along the lines of papers like EigenLorax[1] and for the next one, we have something like LoraHub[2], etc. None of these lines are adequately discussed, or this work is separated from them.

[1] P. Kaushik, A. Vaidya, S. Chaudhari and A. Yuille, "EigenLoRAx: Recycling Adapters to Find Principal Subspaces for Resource-Efficient Adaptation and Inference," 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops

[2] Huang, Chengsong, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du and Min Lin. “LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition.” ArXiv abs/2307.13269 (2023): n. pag.

Questions for authors

please rewrite the paper and have somebody who works in this research area have a read before submitting.

Reviewer_8WM9OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes an approach for transferring LoRA adapters from a source model to a related target model from a different run, a different model scale, or a different architecture. To that end the approach determines a low-rank approximation of the adapter, takes its singular value matrix (or, indeed, a matrix related to it), and aligns it to weights of the target model by aligning the space of the left and right singular vectors of the two spaces via an orthogonal transformation each. The orthogonal transformations are recovered by minimizing the Frobenius norm between the aligned singular vectors. An additional normalization step is performed to ensure that the norms of the singular vectors in the two spaces are comparable. Experiments are performed on eight tasks from the SuperNI benchmark, with various Qwen models as source and target models. Additionally, the transferability to Llama-3.2 is evaluated.

As described below in more detail, limited novelty, incomplete experimental evaluation, and the extremely opaque writing make the paper not suitable for publication in its current form.

Critical strengths
  • The alignment of LoRA adapters or other parameter-efficient fine-tuning methods between a source and a target model is a relevant and useful problem that is being addressed by this paper.
Critical weaknesses
  • The novelty of the paper is extremely limited. The alignment of SVD subspaces between source and target weights using the minimization of the Frobenius norm of the alignment of the left and right singular vectors of the source and target weights has been proposed already in Cross-LoRA (not cited, see below). One difference here is that the transformation matrix is constrained to be orthonormal, which is not the case in Cross-LoRA. Another difference is that this paper normalizes the transformed update. However, critically neither of these two differences is experimentally evaluated. As far as I could see, the Cross-LoRA paper is only on arXiv and not yet been published in a peer-reviewed venue, which makes it debatable whether this should count as prior work. However, given that Cross-LoRA is already over 1 year old and has been compared to in other published papers (e.g., Hochlehner et al., CATS Workshop at ICML 2026), I still believe that this constitutes prior work that should be cited and discussed.
  • The experimental evaluation is incomplete. The paper only compares to fairly naive baselines such as only aligning the left or the right singular vectors, directly splitting the weights, random orthogonal transformations. Relevant comparisons to prior work in the very same area are absent. This includes CrossLoRA, as mentioned above, as well as cited work such as KnOTS, Git Re-Basin, TIES, DARE, and more. Given this incomplete comparison, it is unclear whether the proposed approach yields any benefits over prior work.
  • The paper is extremely difficult to read. It looses itself in undefined or poorly defined terminology and dresses up a relatively simple idea (which, e.g., Cross-LoRA explains in clear and simple terms) in an unnecessarily complex way using Grassmann spaces, fibers, and the like. The benefit of the latter remains unclear. To give some examples of other clarity issues:
    • The notions of core, core-base relations, frames, registered interventions, or ordered responses (all from abstract) are not really defined or well explained. Other notions of zero-core (l. 65), audibility (l. 73), and auditable registration record (l. 148), main matrix (l. 245), or why a relational object (l. 81) is being built also remain unclear.
    • The notation is ambiguous. A and B serve as notation for the target and source models (l. 147), but also for the decomposition of the LoRA adapters (Eq. 2). Similarly, at least according to my understanding, S stands for the singular vectors of the source/target weights but also for the rank-r approximation of the PEFT update.

[Cross-LoRA] F. Xia, M. Liao, Y. Fang, D. Li, Y. Xie, W. Li, Y. Li, D. Xia, J. Huang. “Cross-LoRA: A Data-Free LoRA Transfer Framework across Heterogeneous LLMs.” arXiv:2508.05232, 2025.

Questions for authors
  • What is the benefit of the proposed approach over CrossLoRA and the other cited works, both conceptually and experimentally?
  • What is the benefit of framing the proposed approach in the context of Grassmann spaces, fibers, etc.? What does this framing add?
  • Can you explain the unclear notions named above?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.