- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates what functional information can be reused across independently trained LoRA runs. The authors introduce Bilateral Target-Frame Functional Registration (BTF-FR) with orthogonal Procrustes mapping to transport a donor core into target frames. This method also validates different LLM models and demonstrates compelling performance. However, I find this paper's entire scope and novelty are highly limited, and I don't think I am convinced by the experiments from this paper's experiments part, and further clarification and explanation are highly demanded. The main construction is understandable, but the distinction between pretrained model weights, update subspaces, and the target base needs greater precision. Several important experimental conditions remain operationally underspecified.
Critical strengths
- Using orthogonal is a simple but efficient choice for investigating LoRA's update despite it having been widely used for all kinds of LoRA research, especially for parameter-efficient continual learning and model merging.
- The design for this paper is understandable and easy to follow.
Critical weaknesses
- There are many papers about this LoRA updating, and how does your performance compare to those models? KnOTS [1] already transforms independently trained LoRA updates into an aligned space through joint SVD. GAM [2] explicitly aligns both left and right singular bases using orthogonal Procrustes transformations, making it particularly relevant to the bilateral registration in BTF-FR. TSPA [3] further studies low-rank rotation and scaling with separate functional and parameter alignment objectives. Additionally, W2T [4] uses similar core-based merging. I have no idea how your method is any different compared to these, and this paper seems incremental.
[1] Stoica, George, et al. “Model Merging with SVD to Tie the Knots.” ICLR, 2025. [2] Sun, Qi, et al. “From ‘Weak’ Signals to Strong Models: Preference Delta Aggregation with LoRA Merging.” arXiv, 2026, arXiv:2606.00357. [3] Li, Zijian, et al. “Two-Stage Parameter Alignment for Multi-LoRA Merging in Large Language Models.” ACL. [4] Han, Xiaolong, et al. “W2T: LoRA Weights Already Know What They Can Do.” arXiv, 2026, arXiv:2603.15990.
Questions for authors
- There are many papers about this LoRA updating, and how does your performance compare to those models?
- What is the scalability of your method?
- What's the benefit of your method compared with other merging methods?
- Have you tried QR decomposition in your updating? If not, how's its core matrix different than your core-based combination?