- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes READ (Read-only Expansion of Adapter Deltas), a method for sequentially composing independently trained LoRA adapters into a single model. READ addresses two aspects of LoRA composition: the non-uniqueness of adapter factorization and the directionality of interactions between existing and incoming skills. The evaluation covers Llama-3.2-3B-Instruct and Qwen3-4B on GLUE-6, SuperGLUE-4, Domain-3, and BBH-6.
Critical strengths
- The paper identifies two distinct issues: equivalent factorizations of an individual update can induce different interaction coordinates, and the direction of cross-skill coupling determines which components of the composed update can change.
- Many evaluation results on append-wise analysis, factorization-sensitivity experiment, etc.
Critical weaknesses
- The read-only constraint does not guarantee preservation of old-task functions.
- Analyse the main reason of improvement on these experiments. Eg. why are the canonicalization and directional constraints design useful?
Questions for authors
-
What exactly does the read-only constraint preserve?
-
To what extent are READ's performance gains attributable to its structural constraints rather than training on the union of old and new task data?
-
How is incoming closure defined, and what criteria determine whether a transition is eligible?
-
What evidence supports the authors' interpretation of the negative GLUE macro scores, and how were the reported confidence intervals computed?
-
How does READ differ from recent methods such as LoRAGen (ICLR 2026) and DWF (ICLR 2026), particularly in terms of the composition mechanism?