- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses multimodal adaptation by replacing a single LoRA adapter shared across all modalities with modality-specific LoRA adapters, while retaining global self-attention for cross-modal fusion. The authors evaluate the approach using Llama 3.2-3B, training separate image–text and audio–text models.
Critical strengths
- Clear and meaningful research problem. Effective multimodal adaptation with a frozen pretrained LLM is a practically valuable research direction.
- Simple and intuitive method. MoL introduces modality-specific low-rank updates into the attention projections while preserving global self-attention, supporting both modality specialization and cross-modal interaction. It requires neither a learned router nor modifications to the backbone weights and extends naturally to DoRA.
- Broad experimental coverage. The paper evaluates the proposed method on 21 image–text benchmarks and three audio benchmarks, providing extensive empirical evaluation.
Critical weaknesses
- Limited methodological novelty. The core design primarily applies MoT’s modality-specific parameterization to LoRA, with adapters selected deterministically based on known modality identities. The paper should clarify its distinct technical contributions relative to existing multi-adapter and LoRA expert methods.
- Modest overall gains and substantial variation across tasks. Although MoL and MoD improve the overall average score, performance declines on several tasks. For example, MoL underperforms shared LoRA on the OCR category, CVBench, VSR, and TallyQA.
- Limited evidence of generalizability and scalability. Experiments are conducted only on Llama 3.2-3B and involve pairwise modality combinations. This leaves the method’s applicability to larger models and settings with more modalities insufficiently established.
Questions for authors
- Have the authors sufficiently reviewed existing work on LoRA-based multimodal adaptation? Could the relevant work be discussed more systematically, with clearer distinctions between the proposed approach and prior methods?
- The main text describes the computational overhead as negligible, whereas the appendix reports per-step latency increases of 28.0% and 40.2% for MoL and MoD relative to LoRA. Can the authors provide end-to-end timing results to substantiate this claim?
- When the image and text adapters use the same scaling factor , does the image adapter still exhibit a larger update norm? Is this asymmetry associated with the observed performance gains?