- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes MOCA-MLLM, a framework for studying modality injection order during multimodal LLM fine-tuning. The work aims to examine whether injecting text, vision, and audio at different decoder depths affects downstream performance. The authors first extract modality features using frozen text, vision, and audio encoders. Then they project and sequentially inject these features at three decoder depths according to all six possible modality orders. They further compare sequential injection with an All-at-Input baseline. Experiments on four MLLM backbones, six classification tasks, and twelve datasets report task-dependent performance differences among injection orders.
Critical strengths
- The motivation of this work is inspiring: whether modality injection order affects downstream MLLM adaptation.
- The writing style is good and easy to follow.
Critical weaknesses
- The experimental setting is confusing. The evaluated MLLMs already have pretrained modality encoders and connectors aligned with their backbones, but MOCA replaces them with external MiniLM, CLIP, and Wav2Vec2 features and newly added projections. Since these representations were not jointly pretrained with the backbones, it is unclear whether the original multimodal capabilities remain valid; the results may reflect representation-space mismatch rather than modality order. Comparisons using the native encoders are needed.
- The paper is highly similar to "Order Matters: Exploring Order Sensitivity in Multimodal Large Language Models." Although this work is cited, the main extension appears to be moving from inference-time context ordering to depth-wise modality injection during fine-tuning. The authors should clearly distinguish the two works and justify why the contribution is more than an incremental extension.
- The comparison is limited to All-at-Input. The paper lacks comparisons with native MLLM fine-tuning methods, such as LoRA, making it difficult to assess the practical value of MOCA-MLLM.
- The injection layers are fixed at approximately L/3 and 2L/3. Simply dividing the decoder into three equal parts may not be optimal. Ablation studies with different injection depths and spacing strategies are required.
Questions for authors
See weaknesses. If the authors can adequately address these concerns, especially weakness 1 and 2, I would consider raising my score.