- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes CoMoR, a conflict-aware modality routing framework for adapting frozen large language models to multimodal sentiment analysis. Shared Sentiment Diagnosis produces modality-specific distributions over ordered sentiment anchors using a shared frozen LLM and a shared trainable head. A router receives these distributions, their pairwise Wasserstein-1 distances, and predictive entropies, and scales audio-visual pseudo-token blocks before joint prediction with the original text. Training combines a generative task loss with auxiliary diagnostic supervision. Experiments cover four English and Chinese sentiment benchmarks, with component ablations, alternative routing designs, multiple backbones, and computational analyses.
Critical strengths
- A coherent approach with promising empirical improvements. CoMoR uses modality-specific predictions to modulate continuous inputs while preserving the original evidence for joint prediction. The reported MAE improves over the reproduced MSE-Adapter baseline on all four datasets, including 0.672 to 0.581 on CMU-MOSI and 0.395 to 0.349 on CH-SIMS. Improvements across English and Chinese benchmarks make the approach practically interesting.
- Useful controls and detailed implementation disclosure. The paper evaluates shared versus independent diagnostic heads, alternative disagreement measures, uniform and learned global gates, and feature-driven routing. The appendix reports variability across training seeds and specifies prompts, masking, gradient flow, and numerical decoding. These details make the method easier to assess and reproduce.
Critical weaknesses
- The evidence connecting the diagnostics to effective conflict-aware routing remains limited. All diagnostic branches are supervised using the same multimodal label. Moreover, the text diagnostic branch pools non-contextual token embeddings before the LLM, making it invariant to token order (Appendix A.1). Consequently, disagreement can reflect limitations of the diagnostic branches rather than useful differences in sentiment evidence. The paper appropriately acknowledges that these distributions cannot identify the correct modality, but stronger empirical validation is needed to establish when they provide useful routing signals. Table 13 offers supporting evidence, although it uses one seed and groups defined by the full model itself. The case study also does not isolate the effect of routing, since the final LLM receives the original ordered text and may correct the misleading diagnosis through that path.
- The comparisons do not fully establish the contribution relative to recent LLM adaptation methods or its computational cost. The reproduced MSE-Adapter comparison is valuable, but Table 1 omits the cited Prototype-as-Prompt and MGSI methods, which are directly relevant to the paper’s positioning. Existing routing controls support the design, but do not provide a parameter- or compute-matched comparison with a stronger adapter baseline. Relative to MSE-Adapter, trainable parameters increase from 2.61M to 6.72M, training time increases by approximately 1.95 times, and inference latency increases by 21.8%. These costs make stronger comparisons important. Furthermore, Table 2 compares CoMoR across backbones without the corresponding baseline on each backbone, establishing compatibility more directly than consistent benefits from the proposed mechanism.
Questions for authors
- Diagnostic validity: Can the modality-specific annotations available in CH-SIMS or CH-SIMS v2 be used for evaluation only to examine whether diagnostic disagreement corresponds to independently annotated sentiment differences? This would help distinguish useful modality disagreement from diagnostic errors without changing the training protocol.
- Routing mechanism: Does the high-conflict advantage in Table 13 persist across the other training seeds? A fixed-checkpoint comparison with unit audio-visual scaling or shuffled diagnostic descriptors would also help isolate the contribution of sample-specific routing. 3.Comparisons: Are comparable results available for Prototype-as-Prompt and MGSI under the same splits, features, and metric conventions? If their settings are incompatible, please clarify the specific differences. Is there evidence that the improvement persists against an adapter baseline with a comparable parameter or training-compute budget?
- Backbone dependence: For the alternative LLMs in Table 2, what is the gain over the corresponding MSE-Adapter or uniform-gate baseline? These comparisons would materially affect the assessment of effectiveness across backbones.