- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LAMCC, a framework that leverages semantic information from an auxiliary language to enhance performance in multimodal metaphor detection. The method extracts shared semantics and language-specific information through cross-lingual semantic decoupling, improves metaphor prediction via self-attention-based fusion and adaptive layer selection, and simulates auxiliary language representations during inference using linear mapping. Experiments conducted on two datasets across four vision-language models demonstrate strong performance.
Critical strengths
- The paper explores an interesting direction for multimodal metaphor detection by incorporating information from an auxiliary language. The motivation is reasonable and relevant to the task.
- The proposed method combines semantic disentanglement, attention-based fusion, and adaptive layer selection. The semantic simulation module is also useful, as it avoids requiring translations at inference time.
- The experiments cover two datasets and four VLM backbones, with consistent improvements over SFT. The ablation results also help demonstrate the contributions of the main components.
Critical weaknesses
- The paper does not clearly describe how the training, validation, and test sets are split, or how the Chinese and English samples are organized. More details are needed to ensure reproducibility.
- Although LAMCC achieves the best results, the improvements over some strong baselines are relatively small. For example, the gains over Semantic Disentangled CC-Tuning on MultiMM are only 0.68% and 0.24%. The paper also lacks results from multiple random seeds, making it difficult to assess the significance of these improvements.
- The paper does not provide enough evidence that the improvements come from complementary cross-lingual semantics. Additional comparisons with translation-augmented fine-tuning would help support this claim.
- The formulation of the Decision Maker in Eq. (10) is somewhat unclear. Since the representations from different layers are averaged before computing the layer weights, it is unclear how the model identifies the importance of individual layers.
- The semantic simulation module is not sufficiently evaluated. A comparison between real auxiliary representations, predicted representations, and directly reused main-language representations would help clarify its actual contribution. 6.The paper lacks sufficient visualization and analysis of its key components. Although Figure 3 provides some case studies and layer-wise fusion weights, it does not clearly demonstrate the effectiveness of semantic disentanglement, adaptive layer selection, or semantic simulation. More detailed representation visualizations and layer-wise analyses would help support the proposed mechanisms. In addition, most of the case studies focus on Chinese examples, while examples in English are limited.
Questions for authors
- Could the authors provide more details about the training, validation, and test splits? How are the Chinese and English samples organized, and how is data leakage between the splits avoided?
- The improvements over some strong baselines are relatively small. Could the authors report results across multiple random seeds to demonstrate the stability of these improvements?
- How can the authors demonstrate that the performance gains come from complementary cross-lingual semantics rather than additional training data or model complexity?
- In Eq. (10), the representations from different layers are averaged before computing the layer weights. How does the Decision Maker distinguish the importance of individual layers?
- Could the authors compare the performance using real auxiliary-language representations, simulated representations, and directly reused main-language representations to better evaluate the semantic simulation module?