- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces MIRS, a benchmark containing 11,000 image–text pairs with hierarchical annotations for multimodal sarcasm detection, including sarcasm labels, modality attribution, and explanations. It also proposes IRAC, a retrieval-augmented framework comprising textual, visual, and cross-modal reasoning agents, coordinated by an orchestrator trained with GRPO. The proposed reward combines prediction accuracy, confidence, attribution, and consistency. Experiments on MIRS evaluate both classification performance and explanation quality.
Critical strengths
The main strength is the potentially valuable dataset contribution. MIRS extends conventional binary sarcasm detection with modality attribution and source-specific rationales, which could facilitate more detailed analysis of cross-modal incongruity and model failure modes. Its collection from Weibo, Reddit, and existing datasets potentially provides broader linguistic and cultural coverage, while the reported filtering and human verification procedures are useful efforts toward ensuring data quality.
Critical weaknesses
- The performance advantage is limited: IRAC achieves 86.92% accuracy compared with 86.08% for GPT-4o with retrieval augmentation, while IRAC without GRPO achieves only 85.79%. Without explicit variability estimates or significance tests, these results do not convincingly establish the benefit of the multi-agent design. 2. The comparisons do not adequately control for backbone strength or inference budget. The specialized agents use GPT-5.4, but no standalone GPT-5.4 baseline with the same retrieval context is included. Multiple model calls and additional training further complicate attribution of the gains. 3. The methodological novelty appears limited. The framework largely combines widely used techniques, including retrieval augmentation, prompt-based agent specialization, and GRPO, without a sufficiently distinctive motivation or task-specific design. The paper also does not clearly establish what generalizable methodological insights this combination offers to the broader multimodal reasoning or multi-agent learning community.
Questions for authors
- Have you considered the relationship between sarcasm, irony, and humor when constructing MIRS? How are these concepts distinguished, and are humorous but non-sarcastic examples explicitly included to prevent visual absurdity or playful incongruity from being mistaken for sarcasm? 2. Could you clarify the annotation procedure? Although Appendix A.6.4 describes LLM-generated drafts followed by human verification, the main text provides limited detail. Which model generated the drafts, how many annotators reviewed each sample, were judgments made independently, how were disagreements resolved, and what was the inter-annotator agreement for sarcasm labels and modality attribution? 3. How does IRAC compare with standalone GPT-5.4 using the same knowledge base, and with a single-model multi-call approach under a comparable inference budget?