- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper examines multimodal cascades in cases where the required textual claim is in conflict with the image, thus making 'Conflict' the appropriate output. PAVIR maintains the claim and source requirements, consults an image-only expert when disagreements are detected, and uses a fixed rule to generate 'Answer', 'Conflict', or 'RequestEvidence'. It achieves improvements on composite versions of ScienceQA, MMStar, and MMMU-Pro. On MMStar, complete-decision accuracy rises from 84.83% to 88.67% on 399 expert calls.
Critical strengths
- The factorial study isolates the interaction between source visibility and the final decision rule. Hiding the text helps when 'Conflict' is a valid output, but hurts under answer-only adoption. I think this is a useful and well-controlled finding.
- The paper reports repairs and harms, answer coverage, and conditional answer error alongside aggregate accuracy. Its matched-budget comparisons also help clarify what the acquisition policy contributes.
Critical weaknesses
- The paper does not establish a sufficiently distinct contribution over existing cross-modal conflict work. CLASH [1] already treats conflict as a correct output, evaluates conflicting and non-conflicting cases, and studies improving conflict detection through fine-tuning. PAVIR's distinct addition is a cascade that selectively calls an image-only expert and then compares its answer with a retained claim. However, this amounts to a score-based gate followed by a deterministic option comparison. The appendix's CLASH result does not resolve the novelty question: the paper notes that it includes output-space effects and that conflict-only evaluation can favor always predicting 'Conflict'. I suggest the authors explain more clearly what new capability or general insight this selective-revision procedure provides beyond the existing conflict-detection formulation.
- The evidence for greater applicability is limited. The main results use constructed multiple-choice overlays where claims map to answer options, so source comparison reduces to matching option IDs. On M2CQA, where claims are free-form, source comparison adds only 0.08–0.16 percentage points for the stronger callers (Table 9). On MMStar, answer coverage increases from 25.50% to 31.83%, while conditional answer error also increases from 14.38% to 15.45%, as shown in Table 7. I think these results support a narrower conclusion about the constructed task and its answer-error trade-off than a general claim about multimodal cascades.
References
- [1] Popordanoska, et al., "CLASH: A benchmark for cross-modal contradiction detection." CVPR. 2026.
Questions for authors
- Does the Global uncertainty baseline in Table 3 exclude cases with missing required sources? Could the author report results after excluding those cases while still allowing calls on agreement cases, keeping the uncertainty ranking, revision rule, and call budget fixed?