acceptodds
Withdrawn from ICLR 2027

PAVIR: When Should Multimodal Cascades Revise, Not Adopt?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

What cascades treat as progress, a stronger visual answer, rests on an untested assumption: that the expert's answer is the decision the task scores, even when a required claim can make Conflict correct. We test that assumption on public ScienceQA, MMStar, and MMMU-Pro items with aligned, conflicting, and missing claims, separating who sees the text from whether the expert writes a full answer. keeps the claim and the source requirements, calls an image-only expert only when the sources already disagree, and recomputes Answer, Conflict, or RequestEvidence with one rule. With the rule fixed, isolating the observer raises ScienceQA accuracy by 9.66 points, and the gain sits in the conflict cells; the same isolation lowers accuracy when every output must be an answer. On MMStar, disagreement-gated revision reaches 88.67% at 399/1,200 calls (61 repairs, 15 harms), from 84.83% with no calls; a 256-token 32B model that sees the claim reaches 87.25%, and 88.50% only after the same rule is applied (65 repairs, 21 harms; paired interval ), while calling all 494 eligible disagreements yields 88.58%. The comparison rule, not the ranking score or the size of the generator, is what makes the call useful, and only when Conflict is a legal output.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal model cascades, vision-language models, image-text conflict, selective prediction, expert routing, provenance-aware decision making

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_qiA3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines multimodal cascades in cases where the required textual claim is in conflict with the image, thus making 'Conflict' the appropriate output. PAVIR maintains the claim and source requirements, consults an image-only expert when disagreements are detected, and uses a fixed rule to generate 'Answer', 'Conflict', or 'RequestEvidence'. It achieves improvements on composite versions of ScienceQA, MMStar, and MMMU-Pro. On MMStar, complete-decision accuracy rises from 84.83% to 88.67% on 399 expert calls.

Critical strengths
  • The factorial study isolates the interaction between source visibility and the final decision rule. Hiding the text helps when 'Conflict' is a valid output, but hurts under answer-only adoption. I think this is a useful and well-controlled finding.
  • The paper reports repairs and harms, answer coverage, and conditional answer error alongside aggregate accuracy. Its matched-budget comparisons also help clarify what the acquisition policy contributes.
Critical weaknesses
  • The paper does not establish a sufficiently distinct contribution over existing cross-modal conflict work. CLASH [1] already treats conflict as a correct output, evaluates conflicting and non-conflicting cases, and studies improving conflict detection through fine-tuning. PAVIR's distinct addition is a cascade that selectively calls an image-only expert and then compares its answer with a retained claim. However, this amounts to a score-based gate followed by a deterministic option comparison. The appendix's CLASH result does not resolve the novelty question: the paper notes that it includes output-space effects and that conflict-only evaluation can favor always predicting 'Conflict'. I suggest the authors explain more clearly what new capability or general insight this selective-revision procedure provides beyond the existing conflict-detection formulation.
  • The evidence for greater applicability is limited. The main results use constructed multiple-choice overlays where claims map to answer options, so source comparison reduces to matching option IDs. On M2CQA, where claims are free-form, source comparison adds only 0.08–0.16 percentage points for the stronger callers (Table 9). On MMStar, answer coverage increases from 25.50% to 31.83%, while conditional answer error also increases from 14.38% to 15.45%, as shown in Table 7. I think these results support a narrower conclusion about the constructed task and its answer-error trade-off than a general claim about multimodal cascades.

References

  • [1] Popordanoska, et al., "CLASH: A benchmark for cross-modal contradiction detection." CVPR. 2026.
Questions for authors
  • Does the Global uncertainty baseline in Table 3 exclude cases with missing required sources? Could the author report results after excluding those cases while still allowing calls on agreement cases, keeping the uncertainty ranking, revision rule, and call budget fixed?
Reviewer_okb3OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PAVIR, a framework that selectively consults an image-only expert and applies a unified decision rule to answer questions, flag conflicting evidence, or request additional evidence.

Critical strengths

The paper studies an important task and selectively queries an image-only expert and compares its prediction with a retained textual claim to decide whether to answer, report a conflict, or request missing evidence.

Critical weaknesses
  • Please carefully check your format, where an extra “\” appears in your Openreview caption.

  • The introduction is too short to tell the whole story. In general, the introduction should state motivation, research gap, insights and so on. However, I don’t find them clearly. Meanwhile, Fig.2(a) in line30 appears before Fig.1 in line 38.

  • Limited contribution and novelty.

Questions for authors

Since the main experiments use constructed textual claims and a deterministic option-matching rule, how well does PAVIR generalize to naturally occurring, free-form claims whose relationship with visual evidence is more ambiguous?

Reviewer_r8r5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies a setting where each item comes with an image, a question, and an auxiliary textual claim such as “the answer is X”; the system should output the answer when image and text agree, “Conflict” when they disagree, and “RequestEvidence” when a required claim is missing.

Critical strengths

No

Critical weaknesses

The writing is poor. The paper coins its own terminology without defining it, stuffs the abstract with numbers, ends the introduction after a single paragraph, and offers little logical connection between sentences; it also acknowledges using AI to polish the text. As a result, readers struggle to understand what is actually being compared without repeatedly flipping back to the tables.

Questions for authors

See weekness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.