- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes IVSR, an inference-time method that selectively revises answers in diffusion MLLMs at the first valid Yes/No answer transfer without updating the backbone weights. IVSR selects low-margin answers for visual rereading. It then jointly considers (1) the Yes/No margins after rereading and (2) the cosine similarity between a visual region retrieved by the reread answer state and the contextual target representation in the question. The native answer is changed only when both evidence conditions are satisfied.
Critical strengths
Without updating the backbone weights, IVSR improves over native decoding on all 18 POPE settings across three diffusion MLLMs (Table 1). Its selective inspection keeps the measured additional latency to 1.4–9.8% (Table 2). The authors openly report harmful edits and acknowledge limitations in the available reproduction details.
Critical weaknesses
-
The main comparison uses unequal calibration conditions. IVSR uses 1,200 labeled examples for calibration and configuration selection, while the baselines use their reported defaults. The matched-calibration comparison covers only one setting, where the advantage over the strongest tuned baseline is 0.40 F1 points.
-
A simple baseline is missing. In the diagnostic setting reported in Table 9, IVSR increases the Yes ratio and recall while lowering precision. Simply adjusting the Yes/No decision threshold on the same calibration data might produce a similar effect, but this comparison is not provided.
-
The region–target cosine score is insufficiently validated as a measure of target identity. The target-evidence ablation supports the usefulness of this component, but does not directly show whether e distinguishes similar objects such as a fork and a knife. Compatible hidden spaces alone do not establish this interpretation.
-
The diagnostic evidence has limited scope. The Section 3 diagnostics use only one model and one setting. The spoon example (Figure 2, right) was selected after observing a beneficial change, and its margin magnitude of 0.125 is only one BF16 representable increment at the reported logit scale; full-precision behavior remains untested (Appendix D.4.2). In the persistence experiment, 266/267 initially correct preferences also persist. This supports the claim that errors can persist, but does not establish an error-specific mechanism.
-
Reproducibility is insufficient. The benchmark controller’s tensor-level reread operation is not fully specified, some fitted threshold values are missing, and the profiling hardware is unspecified. The separately implemented diagnostic probe does not resolve the missing details of the benchmark controller.
-
The paper is difficult to follow because of excessive terminology. Existing concepts receive new names, such as “commitment maturity” for the answer margin, and overlapping terminology makes the method harder to understand.
Questions for authors
- Can the authors provide matched-calibration comparisons with the baselines for the other backbones and settings?
- How does IVSR compare with simply adjusting the Yes/No decision threshold on the same calibration data?
- Does e actually distinguish similar objects? For example, what are the e values for the fork/knife and suitcase/backpack pairs in Figure 3, and how well does e alone separate correct from incorrect answers?
- Does the spoon result (Figure 2, right) persist in full precision, and can the authors provide full-precision results on more examples?