- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes OVERTURE, a training-free decoding method for diffusion-based multimodal language models. It detects trajectory deviations termed “Undershoot” during an initial generation pass, then regenerates the response while prioritizing selected positions and deferring EOS/EOT commitment. The paper motivates this approach through a proposed difficulty–timing mismatch and reports improvements over native decoding on two multimodal backbones.
Critical strengths
- The relationship between commitment order and visual grounding is an interesting research question.
- The method requires no additional training and shows improvements over native decoding, particularly on MMaDA.
Critical weaknesses
- Limited model coverage and insufficiently challenging evaluation. The multimodal evaluation is restricted to two early 8B backbones, MMaDA-8B-Base and LaViDa. More importantly, Appendix B reports that LLaDA-V does not exhibit the stable advancing frontier required by the detector and consequently excludes it from intervention evaluation. This raises a concrete concern about architectural generality. Although AMBER and DetailCaps provide useful additional coverage, the evaluation would benefit from tests that more directly distinguish visual grounding from textual shortcuts, such as MathVerse or MMMU-Pro.
- The baseline comparisons do not convincingly establish a competitive advantage. Section 4.1 explicitly states that external baselines are reported under their respective evaluation settings, whereas only Original and OVERTURE are evaluated under matched settings. Consequently, the rankings in Table 1 are difficult to interpret. Even taking the reported numbers at face value, OVERTURE trails the best compared method on five of the six shared metrics on LaViDa. For example, its MathVista and LLaVA-Bench-COCO scores are 41.80 and 70.60, compared with CACD’s 45.70 and 88.70. AMBER and DetailCaps lack comparisons beyond the native scheduler.
- Missing ablations prevent attributing the gains to Undershoot-guided prioritization. OVERTURE combines trajectory-based sentence selection, priority decoding, and EOS/EOT deferral. Section 3.1 states that EOS/EOT deferral remains active even when no sentence is selected. However, the paper does not report EOS-deferral-only or priority-only ablations, nor comparisons with random or confidence-based sentence selection. Matching the maximum generation length does not control for differences in actual response length or premature termination. The reported improvements therefore do not isolate the contribution of the proposed trajectory signal. Section 4.4 also claims weaker downstream performance with Top-2 selection without providing the corresponding quantitative results.
- The central mechanistic claims are insufficiently substantiated. Section 2.3 claims that Undershoot remains associated with errors after controlling for position, generation stage, and confidence, but the corresponding control procedure and quantitative results are not reported. Similarly, Section 2.4 defines a sensitivity measure and describes increasing textual influence, decreasing visual influence, and recoverable visual information, yet does not provide the corresponding quantitative trends, perturbation details, or probing protocol. These omissions make it difficult to assess whether the proposed difficulty–timing mismatch is supported by systematic evidence.
Questions for authors
- Can the method be evaluated on newer or architecturally more diverse dMLLMs and on benchmarks that more directly test dependence on visual evidence? How is applicability determined when a model lacks a stable advancing frontier, as observed for LLaDA-V?
- Can AdaBlock, Wavefront, and CACD be evaluated using the same examples, prompts, decoding settings, and evaluation pipeline as OVERTURE? Under this unified protocol, does OVERTURE offer a consistent advantage, particularly on LaViDa?
- Can you provide EOS-deferral-only and priority-only ablations, together with random and confidence-based sentence-selection controls? Please also report actual response lengths, truncation rates, and quantitative Top-1 versus Top-2 intervention results.
- Can you provide the quantitative text/vision sensitivity results, visual-information probing protocol, and the position-, stage-, and confidence-controlled analysis described in Sections 2.3–2.4? How do these results distinguish the proposed mechanism from alternative explanations?