- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates self-verified Best-of-N selection in unified multimodal models, where a model evaluates and selects among its own generated images using its visual understanding capabilities. The authors argue that the limited effectiveness of self-verification does not necessarily indicate insufficient verification capability, but may instead arise from information loss when continuous preferences are converted into discrete Yes/No judgments. They refer to this as the readout fidelity bottleneck.
As a controlled intervention, the authors introduce Continuous Best-of-N (C-BoN), which directly uses pre-decoding Yes/No logit margins instead of decoded discrete verdicts, while keeping the candidate pools, verification questions, score aggregation, and selection procedure unchanged. They further formulate a finite-observation model showing that repeated stochastic observations converge to the underlying marginal Yes/No preference, which corresponds to the pre-decoding logit margin in the direct-answer setting. Experiments across seven unified multimodal models and four text-to-image benchmarks show improvements over discrete verification in all 28 model–benchmark settings. Additional analyses examine score collisions, tie resolution, ranking reliability, and Best-of-N scaling.
Critical strengths
1. Well-controlled and extensive empirical evaluation. The paper presents a carefully controlled comparison between discrete and continuous readouts, keeping the candidate pools, verification questions, score aggregation, and selection procedure fixed. Experiments across seven models and four benchmarks demonstrate consistent improvements, providing convincing evidence that the readout mechanism itself can substantially affect self-verified selection. The breadth and consistency of these results are the main reason I have no major concerns about the soundness of the empirical claims.
2. Analysis beyond aggregate performance improvements. The authors provide several complementary analyses, including tie resolution, margin quantization, pairwise ranking reliability, and Best-of-N scaling. In particular, Table 4 shows that resolving discrete score ties accounts for 64–81% of the overall improvement, while Tables 5 and 7 show that larger continuous margin gaps correspond to more reliable candidate ranking across all four benchmarks. These findings provide useful evidence for the proposed bottleneck rather than relying solely on aggregate benchmark improvements.
Critical weaknesses
Limited conceptual and theoretical novelty beyond existing confidence-based verification. Although the systematic investigation of readout fidelity is valuable, the central insight that discretizing continuous confidence scores discards potentially useful ranking information is relatively intuitive, particularly in light of existing probability- or logit-based verification methods such as VQAScore and dMLLM-TTS. Notably, VQAScore also serves as the official GenAI-Bench evaluator in this paper (Appendix A.11), which indicates that continuous Yes-probability readout is already standard practice in this area. The authors explicitly acknowledge that a continuous Yes score is not new, and position their contribution around identifying and characterizing the readout bottleneck.
However, I remain uncertain whether the proposed conceptual framing establishes sufficiently new insights beyond the expected consequences of discretizing continuous preferences. The theoretical formulation in Section 3.3 is also relatively straightforward. Equations (5)–(7) rely on standard binomial/multinomial sampling and the strong law of large numbers, while Equation (8) expresses the limiting preference directly through the model logits. Although mathematically sound, these results do not appear to provide substantial additional theoretical understanding beyond formalizing the relationship between repeated discrete observations and continuous confidence scores. Moreover, the empirical validation of this model (Section 4.3, Table 2) is close to agreement by construction, at least for autoregressive verifiers. The predicted condition samples from the Yes/No/other distribution given by the verifier logits, while the observed condition decodes at temperature 1.0 from the same model, and Appendix A.5 itself states that, because the prompt requests an immediate Yes/No answer, the verdict distribution matches the softmax used in Section 3.3. The close agreement is therefore expected and does not constitute an independent test of the model.
Consequently, I find the conceptual and theoretical contributions to be incremental relative to existing confidence-based verification approaches. This is the primary reason for my recommendation of weak rejection.
Minor Presentation Issues. There are several minor presentation inconsistencies, including citation formatting where narrative-style citations appear without appropriate punctuation or parentheses, redundant cross-reference wording such as “Equations equation 5 and equation 6,” and inconsistent hyphenation of terms such as “predecoding” and “pre-decoding.” These issues are straightforward to correct and do not materially affect my overall assessment.
Questions for authors
1. Novelty and significance (could affect my rating). Compared with existing probability- or logit-based verification methods, what specific new or non-obvious insights does the proposed readout fidelity analysis establish? In particular, beyond the agreement reported in Table 2, which is largely expected by construction for autoregressive verifiers, does the finite-observation framework provide additional testable predictions or practical guidance? A prediction that does not follow directly from standard probabilistic sampling and is confirmed experimentally would substantially strengthen the contribution and could change my assessment.
2. Continuous scoring alternatives (clarification). Appendix A.3 shows that positive-only log odds slightly outperforms C-BoN on GenEval (78.51 vs. 78.20). Given that negation pairing is designed to cancel a uniform Yes bias (Appendix A.4), could the authors clarify whether it remains necessary or beneficial under continuous readout? Is the relative performance of these scoring formulations consistent across the other three benchmarks?
3. Masked diffusion verifiers (could affect my rating). For LaViDa-O and LLaDA-o, how are the pre-decoding Yes/No logits obtained, for example, at which answer position and denoising step? For these models, is the agreement between predicted and observed recovery in Table 2 non-trivial, and how do the authors explain the larger deviations for some models (e.g., LaViDa-O at m = 2: 82.17 vs. 84.63)? A clear answer could show that the finite-observation model captures behavior beyond the autoregressive case.