acceptodds
Under review as a conference paper at ICLR 2027

Masked-State Evidence for Correcting Object Hallucinations in Diffusion Vision-Language Models

Abstract

Object hallucinations remain common in diffusion vision–language models (dVLMs). Unlike autoregressive VLMs, which expose a causal prefix that can be rolled back, dVLMs resolve a masked response canvas in parallel. Consequently, commitments need not follow a unique order, and completed bidirectional context can reconstruct an unsupported semantic state after it is reopened. We introduce Evidence-Guided Commitment Correction (EGCC), a training-free framework that treats factual revision as a controlled transition in this diffusion state space. Our key observation is that a dVLM can verify before it answers; at the still-masked answer state of a fixed object-existence probe, a continuous score from the dVLM’s native token distribution better ranks objects by unsupportedness than the probe’s hard decoded answer and remains informative even among commitments that receive the same decoded answer. Across all three dVLM backbones, unsupported object commitments also remain semantic attractors: simply reopening their spans reconstructs the same object in 80–86% of triggered cases. EGCC leverages masked-state evidence to detect these weak commitments and mediate a sparse transition to a locally reachable alternative. It preserves every non-target token by construction, constrains transition to be non-expansive over extracted object commitments, and requires each newly introduced object identified by the extractor to be better supported than the reopened source. Evidence estimation and revision use the same frozen dVLM, without an auxiliary verifier or full-response rollouts. With one numerical policy frozen across models and benchmarks, EGCC reduces measured object hallucination for LLaDA-V, MMaDA, and LaViDA on both COCO CHAIR and AMBER-Generative, with reductions of up to 3.8 CHAIR-S points and 11.7 AMBER Hal points. These results demonstrate masked-state evidence as an actionable interface for post-completion semantic correction in dVLMs.

Primary area:
generative models
Keywords:
Vision-Language Models, Diffusion models, Object hallucination

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mFvAOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free object hallucination correction method for diffusion-based VLMs. It uses the model’s own masked-state estimates of object presence to identify suspicious objects, locally regenerate the corresponding phrases, and select corrected outputs. Experiments on three diffusion-based VLMs, including evaluations on COCO and AMBER and ablation studies, show reduced object hallucination rates.

Critical strengths

The method is not proposed arbitrarily; it is supported by some preliminary analysis.

Critical weaknesses
  1. Does the method become too time-consuming with many objects, limiting its practical value? Do the runtime benchmarks mostly contain only a few objects?
  2. As object hallucination is relatively simple and increasingly rare in stronger models, can the method generalize to harder cases, such as attribute and spatial-relation hallucinations?
  3. The method appears to rely on the model’s inherent confidence, suppressing uncertain outputs while retaining confident ones, rather than genuinely improving the model’s underlying capabilities.
Questions for authors

See Weaknesses

9 comments

Thank you for recognizing the analysis motivating EGCC. We clarify how its cost scales with object proposals, the scope of the hallucination problem studied, and the value of correction with a frozen model.

1. Computational cost and object count (W1)

We quantify the additional cost by proposal count. On COCO random-500, the conservative measured average overhead is 0.52/0.11/0.26 s per image (2.53%/0.50%/4.74%) for LLaDA-V/MMaDA/LaViDA, using one H100 80GB at batch size one (Table 6). Vanilla captions contain 5.27/1.36/5.01 evaluator object mentions per image (Table 16).

The breakdown below groups images by distinct source proposals after within-image phrase deduplication. It excludes zero-proposal images (49/211/95 for LLaDA-V/MMaDA/LaViDA); the overall averages above include all 500 images. Added time covers probing, local correction, and acceptance, and overhead is relative to vanilla generation.

BackboneProposals / imageImagesMean added time (s)Overhead
LLaDA-V12040.2331.13%
LLaDA-V21420.7293.64%
LLaDA-V3–4871.0034.90%
LLaDA-V≥5180.9894.72%
MMaDA12320.1410.65%
MMaDA2500.3701.71%
MMaDA3–460.3311.53%
MMaDA≥510.2911.34%
LaViDA12200.1472.64%
LaViDA21200.3616.50%
LaViDA3–4490.69812.55%
LaViDA≥5161.24022.28%

In the evaluated ≥5-proposal groups, mean added time is 0.989 s for LLaDA-V and 1.240 s for LaViDA. MMaDA's corresponding result is based on a single image.

Total cost includes triggered regeneration and candidate checks. LaViDA reopens 83/500 responses, compared with 35/500 for LLaDA-V and 21/500 for MMaDA. The final policy uses one initial-state forward per distinct source phrase, reuses repeated-phrase scores, and checks new objects only after structural checks pass; selected spans are reopened jointly. Table 6's conservative timing includes additional sparse probe trajectories, as documented in the runtime appendix; the final policy uses initial-state scores without decoding these trajectories.

2. Relevance and scope of object hallucination (W2)

Object hallucination remains substantial in the three dVLMs evaluated here. Vanilla COCO CHAIR-S is 24.8%/9.8%/22.6% for LLaDA-V/MMaDA/LaViDA, and AMBER Hal is 14.34%/18.33%/44.92% (Tables 1–2). EGCC reduces these rates under one frozen policy, demonstrating a practical correction opportunity across the evaluated dVLM families.

OpenCHAIR further supports transfer to generic object proposals. Attribute and spatial-relation correction remain outside the validated scope, as stated in the submitted Limitations, and would require corresponding proposals, evidence queries, and acceptance criteria.

3. Value of evidence-guided correction with a frozen model (W3)

EGCC improves inference-time factual reliability by making better use of the frozen model's evidence. With the backbone fixed, we test whether masked-state evidence supports more factual, informative outputs through constrained local correction.

The score answers an image-conditioned object-existence query before answer decoding. The readout controls show that this information is more diagnostic than the decoded answer to the same query. On full POPE, using the masked-state readout improves binary accuracy by 7.5/11.4/0.5 percentage points across the three backbones (Table 4), showing that useful information is available beyond the model's final verification decision.

The correction benefit also extends beyond deleting uncertain content. Non-target tokens remain fixed. In an additional comparison on the full COCO and AMBER sets, we order EGCC-selected responses by evidence and use evaluation outcomes to choose each deletion control's most favorable prefix within EGCC's supported-object loss budget. EGCC removes more unsupported mentions than object deletion in five of six settings, matching it in the sixth, and exceeds phrase deletion in all six. For example, on AMBER LaViDA, EGCC removes 201 unsupported mentions with 23 supported losses, versus 132/22 for object deletion and 36/23 for phrase deletion.

Direct same-selection comparisons further favor EGCC in grounded detailedness and fluency over both deletion controls in all six settings. Qwen2.5-VL-72B evaluates image-caption pairs with method identities hidden and both presentation orders averaged; object-deletion comparisons have 311 valid pairs after eight unparseable judge outputs, and phrase-deletion comparisons have all 319. These results support a practical benefit from local recommitment: better hallucination reduction within a content-loss budget, together with better judged content and language quality in the direct comparison.

Reviewer_mFvAOpenReview ↗

reply 1

I noticed in your breakdown table that for LLaDA-V, the 3–4 proposals group (87 images) has a mean added time of 1.003s (4.90% overhead), whereas the ≥5 proposals group (18 images) drops to 0.989s (4.72% overhead). Why is the time overhead for ≥5 proposals actually lower than that for 3–4 proposals, and why does this pattern appear for both LLaDA-V and MMaDA? Could you analyze the underlying reason for this?

Reviewer_mCZqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Evidence-Guided Commitment Correction (EGCC), a training-free framework that treats factual revision as a controlled transition in this diffusion state space. For objects extracted from a generated caption, the same frozen model answers an object-existence probe implicitly: its affirmative and negative probabilities at the initial masked answer position define a continuous support score. Low-support objects are selected using absolute and response-relative criteria. The method remasks compact local spans, clamps all other positions, temporarily penalizes source-object forms, and accepts candidates subject to structural constraints and improved evidence for newly introduced objects. Experiments evaluate three diffusion VLM families on COCO CHAIR and AMBER, with an additional OpenCHAIR extension. Controlled analyses investigate evidence quality, source reconstruction, candidate acceptance, and reopening scope.

Critical strengths
  1. The evidence-readout finding is supported by several useful controls. Continuous masked-state scores substantially outperform hard decoded verification answers in unsupported-object ranking, and remain informative within groups receiving the same decoded answer. The volume-matched selector experiments hold the transition and retention mechanisms fixed, helping separate selection quality from intervention frequency. The full-POPE evaluation further supports the usefulness of this readout beyond the generated-object probe pool.

  2. The correction framework is conservative and evaluated with unusually detailed intervention accounting. With one numerical policy frozen across models and benchmarks, EGCC reduces measured object hallucination for LLaDA-V, MMaDA, and LaViDA on both COCO CHAIR and AMBER-Generative, with reductions of up to 3.8 CHAIR-S points and 11.7 AMBER Hal points. These results demonstrate masked-state evidence as an actionable interface for post-completion semantic correction in dVLMs.

Critical weaknesses
  1. The incremental value of local recommitment over evidence-guided deletion or neutralization remains insufficiently isolated. Table 13 reports many deletion outcomes: with source exclusion, LLaDA-V has 28 deletion outcomes versus one replacement, and LaViDA has 49 versus 22. These are transition-job categories, rather than counts of beneficial applied edits or verified grounded replacements. Nevertheless, they motivate a simpler comparison using the identical evidence scores and selected commitments, followed by constrained deletion or neutralization. Existing remasking, exclusion, and scope ablations establish useful properties of the transition, but do not isolate its advantage over this alternative. Such a comparison should assess hallucination reduction jointly with supported-object loss and local grammatical quality.

  2. The candidate-evidence gate has partial component-level attribution, but its incremental effect on final quality remains unclear. Table 5 evaluates the combined structural-and-evidence acceptor. Some independent effects are already documented, and most rejections arise from structural checks. What is missing is a fixed-candidate comparison that retains those checks while disabling only the evidence gate and reports final hallucination and preservation metrics. Counts of reducing and harmful edits alone do not determine changes in those metrics.

Questions for authors
  1. How does EGCC compare with evidence-guided deletion or object neutralization using exactly the same selected commitments and local spans, with text outside those spans preserved? Does local recommitment improve hallucination reduction at comparable supported-object loss, and does it provide a measurable grammatical or descriptive-quality advantage?

  2. Appendix G already classifies the candidates rejected specifically by the evidence gate. With the same candidates and structural checks retained, could you report the final metrics when only that gate is disabled? Hallucination endpoints, supported-object loss, and coverage would clarify the practical effect beyond the existing edit-level accounting.

  3. Could you report paired per-image intervals to quantify uncertainty in the OpenCHAIR and MMaDA improvements? For the generic noun-phrase proposal interface, could you also provide the reducing/neutral/harmful transition accounting and examine whether remaining harmful substitutions still concentrate outside the extractor’s coverage?

5 comments

Thank you for recognizing the evidence-readout controls and detailed intervention analysis. We address the requested comparisons through deletion controls at matched content-loss budgets, the incremental role of evidence acceptance, and uncertainty and failure analysis for open-vocabulary transfer.

1. Local recommitment versus deletion (W1, Q1)

At the same supported-object loss limit, EGCC removes more unsupported mentions than object deletion in five of six settings and matches it in the sixth; it exceeds phrase deletion in all six. We restrict deletion to responses originally selected by EGCC and fix their order by evidence. For each deletion control, we apply deletion to the top kk responses and report the prefix that removes the most unsupported mentions while keeping supported-object loss at or below EGCC's. This gives deletion its most favorable eligible prefix under the same loss limit. The prefix is selected using evaluation outcomes for this comparison; it is not an additional inference policy.

Object deletion removes the selected object spans; phrase deletion removes the local spans used by EGCC, preserving surrounding text except for whitespace/punctuation cleanup. Entries are net unsupported mentions removed / net supported mentions lost, relative to vanilla:

Dataset / backboneEGCCObject deletion within loss budgetPhrase deletion within loss budget
COCO / LLaDA-V19/410/38/3
COCO / MMaDA7/56/44/5
COCO / LaViDA32/929/927/8
AMBER / LLaDA-V34/1226/1212/10
AMBER / MMaDA11/4411/448/42
AMBER / LaViDA201/23132/2236/23

The EGCC unsupported-mention reductions reproduce submitted Tables 1–2. This comparison varies deletion volume within the common selected-response pool to answer the question about comparable content loss. It complements the direct same-selection comparison: applying deletion to all selected responses leaves fewer unsupported mentions but also removes more supported content. In the 319-response comparison below, EGCC retains 6–34 more supported mentions than object deletion per setting, alongside 1–12 more unsupported mentions.

Recommitment also improves grounded detailedness and fluency over both deletion controls. We evaluate the full same-selection outputs on 35/21/83 selected COCO responses for LLaDA-V/MMaDA/LaViDA and 60 randomly sampled selected AMBER responses per model. These are the direct deletion controls, not the loss-budget prefixes above. Qwen2.5-VL-72B scores each image and method-anonymized caption pair in both presentation orders, which are averaged. Detailedness measures visually supported information; fluency measures grammaticality, completeness, and natural phrasing without penalizing brevity itself. Entries are EGCC minus deletion on 1–10 scales:

Dataset / backboneObject deletion: detailedness / fluencyPhrase deletion: detailedness / fluency
COCO / LLaDA-V+0.40/+0.46+0.84/+0.73
COCO / MMaDA+1.17/+1.33+2.29/+1.12
COCO / LaViDA+0.44/+0.70+0.58/+0.86
AMBER / LLaDA-V+0.22/+0.45+1.18/+1.65
AMBER / MMaDA+0.64/+0.91+1.22/+1.28
AMBER / LaViDA+0.30/+0.73+0.57/+0.89

For each of the 24 differences, the paired-image 95% bootstrap interval is above zero with judge scores held fixed. Eight object-deletion pairs have unparseable judge outputs and are excluded (311 valid pairs); phrase-deletion comparisons are complete. Together, the loss-budget and language-quality comparisons identify a benefit beyond deletion: better hallucination reduction within the same content-loss limit, with more fluent and informative outputs in the direct same-selection comparison.

2. Incremental effect of the evidence gate (W2, Q2)

The evidence gate checks newly introduced objects after all structural constraints have been satisfied. It filters replacements that are structurally admissible but insufficiently supported; deletions introduce no new object and bypass this check. With selections and candidates fixed, Appendix G's ten evidence-rejected candidates comprise two reducing, five neutral, and three harmful edits. Adding them back gives:

Acceptance ruleReducingNeutralHarmfulApplied edits
Structural only24130222565
Structural + evidence23929719555

This separates the evidence gate's contribution from the combined acceptor's 44.1% harmful-edit blocking rate. To assess final quality, we keep selections, candidates, and all structural checks fixed and disable only the evidence gate. Each entry below is structural-only → full EGCC. Hallucination and coverage scores are percentages; supported loss is the net loss of supported object mentions relative to vanilla. Coverage denotes COCO object recall or AMBER Cover.

Dataset / backboneCHAIR-I / CHAIR ↓CHAIR-S / Hal ↓Supported loss ↓Coverage ↑
COCO / LLaDA-V5.707 → 5.70721.60 → 21.604 → 463.670 → 63.670
COCO / MMaDA7.036 → 7.0259.00 → 9.006 → 535.705 → 35.780
COCO / LaViDA7.024 → 7.01819.20 → 18.8011 → 953.184 → 53.246
AMBER / LLaDA-V4.172 → 4.08712.151 → 12.15113 → 1237.355 → 37.382
AMBER / MMaDA8.943 → 8.84318.028 → 17.82945 → 4433.006 → 33.024
AMBER / LaViDA6.525 → 6.44433.267 → 33.26723 → 2353.867 → 53.867

The evidence gate preserves or improves the reported final metrics, with a limited but measurable incremental effect. AMBER CHAIR decreases on all three backbones, while supported loss and coverage are maintained or improved. On COCO, unsupported-mention totals are unchanged: the small CHAIR-I improvements on MMaDA and LaViDA reflect better supported-content retention, and LaViDA's CHAIR-S also decreases. These results support the gate's role as an additional check on structurally admissible replacements, rather than the main source of the framework's gains.

3. Paired uncertainty and open-vocabulary failure analysis (Q3)

OpenCHAIR improvements are supported by paired image-level intervals, including on an expanded 1,000-image evaluation of outside-vocabulary objects. Selection thresholds remain frozen. The table distinguishes the submitted 300-image overall result from the expanded outside-vocabulary result. Differences are EGCC minus vanilla; scores are percentages and differences/intervals are percentage points:

Backbone300-image overall difference [95% CI]1,000-image OOV vanilla → EGCC (%)1,000-image OOV difference [95% CI]
LLaDA-V-1.030 [-1.550, -0.500]34.613 → 34.118-0.496 [-0.816, -0.174]
MMaDA-0.420 [-0.670, -0.170]26.052 → 25.698-0.354 [-0.692, -0.018]
LaViDA-0.590 [-1.060, -0.120]35.018 → 33.931-1.087 [-1.533, -0.654]

All listed intervals are paired by image and below zero; differences use unrounded scores. The 300-image intervals use 10,000 resamples, recomputing the hallucination rate from total object counts while holding the official judge decisions fixed. The evaluations cover different object populations, so their effect sizes are not directly comparable.

For MMaDA, paired intervals support lower object-level hallucination rates on both COCO and AMBER. The following comparisons use vanilla as the baseline on the same evaluation images; differences and intervals are in percentage points:

DatasetMetricEGCC − vanillaPaired 95% CI
COCOCHAIR-I-0.904[-1.892, -0.066]
COCOCHAIR-S-0.800[-1.800, 0.000]
AMBERCHAIR-0.213[-0.415, -0.014]
AMBERHal-0.498[-1.062, +0.066]

The object-level intervals exclude zero on both benchmarks. Response-level point estimates also favor EGCC, but their intervals include zero, so we do not claim statistically resolved improvements on those endpoints. These metrics capture different outcomes: removing an unsupported object can improve the object-level rate while leaving the response-level indicator unchanged if another hallucination remains. Thus, the results support a modest reduction in MMaDA's object-level hallucination rates, with greater uncertainty about the fraction of responses containing any hallucination.

The generic-interface analysis identifies the candidate-vocabulary boundary as the main location of residual harmful edits. Source-proposal lexical coverage is 90.2%/85.6%/90.1% for LLaDA-V/MMaDA/LaViDA when matching noun heads or existing aliases. The following table covers all applied edits from the original EGCC on the 1,000-image sample. R/N/H denotes a decrease/no change/increase in official unsupported-object count relative to vanilla; unchanged responses are excluded. An OOV-introducing edit adds an object expression absent from the original response and outside the internal candidate extractor's vocabulary.

BackboneApplied editsR/N/HOOV-introducing among harmful editsHarmful among OOV-introducing edits
LLaDA-V764213/453/9894/98 (95.9%)94/577 (16.3%)
MMaDA6922/34/1312/13 (92.3%)12/57 (21.1%)
LaViDA787270/387/130127/130 (97.7%)127/628 (20.2%)

Most harmful edits therefore introduce expressions outside the candidate vocabulary, consistent with the submitted AMBER analysis. The last two columns distinguish the concentration of failures from the harmful rate of OOV-introducing edits; they measure co-occurrence within an edit, not causal attribution to a particular added object. The expanded results support transfer beyond the predefined vocabulary while making the remaining candidate-validation coverage limitation explicit.

Reviewer_U6zpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EGCC, which uses masked-state probabilities from object-existence probes to identify questionable objects in diffusion vision–language model outputs, then corrects them through local remasking, source-object suppression, and candidate acceptance rules. Experiments on three models demonstrate reduced object hallucination, with analyses of selection, reopening scope, and candidate acceptance.

Critical strengths

The experiments include volume-matched selector comparisons, reopening-scope ablations, and a shared policy selected on a separate development set, controlling several relevant confounders. All three models improve on COCO and AMBER while preserving non-target tokens, with low reported average runtime overhead on COCO.

Critical weaknesses

1 The additional value of the central evidence-consistent recommitment mechanism over simple deletion is insufficiently isolated: deletions substantially outnumber replacements in local transitions, yet a deletion baseline using the same selections is missing. The acceptor experiment also combines structural constraints with evidence checks; the appendix shows that the non-monotonic-evidence rule rejects only three harmful candidates while rejecting two hallucination-reducing candidates, so the combined acceptor’s gains cannot be attributed directly to evidence ordering. 2 The analyses of masked-state evidence and the “semantic reconstruction basin” establish useful empirical phenomena but provide limited explanation of their diffusion-specific nature. A continuous score outperforming a discrete answer, and source suppression reducing source reconstruction, do not alone establish gains beyond combining probabilistic self-verification with lexical exclusion; the trajectory analysis also indicates that the initial state is not uniquely informative. 3 Some incremental gains over the closest control are small and lack uncertainty estimates: on COCO, EGCC removes only 6, 5, and 5 additional unsupported object mentions across the three models compared with decoded self-verification selection. Absolute OpenCHAIR improvements are also only approximately 0.004–0.010, while 18 of the 19 harmful AMBER edits introduce objects outside the internal proposal vocabulary, leaving reliability in broader open-vocabulary settings insufficiently established.

Questions for authors

1 With identical selections, how does directly deleting questionable objects compare with full EGCC in factuality, content preservation, and language quality? How effective is the acceptor with structural constraints alone? 2 Could continuous self-verification controls on a comparable scale and controlled perturbations of surrounding context further distinguish generic probabilistic ranking benefits from the effects of diffusion states and bidirectional context? 3 Could you provide paired confidence intervals against the closest baselines and report proposal coverage and out-of-vocabulary harmful substitution rates on a larger open-vocabulary sample?

3 comments

Thank you for recognizing the controlled comparisons and shared development-set policy. We address the three concerns with matched deletion controls, continuous-readout and context controls, and paired uncertainty estimates with an expanded OpenCHAIR evaluation.

1. Value beyond deletion and structural safeguards (W1, Q1)

With identical selections, EGCC retains more supported content and receives higher grounded-detail and fluency scores than both deletion controls across all six settings. We delete either the selected object spans or the local phrase spans used for reopening, preserving text outside those spans except for whitespace/punctuation cleanup. The comparison covers 319 selected responses: 35/21/83 COCO responses for LLaDA-V/MMaDA/LaViDA and 60 randomly sampled selected AMBER responses per model.

The table reports unsupported/supported mention counts (H/S). The final columns give EGCC's judge-score advantage over each deletion control for grounded detailedness/fluency (D/F):

Dataset / backboneEGCC H/SObject deletion H/SPhrase deletion H/SD/F gain vs. object deletionD/F gain vs. phrase deletion
COCO / LLaDA-V8/1274/1204/116+0.40/+0.46+0.84/+0.73
COCO / MMaDA4/313/253/18+1.17/+1.33+2.29/+1.12
COCO / LaViDA84/43072/40561/397+0.44/+0.70+0.58/+0.86
AMBER / LLaDA-V38/27333/23929/231+0.22/+0.45+1.18/+1.65
AMBER / MMaDA23/20019/18715/165+0.64/+0.91+1.22/+1.28
AMBER / LaViDA37/48632/45232/444+0.30/+0.73+0.57/+0.89

Relative to object deletion, EGCC retains 6–34 more supported mentions per setting, alongside 1–12 more unsupported mentions. On the full COCO and AMBER evaluation sets, we also compare deletion at EGCC's supported-loss limit: we apply each deletion control to the top-k originally selected responses in a fixed evidence order, choosing k from evaluation outcomes to maximize unsupported-mention removal within that limit. EGCC removes more unsupported mentions than object deletion in five of six settings (one tie), and more than phrase deletion in all six.

Qwen2.5-VL-72B scores image-caption pairs on 1–10 scales with method identities hidden and both presentation orders averaged. Detailedness measures visually supported information; fluency measures grammaticality, completeness, and natural phrasing without penalizing brevity itself. For each of the 24 score differences, the paired-image 95% bootstrap interval is above zero, with judge scores held fixed. Eight object-deletion pairs have unparseable judge outputs and are excluded (311 valid pairs); phrase-deletion comparisons are complete. Mention counts use all 319 responses.

Evidence acceptance filters additional harmful edits beyond structural constraints. It checks newly introduced objects only after structural checks pass; deletions introduce no new object and bypass this check. The following counts aggregate all three backbones on AMBER, using Table 5 and Appendix G:

Acceptance ruleReducing edits retainedHarmful edits retained
None25834
Structural only24122
Structural + evidence23919

Structural-only acceptance retains the ten candidates rejected solely by the evidence gate (two reducing, five neutral, three harmful). The gate blocks three additional harmful edits while retaining 239/241 (99.2%) reducing edits; Table 5's 44.1% blocking rate belongs to the combined acceptor. With candidates and structural checks fixed, enabling the gate also lowers AMBER CHAIR by 0.085/0.100/0.081 pp on LLaDA-V/MMaDA/LaViDA. Supported loss and coverage are maintained or improved on both benchmarks. On COCO, unsupported totals are unchanged; the small CHAIR-I gains reflect supported-content retention.

2. Continuous evidence and context-dependent reconstruction (W2, Q2)

The initial readout provides useful continuous evidence before answer decoding. We compare three continuous readouts using identical object probes and pair normalization on all 872/356/704 probes for LLaDA-V/MMaDA/LaViDA. Entries are ROC-AUC / AP:

ReadoutLLaDA-VMMaDALaViDA
Initial masked state0.8908/0.63740.7158/0.29140.9110/0.7268
Last masked state0.8298/0.48670.7150/0.27940.9085/0.7231
Generated text, first answer token remasked0.7994/0.51110.7158/0.29140.9106/0.7266

For the generated-text control, we tokenize the generated answer again, mask its first Yes/No token, and retain the remaining text in the same 16-token answer region, with unused positions masked. Initial evidence is stronger on LLaDA-V and similar to later-state scores on MMaDA and LaViDA. The benefit is early availability in one masked-state forward pass. Separately, full-POPE accuracy improves by 7.5/11.4/0.5 percentage points when both readouts produce binary decisions (Table 4), so the benefit is not confined to comparing continuous rankings with hard answers.

Context perturbations reveal reconstruction effects without source suppression. Holding the target span fixed, we mask equal numbers of left, nearby-right, or distant-right lexical tokens. Entries count exact source reconstruction:

BackboneIntactLeft maskedNearby-right maskedDistant-right masked
LLaDA-V27/3520/3526/3529/35
MMaDA7/77/77/77/7
LaViDA62/8350/8352/8359/83

LaViDA is sensitive to both left and nearby-right context, with smaller changes under distant-right masking; LLaDA-V shows a predominantly left-context effect. MMaDA's short outputs leave only seven cases with sufficient context for all conditions, with no observed change. Masking changes both context and the decoding process, so these results establish context sensitivity without isolating bidirectionality as its sole cause.

EGCC uses native arbitrary-position infilling with non-target tokens clamped and the same frozen model supplying selection and replacement evidence. The scope control illustrates its practical value: on LaViDA, local correction removes 32 unsupported mentions with nine supported losses, versus 41 removals with 60 supported losses for suffix regeneration (Figure 4). Together, these controls support combining early evidence access with native local infilling for correction that preserves all non-target tokens.

3. Uncertainty, efficiency, and open-vocabulary transfer (W3, Q3)

Against the closest selector control, CHAIR-I improves on all three backbones with paired intervals below zero, while correction overhead decreases. Decoded self-verification uses the same probes, local correction and acceptance rules, and number of selected commitments; only the selection readout changes. The additional 6/5/5 unsupported mentions removed measure the benefit of this readout choice within EGCC.

On COCO random-500, we resample image indices jointly across methods and recompute each metric. Differences below are EGCC minus decoded self-verification; all entries are percentage points:

BackboneCHAIR-S differencePaired 95% CICHAIR-I differencePaired 95% CI
LLaDA-V-1.200[-2.400, -0.200]-0.216[-0.444, -0.030]
MMaDA-0.400[-1.000, 0.000]-0.736[-1.538, -0.048]
LaViDA-0.400[-1.000, 0.000]-0.191[-0.410, -0.037]

CHAIR-S intervals include zero on MMaDA and LaViDA. These hallucination reductions accompany lower added runtime: under Table 6's measurement conditions (one H100 80GB, batch size one), correction using decoded selection adds 0.898/0.219/0.443 seconds per image. EGCC's conservative runtime upper bounds reported in Table 6 are 0.520/0.110/0.260 seconds for LLaDA-V/MMaDA/LaViDA. The reduction is approximately 41–50% of correction overhead.

The expanded 1,000-image OpenCHAIR evaluation supports transfer beyond the internal candidate vocabulary without threshold retuning. We report the submitted 300-image overall result and the expanded outside-vocabulary result separately. Differences are EGCC minus vanilla; scores are percentages and differences/intervals are percentage points:

Backbone300-image overall difference [95% CI]1,000-image OOV vanilla → EGCC (%)1,000-image OOV difference [95% CI]
LLaDA-V-1.030 [-1.550, -0.500]34.613 → 34.118-0.496 [-0.816, -0.174]
MMaDA-0.420 [-0.670, -0.170]26.052 → 25.698-0.354 [-0.692, -0.018]
LaViDA-0.590 [-1.060, -0.120]35.018 → 33.931-1.087 [-1.533, -0.654]

The OpenCHAIR intervals are paired by image and below zero; differences use unrounded scores. The 300-image intervals use 10,000 resamples, recomputing the hallucination rate from total object counts while holding the official judge decisions fixed. The two evaluations cover different object populations, so their effect sizes are not directly comparable.

Source-proposal coverage is 90.2%/85.6%/90.1% for LLaDA-V/MMaDA/LaViDA, measured as the fraction of official-evaluator object mentions in vanilla captions matched to same-image source proposals by noun head or existing alias, counting repeated mentions separately. The table covers all applied edits on the 1,000 images using EGCC's generic noun-phrase proposal interface with the candidate-validation rule described in the paper. R/N/H denotes a decrease/no change/increase in official unsupported-object count relative to the original response; unchanged responses are excluded. An OOV-introducing edit adds an object expression absent from the original response and outside the internal candidate extractor's vocabulary.

BackboneApplied editsR/N/HOOV-introducing among harmful editsHarmful among OOV-introducing edits
LLaDA-V764213/453/9894/98 (95.9%)94/577 (16.3%)
MMaDA6922/34/1312/13 (92.3%)12/57 (21.1%)
LaViDA787270/387/130127/130 (97.7%)127/628 (20.2%)

The larger evaluation supports OOV transfer while confirming that most harmful edits introduce expressions outside the candidate vocabulary, consistent with AMBER. The final columns distinguish failure concentration from the harmful rate of OOV-introducing edits; neither attributes harm to a specific added object. Candidate-validation coverage remains a limitation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of additional evidence and clarifications

We thank the reviewers for their careful feedback. As the discussion period draws to a close, we summarize the additional evidence provided in our responses:

  • Correction versus deletion and gate attribution. We added object- and phrase-deletion controls using the same selected targets. Under EGCC’s supported-object loss budget, EGCC removes more unsupported mentions than object deletion in five of six original settings and matches it in the sixth. Direct comparisons also assess supported-content retention and judged language quality. A fixed-candidate ablation isolates the evidence gate’s incremental contribution while retaining structural checks.

  • Evidence readout and uncertainty. We added continuous-readout and context-perturbation controls, clarifying the initial readout’s early availability without claiming it is uniquely informative across all models. Paired image-level intervals support CHAIR-I improvements over decoded self-verification on all three original backbones. The expanded 1,000-image OpenCHAIR evaluation includes outside-vocabulary results, proposal coverage, and harmful-edit accounting.

  • Runtime and relevance to recent models. The stage-level timing breakdown explains the non-monotonic group averages through differing correction workloads. We also evaluated Fast-dVLM and Dynin-Omni, both introduced in 2026, without retuning EGCC. COCO CHAIR-S decreases by 6.6 and 4.0 percentage points, respectively; AMBER results and deletion comparisons further quantify hallucination reduction and content preservation.

These additions clarify EGCC’s contribution as evidence-guided local correction with a frozen backbone, together with its operating trade-offs and current scope of object hallucination.

We will incorporate these experiments and clarifications into the revised manuscript. We would appreciate hearing whether any concerns remain unresolved or require further clarification. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.