- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SchroHallu, a training-free method for hallucination mitigation in LVLMs. It first strengthens visual evidence and decomposes hidden representations into visual-evidence, prior, and residual components. The method then determines whether the prior is beneficial or hallucination-inducing under the current multimodal context, and applies a minimum-norm correction only when needed along the decision-relevant prior direction. Experiments across multiple models and datasets show consistent hallucination reduction while largely preserving perceptual and reasoning abilities. Ablations, case studies, efficiency analysis, and oracle-based evaluations further support the method.
Critical strengths
-
The context-dependent treatment of model priors is a meaningful strength. Instead of assuming that language priors are uniformly harmful, SchroHallu evaluates their role under the current multimodal context and selectively intervenes only when the prior is likely to induce hallucination.
-
The empirical validation is comprehensive and well structured. The method is evaluated across multiple LVLMs and benchmarks, with additional ablations, case studies, efficiency analysis, and oracle-based evaluations that consistently support both its effectiveness and the proposed context-conditioned intervention strategy.
Critical weaknesses
-
The methodological novelty over HulluEdit is not sufficiently clear. SchroHallu’s evidence/prior/residual decomposition and online subspace construction substantially build on HulluEdit. The main additions are context-conditioned prior-utility estimation and decision-axis minimum-norm correction, and the paper should more clearly establish whether these extensions constitute a sufficiently distinct methodological contribution.
-
The technical necessity of the quantum-inspired formulation is limited. The authors explicitly state in Appendix E: Discussion that removing the quantum terminology while keeping the same harmful/beneficial roles, deterministic measurement, and intervention rule leaves the algorithm, outputs, and performance unchanged. This makes the quantum-inspired formulation appear largely conceptual, while its actual technical value remains unclear.
-
The evaluation of generation quality is insufficient. Aggressive hallucination suppression may lead to more conservative, repetitive, or templated outputs, but the paper mainly reports hallucination-oriented metrics such as CHAIR. Additional generation-quality metrics, such as BLEU-5, recall, and repetition/diversity measures, would help verify that hallucination reduction does not come at the cost of text quality.
Questions for authors
-
The methodological novelty of SchroHallu over HulluEdit requires further clarification. The two methods share a similar evidence/prior/residual decomposition and online subspace construction framework, so the independent value of context-conditioned prior-utility estimation and decision-axis correction should be more clearly established.
-
It remains necessary to verify whether hallucination reduction comes at the cost of text generation quality. Additional metrics such as BLEU-5, recall, and repetition/diversity measures would help determine whether the method produces more conservative, repetitive, or templated outputs.
-
The statistical reliability of the reported gains is not yet sufficiently clear. Several improvements over strong baselines are relatively small, so confidence intervals or significance tests would help establish whether these gains are robust.
-
The MME evaluation is incomplete. The paper reports only selected subsets, including Existence, Count, Position, and Color. Reporting the overall MME score would provide a more complete assessment of the method’s effect on multimodal capabilities.
6 comments
[W3 & Q2] Hallucination Reduction Without Degrading Generation Quality
The expanded caption-quality evaluation indicates that hallucination reduction does not rely on conservative or repetitive output. On LLaVA-1.5-7B, CHAIR / decreases from 48.0/14.3 to 21.8/6.5, while BLEU-5 and Recall rise from 2.1 to 3.4 and 74.5 to 76.6. Across all three LVLMs (Table R1), SchröHallu achieves highest Recall and Distinct-2, lowest Repeat-2, and highest or tied-highest BLEU-5 among the compared methods. These results support improved grounding while retaining descriptive content and lexical diversity.
Table R1: CHAIR and generation-quality comparisons (%).
| Model | Method | ↓ | ↓ | BLEU-5↑ | Recall↑ | Repeat-2↓ | Distinct-2↑ |
|---|---|---|---|---|---|---|---|
| LLaVA-1.5-7B | |||||||
| Vanilla | 48.0 | 14.3 | 2.1 | 74.5 | 11.1 | 23.3 | |
| OPERA | 44.6 | 12.8 | 2.2 | 75.6 | 10.3 | 24.1 | |
| VTI | 41.4 | 11.4 | 3.1 | 67.5 | 4.2 | 27.2 | |
| Nullu | 50.2 | 13.8 | 1.9 | 74.0 | 8.1 | 30.2 | |
| SVAR | 25.0 | 6.7 | 2.9 | 69.5 | 30.9 | 18.6 | |
| HulluEdit | 51.4 | 12.9 | 2.2 | 72.5 | 11.3 | 24.4 | |
| SHIELD | 36.6 | 10.3 | 1.8 | 74.1 | 8.4 | 30.1 | |
| LVLMs-Saliency | 35.6 | 8.2 | 1.5 | 71.1 | 42.1 | 25.5 | |
| SchröHallu | 21.8 | 6.5 | 3.4 | 76.6 | 3.8 | 35.4 | |
| LLaVA-1.5-13B | |||||||
| Vanilla | 48.8 | 13.1 | 2.1 | 73.2 | 11.1 | 23.5 | |
| OPERA | 42.6 | 13.2 | 2.3 | 68.8 | 11.7 | 26.4 | |
| VTI | 32.8 | 9.2 | 3.4 | 63.7 | 5.6 | 27.1 | |
| Nullu | 49.2 | 13.3 | 2.0 | 73.1 | 10.7 | 24.6 | |
| SVAR | 28.8 | 8.8 | 2.3 | 68.6 | 21.9 | 24.1 | |
| HulluEdit | 50.6 | 13.4 | 2.3 | 72.8 | 11.3 | 23.8 | |
| SHIELD | 50.0 | 15.3 | 1.2 | 69.2 | 7.1 | 35.7 | |
| LVLMs-Saliency | 39.8 | 8.8 | 1.6 | 69.7 | 33.4 | 28.5 | |
| SchröHallu | 27.8 | 7.9 | 3.5 | 74.0 | 3.0 | 38.5 | |
| DeepSeek-VL-Chat-7B | |||||||
| Vanilla | 25.0 | 7.1 | 1.1 | 63.2 | 10.6 | 33.4 | |
| OPERA | 22.6 | 6.4 | 1.1 | 63.4 | 9.5 | 34.3 | |
| VTI | 24.0 | 8.4 | 1.0 | 65.3 | 35.0 | 21.2 | |
| Nullu | 25.2 | 6.9 | 1.1 | 64.6 | 10.9 | 32.0 | |
| SVAR | 21.4 | 6.7 | 2.1 | 57.6 | 24.8 | 31.2 | |
| HulluEdit | 30.8 | 8.7 | 1.0 | 61.6 | 28.1 | 28.2 | |
| SHIELD | 29.8 | 8.5 | 0.9 | 62.0 | 8.5 | 32.4 | |
| LVLMs-Saliency | 25.8 | 7.3 | 1.1 | 63.3 | 10.3 | 33.5 | |
| SchröHallu | 20.6 | 6.2 | 2.1 | 66.2 | 8.1 | 36.5 |
The caption-quality findings are summarized in Section 5.2 (page 9-10). Full BLEU-5, Recall, Repeat-2, and Distinct-2 comparisons across the three LVLMs are presented in Appendix F.4 (Table 15, page 33).
Response to Reviewer bqR8 (Part 2/2): Statistical Reliability and MME
[Q3] Statistical Reliability of the Improvements
The supplied uncertainty estimates indicate broad nominal statistical support, with explicit exceptions. Of 24 POPE/CHAIR comparisons, 20 have unadjusted , including all 18 POPE entries. On LLaVA-1.5-7B, Random F1 improves by 1.1 pp (95% CI [0.37, 1.83], ) and CHAIR by 3.2 pp (95% CI [0.81, 5.59], ).
Table R2: Differences against the strongest baseline (%; positive gains favor SchröHallu).
| Model / Metric | Best baseline | Ours | Gain (pp) | 95% CI of gain | |
|---|---|---|---|---|---|
| LLaVA-1.5-7B | |||||
| POPE-Ran. Acc. ↑ | SHIELD (90.7) | 91.2 | +0.5 | [0.12, 0.88] | 0.010 |
| POPE-Ran. F1 ↑ | HulluEdit (90.5) | 91.6 | +1.1 | [0.37, 1.83] | 0.003 |
| POPE-Pop. Acc. ↑ | HulluEdit (87.5) | 87.8 | +0.3 | [0.05, 0.55] | 0.019 |
| POPE-Pop. F1 ↑ | HulluEdit (87.6) | 88.2 | +0.6 | [0.17, 1.03] | 0.006 |
| POPE-Adv. Acc. ↑ | LVLMs-Saliency (83.3) | 83.9 | +0.6 | [0.13, 1.07] | 0.012 |
| POPE-Adv. F1 ↑ | HulluEdit (83.4) | 83.8 | +0.4 | [0.06, 0.74] | 0.021 |
| CHAIR ↓ | SVAR (25.0) | 21.8 | +3.2 | [0.81, 5.59] | 0.009 |
| CHAIR ↓ | SVAR (6.7) | 6.5 | +0.2 | [-0.58, 0.98] | 0.615 |
| LLaVA-1.5-13B | |||||
| POPE-Ran. Acc. ↑ | HulluEdit (90.6) | 91.4 | +0.8 | [0.26, 1.34] | 0.004 |
| POPE-Ran. F1 ↑ | HulluEdit (90.8) | 91.2 | +0.4 | [0.08, 0.72] | 0.014 |
| POPE-Pop. Acc. ↑ | HulluEdit (88.0) | 88.7 | +0.7 | [0.23, 1.17] | 0.004 |
| POPE-Pop. F1 ↑ | HulluEdit (88.3) | 88.9 | +0.6 | [0.16, 1.04] | 0.008 |
| POPE-Adv. Acc. ↑ | LVLMs-Saliency (83.4) | 85.8 | +2.4 | [1.55, 3.25] | <0.001 |
| POPE-Adv. F1 ↑ | LVLMs-Saliency (84.1) | 85.0 | +0.9 | [0.25, 1.55] | 0.007 |
| CHAIR ↓ | SVAR (28.8) | 27.8 | +1.0 | [-0.05, 2.05] | 0.063 |
| CHAIR ↓ | SVAR (8.8) | 7.9 | +0.9 | [0.16, 1.64] | 0.017 |
| DeepSeek-VL-Chat-7B | |||||
| POPE-Ran. Acc. ↑ | VTI (90.4) | 90.8 | +0.4 | [0.07, 0.73] | 0.017 |
| POPE-Ran. F1 ↑ | VTI (90.3) | 90.6 | +0.3 | [0.04, 0.56] | 0.025 |
| POPE-Pop. Acc. ↑ | Nullu (88.6) | 89.3 | +0.7 | [0.23, 1.17] | 0.004 |
| POPE-Pop. F1 ↑ | Nullu (88.4) | 88.7 | +0.3 | [0.04, 0.56] | 0.024 |
| POPE-Adv. Acc. ↑ | Nullu (85.5) | 86.0 | +0.5 | [0.13, 0.87] | 0.008 |
| POPE-Adv. F1 ↑ | Nullu (85.6) | 86.2 | +0.6 | [0.15, 1.05] | 0.009 |
| CHAIR ↓ | SVAR (21.4) | 20.6 | +0.8 | [-0.22, 1.82] | 0.125 |
| CHAIR ↓ | OPERA (6.4) | 6.2 | +0.2 | [-0.43, 0.83] | 0.530 |
The statistical-reliability findings are summarized in Section 5.2 (page 9-10), and all 24 metric-wise confidence intervals and significance tests are reported in Appendix F.5 (Table 16, page 34).
Response to Reviewer bqR8 (Part 1/2): Novelty and Generation Quality
Thank you for the constructive review. We clarify our novelty and quantum-inspired motivation, then present additional evidence on generation quality, statistical reliability, and full MME evaluation. The revised manuscript has been uploaded to the submission system, with all changes highlighted in green.
[W1 & Q1] Methodological Novelty Beyond HulluEdit
SchröHallu decides whether and where to intervene, rather than only tuning suppression strength. We build on HulluEdit's evidence/prior/residual decomposition, but differ from its VCR/PCR-guided shrinkage through contextual utility assessment: qtH=DtVtCtPSt combines prior–visual disagreement, visibility, decision relevance, and visual counter-evidence. An edit is considered only if qtHBt>mtT and is restricted to the prior-projected decision axis dtP, leaving visual and residual components unchanged. Removing H/K polarization or the prior-axis certificate raises CHAIR CS from 21.8% to 32.6% or 38.2%, respectively. The novelty is the utility-aware decision principle and localized correction, not the borrowed decomposition or minimum-norm solver.
The distinction from HulluEdit's prior shrinkage is made explicit in Introduction (page 2), Section 2.2 (page 3), and Section 4.2 (page 6). The utility-aware contextual measurement and sufficient decision-axis correction are specified in Sections 4.3–4.4 (pages 7–8), and the contribution of the H/K and prior-axis components is quantified in Table 5 (page 30).
[W2] Why the Quantum-Inspired Perspective Matters
The quantum-inspired view motivates two linked advances: context-conditioned prior utility and intervention only after sufficient harmful evidence. Treating H/K utility as unresolved until interaction with the image, prompt, and decoding state leads to deterministic contextual measurement and sufficiency-gated, decision-local correction. We do not invoke quantum computation or physical quantum states. Relabeling ∣H⟩ and ∣K⟩ would leave the equations unchanged, but not erase their conceptual motivation: this perspective explains why identifying a prior alone does not justify suppressing it. Its value is in guiding the new decision principle, not adding quantum-specific operations.
The conceptual role of the Schrödinger-inspired formulation is clarified in Section 3.3 (page 5). Its computation-preserving relabeling interpretation is explained in Appendix E (page 30), while the operational contextual measurement and associated module ablation appear in Section 4.3 (page 6-7) and Table 4 (page 28), with component terminology clarified in Appendix G (page 32).