- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Visual-Selective On-Policy Self-Distillation (VS-OPSD) to mitigate object hallucination in vision-language models. A frozen teacher receives training-time object cues and provides soft supervision on student-generated responses. The method compares teacher predictions under original and degraded images to identify visually sensitive token positions, then selectively applies sensitivity-weighted distillation. Experiments on LLaVA and Qwen models report reduced hallucination while largely preserving general multimodal capabilities, without additional inference-time computation.
Critical strengths
-
Practical training design. VS-OPSD uses object-category cues rather than preference pairs or complete reference captions, and adds no inference-time computation. This makes it a potentially accessible approach to hallucination mitigation.
-
Consistent improvements over Vanilla OPSD. The method reduces AMBER hallucination rates across three evaluated backbones, with improvements also observed across three paired random seeds on LLaVA-7B. These results provide evidence that the proposed supervision weighting has practical value despite the limited methodological novelty.
Critical weaknesses
- Limited methodological novelty. The method largely builds on existing OPSD and visual-dependence-based distillation. In particular, VA-OPD already uses differences between teacher predictions under original and degraded images to allocate supervision. The main additions are object-cue-conditioned self-distillation, a distribution-level Visual-KL score, and sensitivity-weighted token selection for open-ended hallucination mitigation. While potentially useful, please compare it with your VS-OPSD.
- The hallucination–coverage trade-off is insufficiently characterized. On LLaVA-7B, VS-OPSD reports lower Cover than all other mitigation methods listed in Table 1, although several external references use different settings. This raises concerns that part of the hallucination reduction comes from more conservative content selection. Appendix B.2 provides length-normalized statistics, but these do not rule out selectively omitting uncertain objects or details.
- The necessity of the core design choices is not sufficiently demonstrated. The ablations omit a straightforward baseline that retains all tokens and weights them by Visual-KL. Consequently, it remains unclear whether top-30% selection is necessary or whether continuous weighting alone would achieve comparable results.
Questions for authors
How does the method advance beyond OPSD and VA-OPD? Does the hallucination reduction persist when methods are compared at matched coverage? How does VS-OPSD compare with weighting all tokens by Visual-KL without top-30% selection?
2 comments
Thank you for the detailed review! We provide results on hallucination at matched coverage and an all-token weighting ablation below. (For your Q2 & Q3)
Q2 & W2: Hallucination–coverage trade-off. We acknowledge that hallucination mitigation involves a trade-off between hallucination rate and coverage. We address this concern from four aspects.
(1) Controlled comparison. In the directly comparable setting (Table 1), VS-OPSD achieves coverage similar to Vanilla OPSD on all three backbones while consistently reducing Hal:
| Backbone | Vanilla OPSD Cover / Hal | VS-OPSD Cover / Hal |
|---|---|---|
| Qwen2.5-VL-3B | 61.7 / 26.9 | 61.0 / 25.0 |
| LLaVA-1.5-7B | 48.2 / 25.5 | 47.7 / 22.4 |
| LLaVA-1.5-13B | 49.1 / 23.8 | 49.0 / 21.1 |
On LLaVA-1.5-7B and Qwen2.5-VL-3B, VS-OPSD also exceeds the coverage of the base model (47.7 vs. 46.6 and 61.0 vs. 59.2).
(2) Comparison with SFT. SFT learns token by token from complete reference captions and thus produces more detailed outputs with higher coverage (49.6 on LLaVA-1.5-7B). However, because this learning is off-policy, its Hal even rises above the base model (31.4 vs. 29.8), which defeats the purpose of hallucination mitigation.
(3) Matched coverage experiments. To compare the methods at matched coverage, we prompted the models to produce descriptions at different levels of detail and measured hallucination at each level. Results are averaged over two training seeds (42 and 43) at step 100 on LLaVA-1.5-7B, with generation capped at 512 tokens with no truncation.
| Prompt | Vanilla OPSD Cover ↑ / Hal ↓ | VS-OPSD Cover ↑ / Hal ↓ |
|---|---|---|
| Brief | 34.55 / 5.90 | 34.15 / 5.75 |
| Standard | 48.35 / 25.35 | 48.20 / 24.30 |
| Detailed | 49.55 / 25.65 | 49.60 / 23.00 |
| Comprehensive | 50.45 / 36.50 | 50.60 / 34.55 |
VS-OPSD achieves coverage comparable to Vanilla OPSD at every level, and even slightly higher at the Detailed and Comprehensive levels, while its hallucination rate is consistently lower, with the largest reductions at these two levels (2.65 and 1.95). The hallucination reduction therefore cannot be explained by more conservative descriptions.
(4) Output quality. The general-capability results in Table 1 show that our gains are not obtained by sacrificing output quality. On LLaVA-1.5-7B, VS-OPSD largely preserves MME (1735.34 vs. 1757.46 for the base model) and ScienceQA (63.38 vs. 63.66), consistent with improved visual grounding rather than degraded generation. By contrast, DPO (300 steps) attains a high coverage of 55.0, but its general capability drops severely (MME 1348.46, ScienceQA 62.61), which shows that coverage alone does not fully reflect output quality.
Q3 & W3: Weighting all tokens by Visual-KL without top-30% selection. We thank the reviewer for suggesting this. Table 2(a) in the paper already separates the effects of selection and weighting within the selected set. We further added the all-token weighting variant suggested as a supplementary ablation, adopting the same experimental protocol as Table 1 in the paper:
| Method | Selection | Weighting | CHAIR ↓ | Cover ↑ | Hal ↓ | Cog ↓ |
|---|---|---|---|---|---|---|
| Vanilla OPSD | All | Uniform | 5.7 | 48.2 | 25.5 | 2.6 |
| Top-30% uniform | Top 30% | Uniform | 5.7 | 47.3 | 24.0 | 2.4 |
| All-token weighted | All | Visual-KL | 5.4 | 47.8 | 23.0 | 2.4 |
| VS-OPSD | Top 30% | Visual-KL | 5.0 | 47.7 | 22.4 | 2.3 |
Continuous weighting alone already reduces Hal from 25.5 to 23.0. Adding top-30% selection further reduces Hal to 22.4 and CHAIR from 5.4 to 5.0 at nearly identical coverage (47.7 vs. 47.8), yielding the best results on CHAIR, Hal, and Cog. Selection alone also reduces Hal (24.0). The two components are therefore complementary: selection determines where to distill, and weighting determines how strongly to distill at each position. Top-30% selection is thus beneficial rather than redundant, and its gain does not come at the cost of coverage.
Ethics flag. We believe our work does not raise specific ethical concerns. As described in the Ethics Statement of the paper, this work uses only publicly available models and datasets and does not involve human participants, private data, or personally identifiable information. We would be happy to address any specific concern the reviewer may have.
We hope these results address the reviewer's concerns, and we would be grateful if the reviewer would reconsider the score in light of them. Please feel free to let us know if there are any further questions.
Thank you for the detailed review! Here is our clarification of the differences from OPSD and VA-OPD. (For your Q1)
We thank the reviewer for the detailed review and for recognizing the practical training design, the absence of inference-time computation, and the consistent improvements across backbones and random seeds. Below we address the three concerns in turn.
Q1 & W1: Novelty relative to OPSD and VA-OPD. Our work and the concurrent work VA-OPD both recognize that a controlled visual intervention on the teacher can identify which positions depend on the image, and can therefore guide how supervision is allocated. We discuss this shared insight in Related Work (Section 5) and provide a controlled comparison in Appendix E.1. Below we clarify the differences between the two works in four aspects.
(1) Research setting. OPSD mainly performs self-distillation on verifiable reasoning tasks using reference solutions, and VA-OPD mainly studies transferring visual understanding and reasoning from an external teacher to a student. We instead target open-ended object hallucination mitigation and build a privileged self-teacher from non-verifying object cues. This distinction is described in the Introduction and Related Work.
(2) Visual-sensitivity measure. VA-OPD measures whether visual information increases the teacher’s confidence in the generated token, and rectifies negative log-probability gains to zero. In hallucination, however, the real image may lower the probability of an incorrect token, so such negative gains should not be discarded as uninformative. Visual-KL measures the change in the candidate distribution via KL divergence. It accounts for the probability changes of all candidates, not only the sampled token, and can therefore identify positions that are sensitive to the visual intervention but whose VA is rectified to zero. This design is simple yet effective.
On 200 fixed rollouts, we score every token with both measures using the same teacher, weak image, and top-30% token budget.
Visual-KL covers more hallucinated object mentions than VA (76.6% vs. 57.3%). More importantly, among the 38 mentions that VA completely misses (VA = 0), Visual-KL misses only 16, recovering 22 of them (57.9%).
(3) Rollout-level weighting. VA-OPD normalizes the mean visual advantage across multiple sibling rollouts of the same input to determine response-level distillation weights, with K = 4 by default. VS-OPSD performs sensitivity estimation, token selection, and weighting within a single rollout, without comparisons across responses.
(4) Token-level supervision allocation. VA-OPD retains the distillation loss on both the high- and low-advantage groups, whereas VS-OPSD applies a sensitivity-weighted loss only at the selected high-sensitivity positions. Since a small fraction of tokens carries most of the Visual-KL mass, direct distillation is not needed at every position (see our response to Q3).
Finally, the comparison in Appendix E.1 further shows that, in our open-ended object hallucination setting, VS-OPSD outperforms our reproduced VA-OPD under the same experimental protocol.
Our responses to Q2 and Q3 are provided in the next comment.