- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes ViPSy, a preference-data construction framework for mitigating hallucinations in vision-language models. The policy first captions an image, and a text-to-image model generates synthetic variants from these captions. A judge extracts cues shared between the original image and the variants. These cues guide policy rollouts, from which preferred and dispreferred responses are selected for standard DPO training. Experiments evaluate hallucination mitigation, general multimodal performance, and visual representations across multiple policy models.
Critical strengths
- The reported hallucination reductions are substantial and the paper provides extensive ablations and a practical deployment design
Critical weaknesses
-
The motivation and advantages of semantic image synthesis in step 1 remain unclear. The synthesized images may inherit incorrect descriptions and introduce biases from image-to-text (T2I) conversion (this uncontrollable factor introduces additional noise information and high time costs), so its advantages over direct cue extraction are not yet sufficient. Although Table 6 reports gains, missing cue counts and coverage leave unclear whether these improvements stem from synthesis or stricter filtering that retains fewer cues. Given that image generation accounts for approximately 82% of offline construction cost, synthesis and direct extraction should be compared under similar filtering criteria and computational budgets.
-
The methodological novelty beyond cue construction is limited. Step 2 largely follows established policy sampling, AI judging, and preference optimization, as in RLAIF-V. Hint-guided policy generation has also been explored in RPO, which is cited but not experimentally compared. The main advance therefore appears to lie in Step 1. The paper should clarify what Step 2 adds beyond these precedents and more convincingly establish the distinct value of its cue-construction procedure.
Questions for authors
-
ViPSy requires cues to be supported by all eight pairwise comparisons, which may retain only a small set of salient objects. Could direct extraction achieve similar results by retaining only high-confidence objects that pass repeated verification? Although Table 6 includes repeated extraction with voting, it remains unclear whether both methods use the same retention criterion. The resulting cue counts and coverage should also be compared.
-
A cue correctness of 95% does not by itself establish better cue extraction. For example, retaining only “person” and “car” may yield higher correctness than extracting a dozen objects, while providing less useful information. The paper does not establish whether image synthesis retains its advantage at comparable cue coverage.
-
Image generation accounts for approximately 82% of the default offline preference-construction cost, based on Tables 21 and 22. A more informative comparison would allocate a comparable budget to direct object verification by a strong model, repeated sampling, or stricter filtering, and then compare downstream performance after preference training. The current experiments do not address this question.
-
Contribution of Step 2. Beyond the source and content of the cues, what substantive methodological advance does Step 2 offer over existing AI-ranked policy sampling and hint-guided generation? Since RPO is cited, why is it omitted from the experimental comparisons?
-
Fairness of baseline comparisons. Which Table 1 baselines were retrained under your setup? Were source images, training-set sizes, optimization budgets, and trainable components matched? In particular, did the baselines also update the vision encoder and projector? Please summarize any material differences.
-
The claim of policy-aligned preference construction is insufficiently substantiated. Candidates are generated with cue conditioning (Eq. 5), whereas DPO excludes the cue (Eq. 7). Self-generation therefore does not by itself establish closeness to the original conditional distribution.
4 comments
Response to Reviewer khvB (2/2)
W2/Q4. Methodological novelty of step 2 and comparison with prior methods
The contribution of ViPSy is the construction and utilization of reliable visual cues to generate visually grounded preference data. Below, we clarify what Step 2 adds over each of the two lines of prior work.
Sampling-based methods with a judge model (RLAIF-V): These methods keep preference data close to the policy distribution, but can still underuse visual information, since the policy's rollouts may be driven by language priors rather than image-specific evidence. In Step 2, the reliable visual cues from Step 1 address this underuse of visual information: they guide the target policy to generate diverse responses with varying degrees of visual grounding, enabling the construction of preference pairs that combine visual grounding with policy alignment. As illustrated in Figures 3 and 6, these cues help produce responses with distinguishable levels of visual grounding, making it easier for the judge model to identify preferred and dispreferred responses.
Hint-guided generation (RPO): Unlike RPO, which relies on surrogate ground-truth responses to identify errors and generate corrective hints, ViPSy derives visual cues directly from the input image through semantic synthesis and consensus-based filtering, without requiring reference responses. Moreover, RPO takes the hint-conditioned response as the preferred response and the response generated without the hint as the dispreferred one, whereas in ViPSy the judge model selects both responses from the same cue-conditioned candidates. We did not include RPO in our experimental comparisons because we could not find a publicly available official implementation. We have added this distinction to the paragraph "Mitigating hallucination in VLMs" in Related Work (Appendix B).
Q5. Fairness of baseline comparisons and training configurations
The baselines are described in Appendix E, and the results in Table 1 follow each baseline's original training setup rather than a unified retraining protocol, i.e., none of the Table 1 baselines was re-trained under our setup.
The table below summarizes every training-based Table 1 baseline (LLaVA-1.5-7B) with respect to the items raised by the reviewer. To clarify these differences in detail, we have added this table as Table 25 to the paragraph "Training setups of baselines" in Appendix O.
| Method | Source images | Training-set size | Optimization budget | Vision encoder | Projector | LLM |
|---|---|---|---|---|---|---|
| EFUF | MSCOCO | 23K | 1 epoch | ✗ | ✓ | ✗ |
| LLaVA-RLHF | MSCOCO | 10K | 4 epochs | ✗ | ✗ | ✓ |
| HALVA | Visual Genome | 21.5K | 1 epoch | ✗ | ✗ | ✓ |
| HA-DPO | Visual Genome | 16K | 1 epoch | ✗ | ✗ | ✓ |
| POVID | MSCOCO | 17K | 4 epochs | ✗ | ✓ | ✓ |
| TPR | MSCOCO, ShareGPT4V, VQAv2 | 21K | 1 epoch | ✗ | ✓ | ✓ |
| RLAIF-V | MSCOCO, ShareGPT4V, VQAv2, etc. | 16K | 4 epochs | ✗ | ✓ | ✓ |
| OPA-DPO | RLAIF-V | 4.8K | 2 SFT + 4 DPO epochs | ✓ | ✓ | ✓ |
| SENTINEL | Visual Genome | 8.6K | 1 epoch | ✗ | ✗ | ✓ |
| P²-DPO | RLHF-V | 5.4K | 4 epochs | ✗ | ✗ | ✓ |
| ViPSy (Ours) | Visual Genome | 7K | 1 epoch | ✓ | ✓ | ✓ |
Q6. Policy alignment between cue-conditioned generation and cue-free DPO
We empirically examine distributional closeness between cue-conditioned generation and cue-free generation through KL divergence analysis. Specifically, is the original policy with the method-specific input (the cue for ViPSy and the context for SENTINEL), while is the same policy without it. Both receive identical images and prompts. We evaluate each method on its own preferred and dispreferred responses, since these are the responses used for DPO. As shown in Table 17, ViPSy achieves substantially lower KL divergence than SENTINEL (0.018 vs. 0.074), providing empirical evidence that our preference construction results in smaller deviations from the policy distribution.
Thank you for the detailed reply and the additional experiments. My two main concerns remain.
-
The practical value of image synthesis is still unclear given the high time cost.
- The new results suggest that cues with higher accuracy and better coverage are associated with better downstream hallucination mitigation. Cue quality is therefore central to the paper as a whole. The repeated extraction baseline is too limited to establish what image synthesis adds. Which VLM was used for repeated extraction in W1/Q1–2?
- A more direct comparison would use the same VLM to list candidate objects from the original image and a separate judge model to verify each object against that image. Alternatively, the authors could use a stronger VLM to extract key cues directly and test whether it achieves similar hallucination reduction without the time cost of image generation.
-
Stage 2 offers limited novelty. Antidote [1] and SENTINEL [2] already use visually grounded context to construct preference pairs, with Antidote also removing the object hints during DPO. I still do not see a substantial advance over these earlier approaches.
Overall, I think the core of the paper lies in how Stage 1 constructs visual cues. The current evidence is still insufficient to show that this approach is necessary and practical.
[1] Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception
[2] Mitigating Object Hallucinations via Sentence-Level Early Intervention
Dear Reviewer khvB,
Thank you for your follow-up and for raising these points before the end of the discussion period. We appreciate the opportunity to address them more thoroughly.
RQ1. Is image synthesis worth the cost?; image synthesis vs. direct cue extraction
We first clarify that the repeated-extraction baseline in W1/Q1–2 uses Qwen2.5-VL-72B, the same extractor as ViPSy, with the same consensus-based filtering and aggregation. Following your suggestions, we further evaluated stronger extraction and verification baselines without image synthesis. These comparisons show that ViPSy consistently achieves better downstream hallucination mitigation, while its SD-1.4 variant retains this advantage at a comparable computational cost.
Specifically, we evaluated (i) propose-then-verify, where Qwen2.5-VL-72B lists candidate objects and Qwen3-VL-235B-A22B verifies and merges them against the original image in place of our filtering, and (ii) direct extraction using Qwen3-VL-235B-A22B with our filtering. We tested each with 8, 16, and 32 repetitions. For direct extraction, a cue is retained only if it appears in every repetition, following the same unanimity criterion used in ViPSy.
| Method | VLM | Rep. | Correct. (%) ↑ | Recall (%) ↑ | Avg. Cues ↑ | Coverage (%) ↑ | GPU-hours ↓ | Hal. Avg. ↓ |
|---|---|---|---|---|---|---|---|---|
| Propose-then-verify | Qwen2.5-VL-72B + Qwen3-VL-235B | 8 | 84 | 39.8 | 3.9 | 92 | 13.5 | 5.87 |
| Propose-then-verify | Qwen2.5-VL-72B + Qwen3-VL-235B | 16 | 87 | 40.2 | 4.2 | 90 | 15.9 | 5.34 |
| Propose-then-verify | Qwen2.5-VL-72B + Qwen3-VL-235B | 32 | 89 | 38.8 | 4.3 | 89 | 20.5 | 5.21 |
| Direct extraction | Qwen3-VL-235B | 8 | 87 | 40.8 | 4.3 | 92 | 12.4 | 5.84 |
| Direct extraction | Qwen3-VL-235B | 16 | 92 | 36.8 | 3.4 | 90 | 14.9 | 5.33 |
| Direct extraction | Qwen3-VL-235B | 32 | 95 | 32.4 | 3.0 | 89 | 18.4 | 5.12 |
| ViPSy (SD-1.4) | Qwen2.5-VL-72B | 8 | 93 | 38.4 | 3.3 | 92 | 16.9 | 4.17 |
| ViPSy (SD-3.5-L) | Qwen2.5-VL-72B | 8 | 95 | 40.2 | 3.8 | 94 | 49.4 | 3.72 |
ViPSy with SD-3.5-L achieves both 95% correctness and 40.2% recall. The stronger direct extractor reaches 95% correctness only at 32 repetitions, with recall falling to 32.4%, while propose-then-verify remains at 84-89% correctness. The benefit also holds at a competitive cost: ViPSy with SD-1.4 takes 16.9 GPU-hours, within the baseline range (12.4-20.5), and achieves a lower Hal. Avg. than every baseline (4.17 vs. 5.12-5.87).
The key difference is that ViPSy takes consensus across semantically aligned image variants rather than repeated outputs from the same image, allowing it to filter errors that may persist across repeated extraction.
RQ2. Differences from Antidote and SENTINEL
ViPSy’s Stage 2 differs from Antidote and SENTINEL in how responses are evaluated and selected: it uses an external judge to evaluate each full response against the image, rather than relying on cue conditioning or object presence alone.
Antidote takes the hint-conditioned response as preferred and the hint-free response as dispreferred. Its hints are checked with Grounding-DINO, but the responses themselves are not verified against the image. SENTINEL uses open-vocabulary detectors to check object presence, with context drawn from the policy’s preceding sentences. ViPSy instead evaluates full responses for object, relational, attribute, and spatial accuracy (Table 18).
This distinction matters because accurate cues alone do not guarantee grounded responses: even with 95% cue correctness (Table 16), preferred and dispreferred responses generated with the same cue differ substantially in visual grounding (0.863 vs. 0.441, Table 18). These results highlight why cue quality alone is insufficient for selecting preferred responses.
Thank you again for your thoughtful feedback. We hope these additional comparisons and clarifications help further address your concerns.
Best regards,
Authors
Response to Reviewer khvB (1/2)
We sincerely appreciate your constructive feedback. We respond to each of your questions one-by-one in what follows. We have made every effort to address your concerns thoroughly. If any concerns remain, we would greatly appreciate hearing them and the opportunity to address them further.
W1/Q1-2. ViPSy with versus without semantic image synthesis: retention criterion, cue counts, and coverage
ViPSy is designed to reduce potential synthesis artifacts, including those inherited from incorrect captions and T2I biases, through pairwise comparison with the original image and consensus-based filtering. To further demonstrate the benefits of semantic image synthesis, we compare the cue quality and coverage of ViPSy with and without synthesis.
We have added these results as Table 16, with the corresponding analysis in the paragraph "Cue quality and coverage" in Appendix J and a summary in "Cue-guided preference construction" in Section 3.4.
W1/Q3. Comparison under a comparable computational budget
Under comparable computational budgets (16.9 vs. 16.7 GPU-hours), ViPSy (SD-1.4) achieves substantially better hallucination mitigation (4.17 vs. 5.52) than ViPSy w/o synthesis with 32 repetitions. For this comparison, we extend ViPSy w/o synthesis to 16 and 32 repetitions, retaining cues only when they appear in all 16 or 32 extractions, respectively.
ViPSy (SD-1.4) also outperforms the best setting without synthesis, 16 repetitions (4.17 vs. 5.16), and increasing repetitions does not close the gap (5.94 → 5.16 → 5.52 for 8, 16, and 32 repetitions). In addition, the lighter SD-1.4 reduces the offline cost of ViPSy from 49.4 to 16.9 GPU-hours, while the default ViPSy (SD-3.5-L) achieves the best Hal. Avg. (3.72). Extending to 16 and 32 repetitions also makes the unanimity rule stricter, so the number of repetitions and the filter strictness change together, and the weaker result at 32 repetitions is consistent with its lower cue recall (29.8%, Table 16). Within the unanimity settings we tested, repeated extraction without synthesis does not match ViPSy's hallucination mitigation under comparable computational budgets. We have added the corresponding computational cost analysis to the paragraph "Repeated extraction under a comparable budget" in Appendix J.