- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes DriftDPO, which ranks model-generated image captions using the alignment between generated-token and visual-token representations, then trains on the resulting preference pairs with DPO. It reports reduced object hallucination and increased object coverage without additional inference steps.
Critical strengths
None
Critical weaknesses
-
The central mechanism claim is unsupported. An AUC of 0.589 shows limited association, while selecting captions by cosine similarity does not establish causality. The paper also conflates two different comparisons—hallucinated objects versus background tokens, and hallucinated versus correct objects—to explain its preference reversal. The proposed architectural explanation lacks controlled evidence.
-
The experimental evidence falls far short of the claims. The main text provides verifiable training results primarily for one 3B model, while referring to an appendix absent from the provided PDF for cross-model results and key ablations. Moreover, CHAIRi falls as object coverage rises, but the paper does not report the raw counts needed to establish that the absolute number of hallucinated object mentions decreases.
-
Novelty and baselines are inadequate. Prior work already studies model-generated preference pairs for hallucination mitigation. The paper neither establishes a clear advance over such methods nor compares against relevant training-based baselines, making its “first,” “only,” and “state-of-the-art” claims unconvincing.
Questions for authors
See weakness