- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper characterizes when a contrastive coefficient can promote a ground-truth object token above hallucinated competitors. It identifies configurations that no positive coefficient can correct and proposes TAVCD, a small controller that predicts a signed coefficient from logit statistics at each decoding step. The controller is trained on oracle labels obtained from logged VCD trajectories. Experiments cover five frozen vision-language models on MSCOCO and PASCAL VOC captioning.
Critical strengths
The distinction between pairwise blocking and incompatible promotion intervals is useful. Although the analysis follows directly from an affine logit margin, it explains why increasing a positive coefficient can worsen a particular error and motivates considering negative coefficients. The experiments consistently improve or match VCD on the reported hallucination metrics across both datasets, with no updates to the backbone.
Critical weaknesses
-
The experiments do not yet establish the value of the learned controller over simpler adaptation. On MSCOCO, the CHAIRi reductions relative to VCD are 1.0, 0.1, 1.3, 0.3 and 0.1 percentage points in Table 1's backbone order. These are single-seed results on 500 images, without uncertainty estimates. The paper does tune a global positive coefficient on validation data, but that does not replace a comparison with a simple gate using one tuned signed coefficient. Such a control would distinguish the value of selecting when to intervene from that of learning a 40-bin magnitude distribution. The headline 24-point improvement on LLaVA VOC is also achieved by VCD: both reach CHAIRi 14.0. It is a valid comparison with Regular decoding, but does not establish the added value of TAVCD.
-
The deployment features need clarification. Table 2 defines t/T using the total caption length, which is unavailable before generation terminates. If inference substitutes a maximum generation length or an estimate, the paper should specify it and use the same definition when training the predictor. As written, the online feature computation cannot be reproduced unambiguously.
-
The oracle labels provide a weaker guarantee than the term “structurally uncorrectable” suggests. Appendix B searches a bounded, discrete grid on the stored top-50 support; failure there does not establish that the full real-valued feasible set is empty. A C3 violation excludes positive coefficients, not necessarily negative ones.
-
The recall tradeoff is substantial for BLIP-2 on VOC: 61.6 with VCD falls to 49.0 with TAVCD. CHAIRi improves from 16.4 to 16.2 and CHAIRs from 18.0 to 14.2, but these reported gains accompany considerably less object coverage. Object-mention counts and caption lengths would help characterize this tradeoff.
-
The reported “less than 1%” overhead concerns the head, not the two-branch decoder. End-to-end latency against Regular decoding and VCD is still missing.
Questions for authors
What value replaces the eventual caption length T during inference, and was that same definition used to construct the training features?