- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
SensiEdit predicts head-wise sensitivity to image replacement and uses it to modulate learned spatial biases on visual attention logits. A gate controls when editing occurs, while the LVLM stays frozen. The paper reports object-hallucination reductions across several backbones and examines head weighting, gating, runtime and caption quality.
Critical strengths
The LLaVA result is substantial: CHAIRs/CHAIRi falls from 51.30/16.80 to 22.60/5.32. The full model also retains more recall than the reported equal-head control, 74.07 versus 63.71. The POPE and AMBER gains provide evidence beyond the main captioning experiment.
Critical weaknesses
• The method’s motivation is unclear. Head-wise heterogeneity does not establish the proposed intervention rule. Figure 1 shows that heads respond differently to image replacement, but does not fully explain why low-sensitivity heads require stronger visual attention intervention.
• The relationship between AVS and hallucination is insufficiently established. AVS measures a change in attention weights under image replacement, not whether an object mention is supported by the image.The authors acknowledge that AVS is not a validated causal measure of visual reliance. What is missing is direct evidence that AVS, and its predicted counterpart, track hallucination risk and predict which heads or decoding steps benefit from intervention.
Questions for authors
• On held-out examples, does low AVS predict hallucination and greater benefit from stronger intervention? Can the allocation rule be compared under matched training and bias budgets?
• When trained on the COCO dataset, can it be generalized to other datasets with different distributions, or is retraining required on the dataset with the new distribution?