- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes SCAN for mitigating object hallucination in LVLMs. It retains a frozen, backbone-specific hallucination-associated subspace and trains a lightweight Navigator to predict context-adaptive anchors and direction-wise weights from the orthogonal residual. A closed-form proximal update modifies the subspace coordinates while preserving the residual at the current intervention position. The squared-error analysis characterizes the approximation gap of fixed targets when the conditional mean of the desired anchor varies with context. Experiments report hallucination reductions, hallucination–recall trade-offs, and low decoding overhead in the measured configuration.
Critical strengths
-
The paper provides useful empirical evidence for context-adaptive navigation within a shared subspace. Table 5 examines fixed versus adaptive anchors, uniform versus direction-wise strengths, and alternative basis constructions. Appendix G reports hallucination and recall jointly across intervention strengths. At nearly matched recall, SCAN achieves lower CHAIRs and CHAIRi than the uniform zero-target control. These comparisons support an improved hallucination–recall trade-off within the evaluated setting.
-
Extensive experiments across models and tasks demonstrate that SCAN consistently reduces object hallucinations and achieves a favorable hallucination–recall trade-off. Additional efficiency measurements show that SCAN retains decoding throughput close to that of the original backbone. The results support selective intervention and low overhead under the reported protocols.
Critical weaknesses
-
Comparative coverage is incomplete for closely related subspace-intervention methods. Nullu and HulluEdit appear in the 64-token CHAIR comparison, but not in the POPE or AMBER tables. The Qwen2.5-VL and Qwen3-VL results compare SCAN with Vanilla only. Thus, the experiments provide different levels of comparative evidence across backbones and benchmarks. Table 1 explains that some omitted configurations lack specified hyperparameters; this is a relevant constraint, but the comparative conclusions should remain tied to the configurations actually evaluated.
-
The reporting of per-backbone adaptation and generalization scope could be more complete. The basis and preference pools are constructed from COCO images and annotations. AMBER provides broader benchmark evidence, so cross-dataset generalization should not be described as entirely untested. However, its aggregate results do not separately characterize non-COCO image sources or categories outside the training annotation inventory. Appendix D specifies backbone-specific basis construction and dimension-dependent Navigator architectures, while the detailed optimization settings and approximately 50-minute cost are primarily documented for LLaVA-1.5 7B. A concise per-backbone configuration and cost summary would make the practical adaptation requirements clearer.
Questions for authors
-
Given the backbone-specific basis construction in Appendix D.1, could you provide a per-backbone summary of the intervention layer, rank, inference strength, Navigator training procedure, and offline cost? Please distinguish reused image pools from regenerated response pairs and clarify how the reported defaults were selected.
-
Could you characterize the scope of generalization more explicitly while recognizing the AMBER results already provided? If available, performance separated by image source or categories outside the training inventory would help. Otherwise, please distinguish demonstrated cross-benchmark performance from broader domain robustness without implying that domain-specific rebuilding is necessarily required.
-
Could you extend the Nullu and HulluEdit comparisons to POPE or AMBER where feasible, or explain the implementation constraints? Table 1 already notes unspecified hyperparameters for omitted configurations such as ASCD on Qwen-VL-Chat; please clarify whether a justified configuration can be evaluated and keep the comparative claims appropriately scoped if it cannot.
4 comments
Response to Reviewer a3Gx (Part 2/2): Per-Backbone Adaptation and Generalization
[W2 & Q1] Per-Backbone Configuration and Training Cost
Following the reviewer's suggestion, we added a comprehensive per-backbone summary of intervention configurations, response-pair reuse, and training costs.
Table R4: Per-backbone configurations and Navigator training costs.
| Backbone | Layer | Preference Pairs | Response Source | α | Time (min) |
|---|---|---|---|---|---|
| LLaVA-1.5 7B | 31 | 5,334 | Own | 0.10 | 49 |
| LLaVA-1.5 13B | 39 | 5,334 | LLaVA-1.5 7B | 0.10 | 269 |
| LLaVA-NeXT-7B | 31 | 5,334 | Own | 0.10 | 146 |
| Qwen-VL-Chat | 27 | 5,334 | Own | 0.05 | 58 |
| Qwen2.5-VL-7B | 28 | 5,334 | Own | 0.05 | 44 |
| Qwen3-VL-8B | 28 | 5,334 | Qwen2.5-VL | 0.05 | 35 |
(1) Data Reuse and Training Cost. All backbones use the same MS COCO image pools, with disjoint images for basis construction (3,072 pairs) and Navigator preference learning (5,334 pairs). Each backbone constructs its own rank-128 basis and trains a separate Navigator for 500 steps. Four backbones generate their own response pairs, while LLaVA-1.5 13B and Qwen3-VL reuse those from LLaVA-1.5 7B and Qwen2.5-VL, respectively. Reusing response pairs still requires backbone-specific hidden-state extraction, basis construction, and Navigator training. The reported times cover complete Navigator training runs on one RTX 4090, excluding offline response generation, basis construction, and benchmark evaluation. LLaVA-1.5 13B additionally requires CPU offloading.
(2) Default Configuration Selection. The 500-step checkpoint achieves the lowest preference loss on the independent 533-pair development split of LLaVA-1.5 7B, and this training budget is adopted across backbones. Rank 128 and layer 31 are selected through CHAIR sensitivity analysis on LLaVA-1.5 7B. Other backbones retain rank 128 and use a layer in the last quarter of their decoder blocks without further layer tuning. The inference strength is selected at the knee of the CHAIR hallucination–recall curve (α = 0.10 for LLaVA and 0.05 for Qwen) and then fixed for the reported benchmark evaluations.
Full configurations and costs are provided in Appendix G.3 (Table 15, page 32), with selection details in Appendix G.1–G.2 (pages 30–31).
[W2 & Q2] Scope of Cross-Benchmark Generalization
We thank the reviewer for this insightful suggestion. SCAN demonstrates cross-benchmark transfer to AMBER without additional adaptation, reducing CHAIR from 8.3 to 5.6 and Hal from 37.8 to 23.9, with only a modest decrease in Cover (51.8 → 50.4; Table 3).
Category-Level Generalization Analysis. Since AMBER does not identify the source of each image, we further analyzed generalization by object-category coverage. Our preference data covers the 80 MS COCO categories, and 77 of AMBER's 340 labels map to them under the CHAIR synonym list. We classify AMBER object mentions into in-inventory and out-of-inventory groups and evaluate their hallucination counts and object coverage.
Table R5: Category-level hallucination and coverage analysis on AMBER
| Metric | Vanilla | SCAN | Change |
|---|---|---|---|
| In-inventory hallucinations | 514 | 289 | −44% |
| Out-of-inventory hallucinations | 193 | 191 | −1% |
| Cover (in-inventory) | 86.0 | 83.5 | −2.5 |
| Cover (out-of-inventory) | 37.1 | 36.3 | −0.8 |
Note: Hallucination changes are relative percentages; Cover changes are percentage points.
SCAN reduces hallucinations by 44% on in-inventory categories, which account for 73% of Vanilla's hallucinations. In contrast, out-of-inventory hallucinations, mainly scene-related objects such as sun, beach, and sky, remain largely unchanged, with only a 0.8-point decrease in coverage. These findings demonstrate cross-benchmark effectiveness without additional adaptation, while clarifying that the improvements primarily concentrate on categories represented in the preference data. Broader cross-domain robustness remains to be evaluated.
The analysis is provided in Appendix L.2 (Table 21, page 37), with the generalization scope clarified in Section 4.2 (page 9).
Based on the above clarifications, we hope that our responses have adequately addressed your concerns and warrant a more favorable assessment.
Thank you for the additional experiments and clarifications.
-
The new Nullu and HulluEdit results on POPE and AMBER, together with the ASCD and VTI results on Qwen-VL-Chat, address my main concern about baseline coverage.
-
The revised appendix clarifies the intervention settings, response-pair reuse, training procedure, and Navigator training cost for each backbone. It also makes clear which offline stages are excluded from the reported timings.
-
The AMBER category breakdown answers my question about the scope of transfer.
These additions address my main concerns and strengthen the evidence for SCAN’s hallucination–recall trade-off. I change my rating from 2 (weak rejection) to 3 (weak acceptance).
Thank you for your positive feedback and for raising your score. We are glad our responses have addressed your concerns, and your suggestions have truly helped us make the paper more complete and convincing. We greatly appreciate your time and thoughtful review!
Response to Reviewer a3Gx (Part 1/2): Comprehensive Baseline Comparisons
Thank you for your constructive comments. We have addressed your main concerns through a per-backbone configuration and training-cost summary, a category-level AMBER analysis, and expanded baseline comparisons on POPE, AMBER, and Qwen-VL-Chat. The revised manuscript has been uploaded to the submission system, with changes addressing your comments highlighted in green.
[W1 & Q3-1] Extended Comparisons with Nullu and HulluEdit
Following the reviewer's suggestion, we have extended the Nullu and HulluEdit evaluations to POPE and AMBER using their official implementations under the same prompts, decoding settings, and evaluation protocols as SCAN on LLaVA-1.5 7B.
Table R1: POPE comparison (Accuracy/F1, %)
Table R2: AMBER comparison (official metrics, %)
SCAN consistently outperforms Nullu and HulluEdit across all POPE splits, whereas both baselines fall below Vanilla. On AMBER, SCAN outperforms Nullu across all four metrics. Although HulluEdit achieves lower CHAIR (4.1 vs. 5.6), SCAN preserves substantially higher object coverage (50.4 vs. 43.7) while achieving lower Hal and Cog, demonstrating a more favorable balance between hallucination mitigation and content preservation.
[W1 & Q3-2] Completed ASCD and VTI Comparisons on Qwen-VL-Chat
We have completed the previously unevaluated ASCD and VTI configurations on Qwen-VL-Chat using their official implementations. The previously cited VTI results have been replaced with our own evaluations, ensuring that all entries in Table 1 follow the same protocol.
Table R3: Qwen-VL-Chat comparison on POPE and CHAIR (%).
SCAN achieves the highest POPE accuracy/F1 across all three splits and the lowest CHAIR_s/CHAIR_i, further supporting its effectiveness under consistent evaluation settings. For Qwen2.5-VL and Qwen3-VL, we clarify that the reported results demonstrate improvements over Vanilla rather than comparisons with untested baselines.
The updated AMBER and Qwen-VL-Chat comparisons are provided in Table 3 and Table 1 (page 8), respectively.