- Rating
- 1(−1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1(−1)
Read review
Paper summary
This paper proposes SCAS, a training-free method designed to mitigate hallucinations while enhancing the general reasoning capabilities of VLMs. SCAS leverages a small set of calibration samples to construct basis vectors representing factual and hallucination-related semantic subspaces, which are then used to steer the model's internal representations. Experiments on the Qwen3-VL series and InternVL models demonstrate the effectiveness of the proposed method.
Critical strengths
The paper not only focuses on mitigating multimodal hallucinations but also considers the potential degradation of general capabilities caused by intervention methods. Rather than solely pursuing improvements in hallucination-related metrics, evaluating hallucination mitigation and general capability preservation within a unified framework represents a valuable research direction.
Critical weaknesses
-
The experimental results do not sufficiently support the main motivation and claims. SCAS aims to mitigate hallucinations while enhancing the general capabilities of VLMs, motivated by the difficulty of balancing these two objectives in existing methods. However, the experimental results do not convincingly demonstrate that SCAS achieves a better and more consistent trade-off than existing approaches such as Revis. For example, SCAS underperforms Revis on multiple hallucination-related metrics for Qwen3-VL-2B. Furthermore, the paper identifies the model dependence and limited cross-model stability of existing steering directions as key limitations, yet provides no direct experimental evidence demonstrating how SCAS addresses these issues. Overall, the results do not adequately substantiate the paper's central motivation and claimed contributions.
-
Insufficient discussion of related work and methodological novelty. The paper primarily discusses earlier activation steering methods, with limited coverage of recent advances in activation steering and representation intervention. Apart from Revis, closely related methods are not adequately reviewed or compared, particularly those involving semantic direction extraction, subspace manipulation, and inference-time interventions. Consequently, the limitations of existing approaches highlighted in the Introduction are not sufficiently substantiated, and the methodological novelty of SCAS over prior work remains unclear.
-
The experimental evaluation is not sufficiently comprehensive.
- Limited baseline comparisons. The experiments mainly compare SCAS with Revis and VCD, while VCD follows a different technical paradigm. Comparisons with more closely related activation steering methods are needed to establish the advantages of SCAS. Lack of inference efficiency analysis. SCAS introduces additional subspace projections and matrix operations during inference, but the paper does not report the resulting latency or computational overhead. An efficiency comparison would help assess its practical applicability.
- Insufficient evaluation on larger models. For Qwen3-VL-8B, SCAS is only compared against the original model, without comparisons to existing methods. Moreover, the performance improvements are relatively modest, limiting the evidence for its scalability and comparative advantages.
- Incomplete ablation studies. Although SCAS incorporates several components, the individual contributions of key designs, such as six-state activation extraction, modality debiasing, and hybrid PCA-based subspace construction, are not adequately examined. Additional ablations are needed to justify their necessity and effectiveness.
Other issues: The paper incorrectly applies the ICLR 2026 template.
Questions for authors
no other questions
4 comments
I thank the authors for the rebuttal and new experiments. However, my main concerns remain unresolved.
The rebuttal does not address the central issue. The original abstract motivated the work by the trade-off between mitigating hallucinations and preserving general capabilities, yet the method underperforms baselines on several hallucination benchmarks. The authors did not respond to this point directly; instead, they modified and removed the statement without any explanation. Removing a claim is not the same as resolving the issue.
Moreover, the revised abstract reads more like a summary of experimental results than an abstract. I recommend rewriting it to clearly state the problem, motivation, method, and main contributions, and to accurately reflect the evidence.
Unexplained changes to the baseline results in Table 1. Compared with the original submission, the baseline (ReVis) results in Table 1 have been lowered on multiple metrics, with no explanation provided. This casts doubt on the reliability of the experimental results.
Computational efficiency.
Please provide a direct comparison between the proposed method and the baselines under identical experimental settings, reporting at least inference time and GPU memory usage.
Based on the reasons above, I have lowered my score for this paper. If the authors can adequately address these issues, I would be willing to reconsider my assessment.
Dear Reviewer pt7f,
Part 1/2: The stronger Revis reference and the hallucination–utility trade-off. Thank you. We clarify the stronger-comparator rationale and give direct evidence across both backbones. Q2/I4 denote Qwen3-VL-2B/InternVL3.5-4B. Random/Popular/Adv are RePoPe F1 settings; AMBER-d F1 pools all 14,216 questions; MMBench is local single-pass accuracy. Part 2 gives the direct efficiency comparison.
1. Why the original paper disabled the Revis gate. In our initial RePoPe/AMBER-d checks, the thresholded adaptation produced baseline-equivalent answers. We therefore disabled the gate to obtain a more actively intervening, stronger Revis comparator. This rationale was explicitly stated in the original submission. The configuration is gate disabled (tau=None in the evaluator, “Revis w/o τ” in Table 1); a numeric τ=0 would still apply a gate.
Following the reviews, we additionally used the official Revis calibration code, with backbone-specific adaptation, and completed benchmark reruns. Q2 results are:
| Setting | Random F1↑ | Popular F1↑ | Adv F1↑ | AMBER-d F1↑ | MMBench↑ | MME↑ |
|---|---|---|---|---|---|---|
| Revis original, ungated | 95.29 | 93.23 | 92.26 | 89.90 | 83.93 | 2086.79 |
| Revis calibrated, gated | 95.30 | 92.82 | 91.70 | 89.27 | 83.16 | 2034.29 |
| SCAS | 95.17 | 93.12 | 92.08 | 89.86 | 84.62 | 2101.92 |
The original ungated reference exceeds the newly calibrated configuration on Popular/Adv F1, AMBER-d F1, MMBench and MME. We retain its stronger scores. The original values have not been lowered or replaced: the revised Table 1 reports the original and newly calibrated configurations as separate rows. The rerun also changes hook/index adaptation to follow upstream behavior, so this comparison does not isolate a τ-only causal effect. Appendix L gives the protocol.
2. Small discriminative gaps versus better utility and usable descriptions. On Q2, original Revis's advantages over SCAS are 0.12/0.11/0.18 percentage points on Random/Popular/Adv F1, and 0.04 points on full AMBER-d F1. SCAS instead gains 0.69 MMBench points and 15.13 MME points over that stronger reference. This is direct evidence for the joint motivation: competitive hallucination metrics together with higher general-task utility. SCAS's benefit does not rely on substituting a weaker Revis row.
Free-form generation exposes an additional trade-off. On all 1,004 Q2 AMBER-g images, with a 512-token limit, the original ungated Revis exhibits severe repetition and reduced descriptive coverage. R4 is mean literal four-gram repetition:
| Q2 setting | CHAIR↓ | Coverage↑ | Repetition R4↓ |
|---|---|---|---|
| Revis original, ungated | 4.4 | 44.1 | 62.95% |
| Revis calibrated, gated | 6.0 | 75.5 | 2.04% |
| SCAS | 4.7 | 69.2 | 13.22% |
Thus its slightly lower CHAIR (4.4 vs 4.7) accompanies coverage 44.1 vs 69.2 and repetition 62.95% vs 13.22%. CHAIR alone cannot characterize useful hallucination mitigation. The official calibrated version restores coverage and reduces repetition; we report it separately and do not attribute the original degeneration to every Revis configuration. SCAS offers the reported grounding/utility balance against both references. The small-gap statement above concerns the listed aggregate F1 metrics; subtask and model-specific exceptions remain visible in the paper.
3. InternVL supplies a broader advantage over Revis. On I4, SCAS exceeds both Revis configurations in every RePoPe F1 setting, full AMBER-d F1 and MMBench:
| Setting | Random F1↑ | Popular F1↑ | Adv F1↑ | AMBER-d F1↑ | MMBench↑ | MME↑ |
|---|---|---|---|---|---|---|
| Revis original, ungated | 92.30 | 87.36 | 83.87 | 88.29 | 85.65 | 2237.00 |
| Revis calibrated, gated | 92.05 | 88.34 | 84.99 | 87.92 | 86.77 | 2195.38 |
| SCAS | 92.81 | 89.19 | 86.16 | 88.69 | 87.03 | 2165.25 |
Against original Revis, SCAS improves adversarial F1 by 2.29 points, MMBench by 1.37 points and full AMBER-d F1 by 0.40 points. Revis retains higher MME. Together, the Q2 and I4 comparisons support evaluating the joint balance across hallucination and general capability, with clear SCAS advantages across multiple I4 benchmarks. They support our motivation beyond a single favorable Q2 baseline configuration.
We retained this motivation and rewrote the abstract around the problem, six-state controlled construction, hybrid subspaces and complementary suppression/amplification before summarizing evidence. The construction is fitted separately for each backbone. Fresh reconstruction, component ablations and five same-pool calibration refits further support its contribution; they are not presented as cross-model vector transfer.
Dear Reviewer pt7f,
Part 2/2: Completed direct efficiency comparison. We have now completed a same-session comparison of unmodified decoding, SCAS, both Revis configurations, all three calibrated VISTA variants and VCD on Qwen3-VL-2B. All conditions use one model load on the same RTX 4090, the same 16 images/prompts, batch size 1, BF16/SDPA, greedy decoding and fixed 4/64-token outputs. We use warm-up, rotated method order and five repetitions per image: 1,440 timed calls in total.
| Method | 4 tokens, ms | 64 tokens, ms | 64-token ratio | Peak allocated, MiB |
|---|---|---|---|---|
| Baseline | 226.08 | 2193.27 | 1.000× | 4242.81 |
| SCAS | 233.52 | 2273.14 | 1.036× | 4242.81 |
| Revis (original, ungated) | 225.08 | 2216.05 | 1.010× | 4242.81 |
| Revis (official calibration) | 230.32 | 2218.55 | 1.012× | 4242.81 |
| VISTA-VSV | 527.87 | 3337.52 | 1.522× | 4424.52 |
| VISTA-SLA | 240.28 | 2402.29 | 1.095× | 4242.83 |
| VISTA full | 540.36 | 3389.81 | 1.546× | 4424.52 |
| VCD (paper adapter) | 535.72 | 5508.27 | 2.511× | 4445.99 |
| VCD (cached-noise control) | 437.65 | 4104.09 | 1.871× | 4327.60 |
Timing is CUDA-synchronized model-side wall time, including intervention setup/transfers, generation and VISTA's per-image visual-vector extraction. It excludes model loading, offline calibration, image/noise preprocessing and disk writes. VISTA uses the final calibrated VSV strength 0.01 and SLA coefficient 0.003 at decoder indices 18–23. Revis uses the original ungated setting and its separately identified official calibration; no threshold sweep is involved.
For 64-token outputs, SCAS's mean overhead relative to unmodified decoding is 3.64% (paired image-cluster bootstrap 95% CI [-1.07%, 9.06%], 20,000 replicates). SCAS retains the baseline's measured peak allocated memory and is faster than full VISTA and both VCD implementations in this comparison. All raw calls, token outputs and allocated/reserved memory peaks are retained.
The VCD paper adapter recomputes the noisy prefix without a KV cache. To avoid attributing that implementation cost to the contrastive formula itself, we additionally report an independent noisy-branch cache control. That control was checked step-by-step against Hugging Face's native cached generation; the logits match exactly in the checked forced-prefix case. Cached versus full-prefix BF16 computation can change long greedy outputs, so this is a labeled efficiency control, not a new accuracy result or a replacement for the paper's evaluated VCD configuration.
The revised manuscript now includes the direct comparison in Table 6, with the full component table and protocol in Appendix M.
Dear Reviewer pt7f,
Thank you. New experiments are included in the revised paper (Secs. 4.5–4.6; Apps. J–N). Q2/I4/Q8 denote Qwen3-VL-2B/InternVL3.5-4B/Qwen3-VL-8B; Adv denotes RePoPe adversarial; MMBench is local single-pass accuracy.
1. Contribution and positioning. SCAS reuses Revis's OH-100 factual/hallucinated pairs. Its contribution is their six-state real-image/text-only/blank-image construction, modality and Yes/No projections, mean-plus-PCA bases and asymmetric residualization, followed by complementary suppression/amplification in one forward stream without weight updates. Revised related work distinguishes ASD's prediction contrast, SHARP's feature intervention, VISTA's visual reinforcement/logit augmentation and Dynamic Multimodal Activation Steering. This shared construction is evaluated across architectures with separately extracted subspaces.
2. Completed close comparisons. We adapted upstream VISTA formulas to Qwen and calibrated VSV strength, SLA coefficient and layer window on COCO images disjoint from evaluation images. Fit/holdout images are distinct; parameters are frozen before the rerun.
SCAS leads all three variants on both utility benchmarks, adversarial F1 and aggregate AMBER-d F1, with lower CHAIR. VISTA retains higher caption coverage and Relation accuracy. AMBER-d uses all 14,216 questions and generation all 1,004 images. VISTA is newly generated; baseline/SCAS references reuse matched prior outputs, with full AMBER-d from complete historical evaluations.
SCAS also exceeds upstream-calibrated Revis on these scores:
Revis is newly rerun; these SCAS rows use paper references. The comparisons support joint benefits at the reported settings; the paper also reports task-specific costs.
3. Component contributions. New matched Q2 batch-one controls are:
Joint steering improves MMBench and adversarial F1 over either branch. The hybrid basis improves MME and adversarial F1 over both basis controls; PCA gives slightly higher MMBench. Removing text control reduces all three scores. Thus the selected controls directly support complementary branch, control and basis benefits. A complete reconstructed AMBER-d run also retains aggregate accuracy/F1 gains.
4. Stability and scale. Five same-OH-100-pool bootstrap fits rebuild both subspaces and generate new greedy answers with fixed layer/probe/coefficients:
Every listed fit improves, supporting benefits beyond a selected vector. These are within-pool checks with separate matched baselines, not cross-model vector transfer. Q8 results after fresh state extraction and subspace reconstruction are:
These Q8 evaluations use batch size=1.
These modest, training-free gains extend the construction to 8B. Full refit dispersion/intervals are in the appendix; Q8 MMBench's interval includes zero. An 8B steering-baseline comparison remains outside this study.
5. Practical cost. RTX 4090 measurements use 16 images, five repetitions, batch one, warm-up, balanced baseline/SCAS order and fixed output lengths:
Low measured overhead and unchanged measured peaks support practical plug-in use. Timing includes basis transfers and excludes preprocessing; image-cluster intervals and session details are documented. Mode B terminology, citations and the ICLR 2027 template are corrected.