acceptodds
Under review as a conference paper at ICLR 2027

SCAS: Symmetrical Concept-space Activation Steering for Joint Hallucination Mitigation and Capability Enhancement in VLMs

Abstract

A fundamental challenge in Vision-Language Models (VLMs) lies in the trade-off between mitigating hallucinations and preserving general reasoning capabilities, as existing safety interventions often degrade model utility or introduce severe biases. In this paper, we propose **SCAS** (**S**ymmetrical **C**oncept-space **A**ctivation **S**teering), a lightweight, train-free framework that achieves dual enhancement in both safety and utility. Utilizing only 100 calibration samples, SCAS extracts distinct semantic bases for factual groundedness () and hallucination (). By orthogonalizing these bases, SCAS ensures decoupled, bidirectional steering of intermediate hidden states (), simultaneously suppressing hallucination pathways while reinforcing core reasoning representations. Evaluations on Qwen3-VL-2B, Qwen3-VL-8B, and InternVL3.5-4B demonstrate that SCAS effectively addresses the safety-utility conflict: it significantly reduces visual hallucinations on RePoPe and AMBER, while consistently improving general capabilities on MME and MMBench. Furthermore, on generative captioning, SCAS avoids the decoding collapse suffered by prior methods, maintaining rich descriptive coverage while maintaining safety. This confirms SCAS as a promising solution for co-promoting VLM reliability and general intelligence.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language Models, Hallucination Mitigation, Activation Steering, Representation Learning, Multimodal Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_pt7fOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1(−1)
Read review
Paper summary

This paper proposes SCAS, a training-free method designed to mitigate hallucinations while enhancing the general reasoning capabilities of VLMs. SCAS leverages a small set of calibration samples to construct basis vectors representing factual and hallucination-related semantic subspaces, which are then used to steer the model's internal representations. Experiments on the Qwen3-VL series and InternVL models demonstrate the effectiveness of the proposed method.

Critical strengths

The paper not only focuses on mitigating multimodal hallucinations but also considers the potential degradation of general capabilities caused by intervention methods. Rather than solely pursuing improvements in hallucination-related metrics, evaluating hallucination mitigation and general capability preservation within a unified framework represents a valuable research direction.

Critical weaknesses
  1. The experimental results do not sufficiently support the main motivation and claims. SCAS aims to mitigate hallucinations while enhancing the general capabilities of VLMs, motivated by the difficulty of balancing these two objectives in existing methods. However, the experimental results do not convincingly demonstrate that SCAS achieves a better and more consistent trade-off than existing approaches such as Revis. For example, SCAS underperforms Revis on multiple hallucination-related metrics for Qwen3-VL-2B. Furthermore, the paper identifies the model dependence and limited cross-model stability of existing steering directions as key limitations, yet provides no direct experimental evidence demonstrating how SCAS addresses these issues. Overall, the results do not adequately substantiate the paper's central motivation and claimed contributions.

  2. Insufficient discussion of related work and methodological novelty. The paper primarily discusses earlier activation steering methods, with limited coverage of recent advances in activation steering and representation intervention. Apart from Revis, closely related methods are not adequately reviewed or compared, particularly those involving semantic direction extraction, subspace manipulation, and inference-time interventions. Consequently, the limitations of existing approaches highlighted in the Introduction are not sufficiently substantiated, and the methodological novelty of SCAS over prior work remains unclear.

  3. The experimental evaluation is not sufficiently comprehensive.

  • Limited baseline comparisons. The experiments mainly compare SCAS with Revis and VCD, while VCD follows a different technical paradigm. Comparisons with more closely related activation steering methods are needed to establish the advantages of SCAS. Lack of inference efficiency analysis. SCAS introduces additional subspace projections and matrix operations during inference, but the paper does not report the resulting latency or computational overhead. An efficiency comparison would help assess its practical applicability.
  • Insufficient evaluation on larger models. For Qwen3-VL-8B, SCAS is only compared against the original model, without comparisons to existing methods. Moreover, the performance improvements are relatively modest, limiting the evidence for its scalability and comparative advantages.
  • Incomplete ablation studies. Although SCAS incorporates several components, the individual contributions of key designs, such as six-state activation extraction, modality debiasing, and hybrid PCA-based subspace construction, are not adequately examined. Additional ablations are needed to justify their necessity and effectiveness.

Other issues: The paper incorrectly applies the ICLR 2026 template.

Questions for authors

no other questions

4 comments

Dear Reviewer pt7f,

Thank you. New experiments are included in the revised paper (Secs. 4.5–4.6; Apps. J–N). Q2/I4/Q8 denote Qwen3-VL-2B/InternVL3.5-4B/Qwen3-VL-8B; Adv denotes RePoPe adversarial; MMBench is local single-pass accuracy.

1. Contribution and positioning. SCAS reuses Revis's OH-100 factual/hallucinated pairs. Its contribution is their six-state real-image/text-only/blank-image construction, modality and Yes/No projections, mean-plus-PCA bases and asymmetric residualization, followed by complementary suppression/amplification in one forward stream without weight updates. Revised related work distinguishes ASD's prediction contrast, SHARP's feature intervention, VISTA's visual reinforcement/logit augmentation and Dynamic Multimodal Activation Steering. This shared construction is evaluated across architectures with separately extracted subspaces.

2. Completed close comparisons. We adapted upstream VISTA formulas to Qwen and calibrated VSV strength, SLA coefficient and layer window on COCO images disjoint from evaluation images. Fit/holdout images are distinct; parameters are frozen before the rerun.

Q2 settingMMBench↑MME↑Adv F1↑AMBER-d Overall F1↑CHAIR↓/Coverage↑
Baseline83.162033.2791.5889.146.2/75.7
VISTA-VSV83.512053.7291.7989.306.3/76.0
VISTA-SLA83.082025.5891.4888.966.1/76.2
VISTA full83.422035.4691.6289.186.0/76.2
SCAS, archived vector84.452100.9192.1289.864.9/69.0

SCAS leads all three variants on both utility benchmarks, adversarial F1 and aggregate AMBER-d F1, with lower CHAIR. VISTA retains higher caption coverage and Relation accuracy. AMBER-d uses all 14,216 questions and generation all 1,004 images. VISTA is newly generated; baseline/SCAS references reuse matched prior outputs, with full AMBER-d from complete historical evaluations.

SCAS also exceeds upstream-calibrated Revis on these scores:

Model/settingAdv F1↑AMBER-d Overall F1↑MMBench↑
Q2 Revis calibrated91.7089.2783.16
Q2 SCAS, paper92.0889.8684.62
I4 Revis calibrated84.9987.9286.77
I4 SCAS, paper86.1688.6987.03

Revis is newly rerun; these SCAS rows use paper references. The comparisons support joint benefits at the reported settings; the paper also reports task-specific costs.

3. Component contributions. New matched Q2 batch-one controls are:

SettingMMBench↑MME↑Adv F1↑
Baseline83.162033.2791.58
Full reconstruction84.542097.6492.08
Suppression only83.422091.6791.91
Amplification only83.932027.3091.89
Without text control82.992063.3091.90
Mean only83.762043.2391.76
PCA, rank nine84.622073.0292.00

Joint steering improves MMBench and adversarial F1 over either branch. The hybrid basis improves MME and adversarial F1 over both basis controls; PCA gives slightly higher MMBench. Removing text control reduces all three scores. Thus the selected controls directly support complementary branch, control and basis benefits. A complete reconstructed AMBER-d run also retains aggregate accuracy/F1 gains.

4. Stability and scale. Five same-OH-100-pool bootstrap fits rebuild both subspaces and generate new greedy answers with fixed layer/probe/coefficients:

Model/metricMatched baselineFive-fit mean±SDAbove baseline
Q2 MMBench83.1684.04±0.115/5
Q2 MME2032.792078.68±16.775/5
I4 Adv F184.0885.65±0.715/5

Every listed fit improves, supporting benefits beyond a selected vector. These are within-pool checks with separate matched baselines, not cross-model vector transfer. Q8 results after fresh state extraction and subspace reconstruction are:

Q8 settingMMBench↑MME↑Adv F1↑
Baseline90.382402.6891.45
Rebuilt SCAS90.642408.4191.76

These Q8 evaluations use batch size=1.

These modest, training-free gains extend the construction to 8B. Full refit dispersion/intervals are in the appendix; Q8 MMBench's interval includes zero. An 8B steering-baseline comparison remains outside this study.

5. Practical cost. RTX 4090 measurements use 16 images, five repetitions, batch one, warm-up, balanced baseline/SCAS order and fixed output lengths:

Model4-token overhead64-token overheadPeak allocated: base/SCAS MiB
Q21.40%1.82%4225.78/4225.78
I40.57%1.07%9804.66/9804.66
Q80.73%1.65%16962.70/16962.70

Low measured overhead and unchanged measured peaks support practical plug-in use. Timing includes basis transfers and excludes preprocessing; image-cluster intervals and session details are documented. Mode B terminology, citations and the ICLR 2027 template are corrected.

I thank the authors for the rebuttal and new experiments. However, my main concerns remain unresolved.

The rebuttal does not address the central issue. The original abstract motivated the work by the trade-off between mitigating hallucinations and preserving general capabilities, yet the method underperforms baselines on several hallucination benchmarks. The authors did not respond to this point directly; instead, they modified and removed the statement without any explanation. Removing a claim is not the same as resolving the issue.

Moreover, the revised abstract reads more like a summary of experimental results than an abstract. I recommend rewriting it to clearly state the problem, motivation, method, and main contributions, and to accurately reflect the evidence.

Unexplained changes to the baseline results in Table 1. Compared with the original submission, the baseline (ReVis) results in Table 1 have been lowered on multiple metrics, with no explanation provided. This casts doubt on the reliability of the experimental results.

Computational efficiency.
Please provide a direct comparison between the proposed method and the baselines under identical experimental settings, reporting at least inference time and GPU memory usage.

Based on the reasons above, I have lowered my score for this paper. If the authors can adequately address these issues, I would be willing to reconsider my assessment.

Dear Reviewer pt7f,

Part 1/2: The stronger Revis reference and the hallucination–utility trade-off. Thank you. We clarify the stronger-comparator rationale and give direct evidence across both backbones. Q2/I4 denote Qwen3-VL-2B/InternVL3.5-4B. Random/Popular/Adv are RePoPe F1 settings; AMBER-d F1 pools all 14,216 questions; MMBench is local single-pass accuracy. Part 2 gives the direct efficiency comparison.

1. Why the original paper disabled the Revis gate. In our initial RePoPe/AMBER-d checks, the thresholded adaptation produced baseline-equivalent answers. We therefore disabled the gate to obtain a more actively intervening, stronger Revis comparator. This rationale was explicitly stated in the original submission. The configuration is gate disabled (tau=None in the evaluator, “Revis w/o τ” in Table 1); a numeric τ=0 would still apply a gate.

Following the reviews, we additionally used the official Revis calibration code, with backbone-specific adaptation, and completed benchmark reruns. Q2 results are:

SettingRandom F1↑Popular F1↑Adv F1↑AMBER-d F1↑MMBench↑MME↑
Revis original, ungated95.2993.2392.2689.9083.932086.79
Revis calibrated, gated95.3092.8291.7089.2783.162034.29
SCAS95.1793.1292.0889.8684.622101.92

The original ungated reference exceeds the newly calibrated configuration on Popular/Adv F1, AMBER-d F1, MMBench and MME. We retain its stronger scores. The original values have not been lowered or replaced: the revised Table 1 reports the original and newly calibrated configurations as separate rows. The rerun also changes hook/index adaptation to follow upstream behavior, so this comparison does not isolate a τ-only causal effect. Appendix L gives the protocol.

2. Small discriminative gaps versus better utility and usable descriptions. On Q2, original Revis's advantages over SCAS are 0.12/0.11/0.18 percentage points on Random/Popular/Adv F1, and 0.04 points on full AMBER-d F1. SCAS instead gains 0.69 MMBench points and 15.13 MME points over that stronger reference. This is direct evidence for the joint motivation: competitive hallucination metrics together with higher general-task utility. SCAS's benefit does not rely on substituting a weaker Revis row.

Free-form generation exposes an additional trade-off. On all 1,004 Q2 AMBER-g images, with a 512-token limit, the original ungated Revis exhibits severe repetition and reduced descriptive coverage. R4 is mean literal four-gram repetition:

Q2 settingCHAIR↓Coverage↑Repetition R4↓
Revis original, ungated4.444.162.95%
Revis calibrated, gated6.075.52.04%
SCAS4.769.213.22%

Thus its slightly lower CHAIR (4.4 vs 4.7) accompanies coverage 44.1 vs 69.2 and repetition 62.95% vs 13.22%. CHAIR alone cannot characterize useful hallucination mitigation. The official calibrated version restores coverage and reduces repetition; we report it separately and do not attribute the original degeneration to every Revis configuration. SCAS offers the reported grounding/utility balance against both references. The small-gap statement above concerns the listed aggregate F1 metrics; subtask and model-specific exceptions remain visible in the paper.

3. InternVL supplies a broader advantage over Revis. On I4, SCAS exceeds both Revis configurations in every RePoPe F1 setting, full AMBER-d F1 and MMBench:

SettingRandom F1↑Popular F1↑Adv F1↑AMBER-d F1↑MMBench↑MME↑
Revis original, ungated92.3087.3683.8788.2985.652237.00
Revis calibrated, gated92.0588.3484.9987.9286.772195.38
SCAS92.8189.1986.1688.6987.032165.25

Against original Revis, SCAS improves adversarial F1 by 2.29 points, MMBench by 1.37 points and full AMBER-d F1 by 0.40 points. Revis retains higher MME. Together, the Q2 and I4 comparisons support evaluating the joint balance across hallucination and general capability, with clear SCAS advantages across multiple I4 benchmarks. They support our motivation beyond a single favorable Q2 baseline configuration.

We retained this motivation and rewrote the abstract around the problem, six-state controlled construction, hybrid subspaces and complementary suppression/amplification before summarizing evidence. The construction is fitted separately for each backbone. Fresh reconstruction, component ablations and five same-pool calibration refits further support its contribution; they are not presented as cross-model vector transfer.

Dear Reviewer pt7f,

Part 2/2: Completed direct efficiency comparison. We have now completed a same-session comparison of unmodified decoding, SCAS, both Revis configurations, all three calibrated VISTA variants and VCD on Qwen3-VL-2B. All conditions use one model load on the same RTX 4090, the same 16 images/prompts, batch size 1, BF16/SDPA, greedy decoding and fixed 4/64-token outputs. We use warm-up, rotated method order and five repetitions per image: 1,440 timed calls in total.

Method4 tokens, ms64 tokens, ms64-token ratioPeak allocated, MiB
Baseline226.082193.271.000×4242.81
SCAS233.522273.141.036×4242.81
Revis (original, ungated)225.082216.051.010×4242.81
Revis (official calibration)230.322218.551.012×4242.81
VISTA-VSV527.873337.521.522×4424.52
VISTA-SLA240.282402.291.095×4242.83
VISTA full540.363389.811.546×4424.52
VCD (paper adapter)535.725508.272.511×4445.99
VCD (cached-noise control)437.654104.091.871×4327.60

Timing is CUDA-synchronized model-side wall time, including intervention setup/transfers, generation and VISTA's per-image visual-vector extraction. It excludes model loading, offline calibration, image/noise preprocessing and disk writes. VISTA uses the final calibrated VSV strength 0.01 and SLA coefficient 0.003 at decoder indices 18–23. Revis uses the original ungated setting and its separately identified official calibration; no threshold sweep is involved.

For 64-token outputs, SCAS's mean overhead relative to unmodified decoding is 3.64% (paired image-cluster bootstrap 95% CI [-1.07%, 9.06%], 20,000 replicates). SCAS retains the baseline's measured peak allocated memory and is faster than full VISTA and both VCD implementations in this comparison. All raw calls, token outputs and allocated/reserved memory peaks are retained.

The VCD paper adapter recomputes the noisy prefix without a KV cache. To avoid attributing that implementation cost to the contrastive formula itself, we additionally report an independent noisy-branch cache control. That control was checked step-by-step against Hugging Face's native cached generation; the logits match exactly in the checked forced-prefix case. Cached versus full-prefix BF16 computation can change long greedy outputs, so this is a labeled efficiency control, not a new accuracy result or a replacement for the paper's evaluated VCD configuration.

The revised manuscript now includes the direct comparison in Table 6, with the full component table and protocol in Appendix M.

Reviewer_oTDiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SCAS is a training-free activation-steering method that uses 100 calibration examples to construct factual and hallucination-related subspaces after modality and yes/no debiasing. At inference, it suppresses negative-subspace components and amplifies positive-subspace components. Experiments on three VLMs evaluate hallucination, caption coverage, and general capabilities using RePoPe, AMBER, MME, and MMBench. Results show useful gains, particularly on Qwen3-VL-2B, alongside task-dependent regressions.

Critical strengths
  1. Table 1 shows InternVL RePoPe adversarial false positives decreasing from 414 to 343, with F1 improving from 84.11 to 86.16. On Qwen3-VL-2B, MMBench increases from 83.16 to 84.62 and MME from 2033.27 to 2101.92, alongside reduced hallucination. These results demonstrate simultaneous improvements in at least one configuration without updating model weights.

  2. Table 2 reports SCAS CHAIR/coverage of 4.7/69.2, versus 4.4/44.1 for the evaluated Revis variant. Reporting coverage prevents low hallucination scores caused by uninformative generation from being mistaken for better overall performance.

Critical weaknesses
  1. In Table 3, Qwen AMBER Relation accuracy/F1 falls from 82.69/80.77 to 76.20/76.73. Suppression alone achieves 77.95/77.79, meaning adding the enhancement component further worsens relation performance. Table 1 also shows InternVL MME decreasing from 2192.15 to 2165.25, while Table 2 shows Qwen coverage decreasing from 75.7 to 69.2. These are substantive trade-offs, particularly given the paper’s central claim of preserving or enhancing capabilities while mitigating hallucination.

  2. Section 4.1 sets τ=None, then Section 4.3 attributes repetition collapse to Revis. However, the original method uses thresholding to avoid unnecessary interventions and itself reports collapse without this threshold (§5.5.3, Table 8). Original Revis paper A calibrated, threshold-enabled comparison is needed before the results establish superior robustness over the complete method.

Questions for authors
  1. Does Mode B’s implementation follow Eq. (6)? Its claimed mutual orthogonality does not hold generally. Taking initial bases e1e_1 and (e1+e2)/2(e_1+e_2)/\sqrt{2} gives residualized bases (e1−e2)/2(e_1-e_2)/\sqrt{2} and e2e_2, whose inner product is −1/2-1/\sqrt{2}. Please clarify the implementation and report post-construction overlap. The default Mode A is valid; this issue affects Mode B and Table 4’s interpretation.

  2. Does the captioning advantage persist against threshold-enabled Revis? Please compare CHAIR, coverage, and repetition frequency under model-specific calibration, comparable tuning budgets, and identical generation-length limits.

  3. What directly supports the debiasing interpretation? Section 4.4 reports non-debiased Relation accuracy of 75.18, while full debiased SCAS achieves 76.20, versus baseline 82.69. Please report all three configurations’ “No” rates and relation confusion matrices to establish whether debiasing prevents shortcut negation.

1 comment

Dear Reviewer oTDi,

Thank you. The revised paper adds new experiments (Secs. 4.5–4.6; Apps. J–N) connecting the corrected geometry to direct debiasing evidence and joint utility/grounding gains, including complete calibrated Revis. Q2/I4 denote Qwen3-VL-2B/InternVL3.5-4B; Adv denotes RePoPe adversarial; MMBench is local single-pass accuracy.

1. Main-method geometry and Mode B. Default Mode A preserves the negative basis and residualizes the positive basis against it. Optional Mode B follows Eq. (6), residualizing each basis against the original counterpart. We corrected its name to “two-sided residualization.” For a=(1,0)a=(1,0), b=(cos⁡θ,sin⁡θ)b=(\cos\theta,\sin\theta), with 0<θ<π/20<\theta<\pi/2, the normalized residuals have inner product −cos⁡θ-\cos\theta; they are not generally orthogonal.

Measured normalized projector overlaps on current archived artifacts are:

ModelInitialDefault Mode AMode B
Q20.79961.73×10⁻¹²0.7996
I40.55497.14×10⁻¹³0.5549

Thus the main configuration implements the intended algebraic separation. The method text and ablation interpretation now consistently distinguish Mode A from Mode B.

2. Grounding and utility against complete Revis. We added a new rerun using upstream risk calibration/intervention and 300 separate COCO questions:

ModelDecoder index (zero-based)τ
Q224−0.00093422
I4330.03339844

The following table uses paper baseline/SCAS responses and newly calibrated Revis. Generation covers all 1,004 images with a 512-token limit; R4 is mean literal four-gram repetition.

Model/settingAdv F1↑CHAIR↓Coverage↑R4↓
Q2 baseline91.586.275.71.22%
Q2 Revis w/o τ92.264.444.162.95%
Q2 Revis calibrated91.706.075.52.04%
Q2 SCAS92.084.769.213.22%
I4 baseline84.115.668.00.26%
I4 Revis calibrated84.996.067.90.26%
I4 SCAS86.165.766.50.23%

SCAS has higher adversarial F1 and lower CHAIR than calibrated Revis on both models. Captioning advantages are metric-specific: Revis retains higher coverage and lower Q2 repetition. The earlier collapse is scoped to the unconditional local variant. The rerun also repairs hook/index adaptation; it does not isolate τ or equate historical tuning budgets.

The joint benefit is supported by general utility and full AMBER-d, rather than CHAIR alone. Paper baseline→SCAS results are:

ModelMMBench↑AMBER-d Overall Acc/F1↑
Q283.16→84.6286.27/89.14→86.68/89.86
I486.77→87.0384.84/87.84→85.64/88.69

Full AMBER-d pools all 14,216 questions; Existence and Attribute gains outweigh Relation losses. The revised claims specify these benefits and retain task-level costs, including I4 MME.

3. Direct evidence for response-axis control. The revised table includes all three requested historical Q2 Relation confusion matrices. Y/N means predicted Yes/No:

SettingTrue Y: Y/NTrue N: Y/NAcc↑No rate
Baseline771/20484/60582.6948.62%
Without Yes/No axis588/38726/66375.1863.10%
Full SCAS615/36036/65376.2060.88%

The projection recovers correct positive answers, reduces the No shift and improves accuracy over its removal. This supports partial negation-bias mitigation; full SCAS still has a Relation cost. The paper reports the aggregate gains together with this subtask evidence.

4. Reconstruction and alternative probe. SCAS reuses Revis's OH-100 pairs through our six-state/subspace construction. Fresh extraction and reconstruction retain the benefits. New matched batch-one checks are:

Q2 settingMMBench↑MME↑Adv F1↑Relation Acc/No rate
Baseline83.162033.2791.5882.69/48.62%
Rebuilt SCAS84.542097.6492.0876.08/60.88%
Alternative 20-question probe84.542094.5492.0477.10/59.74%
Without Yes/No axis84.542068.1392.1375.36/63.16%

The alternative probe preserves utility while further reducing the No shift. Its questions replace the original text-only probe, not the OH-100 pairs. A separate full 14,216-question reconstruction confirms aggregate gains:

Full Q2 AMBER-dOverall Acc/F1↑Net correct vs baseline
Historical baseline86.269/89.1420
Rebuilt SCAS, new run86.628/89.818+51

Five same-pool fits retain both Q2 utility improvements in every fit with fixed parameters. These checks strengthen SCAS's evidence beyond the original saved vector. Citations and the ICLR 2027 format are corrected.

Reviewer_gwjXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SCAS (Symmetrical Concept-space Activation Steering), a training-free, inference-time framework that aims to reduce object hallucination in VLMs while preserving or improving general multimodal capability. From 100 calibration samples (OH-100), the authors extract six teacher-forced last-token hidden states per sample at a target decoder layer (real/hallucinated caption x image/text-only/blank-image controls), form raw difference directions, and clean them by projecting out language-bias and visual-bias bases and a Yes/No affirmation-probe axis (M=20 balanced commonsense questions). A hybrid basis (mean direction + top-k=8 PCA components, Gram-Schmidt) yields positive (factual) and negative (hallucination) subspaces, which are mutually orthogonalized (default: asymmetric Mode A). At inference, a phase-aware decode hook steers hidden states as x' = x - alphaP_neg(x-mu) + betaP_pos(x-mu) with optional perturbation-norm clipping. Evaluations on Qwen3-VL-2B, InternVL3.5-4B (main) and Qwen3-VL-8B (supplementary) cover RePoPe, AMBER-d/g, MME, and MMBench against baseline decoding, VCD, and Revis. Reported findings: on InternVL3.5-4B, RePoPe-Adversarial F1 improves 84.11 -> 86.16 with false positives 414 -> 343 and MMBench 86.77 -> 87.03; on Qwen3-VL-2B, SCAS trails Revis slightly on discriminative safety (RePoPe-Adv F1 92.08 vs 92.26; AMBER-d 86.68/89.86 vs 86.75/89.90) but leads on utility (MMBench 84.62 vs 83.93; MME 2101.92 vs 2086.79); on AMBER-g captioning, SCAS attains CHAIR 4.7 with Coverage 69.2 whereas Revis reaches CHAIR 4.4 but collapses Coverage to 44.1 via repetition loops; ablations attribute gains to symmetric alpha/beta steering, Yes/No debiasing (removal induces 63.1% "No" answering and Relation accuracy collapse), and asymmetric orthogonalization; Qwen3-VL-8B results show small uniform gains (e.g., MME +4.98, RePoPe-Adv F1 +0.23).

Critical strengths
  1. The evaluation philosophy is the paper's most valuable contribution: it explicitly guards against the two classic ways hallucination metrics get gamed. Table 2 + Fig. C.1 show that Revis's low CHAIR (4.4) coincides with a Coverage collapse (75.7 -> 44.1) from repetition loops, so SCAS's CHAIR 4.7 at Coverage 69.2 is argued to be the more honest operating point; and the Yes/No debiasing ablation (Sec. 4.4) demonstrates that removing the probe-axis cleanup flips the model into shortcut negation (63.1% "No" predictions; AMBER Relation accuracy 82.69 -> 75.18) even while aggregate safety scores slightly improve. This "metrics can be gamed, and we designed controls for it" stance is consequential: it makes the safety-utility joint evaluation credible in a way that CHAIR-only or accuracy-only reporting is not, and it is a reusable lesson for the subfield.
  2. The method is training-free, consumer-GPU deployable (4x RTX 4090, Appendix A), and plug-in across architectures, and on InternVL3.5-4B it delivers a clear, coherent safety gain where the steering baseline fails: RePoPe-Adversarial F1 84.11 -> 86.16 with FP 414 -> 343 and MMBench +0.26pp, while Revis degrades both safety (F1 83.87, FP 429) and MMBench (-1.12pp) on the same model. The six-state debiasing pipeline (modality-bias projection, affirmation-axis removal, hybrid mean+PCA basis, subspace orthogonalization) is a principled and well-motivated assembly, and the cross-model/cross-scale attempt (2B/4B/8B) is more than many steering papers provide.
Critical weaknesses
  1. The headline "dual enhancement / consistently improving general capabilities" claim is contradicted by the paper's own tables. The abstract states SCAS "consistently improv[es] general capabilities on MME and MMBench," yet on InternVL3.5-4B MME falls from 2192.15 (baseline) to 2165.25 under SCAS - a 26.9-point regression - while Revis raises MME to 2237.00 on the same model; Sec. 4.2 dismisses this as "only negligible variance." Likewise, Table 3 shows full SCAS (alpha=beta=1.0) degrades AMBER Relation F1 from 80.77 to 76.73 (-4.04pp vs baseline) and scores below its own beta-only variant on AMBER-d Overall Accuracy (86.68 vs 86.88), which undercuts "simultaneously suppressing hallucination pathways while reinforcing core reasoning representations." On Qwen3-VL-2B discriminative safety, SCAS is at or below Revis everywhere (RePoPe-Adv F1 92.08 vs 92.26; AMBER-d 86.68/89.86 vs 86.75/89.90), and the Qwen3-VL-8B gains (+0.2-0.3pp) are within plausible noise. As submitted, the central claim that SCAS "resolves" the safety-utility conflict is not supported; it must be scoped to specific models/metrics or substantiated with the missing statistics.
  2. The decisive comparative advantages may be baseline-configuration artifacts, and nothing is statistically tested. Revis is run with tau=None (thresholding disabled), a non-default setting the authors frame as "stronger, more competitive" - but disabling the threshold is a plausible direct cause of the repetition/coverage collapse (75.7 -> 44.1) on which SCAS's entire generative-side narrative rests; no sweep over tau (including its recommended default) is reported. VCD is silently absent from all InternVL3.5-4B tables. All results are single deterministic greedy runs with no seeds, no calibration-resampling dispersion, and no CIs, even though the paper's own Fig. D.2b shows cross-run subspace variability is non-trivial and many decisive margins are <=0.3pp. Finally, the selection protocol for the intervention layer (within the "stable ranges" 15-20 / 22-26 / ~20), alpha/beta, k=8, ortho mode, and the norm-clip constant C (whose value is never given, and whose on/off status in the main results is unclear) is undocumented, leaving test-set tuning unexcludable.
Questions for authors
  1. (Material) How do you reconcile the abstract's "consistently improving general capabilities on MME" with the InternVL3.5-4B MME drop (2192.15 -> 2165.25) while Revis rises to 2237.00? Will the claims be scoped per model/metric?
  2. (Material) Please report Revis with its default/recommended tau and a small tau sweep on AMBER-g: does the Coverage collapse (44.1) persist? Why is VCD absent from the InternVL3.5-4B tables?
  3. (Material) Document the selection protocol for layer, alpha, beta, k, ortho mode, and clipping: which choices were made using only calibration data versus evaluation benchmarks? What is the clip constant C, and was clipping active in Tables 1-2?
  4. (Material) Please provide dispersion: re-extract subspaces over several resamples of the 100 calibration samples and report CIs (or at least min-max) for the key margins - InternVL RePoPe-Adv F1, Qwen-2B MMBench/MME, and the 8B deltas. Are the 8B gains (+0.2-0.3pp) significant under this variability?
  5. (Clarification) Reconcile the subspace-overlap numbers: Sec. 3.4 and Fig. 1 state ~0.5 ("intrinsic 50% overlap"), while Fig. D.2's caption reports overlap > 0.7 with peak ~0.95. Which holds at the chosen intervention layers?
  6. (Clarification) Table 4: please move the body-only evidence into the table (the 63.1% "No" rate and Relation accuracy 82.69 -> 75.18 for the non-debiased variant), and report sensitivity to the M=20 probe set (e.g., a disjoint probe set).
  7. (Clarification) Will code and the exact OH-100 calibration subset be released? Also please fix: Sec. 4.1 cites RePoPe as "Li et al. (2023b)" (that is POPE; RePoPe is Neuhaus & Hein 2025); both the Qwen3-VL and InternVL3.5 technical reports are cited as arXiv:2601.00000 (the same identifier for two different references); and the page header says "ICLR 2026" while the reviewer footer references ICLR 2027 guidelines.
2 comments

Dear Reviewer gwjX,

Part 1/2: Questions 1–3. Thank you. New experiments are included in the revised paper (Secs. 4.5–4.6; Apps. J–N). Q2/I4/Q8 denote Qwen3-VL-2B/InternVL3.5-4B/Qwen3-VL-8B; Adv denotes RePoPe adversarial; MMBench is local single-pass accuracy.

Question 1. Scope of the capability claim. We replaced the universal capability-preservation claim and removed the description of the I4 MME decrease as negligible variance. The revised claims follow the measured model/metric results:

Model/settingAdv F1↑AMBER-d Overall Acc/F1↑MMBench↑MME↑
Q2 baseline91.5886.27/89.1483.162033.27
Q2 SCAS92.0886.68/89.8684.622101.92
I4 baseline84.1184.84/87.8486.772192.15
I4 SCAS86.1685.64/88.6987.032165.25

Q2 improves all four summaries. I4 improves adversarial F1, full AMBER-d and MMBench, while MME decreases by 26.90 points; Revis's higher I4 MME is retained in Table 1. Our claim is joint hallucination/utility gains on Q2 and aggregate hallucination gains on I4. Full AMBER-d pools all 14,216 questions: Existence and Attribute gains outweigh the Relation loss, yielding 59 net additional correct Q2 answers. Table 3 retains every subtask and shows that amplification-only has higher Overall accuracy, while the combined update has higher Overall F1. We therefore report complementary branch benefits with task-dependent effects.

Question 2. Official Revis calibration and the I4 VCD comparison. We used the official Revis repository's calibration code, with backbone-specific adaptation, to obtain model-specific thresholds and completed benchmark reruns for both models. Calibration follows the upstream first-prefill risk collection, backward layer selection and 90th factual-risk percentile on 300 separate COCO questions:

ModelDecoder index (zero-based)τ
Q224−0.00093422
I4330.03339844

The full Q2 captioning comparison covers all 1,004 AMBER-g images with a 512-token limit. R4 is mean literal four-gram repetition. Baseline/SCAS are paper references; calibrated Revis is newly generated:

Q2 settingAdv F1↑CHAIR↓Coverage↑R4↓
Baseline91.586.275.71.22%
Revis w/o τ92.264.444.162.95%
Revis calibrated91.706.075.52.04%
SCAS92.084.769.213.22%

The calibrated method avoids the earlier unconditional coverage collapse. SCAS retains higher adversarial F1 and lower CHAIR than calibrated Revis, while Revis retains higher coverage and lower repetition. The rerun also corrects hook/index adaptation, so the change cannot be attributed solely to τ. These comparisons use the official model-specific calibration rule, with τ fixed before benchmark evaluation (Appendix L).

VCD requires predictions from clean-image and noise-corrupted-image branches at each decoding step, increasing inference work over single-stream steering. On Q2 it has no advantage over SCAS on the four aggregate metrics below:

Q2 methodMMBench↑MME↑Adv F1↑AMBER-d Overall F1↑
VCD83.332058.6891.6888.67
SCAS84.622101.9292.0889.86

Given this added computation and the Q2 comparison, we prioritized complete calibrated Revis and VISTA evaluations instead of extending VCD to I4. We make no I4 VCD performance claim and no measured wall-clock multiplier claim.

Question 3. Selection protocol and clipping. The authors' configuration choices are:

ChoiceSetting and rationale
Decoder indices15/24/20 (Q2/I4/Q8); heuristic intermediate layers
α, β1, 1; empirical unit defaults
PCA componentsk=8; empirical choice, plus the mean (rank at most nine)
OrthogonalizationMode A; preserve the negative basis and orthogonalize the positive basis
ClippingOff: norm_clip=0; no active bound C

OH-100 supplies the activations, centering vectors and fitted bases; the settings above were empirical/geometric choices, not a calibration-score optimization procedure. Mode A is chosen to enforce mutual subspace orthogonality. Archived evaluation-benchmark sweeps are disclosed as exploratory diagnostics, separately from these selection rationales. Fresh reconstruction and all five refits hold layer, coefficients, rank, mode and probe fixed, without selecting the best fit. Clipping was disabled in Tables 1–2: norm_clip=0 bypasses clipping, so no active C was used. The hook changes the last prefill position and each subsequent decoded position. Appendix E.2 documents the selection history.

Dear Reviewer gwjX,

Part 2/2: Questions 4–7. Model abbreviations and metric definitions follow Part 1.

Question 4. Calibration dispersion and small 8B gains. We freshly extract the six states and reconstruct both subspaces. Five additional fits draw 100 OH-100 pairs with replacement, rebuild centers/bases from the cached states and generate fresh greedy answers. They are calibration refits, not repeated sampling with one saved vector. Each row uses its own matched baseline:

Model/metricBaselineFive-fit mean±SDMin–maxAbove baseline
Q2 MMBench83.1684.04±0.1183.93–84.195/5
Q2 MME2032.792078.68±16.772049.39–2091.855/5
I4 Adv F184.0885.65±0.7184.75–86.285/5
Q8 MMBench90.3890.62±0.1790.38–90.814/5
Q8 MME2410.042405.88±8.052394.04–2414.542/5
Q8 Adv F191.5691.70±0.1691.52–91.874/5

Both Q2 utility gains and the I4 adversarial-F1 gain hold in all five fits. Q8 MMBench and adversarial F1 improve in four fits, while MME varies with a lower mean. Thus we do not claim uniformly significant 8B gains. These SDs describe same-pool variability with fixed layer/probe/coefficients, not confidence intervals over independently collected calibration sets.

A separate reconstruction check retains Q2 gains, including complete AMBER-d:

SettingMMBench↑MME↑Adv F1↑Full AMBER-d Acc/F1↑
Baseline83.162033.2791.5886.269/89.142
Rebuilt SCAS84.542097.6492.0886.628/89.818

The first three columns use matched batch-one inference. Full AMBER-d compares all 14,216 newly generated answers with the complete historical baseline, giving 51 net additional correct answers. No Relation-only subsets are stitched into Overall.

Separately, 20,000 paired image-cluster bootstrap replicates yield:

Accuracy changeΔ pointsConditional 95% CI
Q2 MMBench+1.37[0.60, 2.23]
I4 Adv accuracy+2.35[1.77, 2.94]
Q8 MMBench+0.26[−0.09, 0.67]

These intervals concern accuracy conditional on fixed vectors/answers; the F1 dispersion is in the refit table. Q8 MMBench's interval includes zero. Together, the analyses support consistent Q2 utility and I4 adversarial gains while characterizing the smaller 8B effects (Appendices J–K).

Question 5. Which subspace-overlap numbers apply? We removed the universal “intrinsic 50%” claim and distinguish within-fit channel overlap from cross-fit consistency. Full-calibration normalized projector overlaps at the selected layers, recomputed from current archived artifacts, are:

ModelDecoder indexInitial overlapAfter Mode A
Q2150.79961.73e-12
I4240.55497.14e-13
Q8200.80802.61e-12

For the two 50-sample Q2 subsets, within-fit positive/negative overlaps are 0.778/0.755, whereas cross-fit positive/positive and negative/negative overlaps are 0.486/0.515. These are different quantities, not a universal overlap constant. Default Mode A achieves numerical orthogonality across all three models. Mode B is now correctly called two-sided residualization; it is not generally mutually orthogonal. Appendix D and Table D.1 give the definition and measurements.

Question 6. Table 4 and probe sensitivity. Revised Table 4 directly includes baseline/full/no-axis Relation accuracy and No rates, including the requested 82.69→75.18 accuracy change and 63.10% No rate. Appendix F supplies all three historical confusion matrices. A separate matched batch-one experiment replaces the original 20 text-only questions with 20 disjoint questions:

Q2 settingMMBench↑Relation Acc↑No rate
Rebuilt SCAS84.5476.0860.88%
Alternative 20-question probe84.5477.1059.74%
Without Yes/No axis84.5475.3663.16%

The alternative probe retains MMBench and further reduces the No shift. Both probes improve Relation accuracy over omitting the axis, supporting response-axis control while the full method retains its aggregate AMBER-d benefit. This probe-set change does not alter the OH-100 calibration pairs (Appendix K).

Question 7. Release and corrections. Yes. Upon acceptance, we will release the SCAS code, reproduction scripts, exact OH-100 sample IDs/hashes and retrieval instructions, with attribution to Revis. The OH-100 data source is shared; SCAS's contribution is its six-state and complementary-subspace construction. RePoPe is corrected to Neuhaus & Hein (2025); the Qwen3-VL and InternVL3.5 reports now have distinct identifiers, 2511.21631 and 2508.18265. The paper uses the ICLR 2027 template/header. These resources are retained locally now; public release is planned upon acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.