acceptodds
Under review as a conference paper at ICLR 2027

HalluRefuse: Hallucination Mitigation with Knowledge Preserving Activation Steering

Abstract

Large language models (LLMs) perform well in question answering, reasoning, and text generation. However, LLMs may produce plausible but factually incorrect answers when they lack sufficient knowledge or are uncertain. For reliable deployment, LLMs should answer known questions correctly while abstaining from unknown questions. Prior work has shown that activation steering can steer LLMs toward refusal without additional training, but existing methods typically apply interventions to all questions without distinguishing between known and unknown ones. Through empirical analysis, we show that indiscriminate steering induces a trade-off between reducing hallucinations on unknown questions and preserving performance on known questions. We further find that known question representations exhibit a compact spectral structure, revealing a low-dimensional subspace that can be protected. Based on these observations, we propose HalluRefuse, an activation intervention method that mitigates hallucinations while preserving known representations. HalluRefuse estimates the subspace of known representations and applies the intervention to the orthogonal subspace. The derived closed-form intervention guarantees that the protected known representations are unaffected. Extensive experiments across multiple LLMs and question answering benchmarks show that HalluRefuse mitigates hallucinations on unknown questions while achieving the highest average known accuracy across all LLMs.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Hallucination Mitigation, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_McahOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

HalluRefuse addresses the trade-off in conventional activation steering, where reducing hallucinations on unknown questions often degrades accuracy on known questions. The method identifies a low-dimensional subspace of known representations using SVD and protects it by applying interventions only to the orthogonal subspace. A refusal direction is constructed from the activation differences between refused and hallucinated responses, and an intervention matrix is computed in closed form without additional model training. Experiments across three LLMs and three QA benchmarks show that HalluRefuse reduces hallucination rates by 5.88–14.48 percentage points while preserving or improving known-question accuracy.

Critical strengths

1. Empirical analysis of the limitations of existing activation steering methods

  • The paper demonstrates a clear trade-off between hallucination mitigation and known-question accuracy as steering strength increases.

  • The analysis of representation shifts and singular value decomposition (SVD) provides empirical motivation for protecting the known representation subspace.

2. Training-free and efficient intervention

  • The proposed method does not require updating model parameters and derives the intervention matrix using a projection operator and a closed-form solution.

  • The precomputed intervention matrix can be applied during inference without additional online training or explicit known/unknown question classification.

3. Consistent improvements across models and datasets

  • The method consistently reduces hallucination rates while preserving or improving known-question accuracy across three LLMs and three QA benchmarks.

  • The cross-dataset evaluation provides additional evidence of the method's generalization capability beyond the datasets used for constructing the intervention.

Critical weaknesses

1. Dependence on calibration samples and limited generalization

  • The projection matrix PKP_K and intervention matrix Γα\Gamma_\alpha are constructed from sampled known and unknown questions, making them potentially sensitive to the selection of calibration samples.

  • Although cross-dataset experiments are conducted, the effectiveness of the method under substantial domain shifts (e.g., from general QA to medical QA) remains unexplored.

  • The paper lacks a sensitivity analysis examining how different calibration samples affect the stability of the protected subspace and overall performance.

2. Remaining trade-off and lack of adaptive rank selection

  • The intervention subspace rank is determined in advance and remains fixed during inference.

  • Figure 3(b) shows that increasing the rank reduces hallucination rates but also leads to a substantial degradation in known-question accuracy.

  • This suggests that HalluRefuse mitigates rather than fundamentally resolves the trade-off highlighted in the paper. Furthermore, the method lacks a mechanism for dynamically selecting an appropriate rank based on individual inputs or domains.

3. Limited justification for the fixed intervention layer

  • The method applies intervention to the final input token at layer 19 across all evaluated models.

  • However, the paper provides insufficient justification for selecting this particular layer, and the optimal intervention layer may vary across models, input questions, or domains.

  • A layer-wise ablation study would strengthen the paper by examining the sensitivity of the proposed method to intervention location and its implications for generalization.

4. Dependence on the model's existing refusal capability

  • The refusal direction is constructed from the difference between the mean representations of unknown questions that the model already refuses and those for which it produces hallucinated responses.

  • Consequently, if the base model rarely exhibits appropriate refusal behavior, constructing a reliable refusal direction may be challenging.

  • The paper does not sufficiently investigate how the effectiveness of HalluRefuse varies with the base model's initial refusal capability.


Overall, the experimental results demonstrate that HalluRefuse is effective under the evaluated models, datasets, and hyperparameter settings. However, given its dependence on calibration samples, subspace rank, and intervention layer selection, it remains unclear how well the method would generalize to substantially different domains or deployment settings.

Questions for authors
  1. Validity of the theoretical guarantee: The guarantee ΓαHK=0\Gamma_\alpha H_K=0 applies only to the predefined known representations. To what extent can this guarantee be extended to unseen known questions, particularly under domain shifts?

  2. How can the authors ensure that the constructed refusal direction captures refusal behavior rather than confounding factors such as question topics or difficulty?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Mcah

We thank the reviewer for this valuable comment.

W1: Limited Domain Datasets.

We thank the reviewer for this valuable comment. We address this concern through additional empirical evidence.

  • Robustness to calibration data. Our cross-dataset experiments evaluate HalluRefuse using independently constructed projection and intervention matrices from different calibration data sources, as shown in Table 2. Specifically, we construct PKP_K and Γα\Gamma_\alpha using two datasets and evaluate them on the held-out third dataset, without using any target-dataset samples. HalluRefuse achieves the lowest unknown hallucination rate in 8 out of 9 settings, along with the highest average known accuracy across all three LLMs. These results demonstrate that HalluRefuse can generalize across different calibration datasets.

  • Evaluation on a biomedical domain. To further evaluate the applicability of HalluRefuse beyond general-domain QA, we conduct additional experiments on BioASQ-Factoid [1], a biomedical QA benchmark. We construct the interventions using BioASQ-Factoid calibration data and evaluate their effectiveness on the same domain. The average results across four LLMs are reported below.

MethodAverage Known Accuracy ↑Average Known Refusal ↓Average Unknown Hallucination ↓
Baseline94.412.8269.89
5-shot ICL93.024.2177.30
ITI94.414.1968.66
CAA94.413.2568.88
SADI-HEAD73.773.2468.47
Spherical Steering91.202.7868.06
HalluRefuse (Ours)94.411.8465.44

The results show that HalluRefuse achieves the lowest average unknown hallucination rate, reducing it from 69.89% to 65.44%, while maintaining known accuracy at 94.41% and reducing known refusal from 2.82% to 1.84%. These results demonstrate the effectiveness of HalluRefuse in a specialized biomedical QA setting.

Together, these experiments provide empirical evidence of HalluRefuse's robustness across calibration data sources and its applicability to different QA domains, supporting its effectiveness beyond general-domain benchmarks.

W2: Rank selection.

We thank the reviewer for this comment.

  • First, Fig. 3(b) shows that HalluRefuse maintains a favorable balance across a wide range of intervention ranks. Even at r=3686r=3686, known-question performance remains comparable to the original model, while hallucinations are substantially reduced. The sharp degradation occurs at the extreme setting r=d=4096r=d=4096, meaning that the intervention becomes completely unconstrained and no known-representation directions are protected. Consequently, the method effectively reduces to steering without knowledge protection, resulting in severe over-refusal and a substantial drop in known accuracy. This further demonstrates the importance of our knowledge-preserving constraint.

  • Second, regarding adaptive rank selection, Fig. 2(d) and Appendix B.3 reveal that known representations exhibit concentrated spectral energy across models and datasets, with the top 45 directions capturing 92.8%–99.0% of the total energy. This suggests that protecting the dominant high-energy directions while restricting interventions to low-energy directions can effectively preserve knowledge without requiring input-specific rank adaptation. Furthermore, Table 2 demonstrates that our fixed-subspace approach generalizes to unseen datasets.

Overall, our results show that a fixed subspace already achieves effective hallucination mitigation while preserving known-question performance.

AuthorsOpenReview ↗

Response to Reviewer Mcah

W4: Relationship to model's refusal capability.

We thank the reviewer for this insightful comment.

HalluRefuse leverages the existing refusal behavior of the base model to construct the refusal direction, but does not require a high initial refusal rate. As shown in Table 1, the baseline known refusal rates differ substantially between LLaMA-3.1-8B-Instruct (13.57%) and Qwen2.5-7B-Instruct (2.45%). Despite this difference, HalluRefuse achieves comparable relative reductions in unknown hallucination rates, as shown below.

ModelBaseline Known Refusal ↓Unknown Hallucination ↓Relative Reduction ↑
LLaMA-3.1-8B-Instruct13.57%30.36% → 22.66%25.4%
Qwen2.5-7B-Instruct2.45%63.65% → 49.17%22.8%

These results suggest that HalluRefuse can effectively mitigate hallucinations across models with different initial refusal behaviors.

Q1: Validity of the theoretical guarantee.

We agree that the exact protection guarantee should be distinguished from empirical preservation on unseen known questions, particularly under domain shifts.

  • First, regarding the scope of the theoretical guarantee, Lemma 1 establishes exact protection for representations within the protected subspace. Specifically, when the protected basis spans the full calibration known-representation space, we have PKHK=0P_K H_K=0, and thus Δh=λΓαPKh=0\Delta h=\lambda\Gamma_\alpha P_Kh=0 for any h∈span⁡(HK)h\in\operatorname{span}(H_K). Importantly, this guarantee is not restricted to the calibration samples themselves: it also applies to unseen known questions whose representations lie within the protected subspace. For an arbitrary unseen representation hh, the intervention magnitude satisfies

    \leq |\lambda|\lVert\Gamma_\alpha\rVert_2\lVert P_Kh\rVert_2.$$ This bound shows that the intervention magnitude depends on the residual component of an unseen representation outside the protected subspace. When this residual is small, the intervention is correspondingly bounded.
  • Second, when a truncated subspace is used, protection remains exact within the retained subspace, while the intervention on representations outside this subspace can be bounded. Let UkU_k contain the top-kk left singular vectors of HKH_K, and define Pk=I−UkUk⊤P_k=I-U_kU_k^\top. For any representation hh, we have

    \leq |\lambda|\lVert\Gamma_\alpha\rVert_2\lVert P_kh\rVert_2.$$ Moreover, for the calibration known-representation matrix, $$\lVert\Gamma_\alpha P_kH_K\rVert_F \leq \lVert\Gamma_\alpha\rVert_2 \sqrt{\sum_{i=k+1}^{q}\sigma_i^2},$$ where $q=\operatorname{rank}(H_K)$ and $\sigma_i$ denotes the $i$-th singular value of $H_K$. These bounds characterize approximate preservation in terms of the residual outside the retained subspace. The concentrated spectral energy observed in **Fig. 2(d)** further motivates this design.
  • Third, regarding practical preservation, our existing cross-dataset generalization experiments in Table 2 evaluate the method on unseen target datasets. For each target dataset, the intervention is constructed using the other two datasets, without using the target dataset during construction. HalluRefuse improves average known accuracy over the original models by 4.91, 5.55, and 3.59 percentage points on LLaMA, Qwen, and Mistral, respectively. These results provide empirical evidence that HalluRefuse maintains strong known-question accuracy on unseen datasets, complementing the subspace-conditional theoretical guarantee. Nevertheless, these experiments do not establish unconditional protection under arbitrary domain shifts.

We will refine the theoretical statements in Section 4.2 and Theorem 1 to clarify the scope of exact protection and incorporate the residual-based bound for truncated subspaces.

Q2: Validity of the Constructed Refusal Direction.

As described in Section 4.1, our refusal direction is constructed by contrasting refused and hallucinated unknown questions, aiming to capture refusal-related behavior rather than differences in knowledge states.

Furthermore, our cross-dataset generalization experiments in Table 2 provide empirical support for this design. The refusal direction and intervention are constructed using two source datasets and evaluated on an unseen target dataset. HalluRefuse reduces average unknown hallucination rates by 6.72, 7.03, and 4.06 percentage points on LLaMA, Qwen, and Mistral, respectively.

These results suggest that the constructed direction captures transferable refusal-related signals rather than merely dataset-specific topic or difficulty patterns, although potential confounding factors cannot be completely ruled out.

References

[1] Tsatsaronis, George, et al. An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC bioinformatics, 2015.

Thank you for the clarifications. Two concerns remain:

  1. Cross-domain transfer (W1): Can an intervention constructed from general QA transfer directly to biomedical QA, or must the refusal direction and protection subspace be re-estimated for each domain? The in-domain BioASQ experiment does not resolve this question.

  2. Dependence on refusal capability (W4): My concern is how the refusal direction can be constructed when the base model has little or no ability to abstain on unknown questions. Without reliable refused examples, how would the method obtain a meaningful refusal direction? Comparing refusal rates on known questions does not address this issue.

Reviewer_PgXPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This work addresses a practical limitation in current activation steering methods: because they apply the same intervention to all inputs, reducing hallucinations on unknown questions often hurts accuracy on known ones. The authors find that the representations of known questions actually cluster within a compact low-dimensional subspace. Based on this, they introduce HalluRefuse, a training-free approach that only applies interventions to the orthogonal complement of this subspace. They derive a closed-form solution for the method and test it across three LLMs and three QA benchmarks.

Critical strengths
  1. The motivation is well-founded. Applying finer-grained control to avoid over-steering on known questions is a reasonable direction.
  2. The paper makes this case clearly with empirical evidence. The writing is clear, and the ablations are reasonably thorough.
Critical weaknesses
  1. The core method is essentially identical to AlphaSteer (Sheng et al., ICLR 2026), which was published over a year ago. Both methods project benign/known representations into a null space, formulate the same regularized least-squares objective, derive the same closed-form solution, and apply the same inference rule. AlphaSteer is not cited or compared against in this paper. I would expect to see an explicit discussion of the relationship between the two methods and a re-framing of what the actual contribution is. If the authors can clearly articulate what is new and why it matters beyond domain transfer, I am willing to reconsider my score.
  2. The experiments only cover relatively old models without thinking-mode. It would be useful to see whether the method holds up on newer models that reason at test time, since the known/unknown boundary may behave differently for such models.
Questions for authors
  1. What is the relationship between HalluRefuse and AlphaSteer? Are the closed-form solutions mathematically equivalent? If the method is the same, is the contribution primarily the application to a new domain and the empirical analysis?
  2. Figure 2b shows a correlation between representation shift and accuracy drop within a single method as steering strength varies. However, this correlation does not hold cleanly across methods, for example, SADI-HEAD and Spherical Steering.
4 comments
AuthorsOpenReview ↗

Response to Reviewer PgXP

W1&Q1: Compared with AlphaSteer.

We thank the reviewer for pointing out this closely related work. We acknowledge the similarity in the underlying mathematical formulation and will explicitly cite and discuss AlphaSteer to clarify the contributions of our work.

First, the protected set has a fundamentally different nature. AlphaSteer protects benign prompts, where the benign/harmful distinction is a semantic property of the input. In our setting, the known/unknown distinction is model-specific: it is determined by the model’s sampling consistency (Section 2), and the same question can be known to one model and unknown to another. Known and unknown questions are drawn from the same datasets and share format and topic. It is therefore not obvious a priori that a null space estimated from known questions leaves enough steerable components in unknown representations. Our results show that it does across three LLMs and three benchmarks.

Second, we provide an empirical analysis specific to hallucination mitigation. Section 3 shows, across four steering methods, that stronger steering reduces hallucinations at the cost of known-question accuracy, and that this degradation is associated with shifts in known representations. Figure 2(d) and Appendix B.3 further show that known-question representations are spectrally concentrated across all model–dataset pairs. These findings motivate subspace protection specifically for knowledge-aware abstention, and we will present this analysis as our primary contribution.

We will revise the related work and methodology to clarify the relationship with AlphaSteer and distinguish the shared mathematical framework from our contributions in knowledge-boundary analysis, task-specific intervention construction, and empirical evaluation of hallucination mitigation.

W2: Recent Reasoning Models.

To further evaluate the generalizability of HalluRefuse, we conduct additional experiments on the reasoning model Qwen3-8B. The results are summarized below.

MethodAverage Known Accuracy ↑Average Known Refusal ↓Average Unknown Hallucination ↓
Baseline96.461.5381.45
5-shot ICL89.640.8185.31
ITI90.724.0278.71
CAA95.692.1380.60
SADI-HEAD78.564.3675.63
Spherical Steering95.531.8377.84
HalluRefuse (Ours)96.481.3772.76

As shown above, HalluRefuse achieves the lowest average unknown hallucination rate, reducing it from 81.45% to 72.76% (8.69 percentage points) while maintaining known accuracy (96.46% → 96.48%). It also outperforms all compared methods in unknown hallucination across the three datasets. These results further demonstrate the effectiveness of HalluRefuse on a more recent LLM, providing additional evidence of its applicability across different model families and generations. We will include these results in the revised manuscript.

Q2: Representation Shift and Accuracy Degradation.

Thank you for your comment. We would like to clarify that Figure 2b demonstrates an overall negative association between representation displacement and known accuracy across different steering methods, rather than requiring a strict pairwise ordering of accuracy across all methods and steering strengths.

As shown in Figure 2b, all four methods, including SADI-HEAD and Spherical Steering, exhibit a general downward trend in known accuracy as representation displacement increases. Although individual methods exhibit different rates of accuracy degradation and occasional local fluctuations, their overall trends remain consistent with our observation. Importantly, such variations do not invalidate the negative association.

Furthermore, different steering methods modify model representations through distinct intervention mechanisms. Consequently, the same magnitude of representation displacement may have different effects on known accuracy, since the displacement metric DK(λ)D_K(\lambda) captures the magnitude of representation changes but not their directions or functional significance. This explains why different methods need not follow an identical accuracy–displacement curve, without contradicting the overall association observed in Figure 2b.

Our key observation is that larger steering-induced representation shifts are generally associated with greater degradation in known accuracy, a trend supported by all four methods in Figure 2b. This observation motivates HalluRefuse to explicitly protect known representations during steering, rather than allowing indiscriminate perturbations.

AuthorsOpenReview ↗

Invitation for Further Discussion

Dear Reviewer PgXP,

We have now posted a detailed response to your comments. Thank you again for your constructive feedback, which has helped us improve the paper.

We would appreciate it if you could take a look at our rebuttal. If you find that our response has adequately addressed your concerns, we kindly ask that you consider reconsidering your score.

We look forward to your feedback and are happy to answer any further questions.

Best regards,

The Authors

Thank you for the response and the additional experiments.

Regarding W1/Q1: I acknowledge the authors' point that the problem addressed in this paper is more complex than that in AlphaSteer. However, in my view, the current solution is not specific; the proposed approach remains consistent with AlphaSteer. Consequently, the innovation at the methodological level remains limited.

Regarding W2: The results on Qwen3-8B demonstrate that the method remains effective on a new reasoning model, which addresses my concerns. Accordingly, I will increase "Soundness" and "Informativeness."

Regarding Q2: I understand that the current Dk(λ)D_k(\lambda) cannot be directly compared across different methods. However, this also means that the observation—"a larger Dk(λ)D_k(\lambda) correlates with a greater drop in accuracy"—could easily be conflated with the intuitive notion that "more interference leads to a greater performance decline," which limits the persuasiveness of the finding. If the authors could propose a unified metric to quantify the decline in known-answer accuracy, that would be a more valuable contribution than the method itself.

In light of the above, I am maintaining my rating.

AuthorsOpenReview ↗

Response to Reviewer PgXP's Follow-up Comments

Thank you for your thoughtful follow-up and for acknowledging our additional Qwen3-8B results. We appreciate your remaining concerns and would like to provide further clarification.

R1: Relative to AlphaSteer

We acknowledge that HalluRefuse shares the underlying mathematical framework with AlphaSteer. Our contribution lies in a knowledge-aware intervention construction motivated by the model-specific known/unknown boundary and the spectral structure of known representations.

Unlike the semantic benign/malicious distinction in AlphaSteer, whether a question is known depends on the model itself. Our analysis in Section 3 reveals that known representations exhibit concentrated spectral structure, providing an empirical basis for protecting known representations while steering unknown questions toward refusal. Furthermore, our intervention target is specifically constructed from hallucinated and refused responses to unknown questions.

To further validate the importance of knowledge-dependent subspace selection, we conducted an additional controlled experiment on LLaMA-3.1-8B-Instruct (TriviaQA), comparing different intervention subspaces at the same rank (410). The following results focus on known-question preservation:

MethodKnown Accuracy ↑Known Refusal ↓
HalluRefuse96.88%2.03%
Random Subspace35.47%63.83%
Top-Eigen Subspace35.23%64.06%

Despite identical intervention ranks, alternative subspaces cause severe over-refusal and substantially degrade known accuracy. These results demonstrate that knowledge-dependent subspace selection, rather than dimensionality reduction alone, is critical for preserving known-question performance.

We will further clarify that our contributions lie in the knowledge-boundary analysis, task-specific intervention construction, and empirical validation, rather than the shared optimization framework.

R2: Representation Shift and Accuracy Degradation

We agree that DK(λ)D_K(\lambda) measures representation displacement rather than its functional impact, and therefore is not directly comparable across different steering methods. We clarify that Figure 2(b) illustrates a within-method association between representation displacement and known accuracy degradation, rather than a universal cross-method relationship.

The controlled experiment above provides complementary evidence that the choice of intervention directions matters beyond simply restricting intervention dimensionality. We also appreciate your suggestion of a unified functional metric and agree that developing such a metric would be a valuable direction for future research. We will clarify the interpretation and limitations of Figure 2(b) accordingly.

Thank you again for your constructive feedback. We hope these clarifications, together with the additional experimental evidence, help address your remaining concerns. We would be grateful if you could reconsider your overall rating in light of these results and clarifications.

Reviewer_iDL9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

How can one guide a large language model to decline answering questions about unknown topics, while preserving, as much as possible, its ability to correctly answer questions about known topics?

Critical strengths

The research problem holds practical value; rather than focusing solely on reducing the hallucination rate, the paper addresses both "Known Accuracy" and "Unknown Hallucination." This is a sound objective, as a model that consistently refuses to answer—while potentially achieving a very low hallucination rate—would fail to meet the requirements of practical applications. The proposed method is mathematically clear, with Appendix D, in particular, providing a derivation for the regularized optimization problem.

Critical weaknesses

Could you elaborate on the distinctions between this paper and AlphaSteer? Furthermore, the current theorems only guarantee that the constructed protective samples remain resistant to direct intervention under ideal conditions; they do not ensure that accuracy is maintained for all previously unseen (yet known-type) issues. It is also necessary to clarify whether the preserved direction truly corresponds to a zero eigenvalue. In essence, the proposed method adaptively adjusts the intervention intensity based on input activations, while keeping the "refusal" (or rejection) direction fixed.

Questions for authors

How to demonstrate substantial novelty relative to AlphaSteer Whether the actual protection subspace satisfies the condition of exact zero-intervention. Does the performance improvement actually stem from orthogonal protection?

5 comments
AuthorsOpenReview ↗

Response to Reviewer iDL9

We thank the reviewer for this insightful comment.

W1: Compared with AlphaSteer.

We thank the reviewer for pointing out this closely related work. We acknowledge the similarity in the underlying mathematical formulation and will explicitly cite and discuss AlphaSteer to clarify the contributions of our work.

First, the protected set has a fundamentally different nature. AlphaSteer protects benign prompts, where the benign/harmful distinction is a semantic property of the input. In our setting, the known/unknown distinction is model-specific: it is determined by the model’s sampling consistency (Section 2), and the same question can be known to one model and unknown to another. Known and unknown questions are drawn from the same datasets and share format and topic. It is therefore not obvious a priori that a null space estimated from known questions leaves enough steerable components in unknown representations. Our results show that it does across three LLMs and three benchmarks.

Second, we provide an empirical analysis specific to hallucination mitigation. Section 3 shows, across four steering methods, that stronger steering reduces hallucinations at the cost of known-question accuracy, and that this degradation is associated with shifts in known representations. Figure 2(d) and Appendix B.3 further show that known-question representations are spectrally concentrated across all model–dataset pairs. These findings motivate subspace protection specifically for knowledge-aware abstention, and we will present this analysis as our primary contribution.

We will revise the related work and methodology to clarify the relationship with AlphaSteer and distinguish the shared mathematical framework from our contributions in knowledge-boundary analysis, task-specific intervention construction, and empirical evaluation of hallucination mitigation.

W2: Scope of Theoretical Protection.

We clarify the scope and conditions of our theoretical guarantee and provide further theoretical and empirical evidence below.

  • First, regarding the scope of the theoretical guarantee, Lemma 1 establishes exact protection for the entire protected subspace, rather than only the samples used for its construction. Specifically, when the protected basis spans the calibration known-representation space, we have PKHK=0P_K H_K=0, and thus Δh=λΓαPKh=0\Delta h=\lambda\Gamma_\alpha P_Kh=0 for any h∈span⁡(HK)h\in\operatorname{span}(H_K), including unseen representations lying in this subspace.

  • Second, for representations outside the protected subspace, the intervention magnitude can still be bounded. Exact zero-intervention holds when the selected intervention directions lie in the null space of HKHK⊤H_KH_K^\top, corresponding to zero eigenvalues. When directions associated with small nonzero eigenvalues are selected, protection becomes approximate rather than strictly exact. Let UkU_k contain the top-kk left singular vectors of HKH_K, and define Pk=I−UkUk⊤P_k=I-U_kU_k^\top. For any representation hh, we have

    \leq |\lambda|\lVert\Gamma_\alpha\rVert_2\lVert P_kh\rVert_2.$$ Moreover, for the calibration known-representation matrix, $$\lVert\Gamma_\alpha P_kH_K\rVert_F \leq \lVert\Gamma_\alpha\rVert_2 \sqrt{\sum_{i=k+1}^{q}\sigma_i^2},$$ where $q=\operatorname{rank}(H_K)$ and $\sigma_i$ denotes the $i$-th singular value of $H_K$. These bounds show that the intervention magnitude depends on the residual component outside the protected subspace, providing a characterization of approximate protection beyond the exact subspace condition. The concentrated spectral energy observed in **Fig. 2(d)** further motivates this design.
  • Third, regarding generalization to unseen known questions, our existing cross-dataset experiments in Table 2 directly evaluate this setting. Specifically, we construct the intervention using two datasets and evaluate it on the held-out third dataset, without using any target-dataset samples during construction. HalluRefuse improves average known accuracy over the original models by 4.91, 5.55, and 3.59 percentage points on LLaMA, Qwen, and Mistral, respectively. These results provide empirical evidence that HalluRefuse preserves strong known-question performance on unseen datasets, complementing the subspace-conditional theoretical guarantee.

We will refine the statements in Section 4.2 and Theorem 1 to clarify the distinction between exact and approximate protection and incorporate the residual-based bound for truncated subspaces.

AuthorsOpenReview ↗

Response to Reviewer iDL9

Q2: Effectiveness of Orthogonal Protection.

We provide evidence supporting the contribution of orthogonal protection through our existing ablation studies and representation-level observations.

  • First, the subspace rank ablation in Fig. 3(b) directly examines the effect of restricting the intervention subspace. When the intervention rank is r=410r=410, HalluRefuse achieves 96.88% known accuracy with only 2.03% known refusal. In contrast, when the intervention is extended to the full hidden space (r=4096r=4096), effectively removing the subspace protection constraint, known accuracy drops to 36.41%, while known refusal increases to 62.97%. This substantial degradation demonstrates the importance of restricting interventions to protect known representations.

  • Second, our representation-level analysis in Fig. 2(b) provides further support for this mechanism. We observe that larger steering-induced shifts in known representations are generally associated with lower known accuracy. Orthogonal protection directly addresses this issue by restricting the intervention to directions outside the protected known-representation subspace.

  • Third, the intervention strength ablation in Fig. 3(a) further supports the effectiveness of the protection mechanism. As the steering strength increases, unknown hallucination decreases substantially, while known accuracy remains relatively stable. This indicates that the protected intervention can mitigate hallucinations without causing the pronounced degradation of known-question performance associated with unrestricted steering.

Together, these results provide consistent evidence that orthogonal protection plays an important role in preserving known-question performance while mitigating hallucinations. We will further clarify this connection in the ablation discussion in Section 5.5.

Thank you for the detailed response and additional experiments.

I appreciate the clarifications, but I am still having difficulty understanding the precise mechanism and the scope of the theoretical guarantee.

In particular, I would appreciate further clarification on the following points:

Theoretical guarantee vs. practical preservation: My understanding is that exact protection only holds for representations lying within the protected subspace. In practice, when a truncated subspace is used, how does this guarantee translate into preserving the accuracy of unseen known questions? Could you provide a more intuitive explanation or a concrete example?

Role of orthogonal protection: Could you explain more explicitly why the performance improvements should be attributed to orthogonal protection itself, rather than simply restricting the intervention to fewer directions? A controlled comparison would help clarify this distinction.

Difference from AlphaSteer: I understand that the application scenarios differ, but I am still unsure what the essential methodological novelty is relative to AlphaSteer. Could you clarify which part of the proposed intervention is fundamentally different?

Reviewer_Jg5kOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Authors introduce HalluRefuse, an activation steering method designed to reduce hallucinations on unknown questions while preserving accuracy on known questions. The method estimates a known-representation subspace using SVD, then learns a closed-form intervention on orthogonal complement towards a refusal direction at inference time without updating LLM parameters. Experiments across three OSS LLMs and three QA benchmarks show reduced unknown-question hallucinations while preserving known-question accuracy.

Critical strengths
  1. Clear problem motivation. The paper explicitly addresses the hallucination <> accuracy trade-off in activation steering while preventing degradation on known questions.
  2. Simple & interpretable method. Known-subspace protection and closed-form intervention provide a clear geometric formulation without having to fine-tune LLMs with supervision signals.
  3. Consistent empirical improvements. Average unknown hallucination decreases by consistently over the original models in a model agnostic manner. Ablation studies show which components contribute to the gain the most. Cross-dataset emprical design is robust as well.
  4. The visualization of the known and unknown PCA subspace is helpful in understanding the effect of the activation steering.
Critical weaknesses
  1. Insufficient comparison with other baselines: the proposed method is compared against one in-context approach and other activation steering methods without detailed breakdown of how each method differs. One additional baseline that will be helpful to compare against is the models directly fine-tuned to refuse questions with unknown knowledge (sampled via rejection samplings) -- where the refusal behavior is better built into the model parameters.
  2. Outdated underlying base models undermine the claim that the proposed approach is model agnostic. I recommend authors to add additional and more recent baselines to compare against.
  3. Gap between theoretical guarantee & practical preservation. Exact protection holds for representations in the full calibration span, not arbitrary unseen known questions, and truncating that span would make protection only approximated. The rank definition is also somewhat unclear as well -- Fig. 3(b) shows severe over-refusal at full rank, whereas Eq. (8) would give no intervention if this denotes the protected rank.
  4. Nit: Coarse correctness & refusal evaluation. Substring answer matching is inaccurate e.g. as some of the models have hedging behaviors before it correctly answers the question.
Questions for authors

Summarized in the weakness section

4 comments
AuthorsOpenReview ↗

Response to Reviewer Jg5k

We appreciate the reviewer’s comments and suggestions. Below we address each point in detail.

W1: Comparison with fine-tuning-based methods. Thank you for your suggestion. As discussed in the Section 1, fine-tuning-based methods require additional training data, optimization, and model updates, resulting in substantial computational costs. Avoiding these costs is a key motivation of our method. Since fine-tuning-based methods operate under different computational requirements, a direct performance comparison without accounting for training costs would be unfair. Therefore, we focus on representative training-free baselines to ensure a comparison under comparable conditions.

W2: Validation on recent models.

We thank the reviewer for this valuable suggestion. To further evaluate the generalizability of HalluRefuse, we conduct additional experiments on the reasoning model Qwen3-8B. The results are summarized below.

MethodAverage Known Accuracy ↑Average Known Refusal ↓Average Unknown Hallucination ↓
Baseline96.461.5381.45
5-shot ICL89.640.8185.31
ITI90.724.0278.71
CAA95.692.1380.60
SADI-HEAD78.564.3675.63
Spherical Steering95.531.8377.84
HalluRefuse (Ours)96.481.3772.76

As shown above, HalluRefuse achieves the lowest average unknown hallucination rate, reducing it from 81.45% to 72.76% (8.69 percentage points) while maintaining known accuracy (96.46% → 96.48%). It also outperforms all compared methods in unknown hallucination across the three datasets. These results further demonstrate the effectiveness of HalluRefuse on a more recent LLM, providing additional evidence of its applicability across different model families and generations. We will include these results in the revised manuscript.

W3: Scope of theoretical protection.

We thank the reviewer for this insightful comment.

  • First, regarding the scope of the theoretical guarantee, Lemma 1 establishes exact protection for representations within the protected subspace. Specifically, when the protected basis spans the full calibration known-representation space, we have PKHK=0P_K H_K=0, and thus Δh=λΓαPKh=0\Delta h=\lambda\Gamma_\alpha P_Kh=0 for any h∈span⁡(HK)h\in\operatorname{span}(H_K).

  • Second, when a truncated subspace is used, protection remains exact within the retained subspace, while the intervention on representations outside this subspace can be bounded. Let UkU_k contain the top-kk left singular vectors of HKH_K, and define Pk=I−UkUk⊤P_k=I-U_kU_k^\top. For any representation hh, we have

    \leq |\lambda|\lVert\Gamma_\alpha\rVert_2\lVert P_kh\rVert_2.$$ Moreover, for the calibration known-representation matrix, $$\lVert\Gamma_\alpha P_kH_K\rVert_F \leq \lVert\Gamma_\alpha\rVert_2 \sqrt{\sum_{i=k+1}^{q}\sigma_i^2},$$ where $q=\operatorname{rank}(H_K)$ and $\sigma_i$ denotes the $i$-th singular value of $H_K$. These bounds characterize approximate preservation in terms of the residual outside the protected subspace. The concentrated spectral energy observed in **Fig. 2(d)** further motivates this design.
  • Third, regarding practical preservation, our existing cross-dataset generalization experiments in Table 2 evaluate the method on unseen target datasets. When constructing the intervention using the other two datasets, HalluRefuse improves average known accuracy over the original models by 4.91, 5.55, and 3.59 percentage points on LLaMA, Qwen, and Mistral, respectively. These results provide empirical evidence that HalluRefuse maintains strong known-question accuracy on unseen datasets, complementing the subspace-conditional theoretical guarantee.

We will refine the theoretical statements in Section 4.2 and Theorem 1 to clarify the scope of exact protection and incorporate the residual-based bound for truncated subspaces.

W4: Rank definition.

We thank the reviewer for pointing out this ambiguity. We clarify that the rank rr in Fig. 3(b) denotes the intervention subspace rank, rather than the protected subspace rank rKr_K in Eq. (8). Specifically, in our implementation, we select the rr eigenvectors corresponding to the smallest eigenvalues of the known-representation Gram matrix to construct the intervention projector Pr=QrQr⊤P_r=Q_rQ_r^\top. Thus, the intervention and protected subspace dimensions are complementary, i.e., r=d−rKr=d-r_K.

At r=d=4096r=d=4096 in Fig. 3(b), we have Pr=IP_r=I, meaning that the intervention is no longer constrained by the protected subspace. This explains the observed over-refusal, where known accuracy drops to 36.41% and known refusal increases to 62.97%. In contrast, rK=dr_K=d in Eq. (8) would yield PK=0P_K=0, resulting in no intervention.

We will correct it to "intervention subspace rank" to eliminate this ambiguity.

AuthorsOpenReview ↗

Response to Reviewer Jg5k

W5: Accuracy of string-matching-based evaluation.

As discussed in Section 1, a reliable LLM should answer known questions correctly while recognizing its knowledge limitations on unknown questions. Consistent with Yang et al. [1], we evaluate answer correctness and refusal separately using answer matching and refusal phrase detection. Specifically, a response that expresses uncertainty before providing the correct answer is still counted as correct, while the uncertainty expression may also be identified as a refusal. This allows us to evaluate whether the model provides the correct answer and whether it contains refusal-related expressions without treating the two outcomes as mutually exclusive.

References

[1] Yang, Wannan, et al. Hallucination reduction with casal: Contrastive activation steering for amortized learning. ICLR, 2026.

AuthorsOpenReview ↗

Invitation for Further Discussion

Dear Reviewer Jg5k,

We have now posted a detailed response to your comments. Thank you again for your constructive feedback, which has helped us improve the paper.

We would appreciate it if you could take a look at our rebuttal. If you find that our response has adequately addressed your concerns, we kindly ask that you consider reconsidering your score.

We look forward to your feedback and are happy to answer any further questions.

Best regards,

The Authors

AuthorsOpenReview ↗

Invitation for Further Discussion

Dear Reviewer Jg5k,

Thank you again for your valuable feedback. We have provided detailed responses to your concerns, including additional experiments on Qwen3-8B and clarifications regarding the theoretical guarantee, rank definition, and evaluation protocol.

We would greatly appreciate it if you could revisit our rebuttal and share any remaining concerns. If you find that our responses and additional evidence have adequately addressed your concerns, we kindly ask you to consider updating your rating.

Thank you very much for your time and consideration.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers and AC,

We sincerely thank the reviewers for their valuable time and constructive feedback. We are encouraged that the reviewers found our problem "holds practical value" (iDL9) with a "well-founded" motivation (PgXP), our method "simple & interpretable" (Jg5k), "mathematically clear" (iDL9), and "training-free and efficient" (Mcah), and our results show "consistent improvements across models and datasets" (Mcah) with "reasonably thorough" ablations (PgXP). We have summarized the reviewers' comments into the following five categories:

  • Relationship with AlphaSteer
  • Scope of the theoretical guarantee
  • Evaluation on recent models and new domains
  • Sensitivity to hyperparameters
  • Clarification of the method mechanism and evaluation

To address the reviewers' concerns, we have conducted additional experiments and analyses, including:

  • In response to the shared concern from Reviewers iDL9 and PgXP, we explicitly cite and discuss AlphaSteer in Appendix A, and clarify our contributions: a model-specific known/unknown setting, the empirical trade-off and spectral analysis in Section 3, and task-specific intervention construction for hallucination mitigation.
  • In response to the shared concern from Reviewers Jg5k, iDL9, and Mcah, we refine Section 4.2 and Theorem 1 to distinguish exact protection within span(HK)\mathrm{span}(H_K) from approximate protection outside it, and add a residual-based bound in Appendix C.
  • In response to the shared concern from Reviewers Jg5k and PgXP, we add the reasoning model Qwen3-8B to Table 1, where HalluRefuse reduces average hallucination by 8.69 points while maintaining known accuracy.
  • In response to Reviewer Mcah's concern about domain shift, we add biomedical experiments on BioASQ-Factoid in Section 5.4 and Table 3, and highlight the cross-dataset results in Table 2.
  • In response to Reviewer Mcah's question about the intervention layer, we add a layer-wise ablation in Appendix I and Figure 19.
  • In response to Reviewers Jg5k and Mcah regarding the rank, we clarify that rr in Figure 3(b) is the intervention subspace rank (r=d−rKr = d - r_K) and revise the terminology in Section 5.5.
  • In response to Reviewers iDL9, PgXP, and Mcah, we clarify the role of orthogonal protection (Figures 2(b) and 3), the interpretation of DK(λ)D_K(\lambda), and the robustness of the refusal direction across models with different initial refusal rates.
  • In response to Reviewer Jg5k, we clarify our choice of training-free baselines and our evaluation protocol, which scores correctness and refusal separately.

All revisions in the paper are indicated in green font. We believe these revisions have significantly strengthened the paper and addressed the raised concerns.

Thank you very much, Authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.