- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
HalluRefuse addresses the trade-off in conventional activation steering, where reducing hallucinations on unknown questions often degrades accuracy on known questions. The method identifies a low-dimensional subspace of known representations using SVD and protects it by applying interventions only to the orthogonal subspace. A refusal direction is constructed from the activation differences between refused and hallucinated responses, and an intervention matrix is computed in closed form without additional model training. Experiments across three LLMs and three QA benchmarks show that HalluRefuse reduces hallucination rates by 5.88–14.48 percentage points while preserving or improving known-question accuracy.
Critical strengths
1. Empirical analysis of the limitations of existing activation steering methods
-
The paper demonstrates a clear trade-off between hallucination mitigation and known-question accuracy as steering strength increases.
-
The analysis of representation shifts and singular value decomposition (SVD) provides empirical motivation for protecting the known representation subspace.
2. Training-free and efficient intervention
-
The proposed method does not require updating model parameters and derives the intervention matrix using a projection operator and a closed-form solution.
-
The precomputed intervention matrix can be applied during inference without additional online training or explicit known/unknown question classification.
3. Consistent improvements across models and datasets
-
The method consistently reduces hallucination rates while preserving or improving known-question accuracy across three LLMs and three QA benchmarks.
-
The cross-dataset evaluation provides additional evidence of the method's generalization capability beyond the datasets used for constructing the intervention.
Critical weaknesses
1. Dependence on calibration samples and limited generalization
-
The projection matrix and intervention matrix are constructed from sampled known and unknown questions, making them potentially sensitive to the selection of calibration samples.
-
Although cross-dataset experiments are conducted, the effectiveness of the method under substantial domain shifts (e.g., from general QA to medical QA) remains unexplored.
-
The paper lacks a sensitivity analysis examining how different calibration samples affect the stability of the protected subspace and overall performance.
2. Remaining trade-off and lack of adaptive rank selection
-
The intervention subspace rank is determined in advance and remains fixed during inference.
-
Figure 3(b) shows that increasing the rank reduces hallucination rates but also leads to a substantial degradation in known-question accuracy.
-
This suggests that HalluRefuse mitigates rather than fundamentally resolves the trade-off highlighted in the paper. Furthermore, the method lacks a mechanism for dynamically selecting an appropriate rank based on individual inputs or domains.
3. Limited justification for the fixed intervention layer
-
The method applies intervention to the final input token at layer 19 across all evaluated models.
-
However, the paper provides insufficient justification for selecting this particular layer, and the optimal intervention layer may vary across models, input questions, or domains.
-
A layer-wise ablation study would strengthen the paper by examining the sensitivity of the proposed method to intervention location and its implications for generalization.
4. Dependence on the model's existing refusal capability
-
The refusal direction is constructed from the difference between the mean representations of unknown questions that the model already refuses and those for which it produces hallucinated responses.
-
Consequently, if the base model rarely exhibits appropriate refusal behavior, constructing a reliable refusal direction may be challenging.
-
The paper does not sufficiently investigate how the effectiveness of HalluRefuse varies with the base model's initial refusal capability.
Overall, the experimental results demonstrate that HalluRefuse is effective under the evaluated models, datasets, and hyperparameter settings. However, given its dependence on calibration samples, subspace rank, and intervention layer selection, it remains unclear how well the method would generalize to substantially different domains or deployment settings.
Questions for authors
-
Validity of the theoretical guarantee: The guarantee applies only to the predefined known representations. To what extent can this guarantee be extended to unseen known questions, particularly under domain shifts?
-
How can the authors ensure that the constructed refusal direction captures refusal behavior rather than confounding factors such as question topics or difficulty?
5 comments
Response to Reviewer Mcah
W4: Relationship to model's refusal capability.
We thank the reviewer for this insightful comment.
HalluRefuse leverages the existing refusal behavior of the base model to construct the refusal direction, but does not require a high initial refusal rate. As shown in Table 1, the baseline known refusal rates differ substantially between LLaMA-3.1-8B-Instruct (13.57%) and Qwen2.5-7B-Instruct (2.45%). Despite this difference, HalluRefuse achieves comparable relative reductions in unknown hallucination rates, as shown below.
| Model | Baseline Known Refusal ↓ | Unknown Hallucination ↓ | Relative Reduction ↑ |
|---|---|---|---|
| LLaMA-3.1-8B-Instruct | 13.57% | 30.36% → 22.66% | 25.4% |
| Qwen2.5-7B-Instruct | 2.45% | 63.65% → 49.17% | 22.8% |
These results suggest that HalluRefuse can effectively mitigate hallucinations across models with different initial refusal behaviors.
Q1: Validity of the theoretical guarantee.
We agree that the exact protection guarantee should be distinguished from empirical preservation on unseen known questions, particularly under domain shifts.
-
First, regarding the scope of the theoretical guarantee,
\leq |\lambda|\lVert\Gamma_\alpha\rVert_2\lVert P_Kh\rVert_2.$$ This bound shows that the intervention magnitude depends on the residual component of an unseen representation outside the protected subspace. When this residual is small, the intervention is correspondingly bounded.Lemma 1establishes exact protection for representations within the protected subspace. Specifically, when the protected basis spans the full calibration known-representation space, we have , and thus for any . Importantly, this guarantee is not restricted to the calibration samples themselves: it also applies to unseen known questions whose representations lie within the protected subspace. For an arbitrary unseen representation , the intervention magnitude satisfies -
Second, when a truncated subspace is used, protection remains exact within the retained subspace, while the intervention on representations outside this subspace can be bounded. Let contain the top- left singular vectors of , and define . For any representation , we have
\leq |\lambda|\lVert\Gamma_\alpha\rVert_2\lVert P_kh\rVert_2.$$ Moreover, for the calibration known-representation matrix, $$\lVert\Gamma_\alpha P_kH_K\rVert_F \leq \lVert\Gamma_\alpha\rVert_2 \sqrt{\sum_{i=k+1}^{q}\sigma_i^2},$$ where $q=\operatorname{rank}(H_K)$ and $\sigma_i$ denotes the $i$-th singular value of $H_K$. These bounds characterize approximate preservation in terms of the residual outside the retained subspace. The concentrated spectral energy observed in **Fig. 2(d)** further motivates this design. -
Third, regarding practical preservation, our existing cross-dataset generalization experiments in Table 2 evaluate the method on unseen target datasets. For each target dataset, the intervention is constructed using the other two datasets, without using the target dataset during construction. HalluRefuse improves average known accuracy over the original models by 4.91, 5.55, and 3.59 percentage points on LLaMA, Qwen, and Mistral, respectively. These results provide empirical evidence that HalluRefuse maintains strong known-question accuracy on unseen datasets, complementing the subspace-conditional theoretical guarantee. Nevertheless, these experiments do not establish unconditional protection under arbitrary domain shifts.
We will refine the theoretical statements in Section 4.2 and Theorem 1 to clarify the scope of exact protection and incorporate the residual-based bound for truncated subspaces.
Q2: Validity of the Constructed Refusal Direction.
As described in Section 4.1, our refusal direction is constructed by contrasting refused and hallucinated unknown questions, aiming to capture refusal-related behavior rather than differences in knowledge states.
Furthermore, our cross-dataset generalization experiments in Table 2 provide empirical support for this design. The refusal direction and intervention are constructed using two source datasets and evaluated on an unseen target dataset. HalluRefuse reduces average unknown hallucination rates by 6.72, 7.03, and 4.06 percentage points on LLaMA, Qwen, and Mistral, respectively.
These results suggest that the constructed direction captures transferable refusal-related signals rather than merely dataset-specific topic or difficulty patterns, although potential confounding factors cannot be completely ruled out.
References
[1] Tsatsaronis, George, et al. An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC bioinformatics, 2015.
Thank you for the clarifications. Two concerns remain:
-
Cross-domain transfer (W1): Can an intervention constructed from general QA transfer directly to biomedical QA, or must the refusal direction and protection subspace be re-estimated for each domain? The in-domain BioASQ experiment does not resolve this question.
-
Dependence on refusal capability (W4): My concern is how the refusal direction can be constructed when the base model has little or no ability to abstain on unknown questions. Without reliable refused examples, how would the method obtain a meaningful refusal direction? Comparing refusal rates on known questions does not address this issue.
Response to Reviewer Mcah
We thank the reviewer for this valuable comment.
We thank the reviewer for this valuable comment. We address this concern through additional empirical evidence.
Robustness to calibration data. Our cross-dataset experiments evaluate HalluRefuse using independently constructed projection and intervention matrices from different calibration data sources, as shown in Table 2. Specifically, we construct PK and Γα using two datasets and evaluate them on the held-out third dataset, without using any target-dataset samples. HalluRefuse achieves the lowest unknown hallucination rate in 8 out of 9 settings, along with the highest average known accuracy across all three LLMs. These results demonstrate that HalluRefuse can generalize across different calibration datasets.
Evaluation on a biomedical domain. To further evaluate the applicability of HalluRefuse beyond general-domain QA, we conduct additional experiments on BioASQ-Factoid [1], a biomedical QA benchmark. We construct the interventions using BioASQ-Factoid calibration data and evaluate their effectiveness on the same domain. The average results across four LLMs are reported below.
The results show that HalluRefuse achieves the lowest average unknown hallucination rate, reducing it from 69.89% to 65.44%, while maintaining known accuracy at 94.41% and reducing known refusal from 2.82% to 1.84%. These results demonstrate the effectiveness of HalluRefuse in a specialized biomedical QA setting.
Together, these experiments provide empirical evidence of HalluRefuse's robustness across calibration data sources and its applicability to different QA domains, supporting its effectiveness beyond general-domain benchmarks.
We thank the reviewer for this comment.
First, Fig. 3(b) shows that HalluRefuse maintains a favorable balance across a wide range of intervention ranks. Even at r=3686, known-question performance remains comparable to the original model, while hallucinations are substantially reduced. The sharp degradation occurs at the extreme setting r=d=4096, meaning that the intervention becomes completely unconstrained and no known-representation directions are protected. Consequently, the method effectively reduces to steering without knowledge protection, resulting in severe over-refusal and a substantial drop in known accuracy. This further demonstrates the importance of our knowledge-preserving constraint.
Second, regarding adaptive rank selection, Fig. 2(d) and Appendix B.3 reveal that known representations exhibit concentrated spectral energy across models and datasets, with the top 45 directions capturing 92.8%–99.0% of the total energy. This suggests that protecting the dominant high-energy directions while restricting interventions to low-energy directions can effectively preserve knowledge without requiring input-specific rank adaptation. Furthermore, Table 2 demonstrates that our fixed-subspace approach generalizes to unseen datasets.
Overall, our results show that a fixed subspace already achieves effective hallucination mitigation while preserving known-question performance.