acceptodds
Under review as a conference paper at ICLR 2027

K-Shield: Improving the Safety-Utility Trade-off in Refusal Steering with Local Null-Space Constraints

42 views

Abstract

Activation steering offers a promising approach to enhancing the safety of large language models without updating their weights, but inducing refusal on malicious prompts while preserving benign utility remains challenging. Existing null-space methods address this by excluding directions associated with benign activations from the steering map's inputs. However, a direction important for benign behavior in one region of the activation space may enable malicious refusal elsewhere, so protecting it globally can hinder effective steering. We characterize the resulting refusal target reconstruction error and show how local null-space constraints can reduce it. Building on this insight, we propose K-Shield, which uses optimal transport to learn local subspaces and routes inputs to region-specific steering maps and refusal targets. Across three language models, seven jailbreak attacks, and four utility benchmarks, K-Shield consistently improves defense success and aggregate utility over global null-space constrained steering.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
activation steering, LLM safety, jailbreak defense, null-space constraints, optimal transport

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pK3HOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes K-Shield, an activation-sterring method that partitions benign activations into local regions, constructs a region-specific protected subspace and steering map, and routes each input to the nearest protected subspace. The paper is technically polished and reports strong empirical results. However, the central claim—an improved safety–utility trade-off—is not yet established convincingly

Critical strengths
  1. The motivation for local rather than global preservation constraints is intuitive and clearly presented.

  2. The paper is well written and generally easy to follow.

Critical weaknesses
  1. Null-space projection is already established in representation erasure, continual learning, model editing, refusal steering, and increasingly LLL unlearning. Thus, using a null-space constraint alone is not novel. There are already many papers, such as AlphaSteer [r1], Leace [r2], AlphaEdit [r3], that the paper should compared with.

  2. The OT component appears to be a standard entropy-regularized balanced assignment with subspace-reconstruction cost, rather than a novel contribution to OT.

  3. Tables 1 and 2 compare one configuration of each method. K-Shield is tuned separately for each model, while the baselines are reproduced using their released settings. Since defense success and utility depend strongly on steering strength and other hyperparameters, these isolated points do not establish that K-Shield has a better trade-off.

  4. Near-perfect DSR on static prompts is insufficient evidence of robustness. At least one or two strong attacks should be regenerated with the defense in the loop, with comparable attack budgets across methods.

  5. The main tables provide no confidence intervals or significance tests. Several utility improvements are extremely small—for example 0.5–2 percentage points—while MATH and GSM8K contain only 100 questions each. Such differences may be sampling noise.

[r1]. Sheng, Leheng, et al. "Alphasteer: Learning refusal steering with principled null-space constraint." International Conference on Learning Representations. Vol. 2026. 2026.

[r2]. Belrose, Nora, et al. "Leace: Perfect linear concept erasure in closed form." Advances in Neural Information Processing Systems 36 (2023): 66044-66063.

[r3]. Fang, Junfeng, et al. "Alphaedit: Null-space constrained knowledge editing for language models." International Conference on Learning Representations. Vol. 2025. 2025.

Questions for authors

See weaknesses.

5 comments

We thank the reviewer for their thoughtful questions and constructive feedback. We provide our responses below, including additional experimental results and clarifications of our contributions and claims. This response is split into three comments.

Q1. Methodological novelty

We agree that null-space projection is established and do not claim the constraint itself as novel. Our contribution concerns where the constraint is applied. Specifically, we identify a limitation of the global null-space formulation used in prior methods and show how locally chosen protected directions can improve refusal-target approximation.

K-Shield's central methodological contribution is therefore the move from global to local null-space constraints, supported by analysis and empirical evidence in refusal steering. This also suggests a promising extension to other applications of null-space-constrained methods, although its benefits beyond refusal steering remain to be evaluated.

Regarding comparisons, AlphaSteer is our closest baseline and is evaluated in Tables 1–2 across all three models and the evaluated attacks. LEACE and AlphaEdit are relevant methodological precedents but address different objectives: LEACE removes linearly recoverable concept information, whereas AlphaEdit edits factual knowledge in model parameters while preserving unrelated knowledge. Neither directly provides a refusal-steering defense. Adapting them would therefore require additional design choices. We will expand the Related Works discussion to clarify these distinctions and position our contribution more precisely within existing uses of null-space projection.

Q2. Optimal transport novelty

We agree with this characterization. We do not propose a new OT algorithm. We use entropy-regularized OT with subspace-reconstruction costs to learn local subspaces through soft, balanced assignments. Our controlled ablations evaluate the practical value of this component. We will clarify that OT is a supporting component of K-Shield, while the central methodological contribution is the move from global to local null-space constraints.

Q3. Baseline comparison fairness and safety–utility trade-off

We agree that Tables 1–2 compare selected configurations and do not establish a generally superior safety–utility trade-off. To examine sensitivity to these operating points, we evaluated K-Shield and AlphaSteer, our closest baseline, across steering strengths of 0.1–0.5 on all three models, holding their remaining hyperparameters fixed. We focus on steering strength because it directly scales the intervention in both methods and provides a shared inference-time control for varying the safety–utility balance without changing the learned transformations.

The main tables report AlphaSteer's reproduced results at its released experimental settings. The sweep shows that increasing its steering strength can improve DSR beyond those values, but may incur substantial utility degradation or collapse. For example, on Gemma-2, increasing λ\lambda from 0.14 to 0.5 raises DSR from 76.54% to 86.1%, while average utility falls from 57.8% to 0.7%.

Across all three models, K-Shield achieves higher average DSR at every evaluated shared steering strength, with comparable or higher average utility at multiple operating points. Representative comparisons at the same strength are:

ModelSteering strength (λ\lambda)K-Shield: DSR / utilityAlphaSteer: DSR / utility
Llama-3.1-8B-Instruct0.397.11% / 71.3%91.49% / 70.2%
Qwen-2.5-7B-Instruct0.284.40% / 79.3%57.06% / 77.5%
Gemma-2-9B-IT0.296.20% / 69.3%80.1% / 38.4%

These results show that K-Shield’s empirical safety-utility advantage extends beyond the originally reported operating points. However, we acknowledge that the sweep isolates steering-strength sensitivity with the remaining settings fixed. It therefore does not establish superiority across other hyperparameter choices or fully optimised trade-off frontiers. We will revise the contribution statements and conclusions to limit our claims to the evaluated settings and avoid implying a guaranteed or universally better safety–utility trade-off. We have added the experimental setup and full results to Appendix G.1, “Steering Strength and Safety–Utility Trade-off,” with the sweep summarized in Figure 5 and per-attack and per-benchmark results reported in Tables 12–17.

Q4. Adaptive attack robustness

We have added two adaptive white-box GCG evaluations on Llama-3.1-8B-Instruct, comparing the unsteered model, AlphaSteer, and K-Shield. Adversarial suffixes were optimized separately for each configuration, with the defense active and gradients computed through its steering transformation. Each attack used matched settings across configurations: 200 optimization steps, search width 512, top-k 256, and seed 0.

Starting from the 100 AdvBench behaviors used in Table 1, a preliminary GCG run successfully attacked 53 behaviors on the unsteered model. We evaluated the defenses on these 53 behaviors to avoid crediting them for failures of the attack against the original model. Attack Success Rate (ASR) is therefore conditional on this attackable subset. Using the same judge as Table 1, a behavior counted as successfully attacked if any checkpoint at steps 50, 100, 150, or 200 produced a jailbreak.

Explicitly targeting the steering mechanism. In addition to standard GCG, which optimizes the target-response likelihood through the defended model, we evaluated a defense-aware variant that explicitly penalizes the relative steering magnitude:

Lattack=LCE(target)+β∣Ls∣∑l∈Ls∥vl∥2∥hl∥2\mathcal{L}_{\mathrm{attack}} = \mathcal{L}_{\mathrm{CE}}(\mathrm{target}) + \frac{\beta}{|\mathcal{L}_s|}\sum_{l \in \mathcal{L}_s}\frac{\lVert v_l \rVert_2}{\lVert h_l \rVert_2}

Here, vlv_l is the steering intervention and hlh_l is the hidden state at the routed token in layer ll. This objective encourages suffixes that elicit the target response while suppressing the defense’s intervention. A pilot on 10 of the 53 behaviors tested β∈{3,10,30,100}\beta \in \lbrace 3, 10, 30, 100 \rbrace. We selected β=30\beta = 30, the strongest setting against K-Shield and joint-strongest against AlphaSteer, and applied it to both defenses.

AttackUnsteered ASRAlphaSteer ASRK-Shield ASRPaired exact McNemar p
Standard GCG (β = 0)98.1% (52/53)32.1% (17/53)13.2% (7/53)≈ 0.013
Defense-aware GCG (β = 30)—64.2% (34/53)11.3% (6/53)≈ 7 × 10⁻⁹
Either attack succeeds—73.6% (39/53)20.8% (11/53)≈ 6 × 10⁻⁸

Explicitly targeting steering doubles AlphaSteer’s ASR, whereas K-Shield’s ASR does not increase. Taking the per-behavior worst case over both attacks, K-Shield reduces ASR by 52.8 percentage points relative to AlphaSteer. This aggregate gives each defense the same combined budget of two 200-step attacks.

These results support improved robustness under the evaluated matched-budget adaptive attacks, including an objective explicitly targeting steering.

Q5. Statistical reliability and utility evaluation coverage

We would like to clarify that our main objective is to improve DSR while retaining utility close to the unsteered model, rather than to achieve superior utility. However, we agree that differences of 0.5–2 percentage points should not be interpreted as meaningful utility improvements without statistical evidence, and that 100 questions per benchmark provide limited precision. To address these concerns, we have added a paired statistical analysis of K-Shield against AlphaSteer, our main baseline, and expanded evaluation to 500 questions each on MATH-500 and GSM8K.

Statistical reliability. We compare the methods on the same prompts and report 95% confidence intervals for their score differences. We use Newcombe intervals and exact McNemar tests for individual benchmarks. For average DSR, we use a paired bootstrap within attack families (10,000 resamples) for confidence intervals and a stratified paired permutation test for significance, preserving the paper’s equal weighting of the seven families. Significance tests are Holm-corrected across 36 comparisons.

Differences below are K-Shield − AlphaSteer, in percentage points, based on the original evaluation outputs.

ModelAverage DSRXSTest complianceMATHGSM8KAlpacaEval
Llama-3.1-8B+6.3 [4.8, 7.9]0.0 [−2.5, 2.5]−1.0 [−8.4, 6.5]+6.0 [0.4, 12.5]+3.7 [0.8, 6.5]
Qwen2.5-7B+19.5 [16.3, 22.6]−1.6 [−4.3, 0.8]+1.0 [−1.3, 4.5]+3.0 [−1.9, 8.9]+5.4 [1.6, 9.1]
Gemma-2-9B+22.5 [19.7, 25.1]+22.4 [16.8, 27.8]+4.0 [−2.7, 10.6]+1.0 [−6.4, 8.5]+10.7 [7.2, 14.1]

Bold: Holm-adjusted p ≤ 0.05. Brackets show individual, unadjusted 95% confidence intervals; significance is determined by the corrected tests.

K-Shield’s average DSR improvement is statistically significant on all three models. It also significantly improves XSTest compliance on Gemma, indicating reduced over-refusal, and AlpacaEval performance on Qwen and Gemma. The MATH and GSM8K comparisons are not significant after correction. We do not rely on these small differences to claim superior utility or statistical equivalence. Full per-attack results are reported in Appendix G.2, “Significance Tests".

Expanded utility evaluation. We additionally evaluated all three models on the full MATH-500 test set and 500 GSM8K questions:

ModelMethodMATH-500GSM8K (500)
Llama-3.1-8BUnsteered48.084.6
Llama-3.1-8BAlphaSteer47.285.8
Llama-3.1-8BK-Shield47.885.6
Qwen2.5-7BUnsteered73.491.2
Qwen2.5-7BAlphaSteer71.886.6
Qwen2.5-7BK-Shield73.690.4
Gemma-2-9BUnsteered49.088.0
Gemma-2-9BAlphaSteer42.474.0
Gemma-2-9BK-Shield48.686.0

K-Shield’s observed accuracy remains within 2 percentage points of the unsteered model across all three models and both benchmarks. It is numerically close to AlphaSteer on Llama and higher on Qwen and Gemma, with the largest gaps on Gemma: +6.2 points on MATH-500 and +12.0 on GSM8K. These expanded results provide additional empirical evidence of utility retention, separate from the statistical comparisons above.

Reviewer_CP4XOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

his paper addresses the difficulty of balancing safety and normal task capability in LLM refusal steering and proposes K-Shield, a method based on local null-space constraints. The research motivation is clear, the overall method design is reasonable, and mathematical analysis together with relatively thorough ablation experiments verifies the advantage of local projection over global projection. Experimental results show that the method achieves good defense performance across multiple models and jailbreak attack benchmarks while largely preserving normal task capability. However, the theoretical novelty of the paper remains limited: the core mathematical results are mainly based on existing linear algebra theory, and the explanation of actual defense performance is still insufficient. In addition, the paper lacks adaptive attack experiments, and the analysis of the rationality of local subspace partitioning, cross-distribution generalization ability, and computational overhead needs to be strengthened. Overall, the paper has certain methodological innovation and practical value, with relatively solid experimental support, but still contains issues that require further improvement.

Critical strengths
  1. Clear research motivation and reasonable method design. The paper points out that global null-space constraints may overly restrict the information required for refusal steering, and proposes local null-space constraints to improve the trade-off between safety and normal task capability.
  2. Certain theoretical analysis is provided. Starting from target reconstruction error, the paper analyzes the limitations of global projection and provides a basis for the local projection design through mathematical derivations and constructive examples.
  3. Good experimental results and targeted ablation studies. Experiments on three LLMs and seven jailbreak attacks show that K-Shield achieves notable improvements over baselines such as AlphaSteer. In particular, controlled variables verify the additional contribution of local projection relative to independent local steering transformations.
Critical weaknesses
  1. Theoretical novelty is relatively limited. The core theorems are mainly based on classical linear least squares theory, and local null spaces, optimal transport, and input-dependent routing are also largely built on existing techniques, lacking deeper new theoretical or mechanistic findings.
  2. There is a gap between theoretical analysis and actual defense performance. The paper proves that local projection can reduce target reconstruction error under specific conditions, but does not rigorously establish the relationship between reconstruction error and final defense success rate or normal task capability.
  3. Lack of adaptive attack experiments. Current experiments mainly evaluate existing jailbreak attacks and have not fully verified robustness when attackers adaptively optimize against the K-Shield defense mechanism.
  4. Insufficient analysis of the local subspace mechanism and generalization. There is a lack of in-depth explanation of the local subspaces obtained via optimal transport partitioning, as well as stability validation across data distributions and analysis of actual inference overhead.
Questions for authors

First, the connection between theory and actual performance is insufficient. Lower reconstruction error from local projection does not necessarily imply better defense effectiveness and normal task performance, and further verification is needed. Second, adaptive attack experiments are missing. It is necessary to verify whether the method still maintains strong defense capability after attackers optimize against K-Shield. Third, the necessity of the local subspace design needs to be strengthened. The rationality of the optimal transport partitioning should be further explained, and stability and generalization ability across different data distributions should be validated.

5 comments

We thank the reviewer for their constructive feedback and respond to each point below.

W1. Theoretical novelty is limited

We agree that Theorem 1 is a classical least-squares result. Our contribution is what it lets us establish about null-space constrained steering:

  • The projector is the bottleneck, not the map. Theorem 1 shows that the attainable refusal-target error is fixed by the directions the projector retains, so no choice of steering map can recover information removed by the projector.
  • Local maps and local projectors are separable. Eq. 24 (Appendix B.1) splits the total error reduction into a benefit from independent regional maps and an effect of local projectors.
  • The projector effect can be the whole gain. Proposition 2 constructs a case where independent maps behind a global projector give no improvement (Eg=Esplit>0\mathcal{E}_g = \mathcal{E}_{split} > 0), while local projectors give exact reconstruction (Eloc=0\mathcal{E}_{loc} = 0) and leave benign activations unchanged.

The components (null-space projection, entropic OT, routing) are existing tools. The new finding is that a shared projection itself limits steering, and that localizing the protected directions removes this limit in a way that separate steering maps alone cannot.

W2 / Q1. Theory-to-performance connection

We agree that our theory does not guarantee downstream safety or utility, and the Limitations section states it. Theorem 1 and Proposition 2 provide an activation-level rationale for the method, while downstream performance requires empirical validation. This distinction also applies to null-space-constrained steering more broadly, including AlphaSteer. The formal analysis motivates the steering intervention through its effect on activations, without directly guaranteeing refusal rates or task performance.

The paper does provide the following support for the link:

  • Why the target matters. The target rkr_k (Eq. 15) is the refusal-minus-compliance mean difference. Adding this vector directly is the RV ablation, which reaches about 100% DSR on all three models (Table 1) but collapses utility (Table 2). The fitted map ΔkPkh\Delta_k P_k h is trained to reproduce this intervention on malicious inputs only, so reconstruction error measures how far the steering falls short of an intervention known to induce refusal.
  • Controlled evidence. Table 3 fixes the partition, target, α\alpha, γ\gamma, λ\lambda and steered layers, and changes only the projector. Held-out reconstruction error falls from 0.436 to 0.189 and DSR rises from 83.4% to 97.6%. Since nothing else changes, the DSR gain is attributable to the projector change.
  • Utility is governed by a different quantity. Reconstruction error is the malicious-side quantity. Benign behaviour is tied to the perturbation bound in Appendix D.4 (Eqs. 44–45). In the same comparison, relative benign intervention energy falls from 0.0040 to 0.0032 and utility moves from 65.7% to 67.0%.

This is consistent empirical evidence on one model over three seeds, not a formal guarantee. Proving that lower reconstruction error leads to more refusals would require assumptions about how the remaining layers of the model process the steered activation. These layers are nonlinear and such assumptions are difficult to verify for a full LLM, so we support this link empirically. A formal connection is an interesting question for future work.

W3 / Q2. Adaptive attacks

We have added two adaptive white-box GCG attacks on Llama-3.1-8B-Instruct in which the adversarial suffix is optimized directly against K-Shield: the defense is active during optimization and gradients are computed through its steering transformation. Under the per-behavior worst case over both attacks, K-Shield still defends 79.2% (42/53) of behaviors on which the same attack breaks the unsteered model 98.1% of the time.

Setup. Starting from the 100 AdvBench behaviors used in Table 1, a preliminary GCG run successfully attacked 53 behaviors on the unsteered model. We evaluate on these 53 so that a defense is not credited for failures of the attack itself. Attack Success Rate (ASR) is therefore conditional on this attackable subset. Suffixes were optimized separately for the unsteered model, AlphaSteer, and K-Shield with matched settings: 200 optimization steps, search width 512, top-k 256, seed 0. Using the same judge as Table 1, a behavior counts as successfully attacked if any checkpoint at steps 50, 100, 150, or 200 produces a jailbreak.

Attack 1: standard GCG through the defended model. This optimizes the target-response likelihood with K-Shield in the loop.

Attack 2: defense-aware GCG. To target the steering mechanism explicitly, we add a penalty on the relative steering magnitude:

Lattack=LCE(target)+β∣Ls∣∑l∈Ls∥vl∥2∥hl∥2\mathcal{L}_{\text{attack}} = \mathcal{L}_{\text{CE}}(\text{target}) + \frac{\beta}{|\mathcal{L}_s|}\sum_{l \in \mathcal{L}_s}\frac{\|v_l\|_2}{\|h_l\|_2}

where vlv_l is the steering intervention and hlh_l the hidden state at the routed token in layer ll. This rewards suffixes that elicit the target response while suppressing the defense's intervention. A pilot on 10 of the 53 behaviors tested β∈3,10,30,100\beta \in \\{3, 10, 30, 100\\}. We selected β=30\beta = 30, the setting that was strongest against K-Shield, so this choice favors the attacker.

AttackUnsteered ASRAlphaSteer ASRK-Shield ASRPaired exact McNemar p
Standard GCG (β = 0)98.1% (52/53)32.1% (17/53)13.2% (7/53)≈ 0.013
Defense-aware GCG (β = 30)—64.2% (34/53)11.3% (6/53)≈ 7 × 10⁻⁹
Either attack succeeds—73.6% (39/53)20.8% (11/53)≈ 6 × 10⁻⁸

Findings. (i) K-Shield's ASR is 13.2% under standard adaptive GCG and 11.3% under the defense-aware attack, and 20.8% when a behavior counts as broken if either attack succeeds. (ii) Explicitly targeting the steering mechanism doubles AlphaSteer's ASR (32.1% → 64.2%) but does not increase K-Shield's (7 vs. 6 of 53). (iii) Under the worst case over both attacks, K-Shield's ASR is 52.8 percentage points below AlphaSteer's at the same combined attack budget.

W4 / Q3. Local subspace design and rationale for the OT partition

Local subspaces are motivated by the geometric heterogeneity of benign activations: directions that need to be suppressed globally may not need to be suppressed in every region. Our ablations support this design. Increasing KK from 1 to 2 raises average DSR from 94.2% to 99.1%, while K=3K=3 achieves 99.4% (Appendix E.6). Table 3 further shows that local projectors contribute to this improvement beyond the benefit of using multiple steering maps.

Each learned region groups benign activations that can be reconstructed well by a common low-dimensional subspace (Eq. 5). Optimal transport learns these subspaces through soft, balanced assignments. These regions are therefore geometric and need not correspond to semantic categories such as instruction following or mathematical reasoning. The condition underlying our analysis is that benign activations within a region span fewer directions than benign activations collectively (ρk<ρ\rho_k < \rho). This allows a local projector to preserve directions that the global projector would remove (Section 4.1). Three empirical findings support this interpretation:

  • Local projectors retain more directions. At layer 13 of Gemma-2-9B-IT, local projectors retain 210–238 residual directions across three seeds, compared with 175 for the global projector (Table 5).
  • How the regions are formed matters. K-means partitioning followed by the same local steering achieves 69.46% average DSR, compared with 96.1% for K-subspaces and 99.37% for K-Shield (Table 4).
  • Matching inputs to regions matters. Replacing the learned routing with random routing, while keeping all projectors and maps fixed, reduces utility from 71.0% to 56.1% (Section 6.3).

Whether the learned regions also align with interpretable prompt categories is an interesting question that we leave to future work.

Regarding why we use OT rather than other partitioning or clustering algorithms, Table 4 ablates the individual design choices underlying our formulation:

  • Subspace reconstruction cost. We learn the partition using subspace reconstruction residuals (Eq. 5), consistent with the geometry of the projectors and the inference routing criterion (Eq. 13). The K-means versus K-subspaces comparison supports this choice over centroid distance.
  • Balanced assignments. Hard, unbalanced assignment required empty-cluster reinitialization in 4 of 10 restarts, and one subspace absorbed most benign activations (Appendix E.3). Balance prevents this concentration and helps ensure that each region has sufficient data to estimate its projector and steering map.
  • Soft assignments combined with balance. Balance or softness alone achieves 97.5% and 97.8% DSR, respectively; combining them achieves 99.37%. The standard deviation over ten restarts also falls from 2.8 to 0.2 percentage points relative to K-subspaces, supporting improved stability across initializations.

Together, these results motivate OT as a way to combine geometry-aware partitioning with soft, balanced assignments.

W4 / Q3. Inference overhead

We measured overhead on Llama-3.1-8B-Instruct (bf16, one A100), relative to the unsteered model.

  • Steering is a fixed per-prompt cost. The intervention is computed once per prompt during prefill, and decoding steps are not modified. Per-token generation speed is therefore unchanged by construction.
  • Measured latency. Our current implementation adds a median of 41.6 ms per prompt. This is a one-time delay before the first generated token, not a per-token cost. For a 256-token response, which takes about 7s in total, it amounts to 0.6% of the generation time.
  • Measured footprint. It adds 780.7M parameters (+9.7%) and 2.9 GiB of peak memory.

As noted in our Limitations, maintaining several projectors and maps costs more than a single global one. The numbers above also reflect an unoptimized implementation, not the cost the method requires. Our current implementation rebuilds each dense projector on every call and stores each steering map as a dense d×dd \times d matrix. Both are avoidable without changing the intervention:

  • Maps. Because the regional targets repeat one refusal vector, each fitted map has rank one (Eq. 17). It can be stored as two vectors, and applying it costs O(d)O(d).
  • Routing. ∥Pkh∥2\|P_k h\|^2 can be computed from a basis of the residual subspace without forming the projector, at O(Kdqk)O(K d q_k) per steered layer, where qkq_k is the residual dimension.

By our analysis, this factored form needs about 278M added parameters (+3.5%) and roughly 500 times fewer multiply-adds than the current implementation. Subspace learning and map fitting are one-time offline costs.

Reviewer_WHynOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

K-Shield improves the safety–utility trade-off in LLM refusal steering by replacing a single global null-space projector with local projectors learned via optimal transport, routing each input by its smallest projection residual. The paper theoretically characterizes the reconstruction advantage of local projection and validates the method across three LLMs, seven jailbreak attacks, and four utility benchmarks.

Critical strengths
  • The algorithm is well-motivated and intuitive. This paper focuses on the drawbacks of AlphaSteer and proposes reasonable improvements. I like the figures in this paper. It is intuitive and easy to understand.

  • The defense design is practical and effective. The proposed K-Shield is reasonable and achieves a good balance between jailbreak defense success rate and benign utility across various models and benchmarks.

  • The theoretical and mechanistic analysis is clear. Modeling target approximation error under null-space projections (Theorem 1) and showing how local projections preserve more steering directions than a global projection (Table 3) provides a clear perspective.

Critical weaknesses

Lack of explanatory and visualization experiments. This paper heavily follows the experimental setup of AlphaSteer [1], and the main text mainly reports final numerical results without distinctive explanatory experiments. For example, can the authors visualize how the proposed method demonstrates its superiority in the embedding space, and whether the clustering effect of different benign activations can be clearly visualized?

Lack of more recent baselines and harder reasoning benchmarks. AlphaSteer [1] is from last year; are there more recent methods available for comparison? Additionally, on more challenging mathematical benchmarks such as AIME 2024, can the proposed method still maintain its advantage?

[1] AlphaSteer: Learning Refusal Steering with Null-Space Constraints. 2025.

Questions for authors

Why are the reported results of AlphaSteer inconsistent with those in its original paper? Were there any modifications to the experimental settings?

3 comments

We thank the reviewer for their constructive feedback and respond to each point below. The visualisations requested under Weakness 1 are still in preparation and will be posted as a follow-up.

Q1: Why do the AlphaSteer results differ from the original paper?

We reproduced AlphaSteer using the authors’ released codebase and the exact experimental settings described in their paper, without modifying those settings. Two co-authors independently attempted reproduction but could not recover the published results. We have not established the source of the discrepancy and therefore report the results we obtained. All methods in our tables, including K-Shield and AlphaSteer, were evaluated using the same pipeline to ensure a consistent comparison. On Llama-3.1, where our reproduced AlphaSteer results closely match those reported in the original paper, K-Shield still improves average DSR from 93.43% to 99.37%, with Cipher DSR increasing from 68% to 100%.

W2: More recent baselines and harder reasoning benchmarks

More recent baselines. AlphaSteer first appeared on arXiv in 2025 and was published at ICLR 2026. To our knowledge it is the most recent peer-reviewed null-space-constrained refusal steering method, which makes it the direct comparator for our contribution (replacing a global null-space constraint with local ones). We are aware of the more recent "Steering Beyond the Support" (ICML 2026) [1], which trains a potential-based steering field adversarially on simulated jailbroken activations. However, we could not find a public implementation and its evaluation protocol differs from ours, so a faithful comparison was not possible within the rebuttal period. We will discuss it in Related Work and add a comparison in the final version if code becomes available. We view the two approaches as complementary: [1] targets generalisation to unseen jailbreaks through the training procedure, while K-Shield targets how the benign-preserving constraint is constructed, and the two could in principle be combined.

Harder reasoning benchmarks. We added CollegeMath (university-level problems) and ran AIME-2024 as requested. We also enlarged the existing evaluations from 100 to 500 problems each (the full MATH-500 test set and a 500-problem GSM8K subset) to tighten the estimates. CollegeMath is harder than MATH-500 for all three backbones (unsteered accuracy: 48.0% → 33.0% on Llama-3.1, 73.4% → 53.0% on Qwen2.5, 49.0% → 43.0% on Gemma-2).

ModelMATH-500GSM8K (500)CollegeMath (n=100)AIME-2024 (n=30)
Llama-3.1-8B-Instruct48.084.633.03.3
+ AlphaSteer47.285.836.06.7
+ K-Shield47.885.635.010.0
Qwen2.5-7B-Instruct73.491.253.013.3
+ AlphaSteer71.886.657.06.7
+ K-Shield73.690.453.016.7
Gemma-2-9B-IT49.088.043.00.0
+ AlphaSteer42.474.033.00.0
+ K-Shield48.686.041.00.0

K-Shield preserves reasoning accuracy on the harder benchmark. On CollegeMath, K-Shield's accuracy relative to the unsteered model is two problems higher on Llama-3.1, unchanged on Qwen2.5, and two problems lower on Gemma-2. This is the pattern expected from sampling noise (n=100, standard error ≈ 5 points) rather than from a systematic loss of reasoning ability. The 500-problem MATH and GSM8K results agree: K-Shield stays within 1 point of the unsteered model in five of six cases and within 2 points in the sixth (Gemma-2 on GSM8K).

Comparison with AlphaSteer. K-Shield's advantage over AlphaSteer is in defense success at preserved utility (Tables 1 and 2); we do not claim higher reasoning accuracy. On CollegeMath, AlphaSteer scores above K-Shield on Llama-3.1 (by 1 problem) and Qwen2.5 (by 4). In both cases it also scores above the unsteered model (by 3 and 4), so at n=100 we do not read these gaps as a reliable difference between the methods. On Gemma-2, AlphaSteer loses 10.0 points on CollegeMath, and the drop is confirmed at n=500 (−6.6 on MATH-500, −14.0 on GSM8K), while K-Shield stays within 2 points of the unsteered model on all three. Across the three backbones, K-Shield is therefore the method whose reasoning accuracy consistently remains at the unsteered level.

AIME-2024. We report AIME-2024 for completeness but draw no conclusion from it. The unsteered backbones are at or near floor (1/30, 4/30 and 0/30), so the benchmark cannot distinguish a defense that preserves utility from one that degrades it. CollegeMath, where the backbones score 33–53%, is more informative about harder reasoning.

Evaluation protocol. All generations use a fixed 4096-token budget in every condition, and responses that reach it without a final answer are scored incorrect.

[1] Steering Beyond the Support: Adversarial Training on Unsupervised Jailbroken Activation Simulation. 2026.

Reviewer_WHynOpenReview ↗

Thanks

Thanks. I would like to maintain a positive score of 3.

Dear Reviewer WHyn,

Thank you for your reply and for maintaining your positive assessment of our work. Following up on our initial response, we have now added the explanatory visualizations you requested in Appendix H of the revised paper. Appendix H.1 visualizes how benign activations are partitioned into regions and routed at each steered layer (e.g., MATH-500 and GSM8K prompts consistently share one region, while AlpacaEval prompts span several), and Appendix H.2 shows that these regions are assigned distinct steering directions. As these were not yet available when you replied, we would be grateful if you could take them into account in your final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.