acceptodds
Under review as a conference paper at ICLR 2027

Efficient Refusal Ablation in LLM through Optimal Transport

Abstract

Safety-aligned language models refuse harmful requests through learned refusal behaviors encoded in their internal representations. Recent activation-based jailbreaking methods circumvent these safety mechanisms by applying orthogonal projections to remove refusal directions, but these approaches treat refusal as a one-dimensional phenomenon and ignore the rich distributional structure of model activations. We introduce a principled framework based on optimal transport theory that transforms the entire distribution of harmful activations to match harmless ones. By combining PCA with closed-form Gaussian optimal transport, we achieve efficient computation in high-dimensional representation spaces while preserving essential geometric structure. Across six models (Llama-2, Llama-3.1, Qwen-2.5; 7B-32B parameters), our method achieves higher attack success rates than state-of-the-art baselines while maintaining comparable perplexity, demonstrating superior preservation of model capabilities. Critically, we discover that layer-selective intervention—applying optimal transport to 1-2 carefully chosen layers at approximately 40-60% network depth—substantially outperforms full-network interventions, revealing that refusal mechanisms may be localized rather than distributed. Our analysis provides new insights into the geometric structure of safety representations and suggests that current alignment methods may be vulnerable to distributional attacks beyond simple direction removal.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Optimal transport, refusal ablation, LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_39uxOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a representation-level jailbreaking method based on PCA and Gaussian Optimal Transport (OT). Unlike previous refusal ablation methods that primarily manipulate a single refusal direction, the proposed approach models harmful and harmless activations as Gaussian distributions and learns an affine transformation to align their means and covariance structures. PCA is introduced to reduce computational complexity, while layer-selective intervention is employed to improve attack effectiveness and preserve model utility.

Experiments on six LLMs from the Llama and Qwen families show improvements over RFA and AcT in attack success rate (ASR), with varying effects on generation quality.

Critical strengths
  1. Clear presentation. The paper is well-structured and easy to follow. The motivation, mathematical formulation, and implementation are clearly presented.

  2. Practical implementation. The proposed method combines PCA with closed-form Gaussian OT, providing a computationally tractable approach to activation manipulation. The experiments cover multiple model families and include ablations on intervention layers and PCA dimensionality.

Critical weaknesses

1. Limited methodological novelty and insufficient justification of the distributional assumption.

The main contribution is to extend one-dimensional refusal ablation to multidimensional distribution matching using Gaussian OT. However, previous studies have already investigated the multidimensional structure of safety representations, and the proposed transformation remains an affine mapping. It is therefore unclear what fundamentally new insight this formulation provides beyond existing representation-level methods.

Moreover, the Gaussian assumption is not sufficiently validated. Although the paper provides PCA visualizations, these do not establish that the high-dimensional activation distributions are adequately modeled by Gaussians, or that covariance matching is essential to the observed improvements.

2. Insufficient comparison with recent work and stronger safety defenses.

The experiments primarily compare against RFA and AcT, without adequately considering more recent research on latent-space attacks and safety robustness, such as Gao et al. (ACL 2025) and Gu et al. (ACL 2026). While the paper includes a layer-matched comparison, its scope is limited.

Furthermore, the evaluation does not investigate the proposed attack against stronger adversarially trained models, such as those obtained through CAT or MixAT. Although these are defense methods rather than directly comparable attack baselines, evaluating against such models would provide important evidence regarding the effectiveness and generalizability of the proposed attack.

Consequently, the current experiments are insufficient to demonstrate a substantial advance over recent developments in representation-level jailbreaking.

3. Limited reliability of the jailbreak evaluation.

The paper primarily relies on LlamaGuard-2 to measure attack success rates. However, LlamaGuard-2 is a relatively old safety moderation model, and more recent approaches, such as GuardReasoner (Liu et al., 2025), have demonstrated substantial improvements in safety classification.

Without cross-validation using stronger safety evaluators or human annotations, the reliability of the reported ASR remains uncertain. This is particularly important because several improvements over AcT are relatively small, and evaluator errors could potentially affect the resulting method rankings.

  • Gao, L., Geng, J., Zhang, X., Nakov, P., & Chen, X. (2025). Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models. ACL 2025.
  • Gu, T., et al. (2026). Probing the Safety Robustness of LLMs in Latent Space. ACL 2026.
  • Liu, Y., et al. (2025). GuardReasoner: Towards Reasoning-based LLM Safeguards. ICLR 2025 Workshop.
Questions for authors
  1. Novelty and Gaussian assumption: What evidence supports the Gaussian assumption for harmful and harmless activation distributions? Can the authors clarify what advantages Gaussian OT provides beyond existing multidimensional representation manipulation methods?

  2. Evaluation reliability: Can the authors validate the reported ASR using a stronger safety evaluator, such as GuardReasoner, or provide human evaluation on a subset of outputs? In particular, do the relative rankings of the compared methods remain consistent?

  3. Comparison with recent work: How does the proposed approach differ from the representation-level safety analyses in Gao et al. (2025) and Gu et al. (2026)? Can the authors clarify its distinct contribution relative to these studies and discuss its applicability to models strengthened by adversarial training?

Reviewer_iD5kOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce a framework, combining PCA with closed-form Gaussian optimal transport; they achieve efficient computation in high-dimensional representation spaces while preserving essential geometric structure. The authors achieve a higher ASR and also find that when applying the techniques to certain layers, they can achieve even better intervention.

Critical strengths

The authors propose a method that achieves higher ASR compared to the baselines. The authors also observe that when applying the intervention to certain layers, the ASR can be even higher compared to full network intervention.

Critical weaknesses

First, I am not convinced by the scenarios of attacking the representation space of the model. Specifically, why can we assume that the attacker has access to the weights of the model? Do we have specific and well-established scenarios for this?

Please also include experiments on Qwen 3, Qwen 3.5, and other models.

The writing needs some polish. Currently, in the abstract and introduction, it is hard to grasp the overview of the proposed method.

Questions for authors

See weaknesses.

In addition,

Why do the authors care about geometric structure preservation?

Reviewer_pVCDOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an optimal transport (OT) based framework that reformulates refusal ablation in LLMs as a distribution matching problem. The authors use PCA dimensionality reduction and closed-form Gaussian OT (PCA-OT1/PCA-OT2) to apply affine transformations via forward hooks on selected layers during inference, mapping the activation distributions of harmful prompts to those of harmless ones, thereby bypassing the model's safety refusal. Experiments on six models (Llama-2, Llama-3.1, Qwen-2.5) show that the method achieves significantly higher Attack Success Rate (ASR) than baselines such as RFA and AcT, while having minimal impact on model capability (similar PPL). The paper also finds that refusal mechanisms are concentrated around 40–60% of network depth, and intervening in only 1–2 layers suffices for high ASR.

Critical strengths
  1. Novel perspective: It extends refusal ablation from one-dimensional direction removal to multi-dimensional distribution matching, offering a richer geometric viewpoint and inspiring follow-up work.

  2. Solid experimental results: The method systematically outperforms baselines in ASR across multiple mainstream models with little degradation in model capability, demonstrating practical threat potential.

  3. Important observation: It reveals the localized nature of refusal mechanisms (concentrated in middle layers), providing clues for understanding the internal representation structure of safety-aligned models

Critical weaknesses

The core idea of this paper is inspiring, and it naturally raises several deeper questions that deserve investigation from different angles.

  1. Boundaries of the low-rank clustering assumption: The paper observes that activations of harmful prompts form low-rank clusters in the representation space, which makes a single distribution mapping effective. An interesting question is whether neural network training inherently leads to such low-rank bias. If so, can this bias be deliberately broken through careful design (e.g., special regularization or randomized training)? Exploring this would not only test the robustness of the method but also help us understand fundamental properties of learned representations.

  2. Two dimensions of distribution specificity remain underexplored:

  • Layer sparsity: Is it really possible to control refusal behavior with as few layers as possible? Different models (e.g., Llama vs. Qwen) show very different sensitivity along this dimension, yet the paper does not explain why. Understanding why some models require more layers could reveal different design philosophies behind safety alignment.

  • Randomness within layer representations: Even if we know which layers are critical, are the features inside those layers themselves highly random and difficult to capture stably? For instance, are Qwen's representations inherently more dispersed or harder to align than Llama's? The reasons behind these differences merit further study.

  1. Extending from attack to defense: The paper's idea naturally invites a reverse question: if a model were deliberately designed to exploit the above properties (e.g., distributing refusal features across many layers, or making intra-layer representations more random), how much would the effectiveness of such distribution-matching attacks be affected? This question is arguably more important than the attack method itself, because it can deepen our understanding of model internals and potentially inspire new defense strategies. Discussing defense is not about criticizing the method; rather, it reflects that the phenomenon is profound and deserves examination from multiple perspectives.
Questions for authors
  1. Inevitability of low-rank bias: Does neural network training naturally lead to low-rank clustering of representations? If so, can this bias be deliberately disrupted through design choices (e.g., special regularization or randomized training) to render such attacks ineffective?

  2. Measuring distribution specificity: Is there a quantifiable metric (e.g., effective rank of activation distributions, Wasserstein distance) that can predict a model's vulnerability to this type of attack? How do different training strategies (e.g., RLHF, DPO) affect this metric?

  3. Insights from a defense perspective: Can a lightweight randomization strategy (e.g., adding noise during inference, spreading refusal features across many layers during training) completely break distribution alignability without significantly degrading model performance? What would such defense attempts tell us about the structure of model internals?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.