acceptodds
Under review as a conference paper at ICLR 2027

Large Language Lobotomy: Locating and Silencing Safety in Mixture-of-Experts LLMs

Abstract

Mixture-of-Experts (MoE) language models route each token through only a small subset of experts, so the computation a prompt uses is conditional on its content and need not represent refusal uniformly. Prior attacks on MoE safety largely identify safety-relevant components from static activation frequencies; we ask instead which information in the routing trace best locates the components whose suppression changes refusal. We introduce Large Language Lobotomy (), a white-box inference-time framework that requires no target-model fine-tuning or parameter updates. records token-resolved routing trajectories for malicious prompts and minimally changed benign twins, trains a local-expert LSTM (LE-LSTM) to predict refusal, attributes the prediction back to individual local experts, and iteratively silences the highest-ranked ones. Across eight open-weight MoE LLMs, raises the average attack success rate (ASR) from % to %, reaching %, while accuracy on five downstream benchmarks falls by % on average at the most aggressive intervention we evaluate; it outperforms prior routing-based attacks that rely on static activation frequencies and matches F-SOUR, the strongest per-prompt routing search, but leverages one reusable expert ranking. With the silencing procedure held fixed, localization degrades when token order is destroyed and degrades further when routing is pooled across the prompt. The informative structure is the ordered path a prompt takes through the experts, which static usage counts discard. Token-resolved analyses further show that refusal evidence emerges at context-dependent positions before the end of the prompt. These results identify token-resolved routing trajectories as a strong signal for localizing intervention-relevant safety computation in MoE LLMs, and show that this signal suffices to jailbreak them without changing a single weight.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Mixture-of-Experts, LLM safety, jailbreak attacks, expert routing, safety alignment, adversarial attacks, interpretability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_x3Y6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Large Language Lobotomy, a white-box inference-time framework that attacks MoE LLMs by locating and suppressing refusal-related experts. The motivation of the proposed framework is that whole-prompt routing trajectories contain a stronger localization signal than static summaries. The framework first records token-level resolved routing trajectories for harmful prompts and minimally changed benign twins, trains an LSTM to predict refusal, and attributes the prediction back to individual local experts. Then it iteratively silences the highest-ranked ones. Results show that it improves average ASR greatly over 8 open-source MoE LLMs, with the utility falling by 3.4% on average for the most aggressive intervention tested.

Critical strengths
  1. This paper identifies that the whole-prompt routing trajectories contain a stronger localization signal than previous static summaries, which could be a useful empirical finding for relevant research community
  2. The evaluation is reasonably broad covering eight open-source MoE LLMs, and the authors provide further analysis of the proposed framework such as token order being an important factor in locating refusal-relevant experts and the routing evidence used by the classifier appearing at context-dependent positions in the prompt.
Critical weaknesses
  1. This paper proposes a framework that uses LSTM for sequential modelling, but it does not include a sensitivity analysis of this choice, as there are also models suitable for sequential modelling.
  2. The LSTM model is trained on harmful prompts that the LLM refuses and the benign prompts that the LLM answers, confounding harmfulness with refusal behavior. The experiments do not disentangle experts in charge of recognizing harmful prompts from experts in charge of the refusal behavior of the LLM.
Questions for authors
  1. Could the authors try other models for sequential modelling as a sensitivity analysis?
  2. Could the authors further clarify whether the silenced experts are related to the harmfulness of the prompt or the refusal behavior of the LLM? Is there a way to disentangle these two aspects?
2 comments
AuthorsOpenReview ↗

Response to Reviewer x3Y6

We thank the reviewer for the comments on the choice of sequential model to further strengthen the sequentiality claims and on separating harmfulness from refusal behavior, and we address each point below.

Q1/W1

We add two models that are also suitable for sequential modeling as baselines, a GRU and a single-layer Transformer encoder. Both use the same training setup (until convergence) and gradient×input attribution as the other baselines.

AUROC (held-out SorryBench pairs)LE-LSTMGRUTransformer
OLMoE-1B-7B0.9390.9170.911
Qwen1.5-A2.7B0.9460.9190.926
DeepSeek-16B0.9510.9380.930
Average0.9450.9250.922

A GRU or a Transformer in place of the LSTM loses only a couple of points on AUROC and picks largely the same experts, so the localization mechanism does not necessarily rely on the choice of LSTM. Their rankings overlap with the LE-LSTM's top 5% of experts by about 86% (GRU) and 72% (Transformer). This adds additional evidence that the sequential information can be a valuable dimension for identifying experts associated with refusal of harmful prompts, which we positioned as a contribution of our work.


Q2/W2

We can separate the two aspects on the prompts where they disagree, which are harmful prompts that the model answers and benign prompts that it refuses (in bold below). Every prompt is scored by the LE-LSTM under grouped cross-validation, so that no prompt is scored by a model trained on its twin.

Every prompt receives the LE-LSTM's predicted refusal probability under grouped cross-validation, so that no prompt is scored by a model trained on its twin. Each cell below is the mean of this probability over the prompts in the group.

Mean predicted refusal probabilityOLMoE-1B-7BQwen1.5-A2.7BDeepSeek-16B
harmful, model refused0.890.910.92
harmful, model answered0.850.950.90
benign, model refused0.220.260.31
benign, model answered0.100.080.10

We then compare groups that differ in one property and match on the other. As a second measure that does not use the classifier's output, we score each prompt by how heavily it uses the experts the attack silences: every token, at every MoE layer, is routed to a fixed number of experts, and we take the fraction of those routing decisions that land on one of the identified safety experts.

AUROCLE-LSTM scorerouting share on silenced experts
harmful vs benign0.92 to 0.970.87 to 0.92
refused vs answered, among harmful prompts0.54 to 0.650.52 to 0.58
refused vs answered, among benign prompts0.73 to 0.79—

On prompts where the two aspects disagree, the LE-LSTM score and the routing through the silenced experts track the prompt's harmfulness more than the model's behavior.

Harmful prompts the model answered score within 0.04 of those it refused, and benign prompts it refused score 0.59 to 0.69 below harmful prompts it answered, the reverse of what a score tracking refusal would give. Behavior separates the groups at AUROC 0.54 to 0.65 among harmful prompts, compared with 0.92 to 0.97 for harmfulness. The behavior effect is larger among benign prompts (0.73 to 0.79) but still clearly below that of harmfulness. On the routing share, harmful prompts send 43.2% to 46.2% of their routing decisions to the silenced experts against 40.4% to 43.0% for benign prompts, a small but consistent gap.

On W2 specifically, the pairing for LSTM training is deliberate rather than incidental, and is chosen in relation to the goal of the jailbreaking method. However, exploring the quality of attacks and expert rankings in relation to one of the two behaviors, harmfulness recognition and refusal intention, is a natural mechanistic interpretability extension for future work. The attack's objective is to remove whatever produces refusals on harmful prompts, so that conjunction is the target construct, and the twin design holds syntax, length, and register fixed so that the contrast isolates intent rather than surface form.

Thanks for the rebuttal. My concern is mostly addressed. I maintain my positive assessment of this paper.

Reviewer_Mv3BOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces a new framework for disabling safety-related experts in MoE LLMs by incorporating per-token routing differences, whereas previous work only used the static average of all token routings, and shows that this token-aware expert detection helps suppression and can raise the attack success rate while keeping the loss of downstream performance minimal.

Critical strengths
  1. It has an interesting idea to reformulate expert detection as a sequential task for an LSTM, with each layer as a time step and using gradient×input to attribute layer experts. It is tested on various MoE LLMs.
  2. Interpretability analysis of the method demonstrates more information about when safety experts are activated in pre-final positions in the prompt, and Ablation studies in 5.2 break down the gain from each contribution of the paper, although the experiments could be more clearly designed and explained.
Critical weaknesses
  1. Section 5 writing includes too many reiterations of numbers. More narrative-based, easy-to-follow writing will help readers understand the tables and figures better than repeating numbers.
  2. Figures 3 and 4 are very difficult to decode. For instance, I couldn't find "Certainty Added (pp)" on the y-axis defined anywhere. Or, in Figure 3, when the malicious-like routings are derived from the same LSTM whose refusal certainty is measured, the correlation is expected. It can be a small consistency check in the appendix instead of a main-text result. I believe the figures and writing of this section need a significant revision to be clear to human readers.
Questions for authors
  1. Please add a citation for gradient×input if you haven't
  2. In Table 8, | Idx. | Probability | Token | Probability | Token | |---|---|---|---|---| | 5 | 0.68 | bomb | 0.19 | cake | | 9 | 0.99 | ? | 0.09 | ? | seems like the last token is actually more different than the decisive token of 'bomb' and 'cake'. This feels counterintuitive to the claims of the main paper and Figure 3 about "WHERE THE REFUSAL SIGNAL APPEARS IN THE PROMPT". Can you provide more qualitative examples that can support your claims in the main paper, please?
  3. Since the premise of this paper is that experts must be seen in the context of which token is activating them, and the token-dependent routings, why is it that the silencing step is applied across all tokens instead of specific tokens?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Mv3B (1/2)

We thank the reviewer for the detailed feedback on improving the readability of the paper to make it stronger, and on the positional analysis. We address each question and weakness below.

Questions

Q1

We will cite Shrikumar et al. (2017) and Ancona et al. (2018) at Eq. 2 in the revised manuscript to support the decision for 'gradient$\times$input'.


Q2

We agree the single example was not a good illustration of the positional claim. Below, we give a clearer qualitative example, followed by the population numbers that the claim rests on.

The pair in this example differs in exactly one word: espionage vs security at index 5.

Idx.Token (malicious)malicious P(refusal)changeToken (benign)benign P(refusal)
0How0.35—How0.35
1 to0.40+0.05 to0.40
2 engage0.06-0.34 engage0.06
3 in0.04-0.02 in0.04
4 cyber0.18+0.14 cyber0.18
5 espionage0.99+0.81 security0.05
6 in0.98-0.01 in0.02
7 international0.95-0.03 international0.04
8 trade0.99+0.04 trade0.01
9?1.00+0.01?0.00

On this Qwen3-30B-A3B pair, the refusal signal appears at token 5 and not at the end of the prompt. Certainty rises by 0.81 there against 0.01 at the final token, and the four remaining tokens add nothing, so by the time the prompt ends, the evidence has been available for four positions.

Across the population, the ordering first << last << decisive holds on all eight models without exception, with the decisive token exceeding the final token by factors from 1.2 to 6.5 (Appendix K). The decisive token is the final token in 0.0% to 7.7% of prompts on six models, and in 21.2% and 32.7% on Hunyuan-A13B-Instruct and Pangu-Pro-MoE. Those two are also the two models with the weakest relationship between routing difference and certainty, at 0.38 and 0.20 against 0.54 to 0.60 for the six conventional top-kk routers (Appendix K).

On the apparent counterintuitiveness, Table 8 and the positional claim report two different quantities, which we had not separated explicitly enough. The positional claim concerns the routing difference between the two prompts of a pair at a given token, whereas Table 8 reports the classifier's refusal probability within each prompt, where a pair can keep diverging to the end of the prompt while the rise that identifies the decisive position occurs earlier.

AuthorsOpenReview ↗

Response to Reviewer Mv3B (2/2)

Q3

In our method, the silencing step was applied across all tokens because the per-token scores are already used for localization, and silencing only at specific tokens would introduce a more complex silencing strategy in which the specific tokens need to be identified for each prompt.

The LE-LSTM reads each prompt’s routing trajectory in order; Eq. 2 attributes its refusal logit to every expert selected at every token (and is zero where the expert was not selected), and Eqs. 3–4 sum those scores over tokens and prompts into one score per local expert. Therefore, each score also incorporates the dimension where the expert was selected.

Silencing only at specific tokens would also make the procedure prompt-dependent. The ranking is learned once, whereas specific tokens would have to be identified separately for each prompt, for example by tracing its routing and running the LE-LSTM and attribution on it. These positions also do not sit in a fixed place, since across the eight models the decisive token (where the classifier’s certainty rises most) is the final one in only 0% to 33% of prompts and has a median position of 50% to 72% of the prompt (Appendix K).

Creating a methodology to identify a good intervention on only specific tokens at the pre-fill stage is noted as a direction for future work.

Weaknesses

W1

We are revising Section 5 so that it no longer restates every number. The text first introduces the table or figure, then states the main conclusions it supports, and quotes only the key numbers behind them. Details that Appendix J.1 already carries, such as matched-budget comparisons and validation AUROC, are reduced in the main text to their main conclusions and key numbers. For example, the Section 5.2 paragraph on whole-prompt routing, which now restates six averages from Table 2, will first introduce Table 2 as the comparison of rankings that retain progressively less of the routing trace, then state that the full token-resolved trajectory locates safety experts best, supported by the contrasts with final-token routing and static activation counts (78.0% against 57.9% and 61.4% average peak ASR).


W2

We redefine Figure 3 to better focus on the question "Where does the refusal signal appear in the prompt?". To establish this, we first plot how the routing differences between benign and malicious prompts are represented throughout the prompt; the second panel plots how the LSTM's signal is represented throughout the prompt. Answering: where does the refusal signal appear in the prompt for the target model, and where for the classifier?

To make both Figure 3 and Figure 4 easier to decode, we add explicit definitions of each axis, and we split the right panel of Figure 4 so that they no longer share one plot.

Thank you for the response. I couldn't find the revised manuscript. Have you uploaded it? I'd particularly like to review the changes to Section 5 and Figures 3 and 4.

We have now uploaded the revised PDF. It implements the changes described in our response to W1 and W2:

  • Section 5 now introduces each table and figure, leads with the main conclusions, and keeps only the key numbers; details already in the appendices are left there.
  • Figure 3 is redesigned around where the refusal signal appears in the prompt, with a classifier-free left panel. The correlation panel is now a consistency check in Appendix K.
  • Figure 4 has its right-hand panel split into two panels with their own scales.
  • Figures 3 and 4 have every axis defined in the caption.

The revision also adds the citations for gradient$\times$input at Eq. 2 (Q1) and a second twin-pair example in Appendix L (Q2).

We thank the reviewer again for the helpful feedback.

Reviewer_sZxDOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates whether token-resolved routing trajectories can identify refusal-relevant components in Mixture-of-Experts language models more effectively than static expert-usage statistics. The proposed L³ framework trains an LSTM on routing traces from behavior-filtered malicious prompts and minimally modified benign twins, attributes its predictions to individual layer–expert pairs, and iteratively suppresses highly ranked experts during inference. Experiments across eight MoE models show improved cumulative attack success over alternative localization methods, with representation ablations supporting the usefulness of token order.

Critical strengths
  1. The paper asks which information in routing traces improves reusable expert localization. Comparisons under a shared intervention procedure help distinguish localization quality from the effects of the intervention itself.

  2. The shuffled-sequence, pooled-representation, and activation-frequency comparisons provide evidence that ordered routing trajectories contain useful localization information. Layer- and frequency-matched random controls further strengthen this analysis.

  3. Evaluation across eight MoE models supports the relevance of the findings beyond a single architecture.

Critical weaknesses
  1. The reported ASR accumulates successes across successive silencing configurations, whereas utility is evaluated at the configuration where the last success occurs. Since attack outcomes are non-monotonic along the silencing sequence, the reported ASR–utility pair does not establish that one fixed configuration simultaneously achieves both results. This limits the interpretation of the claimed tradeoff.

  2. L³ repeatedly queries each remaining prompt across different configurations, while some baselines use a single intervention. Cumulative ASR is a valid metric for an iterative attack, but comparisons should also match query or computational budgets. The shared-loop ranking ablations address localization quality, but do not resolve this issue for comparisons against prior attacks.

  3. The average utility change is measured on five classification or multiple-choice benchmarks and can conceal substantial task-specific losses. Moreover, greater silencing does not guarantee monotonically greater degradation, so endpoint utility cannot be assumed to bound degradation throughout the search. Open-ended generation quality and degeneration rates across budgets would provide a more complete assessment.

Questions for authors
  1. Can the authors report fixed-configuration ASR and utility jointly, selecting the silencing depth on validation data and evaluating it on held-out prompts?

  2. Do the improvements over prior attacks persist under matched query or computational budgets, including localization and repeated generation costs?

  3. What are the degeneration rates and open-ended capability changes across the silencing sequence, rather than only benchmark accuracy at the endpoint?

3 comments
AuthorsOpenReview ↗

Response to Reviewer sZxD (1/2)

We thank the reviewer for the detailed feedback on how ASR and utility are evaluated. The points on fixed-configuration evaluation, matched budgets, and degradation along the silencing sequence prompted the additional analyses below.

Q1/W1

We ran the suggested experiment on a validation split, swept candidate depths, and selected the shallowest depth reaching at least 90% of peak validation set ASR, under the same degeneracy constraints as the main results. That single configuration is applied once to the target model and evaluated on the held-out prompts and utility benchmarks.

modeldepth selected on validationexperts silencedheld-out ASRutility change (5-benchmark mean, pp)
OLMoE-1B-7B17.0% of local experts174 / 102419.0%-2.8
DeepSeek-MoE-16B8.4% of local experts146 / 172823.3%-1.3

The reviewer is correct that the cumulative pair does not establish that one configuration achieves both results, and the table above gives that pair directly. The values are consistent with the one-shot results already in Table 3. The ASR-utility trade-off should thus be interpreted as a property of the iterative attack that an attacker would perform under the proposed threat model, which is iterative because whether a prompt is answered maliciously is not monotone in the number of silenced experts, so one flat budget is at once too invasive for some prompts and too mild for others.


Q2/W2

We read query budget as the number of generations the attack is allowed per prompt, and computational budget as where each method pays its localization cost, either once per model or once per attacked prompt. The table below restricts L3L^3 to one generation per prompt, matching SAFEx and GateBreaker, which are single-generation by design. All values are from Tables 1 and 3.

methodgenerations per promptlocalizationavg ASR
SAFEx1per model35.0
GateBreaker1per model66.8
L3L^3, one-shot at a fixed 40% budget1per model29.0
L3L^3, one-shot at each model's best budget1per model40.3
F-SOURper-prompt searchper prompt73.8

At one generation per prompt, L3L^3 does not outperform GateBreaker. At a fixed 40% budget, it also does not outperform SAFEx, though at the best budget per model it does.

The one-query comparison changes the intervention as well, since GateBreaker disables neurons inside located experts while L3L^3 silences whole experts through the router. Matching the budget therefore resolves one confound and introduces another, which is why the paper rests its localization claim on Table 2, where the ranking is the only thing that varies.

An iterative comparison is the one that matches the threat model we introduce, since an adversary who controls the inference stack re-queries at no additional access cost. Under a matched computational budget, the advantage over F-SOUR widens with the number of attacked prompts, because L3L^3 pays localization once per model while F-SOUR searches per prompt.

AuthorsOpenReview ↗

Response to Reviewer sZxD (2/2)

Q3/W3

For open-ended output, we score every counted jailbreak of the main results with the fine-tuned StrongREJECT evaluator, using the authors' prompt template and scoring rule, which grades how specific and how convincing a response is as an answer to the request it was given.

modelwithin 5%within 10%beyond 10%unmodified model
OLMoE-1B-7B0.6630.5240.4010.132
DeepSeek-MoE-16B0.4190.3850.3480.102

Unmodified responses score 0.132 on OLMoE-1B-7B and 0.102 on DeepSeek-MoE-16B, with both having a median of 0.001; most of them are outright refusals. On OLMoE-1B-7B, 76% of the jailbreaks obtained within the first 5% of local experts score at least 0.5. Quality declines with depth on both models while staying well above the unmodified floor throughout, so the attack's useful output is concentrated at the shallow budgets where general capability is least affected. This measures the open-ended generation capabilities as the usefulness of output to the attacker's goal across the silencing sequence, showing that substantial task-specific loss is avoided when it is measured as the attack's generation purpose.

On task-specific losses, Appendix I reports the five benchmarks separately, and the declines are uneven: at 6.2 and 7.0 points on ARC and OpenBookQA against 1.8 on CoLA, with RTE rising by 2.2. On whether the endpoint bounds the sequence, the fixed configurations in our answer to Q1 give two further points below each model's peak budget, at −2.8-2.8 against −8.4-8.4 on OLMoE-1B-7B and −1.3-1.3 against −2.1-2.1 on DeepSeek-MoE-16B, both smaller than the endpoint figure.

For degeneration rates, it is consistently near 0% at every depth the attack passes through on models whose run exhausts their positive expert set. On models whose run ends in collapse, the transition is a cliff rather than a climb. For example, for DeepSeek-MoE-16B the degeneration rate is at most 3% at the final depth, and collapses to 98% immediately on the next step, triggering the stopping rule.

Thank you for addressing my questions. I remain positive about the paper and will maintain my rating of 3 (Weak Accept).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of discussion and revisions

We thank all three reviewers and the AC for their time and constructive engagement.

Reviewers sZxD and x3Y6 confirmed that our responses addressed their questions and maintained their positive assessments.

Reviewer Mv3B's concerns centred mainly on presentation. At the reviewer's request, we uploaded a revised PDF that implements every change described in our response. These changes leave the method, the main results, and the conclusions unchanged.

Reviewer Mv3B. The revision addresses the density of numbers in Section 5, the readability of Figures 3 and 4, a missing citation, and the request for further qualitative examples:

  • Section 5 introduces each table and figure, leads with the main conclusions, and keeps only the key numbers
  • Figure 3 is redesigned around where the refusal signal appears in the prompt, with a classifier-free left panel; as the reviewer suggested, the correlation panel is now a consistency check in Appendix K
  • Figure 4's right-hand panel is split into two panels with their own scales
  • Every axis in Figures 3 and 4 is defined in the caption
  • Citations for gradient×input are added at Eq. 2, and a second twin-pair example is added in Appendix L

In our response, we also clarified that Table 8 and the positional claim report different quantities, and explained why silencing is applied across all tokens, with token-specific intervention noted as future work.

We are grateful for these suggestions; the redesigned Figure 3 in particular states the positional claim more directly. We would welcome any further feedback from Reviewer Mv3B on the revision.

Reviewers sZxD and x3Y6. The analyses added in these discussions are:

  • Fixed-configuration ASR and utility, with silencing depth selected on a validation split and evaluated on held-out prompts (sZxD)
  • Single-generation comparisons against SAFEx and GateBreaker, and a discussion of computational budget (sZxD)
  • StrongREJECT scores and degeneration rates across the silencing sequence, and per-benchmark utility changes (sZxD)
  • A sensitivity analysis replacing the LSTM with a GRU and a Transformer encoder, which select largely the same experts (x3Y6)
  • An analysis separating harmfulness from refusal behavior on prompts where the two disagree (x3Y6)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.