- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Large Language Lobotomy, a white-box inference-time framework that attacks MoE LLMs by locating and suppressing refusal-related experts. The motivation of the proposed framework is that whole-prompt routing trajectories contain a stronger localization signal than static summaries. The framework first records token-level resolved routing trajectories for harmful prompts and minimally changed benign twins, trains an LSTM to predict refusal, and attributes the prediction back to individual local experts. Then it iteratively silences the highest-ranked ones. Results show that it improves average ASR greatly over 8 open-source MoE LLMs, with the utility falling by 3.4% on average for the most aggressive intervention tested.
Critical strengths
- This paper identifies that the whole-prompt routing trajectories contain a stronger localization signal than previous static summaries, which could be a useful empirical finding for relevant research community
- The evaluation is reasonably broad covering eight open-source MoE LLMs, and the authors provide further analysis of the proposed framework such as token order being an important factor in locating refusal-relevant experts and the routing evidence used by the classifier appearing at context-dependent positions in the prompt.
Critical weaknesses
- This paper proposes a framework that uses LSTM for sequential modelling, but it does not include a sensitivity analysis of this choice, as there are also models suitable for sequential modelling.
- The LSTM model is trained on harmful prompts that the LLM refuses and the benign prompts that the LLM answers, confounding harmfulness with refusal behavior. The experiments do not disentangle experts in charge of recognizing harmful prompts from experts in charge of the refusal behavior of the LLM.
Questions for authors
- Could the authors try other models for sequential modelling as a sensitivity analysis?
- Could the authors further clarify whether the silenced experts are related to the harmfulness of the prompt or the refusal behavior of the LLM? Is there a way to disentangle these two aspects?
2 comments
Thanks for the rebuttal. My concern is mostly addressed. I maintain my positive assessment of this paper.
Response to Reviewer x3Y6
We thank the reviewer for the comments on the choice of sequential model to further strengthen the sequentiality claims and on separating harmfulness from refusal behavior, and we address each point below.
Q1/W1
We add two models that are also suitable for sequential modeling as baselines, a GRU and a single-layer Transformer encoder. Both use the same training setup (until convergence) and gradient×input attribution as the other baselines.
A GRU or a Transformer in place of the LSTM loses only a couple of points on AUROC and picks largely the same experts, so the localization mechanism does not necessarily rely on the choice of LSTM. Their rankings overlap with the LE-LSTM's top 5% of experts by about 86% (GRU) and 72% (Transformer). This adds additional evidence that the sequential information can be a valuable dimension for identifying experts associated with refusal of harmful prompts, which we positioned as a contribution of our work.
Q2/W2
We can separate the two aspects on the prompts where they disagree, which are harmful prompts that the model answers and benign prompts that it refuses (in bold below). Every prompt is scored by the LE-LSTM under grouped cross-validation, so that no prompt is scored by a model trained on its twin.
Every prompt receives the LE-LSTM's predicted refusal probability under grouped cross-validation, so that no prompt is scored by a model trained on its twin. Each cell below is the mean of this probability over the prompts in the group.
We then compare groups that differ in one property and match on the other. As a second measure that does not use the classifier's output, we score each prompt by how heavily it uses the experts the attack silences: every token, at every MoE layer, is routed to a fixed number of experts, and we take the fraction of those routing decisions that land on one of the identified safety experts.
On prompts where the two aspects disagree, the LE-LSTM score and the routing through the silenced experts track the prompt's harmfulness more than the model's behavior.
Harmful prompts the model answered score within 0.04 of those it refused, and benign prompts it refused score 0.59 to 0.69 below harmful prompts it answered, the reverse of what a score tracking refusal would give. Behavior separates the groups at AUROC 0.54 to 0.65 among harmful prompts, compared with 0.92 to 0.97 for harmfulness. The behavior effect is larger among benign prompts (0.73 to 0.79) but still clearly below that of harmfulness. On the routing share, harmful prompts send 43.2% to 46.2% of their routing decisions to the silenced experts against 40.4% to 43.0% for benign prompts, a small but consistent gap.
On W2 specifically, the pairing for LSTM training is deliberate rather than incidental, and is chosen in relation to the goal of the jailbreaking method. However, exploring the quality of attacks and expert rankings in relation to one of the two behaviors, harmfulness recognition and refusal intention, is a natural mechanistic interpretability extension for future work. The attack's objective is to remove whatever produces refusals on harmful prompts, so that conjunction is the target construct, and the twin design holds syntax, length, and register fixed so that the contrast isolates intent rather than surface form.