acceptodds
Under review as a conference paper at ICLR 2027

Attribution-guided Pruning for Circuit Discovery and Targeted Correction in Language Models

Abstract

The parameters of Language Models (LM) encode knowledge acquired during training and jointly support many behaviors at inference. Identifying the components associated with a particular behavior provides insight into its implementation, while modifying them offers targeted control but can affect general capabilities. Circuit discovery and model correction address these respective goals. We connect them through attribution-guided pruning, which uses attribution methods to score neurons and individual weights. For circuit discovery, we retain highly attributed components. For correction, we compare component attribution on behavior-eliciting prompts and diverse reference text, then prune components more strongly associated with the target behavior. We evaluate LRP and one-step gradient saliency, with Wanda as a pruning baseline where applicable. On Indirect Object Identification, LRP yields the sparsest structured circuits across the three tested checkpoints and remains competitive for individual-weight pruning. In held-out evaluations across nine LM checkpoints from 125M to 8B parameters, attribution-guided pruning reduces the mean toxicity score by 43.2-77.6% on prompts whose unpruned continuations exhibit the target behavior, defined in this study by high toxicity, insult, and obscenity scores. Across three models, it improves response uniqueness by 39.8-86.6% on repetition-eliciting prompts. All selected corrections retain at least 90% of their unpruned HellaSwag and ARC-Challenge normalized accuracy and keep WikiText2 perplexity within baseline. Both LRP and gradient saliency support targeted correction, with gradients yielding the best selected operating point for several models. The results show that the same parameter-level pruning framework can yield sparse task circuits and suppress selected generation behaviors.

Primary area:
interpretability and explainable AI
Keywords:
Interpretability, Circuit Discovery, Attribution Methods, Targeted Model Correction, Pruning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kDDKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes an attribution-guided pruning framework that unifies circuit discovery and targeted behavioral correction in language models. The method uses Layer-wise Relevance Propagation (LRP) and gradient saliency to identify important neurons and weights. For circuit discovery, highly attributed components are retained to extract sparse task-specific subnetworks. For behavioral correction, attribution scores on undesired behavior prompts are contrasted with those on general text to identify and prune behavior-associated parameters. Experiments on IOI, toxicity, and repetition tasks show that the method can extract sparse circuits and suppress targeted behaviors while largely preserving performance on selected general-purpose benchmarks.

Critical strengths
  1. The paper connects circuit discovery and targeted behavioral correction through a unified parameter-level attribution and pruning framework, providing a practical approach to analyzing and intervening on model behaviors without fine-tuning.
  2. The authors evaluate multiple attribution methods, pruning granularities, and model checkpoints. They also assess behavioral changes alongside general utility and conduct necessity and sufficiency experiments to investigate the functional relevance of the identified components.
Critical weaknesses
  1. The methodological novelty remains somewhat limited, as the framework primarily combines existing attribution methods with parameter pruning. The distinction from existing attribution-based pruning and circuit discovery approaches is not sufficiently established.
  2. The behavioral correction results rely on narrowly selected prompts and limited evaluation metrics. In particular, some toxicity evaluation sets are very small, while general capability preservation is assessed using only WikiText2, HellaSwag, and ARC-Challenge. The generalization of the method to broader prompt distributions and other capabilities remains unclear.
Questions for authors
  1. The authors should provide more direct comparisons with existing circuit discovery and attribution-based pruning methods under matched pruning budgets and evaluation settings. This would help clarify the methodological novelty and establish whether the proposed framework offers consistent advantages over existing approaches.
  2. The authors should evaluate behavioral correction on larger, independently held-out datasets using additional behavioral and utility metrics. They should also test the sensitivity of the results to different reference datasets and behavior definitions to better establish the robustness and generalizability of the proposed method.
4 comments
AuthorsOpenReview ↗

Clarification of distinction of our work from other circuit discovery methods, and new experiments on math circuits for arithmetic and GSM8K

Thank you for the constructive questions. We clarify both the direct comparison and the scope of our generalization evidence below.

Comparison with existing methods. Many circuit-discovery methods construct circuits over an activation-level computation graph. Our question is different: which learned parameters can be retained or removed to obtain a fixed subnetwork for the behavior? We therefore evaluate parameter interventions at both neuron and individual-weight granularity. The individual-weight scores arise directly from LRP's relevance decomposition to connections or from gradients with respect to weights; they are not formed by multiplying a neuron score by weight values. An activation-graph node or edge has no one-to-one correspondence with a scalar parameter, so matching ACDC or EAP to an individual-weight pruning budget would compare different intervention units.

We therefore compare with RelP at neuron granularity, where both methods share the same intervention unit and propagate relevance, although RelP introduces LRP into an activation-patching graph. On the same six seeded IOI sets and matched top-k budgets, 18.4-45.1% of the retained neurons overlap at 5% retention, versus 5% expected from two random top-5% sets. Thus, their overlap is well above the random expectation, but they do not produce the same circuit. This is the intended purpose of the comparison. We test intervention faithfulness separately within our parameter-space framework by comparing LRP, gradient saliency, and Wanda at matched pruning budgets using LeRF and MoRF. We will make these two comparison questions explicit in the revision.

Circuit discovery beyond IOI. We added exact arithmetic and GSM8K using google/gemma-3-4b-it and Qwen/Qwen3-4B. The arithmetic benchmark is synthetically generated and balances addition, subtraction, multiplication, and exact-integer division; we evaluate exact final-answer accuracy. GSM8K contains natural-language word problems that usually require several reasoning steps. It is the harder task in this pair, while arithmetic gives us a controlled setting between IOI and multi-step reasoning.

For each completed model-task pair and attribution method, we use three seeded sets of 128 correctly solved training examples and attribute the extracted final-answer tokens. Evaluation uses separate, fixed sets of 500 arithmetic questions and 660 randomly selected GSM8K test questions. We apply unstructured LeRF independently to all weights of each MLP projection type, using matched 0-100% pruning rates in 5% steps. The complete curves with seed-wise min-max bands are the most informative presentation because they show both gradual degradation and sharp transitions. Due to the response-space limit, we summarize them below. Each entry is the largest removal rate before mean accuracy first falls below 95%/90% of its unpruned value. D/G/U denote down/gate/up projections.

Model/task (unpruned)LRP 95% (D/G/U)LRP 90% (D/G/U)Grad 95% (D/G/U)Grad 90% (D/G/U)
Gemma Arithmetic (93.6%)15/30/2020/45/2510/5/010/15/0
Qwen Arithmetic (98.6%)70/10/7075/10/8050/0/4555/0/50
Gemma GSM8K (88.5%)5/35/4015/45/5010/15/1010/25/10
Qwen GSM8K (90.8%)55/10/5555/10/6020/0/2030/0/30

As in the submitted IOI experiments, LRP supports sparser retained circuits within the targeted projection family than Gradient at matched budgets: across the 12 comparisons, its 95%/90%-accuracy removal limits are larger in 11/12 and 12/12 model-task-projection settings, respectively. The math circuits are less sparse than the IOI circuits, consistent with arithmetic and especially multi-step GSM8K relying on more distributed parameter support. Nevertheless, the results show substantial removable fractions beyond IOI, while also showing that circuit compactness depends on the task, model, and projection type. We will provide all curves in the revision/supplement rather than only selected layers.

Second-dataset correction evaluation. Please see our response to Reviewer 85fz for the reserved ToxiGen protocol and fixed-setting results. This study tests whether the framework can be adapted to a second toxicity dataset; it does not claim that one mask transfers across datasets or models.

Please also see our response to Reviewer EyoC for the G, -T, and G-T ablation and the complete WinoGrande/PIQA evaluation. The contrast is best in five of six model-method cases, but this does not test replacement of C4 by a different general-reference corpus. We will add reference-corpus sensitivity as a remaining limitation.

AuthorsOpenReview ↗

Additional message to Reviewer kDDK: Weight amplification

We additionally tested whether our framework can support a different parameter intervention: strengthening selected behavior-associated weights rather than setting them to zero. This is a preliminary experiment, not part of the submitted claims. On google/gemma-3-4b-it and Qwen/Qwen3-4B, we use LRP or Gradient to score weights from 128 correctly solved training examples for exact arithmetic and GSM8K. We multiply selected weights from down_proj, gate_proj, or up_proj by coefficients in {1.01, 1.1, 1.2, 1.5, 2.0} and select separately for each of three seeds on the development set. We retain only settings satisfying the strict utility constraints: PPL <= 1.10x baseline and HellaSwag/ARC-Challenge >= 95% of baseline. The table reports development accuracy; layers and coefficients are ordered by seed 0/1/2.

Model/taskMethodOriginal -> mean selectedGainLayer by seedCoefficient by seed
Gemma ArithmeticLRP92.58 -> 96.48+3.91 ppdown/down/down1.2/1.2/1.2
Grad92.58 -> 95.51+2.93 ppdown/down/-2.0/1.5/-
Gemma GSM8KLRP88.67 -> 92.06+3.39 ppgate/gate/gate1.1/1.2/1.2
Grad88.67 -> 91.93+3.26 ppup/up/down1.2/1.1/1.5
Qwen ArithmeticLRP99.22 -> 100.00+0.78 ppgate/gate/up2.0/1.5/2.0
Grad99.22 -> 100.00+0.78 ppgate/up/gate1.2/2.0/1.1
Qwen GSM8KLRP94.53 -> 95.31+0.78 ppgate/up/gate2.0/2.0/1.2
Grad94.53 -> 95.44+0.91 ppdown/gate/gate1.1/2.0/1.2

All 23 available seed-specific selections satisfy the strict utility criteria; one Gemma Arithmetic Gradient seed is incomplete and is not included. Gemma shows the clearest gains, while Qwen starts near the ceiling and therefore has less room to improve. These development-set results provide initial evidence that our framework can guide amplification as well as removal on two math behaviors; they do not yet establish general capability improvement. Still, every reported model-task-method mean improves over its unamplified baseline while satisfying the strict utility criteria. The gains are modest, especially for Qwen whose initial accuracy is already 94.53-99.22%, but their consistent direction is an encouraging sign that targeted amplification merits broader study. We plan to extend this direction to other model families, including Llama, and tasks with lower initial accuracy.

AuthorsOpenReview ↗

General response to reviewer kDDK and AC

Thank you again for asking for broader circuit-discovery evidence and clearer comparisons. During the rebuttal, we clarified why scalar-parameter budgets do not directly match activation-graph edges and made the purpose of the shared-neuron RelP comparison explicit. We also completed 0-100% LeRF curves for exact arithmetic and GSM8K on Gemma and Qwen. LRP permits more removal than Gradient in 11/12 projection cases at 95% accuracy retention and 12/12 at 90%. In addition, we added the preliminary weight-amplification experiment above and a held-out second-dataset correction study. We believe these additions address the main concerns about evaluation beyond IOI, comparison scope, and broader applicability.

If any concern remains, we would be grateful for your feedback and the opportunity to discuss it further.

Thank you for the detailed response and additional experiments.

I appreciate the clarification of the distinction between parameter-level circuit discovery and activation-graph methods. The new experiments are particularly helpful and strengthen the evidence that the framework generalizes beyond IOI.

However, the RelP comparison mainly measures neuron overlap rather than comparative circuit faithfulness under matched pruning budgets. Thus, my concern regarding comparisons with existing methods remains unaddressed and I will maintain the current rating.

Reviewer_EyoCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Attribution-guided Pruning, which uses LRP and Gradient Saliency to rank and prune model parameters for Circuit Discovery, Targeted Correction, and Model Compression. For Targeted Correction, the method uses differential attribution to identify parameters for pruning, aiming to suppress undesired behaviors while preserving general model capabilities.

Critical strengths
  1. The paper evaluates Circuit Discovery, Targeted Correction, and Model Compression across different models, providing relatively broad experimental coverage.

  2. The authors analyze the effectiveness of the proposed framework through LeRF, MoRF, different pruning configurations, and multi-seed experiments, while considering the trade-off between behavioral correction and general capability preservation.

Critical weaknesses

1. Limited novelty and unclear distinctions from prior work.

The paper mainly combines existing LRP, Gradient Saliency, and Parameter Pruning techniques without substantial algorithmic improvements. Prior works, including Pruning by Explaining Revisited [1], Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition [2], and On Localizing and Deleting Toxic Memories in Large Language Models [3], have already explored attribution-guided pruning, circuit discovery, and the localization and removal of toxicity-related model components. The authors should clarify the fundamental differences between their method and these existing approaches, as well as its independent contribution.

2. Insufficient motivation and ablation studies for the core differential attribution strategy.

The authors directly subtract Undesired Attribution from General Attribution to select parameters for pruning. However, the motivation for this simple subtraction is not sufficiently justified. The paper does not explain why these attribution scores can be directly compared, why equal weighting is appropriate, or how their difference reliably captures behavior-specific importance. Moreover, the attribution target is not directly related to Toxicity or Repetition, and the two reference datasets have different distributions, making this strategy appear largely heuristic. The paper also lacks key ablations, such as General-only, Undesired-only, and different weighting coefficients, making it difficult to establish the necessity and independent benefits of the proposed differential attribution.

3. Limited generalization evaluation for Circuit Discovery and unclear computational efficiency.

Circuit Discovery is evaluated only on the IOI task, without experiments on other types of tasks or more complex reasoning settings. It is therefore unclear whether the method generalizes to more complex circuits. Furthermore, the framework requires recomputing attribution for different tasks and searching over pruning ratios, layer types, and granularities to determine the final configuration. The authors should clarify the computational cost of this search procedure and demonstrate its advantages over existing methods in terms of circuit quality or efficiency.

References

[1] Hatefi et al. (2024). Pruning by Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers. ECCV 2024 eXCV Workshop. https://arxiv.org/abs/2408.12568

[2] Hsu et al. (2025). Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition. ICLR 2025. https://proceedings.iclr.cc/paper_files/paper/2025/hash/916ee60e315531d6b3954af8a8dc3437-Abstract-Conference.html

[3] Das et al. (2025). On Localizing and Deleting Toxic Memories in Large Language Models. Findings of NAACL 2025. https://aclanthology.org/2025.findings-naacl.129/

Questions for authors

see weaknesses

2 comments
AuthorsOpenReview ↗

Clarification of the distinctions from related work, attribution target, ablation of `G`, `G-T`, and `-T`, plus full Winogrande and PIQA evluations

Thank you for identifying where the distinction from prior work and the differential score require a clearer explanation. Our contribution is a unified parameter-level framework in which attribution-guided pruning extracts sparse behavioral circuits and suppresses selected generation behaviors.

Relation to prior work. We already cite and discuss Pruning by Explaining Revisited in Related Work, but agree that the distinction was not prominent enough. Its reported experiments use attribution-guided structured compression of CNNs and vision transformers while preserving classification. Our experiments study behavior-conditioned circuits and correction in autoregressive language models, including interventions on individual scalar weights as well as structured components. We will state this relationship more directly.

CD-T recursively decomposes transformer computations to extract task circuits, including attention-head components. Our parameter scores can be aggregated by head; they additionally support direct interventions on learned parameter groups and individual weights, producing a fixed pruned subnetwork used for correction. Das et al. use causal tracing to identify MLP layers and MEMIT-style updates or deletion for toxic generation in GPT-2-XL. We instead rank neurons or scalar weights, zero selected components without optimizing a weight update, and test multiple model families and target behaviors under fixed utility constraints. We will add both papers and these distinctions to Related Work.

At weight granularity, LRP does not obtain a scalar-weight score by multiplying a neuron score by weight magnitude. It decomposes propagated relevance to each connection using its input and weight. Gradient saliency directly differentiates the selected output with respect to each weight.

Why use differential attribution? Both terms use the same model, components, attribution rule and target definition, and each is averaged over its sample count. Their distributions intentionally differ: target examples elicit the selected behavior, while C4 represents broader language modeling. Under lowest-score pruning, G-T prioritizes components whose target attribution is large relative to their reference attribution. It does not prove a pure separation of behavioral mechanisms; we will clarify this interpretation.

The attributed output depends on the behavior studied. IOI attributes the target answer prediction; the submitted correction experiments attribute a next-token prediction conditioned on the behavior example up to that position. For ToxiGen, we average the logits assigned to the observed continuation tokens, each conditioned on the prompt and preceding continuation. We attribute model predictions, not external evaluator scores.

We completed our ablation comparing G, -T, and G-T. Our pruning procedure always removes the lowest score; for -T, its lowest values are precisely the components with the largest target attribution. For each arm and tier, we choose the setting by mean toxicity on the attribution data across five seeds and report its held-out reduction. Every entry satisfies the stated seed-averaged utility constraints.

Model/methodTierG-TG-T
OPT/LRPStrict / moderate49.6 / 49.647.2 / 51.160.8 / 71.2
OPT/GradientStrict / moderate47.0 / 47.036.3 / 41.656.1 / 60.2
TinyLlama/LRPStrict / moderate27.9 / 27.943.6 / 55.238.6 / 39.8
TinyLlama/GradientStrict / moderate35.6 / 55.243.2 / 45.369.7 / 72.7
Qwen2-1.5B/LRPStrict / moderate15.6 / 24.613.3 / 66.862.2 / 75.9
Qwen2-1.5B/GradientStrict / moderate20.8 / 34.048.0 / 56.762.7 / 74.4

G-T is best in five of six model-method cases under both tiers, while -T is better for TinyLlama/LRP. These are three points in the weighted family alpha G - beta T: (1,0), (0,1), and (1,1). The reviewer's question about other coefficients is valuable. We use unit weights to avoid another tuned parameter in an already broad search; coefficient sensitivity is useful future work.

Additional utility and cost. We evaluated every Table 2 intervention on the complete WinoGrande and PIQA data splits. PIQA remains within 1.28 percentage points of baseline for all 12 model-behavior settings. WinoGrande remains within 3 points for 9/12 settings; the largest drop is 5.52 points for Qwen3-4B-Instruct. These tasks were not used to reselect operating points. Our measured adaptation cost and the absence of a universally transferable pruning setting are reported in our response to Reviewer 85fz. We also added new experiments on ToxiGen correction (Reviewer 85fz) and on arithmetic and GSM8K as circuit-discovery behaviors beyond IOI (Reviewer kDDK).

AuthorsOpenReview ↗

General response to reviewer EyoC and AC

Thank you again for helping us identify where the distinctions and motivation needed to be clearer. During the rebuttal, we clarified the differences from CD-T, toxic-memory editing, and prior attribution-guided pruning; completed the G, -T, and G-T ablation on three models; and added full WinoGrande and PIQA evaluations. We also added held-out ToxiGen correction and arithmetic and GSM8K circuit discovery. We believe these additions address the requested contrastive-score ablation, broader utility evaluation, and much of the concern about behavior and dataset coverage.

If any part remains unclear or unaddressed, we would greatly appreciate your feedback and further discussion.

Reviewer_85fzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses attribution-guided pruning for circuit discovery and model correction, with LRP or gradient saliency scoring neurons and individual weights. In the IOI experiments, LRP gives the sparsest structured circuits among the methods. For model correction, the authors contrast attribution scores from behaviour-eliciting prompts with those from diverse reference text, then prune based on this difference to reduce held-out toxicity and repetition while largely preserving utility.

Critical strengths
  1. The setting is clear. The comparison between LRP and gradients keeps the model, layer type, pruning granularity, and pruning count fixed. With these factors controlled, LRP reduces held-out toxicity more than gradients in 112 of 114 comparisons under the strict utility tier and 178 of 185 under the moderate tier.

  2. Attribution is cheap. The method requires only one forward and backward pass per reference sample, with no fine-tuning or inference-time steering. The resulting parameter mask stays fixed, so the pruned model can be evaluated without recomputing attribution scores.

  3. The evaluation protocol is careful. The correction experiments use five-fold held-out evaluation, with entire prompt families held out together for repetition. Operating points are chosen under predefined utility tiers without consulting held-out scores. The final settings are then checked on the full HellaSwag and ARC-Challenge sets across nine checkpoints, including models with up to 8B parameters.

Critical weaknesses
  1. The method is costly to adapt because its pruning settings must be tuned separately for each model and behaviour, requiring repeated generation and evaluation across a new sweep. Appendix H reports only the time for one attribution pass, so this adaptation cost remains unreported. Toxicity is tested only on held-out prompts from the same filtered pool, leaving it unclear whether the chosen settings transfer to other datasets or mixed inputs, or can correct multiple behaviours together.

  2. The target behaviour is rare, so the evaluation sets are small. Even on RealToxicityPrompts, Instruct models produce continuations that pass the toxicity, insult, and obscene filters for only 0.06–0.15% of prompts, leaving 12–29 held-out prompts per fold (Table 6). This makes the reported reductions unstable, and pruning affects all inputs to fix a behaviour that rarely occurs.

  3. No generated examples are shown. The correction results rely only on classifier scores, so it is unclear whether the lower toxicity comes from actually removing the behaviour or from degraded generation quality, especially since perplexity rises by up to 1.20×.

Questions for authors
  1. What is the total compute of the correction sweep for one model and one behaviour? Can a setting selected on one model or behaviour be transferred to another without repeating the sweep?

  2. Could the authors evaluate the pruned models on established safety datasets with larger prompt sets, such as ToxiGen or HarmBench, to show that the correction holds beyond the filtered RealToxicityPrompts subset?

  3. Could the authors show generated examples before and after pruning, on both eliciting and ordinary prompts, to clarify whether the lower toxicity reflects removal of the behavior or degraded generation?

4 comments
AuthorsOpenReview ↗

Report of adaptation cost, new ToxiGen experiments on three models, and illustrative generated examples.

Thank you for the careful review. We address adaptation cost, evaluation beyond the filtered RealToxicityPrompts pool, and generated examples below.

Adaptation cost. Attribution is computed once per model, behavior, method, and data split, then reused across layers and counts. Each candidate requires generation and utility evaluation. On Gemma-3-4B-Instruct, attributing 128 ToxiGen prompts takes about two minutes per method. Running the three random-seed jobs in parallel, the full 18-curve, 16-count sweep took 6.26 hours wall-clock on one RTX PRO 6000 Blackwell; additional devices can parallelize the curves further. No setting is universally strongest, but unstructured up_proj pruning frequently appears among the strongest settings.

ToxiGen evaluation. We evaluate Gemma-3-4B-Instruct, Qwen3-4B-Instruct, and Llama-3-8B-Instruct on released ToxiGen prompts. For each model, we retain non-empty continuations with hateful probability at least 0.9 under the pinned tomh/toxigen_roberta classifier. For each of three seeds, 128 prompts are used for attribution and 128 for setting selection. Final evaluation regenerates on a separate 256-prompt set and uses full WikiText2, HellaSwag, ARC-Challenge, WinoGrande, and PIQA evaluations.

All interventions use U, unstructured weight pruning. Each row fixes one layer and count across seeds 0/1/2 using development results only. S/M denote strict/moderate. Scores below are final held-out mean ± SD; B→P is baseline to pruned, and H/A are full-set normalized accuracies. Pass counts seeds satisfying the PPL/HellaSwag/ARC tier. up, down, and gate denote up_proj, down_proj, and gate_proj.

ModelMethod/tierFixed layer; countHateful B→PReductionPPL B→PH/A B→PPass
Qwen3-4B-InstructGrad/Sup; 50k.958→.688±.02628.1%13.66→14.78.684/.538→.686/.5233/3
LRP/Mup; 7k.958→.710±.02325.8%13.66→15.99.684/.538→.660/.5363/3
Grad/Mup; 100k.958→.564±.02041.1%13.66→15.65.684/.538→.681/.5183/3
Gemma-3-4B-InstructGrad/Sdown; 1k.899→.630±.03929.9%17.38→18.26.742/.572→.728/.5492/3
LRP/Mup; 2k.899→.172±.00980.9%17.38→19.15.742/.572→.705/.5111/3
Grad/Mup; 20k.899→.179±.01680.1%17.38→19.52.742/.572→.713/.5333/3
Llama-3-8B-InstructGrad/Sup; 500k.974→.884±.0149.2%8.29→8.68.756/.560→.759/.5250/3
LRP/Sgate; 15k.974→.904±.0087.1%8.29→8.76.756/.560→.750/.5653/3
Grad/Mup; 1m.974→.844±.00613.3%8.29→8.91.756/.560→.759/.5133/3
LRP/Mup; 20k.974→.827±.01815.1%8.29→8.56.756/.560→.745/.4870/3

Qwen confirms 25.8-41.1% suppression with 3/3 passes. Gemma Grad/M gives the largest fully passing reduction, 80.1%. Gemma LRP/M reaches 80.9% but passes in 1/3 seeds. Llama has smaller fully passing reductions with LRP/S (7.1%) and Grad/M (13.3%). The other failures cross the ARC threshold, rather than showing general collapse. Strict LRP selects zero pruning on Qwen and Gemma and is omitted. Thus, the framework adapts to a second toxicity dataset across three model families; this is not transfer of one mask or general detoxification.

Generated examples. We show two eliciting examples and two ordinary-prompt controls. RTP scores use Detoxify; the others use hateful probabilities from tomh/toxigen_roberta. These examples are illustrative; aggregate evidence is above. We will include further examples in the revision.

Dataset/model (setting)Prompt excerptUnpruned →\rightarrow pruned continuationClassifier score
RTP/Llama (Grad/M; struct. gate, 5k)“This movie is about a fucked up guy who keeps some kid chained up...”“does all sorts of sick shit to him...” →\rightarrow “is trying to make him into a ‘perfect’ boy. The kid is trying to escape...”.993 →\rightarrow .006
ToxiGen/Qwen (Grad/M; unstruct. up, 100k)“everyone with a mental disability is violent”“mentally disabled people are a threat to society...” →\rightarrow “the mentally disabled are not a threat to society...”.992 →\rightarrow .100
Ordinary/Qwen (Grad/S; unstruct. up, 50k)“Renewable energy technologies have become increasingly affordable...”“the cost of solar and wind power has dropped... cleaner... sources...” →\rightarrow “the cost... has dropped... integration... into the existing power grid...”.00062 →\rightarrow .00062
Ordinary/Gemma (LRP/M; unstruct. up, 2k)Same prompt“competitive with traditional fossil fuels... intermittency... challenge...” →\rightarrow “deployment... driven by climate concerns, energy security, and economic development...”.00060 →\rightarrow .00062
AuthorsOpenReview ↗

General response to reviewer 85fz and AC

Thank you again for the careful review. We wanted to briefly summarize the new evidence most relevant to your concerns:

During the rebuttal, we measured the complete adaptation cost, added a held-out ToxiGen study on Gemma, Qwen, and Llama with five full utility benchmarks, and reported fixed settings that did and did not satisfy every utility constraint. We separately included generated examples and ordinary-prompt controls. The results show that our framework can be adapted to a second toxicity dataset while satisfying the utility constraints for several settings. We believe this addresses the main questions about adaptation cost, evaluation beyond the original filtered data, and qualitative behavior.

If any concern remains, we would be grateful if you could let us know and engage with us in the discussion.

Reviewer_85fzOpenReview ↗

answer to authors' comments

The authors addressed some of my concerns and some remain:

  1. Adaptation cost. The timing is for one specific, high-end GPU. No normalised cost (e.g. GPU-hours) is given, making it hard to compare across hardware or to other methods.
  2. Transfer. Though 'up_proj' frequently appears, but this falls short of demonstrating transferability.
AuthorsOpenReview ↗

answer to reviewer's comment

Thank you for your newly brought up concerns.

  1. We like to point the reviewer to appendix H, where we have stated the attribution cost in our submission (with apologies for not placing it more dominantly due to lack of space). We have stated the GPU hours in our initial response (at https://openreview.net/forum?id=dw60kMjybk&noteId=ygJP0TSxTl) which is 6.26 GPU-hours for the ToxiGen task on Gemma-3-4B-Instruct. We additionally report that we use the same amount of operating points (x288) and curves (x18) to compute for each model, it took 9.34 GU hours for Qwen3-4B and 7.39 GPU-hours for Llama-3-8B-Instruct on the same GPU. This information allows to compare across hardware and to other methods. Please respond to us if this point is not resolved already by this. 
  2. ‘up_proj’ (similar to ‘fc1’ in OPT model) is a common layer in almost all transformers and is observed in 9/12 settings from Table 2, 6/12 settings from Table 12, and 7/10 settings from the table in our initial comment. Both of these observations make it one of the most suitable targets for model interventions. Which other transferability does the reviewer have in mind? We look forward to further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.