acceptodds
Under review as a conference paper at ICLR 2027

WASD: Locating Critical Neurons as Sufficient Conditions for Explaining and Controlling LLM Behavior

Abstract

Precise behavioral control of large language models (LLMs) is critical for complex applications. However, existing methods often require task-specific training, lack natural language controllability, or compromise semantic coherence.To bridge this gap, we propose WASD (unWeaving Actionable Sufficient Directives), a mechanistic interpretability framework that explains model behavior by identifying intervention-validated sufficient neural conditions for token generation. WASD uses Circuit Tracer as an attribution and activation extractor, but does not directly treat high-attribution neurons as explanations. Instead, it constructs neuron-activation predicates under prompt perturbations and searches for a compact, pruned rule set whose intervention preserves the target behavior. Experiments on SST-2, CounterFact, and IOI across four models yield 94.83%–99.48% mean fidelity for compact rules at a target threshold of 0.95; protocol differences limit direct comparisons with attribution baselines. Neutral-prefix evaluations at the same threshold show lower mean instability in all twelve settings. Cross-lingual completion and QA on Gemma-2-2B and Qwen3-8B demonstrate language control and task-dependent trade-offs between accuracy and Chinese explanation rate.

Primary area:
interpretability and explainable AI
Keywords:
curcuit tracing, rule-based explanation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4pudOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes WASD, a method that finds a small set of neurons that is sufficient to keep a model's output unchanged. WASD uses Circuit Tracer to rank candidate neurons. It then uses a greedy search with interventions to select the neurons that keep the output stable under input perturbations. The experiments cover three tasks on four models. The paper also uses the found neurons to control the output language.

However, the ablation studies of the method are not sufficient. Many details of the method and the experiments are also not clearly explained. This makes it hard to judge the overall quality of the paper. The presentation of this paper should be improved.

Critical strengths
  1. The experiments cover four models from three model families.
  2. The method is tested on several datasets and tasks.
  3. The paper raises a useful point: high attribution does not mean that a neuron is sufficient for the output. It checks this with interventions instead of trusting the attribution scores.
  4. The found rules are small. Most rules use only two to eight neurons, which helps interpretability.
Critical weaknesses
  1. The abstract does not clearly state the contribution over prior work, and does not address whether WASD avoids the semantic incoherence problem that intervention methods typically cause.
  2. Key details about the data and the perturbations are missing. For SST-2, the prompt template and the label tokens are not given. The rule search builds the neighborhood by word deletion or replacement, as stated in Definition 2.5. However, the paper does not say how to do these modifications. The evaluation then uses a different perturbation, the neutral prefixes in Appendix B. The paper should explain why the search and the evaluation use different perturbations. Each task needs one full example with the original prompt, a perturbed prompt, and the target token.
  3. The fidelity evaluation is circular. WASD selects rules until precision reaches 0.95, then reports the same measure as the main result. The Circuit Tracer baselines are not optimized for this measure, so Table 1 mainly shows that a method that optimizes a metric scores higher on it. The authors should add a baseline that uses the same greedy search with a random neuron order.
  4. Each rule is built for one prompt and evaluated only on that same prompt. The paper does not test whether a rule works on other prompts of the same task, so the rules explain only single outputs, not the model behavior claimed in the title.
  5. WASD's perplexity is 250.79 on Gemma and 227.87 on Qwen. A high Chinese rate does not say much when the outputs are not fluent. On the QA task, WASD drops from 33% to 24% accuracy on MMLU-Pro for Qwen3-8B, and ICL achieves higher pooled accuracy on both models. The abstract criticizes methods that trade task performance for control, but WASD shows the same pattern.
  6. The paper only compares with Circuit Tracer and in-context learning, which is limited in showing the performance of the proposed approach.
  7. The datasets in Section 4 contain only short inputs, and the Chinese prompts in Appendix C are also very short. Section 4 evaluates only the next token, and the generation length in Section 5 is not reported. It is unclear whether the method works on longer texts.
Questions for authors
  1. What is the value of λ\lambda for each model and task? What candidate values are tested in the calibration? The best λ\lambda may differ from prompt to prompt, but the paper uses one λ\lambda for each model and task. In addition, λ\lambda is chosen on a small calibration subset, so it may not be the best value for the test prompts. How sensitive are the fidelity and the rule size to λ\lambda?
  2. Why are Figures 1 and 2 blurry? Besides, table lines overlap with the text.
  3. The term "output token" at lines 114 and 116 is undefined. Does it refer to the next predicted token or a token from the generation?
Reviewer_2gPqOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper's premise is that precise control for critical applications (a) requires task-specific training, (b) lacks natural language interfaces for control, or (c) compromises semantic coherence. The proposed framework WASD identifies sufficient neural conditions for token generation based on mech-interp. It uses Circuit Tracer to construct rule sets out of neuron-activation predicates under prompt perturbations that preserve a target behavior.

Critical strengths

None

Critical weaknesses

The paper's weaknesses are mostly related to presentation and value to the community.

Abstract

  • (W1) It's not immediately obvious (1) what the relationship between prompt perturbations and interventions is, (2) what the goal of the framework is ("explaining model behavior" vs. "preserving a target behavior" vs. "controlling the model").
  • (W2) The fidelity evaluation does not follow from the premise of the paper. Why is this the main number to report and not the success of control?
  • (W3) The motivation for using a circuit discovery method as a baseline for the control task is weak, because Circuit Tracer's main purpose is to explain or verbalize model behavior on a subgraph-level. A steering method would be a fairer comparison to introduce the research gap. I do not see the immediate benefit for the community in terms of practical use cases where you would choose WASD over steering.

Minor points:

  • (C1) Why is it important to hedge in the Abstract with "protocol differences limit direct comparisons with attribution baselines"? This breaks the reading flow and might be misleading at worst.
  • (C2) The meaning of "neutral-prefix evaluations" is not easily inferred at this point. Consider a simpler phrasing. It is also unclear if instability refers to the controllability or something else.

I mention most of these points under this heading, because the paper already fell apart for me at this point.

Main body

  • (W4) After trying to decipher Figure 2, the predicate construction and heuristic search steps remain very intransparent. Concrete examples and a clearer explanation of what their purposes are would help the usefulness of the figure.
  • (W5) The relevance of the three first datasets (SST-2, CF, IOI) with respect to controllability remains unclear.
  • (W6) There is a lack of qualitative analysis of the produced explanations and identified circuits.
Questions for authors
  1. Why is fidelity the main number to report and not the success of control?
  2. Why did you choose Circuit Tracer as a baseline and not a steering method?
  3. Could you provide concrete example explanations and a qualitative comparison between CT and WASD?
  4. Could you provide examples for the predicate construction and heuristic search steps?
Reviewer_B9Z3OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper propose WASD, a method to eplain and controll LLM behaviour by neuron activation rules. The method first using Circuit Tracer( existing work) to find important neurons, then applies interventions. it keep the neorons that can perserve the original output.

The paper experiment on SST-2, COunterFact, IOI with 4 models (report the better fidelity) is translation.

Critical strengths

Main logic is coherent. The experiments supported the claim: WASD can finds better output-keeping rules more effectively than circuit tracer topk Author evaluate on multipule modles and tasks, and admit the trade-off between language control and QA accuracy

Critical weaknesses
  1. Should evaluate on broader behavior control to show more application potential
  2. QA accuracy downgrade limited the application potential.
  3. The computational cost is not fully matched to the baselines, need to be clarifiy
Questions for authors

Will this method generalize to other task like formatting, factural attributes, etc? What is the size of rules, and the counts of forward-pass compared to barely circuit tracer.

Suggestion:

  1. figure1 and 2 are raster iamges, not vector graphics.
  2. row spacing in tables looks bad.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.