- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes WASD, a method that finds a small set of neurons that is sufficient to keep a model's output unchanged. WASD uses Circuit Tracer to rank candidate neurons. It then uses a greedy search with interventions to select the neurons that keep the output stable under input perturbations. The experiments cover three tasks on four models. The paper also uses the found neurons to control the output language.
However, the ablation studies of the method are not sufficient. Many details of the method and the experiments are also not clearly explained. This makes it hard to judge the overall quality of the paper. The presentation of this paper should be improved.
Critical strengths
- The experiments cover four models from three model families.
- The method is tested on several datasets and tasks.
- The paper raises a useful point: high attribution does not mean that a neuron is sufficient for the output. It checks this with interventions instead of trusting the attribution scores.
- The found rules are small. Most rules use only two to eight neurons, which helps interpretability.
Critical weaknesses
- The abstract does not clearly state the contribution over prior work, and does not address whether WASD avoids the semantic incoherence problem that intervention methods typically cause.
- Key details about the data and the perturbations are missing. For SST-2, the prompt template and the label tokens are not given. The rule search builds the neighborhood by word deletion or replacement, as stated in Definition 2.5. However, the paper does not say how to do these modifications. The evaluation then uses a different perturbation, the neutral prefixes in Appendix B. The paper should explain why the search and the evaluation use different perturbations. Each task needs one full example with the original prompt, a perturbed prompt, and the target token.
- The fidelity evaluation is circular. WASD selects rules until precision reaches 0.95, then reports the same measure as the main result. The Circuit Tracer baselines are not optimized for this measure, so Table 1 mainly shows that a method that optimizes a metric scores higher on it. The authors should add a baseline that uses the same greedy search with a random neuron order.
- Each rule is built for one prompt and evaluated only on that same prompt. The paper does not test whether a rule works on other prompts of the same task, so the rules explain only single outputs, not the model behavior claimed in the title.
- WASD's perplexity is 250.79 on Gemma and 227.87 on Qwen. A high Chinese rate does not say much when the outputs are not fluent. On the QA task, WASD drops from 33% to 24% accuracy on MMLU-Pro for Qwen3-8B, and ICL achieves higher pooled accuracy on both models. The abstract criticizes methods that trade task performance for control, but WASD shows the same pattern.
- The paper only compares with Circuit Tracer and in-context learning, which is limited in showing the performance of the proposed approach.
- The datasets in Section 4 contain only short inputs, and the Chinese prompts in Appendix C are also very short. Section 4 evaluates only the next token, and the generation length in Section 5 is not reported. It is unclear whether the method works on longer texts.
Questions for authors
- What is the value of for each model and task? What candidate values are tested in the calibration? The best may differ from prompt to prompt, but the paper uses one for each model and task. In addition, is chosen on a small calibration subset, so it may not be the best value for the test prompts. How sensitive are the fidelity and the rule size to ?
- Why are Figures 1 and 2 blurry? Besides, table lines overlap with the text.
- The term "output token" at lines 114 and 116 is undefined. Does it refer to the next predicted token or a token from the generation?