- Rating
- 3
- Confidence
- 2(+1)
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CANS, a neuron-level steering framework that balances behavioral control with capability preservation. CANS combines target-token log-probability gradients with a diagonal Fisher approximation of output-distribution drift to derive closed-form neuron scaling coefficients. Experiments on safety, toxicity, and hallucination show favorable steering–capability trade-offs across Qwen and Llama models.
Critical strengths
- The method explicitly incorporates preservation costs into neuron-wise interventions, providing a coherent formulation of the steering–capability trade-off.
- The closed-form solution avoids iterative fine-tuning, and scaling coefficients can be folded into model weights without additional inference operations.
- Evaluation covers three behavioral domains and multiple capability benchmarks, with substantial improvements in several settings.
Critical weaknesses
- Capability preservation relies on distributional stability over reference text. Although the paper acknowledges this proxy and provides empirical support, its coverage of broader capabilities remains uncertain.
- Target-span likelihood may capture lexical preferences rather than the intended behavior. The reported improvements do not fully establish whether safety gains avoid over-refusal or hallucination reductions reflect more informative, correct answers.
- CANS-Grad-Only changes both the preservation mechanism and the intervention scheme by uniformly scaling a selected subset. Consequently, this comparison does not cleanly isolate the contribution of Fisher weighting.
Questions for authors
I rewrite the weakness above into the questions below:
- How robust is capability preservation on tasks substantially different from the reference corpus, such as coding and multilingual reasoning?
- Can the authors report benign-request refusal rates and distinguish correct, informative answers from abstentions in hallucination evaluation?
- Can the authors compare against scaling all neurons proportionally to their gradients, using the same clipping and tuning protocol, to isolate the benefit of Fisher weighting?
9 comments
Llama Family (Llama-3-8B-Instruct Safety / Llama-2-7B Toxicity / Llama-3.1-8B-Instruct Hallucination):
| Method | Safety: ASR ↓ | Safety: MGSM ↑ | Safety: MBPP ↑ | Toxicity: Toxic Rate ↓ | Toxicity: MGSM ↑ | Toxicity: MBPP ↑ | Hallu.: T&I Failure ↓ | Hallu.: MGSM ↑ | Hallu.: MBPP ↑ |
|---|---|---|---|---|---|---|---|---|---|
| Unmodified | 14.00 | 51.82 | 57.20 | 37.33 | 12.36 | 19.20 | 58.50 | 68.73 | 59.40 |
| SFT | 10.00 | 54.55 | 57.20 | 17.83 | 11.64 | 20.00 | 37.75 | 69.27 | 58.40 |
| CAA | 10.50 | 27.64 | 36.60 | 34.50 | 10.91 | 19.20 | 59.25 | 68.36 | 58.20 |
| AurA | 0.00 | 0.00 | 0.00 | 21.50 | 10.91 | 18.80 | 52.50 | 65.45 | 57.60 |
| SN-Tune | 8.75 | 50.91 | 57.60 | 33.67 | 11.45 | 19.20 | 47.25 | 56.36 | 59.00 |
| DAP | 12.25 | 50.73 | 56.80 | 35.67 | 11.64 | 20.20 | 64.25 | 70.00 | 59.60 |
| H-Neurons | 7.50 | 23.27 | 35.80 | 32.33 | 9.09 | 18.60 | 56.50 | 69.45 | 59.00 |
| CANS-Grad-Only | 8.75 | 30.73 | 53.20 | 19.33 | 3.64 | 6.60 | 46.50 | 69.09 | 59.20 |
| CANS (Ours) | 8.75 | 54.36 | 58.20 | 18.83 | 11.64 | 19.40 | 46.50 | 69.45 | 59.20 |
Qwen3-4B (Safety / Toxicity / Hallucination):
| Method | Safety: ASR ↓ | Safety: MGSM ↑ | Safety: MBPP ↑ | Toxicity: Toxic Rate ↓ | Toxicity: MGSM ↑ | Toxicity: MBPP ↑ | Hallu.: T&I Failure ↓ | Hallu.: MGSM ↑ | Hallu.: MBPP ↑ |
|---|---|---|---|---|---|---|---|---|---|
| Unmodified | 15.50 | 74.55 | 65.00 | 19.17 | 74.55 | 65.00 | 39.50 | 74.55 | 65.00 |
| SFT | 8.00 | 73.82 | 64.40 | 4.50 | 73.45 | 63.80 | 37.00 | 73.64 | 65.60 |
| CAA | 14.00 | 49.45 | 53.80 | 19.50 | 73.82 | 65.60 | 38.00 | 73.82 | 65.20 |
| AurA | 0.00 | 0.00 | 0.00 | 10.83 | 72.91 | 65.20 | 39.00 | 68.00 | 61.00 |
| SN-Tune | 15.75 | 72.91 | 65.20 | 20.00 | 74.18 | 64.60 | 37.75 | 73.45 | 65.20 |
| DAP | 16.50 | 73.45 | 65.80 | 17.83 | 73.45 | 64.60 | 38.50 | 74.18 | 65.00 |
| H-Neurons | 16.50 | 74.00 | 65.00 | 16.83 | 74.73 | 65.80 | 37.00 | 74.73 | 65.40 |
| CANS-Grad-Only | 7.50 | 73.45 | 65.40 | 4.67 | 71.09 | 62.20 | 35.25 | 72.55 | 64.60 |
| CANS (Ours) | 8.00 | 75.27 | 65.40 | 4.17 | 73.64 | 63.20 | 34.25 | 71.64 | 66.20 |
Key Takeaways:
- Robust Preservation on Coding and Multilingual Math: Even though our Fisher diagonal is estimated purely on unlabeled English prose (WikiText-2), CANS robustly preserves both MBPP (code generation) and MGSM (multilingual math) across all six settings—frequently matching or slightly exceeding the unmodified model (e.g., on Llama-3-8B Safety, CANS reaches 54.36% on MGSM and 58.20% on MBPP vs. and unmodified; on Qwen3-4B Safety, CANS reaches 75.27% on MGSM and 65.40% on MBPP vs. and unmodified).
- Sharp Contrast with Unregularized and Heuristic Baselines: Without Fisher curvature protection, methods that achieve strong steering often suffer severe collapses on out-of-distribution capabilities. For instance, on Llama-3-8B Safety, H-Neurons and CAA drop MGSM from to 23.27% and 27.64%, and MBPP from to 35.80% and 36.60%. Similarly, CANS-Grad-Only () drops MGSM to 30.73% on Llama-3-8B Safety (vs. 54.36% for CANS) and collapses on Llama-2-7B Toxicity (dropping MGSM from to 3.64% and MBPP from to 6.60%, whereas CANS maintains 11.64% and 19.40%).
- Mechanistic Insight: These results demonstrate that specialized tasks such as code synthesis and multilingual reasoning rely on shared foundational FFN circuits (e.g., logical deduction, state tracking, and syntactic composition) that are already active during general text continuation. By penalizing neurons with high Fisher curvature on reference continuations, CANS effectively protects these shared computational backbones across diverse downstream tasks.
We sincerely thank the reviewer for the constructive and thoughtful feedback. We have fully incorporated every experimental and methodological suggestion you raised across the entire revised manuscript:
Response to W1 & Q1 (Robustness of Capability Preservation on Coding and Multilingual Reasoning)
We thank the reviewer for this constructive suggestion. To rigorously test whether distributional stability on our general-purpose reference corpus (WikiText-2) preserves capabilities on tasks substantially different from encyclopedic English prose, we expanded our evaluation suite in the revised paper with two additional benchmarks:
We re-evaluated all methods across all models and domains on MBPP and MGSM, and updated the aggregated capability score across the entire revised manuscript to the equal-weight mean of all five benchmarks (IFEval, BBH, MMLU, MBPP, and MGSM; see updated Figure 2, Figure 3, and Appendix B). We report detailed results for MGSM and MBPP below, alongside domain-specific metrics for all six model–domain settings. All values are expressed as percentages. The corresponding plots appear in Figures 10 and 11 of Appendix B (p. 22-23).