- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2(+1)
Read review
Paper summary
This paper investigates whether natural-language descriptions produced by Natural Language Autoencoders (NLAs) can serve as a causal intervention channel for language-model refusal behavior. The authors construct steering directions by encoding two human-written sentences describing refusal and compliance, subtracting their reconstructed activation vectors, and injecting the resulting direction into the model's residual stream.
The paper evaluates this approach through controlled interventions, dose-response analyses, semantic placebo comparisons, and tests of model competence. The results show that the language-derived direction systematically influences refusal behavior in Qwen2.5-7B, while having a relatively small effect on an exact-match task battery at the evaluated dose. The authors also investigate the limitations of norm-matched random-vector controls and report negative results concerning cross-model transfer and autoencoder retraining.
Critical strengths
The paper investigates whether natural-language descriptions of model activations can be used for causal interventions on refusal behavior. This is a relevant question for understanding the relationship between interpretability and model behavior.
Critical weaknesses
-
Poor readability and fragmented presentation. The manuscript is difficult to follow due to its fragmented organization and experiment-log-like narrative. The core methodology, including the NLA reconstruction process, steering direction construction, and intervention procedure, is scattered across multiple sections rather than presented in a unified manner. Moreover, numerous experiment identifiers, registration details, statistical values, and cross-references interrupt the main argument. Although reporting unsuccessful experiments is valuable, the current presentation makes it difficult to identify the central contributions and understand how the different experiments support them. Substantial improvements in organization and clarity are needed.
-
Insufficient clarity regarding the interpretation and practical significance of the causal results. Although the paper presents several intervention and control experiments, the precise conclusions supported by these results are not always easy to distinguish. In particular, changes in refusal behavior do not necessarily establish that the natural-language descriptions faithfully capture the underlying model states. The paper acknowledges this distinction, but its implications for the central contribution could be explained more clearly. In addition, the competence-preservation experiment uses substantially fewer injection positions per item than the refusal experiments, limiting the direct comparability of these results. A clearer separation between demonstrated causal effects, interpretability claims, and practical implications would strengthen the paper.
Questions for authors
Q1. Could the authors substantially improve the organization and readability of the manuscript? In particular, a unified overview of the complete NLA-based intervention pipeline, together with a consolidated table summarizing the main experiments, their settings, and key findings, would help readers better understand the methodology and evaluate the contributions.
Q2. Could the authors clarify the precise causal and interpretability claims supported by the experiments? Specifically, to what extent do the observed behavioral changes validate the semantic faithfulness of the natural-language descriptions, rather than merely demonstrating that their reconstructed vectors can influence refusal behavior? Additionally, could the authors clarify how the difference in injection schedules between the competence-preservation and refusal experiments affects the interpretation of the reported performance trade-off?
Overall Assessment: While the research question is potentially interesting and the authors attempt to provide several experimental controls, I found the manuscript difficult to follow and its central contributions insufficiently clearly presented. The concerns regarding robustness and the interpretation of the causal evidence further limit my confidence in the conclusions. I would be willing to reconsider my assessment if the authors can substantially clarify the methodology, experimental findings, and scope of their claims during the rebuttal.
6 comments
We have now uploaded the revised manuscript and updated supplementary archive incorporating all structural and methodological changes requested in your review:
- Unified pipeline and experiment index (W1 / Q1): §1.1 (Figure 1, p. 2) and §3 (pp. 3–4) present the four-step write-back pipeline and grading rule in one place, indexed by the consolidated experiment summary in Table 1 (p. 4). Pre-registration codes and bootstrap intervals are moved to Appendices A and C–E, and each empirical section opens with its primary test while caveats are consolidated once in §8 (pp. 8–9).
- Causal claims vs. semantic faithfulness and matched competence (W2 / Q2): §1.2 (p. 2) explicitly separates what is shown causally, what is shown predictively, and what is not shown (in particular, aggregate behavioural coupling does not establish single-reading faithfulness). §4.6 adds write-back from the NLA's own readings (+0.756 vs. +0.000 shuffled), and §5.2 (Table 2, p. 6) reports the matched 100-problem GSM8K evaluation steering generated tokens only: at α = −0.75, accuracy holds for both the written pair (85% vs. 85% unsteered, median 277 steered positions) and the readings direction (84% vs. 88% unsteered and 81% shuffled, median 247.5 steered positions), before degrading at α = −1.
We hope the reorganised manuscript and matched experiments address your concerns, and we welcome any further questions during the discussion period.
I believe this paper explores an interesting and potentially valuable research direction, and I appreciate the authors' efforts in conducting extensive experiments and providing detailed analyses. However, I still think the presentation of the paper could be substantially improved. In particular, I encourage the authors to better highlight the motivation behind the work and provide a more concise and unified description of the proposed methodology, so that readers can more easily understand the main idea and the reasoning behind the approach.
In my view, a good research paper should not only address a meaningful problem but also clearly explain why the problem matters and how the proposed method solves it. A clearer motivation, a more coherent methodological explanation, and a better-organized presentation of the experimental findings would significantly strengthen this work.
I appreciate the authors' efforts and believe the research has potential. I encourage them to continue improving the manuscript, particularly in terms of clarity and presentation. For now, I will maintain my original rating, and I hope these suggestions will be helpful in further strengthening the paper.
Response to Reviewer TCre
W1 / Q1. Organisation, a unified pipeline, and one summary table.
We agree. The next version will describe the method in one section. Here it is in four steps.
The draft summary table:
Rows marked * are new and use layer 20, where the written pair gives +0.612. The §4.5 row uses layer 21.
W2 / Q2. What the causal results support.
Behavioural change alone does not validate the semantic faithfulness of the descriptions. It shows that the reconstructed vectors move refusal, and in the direction the sentences describe. The paper's claims fall into three groups.
Shown, causal. Adding the difference between the reconstructions of two written sentences changes graded refusal with dose: median ρ = +0.412 against a placebo's +0.000. Four of five wordings reproduce this, and two pairs on other topics do not. So the effect depends on what the sentences say. A direction built from the NLA's own readings, written back, also moves refusal (+0.756). With labels shuffled, it does not (+0.000). ActAdd on the same two sentences does not move refusal (+0.000 against +0.612).
Shown, predictive. The NLA's reading of an item predicts which items lose refusal under a fitted direction (ρ = +0.51). It does so less well than the raw activation (ρ = +0.58).
Not shown. That any single reading is a faithful description of the model's state. That the reconstructor's advantage over ActAdd holds beyond one sentence pair and one layer. That the written pair transfers to a second family. That the effect survives retraining the NLA.
The next version will state these three groups in §1 and use them in the abstract and the conclusion.
On the injection schedules: the competence test steered a median of 58 positions per item, prompt included, not about 3. The refusal sweep steers a median of 255 positions. So the 2-point figure cannot be read as the cost of the refusal intervention. The matched test in the general comment replaces it. At the dose where refusal falls 38 points, GSM8K accuracy is unchanged. The next version will report both tests in §5.2.