- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates whether interventions that suppress a reasoning model’s verbalized evaluation awareness reduce refusal. The authors identify neurons whose activations differ between evaluation-aware sentences and other reasoning, then replace their activations with average non-awareness values, substantially suppressing such statements. In QwQ-32B, this intervention also reduces refusal on examples where neither condition contains evaluation-aware statements, showing that removing those statements alone cannot explain the effect. Preserving the intervention’s output component along a refusal-associated direction partially restores refusal while maintaining awareness-statement suppression. These results suggest that the selected neurons contribute to both behaviors, so simultaneous changes in verbalized awareness and refusal do not establish that awareness causes refusal.
Critical strengths
A clear causal question. Suppressing evaluation-related representations and observing reduced refusal does not establish that awareness causes refusal: the intervention might directly disrupt refusal. This is an important distinction.
Useful controls and transfer tests. They compare with neuron-count- and layer-matched random interventions, demonstrate suppression across benchmarks and model checkpoints, and check general capabilities.
Critical weaknesses
Their neuron selection may capture evaluation-related language rather than awareness. They contrast sentences explicitly mentioning evaluation with other reasoning. This can select neurons representing words such as “test,” sentence content, or general caution. Successful suppression establishes control over verbalization more clearly than control over awareness. The probe only partly addresses this, because it is constructed from representations of answering “Testing” versus “Deployment.”
Their “direct effect on refusal” argument is incomplete. Refusal decreases even when neither condition verbalizes awareness, but the model could still have unverbalized awareness that the intervention changes. Besides, refusal directions are not unique and difference-in-mean is just one of them; there could be other directions leading to refusal/ acceptance as well.
Questions for authors
Why associate refusal with evaluation awareness? These two concepts feel distant to me, and I am convinced of the necessity to even disentangle them.
3 comments
Kind reminder to reviewer ni48
Thank you again for your valuable review and constructive feedback on our paper. As we have not yet received a response, we would like to kindly remind Reviewer ni48 that we have addressed your concerns. Specifically, we argue that the neuron selection captures more than only evaluation-related language, as shown in EvalAwareBench and probe as shown in the appendix (Before revision). Regarding to the second concern, we add analysis on how the intervention changes unverbalized evaluation awareness. The updates have been incorporated into the revised paper, which we believe has been significantly strengthened as a result. We would greatly appreciate it if you could review these updates and kindly reconsider your score in light of our revisions to address your comments.
Thank you for your response. After reviewing it and other reviewers' comments, I will maintain my score.
Response to reviewer ni48
We thank the reviewer for the feedback. We respond to each concern below and add one new analysi and we have uploaded a revised PDF with the changes marked in red.
We agree on the scope: the neurons are identified from verbalization of evaluation awareness. However, 2 results indicate that the effect is not limited to words such as "test".
On the probe. Its direction is extracted at the Testing/Deployment answer tokens, but it is validated on held-out items using answer-free prefixes against the true labels: AUROC 0.81, versus 0.48 for random directions (App. D.2). It therefore reads more than the answer token. We agree it is imperfect, however, to our understanding, this construction is an established way of measuring internal evaluation awareness (Nguyen et al., 2025, Probing and Steering Evaluation Awareness of Language Models).
We agree, and our analysis did not make this clear. In the revision we add a causal diagram (Fig. 4) with three pathways from the intervention to refusal: (i) removing VEA text from the CoT, (ii) weakening unverbalized evaluation awareness (abbreviated unVEA in the pdf), and (iii) changing the neurons' write to the refusal direction. A mediation analysis (new Sec. 4.4, Eq. 8) estimates each share on QwQ-32B SorryBench (440 questions × 3 seeds) from three sets of rollouts: (A) no intervention, (B) VEA-neuron intervention, and (C) refusal-component-preserving intervention, which differs from B only in pathway (iii); Sec. 4.3 now states this explicitly.
Table E.2 (new App. E.5) reports each term's share of the total decrease with 95% bootstrap intervals. Across N = 5,000 to 20,000, (i) accounts for 4–10% of the decrease, (ii) for 39–47%, and (iii) for 48–57%. Unverbalized awareness can therefore account for at most 39–47%. Sec. 4.2 now states that the experiment on rollouts without VEA does not separate pathways (ii) and (iii).
Refusal is the quantity that safety benchmarks measure, so if evaluation awareness raises it, evaluations overstate deployment safety. This is why we care about evaluation awareness. Prior work reports this link: evaluation-aware reasoning increases refusal, and steering evaluation awareness changes safety behavior (Sec. 1). Our results show it as well: VEA and refusal decrease together under intervention (Fig. 3b).
This association is why disentangling is necessary. The mediation analysis (Sec. 4.4; Table E.2, App. E.5) shows that removing VEA text explains at most 10% of the refusal decrease, while about half is a direct write to the refusal direction. Without separating the two, any experiment that suppresses evaluation awareness and observes less refusal would be read as "awareness causes safer behavior", whereas part of the effect arises because the same neurons carry both awareness and refusal. We therefore consider this a core contribution of our paper.
We hope these clarifications and the additional analysis address your concerns. If so, we would be grateful if you would consider updating your score.