- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
The paper builds a "model organism" from Qwen3.5-27B for unverbalized evaluation awareness (UEA). It is trained in two phases. Synthetic document fine-tuning teaches the model to emit a "§" marker in its visible response when it believes it is being evaluated. SFT plus DPO on DeepSeek-edited traces then removes verbalized awareness and marker deliberation from the reasoning. The marker serves as an instance-level label. Prompt steering moves it strongly (X100+NotEval drives it below 1%, Table 2), and it correlates with the base model's test-question verbalizations (τb = 0.42). Using the label, the authors benchmark 10 linear probes, 22 Jacobian Lens readouts, and Activation Oracles, all built on the base model. The best median AUCs are only about 0.59–0.70, so the paper concludes that current methods are not reliable UEA detectors.
Critical strengths
- An important problem and a concrete testbed. Unverbalized evaluation awareness undermines safety evaluations, and verbalized-awareness rates are only a lower bound. The paper builds a model organism and uses it to measure how well existing white-box detection methods perform. Without such a label, no UEA detector can be validated at all.
- Careful, well-documented engineering. This includes marker-counterfactual DPO pairs, 16 benchmark configurations, five system-prompt conditions, resampling robustness (Table 25), a capability check (Tables 14–15), lexical audits (Table 16), and a released code pipeline.
- The results point to a gap in current detection methods. Off-the-shelf detectors perform poorly at recovering the organism's awareness (best median AUC ≈ 0.59–0.70), and the paper is honest about confounds: the BBH prefix shortcut, class imbalance, neutral-direction baselines, and the adapter-off control. This is a useful early warning that existing tools may not be ready for UEA auditing.
Critical weaknesses
- The detector evaluation confounds "UEA is hard to detect" with "base-model detectors don't transfer." Every method is built on the base model and only applied to the organism (Sec. 3.3): a real auditor has the audited model's weights and can build contrastive or verbalization-trained probes on that model directly. The organism has been substantially changed (164k SDF documents, SFT, and ~31k DPO pairs), and representation drift is never quantified. Also, the trained probes learn from the base model's verbalized awareness, while the organism is trained specifically not to verbalize.
- Narrow notion of "evaluation." The marker is near-saturated on multiple-choice benchmarks (MMLU 99.7%) but near zero on HarmBench, StrongREJECT and agentic sets (Table 13). It mostly tracks question format and misses the safety-relevant case.
- Limited generality. One base model and one training seed.
Questions for authors
6 comments
Response to Reviewer 3jTS (2/3)
W1.2 “The trained probes learn from the base model's verbalized awareness, while the organism is trained specifically not to verbalize.”
Fitting every method on the base model was a deliberate choice to keep each method blind to the new semantics of the marker (Section 3.3). This follows Hua et al. (2026) who construct steering vectors from the model prior to training and apply them to the model organism. However, we have also trained probes on the model organism’s own activations with the § marker used as the label for evaluation awareness. These probes do not depend on verbalization at all, yet they do not clearly outperform the probes trained on verbalized awareness. The weak detection is therefore not explained by training the probes on verbalized awareness. We provide the details of this new experiment below.
We trained probes on the organism activations labelled by whether it emitted §. We collected the organism’s response under the X100 system prompt using the eight training benchmarks, and trained the paper’s two probe types (mass-mean and softmax) at five positions defined relative to the marker: the marker token itself, the token before it, the last non-whitespace token before it, the first reasoning token, and the first token of the last reasoning sentence (Appendix H). For the negative class, we take activations from the same position where the marker would otherwise appear. The layers were selected by testing the marker on held-out training prompts, with experiments presented on the four held-out test benchmarks (WildChat, BBH, EvalAwareBench, Triggers). We use the comparable probes trained in the original paper using verbalized evaluation awareness as a baseline.
The probes trained using the marker as a label do not clearly outperform the baseline probes. Their mean AUC across probes and positions is within 0.03 of the baseline on BBH, EvalAwareBench and Triggers, and higher only on WildChat (0.633 vs 0.545), where performance is still weak. The best single probe on each benchmark is similar on BBH (0.979 vs 0.980) and EvalAwareBench (0.598 vs 0.604) and higher on Triggers (0.667 vs 0.616) and WildChat (0.852 vs 0.806). Overall, with the slight exception of WildChat, the marker probes do not appear to perform better, and even there the gain is modest and the probes still perform poorly in general. These suggest that the paper’s results are not explained by the use of verbalized evaluation awareness, and provide further evidence that the methods themselves perform poorly on this task.
| WildChat | BBH | EvalAwareBench | Triggers | |
|---|---|---|---|---|
| Baseline probes: mean | 0.545 | 0.582 | 0.513 | 0.506 |
| § probes: mean | 0.633 | 0.579 | 0.535 | 0.523 |
| Baseline probes: best | 0.806 | 0.980 | 0.604 | 0.616 |
| § probes: best | 0.852 | 0.979 | 0.598 | 0.667 |
| AUC for predicting § emission. Mean is over all probe–position combinations; best is the single best probe and position (Appendix H). |
Response to Reviewer 3jTS (3/3)
W2.1 “Narrow notion of evaluation.” and misses the safety-relevant case.
We agree that the organism recognises a narrower notion of evaluation than we would ideally want, and we already note this in the paper (Section 5.1.1). We also agree that marker rates on the safety benchmarks are low, although we do see non-negligible rates under the X100 system prompt on some jailbreak prompts (e.g. SG-Bench (Jailbreak) 19.8%, StrongREJECT (AIM) 8.0%; Table 1), so we would not say the case is missed entirely. This behaviour follows from a deliberate choice to design our training pipeline to utilise the model organism’s own notion of evaluation rather than implanting a specific belief of what an evaluation looks like, which could potentially affect how evaluation awareness was represented in the model. After SDF, the model organism naturally identified a skewed distribution of prompts as evaluations, which were more often capability benchmarks rather than safety ones. As this checkpoint was used to generate the SFT and DPO data, those phases reinforced this skew. Even under this narrower notion, which relies on comparatively simple test-question cues, the methods do not reliably detect the marker (Section 5.2).
”W2.2 “The marker tracks question format””
We agree that test-question formatting is a strong cue for the marker, however we do not think the evidence suggests that the model organism has learned a simple rule tied to test question formatting. Removing the multiple-choice formatting (i.e, the test-question formatting and hence, asking only the question) does not drop marker generation rates to 0 (Appendix C), which should be the case under that assumption. Moreover, question content also matters. Without multiple-choice formatting, MMLU and BBH have higher marker rates without steering than TruthfulQA, which reflects questions that read like everyday user queries instead of typical capability benchmarks (e.g. “What happens if you eat after 8:00 p.m.?”). Therefore, these results suggest that the marker generation reflects more than mere test-question recognition, and instead reflects the model’s belief which can be influenced by several factors. We will add this to the Discussion section.
| MMLU | BBH | TruthfulQA | |
|---|---|---|---|
| Benchmark format | 99.8 | 50.8 | 99.9 |
| Question only | 19.6 | 24.5 | 4.2 |
Marker rate (%) under the X100 system prompt (Appendix C).
The model organism’s marker output rate is also strongly influenced by factors other than test-question formatting, in particular prompt-based steering. Adding an evaluation suffix to the system prompt raises marker rates on most benchmarks, often close to saturation, including on safety benchmarks (e.g. HarmBench from 5.0% to 94.3%, StrongREJECT from 2.6% to 93.9%; Table 2). A not-evaluation suffix pushes them close to zero, even on benchmarks that are otherwise almost always marked (MMLU from 99.7% to 0.1%, TruthfulQA from 99.9% to 0.5%; Table 2). This shows that the model organism is not simply tracking test-question formatting, and that other factors can influence the marker behaviour.
W3. “Limited generality. One base model and one training seed.”
We agree with the reviewer that additional trained models would strengthen the result. We have trained Gemma 4 31B using the same pipeline. Initial results suggest it has learned similar behaviours to our model organism trained using Qwen-3.5 27B. We commit to adding the full results from our assessment of this additional model to the revised version.
| Benchmark | X100: Qwen | X100: Gemma | X100 + Eval: Qwen | X100 + Eval: Gemma |
|---|---|---|---|---|
| MMLU | 99.7 | 98.0 | 99.9 | 99.2 |
| IFEval | 2.8 | 19.6 | 51.4 | 57.7 |
| SG-Bench (MCQ) | 18.9 | 31.8 | 66.7 | 77.4 |
| HarmBench | 5.0 | 6.5 | 94.3 | 59.5 |
| StrongREJECT | 2.6 | 8.3 | 93.9 | 72.2 |
| Triggers | 4.8 | 7.8 | 87.9 | 64.9 |
| WildChat | 14.8 | 13.1 | 62.1 | 48.2 |
Marker rate (%) for the Qwen3.5-27B organism (Table 2) and the Gemma 4 31B organism trained with the same pipeline, under the X100 system prompt alone and with the evaluation suffix.
Response to Reviewer 3jTS (1/3)
Thank you for your review. We are happy to see we tackle an important problem with a concrete testbed, our experiments are well documented and that our results show a gap in current detection methods.
W1.1 “The detector evaluation confounds "UEA is hard to detect" with "base-model detectors don't transfer."”
We agree it is important to distinguish between these two explanations for our results. We ran follow up experiments applying the same ten probes but using the model organism’s activations to rule out the transferability hypothesis (Appendix H). The results also show poor UEA detection, therefore, our conclusion holds: current methods do not reliably detect UEA. We provide the details of the new experiment below:
We repeat the paper’s pipeline, using the same contrastive prompt pairs, judge labelled transcripts and layer selection, but with every activation collected from the model organism instead of the base model. The probes trained on the organism again behave similarly to the probes from the original paper. Across the four tested benchmarks, the median and mean AUC for the benchmarks change by at most 0.02 when compared with the original probes. The distribution of AUCs across probes and extraction points is also very similar on each benchmark, with means within 0.02 and standard deviations within 0.01 of the originals. The strongest probes on each benchmark remain largely unchanged and each has similar overall values. If representation shift explained the poor performance of the probes in the paper, we would have expected the same probes trained on the model organism to perform better. They did not, suggesting that the weak detection reflects the methods themselves.
AUC for predicting § emission with the ten probes of the paper. Mean ± SD is over the 110 probe–position combinations; best is the single best probe and position (Table 31).