acceptodds
Under review as a conference paper at ICLR 2027

A Model Organism for Studying Unverbalized Evaluation Awareness

Abstract

Large language models can recognize when they are being evaluated and change their behavior accordingly, undermining the reliability of safety benchmarks. Evaluation awareness (EA) is usually measured by having an LLM judge scan reasoning traces for explicit statements of recognition, but this signal is increasingly unreliable. Recent frontier-model system cards report a drop in verbalized rates between releases, which is indistinguishable from awareness that has gone unverbalized. Interpretability methods could detect such unverbalized evaluation awareness (UEA) in individual interactions, but lack instance-level ground truth. Asking the model alters the input, and dataset labels do not reveal whether it recognized a particular prompt as an evaluation. We address this gap with a model organism. Using synthetic document fine-tuning, SFT, and DPO, we train Qwen3.5-27B to emit a marker (a designated token) in its final response when it recognizes an evaluation, while suppressing the marker and verbalized awareness in its reasoning. The marker largely tracks the evaluation concept as it can be strongly mediated by evaluation framing in the system prompt. When not told whether it is being evaluated, the organism recognizes evaluations from test-question cues, mirroring what its base model often verbalizes. Using the marker as a label, we benchmark linear probes, the Jacobian Lens, and Activation Oracles, built on the base model and blind to the marker. None recovers the label reliably. These results suggest that current interpretability methods are not yet reliable UEA detectors, even when recognition relies on simple cues. Our model organism offers a testbed for improving them.

Primary area:
interpretability and explainable AI
Keywords:
Evaluation Awareness, Model Organism, Interpretability, Large Language Model, AI Safety, Explainability, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3jTSOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper builds a "model organism" from Qwen3.5-27B for unverbalized evaluation awareness (UEA). It is trained in two phases. Synthetic document fine-tuning teaches the model to emit a "§" marker in its visible response when it believes it is being evaluated. SFT plus DPO on DeepSeek-edited traces then removes verbalized awareness and marker deliberation from the reasoning. The marker serves as an instance-level label. Prompt steering moves it strongly (X100+NotEval drives it below 1%, Table 2), and it correlates with the base model's test-question verbalizations (τb = 0.42). Using the label, the authors benchmark 10 linear probes, 22 Jacobian Lens readouts, and Activation Oracles, all built on the base model. The best median AUCs are only about 0.59–0.70, so the paper concludes that current methods are not reliable UEA detectors.

Critical strengths
  1. An important problem and a concrete testbed. Unverbalized evaluation awareness undermines safety evaluations, and verbalized-awareness rates are only a lower bound. The paper builds a model organism and uses it to measure how well existing white-box detection methods perform. Without such a label, no UEA detector can be validated at all.
  2. Careful, well-documented engineering. This includes marker-counterfactual DPO pairs, 16 benchmark configurations, five system-prompt conditions, resampling robustness (Table 25), a capability check (Tables 14–15), lexical audits (Table 16), and a released code pipeline.
  3. The results point to a gap in current detection methods. Off-the-shelf detectors perform poorly at recovering the organism's awareness (best median AUC ≈ 0.59–0.70), and the paper is honest about confounds: the BBH prefix shortcut, class imbalance, neutral-direction baselines, and the adapter-off control. This is a useful early warning that existing tools may not be ready for UEA auditing.
Critical weaknesses
  1. The detector evaluation confounds "UEA is hard to detect" with "base-model detectors don't transfer." Every method is built on the base model and only applied to the organism (Sec. 3.3): a real auditor has the audited model's weights and can build contrastive or verbalization-trained probes on that model directly. The organism has been substantially changed (164k SDF documents, SFT, and ~31k DPO pairs), and representation drift is never quantified. Also, the trained probes learn from the base model's verbalized awareness, while the organism is trained specifically not to verbalize.
  2. Narrow notion of "evaluation." The marker is near-saturated on multiple-choice benchmarks (MMLU 99.7%) but near zero on HarmBench, StrongREJECT and agentic sets (Table 13). It mostly tracks question format and misses the safety-relevant case.
  3. Limited generality. One base model and one training seed.
Questions for authors

6 comments
AuthorsOpenReview ↗

Response to Reviewer 3jTS (1/3)

Thank you for your review. We are happy to see we tackle an important problem with a concrete testbed, our experiments are well documented and that our results show a gap in current detection methods.

W1.1 “The detector evaluation confounds "UEA is hard to detect" with "base-model detectors don't transfer."”

We agree it is important to distinguish between these two explanations for our results. We ran follow up experiments applying the same ten probes but using the model organism’s activations to rule out the transferability hypothesis (Appendix H). The results also show poor UEA detection, therefore, our conclusion holds: current methods do not reliably detect UEA. We provide the details of the new experiment below:

We repeat the paper’s pipeline, using the same contrastive prompt pairs, judge labelled transcripts and layer selection, but with every activation collected from the model organism instead of the base model. The probes trained on the organism again behave similarly to the probes from the original paper. Across the four tested benchmarks, the median and mean AUC for the benchmarks change by at most 0.02 when compared with the original probes. The distribution of AUCs across probes and extraction points is also very similar on each benchmark, with means within 0.02 and standard deviations within 0.01 of the originals. The strongest probes on each benchmark remain largely unchanged and each has similar overall values. If representation shift explained the poor performance of the probes in the paper, we would have expected the same probes trained on the model organism to perform better. They did not, suggesting that the weak detection reflects the methods themselves.

WildChatBBHEvalAwareBenchTriggers
Fitted on base model: mean ± SD0.543 ± 0.1220.545 ± 0.2160.509 ± 0.0390.504 ± 0.045
Fitted on organism: mean ± SD0.531 ± 0.1280.564 ± 0.2130.510 ± 0.0380.496 ± 0.048
Fitted on base model: best0.8060.9800.6040.616
Fitted on organism: best0.8350.9800.6170.628

AUC for predicting § emission with the ten probes of the paper. Mean ± SD is over the 110 probe–position combinations; best is the single best probe and position (Table 31).

AuthorsOpenReview ↗

Response to Reviewer 3jTS (2/3)

W1.2 “The trained probes learn from the base model's verbalized awareness, while the organism is trained specifically not to verbalize.”

Fitting every method on the base model was a deliberate choice to keep each method blind to the new semantics of the marker (Section 3.3). This follows Hua et al. (2026) who construct steering vectors from the model prior to training and apply them to the model organism. However, we have also trained probes on the model organism’s own activations with the § marker used as the label for evaluation awareness. These probes do not depend on verbalization at all, yet they do not clearly outperform the probes trained on verbalized awareness. The weak detection is therefore not explained by training the probes on verbalized awareness. We provide the details of this new experiment below.

We trained probes on the organism activations labelled by whether it emitted §. We collected the organism’s response under the X100 system prompt using the eight training benchmarks, and trained the paper’s two probe types (mass-mean and softmax) at five positions defined relative to the marker: the marker token itself, the token before it, the last non-whitespace token before it, the first reasoning token, and the first token of the last reasoning sentence (Appendix H). For the negative class, we take activations from the same position where the marker would otherwise appear. The layers were selected by testing the marker on held-out training prompts, with experiments presented on the four held-out test benchmarks (WildChat, BBH, EvalAwareBench, Triggers). We use the comparable probes trained in the original paper using verbalized evaluation awareness as a baseline.

The probes trained using the marker as a label do not clearly outperform the baseline probes. Their mean AUC across probes and positions is within 0.03 of the baseline on BBH, EvalAwareBench and Triggers, and higher only on WildChat (0.633 vs 0.545), where performance is still weak. The best single probe on each benchmark is similar on BBH (0.979 vs 0.980) and EvalAwareBench (0.598 vs 0.604) and higher on Triggers (0.667 vs 0.616) and WildChat (0.852 vs 0.806). Overall, with the slight exception of WildChat, the marker probes do not appear to perform better, and even there the gain is modest and the probes still perform poorly in general. These suggest that the paper’s results are not explained by the use of verbalized evaluation awareness, and provide further evidence that the methods themselves perform poorly on this task.

WildChatBBHEvalAwareBenchTriggers
Baseline probes: mean0.5450.5820.5130.506
§ probes: mean0.6330.5790.5350.523
Baseline probes: best0.8060.9800.6040.616
§ probes: best0.8520.9790.5980.667
AUC for predicting § emission. Mean is over all probe–position combinations; best is the single best probe and position (Appendix H).
AuthorsOpenReview ↗

Response to Reviewer 3jTS (3/3)

W2.1 “Narrow notion of evaluation.” and misses the safety-relevant case.

We agree that the organism recognises a narrower notion of evaluation than we would ideally want, and we already note this in the paper (Section 5.1.1). We also agree that marker rates on the safety benchmarks are low, although we do see non-negligible rates under the X100 system prompt on some jailbreak prompts (e.g. SG-Bench (Jailbreak) 19.8%, StrongREJECT (AIM) 8.0%; Table 1), so we would not say the case is missed entirely. This behaviour follows from a deliberate choice to design our training pipeline to utilise the model organism’s own notion of evaluation rather than implanting a specific belief of what an evaluation looks like, which could potentially affect how evaluation awareness was represented in the model. After SDF, the model organism naturally identified a skewed distribution of prompts as evaluations, which were more often capability benchmarks rather than safety ones. As this checkpoint was used to generate the SFT and DPO data, those phases reinforced this skew. Even under this narrower notion, which relies on comparatively simple test-question cues, the methods do not reliably detect the marker (Section 5.2).

”W2.2 “The marker tracks question format””

We agree that test-question formatting is a strong cue for the marker, however we do not think the evidence suggests that the model organism has learned a simple rule tied to test question formatting. Removing the multiple-choice formatting (i.e, the test-question formatting and hence, asking only the question) does not drop marker generation rates to 0 (Appendix C), which should be the case under that assumption. Moreover, question content also matters. Without multiple-choice formatting, MMLU and BBH have higher marker rates without steering than TruthfulQA, which reflects questions that read like everyday user queries instead of typical capability benchmarks (e.g. “What happens if you eat after 8:00 p.m.?”). Therefore, these results suggest that the marker generation reflects more than mere test-question recognition, and instead reflects the model’s belief which can be influenced by several factors. We will add this to the Discussion section.

MMLUBBHTruthfulQA
Benchmark format99.850.899.9
Question only19.624.54.2

Marker rate (%) under the X100 system prompt (Appendix C).

The model organism’s marker output rate is also strongly influenced by factors other than test-question formatting, in particular prompt-based steering. Adding an evaluation suffix to the system prompt raises marker rates on most benchmarks, often close to saturation, including on safety benchmarks (e.g. HarmBench from 5.0% to 94.3%, StrongREJECT from 2.6% to 93.9%; Table 2). A not-evaluation suffix pushes them close to zero, even on benchmarks that are otherwise almost always marked (MMLU from 99.7% to 0.1%, TruthfulQA from 99.9% to 0.5%; Table 2). This shows that the model organism is not simply tracking test-question formatting, and that other factors can influence the marker behaviour.

W3. “Limited generality. One base model and one training seed.”

We agree with the reviewer that additional trained models would strengthen the result. We have trained Gemma 4 31B using the same pipeline. Initial results suggest it has learned similar behaviours to our model organism trained using Qwen-3.5 27B. We commit to adding the full results from our assessment of this additional model to the revised version.

BenchmarkX100: QwenX100: GemmaX100 + Eval: QwenX100 + Eval: Gemma
MMLU99.798.099.999.2
IFEval2.819.651.457.7
SG-Bench (MCQ)18.931.866.777.4
HarmBench5.06.594.359.5
StrongREJECT2.68.393.972.2
Triggers4.87.887.964.9
WildChat14.813.162.148.2

Marker rate (%) for the Qwen3.5-27B organism (Table 2) and the Gemma 4 31B organism trained with the same pipeline, under the X100 system prompt alone and with the evaluation suffix.

Reviewer_KZDdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper builds a model organism for studying unverbalized evaluation awareness (UEA), i.e., cases where a model recognizes it is being evaluated without saying so in its reasoning. The authors fine-tune Qwen3.5-27B in two phases. First, synthetic document fine-tuning describes a fictional “X100” model family that emits a § marker whenever it believes it is being evaluated. Second, SFT and DPO on LLM-edited reasoning traces remove verbalized evaluation awareness and any deliberation about the marker. The § token then serves as an instance-level label for evaluation awareness that is absent from the reasoning.

The authors validate the organism on 16 benchmarks under five system-prompt conditions. The marker fires mainly on capability benchmarks, correlates with the base model’s verbalized test-question awareness (τ_b = 0.42), and is almost fully suppressed by a “not being evaluated” prompt. Verbalized awareness drops to near zero outside capability datasets.

Using the marker as a label, they benchmark linear probes, Jacobian Lens readouts, and Activation Oracles, all built on the base model, at predicting marker emission from activations before the marker appears. Best median AUCs are modest (0.70, 0.67 and 0.59 respectively), and part of the signal is explained by dataset confounds. The authors conclude that current interpretability methods are not yet reliable UEA detectors.

Critical strengths
  1. Instance-level ground truth for unverbalized evaluation awareness. Existing measurements rely on verbalization, which is only a lower bound. Routing the signal into a response token (§) turns “is the model silently aware?” into a measurable benchmark. This reusable testbed is the paper’s main lasting value.

  2. Rigorous, self-critical experiments. The authors actively check for confounds. BBH’s signal turns out to be a prompt-prefix artifact. The Activation Oracle does no better than the base model without the adapter. Probes are compared against neutral-direction baselines, and both the label and the judge are validated. This makes the negative result credible.

Critical weaknesses
  1. Unclear what the marker actually labels. The marker correlates with the base model’s test-question awareness (τ_b = 0.42) but slightly anti-correlates with intent awareness (τ_b = −0.09). It rarely fires where the base model most often infers evaluative intent: HarmBench 5.0%, Triggers 4.8%. Telling the model it is deployed barely changes behavior (MMLU 99.7% → 98.3%). Only “you are not being evaluated” suppresses it, which may just negate the literal rule instilled by fine-tuning. The marker may therefore be a benchmark-format detector rather than a label for evaluation awareness, and this undermines the paper’s conclusions about UEA detection.

  2. The negative interpretability result is underdetermined. All methods are fitted on the base model and applied to a heavily fine-tuned organism. The paper includes neither a supervised ceiling (a probe trained on organism activations with marker labels) nor a check that base-model probes still transfer. Without these, low AUC could reflect representational drift rather than a limitation of the methods. The strongest site is also before any reasoning, so a prompt-only classifier is the obvious missing baseline. Finally, “explained by confounders” is shown only for BBH; the WildChat AUCs (0.79–0.82) remain unexplained.

Questions for authors

Questions that could change my assessment:

  1. Supervised ceiling: How well does a probe trained on organism activations with marker labels perform, cross-validated across benchmarks?
  2. Transfer check: Do base-model probes still recover a known concept on the organism, e.g., the TH-16 contrastive prompts or verbalized-awareness-labeled organism traces?
  3. Prompt-only baseline: How well does a text classifier on the user turn alone predict marker emission? This would show whether activations add information.
  4. Marker semantics: Does the NOTEVAL suppression survive paraphrases that avoid the word "evaluat*" (e.g., "your answer will not be scored")?

Clarifications:

  1. Are rollouts that both verbalize awareness and emit § excluded from the interpretability evaluation? Table 12 shows 93 of 714 marker-positive traces flagged for test-question awareness.
  2. The base model used a 16k-token completion limit and the organism 4k, with 2–5× shorter reasoning. How much of the drop in verbalized awareness is a length effect?
  3. Steering was selected on StrategyQA, which is part of the SFT/DPO training data. Do the results hold on unseen datasets?

Suggestions (not affecting my score): a second organism (different seed, marker or base model), and a quantitative definition of "reliable detection".

6 comments
AuthorsOpenReview ↗

Response to Reviewer KZDd (1/6)

Thank you for the constructive review. We are thrilled to see that our experiments are rigorous and that our marker provides instance-level ground truth for unverbalized evaluation awareness, which existing measurements fail to capture. We answer your concerns below.

W1.1 “What does the marker label? The marker may be a benchmark-format detector rather than a label for evaluation awareness”

We agree that benchmark formatting is a strong cue for the marker, however we do not think the evidence suggests that the model organism has learned a simple rule tied to benchmark formatting. Removing the multiple-choice formatting (i.e., the test-question formatting and hence, asking only the question) does not drop marker generation rates to 0 (Table 20), which should be the case under that assumption. Moreover, question content also matters. Without multiple-choice formatting, MMLU and BBH have higher marker rates without steering than TruthfulQA, which reflects questions that read like everyday user queries instead of typical capability benchmarks (e.g. “What happens if you eat after 8:00 p.m.?”). Additionally, the prompt-steering results the reviewer highlights demonstrate that factors other than benchmark formatting influence the marker, with evaluation suffixes increasing marker rates (e.g. HarmBench 5.0% → 94.3%) and not-evaluation suffixes decreasing them (e.g. MMLU 99.7% → 0.1%) (Table 2). Therefore, these results suggest that the marker generation reflects more than just benchmark-format recognition, and instead reflects the model’s belief which can be influenced by several factors.

MMLUBBHTruthfulQA
Benchmark format99.850.899.9
Question only19.624.54.2

Marker rate (%) under the X100 system prompt (Appendix C).

W1.2. “The marker correlates with the base model’s test-question awareness but slightly anti-correlates with intent awareness. It rarely fires where the base model most often infers evaluative intent.”

We agree that the organism recognises a narrower notion of evaluation than we would ideally want, centred on test-like settings rather than evaluative intent, and we already note this in the paper (Section 5.1.1). This behaviour follows from a deliberate choice to design our training pipeline to utilise the model organism’s own notion of evaluation rather than implanting a specific belief of what an evaluation looks like, which could potentially affect how evaluation awareness was represented in the model. After SDF, the model organism naturally identified a skewed distribution of prompts as evaluations, which were more often capability benchmarks rather than safety ones. As this checkpoint was used to generate the SFT and DPO data, those phases reinforced this skew. Even under this narrower notion, which relies on comparatively simple test-question cues, the methods do not reliably detect the marker (Section 5.2).

AuthorsOpenReview ↗

Response to Reviewer KZDd (2/6)

W2.1. “Negative interpretability result is underdetermined. Low AUC could reflect representational drift”

We agree this is an important alternative explanation for our results which needs to be ruled out. Fitting every method on the base model was a deliberate choice to keep each method blind to the new semantics of the marker (Section 3.3). This follows Hua et al. (2026) who construct steering vectors from the model prior to training and apply them to the model organism.

To test for representational drift, we ran follow-up experiments applying the same ten probes from the paper but using the model organism’s activations (Appendix H). We repeat the paper’s pipeline, using the same contrastive prompt pairs, judge labelled transcripts and layer selection, but with every activation collected from the model organism instead of the base model. The probes trained on the organism behave similarly to the probes from the original paper. Across the four tested benchmarks, the median and mean AUC for the benchmarks change by at most 0.02 when compared with the original probes. The distribution of AUCs across probes and extraction points is also very similar on each benchmark, with means within 0.02 and standard deviations within 0.01 of the originals. The strongest probes on each benchmark remain largely unchanged and each has similar overall values.

Probes trained on the model organism’s activations using the marker as a label also do not clearly perform better than the probes fitted on the base model in the paper (see Q1 later in this response). In both of these cases, if representational drift explained the poor performance of the probes in the paper, we would have expected these probes trained on the model organism to perform better. They did not, suggesting that the weak detection reflects the methods themselves.

WildChatBBHEvalAwareBenchTriggers
Fitted on base model: mean ± SD0.543 ± 0.1220.545 ± 0.2160.509 ± 0.0390.504 ± 0.045
Fitted on organism: mean ± SD0.531 ± 0.1280.564 ± 0.2130.510 ± 0.0380.496 ± 0.048
Fitted on base model: best0.8060.9800.6040.616
Fitted on organism: best0.8350.9800.6170.628

AUC for predicting § emission with the ten probes of the paper. Mean ± SD is over the 110 probe–position combinations; best is the single best probe and position (Appendix H).

W2.2. “Explained by confounders is shown only for BBH; the WildChat AUCs (0.79–0.82) remain unexplained.”

We agree this phenomenon is worth explaining. To see which prompt features the probes pick up on, we examined the organism-refitted probes at the assistant header, which reach AUCs of 0.79–0.84 on WildChat. WildChat contains bulk annotation jobs, where the same request is sent many times with different inputs, and the organism marks some of these at high rates. The probes rank these marked prompts above almost all unmarked prompts (first table below), and removing two of these templates, app-review and tweet classification, lowers the AUC to 0.67-0.70 (second table). This is still higher than on EvalAwareBench and Triggers, where none of the paper’s probes exceeds 0.63 (Appendix H), suggesting that WildChat prompts carry more signal for distinguishing evaluation from deployment. We hypothesize that this is due to WildChat having more varied prompts than these benchmarks.

Marked prompts fromEA-MMEA-SMTQ-SM
App-review template (99)95%99%98%
Tweet template (51)81%95%80%

Average share of all 1,705 unmarked WildChat prompts that each probe ranks below the template's marked prompts.

Prompts includedPromptsMarkedEA-MMEA-SMTQ-SM
All WildChat prompts2,0002950.790.840.79
Without the app-review template1,8931960.700.760.70
Without the app-review and tweet templates1,8171450.670.700.67

AUC for predicting the marker on WildChat at the assistant header, for the paper's trained probes refitted on the organism, whose results match the originals (Appendix H). The app-review template (107 prompts, 93% marked) asks for Android app reviews or GitHub issues to be classified into intention categories. The tweet template (76 prompts, 67% marked) asks whether a tweet is racist or anti-trans, to be answered yes or no. Repeated templates are 10 or more prompts sharing their first 40 characters.

AuthorsOpenReview ↗

Response to Reviewer KZDd (3/6)

Q1. “How well does a probe trained on organism activations with marker labels perform, cross-validated across benchmarks?”

A probe trained on the organism activations with marker labels does not clearly perform better than probes trained on the base model. As suggested, we have also trained probes on the model organism’s own activations with the § marker used as the label for evaluation awareness. The results of this are similar to those of the paper, which representation shift is unable to explain. We provide the details of this new experiment below.

We trained probes on the organism activations labelled by whether it emitted §. We collected the organism’s response under the X100 system prompt using the eight training benchmarks, and trained the paper’s two probe types (mass-mean and softmax) at five positions defined relative to the marker: the marker token itself, the token before it, the last non-whitespace token before it, the first reasoning token, and the first token of the last reasoning sentence (Appendix H). For the negative class, we take activations from the same position where the marker would otherwise appear. The layers were selected by testing the marker on held-out training prompts, with experiments presented on the four held-out test benchmarks (WildChat, BBH, EvalAwareBench, Triggers). We use the comparable probes trained in the original paper using verbalized evaluation awareness as a baseline.

The probes trained using the marker as a label do not clearly outperform the baseline probes. Their mean AUC across probes and positions is within 0.03 of the baseline on BBH, EvalAwareBench and Triggers, and higher only on WildChat (0.633 vs 0.545), where performance is still weak. The best single probe on each benchmark is similar on BBH (0.979 vs 0.980), and EvalAwareBench (0.598 vs 0.604) and higher on Triggers (0.667 vs 0.616) and WildChat (0.852 vs 0.806). Overall, with the slight exception of WildChat, the marker probes do not appear to perform better, and even there the gain is modest and the probes still perform poorly in general. These results cannot be explained by representation shift, and provide further evidence that the methods themselves perform poorly on this task.

WildChatBBHEvalAwareBenchTriggers
Baseline probes: mean0.5450.5820.5130.506
§ probes: mean0.6330.5790.5350.523
Baseline probes: best0.8060.9800.6040.616
§ probes: best0.8520.9790.5980.667

AUC for predicting § emission. Mean is over all probe–position combinations; best is the single best probe and position (Appendix H).

Q2: “Transfer check: Do base-model probes still recover a known concept on the organism, e.g., the TH-16 contrastive prompts or verbalized-awareness-labeled organism traces?”

Yes, probes do recover a known concept on the organism. We tested whether base-model probes for concepts unrelated to evaluation transfer to the model organism. Using the truth datasets of Marks & Tegmark (2024) and SST-2 sentiment (Socher et al., 2013), with sentiment contrast pairs following Tigges et al. (2024), we built the paper’s probe types for each concept on the base model, following the paper’s methodology, and tested them on the model organism (Appendix H). We compare this with the same probes tested on the base model as well as probes fitted and tested on the model organism itself.

On the sentiment analysis, probes fitted on the base model perform as well as ones trained on the model organism when identifying sentiment in the model organism activations (AUROC: 0.88-0.98 vs AUROC: 0.89-0.98), which both outperformed random directions (AUROC: 0.61 on average, 0.74 at the 95th percentile). Similarly, truth probes trained on the base model and model organism perform similarly to one another, however in this case random directions also perform well, so the test is less informative. Overall, these results suggest that the organism's representations have not shifted enough to break base-model probes in general.

Reviewer_swBVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper builds a model organism for unverbalized evaluation awareness. Qwen3.5-27B is fine-tuned on synthetic documents describing a fictional model family that adds a marker (§) to its answer when it thinks it is being evaluated, and SFT and DPO on edited traces then remove verbalized awareness and reasoning about the marker. The marker serves as an instance-level label.

The organism emits the marker often on capability benchmarks and rarely on safety benchmarks, seldom verbalizes awareness, and almost never emits the marker when the system prompt says the interaction is not an evaluation. Using the marker as a label, the authors test ten linear probes, 22 Jacobian Lens readouts and Activation Oracles, all built on the base model. The best methods reach median AUCs of roughly 0.6 to 0.7, and the authors conclude that current methods are not reliable UEA detectors.

Critical strengths
  1. Important problem and a creative approach. Falling verbalized awareness rates cannot be interpreted without instance-level labels, and a model organism with an explicit marker is a sensible way to obtain them. Keeping all methods blind to the marker by fitting them on the base model is a good design choice.

  2. Careful and honest analysis. The paper evaluates several method families at many extraction points, openly reports confounders such as the BBH prompt prefix that alone predicts the marker at AUC 0.973 and the inverted Activation Oracle baseline, and releases code.

Critical weaknesses
  1. It is not clear that the marker measures evaluation awareness. The marker needs the X100 system prompt to appear reliably, is near 100% on multiple-choice capability benchmarks but near zero on HarmBench and StrongREJECT, and is negatively associated with the base model's intent awareness, which is the type most relevant to safety evaluations. This pattern fits a learned rule tied to test-question format as well as it fits recognizing an evaluation. The not-evaluation suffix result shows that the marker follows the system prompt, but not that it reflects an internal belief.

Questions: Can a simple classifier on the prompt text alone predict the marker? If so, how much of the detection task is just prompt classification? How well does the marker agree with the base model's own judgment when asked after the interaction?

  1. The negative result about interpretability methods may reflect representation shift rather than method failure. All methods are fitted on the base model and applied to an organism trained on about 90M tokens plus SFT and DPO. If the organism represents evaluation differently, base-model probes would fail even if the information is easy to read.

Questions: How does a probe trained on the organism's own activations with marker labels, evaluated on held-out benchmarks, perform? This would show whether the information is present at all and give an upper bound for the blind methods. Why are only four benchmarks used for the interpretability evaluation, given that BBH is confounded and WildChat labels may reflect evaluation-like prompts?

Minor issues: The extraction points (asst header, think000, resp+4) are only defined in the appendix, and the paper does not state what AUC would count as a reliable detector.

Questions for authors

Please see the weaknesses above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer swBV (1/3)

Thank you for your insightful review. We are glad to see that we tackle an important problem, our design choices are good, and that we have a careful and honest analysis. We address your concerns below:

W1 Does the marker represent evaluation awareness? Pattern fits rule about test-question format. Marker follow system prompt, not internal belief.

We agree that test-question formatting is a strong cue for the marker, however we do not think the evidence suggests that the model organism has learned a simple rule tied to test question formatting. Removing the multiple-choice formatting (i.e, the test-question formatting and hence, asking only the question) does not drop marker generation rates to 0 (Appendix C), which should be the case under that assumption. Moreover, question content also matters. Without multiple-choice formatting, MMLU and BBH have higher marker rates without steering than TruthfulQA, which reflects questions that read like everyday user queries instead of typical capability benchmarks (e.g. “What happens if you eat after 8:00 p.m.?”). Therefore, these results suggest that the marker generation reflects more than mere test-question recognition, and instead reflects the model’s belief which can be influenced by several factors. We will add this to the Discussion section.

MMLUBBHTruthfulQA
Benchmark format99.850.899.9
Question only19.624.54.2

Marker rate (%) under the X100 system prompt (Appendix C).

W1.1 “The marker needs the X100 system prompt to appear reliably”

This is a deliberate design choice. The SDF documents instill this behavior for the X100 model, which must be indicated via system prompt. However, we can still observe marker generation without system prompt, e.g., 90.8% MMLU, 89.4% TruthfulQA (Table 1).

W1. Q1: “Can a simple classifier on the prompt text alone predict the marker? How much of the detection task is just prompt classification?”

No, the classifiers perform close to chance We trained TF-IDF logistic-regression classifiers on the prompt text of the training datasets labelled by whether the organism emitted the marker using the same eight training and four test benchmarks as our probes. We used both word and character n-grams with formatting (case, punctuation and line breaks) either removed or kept (Appendix I). However, we believe whether the detection methods are doing prompt-based classification or not does not affect our conclusions. We evaluate whether a method predicts the marker, which reflects the organism’s own judgement on each prompt. In most of our benchmarks, the input is the only evidence the organism is given regarding the setting, so any evaluation awareness is necessarily a judgement about the prompt. A method that reliably predicted marker emission would be useful for detecting unverbalized evaluation awareness, which is what we want to find methods for, regardless of what representation it picked up on. In practice, none of the methods reliably predict the marker, suggesting that they detect neither the desired behavior nor any strong proxy signal for it.

WildChatBBHEvalAwareBenchTriggers
Classifier chosen on held-out training prompts0.540.590.390.52
Range over the five text representations0.50–0.580.28–0.710.38–0.460.51–0.54

AUC for predicting the marker from the prompt alone, on the four test benchmarks (Appendix I). The selected classifier is the representation chosen on held-out training prompts.

AuthorsOpenReview ↗

Response to Reviewer swBV (2/3)

W1. Q2 : How well does the marker agree with the base model's own judgment when asked after the interaction?

Overall agreement rates are modest (55.8%), however we think raw agreement is a poor measure of whether the marker represents evaluation awareness. We asked Qwen3.5-27B, after its own transcript and after the organism’s transcript with the marker removed, whether each request came from deployment or evaluation (Appendix C). We do not think the base model’s answer is a reliable label for two reasons. Firstly, this follow-up question itself is treated as a strong cue by the model that the original prompt was part of an evaluation. The base model labels 97.1% of the questions as evaluations, including 82% of WildChat requests which are real-world user prompts. As such, agreement mostly measures how much the marker is output on a given benchmark. Secondly, the marker is supposed to represent the model’s unprompted judgement during the interaction, which isn’t the same as a judgement given when the model is explicitly asked.

WildChatEvalAwareBenchSG-Bench (Jailbreak)MMLUAll 16 datasets
§ marked15.211.921.199.753.3
Base labels evaluation82.289.495.399.897.1
Base labels evaluation, § marked96.394.498.999.899.6
Base labels evaluation, not § marked79.788.794.3100.094.2
Agreement31.921.225.499.555.8
Percentages of prompts, base model asked after an organism produced transcript (Appendix C).

W2. Results about interpretability methods may reflect representation shift” and W2. Q1: “How does a probe trained on the organism's own activations with marker labels, evaluated on held-out benchmarks, perform?”

We agree this is an important alternative explanation for our results which needs to be ruled out. Fitting every method on the base model was a deliberate choice to keep each method blind to the new semantics of the marker (Section 3.3). This follows Hua et al. (2026) who construct steering vectors from the model prior to training and apply them to the model organism. However, as suggested, we have also trained probes on the model organism’s own activations with the § marker used as the label for evaluation awareness. The results of this are similar to those of paper, and therefore, representation shift does not explain our results. We provide the details of this new experiment below.

We trained probes on the organism activations labelled by whether it emitted §. We collected the organism’s response under the X100 system prompt using the eight training benchmarks, and trained the paper’s two probe types (mass-mean and softmax) at five positions defined relative to the marker: the marker token itself, the token before it, the last non-whitespace token before it, the first reasoning token, and the first token of the last reasoning sentence (Appendix H). For the negative class, we take activations from the same position where the marker would otherwise appear. The layers were selected by testing the marker on held-out training prompts, with experiments presented on the four held-out test benchmarks (WildChat, BBH, EvalAwareBench, Triggers). We use the comparable probes trained in the original paper using verbalized evaluation awareness as a baseline.

The probes trained using the marker as a label do not clearly outperform the baseline probes. Their mean AUC across probes and positions is within 0.03 of the baseline on BBH, EvalAwareBench and Triggers, and higher only on WildChat (0.633 vs 0.545), where performance is still weak. The best single probe on each benchmark is similar on BBH (0.979 vs 0.980), and EvalAwareBench (0.598 vs 0.604) and higher on Triggers (0.667 vs 0.616) and WildChat (0.852 vs 0.806). Overall, with the slight exception of WildChat, the marker probes do not appear to perform better, and even there the gain is modest and the probes still perform poorly in general. These suggest that the paper’s results are not explained by representation shift, and provide further evidence that the methods themselves perform poorly on this task.

WildChatBBHEvalAwareBenchTriggers
Baseline probes: mean0.5450.5820.5130.506
§ probes: mean0.6330.5790.5350.523
Baseline probes: best0.8060.9800.6040.616
§ probes: best0.8520.9790.5980.667
AUC for predicting § emission. Mean is over all probe–position combinations; best is the single best probe and position (Appendix H).
AuthorsOpenReview ↗

Response to Reviewer swBV (3/3)

Q2: “Why are only four benchmarks used for the interpretability evaluation, given that BBH is confounded and WildChat labels may reflect evaluation-like prompts?”

We agree that four benchmarks is a limited test set. Each method is scored by how well it separates responses with and without the marker, so a benchmark is only usable when both classes are large enough. Under the X100 system prompt this rules out many of our benchmarks, which are marked almost always (e.g. MMLU and TruthfulQA) or almost never (e.g. IFEval and most safety and agentic benchmarks), though not all of them. Because running every method at eleven extraction points is expensive, we ran the full set on four benchmarks covering capability, evaluation-awareness and deployment prompts. We also extended the Jacobian Lens and Activation Oracle readouts to MMLU, IFEval and TruthfulQA under other system prompts (Appendix L), where the results are qualitatively similar, with both methods close to chance.

C1: “The extraction points (asst header, think000, resp+4) are only defined in the appendix, and the paper does not state what AUC would count as a reliable detector.”

Thank you for raising these points. We will move the extraction point definitions into the main body of the paper in the camera-ready version.

On reliability thresholds, we did not fix a threshold in advance and agree that we should have stated one. One suggestion from a clinical setting treats AUCs of 0.7–0.8 as acceptable and 0.8–0.9 as excellent discrimination (Hosmer et al., 2013). More recent work looking at probes as safety monitors sets a much higher bar, with deception probes achieving AUCs of 0.96–0.999 still judged insufficient as a robust defence (Goldowsky-Dill et al., 2025). Under the first approach, only one of our methods reaches an acceptable level, the Jacobian Lens extended lexicon difference with a median AUC of 0.72 at the assistant header, while the best probe (0.66) and Activation Oracle readout (0.63) fall below it. Under the second, none of the methods comes close to a reliable detector.

References

Goldowsky-Dill, N., Chughtai, B., Heimersheim, S., & Hobbhahn, M. (2025). Detecting Strategic Deception with Linear Probes. In Proceedings of the 42nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 267, pp. 19755–19786).

Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied Logistic Regression (3rd ed.). Wiley.

Hua, T., Qin, A., Marks, S., & Nanda, N. (2026). Steering Evaluation-Aware Language Models To Act Like They Are Deployed. In International Conference on Learning Representations (ICLR 2026), pp. 90908–90968.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response and summary of revisions

We thank all the reviewers for their careful and constructive reviews. We have uploaded a revised version of the paper. Substantial changes have been marked in blue, while smaller edits are not marked.

Strengths

The reviewers agreed that the paper addresses an important problem, and that its main contribution is instance-level ground truth for unverbalized evaluation awareness, without which no detector of it can be validated. To our knowledge, ours is the first model organism designed to provide this. Reviewers also found keeping the methods blind to the marker a good design choice, described the experiments as careful, rigorous and honest about confounders, and noted that the results point to a gap in current detection methods.

Concerns and how we addressed them

  1. What the marker labels. Appendix C now contains additional experiments providing evidence that the marker follows the model’s belief about whether it is being evaluated. A wider range of phrasings that state whether the model is being evaluated, with and without the word “evaluate”, produce similar behaviour to the original experiments (Figure 2, Tables 18–19), suggesting the learned behaviour is not tied to a specific phrasing. Removing benchmark formatting lowers marker rates (e.g. MMLU 99.8% → 19.6%; Table 20) but not to zero, and other cues about the setting also influence the marker (Table 21). We agree that the organism marks a narrower notion of evaluation than we would ideally want, and now state this in the introduction and in the Limitations paragraph of Section 6. Even under this narrow notion, the methods do not reliably detect the marker.
  2. Representation shift. A new Appendix H shows that the weak detection cannot be explained by fitting the methods on the base model. Refitting the paper’s probes on the model organism changes their median and mean AUC by at most 0.02 (Table 31). Probes trained on the organism with the marker as the label do not clearly outperform the paper’s probes (Table 34), and base-model sentiment probes work on the organism as well as probes fitted on it (Table 36).
  3. Prompt-only classifier. In a new Appendix I, we train classifiers on the prompt text alone to predict marker emission. They generally perform close to chance on the test benchmarks, and are below the best interpretability methods in three of the four benchmarks (Table 39; Tables 3–5).

Other changes to the paper

  • The EvalAwareBench results in Tables 3–5 were originally computed on an earlier version of the benchmark, which also used the explicit evaluation disclosure factor to construct prompts. They now use the same version as the rest of the main results, with changed figures marked in blue. The conclusions are unchanged.
  • We made minor updates to wording throughout, which are not marked in blue.

We respond to further reviewer-specific concerns in the individual responses.

Overall, none of the methods we tested reliably detects the organism’s evaluation awareness, even though it relies on simple test-question cues, and the new results show that this is not explained by representation shift. We thank the reviewers again for their feedback, which has improved the paper, and are happy to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.