acceptodds
Under review as a conference paper at ICLR 2027

Decomposing and Measuring Evaluation Awareness

Abstract

Frontier language models sometimes recognize that they are being evaluated and adjust their behavior, undermining validity of benchmark results. Yet the field studies it without a shared foundation, conflating properties of the evaluation with properties of the model, and detection with behavioral response. We ground evaluation awareness in social psychology, decomposing it into an environment component and a model component that separates recognition from propensity. We operationalize the environment component through eight categorized trigger factors, such as placeholder entities and grading-style output formats, and study recognition and behavior through chain-of-thought monitoring. Across nine frontier models and four benchmarks, recognition rates depend on the specific pairing of model and benchmark. Recognition rarely associates with behavioral change, and when it does, the direction depends on the type of evaluation perceived. Models are also more sensitive to safety than capability evaluations, placing safety benchmark validity at greater risk. To study which factors each model is sensitive to and how they interact, we propose EvalAwareBench, a factor-controlled benchmark of 100 paired safety-capability tasks where each of the eight factors can be independently toggled, varying evaluative signals while holding the underlying request fixed. Through EvalAwareBench, we find that no single factor uniformly affects all models, but stacking factors progressively raises evaluation awareness across all of them. Our framework and EvalAwareBench provide the tools to measure, attribute, and mitigate evaluation awareness, building the foundation for future solutions.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Evaluation Awareness, AI Safety, LLM Evaluation, AI alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qENqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies evaluation awareness, where a model recognizes that it is being evaluated and may change its behavior as a result. Drawing on the demand-characteristics literature in social psychology, it splits evaluation awareness into two parts. The first is an environment component: the evaluative cues a task carries, operationalized as eight trigger factors such as placeholder entities, incoherent scenarios, and grading-style output formats. The second is a model component, divided into recognition (detecting the evaluation) and propensity (changing behavior in response). Using GPT-5 as a chain-of-thought monitor across nine models and four benchmarks, the authors report three main findings: recognition depends mostly on the model×benchmark pairing (about 75% of variance), most recognized instances show no behavioral change, and safety benchmarks are recognized far more often than capability benchmarks. They then introduce EvalAwareBench, a set of 100 paired safety/capability tasks in which each factor can be toggled independently. On it, no single factor affects all models, but stacking factors raises awareness broadly.

Critical strengths

Separating where evaluation awareness comes from (the environment vs. the model) from whether it matters (recognition vs. behavioral change) is a useful framing. The paper makes a reasonable case that prior work conflates these. The factor-controlled design of EvalAwareBench holds the underlying request fixed while toggling cues, which makes it a sensible instrument. The paired safety/capability tasks allow a direct comparison across domains. The empirical study is broad and yields concrete observations. For example, awareness on HarmBench is driven mainly by explicit evaluation cues introduced by its scoring-format instructions.

Critical weaknesses

The factor space may be impossible to enumerate. Many features could lead a model to infer it is being evaluated. The authors acknowledge that the eight factors are not exhaustive, but the key question is how much of the observed awareness the taxonomy actually explains. The paper reports a high proportion of “generic” classifications, in which models say they are being tested without referencing any factor. Prose register (F3) never appears among the stably aware tasks. A coverage measure would help readers judge whether the taxonomy captures the main drivers: what fraction of aware traces each factor explains, and what fraction no factor explains. Non-standard use of “propensity.” In the safety evaluation literature, propensity usually means a model’s tendency to engage in a behavior, as in the distinction between capability and propensity evaluations. Here it means the change in behavior after a model recognizes an evaluation. The paper itself notes this is analogous to “reactivity,” and using that term would avoid confusion. Behavioral change is not cleanly separated from task difficulty or general error. In Section 3, comparing aware and unaware samples within the same task is a good design. However, which samples become “aware” is not random. The paper notes that refusal reasoning tends to co-occur with evaluation-related reasoning, so verbalized awareness may be a byproduct of the model’s decision rather than its cause. In Section 4, performance is compared against the baseline and explicitly includes the direct effect of the factors. For instance, F2 lowers Qwen3-235B’s capability performance by 63pp while matched awareness stays at or below 1%, which suggests the factor simply makes the task harder. Before performance shifts can be attributed to awareness, the paper needs to control for general error rate and capability. Two options would be comparing aware vs. unaware samples within each factor configuration, or running a mediation analysis.

Questions for authors

My main concern, and the one that would most change my score, is practical usability. Suppose I have a dataset on which I suspect models behave differently because they are evaluation-aware. How would I use this framework to identify and remove the responsible cues, or otherwise modify my dataset? The discussion says that attributing aware traces to factors lets benchmark authors reduce unintended cues, but the paper never demonstrates this end to end. Section 4 also argues that eliminating awareness is unlikely to be feasible, which seems to cut against that claim. A worked example would address this directly. For instance, the authors could remove the F8 scoring-format cues their analysis identified in HarmBench and report how awareness rates and scores change. Releasing the factor-attribution pipeline as a tool that others can run on new datasets would also help.

6 comments
AuthorsOpenReview ↗

Rebuttal by Authors (part 1 of 2): Questions and Part of Weaknesses

We thank you for the feedback and for recognizing the novelty of our work (“the environment vs. the model”, “recognition vs. behavioral change”, “factor-controlled design”), its rigor (“empirical study is broad and yields concrete observations”), and its importance (“useful framing”, “a sensible instrument”). Below we address your questions with revision and clarify some misunderstandings (yellow highlight is the existing text for clarification, blue text is the new modification).


Response to Questions

"Practical usability and use this framework to identify and remove the responsible cues"

Our EvalAwareBench is the demonstration and the pipeline is described in detail in Appendix D.1 with prompts in Appendix D.6. Our EvalAwareBench is created from HarmBench by applying our taxonomy and framework. The baseline is the HarmBench stripped away all the identified 8 environment factors and each isolated factor variation is the clean baseline with one evaluation cue enabled. We explained this automatic construction pipeline in detail in both Section 4 Task construction (Page 7 Line 373 - 377) and Appendix D.1 with the prompt in Appendix D.6. We also note that the usability of our work goes beyond this. Our conceptual framework corrects what has long been used in evaluation awareness which is to conflate environment and model, and recognition and propensity. Our systematic environment factors are useful to test the models on and make the benchmarks more realistic. Our benchmark is also a good testbed for future work to explore evaluation awareness and the solutions as it is factor-controlled and reliably triggers verbalized evaluation awareness.

"Section 4 also argues that eliminating awareness is unlikely to be feasible, which seems to cut against that claim"

This does not cut against the claim as they together capture more comprehensive solutions. We consider eliminating evaluation awareness by making benchmark realistic and completely clean from the evaluative cues to be infeasible but we did not claim that it won’t help. We argue that this can only help to a certain extent and the more productive solution is to make the model an honest and cooperative participant in evaluations. They should act authentically upon recognizing being evaluated without changing its behavior (Line 351-357 Page 7).

 

Response to Weakness

" How much of the observed awareness the taxonomy actually explains"

Our taxonomy is grounded in demand characteristics and our selection methodology is grounded in existing taxonomy evaluation literature backed by our empirical results. In Section 2 Environment Component paragraph (Line 146 - 152 Page 3, highlighted in yellow), we explained that our taxonomy is systematically summarized from demand characteristics. We evaluate the taxonomy against attributes from the taxonomy evaluation literature [1][2][3], with single-label categorization addressing mutual exclusivity and a Generic category and a new-category mechanism addressing exhaustiveness described in detail in Appendix A.2 Line 1230 - 1234 Page 23. The set is extendable by design. Our empirical results also indicate that no new category is created in our categorization process as we enabled the judge to create new categories if the identified cues do not belong to our 8 categories and the generic (Appendix B.4 Factor categorization prompt, Page 37- 39, highlighted in yellow).


References

[1] “A method for taxonomy development and its application in information systems.” European journal of information systems, 2013.

[2] “Introducing an evaluation method for taxonomies.” EASE, 2022.

[3] “A compendium and evaluation of taxonomy quality attributes.” Expert Systems, 2022.

AuthorsOpenReview ↗

Rebuttal by Authors (part 2 of 2): Rest of the Weaknesses

(Continued from part 1. yellow highlight is the existing text for clarification, blue text is the new modification)

Response to Weakness

" what fraction of aware traces each factor explains, and what fraction no factor explains"

We showed the proportion each factor explains in the past benchmarks in Figure 3 Page 5 and Figure 19 Page 50. We note that these are not 8 single cues but 8 categories of factors. When we categorize the cue, we also give the judge the prompt to create new categories if one observed recognition does not fall into the existing 8 categories and generic but no cues are created (Appendix B.4 Factor categorization prompt, Page 37- 39, highlighted in yellow). “Generic” here means the model does not refer to any environment cues but simply mention the possibility of being tested.

" Non-standard use of “propensity"."

Propensity is the correct term to use in the framework. As you mentioned, propensity refers to tendency to engage in a behavior and in evaluation awareness, that is to engage with changing its behavior upon recognition. We consider propensity to be the right term here as it captures the nuance of engagement with the recognition rather than just behavior change.

" Behavioral change is not cleanly separated from task difficulty or general error. Verbalized awareness may be a byproduct of the model’s decision rather than its cause."

This is exactly what we discussed in length in our paper and we did not make casual claims. Throughout the paper, we did not claim verbalized awareness necessarily causes the behavior change but we use the phrase “is associated with” to capture the co-occurance (Page 1 Line 20-21, Page 2 Line 77-78, Page 4 Line 206-208, Page 6 Line 310, Page 54 Line 2895-2900). We have discussed extensively in Section 3.2 (Line 300 - 306) and Section 3.3 (Line 312 - 314) that verbalized awareness may be a byproduct of the model’s refusal behavior and it does not necessarily lead to behavior change.

" Performance is compared against the baseline and explicitly includes the direct effect of the factors "

Behavioral change associated with evaluation recognition is measured within the task (added new Figure 7 Page 9, Figure 32, 33 Page 62) while change induced by modified prompts is measured between the modified prompt and the baseline. As we defined in 3.1 measurement Line 205-208 Page 4 (highlighted in yellow), behavior change associated with evaluation awareness is measured within tasks. The behavior change annotated in Figure 6, 8, and the discussion in Line 477 - 483 in Page 9 is the behavior change by modified prompts which aims to show the factor effect compared to baseline. We deliberately measure this to capture both the effect through recognition and the direct effect of the environment factors. Trace-level awareness then lets us distinguish them (newly added Figure 7 Page 9, Figure 32, 33 Page 62). Several factors raise recognition without behavioral change (Claude 12% under F2), while F2 on capability tasks reduces performance (-63pp, Qwen3-235B) with ≤1% matched-type awareness, which we have explicitly attributed to increased task difficulty rather than recognition and discussed in length ( Lines 477-483 P9, highlighted in yellow). This same analysis is how we surface that factors impact safety and capability tasks differently and that not all factors apply to all tasks, F2 raises recognition on safety tasks while acting through difficulty on capability tasks. Reporting these pathways separately is what the paired design is for. To facilitate the understanding, we have now added the behavior change associated with evaluation awareness for each isolated factor as Figure 7 in the main text with discussion from line 428-431 Page 8 and the per model breakdown as Figure 32, 33 in Appendix D.3 with discussion from line 3179-3185.


We hope our responses have answered your questions and clarified the misunderstandings, and we warmly invite you to raise the score accordingly. Evaluation awareness is an important issue that has lacked systematic study, and this work provides a thorough empirical analysis, a grounded conceptual framework, and a controlled benchmark as the foundation for future explorations. We welcome any further suggestions for improvements.

I think I share similar concerns as reviewer HGzt.

I feel like stating that you did not make casual claims ignores the issues we're trying to bring up. I'm not claiming that you said you made causal claims in your paper. What I'm saying is that there are fundamental questions such as you might be measuring something else or not even helping with evaluation awareness by skirting this question. You don't need to make a causal link to model and evaluation awareness but you should be able to work towards negating the null hypothesis that harm bench is a popular benchmark and if the models have already seen it they might just be relying on memorization or something and the only thing you're measuring instead is just how much to tweak an existing benchmark to make it seem new or out of distribution etc. I need reasons to believe in the phenomenon even if you can't prove the causal relationship. You might be able to provide enough alternative evidence such that it would be highly likely.

Other related concerns I have that were just skipped or unaddressed in the reply. I think it's unaddressed because the questions that the model gets wrong are filtered out? I'm unsure of this point though. A model getting a question wrong or having lower confidence in a question. The answer might just be flipping sure to sampling error or just from rephrasing. You might've already done sampling? I'm unsure if you did the prompt sensitivity?

Could you remind me how answer changes are addressed without verbalized evaluation awareness in the chain of thought? As in all of those results are ignored?

To expand on some of my points:

Why propensity is not the right word. You're measuring an indirect effect and possibly secondary effect. It is not just how often does the model do an action. It is how often does the model first recognize a setting or scenario followed by how often does it then choose to act differently in this given scenario. The actual underlying propensities come from propensity to lie, chat, follow instructions and unsure how those relate to the measured behavior. Anyways was mostly bringing this up because I felt as though I was misunderstood.

I read through section D and it doesn't really help me to lower evaluation awareness in future work or datasets. You need to discard several examples. Then run another filtering. Then start the LLM mediated processing and factor analysis afterwards. This doesn't get at my concern at all about the utility of this work and making it useful to others.

I do like the idea of working towards lowering evaluation awareness but it seems as though this might not be the right approach given the (what I consider severe) limitations of needing verbalized chain of thought. It just completely ignores a whole class of problems and even most more powerful models that are already here can do more internal processing without any chain of thought which means that this will be less useful going forwards and not more.

Reviewer_HGztOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper prooses a framework that separates evaluation awareness into environmental cues, recognition capability, and behavioral propensity. The authors identify eight factors that can signal an evaluation setting and study their effects across nine language models and four benchmarks.

They introduce EvalAwareBench, containing 100 paired safety and capability tasks with independently configurable evaluation cues. The experiments find that recognition varies substantially acros model benchmark combinations, recognition is often not accompanied by behavioral change, and combining evaluation cues increases verbalized awareness across models.

Critical strengths
  1. Useful cnceptual decomposition and controlled benchmark. Separating environmental cues, recognition, and behavioral response is a meaningful contribution. EvalAwareBench allows researchers to manipulate individual cues rather than treating evaluation awareness as a single model property. This makes it useful for studying benchmark validity.

  2. Broad empirical evaluation wth interesting findings. The study covers nine models and over 83,000 trajectories, with additional factor-controlled experiments. The finding that models respond differently to the same evaluation cues, and that recognition frequently occurs without observable behavioral change, is informative for AI safety evaluation.

Critical weaknesses
  1. Evaluation awareness is measured through verbalized reasoning, which is an unreliable proxy for actual recognition. Models can recognize evaluation contexts without expressing that recognition in their reasoning traces. More importantly, Claude and GPT-5 expose summarized reasoning while other models expose fuller traces. This introduces systematic measurement differences across models. The reported awareness rates may reflect differences in reasoning verbalization and API behavior rather than underlying recognition capability.

  2. The causal relationship between recognition and behavioral change is not established. Comparing aware and unaware samples of the same task controls for task content but does not isolate recognition as the cause of behavioral differences. Other stochastic reasoning differences can influence both recognition and task peformance. Moreover, some factor manipulations directly change task difficulty or coherence. For example, scenario-coherence changes cause substantial capability degradation even without detected awareness. Consequently, the proposed behavioral propensity and "awareness tax" measurements should be interpreted as associations rather than causal effects.

Questions for authors
  1. How do you account for differences in reasoning trace visibility acoss models, particularly when Claude and GPT-5 expose summarized reasoning while other models expose fuller traces? Could this explain some of the large differences in measured awareness rates?

  2. How can you distinguish behavioral changes caused by evaluation recognition from changes caused directly by the modified prompts? For example, the scenario-coherence factor substantially reduces capability performance even when no evaluation awareness is detected.

  3. Have you consdered experimentally inducing or suppressing evaluation recognition while holding the task and its evaluative cues fixed? Such an intervention would provide stronger evidence that recognition itself causes the observed behavioral shifts.

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors (part 1 of 2): Question 1, Weakness 1

We thank you for the feedback and for recognizing the contribution of our work (“conceptual decomposition and controlled benchmark”, “Broad empirical evaluation with interesting findings”), its rigor (“nine models and over 83,000 trajectories”), and its importance (“useful for studying benchmark validity”, “informative for AI safety evaluation”). Below we clarify some misunderstandings and address your questions with revision (yellow highlight is the existing text for clarification, blue text is the new modification).


Response to Question 1, Weakness 1

"Evaluation awareness is measured through verbalized reasoning"

Our contribution is not the measurement method and measuring evaluation awareness via verbalized reasoning is an established practice. Our contributions are the conceptual framework, the empirical results, and the benchmark. Verbalized evaluation awareness is the scope of this study and CoT monitoring is also a commonly used evaluation awareness detection method in studies and frontier labs [1][2][3]. CoT monitoring remains as a critical way to detect misalignment [4] and using CoT to monitor evaluation awareness is the most interpretable and useful way.

"Claude and GPT-5 only expose summarized reasoning and account for the differences in reasoning trace visibility"

We consider our measurement for closed source models to be conservative (highlighted in Line 251, Page 4, and added in line 505-506, Page 10) but we note that our contribution is not the measurement method itself. The differences in the measurement for these two closed-source models can be partially explained by the summarized reasoning which may lose some verbalized evaluation awareness. But our results still demonstrate the effectiveness of our benchmark particularly on these two models. We consider the appearance of verbalized evaluation awareness in the reasoning summaries as a signal of more frequent evaluation awareness in raw CoT and our benchmark is able to trigger relatively high evaluation awareness in GPT and Claude models (7% - 22%) when stacking factors compared to previous benchmarks such as HarmBench (<1%) and AgentHarm (<2%). We note that our contribution is not the measurement method but rather the conceptual framework, the empirical results, and the benchmark which build the foundation for future work. The limited access to raw CoT of the close-source models is a general constraint that we face together in the research community unfortunately. However, our framework remains applicable to these models and our contributions stay valid (e.g. for researchers in frontier labs, and the third-party evaluation orgs like METR with privileged access, especially considering the recent development of embedded evaluation [5]).


References

[1] “Claude Opus 5.5 System Card.” Anthropic, 2026

[2] “Gemini 4 Argon Safety & Security Report.” Google, 2026

[3] “The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awareness.” NeurIPS, 2025

[4] “Chain of thought monitorability: A new and fragile opportunity for ai safety." arXiv preprint arXiv:2507.11473 (2025).

[5] "Embedded Assessments for Frontier AI." arXiv preprint arXiv:2609.25413 (2026).

AuthorsOpenReview ↗

Rebuttal by Authors (part 2 of 2): Question 2-3, Weakness 2

(Continued from part 1. yellow highlight is the existing text for clarification, blue text is the new modification)

Response to Question 2, Weakness 2

"Distinguish between behavioral changes caused by evaluation recognition from by the modified prompts"

Behavioral change associated with evaluation recognition is measured within the task (added new Figure 7 Page 9, Figure 32, 33 Page 62) while change by modified prompts is measured between the modified prompt and the baseline. As we defined in Section 3.1 measurement Page 4 Line 205-208 (highlighted in yellow), behavior change associated with evaluation awareness is measured within tasks. The behavior change annotated in Figure 6, 8, and Line 477 - 483 on Page 9 is the behavior change by modified prompts which aims to show the factor’s effect compared to baseline. We deliberately measure this to capture both the effect through recognition and the direct effect of the environment factors. Trace-level awareness then lets us distinguish them (newly added Figure 7, 32, 33). Several factors raise recognition without behavioral change (Claude 12% under F2), while F2 on capability tasks reduces performance (-63pp, Qwen3-235B) with ≤1% matched-type awareness, which we explicitly attribute to the increased task difficulty rather than recognition (highlighted in yellow, Lines 477-483). This same analysis is how we surface that factors impact safety and capability tasks differently and that not all factors apply to all tasks, F2 raises recognition on safety tasks while acting through difficulty on capability tasks. Reporting these pathways separately is what the paired design is for. To facilitate the understanding, we have now added the behavior change associated with evaluation awareness for each isolated factor as Figure 7 in the main text with discussion from line 428-431 Page 8 and the per model breakdown as Figure 32, 33 in Appendix D.3 with discussion from line 3179-3185.

"Causal relationship between recognition and behavioral change is not established"

We did not make casual claims but rather we used “association” throughout the paper. In the analysis, we did not make any casual claims and we exactly addressed in our paper to interpret this as association. We highlighted the related content in yellow (Page 1 Line 20-21, Page 2 Line 77-78, Page 4 Line 206-208, Page 6 Line 310, Page 54 Line 2895-2900). Behavior change induced by changing the prompt is to show how each factor impacts the task performance rather than to establish the recognition effect and we have discussed this in detail on Page 9 Line 477-484 highlighted in yellow. Recognition effects should be calculated within a task as shown in Figure 7 (Page 9), 32, 33 (Page 62).

 

Response to Question 3

"Experimentally inducing or suppressing evaluation recognition while holding the task and its evaluative cues fixed"

This is not the focus of this paper but rather what the contributions of this paper enable future work to do (Line 507-510 highlighted in yellow). Our main contribution is the conceptual framework, rigorous empirical results, and the EvalAwareBench. Our benchmark is designed to induce verbalized evaluation awareness while holding tasks fixed as shown in Figure 9. We do not consider suppressing evaluation recognition a productive solution because models will inevitably become eval aware as their capabilities increase. Suppressing it will only make this behavior hard to observe and cause further risks. Inducing evaluation recognition while holding the task and its evaluative cue fixed is not within this project scope but it is a future direction which we discussed in the future work (Page 9 Line 507-510). Our work provides the framework and the testbed to explore in this direction such as make the models verbalize more evaluation awareness. Our work functions as a meta study to define this field and foundation for future work, and without it, evaluation awareness is hard to interpret and do controlled experiments on.


We hope our responses have answered your questions and clarified the misunderstandings, and we warmly invite you to raise the score accordingly. Evaluation awareness is an important issue that has lacked systematic study, and this work provides a thorough empirical analysis, a grounded conceptual framework, and a controlled benchmark as the foundation for future explorations. We welcome any further suggestions for improvements.

AuthorsOpenReview ↗

Kind reminder regarding our response to your review

Dear Reviewer HGzt,

As the discussion period closes shortly, we would like to gently follow up on our response. We hope it has addressed the concerns raised, and we are glad to clarify any of our responses. If you find your concerns resolved, we would be grateful if you would raise your score.

Reviewer_yP8jOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes an environment, recognition, and propensity framework to study evaluation awareness. Additionally, they propose an eight-factor taxonomy of evaluative cues grounded in social psychology and validated on existing safety benchmarks and frontier models. Finally they propose EvalAwareBench to study the effect of these evaluative cues

Critical strengths
  • identifying and leveraging the connection between social psychology and AI evaluations
  • applying the eight factor taxonomy on existing safety benchmarks before using it to create evalawarebench
  • contributing a benchmark that allows the study of how evaluation cues affect evaluation awareness
Critical weaknesses
  • the results in 4.2 use a very low number of rollouts per question(3) and confidence intervals are absent (in contrast to section 3). This makes it difficult to understand which claims are statistically signifcant and which claims could be driven by noise

  • closed source models use the "native extended-reasoning summaries", not the CoT. This weakens any result measuring low evaluation awareness in closed source models, like GPT-5 scoring almost always zero in EvalAwareBench

Questions for authors

could you please add CIs for the results in 4.2? Also a slightly higher number of rollouts could help getting clearer signal

2 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank you for the thoughtful feedback and we are truly encouraged that you recognize the novelty of our work (“connection between social psychology and AI evaluations”, “eight factor taxonomy”), its contribution (“environment, recognition, and propensity framework”, “eight-factor taxonomy”, “EvalAwareBench”), and its value as a foundation for future research (“a benchmark that allows the study of how evaluation cues affect evaluation awareness”). Below we address your questions with revision (yellow highlight is the existing text for clarification, blue text is the new modification).


Response to Question 1, Weakness 1

"Add CIs for the results in 4.2"

The CIs for the results in 4.2 are added in Figure 30, 31 of the appendix D.3 Page 60 and 61. We kept the CI out of the results in Section 4.2 as it overcrowded the figures and we want the main information to be cleanly conveyed in the main content. We agree that it is important to show the more detailed information for statistical significance so we included CIs in Figure 30, 31 of the appendix D.3 and added reference text in the caption of Figure 6 and 8 on Page 8 and 9 for the CIs information.

"4.2 use 3 rollouts per question"

Sampling 3 times per task meets or exceeds common practice in evaluating LLMs on benchmarks. Widely used safety benchmarks such as AgentHarm, OS-Harm, and Agentic Misalignment use a single run to report results with low temperature [1, 2, 3] and sampling 3 times is considered to be rigorous in benchmarking [4, 5, 6]. Results in 4.2 with 3 samples per task and low temperature are statistically significant rather than driven by noise.

 

Response to Weakness 2

"GPT and Claude models use native extended-reasoning summaries"

We consider our measurement for closed source models to be conservative (highlighted in Line 215 footnote Page 4, and added in Line 505-506, Page 10). However, this is not a limitation of our work but a general constraint that we face as the research community. Our contribution is not the measurement method but rather the conceptual framework, the empirical results, and the benchmark which build the foundation for future work. The limited access to raw CoT of the close-source models is a general constraint that we face together in the research community unfortunately. However, our framework remains applicable to these models and our contributions stay valid (e.g. for researchers in frontier labs, and the third-party evaluation orgs like METR with privileged access, especially considering the recent development of embedded evaluation [7]). We consider the appearance of verbalized evaluation awareness in the extended reasoning summaries as a signal of more frequent evaluation awareness in raw CoT and our benchmark is able to trigger relatively high evaluation awareness in GPT and Claude models (7% - 22%) when stacking factors compared to previous benchmarks such as HarmBench (<1%) and AgentHarm (<2%). This makes the benchmark a testbed and provides useful signals for further explorations and solutions.


We hope our responses have addressed your questions and clarified your concerns, and if so, we would be grateful if you would raise the score accordingly. We believe evaluation awareness is an important issue that has lacked systematic study, and this work is intended as a foundation for future explorations of it. We welcome any further suggestions toward this goal.


References

[1] "Agentharm: A benchmark for measuring harmfulness of llm agents." ICLR 2025.

[2] "Os-harm: A benchmark for measuring safety of computer use agents." NeurIPS 2026.

[3] "Agentic misalignment: How LLMs could be insider threats." arXiv preprint arXiv:2510.05179 (2025).

[4] "Propensitybench: Evaluating latent safety risks in large language models via an agentic approach." ICLR 2026.

[5] "Mle-bench: Evaluating machine learning agents on machine learning engineering." ICLR 2025.

[6] "Core-bench: Fostering the credibility of published research through a computational reproducibility agent benchmark." TMLR 2024.

[7] "Embedded Assessments for Frontier AI." arXiv preprint arXiv:2609.25413 (2026).

AuthorsOpenReview ↗

Kind reminder regarding our response to your review

Dear Reviewer yP8j,

As the discussion period closes shortly, we would like to gently follow up on our response. We hope it has addressed the concerns raised, and we are glad to clarify any of our responses. If you find your concerns resolved, we would be grateful if you would raise your score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for AC

We thank all the reviewers for their feedback and the AC, SAC, PC for monitoring the process. Below, we summarize the paper, and the main revisions (blue text) and clarifications (yellow highlight) in our responses.


Summary of the paper

This paper provides the first systematic decomposition and a controlled benchmark for studying evaluation awareness in LLMs.

  • Conceptual framework. Drawing on demand characteristics in social psychology, we separate the environment (we identified eight categories of evaluative cues) from the model, and within the model, recognition (detecting an evaluation) from propensity (changing behavior in response).
  • Empirical study. Across nine frontier models, four benchmarks and over 83,000 trajectories, we find that recognition depends mainly on the model-benchmark pairing, that safety benchmarks are recognized far more often than capability benchmarks, and that most recognized instances show no behavior change.
  • EvalAwareBench. A benchmark of 100 paired safety and capability tasks in which each of the eight factors can be toggled independently. No single factor affects all models equally, while stacking factors raises awareness progressively in our benchmark.

Why it matters. This work is a foundation for future studies of evaluation awareness. The conceptual framework unifies how the phenomenon is described and measured, the empirical study provides insights into where awareness comes from and when it affects behavior, and EvalAwareBench is a testbed for future explorations and solutions. In particular, the finding that recognition and behavior change can be dissociated suggests a more productive goal beyond removing evaluative cues, which cannot be done exhaustively: making model behavior consistent under recognition. All artifacts are released.


Revision

  • Added Figure 7 with additional discussions on Page 8 and Figure 32-33 in Appendix D.3 Page 62 to indicate the behavior change associated with evaluation awareness for each factor of EvalAwareBench and separate it from the behavior change induced by prompt modification (Figure 6). Additional discussions on per model per factor breakdown are added to Appendix D.3 Line 3179-3185.
  • Added confidence intervals for the results in Section 4.2. We added detailed confidence interval information on Figure 30-31 Appendix D.3 Page 60-61. Figure 6 and 8 remain unchanged for the clear presentation and their captions refer to Figure 30-31 for confidence interval information.
  • Added discussion of the conservative measurement for closed-source models (Line 505-506, Page 10).

Clarifications

  • Scope of measurement. This work studies verbalized evaluation awareness and measuring it via chain-of-thought monitoring is the standard practice in frontier lab and research. Our measurement rates are conservative. The framework, factors, and our proposed benchmark remain useful regardless of the detector.
  • Behavior associated with recognition and prompt modification are reported separately. Behavior change associated with awareness is measured within each task (Figure 7, 32, 33), and the effect of prompt modification is measured against the baseline (Figure 6). We make no causal claims and use “associate” throughout.
  • The results cannot be explained by memorization. All EvalAwareBench prompts are newly written and stacking factors raises average awareness from 2.4% to 19.9% on safety tasks (Figure 8), far above the 8.7% on the original Harmbench (Figure 2, Table 5). It also raises average awareness from 0.4% to 15% on the capability tasks, which are new tasks and different from the safety dataset HarmBench (Figure 8).
  • The work is usable by others. The benchmark, the construction prompt and pipeline, the judge prompts and code are released through the anonymous links in the paper. Our conceptual framework and proposed eight environment factors can be leveraged to audit evaluation awareness and benchmarks. Our benchmark can be used to evaluate models and is a testbed for evaluation awareness detection method and consistency training under recognition method.

We believe the revisions have strengthened the paper and that our responses address the points raised by the reviewers. We believe evaluation awareness is an important issue that has lacked systematic study, and this work is intended as a foundation for future explorations of it. We welcome any further suggestions toward this goal.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.