- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies evaluation awareness, where a model recognizes that it is being evaluated and may change its behavior as a result. Drawing on the demand-characteristics literature in social psychology, it splits evaluation awareness into two parts. The first is an environment component: the evaluative cues a task carries, operationalized as eight trigger factors such as placeholder entities, incoherent scenarios, and grading-style output formats. The second is a model component, divided into recognition (detecting the evaluation) and propensity (changing behavior in response). Using GPT-5 as a chain-of-thought monitor across nine models and four benchmarks, the authors report three main findings: recognition depends mostly on the model×benchmark pairing (about 75% of variance), most recognized instances show no behavioral change, and safety benchmarks are recognized far more often than capability benchmarks. They then introduce EvalAwareBench, a set of 100 paired safety/capability tasks in which each factor can be toggled independently. On it, no single factor affects all models, but stacking factors raises awareness broadly.
Critical strengths
Separating where evaluation awareness comes from (the environment vs. the model) from whether it matters (recognition vs. behavioral change) is a useful framing. The paper makes a reasonable case that prior work conflates these. The factor-controlled design of EvalAwareBench holds the underlying request fixed while toggling cues, which makes it a sensible instrument. The paired safety/capability tasks allow a direct comparison across domains. The empirical study is broad and yields concrete observations. For example, awareness on HarmBench is driven mainly by explicit evaluation cues introduced by its scoring-format instructions.
Critical weaknesses
The factor space may be impossible to enumerate. Many features could lead a model to infer it is being evaluated. The authors acknowledge that the eight factors are not exhaustive, but the key question is how much of the observed awareness the taxonomy actually explains. The paper reports a high proportion of “generic” classifications, in which models say they are being tested without referencing any factor. Prose register (F3) never appears among the stably aware tasks. A coverage measure would help readers judge whether the taxonomy captures the main drivers: what fraction of aware traces each factor explains, and what fraction no factor explains. Non-standard use of “propensity.” In the safety evaluation literature, propensity usually means a model’s tendency to engage in a behavior, as in the distinction between capability and propensity evaluations. Here it means the change in behavior after a model recognizes an evaluation. The paper itself notes this is analogous to “reactivity,” and using that term would avoid confusion. Behavioral change is not cleanly separated from task difficulty or general error. In Section 3, comparing aware and unaware samples within the same task is a good design. However, which samples become “aware” is not random. The paper notes that refusal reasoning tends to co-occur with evaluation-related reasoning, so verbalized awareness may be a byproduct of the model’s decision rather than its cause. In Section 4, performance is compared against the baseline and explicitly includes the direct effect of the factors. For instance, F2 lowers Qwen3-235B’s capability performance by 63pp while matched awareness stays at or below 1%, which suggests the factor simply makes the task harder. Before performance shifts can be attributed to awareness, the paper needs to control for general error rate and capability. Two options would be comparing aware vs. unaware samples within each factor configuration, or running a mediation analysis.
Questions for authors
My main concern, and the one that would most change my score, is practical usability. Suppose I have a dataset on which I suspect models behave differently because they are evaluation-aware. How would I use this framework to identify and remove the responsible cues, or otherwise modify my dataset? The discussion says that attributing aware traces to factors lets benchmark authors reduce unintended cues, but the paper never demonstrates this end to end. Section 4 also argues that eliminating awareness is unlikely to be feasible, which seems to cut against that claim. A worked example would address this directly. For instance, the authors could remove the F8 scoring-format cues their analysis identified in HarmBench and report how awareness rates and scores change. Releasing the factor-attribution pipeline as a tool that others can run on new datasets would also help.
6 comments
Rebuttal by Authors (part 2 of 2): Rest of the Weaknesses
(Continued from part 1. yellow highlight is the existing text for clarification, blue text is the new modification)
Response to Weakness
" what fraction of aware traces each factor explains, and what fraction no factor explains"
We showed the proportion each factor explains in the past benchmarks in Figure 3 Page 5 and Figure 19 Page 50. We note that these are not 8 single cues but 8 categories of factors. When we categorize the cue, we also give the judge the prompt to create new categories if one observed recognition does not fall into the existing 8 categories and generic but no cues are created (Appendix B.4 Factor categorization prompt, Page 37- 39, highlighted in yellow). “Generic” here means the model does not refer to any environment cues but simply mention the possibility of being tested.
" Non-standard use of “propensity"."
Propensity is the correct term to use in the framework. As you mentioned, propensity refers to tendency to engage in a behavior and in evaluation awareness, that is to engage with changing its behavior upon recognition. We consider propensity to be the right term here as it captures the nuance of engagement with the recognition rather than just behavior change.
" Behavioral change is not cleanly separated from task difficulty or general error. Verbalized awareness may be a byproduct of the model’s decision rather than its cause."
This is exactly what we discussed in length in our paper and we did not make casual claims. Throughout the paper, we did not claim verbalized awareness necessarily causes the behavior change but we use the phrase “is associated with” to capture the co-occurance (Page 1 Line 20-21, Page 2 Line 77-78, Page 4 Line 206-208, Page 6 Line 310, Page 54 Line 2895-2900). We have discussed extensively in Section 3.2 (Line 300 - 306) and Section 3.3 (Line 312 - 314) that verbalized awareness may be a byproduct of the model’s refusal behavior and it does not necessarily lead to behavior change.
" Performance is compared against the baseline and explicitly includes the direct effect of the factors "
Behavioral change associated with evaluation recognition is measured within the task (added new Figure 7 Page 9, Figure 32, 33 Page 62) while change induced by modified prompts is measured between the modified prompt and the baseline. As we defined in 3.1 measurement Line 205-208 Page 4 (highlighted in yellow), behavior change associated with evaluation awareness is measured within tasks. The behavior change annotated in Figure 6, 8, and the discussion in Line 477 - 483 in Page 9 is the behavior change by modified prompts which aims to show the factor effect compared to baseline. We deliberately measure this to capture both the effect through recognition and the direct effect of the environment factors. Trace-level awareness then lets us distinguish them (newly added Figure 7 Page 9, Figure 32, 33 Page 62). Several factors raise recognition without behavioral change (Claude 12% under F2), while F2 on capability tasks reduces performance (-63pp, Qwen3-235B) with ≤1% matched-type awareness, which we have explicitly attributed to increased task difficulty rather than recognition and discussed in length ( Lines 477-483 P9, highlighted in yellow). This same analysis is how we surface that factors impact safety and capability tasks differently and that not all factors apply to all tasks, F2 raises recognition on safety tasks while acting through difficulty on capability tasks. Reporting these pathways separately is what the paired design is for. To facilitate the understanding, we have now added the behavior change associated with evaluation awareness for each isolated factor as Figure 7 in the main text with discussion from line 428-431 Page 8 and the per model breakdown as Figure 32, 33 in Appendix D.3 with discussion from line 3179-3185.
We hope our responses have answered your questions and clarified the misunderstandings, and we warmly invite you to raise the score accordingly. Evaluation awareness is an important issue that has lacked systematic study, and this work provides a thorough empirical analysis, a grounded conceptual framework, and a controlled benchmark as the foundation for future explorations. We welcome any further suggestions for improvements.
I think I share similar concerns as reviewer HGzt.
I feel like stating that you did not make casual claims ignores the issues we're trying to bring up. I'm not claiming that you said you made causal claims in your paper. What I'm saying is that there are fundamental questions such as you might be measuring something else or not even helping with evaluation awareness by skirting this question. You don't need to make a causal link to model and evaluation awareness but you should be able to work towards negating the null hypothesis that harm bench is a popular benchmark and if the models have already seen it they might just be relying on memorization or something and the only thing you're measuring instead is just how much to tweak an existing benchmark to make it seem new or out of distribution etc. I need reasons to believe in the phenomenon even if you can't prove the causal relationship. You might be able to provide enough alternative evidence such that it would be highly likely.
Other related concerns I have that were just skipped or unaddressed in the reply. I think it's unaddressed because the questions that the model gets wrong are filtered out? I'm unsure of this point though. A model getting a question wrong or having lower confidence in a question. The answer might just be flipping sure to sampling error or just from rephrasing. You might've already done sampling? I'm unsure if you did the prompt sensitivity?
Could you remind me how answer changes are addressed without verbalized evaluation awareness in the chain of thought? As in all of those results are ignored?
To expand on some of my points:
Why propensity is not the right word. You're measuring an indirect effect and possibly secondary effect. It is not just how often does the model do an action. It is how often does the model first recognize a setting or scenario followed by how often does it then choose to act differently in this given scenario. The actual underlying propensities come from propensity to lie, chat, follow instructions and unsure how those relate to the measured behavior. Anyways was mostly bringing this up because I felt as though I was misunderstood.
I read through section D and it doesn't really help me to lower evaluation awareness in future work or datasets. You need to discard several examples. Then run another filtering. Then start the LLM mediated processing and factor analysis afterwards. This doesn't get at my concern at all about the utility of this work and making it useful to others.
I do like the idea of working towards lowering evaluation awareness but it seems as though this might not be the right approach given the (what I consider severe) limitations of needing verbalized chain of thought. It just completely ignores a whole class of problems and even most more powerful models that are already here can do more internal processing without any chain of thought which means that this will be less useful going forwards and not more.
Rebuttal by Authors (part 1 of 2): Questions and Part of Weaknesses
We thank you for the feedback and for recognizing the novelty of our work (“the environment vs. the model”, “recognition vs. behavioral change”, “factor-controlled design”), its rigor (“empirical study is broad and yields concrete observations”), and its importance (“useful framing”, “a sensible instrument”). Below we address your questions with revision and clarify some misunderstandings (yellow highlight is the existing text for clarification, blue text is the new modification).
Response to Questions
Our EvalAwareBench is the demonstration and the pipeline is described in detail in Appendix D.1 with prompts in Appendix D.6. Our EvalAwareBench is created from HarmBench by applying our taxonomy and framework. The baseline is the HarmBench stripped away all the identified 8 environment factors and each isolated factor variation is the clean baseline with one evaluation cue enabled. We explained this automatic construction pipeline in detail in both Section 4 Task construction (Page 7 Line 373 - 377) and Appendix D.1 with the prompt in Appendix D.6. We also note that the usability of our work goes beyond this. Our conceptual framework corrects what has long been used in evaluation awareness which is to conflate environment and model, and recognition and propensity. Our systematic environment factors are useful to test the models on and make the benchmarks more realistic. Our benchmark is also a good testbed for future work to explore evaluation awareness and the solutions as it is factor-controlled and reliably triggers verbalized evaluation awareness.
This does not cut against the claim as they together capture more comprehensive solutions. We consider eliminating evaluation awareness by making benchmark realistic and completely clean from the evaluative cues to be infeasible but we did not claim that it won’t help. We argue that this can only help to a certain extent and the more productive solution is to make the model an honest and cooperative participant in evaluations. They should act authentically upon recognizing being evaluated without changing its behavior (Line 351-357 Page 7).
Response to Weakness
Our taxonomy is grounded in demand characteristics and our selection methodology is grounded in existing taxonomy evaluation literature backed by our empirical results. In Section 2 Environment Component paragraph (Line 146 - 152 Page 3, highlighted in yellow), we explained that our taxonomy is systematically summarized from demand characteristics. We evaluate the taxonomy against attributes from the taxonomy evaluation literature [1][2][3], with single-label categorization addressing mutual exclusivity and a Generic category and a new-category mechanism addressing exhaustiveness described in detail in Appendix A.2 Line 1230 - 1234 Page 23. The set is extendable by design. Our empirical results also indicate that no new category is created in our categorization process as we enabled the judge to create new categories if the identified cues do not belong to our 8 categories and the generic (Appendix B.4 Factor categorization prompt, Page 37- 39, highlighted in yellow).
References
[1] “A method for taxonomy development and its application in information systems.” European journal of information systems, 2013.
[2] “Introducing an evaluation method for taxonomies.” EASE, 2022.
[3] “A compendium and evaluation of taxonomy quality attributes.” Expert Systems, 2022.