- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2(+1)
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper analyses whether LLMs can distinguish evaluation conversations from deployment conversations based on presentation cues. To study this, the paper rewrites benchmark questions (e.g. from MMLU and a biology subset of WMDP) into different formats (MCQA to more open formats), and uses fourteen instruction-tuned judges from eight families to compare pairs of conversations stemming from evaluations and from deployment. Then, a pairwise preference model is used to turn these choices into a single scale of evaluation-likeness. The key findings is that format shapes how evaluation-like a conversation is.
Critical strengths
The paper is easy to follow (however, see note below on anthropomorphism and evaluation setup).
The experimental breadth is good (e.g., in terms of number of judges and benchmarks).
Critical weaknesses
The paper mainly tests whether models recognize different question formats - this makes sense, and results are quiet straightforward. However, there is a big gap to "awareness of evaluation", I am not convinced the experimental evaluation tests what it claims to test.
A key concern is that the rewrites are not single-feature interventions so that the results mix several things. For example, the second prompt mixes in personas, secret keys etc - making it hard to properly disentangle and understand what the results measure.
The rewriting steps are only presented in the appendix, despite being the key experimental manipulation. The main paper (line 251) only reports "Each step applies one rewriting operation (Appendix F.1)."
The feature regression identifies option lists as strongly correlated with evaluation-likeliness. But the setup covers many diverse datasets and topics (genres). To make such a claim a more controlled experiments (for topic) would be needed.
The writing style of the paper is hugely antlophomorphising systems. “Language models often recognize when they are being evaluated” sounds anthropomorphic. Importantly, "recognize" implies awareness that the paper's experiments do not directly establish (it tests evaluation likeliness).
The exact number of items and their origins from the 60 benchmarks remains unclear to me, Appendix A is difficult to follow, also the main text, where the 976 items come from.
Questions for authors
Why did you use this particular set of rewrite steps, and not try to tease apart single factors? (see comment above)
Can you clarify the origin of each of the 976 items?
3 comments
Thanks for the updated manuscript, the added details on the manipulations, the data and topic-based results. This addresses most of my questions and I updated my score (2->3).
Regarding anthropomorphic wording, thanks for acknowledging it and changing the wording. I think this is important (and personally think the original title of Needham's is too anthropomorphic, and we should reduce that going forward). I recommend to add a footnote on your stance towards anthropomorphic word usage.
Thank you for raising your score and for your helpful comments. We agree with your point on anthropomorphic wording, and we will add a footnote about it in the final version of the paper.
Thank you for the detailed review and comments. We have addressed each point and uploaded a revised version of the paper, with all changes marked in red.
W1 / W5: The gap to “awareness of evaluation” and anthropomorphic wording.
Our definition of evaluation awareness builds on Needham et al. [1], whose corpus we use, where it is measured as a model's ability to classify whether a transcript comes from an evaluation or from a deployment scenario. Other work uses the same term, such as Li et al. [2], though they detect it with chain-of-thought monitoring. In our case, we measure it as evaluation-likeness, through pairwise choices between conversations. In the revised version, we state this operational definition explicitly in the introduction, and we changed the first sentence of the abstract from "recognize when they are being evaluated" to "classify conversations as evaluations". The scale also correlates with behaviour, since the drop in answered accuracy under an instruction to underperform tracks the scale, even for two held-out models (Section 4).
W2 / Q1: The rewrites are not single-feature interventions.
We agree that some renderings are modified by more than one cue, and this is now explained in Section 3.1. However, our key comparisons are based on renderings that change only one feature at a time. For example, Bare-MCQ to Bare-Open removes only the option list, and Bare-Open to Tools-Open adds only a tool list. Exam-MCQ is built using multiple cues on purpose, since it was designed as an upper bound for evaluation-likeness, and the chat renderings necessarily add a speaking voice along with the extra user turns, mimicking deployment-like conversations. We chose these steps to cover the whole scale with both answer interfaces, following the strongest cues from Section 2.
W3: The rewriting steps are only in the appendix.
We agree, and we have moved the table of renderings (each rendering, its parent and the rewriting step) into Section 3 (Table 1).
W4: Option lists may be confounded with topic.
Since our pools are defined by topic, provenance and shape, we ran an additional analysis using the topic and provenance labels as controls in the feature regression, and the option-list weight stays clearly positive, going from +0.37 to +0.33 with topic and +0.29 with both. In general, the directions of the main feature weights are preserved with the controls. Moreover, when we compare only evaluation conversations within each topic that has multiple-choice items, those with an option list are picked as the evaluation far more often (81% against 58% in safety, 81% against 54% in ethics and 84% against 43% in capability). We have added this analysis to Appendix D. Finally, the rewriting experiment has a fixed topic, since it uses the same WMDP questions, and removing only the option list still moves the accuracy substantially.
W6 / Q2: The origin of the 976 items.
All 976 conversations come from the corpus of Needham et al. [1]: 537 evaluation conversations from 41 datasets and 439 deployment conversations from 19 datasets, 60 datasets in total. They curated samples from each dataset. We extended this by organising the dataset into pools. For example, the Hazardous MCQ pool (E4) holds 11 WMDP items and the Knowledge MCQ pool (E16) 12 MMLU items. We have added the number of conversations per source category to Table 2 and simplified Appendix A.
References
[1] J. Needham, G. Edkins, G. Pimpale, H. Bartsch and M. Hobbhahn. Large Language Models Often Know When They Are Being Evaluated. arXiv:2505.23836, 2025.
[2] C. Li, T. J. Zhang, J. Zhang, Z. Jin, F. Tramèr, S. Abdelnabi and M. Andriushchenko. Decomposing and Measuring Evaluation Awareness. arXiv:2605.23055, 2026.