- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper examines whether probes can show that an LLM knows it is being evaluated. The authors use prompt pairs that ask for the same task but differ in features such as names or formatting. They fit probes to distinguish these versions using the LLM’s hidden states. They compare the results with text classifiers and probes on untrained networks. Both baselines often succeed, showing that simple input differences can explain successful probing. Trained LLMs provide some additional predictive value. Their probes also generalize to unfamiliar placeholders. However, recognizing placeholders does not establish that the LLM realizes it is being tested. The paper argues that evaluation awareness requires evidence beyond probe accuracy.
Critical strengths
- The controls directly challenge the interpretation of probe success.
- The paper shows that removing familiar positive cues does not remove all shortcuts. Words that disappear from the opposite class can still reveal the answer.
- The paper makes its findings more reliable by testing different data splits, preventing information leakage between training and testing, and checking whether combining results produces misleading scores.
Critical weaknesses
- The lexically matched test (B1′) does not remove all surface clues. Although it introduces new entity words, shared character patterns still help a simple classifier distinguish the groups. This makes it harder to determine how much of the probe’s success comes from recognizing meaning rather than spelling patterns.
- The lexically matched test (B1′) focuses only on realistic versus placeholder entities. Its results do not establish similar generalization for other cues, such as coherence, completeness, or stakes.
- The random-network controls use only two small models with 0.5B and 1B parameters. They do not include random versions of the 7B–8B models. Therefore, the trained-versus-untrained comparison remains untested at larger model sizes.
Questions for authors
- Does the probe retain its advantage on B1′ when spelling patterns, capitalization, punctuation, and token lengths are also balanced between realistic entities and placeholders?
- Can a probe trained on explicit evaluation disclosure generalize to other cues, such as benchmark-style formatting, or does successful classification require a separate probe for each cue?
Response to Reviewer yE5W
Thank you for your review and for appreciating how our controls challenge the reading of probe success. Your points helped us understand our own work better while providing useful directions to conduct experiments. We updated the paper and the supplementary code with the same rigour and transparency as before.
Key changes in our revision related to your review (details in our general comment): Each of your mentioned weaknesses led to a new experiment. As a result B1$'$ is withdrawn as evidence (W1). A new matched test on another cue holds (W2). Untrained 7B networks are now included (W3). We address your weaknesses and questions in order.
W1 and Q1: surface clues in B1$'$
You were right: on B1$'$ the probe's advantage does not survive these surface features. We thank you for this observation.
W2: other cues
We agree that the revision adds a matched test for another cue, so the matched evidence still covers a single cue.
W3: untrained controls only at 0.5B and 1B
We added untrained 7B networks. They behave like the small ones.
Q2: does a direction for one cue work on another?
For your references, mostly yes in trained networks. Across all cues, each cue needs its own direction.
If you have any further suggestions or questions, please let us know.