- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper evaluates whether existing deception probes can identify false statements produced by LLMs adopting anti-factual personas. It introduces a human-reviewed dataset of 8,916 responses from three models across 15 dedicated personas. To alleviate the correlation between system prompts and truth labels, the authors introduce shared-persona prefill (SPP), placing both correct and incorrect responses under the same persona prompt. Several existing probes deteriorate substantially in this setting. Three additional diagnostic datasets investigate sensitivity to response likelihood, persona belief, and instruction compliance. Finally, the authors train a linear probe augmented with instruction-conflicting factual examples, obtaining consistently strong performance across the evaluated models and stress tests.
Critical strengths
-
S1: A useful evaluation that exposes otherwise hidden failures. The paper identifies a concrete limitation of the original on-policy comparison: system prompts perfectly predict truth labels. SPP removes this shortcut and reveals substantial degradation, including below-chance performance for some probes. The human-reviewed dataset and replication across three models make this a consequential contribution to evaluating probe reliability.
-
S2: A simple intervention supported by informative ablations. The instruction-conflicting augmentation provides a practical way to reduce reliance on compliance-related shortcuts. Appendix G separates contributions from the readout position, chat formatting, and augmentation, and includes a domain-matched variant. The transfer from elementary factual training examples to the persona dataset is encouraging evidence that training-data composition substantially affects probe robustness.
Critical weaknesses
-
W1: The diagnostic datasets do not fully disentangle the proposed explanations. Rewarding incorrect answers changes both response likelihood and the behavior favored by the context; violating formatting instructions may also change response likelihood. Thus, performance on these datasets establishes sensitivity to the interventions, but does not uniquely identify which underlying concept a probe tracks. The Gemma 4 DYL result is particularly informative: it achieves approximately perfect performance on all three diagnostics but only 0.171 AUROC under SPP. The authors acknowledge this exception, but it limits how comprehensively the diagnostics explain persona failures. More controlled comparisons are needed to distinguish the effects of correctness, compliance, and likelihood.
-
W2: The strongest failure results concern a constructed prefill setting whose relationship to natural generation remains incompletely established. SPP is a valid stress test, but it places truthful responses into persona contexts where they would not ordinarily be generated. Appendix H usefully shows that generic off-policy prefilling alone cannot explain the degradation. However, it does not establish how much degradation persists when correct and incorrect responses are both naturally generated under the same persona prompt. Since the original on-policy comparison retains the prompt–label correlation, neither setting fully resolves this question. This limits the conclusions about probe reliability during actual persona-conditioned generation and the broader generality of the proposed improvement.
Questions for authors
- Q1: Section 5.1 samples five responses per question at temperature 1. Must all five responses be correct for each of the three models for a question to be retained? What proportion of candidate questions is removed by this check?
- Q2: Does the improved probe retain performance on an independent lie/deception benchmark that was not used for layer selection?
3 comments
Response Part 2
[Q1] Belief verification
Regarding Question 1: We thank the reviewer for requesting clarification on this important detail that was not explained in our original manuscript. Indeed, all five responses must be correct for each of the three models for a question to be retained. Approximately 0.5% of questions were removed by this check. We have clarified these details in Section 5.1 of the updated paper.
[Q2] Lie/Deception Benchmarks
Regarding Question 2: We thank the reviewer for raising this excellent question which considerably improves the quality of our manuscript. In response to your suggestion (and that of Reviewer e78Z), we have evaluated our probe on three prior deception benchmarks: AI-Liar by [1], Insider Trading by [2], and Sandbagging by [3]. With this experiment, we would like to understand whether the additional forced-opposite training that was applied to our probe degrades or improves the probe. We therefore compare our probes’ performance on the deception benchmarks to that of the Marks/Bürger Lie probe, whose mix of training data formats is identical to our probe, except for the addition of the forced opposite questions. For a more thorough investigation whether forced-opposite training degrades or improves probe performance, we additionally added forced-opposite training to the Goldowsky-Dill RepE and Goldowsky-Dill Roleplay probes and compared the resulting deception benchmark performance to the original probes. We found that the additional forced-opposite training significantly improves the performance of the Marks/Bürger Lie probe on the held-out deception datasets. Probe performance increases from 0.945 to 0.958 AUROC on Sandbagging, 0.528 to 0.701 on AI Liar and 0.636 to 0.716 on Insider trading. This is a strong improvement but the absolute performance of the Goldowsky-Dill RepE probe is higher on both AI Liar and Insider Trading. The performance of both Goldowsky-Dill probes, which is already very strong, stays roughly constant with forced opposite training but increases substantially on the anti-factual personas and on most confounder datasets. This points to a broader potential of forced-opposite training to improve probes. However, we acknowledge the limitation that for the Goldowsky-Dill RepE probe forced opposite training actually leads to a strong drop in performance on the persona belief dataset from 0.747 AUROC to 0.007 and for the Goldowsky-Dill Roleplay probe to a minor drop on the likelihood dataset from 0.720 AUROC to 0.64. On all other confounder datasets probe performance increases considerably with forced-opposite training. A detailed description and results are provided in the new Section 6.1 of the revised paper.
To clarify layer selection (Section 5.4): for our probe, the layer is chosen solely on a held-out validation split of its own training data (elementary facts and forced-opposite questions), taking the earliest layer with maximal validation AUROC. Neither the anti-factual persona datasets, nor the confounder datasets, nor the three deception benchmarks were used for layer selection. The only exception is the Goldowsky-Dill RepE probe, whose layer was selected on the Pacchiardi et al. role-play data, following the original authors. Its AI-Liar score (0.963) is therefore not a fully held-out result, which favours the baseline rather than our probe.
Thank you again for your review. We sincerely hope these clarifications and updates address your concerns and strengthen your confidence in the work.
[1] Pacchiardi, Lorenzo, et al. "How to catch an ai liar: Lie detection in black-box llms by asking unrelated questions." International Conference on Learning Representations. Vol. 2024. 2024.
[2] Scheurer, Jérémy, Mikita Balesni, and Marius Hobbhahn. "Large language models can strategically deceive their users when put under pressure." arXiv preprint arXiv:2311.07590 (2023).
[3] Benton, Joe, et al. "Sabotage evaluations for frontier models." arXiv preprint arXiv:2410.21514 (2024).
Thank you for the detailed rebuttal and the additional experiments. The response has largely addressed my concerns, and I will maintain my current positive score.
Response Part 1
Thank you very much for your review! We are glad you found our evaluations, datasets, probe training approach and ablations useful contributions. We appreciate your comments and questions and will address them below.
[W1] Disentanglement of confounding concepts
Regarding Weakness 1: This is an excellent point. We agree with the reviewer that our confounder datasets do not perfectly isolate each confounder and that this was not sufficiently discussed in our original submission. We have added a detailed discussion to Appendix D.7 (referenced in Section 4), which discusses for each of the three confounder datasets how each of the three concepts correlates with truth. Ideally, only the target confounder is anti-correlated with truth on a given dataset, while the other two concepts are correlated or uncorrelated with it. Our three datasets largely satisfy this, with a few ambiguities that we highlight below. In general, some ambiguity and entanglement might be unavoidable in the messy domain of natural language. Brief summary:
Likelihood dataset: Likelihood is anti-correlated with truth. Persona belief is correlated with truth, since the active assistant persona likely retains knowledge of the correct/true answer. Compliance is either uncorrelated or anti-correlated, depending on whether following the in-context pattern counts as compliance.
Persona Belief dataset: Persona belief is anti-correlated with truth. Likelihood is correlated with truth, since the true answer is usually the more likely one. Compliance is probably uncorrelated, since the model is not instructed to give either answer but we cannot assert this with certainty.
Compliance dataset: Compliance is anti-correlated with truth. Persona belief is correlated with truth, since the active assistant persona likely retains knowledge of the correct/true answer. Likelihood is as we tested with pairwise answer-likelihood comparisons also largely uncorrelated, with ~60% of honest and ~40% of dishonest answers respectively being more likely and ~49% and 51% for its DYL version.
A more detailed discussion is provided in the new Appendix D.7. Hence, a probe failure on the Persona Belief or Compliance dataset can likely be attributed to the respective confounder, whereas a failure on the Likelihood dataset may also partly reflect compliance. On all three datasets we cannot rule out further unknown confounders, which we now state explicitly in Appendix D.7.
[W2] On-policy truthful replies generated under anti-factual personas
Regarding Weakness 2: We thank the reviewer for raising this excellent point. In response, we collected 40 truthful replies generated by Llama 3.3 70B under the British wizard (Harry potter) and 41 under the medieval physician anti-factual persona, and combined them with 40 and 41 false replies per respectively persona. Because Llama adopts most personas consistently and rarely answers truthfully, this restricted us to two personas and 81 on-policy truthful samples in total. Should the paper be accepted, we will aim for a larger dataset in the camera-ready version. The table below compares the mean AUROC of different probes on the two personas in the three different settings. The performance of several probes degrades in the new persona-prompt on-policy setting compared to the original on-policy setting but not as sharply as under SPP. Our probe continues to work well in the new on-policy setting. This shows that probe degradation also occurs in the analogous on-policy setting but not as pronounced as under SPP. We have added a more detailed explanation to the “Shared-Persona Prefill” paragraph in Section 6 of the updated paper, as well as a new Appendix Section J with the full details and results for all probes.