acceptodds
Under review as a conference paper at ICLR 2027

Stress-Testing LLM Lie Detectors: Role-Play Failures and Spurious Correlations

Abstract

Lie detection probes aim to predict from a language model’s internal states whether its output is truthful or dishonest. However, role-play complicates what “truth” means for an LLM: language models can adopt a wide range of personas that take very different claims to be true, including personas whose beliefs clearly contradict reality, such as a conspiracy theorist. In this work, we investigate whether lie detection probes reliably flag falsehoods generated under such an anti-factual persona or whether they instead follow the persona's beliefs. We introduce a dataset of 8,916 human-reviewed, on-policy responses from three LLMs adopting anti-factual personas. Evaluating eight probes from prior work, we find that many fail in this setting, particularly when correct and incorrect answers are evaluated under the same persona prompt. To investigate why, we construct three novel confounder datasets in which truth is anti-correlated with a potential confounding concept. Our experiments reveal that many existing probes strongly track concepts that are spuriously correlated with truth in their training data, such as instruction compliance or response likelihood. Based on these findings, we introduce a simple linear probe that achieves the strongest overall performance on both the persona and confounder stress tests. Our results suggest that current probes are far from reliable and highlight the need for training data in which truth is decorrelated from confounding concepts.

Primary area:
interpretability and explainable AI
Keywords:
language model interpretability, deception detection, truthfulness, probing classifiers, activation-based monitoring, spurious correlations, role-playing personas, AI safety

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tH6LOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper evaluates whether existing deception probes can identify false statements produced by LLMs adopting anti-factual personas. It introduces a human-reviewed dataset of 8,916 responses from three models across 15 dedicated personas. To alleviate the correlation between system prompts and truth labels, the authors introduce shared-persona prefill (SPP), placing both correct and incorrect responses under the same persona prompt. Several existing probes deteriorate substantially in this setting. Three additional diagnostic datasets investigate sensitivity to response likelihood, persona belief, and instruction compliance. Finally, the authors train a linear probe augmented with instruction-conflicting factual examples, obtaining consistently strong performance across the evaluated models and stress tests.

Critical strengths
  • S1: A useful evaluation that exposes otherwise hidden failures. The paper identifies a concrete limitation of the original on-policy comparison: system prompts perfectly predict truth labels. SPP removes this shortcut and reveals substantial degradation, including below-chance performance for some probes. The human-reviewed dataset and replication across three models make this a consequential contribution to evaluating probe reliability.

  • S2: A simple intervention supported by informative ablations. The instruction-conflicting augmentation provides a practical way to reduce reliance on compliance-related shortcuts. Appendix G separates contributions from the readout position, chat formatting, and augmentation, and includes a domain-matched variant. The transfer from elementary factual training examples to the persona dataset is encouraging evidence that training-data composition substantially affects probe robustness.

Critical weaknesses
  • W1: The diagnostic datasets do not fully disentangle the proposed explanations. Rewarding incorrect answers changes both response likelihood and the behavior favored by the context; violating formatting instructions may also change response likelihood. Thus, performance on these datasets establishes sensitivity to the interventions, but does not uniquely identify which underlying concept a probe tracks. The Gemma 4 DYL result is particularly informative: it achieves approximately perfect performance on all three diagnostics but only 0.171 AUROC under SPP. The authors acknowledge this exception, but it limits how comprehensively the diagnostics explain persona failures. More controlled comparisons are needed to distinguish the effects of correctness, compliance, and likelihood.

  • W2: The strongest failure results concern a constructed prefill setting whose relationship to natural generation remains incompletely established. SPP is a valid stress test, but it places truthful responses into persona contexts where they would not ordinarily be generated. Appendix H usefully shows that generic off-policy prefilling alone cannot explain the degradation. However, it does not establish how much degradation persists when correct and incorrect responses are both naturally generated under the same persona prompt. Since the original on-policy comparison retains the prompt–label correlation, neither setting fully resolves this question. This limits the conclusions about probe reliability during actual persona-conditioned generation and the broader generality of the proposed improvement.

Questions for authors
  • Q1: Section 5.1 samples five responses per question at temperature 1. Must all five responses be correct for each of the three models for a question to be retained? What proportion of candidate questions is removed by this check?
  • Q2: Does the improved probe retain performance on an independent lie/deception benchmark that was not used for layer selection?
3 comments
AuthorsOpenReview ↗

Response Part 1

Thank you very much for your review! We are glad you found our evaluations, datasets, probe training approach and ablations useful contributions. We appreciate your comments and questions and will address them below.

[W1] Disentanglement of confounding concepts

Regarding Weakness 1: This is an excellent point. We agree with the reviewer that our confounder datasets do not perfectly isolate each confounder and that this was not sufficiently discussed in our original submission. We have added a detailed discussion to Appendix D.7 (referenced in Section 4), which discusses for each of the three confounder datasets how each of the three concepts correlates with truth. Ideally, only the target confounder is anti-correlated with truth on a given dataset, while the other two concepts are correlated or uncorrelated with it. Our three datasets largely satisfy this, with a few ambiguities that we highlight below. In general, some ambiguity and entanglement might be unavoidable in the messy domain of natural language. Brief summary:

Likelihood dataset: Likelihood is anti-correlated with truth. Persona belief is correlated with truth, since the active assistant persona likely retains knowledge of the correct/true answer. Compliance is either uncorrelated or anti-correlated, depending on whether following the in-context pattern counts as compliance.

Persona Belief dataset: Persona belief is anti-correlated with truth. Likelihood is correlated with truth, since the true answer is usually the more likely one. Compliance is probably uncorrelated, since the model is not instructed to give either answer but we cannot assert this with certainty.

Compliance dataset: Compliance is anti-correlated with truth. Persona belief is correlated with truth, since the active assistant persona likely retains knowledge of the correct/true answer. Likelihood is as we tested with pairwise answer-likelihood comparisons also largely uncorrelated, with ~60% of honest and ~40% of dishonest answers respectively being more likely and ~49% and 51% for its DYL version.

A more detailed discussion is provided in the new Appendix D.7. Hence, a probe failure on the Persona Belief or Compliance dataset can likely be attributed to the respective confounder, whereas a failure on the Likelihood dataset may also partly reflect compliance. On all three datasets we cannot rule out further unknown confounders, which we now state explicitly in Appendix D.7.

[W2] On-policy truthful replies generated under anti-factual personas

Regarding Weakness 2: We thank the reviewer for raising this excellent point. In response, we collected 40 truthful replies generated by Llama 3.3 70B under the British wizard (Harry potter) and 41 under the medieval physician anti-factual persona, and combined them with 40 and 41 false replies per respectively persona. Because Llama adopts most personas consistently and rarely answers truthfully, this restricted us to two personas and 81 on-policy truthful samples in total. Should the paper be accepted, we will aim for a larger dataset in the camera-ready version. The table below compares the mean AUROC of different probes on the two personas in the three different settings. The performance of several probes degrades in the new persona-prompt on-policy setting compared to the original on-policy setting but not as sharply as under SPP. Our probe continues to work well in the new on-policy setting. This shows that probe degradation also occurs in the analogous on-policy setting but not as pronounced as under SPP. We have added a more detailed explanation to the “Shared-Persona Prefill” paragraph in Section 6 of the updated paper, as well as a new Appendix Section J with the full details and results for all probes.

Probe (mean of British wizard and medieval physician)Original on-policySPPBoth on-policy, same persona prompt
Cooney DYL0.770.310.66
Goldowsky-Dill RepE0.750.450.79
Zou RepE0.890.750.63
Goldowsky-Dill Roleplay0.920.790.84
Cundy DolusChat0.910.840.91
Ours0.960.950.98
AuthorsOpenReview ↗

Response Part 2

[Q1] Belief verification

Regarding Question 1: We thank the reviewer for requesting clarification on this important detail that was not explained in our original manuscript. Indeed, all five responses must be correct for each of the three models for a question to be retained. Approximately 0.5% of questions were removed by this check. We have clarified these details in Section 5.1 of the updated paper.

[Q2] Lie/Deception Benchmarks

Regarding Question 2: We thank the reviewer for raising this excellent question which considerably improves the quality of our manuscript. In response to your suggestion (and that of Reviewer e78Z), we have evaluated our probe on three prior deception benchmarks: AI-Liar by [1], Insider Trading by [2], and Sandbagging by [3]. With this experiment, we would like to understand whether the additional forced-opposite training that was applied to our probe degrades or improves the probe. We therefore compare our probes’ performance on the deception benchmarks to that of the Marks/Bürger Lie probe, whose mix of training data formats is identical to our probe, except for the addition of the forced opposite questions. For a more thorough investigation whether forced-opposite training degrades or improves probe performance, we additionally added forced-opposite training to the Goldowsky-Dill RepE and Goldowsky-Dill Roleplay probes and compared the resulting deception benchmark performance to the original probes. We found that the additional forced-opposite training significantly improves the performance of the Marks/Bürger Lie probe on the held-out deception datasets. Probe performance increases from 0.945 to 0.958 AUROC on Sandbagging, 0.528 to 0.701 on AI Liar and 0.636 to 0.716 on Insider trading. This is a strong improvement but the absolute performance of the Goldowsky-Dill RepE probe is higher on both AI Liar and Insider Trading. The performance of both Goldowsky-Dill probes, which is already very strong, stays roughly constant with forced opposite training but increases substantially on the anti-factual personas and on most confounder datasets. This points to a broader potential of forced-opposite training to improve probes. However, we acknowledge the limitation that for the Goldowsky-Dill RepE probe forced opposite training actually leads to a strong drop in performance on the persona belief dataset from 0.747 AUROC to 0.007 and for the Goldowsky-Dill Roleplay probe to a minor drop on the likelihood dataset from 0.720 AUROC to 0.64. On all other confounder datasets probe performance increases considerably with forced-opposite training. A detailed description and results are provided in the new Section 6.1 of the revised paper.

To clarify layer selection (Section 5.4): for our probe, the layer is chosen solely on a held-out validation split of its own training data (elementary facts and forced-opposite questions), taking the earliest layer with maximal validation AUROC. Neither the anti-factual persona datasets, nor the confounder datasets, nor the three deception benchmarks were used for layer selection. The only exception is the Goldowsky-Dill RepE probe, whose layer was selected on the Pacchiardi et al. role-play data, following the original authors. Its AI-Liar score (0.963) is therefore not a fully held-out result, which favours the baseline rather than our probe.

Thank you again for your review. We sincerely hope these clarifications and updates address your concerns and strengthen your confidence in the work.

[1] Pacchiardi, Lorenzo, et al. "How to catch an ai liar: Lie detection in black-box llms by asking unrelated questions." International Conference on Learning Representations. Vol. 2024. 2024.

[2] Scheurer, Jérémy, Mikita Balesni, and Marius Hobbhahn. "Large language models can strategically deceive their users when put under pressure." arXiv preprint arXiv:2311.07590 (2023).

[3] Benton, Joe, et al. "Sabotage evaluations for frontier models." arXiv preprint arXiv:2410.21514 (2024).

Thank you for the detailed rebuttal and the additional experiments. The response has largely addressed my concerns, and I will maintain my current positive score.

Reviewer_cfYgOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the robustness of existing LLM lie detection probes under anti-factual role-playing personas. The authors construct a dataset of 8,916 human-reviewed responses from three LLMs across 15 personas, and evaluate existing truth and deception probes under both on-policy and shared-persona prefill (SPP) settings.

Critical strengths
  1. The paper addresses an important limitation of existing lie detection probes. In particular, the SPP setting is a useful addition, since the original on-policy evaluation allows the system prompt itself to predict whether a response is truthful or false. By placing both responses under the same persona prompt, the authors provide a more controlled evaluation of whether probes rely on contextual shortcuts rather than factual correctness.

  2. The analysis of spurious correlations is useful and supported by extensive experiments across three models. I also appreciate that the authors go beyond identifying failures and propose a simple training modification to address them. The strong results of the proposed probe suggest that training data composition can play an important role in probe robustness.

Critical weaknesses
  1. The definition of lying and the evaluation objectives are not fully aligned. The paper defines lying relative to the default assistant's beliefs, but a statement that contradicts objective truth may still be consistent with the active persona's beliefs. For example, a historical astronomer expressing a geocentric view is factually incorrect but not necessarily lying. This distinction also matters when comparing probes originally designed for different tasks, including factual truth and strategic deception detection. Therefore, some reported failures may reflect a mismatch between probe objectives and evaluation labels rather than failures in deception detection.

  2. The experimental design may not fully isolate the proposed confounders. In SPP, truthful responses generated under the default assistant are inserted into anti-factual persona contexts, potentially introducing off-policy distribution shifts. Moreover, likelihood, persona belief, and instruction compliance may remain correlated, making it difficult to attribute probe failures to individual factors. Since the proposed probe is trained on instruction-conflicting examples, its strong performance on closely related evaluations may not establish robustness to unseen confounders.

Questions for authors
  1. Why is default-assistant belief preferred over active-persona belief when evaluating deception? How is temporal knowledge considered, particularly for historical personas?

  2. Could the authors provide results using truthful and false responses generated under the same persona prompt, rather than relying on off-policy SPP examples?

  3. How do the authors ensure that the three confounders are independently controlled, and that the proposed probe generalizes beyond the confounders explicitly addressed during training?

5 comments
AuthorsOpenReview ↗

Response Part 1

Thank you very much for your review! We are glad you found SPP a useful addition and liked our analysis of spurious correlations and our training approach for fixing them. We appreciate the comments and questions you raised. We hope our response and additional experiments will convince the reviewer of the soundness and significance of our framework and experiments.

[W1] Monitoring- and safety-motivated definition of lying

Regarding the “definition of lying” in Weakness 1, and the reviewers question “Why is default-assistant belief preferred over active-persona belief when evaluating deception?”: We thank the reviewer for raising this important point which is central to our work and was not clear enough in our original submission. We agree with the reviewer that a historical falsehood generated by a model that has adopted a historical persona, is not necessarily a lie with respect to this persona. However, from a monitoring perspective it would be undesirable and potentially dangerous if lie detectors were to flag lies in accordance with the active persona’s belief and not the default assistant’s belief. If we were to flag falsehoods generated under anti-factual personas as truthful, models adopting anti-factual personas, either via persona drift or strategic anti-factual persona adoption, could output a large variety of harmful false outputs that would go unnoticed by the lie detector. Therefore, we argue, it would be desirable for a lie detector to track lies with respect to the default assistant’s belief. We fully agree with the reviewer that lie detection probes could in principle track lies with respect to the beliefs of the currently active persona and might therefore not flag falsehoods generated under an anti-factual persona. Lie detection probes are typically trained to flag statements “the model believes to be false”. A central point our submission raises is that this approach to lie detection is too simplistic due to the roleplaying nature of language models. Because language models are able to simulate a wide range of different personas, among which the post-trained default assistant persona occupies a special position, it is unclear which “beliefs” a lie detection probe will track: those of the currently active persona or those of the default assistant persona. We argue in our paper that this is a critical aspect of building lie detectors for LLMs and has largely been overlooked in prior work. We have made this important point clearer in our introduction (Line 72-76 of our revised manuscript). We thank the reviewer for requesting clarification on this central point.

[Q1] Historical personas

Regarding the second part of Question 1 (“How is temporal knowledge considered, particularly for historical personas?”): We apply the same criterion here: labels are defined relative to the default assistant’s present-day beliefs, not the knowledge available in the persona’s era.

[W1] Probe objectives

Regarding “reported failures may reflect a mismatch between probe objectives and evaluation labels” in Weakness 1: We thank the reviewer for raising this important point which was not explained in sufficient detail in our original submission. We fully agree that the falsehoods generated by the anti-factual personas are not clear instances of deception. Therefore, while truth probes and lie probes, under our definition of lying, should flag these falsehoods, it is not clear whether deception probes should flag them. However, as we argue at the beginning of Section 5.2 of our original submission, “while some probes are presented as deception probes by their developers, often their training signal is effectively that of a lie detector”. Concretely, the only probes presented as deception detectors by their developers are the Goldowsky-Dill RepE, Goldowsky-Dill Roleplay, MacDiarmid CP and Goldowsky-Dill SD probes. All other probes are either truth or lie probes. The Goldowsky-Dill Roleplay probe is trained on roleplaying scenarios where the LLM lies about facts provided in context and is therefore well described as a lie detector. Hence, the Goldowsky-Dill Roleplay probe can be expected to catch the falsehoods generated under the anti-factual personas under our definition of lying. The Goldowsky-Dill RepE probe is trained on contrastive pairs where the model is either instructed to be honest or dishonest. In our view, such a probe trained to flag dishonest statements, can also be expected to catch lies. The two probes MacDiarmid CP and Goldowsky-Dill SD are each trained on a single Yes/No contrast pair to the questions “Are you a helpful AI?” or “Are you being deceptive?”, respectively. We agree with the reviewer that these two probes are not well described as truth or lie detection probes. However, we believe it is still interesting to test whether they might catch falsehoods generated under an anti-factual persona. We have added a detailed discussion to Appendix B.3 of the revised paper.

AuthorsOpenReview ↗

Response Part 2

[W2&Q2] On-policy truthful replies generated under anti-factual personas

Regarding “In SPP, truthful responses generated under the default assistant are inserted into anti-factual persona contexts, potentially introducing off-policy distribution shifts” in Weakness 2, and Question 2: We thank the reviewer for raising this excellent point and making the suggestion to evaluate probes on on-policy truthful replies generated under an anti-factual persona prompt. We collected 40 truthful replies generated by Llama 3.3 70B under the British wizard (Harry potter) and medieval physician anti-factual personas, respectively, and combined them with 40 false replies per persona. Because Llama adopts most personas consistently and rarely answers truthfully, this restricted us to two personas and 80 on-policy truthful samples in total. Should the paper be accepted, we will aim for a larger dataset in the camera-ready version. The table below compares the mean AUROC of different probes on the two personas in the three different settings. The performance of several probes degrades in the new persona-prompt on-policy setting compared to the original on-policy setting, including Zou Repe and Cooney DYL degrading to near chance discrimination on the British wizard persona, yet overall not as sharply as under SPP. Our probe continues to work well in the new on-policy setting. This shows that probe degradation also occurs in the analogous on-policy setting but not as pronounced as under SPP. We have added a more detailed explanation to the “Shared-Persona Prefill” paragraph in Section 6 of the updated paper, as well as a new Appendix Section J with the full details and results for all probes. In addition, in Appendix H of our original submission (now Appendix I), we prefill both responses under the helpful assistant system prompt and show that probes perform well on this off-policy setting.

Probe (mean of British wizard and medieval physician)Original on-policySPPBoth on-policy, same persona prompt
Cooney DYL0.770.310.66
Goldowsky-Dill RepE0.750.450.79
Zou RepE0.890.750.63
Goldowsky-Dill Roleplay0.920.790.84
Cundy DolusChat0.910.840.91
Ours0.960.950.98

[W2 & Q3] Disentanglement of confounding concepts

Regarding “likelihood, persona belief, and instruction compliance may remain correlated” in Weakness 2, and Question 3 (“How do the authors ensure that the three confounders are independently controlled?”): This is an excellent point. We agree with the reviewer that our confounder datasets do not perfectly isolate each confounder and that this was not sufficiently discussed in our original submission. We have added a detailed discussion to Appendix D.7 (referenced in Section 4), which discusses for each of the three confounder datasets how each of the three concepts correlates with truth. Ideally, only the target confounder is anti-correlated with truth on a given dataset, while the other two concepts are correlated or uncorrelated with it. Our three datasets largely satisfy this, with a few ambiguities that we highlight below. In general, some ambiguity and entanglement might be unavoidable in the messy domain of natural language. Brief summary:

Likelihood dataset: Likelihood is anti-correlated with truth. Persona belief is correlated with truth, since the active assistant persona likely retains knowledge of the correct/true answer. Compliance is either uncorrelated or anti-correlated, depending on whether following the in-context pattern counts as compliance.

Persona Belief dataset: Persona belief is anti-correlated with truth. Likelihood is correlated with truth, since the true answer is usually the more likely one. Compliance is probably uncorrelated, since the model is not instructed to give either answer but we cannot assert this with certainty.

Compliance dataset: Compliance is anti-correlated with truth. Persona belief is correlated with truth, since the active assistant persona likely retains knowledge of the correct/true answer. Likelihood is as we tested with pairwise answer-likelihood comparisons also largely uncorrelated, with ~60% of honest and ~40% of dishonest answers respectively being more likely and ~49% and 51% for its DYL version.

A more detailed discussion is provided in the new Appendix D.7. Hence, a probe failure on the Persona Belief or Compliance dataset can likely be attributed to the respective confounder, whereas a failure on the Likelihood dataset may also partly reflect compliance. On all three datasets we cannot rule out further unknown confounders, which we now state explicitly in Appendix D.7.

AuthorsOpenReview ↗

Response Part 3

[W2 & Q3] Generalization to confounders and external benchmarks

Regarding “its strong performance on closely related evaluations may not establish robustness to unseen confounders” in Weakness 2, and Question 3 (“How do the authors ensure that the proposed probe generalizes beyond the confounders explicitly addressed during training?”): We fully agree with the reviewer that our probes’ strong performance on the closely related confounder datasets may not establish robustness to unseen confounders. As we mention in our Limitations Section, “it is likely that there exist many more spurious confounders beyond those we identified in this paper and our probe might fail under distribution shifts due to those confounders.”. We do not make the claim that our probe will always be robust under other unknown confounders and we highlight this intrinsic difficulty in the construction of probes. However, we would like to note that while forced-opposite training is specifically designed to remove the compliance and likelihood confounders, this is not the case for the persona belief confounder. Even though forced-opposite training is not specifically designed to disentangle the persona belief confounder from truth, it also improves our probes’ accuracy on the persona belief confounder dataset. Moreover, following your concerns and the suggestions of reviewers tH6L and e78Z, we have now tested whether the forced-opposite training we introduce with our probe also improves performance on held-out deception benchmarks. We found that forced-opposite training strongly improves performance on these held-out deception benchmarks. This shows that eliminating the confounders identified in this paper can also improve performance on held-out deception datasets with a-priori unknown confounders. We provide full details in our response to question 4 of reviewer e78Z (which we do not copy here for brevity) and in Section 6.1 of the revised paper. In general, we agree with the reviewer that probe generalization beyond the training data remains an intrinsically hard problem and we do not claim to have fully solved it. As we mention in our Discussion Section, we view “the construction of much broader training datasets and the targeted search for confounders in these datasets” as an exciting future research direction.

Thank you again for taking the time to review the paper and providing helpful feedback. Do the above responses and actions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?

Reviewer_e78ZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether truth/lie/deception probes flag falsehoods according to a model's default assistant beliefs or according to the beliefs of a currently role-played persona. The authors (1) build a human-reviewed dataset of 8,916 on-policy responses from three open-weight LLMs (Llama 3.3 70B, Gemma 3 27B, Gemma 4 31B) across 15 anti-factual personas (reality-based, fictional, historical); (2) evaluate eight probes from prior work, finding that most degrade sharply under "shared-persona prefill" (SPP), where both truthful and persona-consistent responses are prefilled under the anti-factual system prompt, removing the system prompt as a label-predictive cue; (3) construct three confounder test sets in which truth is anti-correlated with response likelihood, persona belief, or instruction compliance, showing every prior probe fails at least one; and (4) propose a probe trained on elementary facts augmented with "forced-opposite" questions, which achieves ≥0.9 AUROC in all settings and perfect scores on the confounder tests.

Critical strengths
  1. This paper focuses on an interesting research question.

  2. The SPP construction is a genuinely useful contribution. Prior probe training/eval data often lets the system prompt predict the label; SPP cleanly removes this confound.

  3. Good ablations. Appendix G cleanly decomposes the gains (EOT token → chat template → forced-opposite), and Appendix H rules out a generic "off-policy prefill hurts probes" explanation for SPP failures.

Critical weaknesses
  1. The "our probe" evaluation is partially circular. The forced-opposite training format is essentially the compliance confounder rendered as training data, so scoring 1.000 on that test is close to train/test alignment rather than evidence of robustness to unknown confounders. The genuinely informative result is the 0.976 under SPP; the abstract's claim of "passing the three confounder tests" overstates what is shown. Evaluation on held-out, unanticipated confounders would be far more convincing.

  2. The link between the confounders and SPP failures is weaker than presented. Both SPP and the confounder tests are constructed so that truth is anti-correlated with the same factors, so showing probes fail on both is close to restating one finding twice. The paper's own data undercuts the claim: on Gemma 4, Cooney DYL passes all three confounder tests yet scores 0.171 under SPP. Section 6's causal framing ("these confounding concepts... explain the failures") should be hedged accordingly.

  3. Llama 3.3 70B is both a subject model and the filtering judge; the final human verification was done by one author with no reported agreement check or second-annotator audit on a subset.

Questions for authors
  1. Are probes trained separately per model? Do you have any cross-model transfer results (train on Llama, test on Gemma)? This matters for practical monitoring.

  2. What fraction of generated question–response pairs was rejected by the judge/human review per persona, and were rejection modes analyzed?

  3. Can you report CIs (e.g., bootstrap) for the main AUROC table, and per-persona results for prior probes under SPP?

  4. How does your probe perform on prior deception benchmarks (Pacchiardi role-play, Goldowsky-Dill setups)? Does forced-opposite training degrade them?

4 comments
AuthorsOpenReview ↗

Response Part 1

Thank you very much for your review! We are glad you find the research question of our paper interesting and SPP a useful contribution. We appreciate the comments and questions you raised. We hope our response and additional experiments will convince the reviewer of the soundness and significance of our work.

[W1] Confounder-Truth Disentanglement

Regarding “our probe evaluation is partially circular” in Weakness 1: We thank the reviewer for raising this important point which was not communicated in sufficient nuance in our introduction and abstract. Indeed, the forced-opposite training is an addition to the training data that is specifically designed to remove the compliance and likelihood confounders. However, we would like to note that the likelihood and compliance confounder datasets differ from the training dataset in both domain and format. Nevertheless, we agree with the reviewer that the perfect AUROC of our probe on the two corresponding confounder datasets is not surprising. In Section 6 of our original submission, we explicitly stated that we “specifically constructed our training dataset in a way that disentangles these confounding concepts from truth” and that the strong probe performance on the respective confounder datasets should therefore “not come as a big surprise” (Line 460-462 of our original submission). However, we fully agree with the reviewer that this fact was not communicated with appropriate nuance in our abstract and introduction. We have now reformulated this important point in both the abstract and introduction accordingly.

[W1] Generalization to confounders and external benchmarks

Regarding “robustness to unknown confounders” in Weakness 1: We thank the reviewer for pointing this out. As we mention in our Limitations Section, “it is likely that there exist many more spurious confounders beyond those we identified in this paper and our probe might fail under distribution shifts due to those confounders.”. We do not make the claim that our probe will always be robust under other unknown confounders and we highlight this intrinsic difficulty in the construction of probes. However, we would like to note that while forced-opposite training is specifically designed to remove the compliance and likelihood confounders, this is not the case for the persona belief confounder. Even though forced-opposite training is not specifically designed to disentangle the persona belief confounder from truth, it also improves our probes’ accuracy on the persona belief confounder dataset. Moreover, following the great suggestion of the reviewer (question 4), we have now tested whether the forced-opposite training we introduce with our probe improves or degrades performance on held-out deception benchmarks. We found that forced-opposite training considerably improves performance on these held-out deception benchmarks (+0.173 on AI-Liar, +0.080 on Insider Trading Report, and +0.013 on Sandbagging). We provide full details in our response to question 4. This shows that eliminating these confounders can also improve performance on held-out deception datasets with a-priori unknown confounders.

[W2] Explaining probe failures under SPP

Regarding Weakness 2: We thank the reviewer for raising this important point. We agree with the reviewer that our confounder datasets are specifically constructed such that truth is anti-correlated with the same confounders as under SPP. However, it is not at all obvious that eliminating these confounders from probe training data would also lead to improved performance under SPP. It might have been the case that the probe failures under SPP had other causes than these confounders. In that case, eliminating the compliance and likelihood confounders via forced-opposite training, would not have led to improved performance under SPP, only on the confounder datasets. In response to this excellent point of the reviewer, we have now conducted additional experiments, applying forced opposite training to the Goldowsky-Dill RepE probe that previously failed under SPP (AUROC 0.438). In Appendix H of the revised manuscript, we show that the Goldowsky-Dill RepE probe with additional forced-opposite training performs much better under SPP (AUROC 0.760). This provides more evidence that the probe failures under SPP are, to a certain extent, driven by the identified confounders. However, we fully agree with the reviewer that the identified confounders are not the whole explanation for probe failures under SPP and other factors or confounders remain to be identified. As we mentioned in our original manuscript, the fact that Cooney DYL achieves near-perfect AUROC on all three confounder tests, yet only 0.171 under SPP, points to the fact “that likely further confounding factors remain to be identified“.

AuthorsOpenReview ↗

Response Part 2

[W3] Labelling agreement check and detailed statistics

Regarding Weakness 3: We thank the reviewer for raising this important point. To verify the Llama judge, we relabeled a random subset of 283 question-answer pairs drawn before judge filtering, so including both accepted and rejected pairs, with GPT-5-nano and measured an agreement of 97.5%. In addition, to validate the manual human review, a second author independently labeled a random sample of 100 questions-answer pairs (pre-filtering, including rejected pairs) and agreed with the first author on 97 of them. We have added a description of this additional validation to Section 3.1 of the revised manuscript, including detailed tables of annotator-annotator agreement, judge-judge agreement and per-persona rejection rates.

[Q1] Cross-model transfer

Regarding Question 1: Yes, each of our probes is trained separately per model. To our understanding, training a probe on one model and testing the probe on another model comes with significant complications. For instance, the dimensionality of the activation vectors often differ between models and the representation spaces need not align. To test a probe on another model, representational alignment techniques, such as [1], would be needed which we believe to be out of scope for our paper. In case we misunderstood your question (if for instance you were interested in the performance of a probe trained on Llama, tested on anti-factual persona responses generated by Gemma, prefilled off-policy for Llama), please clarify and we will gladly provide you with the respective experimental results.

[Q2] Judge sample rejection modes

Regarding Question 2: We thank the reviewer for this question. The judge reviewer rejected approximately 95% of all questions generated via the agentic pipeline detailed in Section 3.2. This is due to the extensive refinement to both the anti-factual persona prompts and questions until the persona was reliably elicited. From the dataset that passed judge filtering, the human reviewer rejected 125 out of 1630 questions. We have added a detailed description, as well as your requested analysis of rejection modes, to Appendix A.2 of the paper.

[Q3] Confidence intervals and per-persona results

Regarding Question 3: We thank the reviewer for this excellent suggestion. We have added confidence intervals to our main AUROC table (Table 2) and added per-persona results for prior probes under SPP in Appendix E.1. The confidence results include the Ours probe having an SPP mean-AUROC of 0.976 [0.970, 0.982]. The per-persona results table additionally includes that the DYL is below chance on 13/15 and Goldowsky-Dill RepE on 9/15 personas.

AuthorsOpenReview ↗

Response Part 3

[Q4] Deception Benchmarks

Regarding Question 4: We thank the reviewer for this great suggestion which considerably strengthens our manuscript. We have evaluated our probe on three prior deception benchmarks: AI-Liar by [2], Insider Trading by [3], and Sandbagging by [4]. To evaluate whether the forced-opposite training degrades or improves the probe, we compare its performance on the deception benchmarks to that of the Marks/Bürger Lie probe, whose mix of training data formats is identical to our probe, except for the addition of the forced opposite questions. For a more thorough investigation whether forced-opposite training degrades or improves probe performance, we additionally added forced-opposite training to the Goldowsky-Dill RepE and Goldowsky-Dill Roleplay probes and compared the resulting deception benchmark performance to the original probes. We found that the additional forced-opposite training significantly improves the performance of the Marks/Bürger Lie probe on the held-out deception datasets. Probe performance increases from 0.945 to 0.958 AUROC on Sandbagging, 0.528 to 0.701 on AI Liar and 0.636 to 0.716 on Insider trading. The performance of both Goldowsky-Dill probes, which is already very strong, stays roughly constant with forced opposite training but increases substantially on the anti-factual personas and on most confounder datasets. However, we acknowledge the limitation that for the Goldowsky-Dill RepE probe forced opposite training actually leads to a strong drop in performance on the persona belief dataset from 0.747 AUROC to 0.007 and for the Goldowsky-Dill Roleplay probe to a minor drop on the likelihood dataset from 0.720 AUROC to 0.64. On all other confounder datasets probe performance increases considerably with forced-opposite training. A detailed description and results are provided in the new Section 6.1 of the revised paper.

Thank you again for taking the time to review the paper and providing helpful feedback. Do the above responses and actions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?

[1] Puri, Bruno, et al. "Atlas-alignment: Making interpretability transferable across language models." arXiv preprint arXiv:2510.27413 (2025).

[2] Pacchiardi, Lorenzo, et al. "How to catch an ai liar: Lie detection in black-box llms by asking unrelated questions." International Conference on Learning Representations. Vol. 2024. 2024.

[3] Scheurer, Jérémy, Mikita Balesni, and Marius Hobbhahn. "Large language models can strategically deceive their users when put under pressure." arXiv preprint arXiv:2311.07590 (2023).

[4] Benton, Joe, et al. "Sabotage evaluations for frontier models." arXiv preprint arXiv:2410.21514 (2024).

Dear Reviewer,

We’re writing to follow up on our rebuttal, as we haven’t seen any further comments and the discussion period ends tomorrow (Nov 18, AoE). In our rebuttal, we carefully addressed all the concerns you raised, and made sure to clarify all the questions. We conducted additional evaluations to demonstrate that forced-opposite training improves performance on three held-out deception benchmarks, improved the framing of our mentioned claims and added a second-annotator agreement check, confidence intervals and per-persona results. If there are any remaining questions or concerns, we would truly appreciate the chance to clarify.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.