acceptodds
Withdrawn from ICLR 2027

A Lie Detector Test for Language Models: Reading Knowledge a Model Won’t Reveal

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language models can hold knowledge they do not report. A model may sandbag on a capability evaluation, or answer against what it internally knows, and its outputs alone cannot tell whether it is hiding an answer or simply does not have one. We borrow the Concealed Information Test, a forensic method that identifies guilty knowledge by presenting a suspect with the true detail among plausible decoys and measuring a stronger response to the item they recognize. Our method, Probe of Internal Recognition (PIR), does the same inside a model. It presents a question with its candidate answers and reads, from the model's internal states, which candidate the model recognizes as correct. PIR is reference-free, needing no honest reference model and no labeled truth corpus. Across eight models from five families (Gemma, Qwen, Llama, Mistral, and Phi), PIR recovers the recognized answer at 0.70 to 0.87 balanced accuracy, well above the 0.28 to 0.40 unknown-item baseline and the 0.25 chance rate. It stays readable across every form of concealment we test, from prompted deception and trained sandbagging to external password-locked and circuit-broken checkpoints, with recognition between 0.85 and 0.93. When the model hides a known answer, recognition stays high. When unlearning removes the knowledge, recognition drops to the level of a question the model never knew. PIR therefore separates a model that will not answer from one that cannot, which supports sandbagging audits and unlearning verification. The signal is causal, adds information beyond black-box behavioral cues, and extends from multiple-choice questions to free-form generation.

Primary area:
interpretability and explainable AI
Keywords:
We adapt the forensic Concealed Information Test to read a model's concealed answer from its internal recognition of the correct option among foils, and show the readout tells concealment apart from genuine unlearning.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ooYkOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Probe of Internal Recognition (PIR), which learns a direction from the difference between correct and distractor option activations and uses it to decode which candidate answer a model appears to recognize internally. The authors evaluate this readout on multiple-choice benchmarks across several model families, prompted and trained concealment, external sandbagging models, and RMU/NPO unlearning checkpoints. They also report an output–readout divergence detector, a steering intervention, and an extension to free-form questions. Their main claim is that the recognition signal survives concealment but disappears when knowledge is unlearned.

Critical strengths

The item-contrast readout is simple and testable. Results on external concealment models, together with the steering control, provide evidence that the measured signal is not merely a copy of the model’s emitted response.

Critical weaknesses
  • Limited novelty beyond the application. Eq. (1) is a mass-mean probe [1] over option spans, and reading latent knowledge from models trained to answer incorrectly was shown by Mallen et al. [2]. What is new is the concealment-versus-unlearning boundary and the deployment detector, which is where the concerns below concentrate.

  • Option-order confounding is not tested. Under an autoregressive model, an option's last-token activation depends on the options before it. Balanced accuracy aggregates over positions and does not show whether PIR decodes the same question consistently after its options are reordered. The below-chance numbers the paper reports (off-distribution unknown-item baseline 0.21, untriggered anti-probe 0.11, against 0.25 chance) indicate the direction tracks an option-level feature that anti-correlates with correctness, so the central decoding result may rest on presentation order rather than a stable recognition signal.

  • The unlearning result does not establish the claimed hiding-versus-erasure boundary. Erasure is defined behaviorally (Sec. 3.4), so PIR adds nothing on that side. The paper reads a baseline-level readout as evidence the knowledge is gone (Sec. 5), but Sec. 4.6 shows capability can be fully retained while PIR falls to the same baseline. A silent readout is therefore consistent with erasure, with suppression at the read locus, and with RMU’s intended scrambling of the very representations PIR measures. The domain-specific result also rests on a single in-house RMU run with an unspecified protocol, and bootstrap intervals do not capture across-run variability.

  • External validity. The naturalistic organism gives 0.63 on the concealed subset against 0.81 on the full known set, so the signal weakens as concealment becomes more natural. The strongest numbers come from prompt deception and prefix triggers, the settings least representative of real sandbagging.

  • Presentation. Table 3 reports raw-readout numbers the paper itself calls depressed for gemma-3, Table 1 mixes single-layer and multilayer internal detectors, and Sec. 4.6 embeds a retracted result inline, which makes it unclear which numbers are final.

[1] Marks, S., & Tegmark, M. (2023). The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. arXiv preprint arXiv:2310.06824. [2] Mallen, A., Brumley, M., Kharchenko, J., & Belrose, N. (2023). Eliciting latent knowledge from quirky language models. arXiv preprint arXiv:2312.01037.

Questions for authors
  • On the same held-out questions, how often does PIR’s decoded answer remain unchanged across option permutations?

  • Why are the off-distribution unknown-item baseline (0.21) and the untriggered anti-probe accuracy (0.11) below the 0.25 chance rate? Which option feature is the direction tracking on items the model does not know?

  • Please specify the in-house RMU runs: forget/retain sets, targeted layers, hyperparameters, checkpoint-selection criterion, and the elicitation protocol used to confirm failed recovery.

Reviewer_hVbFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Probe of Internal Recognition (PIR), a method for recovering knowledge concealed by language models through contrastive analysis of internal activations. PIR constructs a recognition direction from correct answers and distractors, allowing it to identify answers that models internally recognize but do not output. Experiments across multiple model families and concealment mechanisms demonstrate its effectiveness in recovering hidden answers, distinguishing concealment from unlearning, and detecting sandbagging. The authors further investigate causal steering, extend PIR to free-form generation, and examine its vulnerability to adversarial evasion.

Critical strengths
  1. Comprehensive experimental evaluation. The paper evaluates PIR across eight models from five families and multiple concealment mechanisms, including prompted deception, trained sandbagging, and external password-locked and circuit-broken models. The results consistently demonstrate that internal activations can reveal answers that models deliberately conceal.

  2. Meaningful analysis beyond recognition accuracy. The paper investigates the distinction between concealment and unlearning, provides causal evidence through activation steering, and shows that internal signals can complement black-box detection. These experiments strengthen contribution and highlight potential applications in model safety auditing.

Critical weaknesses
  1. Limited methodological novelty. The core method relies on a straightforward contrastive activation direction, closely related to existing linear probing and latent knowledge elicitation methods (Burns et al., 2023; Mallen et al., 2024; Orgad et al., 2025). In particular, Mallen et al. (2024) already demonstrated that linear probes can recover knowledge from models fine-tuned to produce incorrect answers. The paper lacks direct comparisons with these closely related methods, making it difficult to assess the specific advantages and novelty of PIR.

  2. Insufficient evidence for knowledge erasure and limited practical applicability. The drop in recognition accuracy after unlearning does not necessarily imply knowledge erasure, as prior work has shown that apparently forgotten knowledge can remain recoverable (Łucki et al., 2025; Hu et al., 2025). Section 4.6 further demonstrates that models can retain their capabilities while reducing PIR accuracy to the unknown-item baseline, weakening the proposed concealment-versus-erasure distinction. Moreover, deployment-time detection requires access to the pre-modification checkpoint, limiting its applicability to realistic third-party safety evaluations.

Questions for authors
  1. Concealment vs. erasure: Given the anti-probe results in Section 4.6, how can PIR reliably distinguish genuine knowledge erasure from changes in internal representations that make the knowledge inaccessible to the probe?

  2. Comparison with existing methods: How does PIR compare with established linear probing or latent knowledge elicitation methods under the same experimental settings? What advantages are specific to the proposed item-contrast formulation?

  3. Deployment practicality: Can the concealment detector in Section 4.4 and free-form extension in Section 4.5 operate using only the deployed model, without access to its pre-modification checkpoint?

  4. Causal steering controls: How exactly is the output direction constructed in Section 4.2? Could the authors also report a no-intervention baseline to better quantify the causal effect of recognition steering?

  5. Transferability of rocognition direction: The transferability among multi-choice tasks have been investigated. Can the recognition direction transfer from multiple choice questions to free form questions? It may help the free form extension avoid the need of base pre-modification checkpoint.

Reviewer_FUWxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PIR, a method to read which answer a language model internally recognizes as correct, even when it outputs a different one. It adapts the Concealed Information Test from lie detection. For each multiple-choice option, PIR takes the hidden state at the option's last token. It projects that state onto a direction defined as the mean difference between correct and incorrect options on questions the model knows. The top-scoring option is the recognized answer, and a mismatch with the model's output flags concealment. The readout recovers hidden answers across 8 models from 5 families. It works under prompted deception, trained sandbagging, and on external password-locked and circuit-broken models. After unlearning (RMU, NPO), it drops to the level of questions the model never knew. From this, the authors argue that PIR can tell a model hiding knowledge apart from one whose knowledge was erased. They also show through steering that the direction is causal, that it adds to black-box detectors, and that the method extends to free-form answers.

Critical strengths
  1. The method is simple and cheap (it needs only a few labeled questions). It works across many model families and on sandbagging models built by other groups, not just the authors' own.

  2. The authors are upfront about the limits. They show that a model trained against the probe can defeat it, and they retract an earlier result that turned out to be an in-sample artifact.

Critical weaknesses
  1. The claim that PIR separates hiding from erasure is not supported. RMU works by scrambling the model's internal activations on the unlearned topic, and that is exactly where PIR reads. So a drop in the readout does not show that the knowledge is gone. Prior work cited by the authors shows that RMU knowledge comes back with light finetuning. The paper's own anti-probe model also keeps its capability while PIR reads it as unknown. So a low readout cannot certify erasure, which is one of the two main uses the paper claims.

  2. The method is not very novel, and it is not compared with other white-box methods. It is a standard mean-difference probe, close to Marks & Tegmark (2024). The only baselines are text-only and black-box ones.

Questions for authors

Clarifications (could change my assessment):

  1. If the RMU models are finetuned until their WMDP accuracy comes back, does the PIR readout come back too?
  2. How does PIR compare with other white-box methods, such as a linear probe on the answer token or noise injection (Tice et al., 2025)?
  3. Can the deployment detector work without the base checkpoint? For example, could it use a direction fit on the deployed model's own questions in a domain where the model does not sandbag?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.