- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates whether probes trained on LLM internal activations can detect knowing falsehoods, defined as statements the model believes to be false, and generalize beyond their training settings. Its main contributions are FIBS, a dataset of 161,760 labelled conversations spanning 29 families, and an axial probe architecture that combines information across layers and token positions. The authors examine how training-data quantity and diversity, probe architecture, and model size affect performance. Validation includes scaling experiments and ablations, comparisons with text-based monitors on the SHADE-Arena sabotage benchmark, and evaluations involving cue-following, political censorship, and hidden-goal denials. The strongest probes achieve 0.988 AUROC on SHADE-Arena, outperforming the Haiku 4.5 monitor, and demonstrate transfer to several introspective falsehood settings, although performance varies across hidden-goal models.
Critical strengths
-
Systematic investigation of whether knowing falsehoods can be detected from LLM internal representations. The authors combine a large, diverse dataset with comparisons across probe architectures, model sizes, and evaluation settings. This breadth provides substantial empirical support for their finding that probes can detect knowing falsehoods and generalize to several settings beyond their training data.
-
Learned goals: Probes trained on contextual falsehoods can detect dishonest denials of learned hidden goals, revealing information unavailable from the visible text alone.
Critical weaknesses
- Limited validation in practical settings: Although the probes perform well in controlled experiments, their effectiveness in deployment remains uncertain.
- Unclear connection between detection and prevention: The paper demonstrates that some falsehoods and hidden-goal denials can be detected, but does not establish how these detections translate into reduced harm. A practical monitoring system needs an intervention strategy, and evidence that these interventions prevent harmful actions or improve honesty.
Questions for authors
- Q1: Practical deployment: What evidence supports the probes’ effectiveness beyond the controlled settings evaluated here, particularly on unfamiliar tasks, other model families, or agents attempting to evade monitoring? How would false-positive rates and computational overhead affect their practical usefulness?
- Q2: From detection to prevention: How do you envision integrating the probes into a system that intervenes when dishonesty is detected? Since monitoring occurs after tool calls, could detection happen early enough to prevent harm, and what additional evaluation would be needed to establish that the proposed monitoring approach improves safety?
Request for feedback
We thank the reviewers once again for thoughtfully engaging with our submission, their overall positive feedback, and their suggestions which have greatly improved our paper.
With only 2 days left in the rebuttal period, we would love to hear feedback on the substantial improvements we made to the paper based on reviewer feedback, including a more realistic evaluation setting (BashArena), a new model family for scaling laws (Nemotron) and a frontier open-weight model (Kimi K3).