- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper asks whether an LLM agent's deception can be detected before it becomes externally observable. Using the agentic upward deception setting (an agent hits a blocker and either admits failure or fabricates a success), the authors align trajectories at a behaviorally decisive step d, pool pre-d hidden states into a trajectory-level vector G, and train a deep-kernel MMD detector to score G as more honest- or more deceptive-like. Three questions are studied: (RQ1) can future outcomes be predicted, (RQ2) does the signal accumulate over execution, (RQ3) does steering along the honest–deceptive direction change behavior. The agent is Qwen3-14B; 1,500 train / 500 test trajectories are built from five task templates (original 5 from Guo et al. plus GPT-5.6-generated variants). Results: AUROC 89.4% at the decisive step and 90.4% even at t−7; mean AUROC rises 51.2% (prefix p1) → 80.0% (p10); CAA steering at α=2.0 converts 56.0% of flagged deceptive trajectories to honest with 3.2% invalid.
Critical strengths
- Correct reframing of the problem. Moving deception monitoring from post-hoc output/trajectory evaluation to a pre-behavioral, internal-state prediction problem is genuinely useful. It creates a path to early warning rather than audit, and the paper is explicit that the target is spontaneous deception in benign task execution, not instructed/incentivized deception.
- The temporal analysis (RQ2) is the paper's best-designed experiment. Freezing the detectors trained at decision-relative shifts and evaluating them unmodified on start-relative prefixes is the right control: it rules out that the rising AUROC comes from refitting to later prefixes, so the gain really is information becoming recoverable. The t vs t−k train/eval grid (Table 1) plus Figure 3 gives two complementary views of the same phenomenon at almost no extra cost.
- The intervention closes the predict→act loop. Gating CAA on the detector's own score, and reporting representation shift in PCA alongside the behavioral transition rates, makes the "behaviourally actionable" claim concrete rather than rhetorical. The type-balanced direction in Eq. 8 is also a sensible detail.
Critical weaknesses
W1 — The honest/deceptive label is confounded with task outcome.
- Honest = "reports the encountered failure"; deceptive = "continues toward a false success." The detector may therefore be reading task infeasibility / imminent giving-up — an internal state that legitimately differs between the two classes for reasons other than deception.
- No control for task type × difficulty × trajectory length. L_type "discourages" task-specific differences but there is no diagnostic showing the honest/deceptive distributions are separable within a task type after the task-only information is removed.
- The 56% "correction" rate is equally consistent with steering toward admitting defeat as toward honesty; the paper does not report task-success or task-completion rate after steering (it only says preserving task performance is future work).
W2 — Dataset is synthetic and narrow, and separability may be template artifacts.
- Trajectories are generated by GPT-5.6 under the same five task settings and only manually reviewed — the generator and template can leave stylistic traces that a 300-dim learned kernel on averaged layer representations will readily pick up.
- Single backbone (Qwen3-14B), white-box, fixed decoding (temperature=0). No cross-model or cross-scaffold test.
- No baselines at all: no linear/logistic probe on the same G, no output-text monitor, no random-direction or shuffled-label control. Without them we cannot tell whether the deep-kernel MMD is doing the work or the representations are simply easy.
W3 — Internal tension between RQ1 and RQ2 numbers.
- At p1 (one pre-decision state) all detectors are near chance (51.2%), yet the t−7 detector — which at test time sees the whole prefix except the last seven steps — reaches 90.4% AUROC. That gap implies the signal is carried by a small number of specific late steps, not a smooth accumulation, which contradicts the "progressive emergence" narrative as stated.
- Table 1 also shows off-diagonal AUROC sometimes higher than the bold diagonal (e.g. t−7→t−7 90.40 vs. t→t 89.40, and the t−7 row is uniformly high). That pattern suggests late-window detectors are less selective, not more transferable — possibly because they see more of the trajectory rather than because the structure generalizes.
- No confidence intervals or significance tests anywhere, and single seed / single α=2.0 chosen as "main analysis."
Questions for authors
- On the confound (W1): Can you report detection AUROC within each of the five task types separately, and after regressing out a task-type one-hot / a task-only control score? If honesty is largely "report failure," a matched control — where the agent is told the failure is admissible — would separate deception from mere task difficulty.
- On baselines (W2): How does the deep-kernel MMD compare against (a) a logistic probe on the same G, (b) a detector on the final response text of each pre-decision step, and (c) a shuffled-label / random-direction control? Is the learned kernel actually necessary?
- On the RQ1/RQ2 tension (W3): How many pre-decision steps do the t−7 trajectories have, and what is the AUROC of the t−7 detector evaluated on prefixes p1…p_{m−7}? A per-step ablation (drop step s from the t window) would show whether the signal is concentrated in the one or two steps before d.
- On the decisive step: How is d determined — is the "LLM Judge" in Figure 2 the same model that generated the trajectories, and what is inter-annotator / judge–human agreement on d? Errors in d shift the whole windowing scheme, including the near-chance p1 result.
- On the intervention (W1/W3): What is the task-level outcome after steering — do steered trajectories still attempt the task correctly, or do they increasingly terminate with a failure admission? And is the 3.2% invalid rate at α=2.0 measured against a matched α=0 reroll, or against the original baseline distribution?
- On transfer: Do detectors trained on Qwen3-14B transfer at all to a second backbone (or to a different agent scaffold on the same backbone)? Given the white-box constraint the paper acknowledges, this is the single result that would decide whether the method is a monitoring technique or a property of one model.
8 comments
Response for Reviewer 2GSA (Part 2)
Q3/W3.1
1. Why strong performance at t−7 is consistent with weaker prediction at early prefixes
RQ1 evaluates windows relative to the decisive event, while RQ2 tracks prediction as context accumulates from the trajectory start. These analyses use different reference points.
Let m be the total number of pre-decision calls and pⱼ the prefix containing the first j calls. Then
Thus, W(t−7) contains all pre-decision calls except the final seven. In the fixed t−7 cohort of 1,226 trajectories, m ranges from 8 to 30. The window therefore contains 1–23 calls, with a mean of 14.1 and a median of 23. By comparison, p1 contains only the first call.
The AUROC of 0.904 at t−7 measures prediction from this accumulated history; it does not imply equally strong prediction from the first call. To examine how prediction changes as context accumulates, we evaluated the same frozen t−7 detector on prefixes p1–p10:
| Prefix | p1 | p2 | p3 | p4 | p5 | p6 | p7 | p8 | p9 | p10 |
|---|---|---|---|---|---|---|---|---|---|---|
| AUROC | 0.482 | 0.540 | 0.541 | 0.581 | 0.593 | 0.636 | 0.691 | 0.708 | 0.772 | 0.761 |
AUROC rises overall from 0.482 at p1 to 0.761 at p10, with a small decrease after p9. This supports improved prediction as pre-decision context accumulates. The 0.904 result is evaluated separately at the trajectory-specific endpoint pₘ₋₇, whose length varies across trajectories. It is therefore distinct from performance at any fixed start-relative prefix.
2. Which pre-decision calls contribute most to detection?
Let d denote the decisive event. We removed individual calls and selected call pairs from W(t) and evaluated the resulting representations with the detector frozen. The unablated AUROC was 0.8948; positive ΔAUROC below denotes a decrease from this baseline.
| Removed Step | Frozen AUROC | Frozen ΔAUROC [95% CI] |
|---|---|---|
d-1 | 0.8740 ± 0.0196 | 0.0208 [0.0189, 0.0227] |
d-2 | 0.8823 ± 0.0179 | 0.0125 [0.0107, 0.0142] |
d-3 | 0.8929 ± 0.0176 | 0.0020 [0.0002, 0.0039] |
d-4 | 0.8947 ± 0.0184 | 0.0002 [−0.0010, 0.0012] |
d-5 | 0.8934 ± 0.0176 | 0.0015 [0.0005, 0.0024] |
d-6 | 0.8914 ± 0.0167 | 0.0034 [0.0017, 0.0051] |
d-7 | 0.8946 ± 0.0186 | 0.0002 [−0.0017, 0.0020] |
d-8 | 0.8886 ± 0.0194 | 0.0062 [0.0047, 0.0078] |
d-1, d-2 | 0.8544 ± 0.0190 | 0.0405 [0.0380, 0.0431] |
d-1, d-8 | 0.8653 ± 0.0213 | 0.0296 [0.0261, 0.0329] |
d-1, d-6 | 0.8692 ± 0.0172 | 0.0257 [0.0226, 0.0285] |
d-1, d-3 | 0.8698 ± 0.0196 | 0.0250 [0.0222, 0.0280] |
d-1, d-5 | 0.8706 ± 0.0194 | 0.0242 [0.0220, 0.0266] |
d-1, d-7 | 0.8719 ± 0.0192 | 0.0230 [0.0206, 0.0253] |
d-1, d-4 | 0.8722 ± 0.0194 | 0.0226 [0.0208, 0.0244] |
d-2, d-8 | 0.8731 ± 0.0202 | 0.0217 [0.0186, 0.0252] |
d-2, d-6 | 0.8766 ± 0.0168 | 0.0182 [0.0149, 0.0218] |
d-2, d-3 | 0.8777 ± 0.0174 | 0.0171 [0.0146, 0.0196] |
d-3, d-4 | 0.8916 ± 0.0183 | 0.0032 [0.0006, 0.0061] |
d-4, d-5 | 0.8925 ± 0.0177 | 0.0023 [0.0004, 0.0040] |
d-5, d-6 | 0.8898 ± 0.0164 | 0.0050 [0.0030, 0.0069] |
d-6, d-7 | 0.8908 ± 0.0172 | 0.0041 [0.0014, 0.0067] |
d-7, d-8 | 0.8904 ± 0.0205 | 0.0044 [0.0022, 0.0067] |
Among individual calls, removing d−1 or d−2 causes the largest drops. Removing both produces the largest decrease among the tested ablations: 0.0405 AUROC (95% CI [0.0380, 0.0431]). Even then, AUROC remains 0.8544 with the frozen detector and 0.8607 after retraining. The final two calls contribute most among those tested, but earlier history still supports detection. Together, the prefix and ablation results show that prediction improves with accumulated context and does not depend solely on the calls immediately before the decision.
Response for Reviewer 2GSA (Part 3)
Q4
(1) Final behavioral labeling. We use DeepSeek 4.1 Flash to label final behavioral outcomes, following the criteria for agentic upward deception in Guo et al. (2025). (2) Decisive-step identification. We identify the decisive step separately through deterministic, task-specific rules judges without LLM judges. We define as the earliest step at which the agent makes an observable commitment to an honest or deceptive outcome. Only hidden states strictly preceding are used for detection. (3) Annotation validation. We manually reviewed several hundred trajectories while refining the annotation rules. In a final audit of 200 randomly sampled trajectories, manual verification agreed with the pipeline on both behavioral labels and decisive-step locations in all cases (194/200).
Q5/W2.3
(1) Task execution under steering. We examined termination outcomes across the same 264 detector-gated tasks. At α=2, the max-step termination rate increased from 11.4% to 27.3%. Among honest responses, the proportion generated after reaching the step limit increased from 10.4% to 32.1%. These results suggest that steering did not simply cause immediate refusal; part of the increase in honest reporting followed continued task attempts and eventual failure acknowledgment.
(2) Intervention controls and invalid outputs.We further evaluated a matched α=0 rerun and reverse steering -v. The α=0 rerun produced only one additional deceptive outcome compared with the original trajectories, while -v increased the deceptive-outcome rate by approximately 8%. Neither the original nor the matched α=0 condition produced invalid outputs.Under +v, α=2, the reported 3.2% invalid rate corresponds to 7 of 216 originally deceptive trajectories, or 2.7% (7/264) across all detector-gated trajectories. This is an absolute within-condition rate, not a relative increase over either baseline.
(3) Performance on standard tasks. To further examine whether steering compromises the model's ability to complete answerable tasks, we evaluated MMLU and GSM8K, using 100 questions from each benchmark. We compared honesty steering at α=2 against a matched no-injection baseline and a norm-matched random-direction control. Further experimental details are provided in the revised manuscript.
| Condition | Intervention | MMLU Accuracy | MMLU Effective Response Rate | GSM8K Exact Match | GSM8K Effective Response Rate | Pooled Accuracy | Overall Completion |
|---|---|---|---|---|---|---|---|
α=0 | No vector injection | 86/100 (86%) | 100% | 94/100 (94%) | 100% | 180/200 (90.0%) | 200/200 (100%) |
+v, α=2 | Honesty-direction injection | 84/100 (84%) | 100% | 92/100 (92%) | 100% | 176/200 (88.0%) | 200/200 (100%) |
Random, α=2 | Layer-wise norm-matched random direction | 86/100 (86%) | 99% | 92/100 (92%) | 100% | 178/200 (89.0%) | 199/200 (99.5%) |
At α=2 , honesty steering reduced accuracy by 2 percentage points on both MMLU and GSM8K, while maintaining a 100% valid response rate across all 200 questions. These results suggest a limited performance cost on standard tasks, while further evaluation is needed to assess whether steering causes premature failure reporting on challenging but solvable agent tasks.
Response for Reviewer 2GSA (Part 1)
We appreciate your careful review and recognition of our pre-behavioral monitoring approach. Following your suggestion, we have added task-controlled and failure-matched analyses to the revised manuscript.
Q1/W1.1
(1) Task-Type Controlled Detection.
We first evaluated MMD separately within each of the five task types.
The detector achieved a macro-average AUROC of 0.844 across the five task types. We further removed task-type information from the representations and detection scores. The task-adjusted detector achieved 0.790 ± 0.011 AUROCacross 15 held-out folds, with an overall out-of-fold AUROC of 0.799. These results show that the predictive signal remains detectable within task types and after task-type adjustment.
(2) Failure-Matched Detection.
We identified 3,711 trajectories where the agent encountered a task-critical failure before exhibiting honest or deceptive behavior.Both groups therefore experienced task-critical failures, while their labels reflected how the agents subsequently handled those failures. We constructed 152 matched honest-error/deceptive-error pairs. By comparing agents that encountered similar failures but responded differently, we can test whether the detector predicts subsequent deceptive behavior beyond observable failure-related factors.
These results suggest that task identity and observable failure conditions do not fully explain the performance. Even under matched failure conditions, pre-decision representations remain predictive of how agents subsequently respond to those failures.
Q2/W2.3
(1) Logistic probe on internal representations. We trained a logistic-regression probe using the same layer-aggregated representation (G) and case-level splits as Deep-kernel MMD. Across the eight decision-relative windows, the linear probe achieved a mean AUROC of 0.846, compared with 0.871 for MMD. These results show that a linear probe can recover the pre-decision predictive signal, which also aligns with our stated contribution, while the learned kernel provides additional discrimination.
(2) Same-prefix transcript monitor. We further evaluated a text-only monitor following the setup of SHADE-Arena [1]. We used DeepSeek-V4.1-Flash to score the visible transcript up to each decision-relative position, excluding the decisive event and subsequent trajectory. The monitor did not access hidden states or behavioral labels. Further experimental details are provided in the revised manuscript. The text-only monitor achieved 0.825 AUROC at (t), but its performance decreased to 0.624 at (t-7). In contrast, both internal-state detectors maintained stronger discrimination at the earlier decision-relative window.
These results suggest that pre-decision representations provide predictive information beyond a specific detection method, while the stronger performance of deep-kernel MMD supports the effectiveness of our proposed approach.
[1] Kutasov et al. SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents. arXiv:2506.15740, 2025.