acceptodds
Under review as a conference paper at ICLR 2027

Before Agent Tells The Lie: Has Deception Already Been Represented?

Abstract

Large language model (LLM)-based agents can exhibit deceptive behavior during task execution, including hiding failures, fabricating results, or falsely signaling task completion. Existing monitoring approaches mainly detect deception after it appears in observable actions or outputs. In this paper, we investigate whether deceptive behavior can be predicted from an agent’s internal representations before it becomes externally visible. We frame deception monitoring as a trajectory-level representation analysis problem and align agent trajectories around key decision points. Using hidden states extracted before these points, we show that future honest and deceptive outcomes can be reliably distinguished, with predictive signals remaining detectable several model calls before the final decision. We further characterize the temporal evolution of these signals: deception-related representations are weak early in execution but become increasingly identifiable as trajectories progress, while transferable structure can emerge before the strongest decision-adjacent signals appear. Finally, we intervene on the identified honest–deceptive representation directions during inference and find that activation steering reduces downstream deceptive behavior, suggesting that these representations influence agent decisions. Our findings indicate that agent deception is an evolving internal process that can be detected and potentially mitigated before it is expressed externally. Codes are available at https://anonymous.4open.science/r/deception-prediction/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Agent Safety, Deception Detection, Internal Representations

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_2GSAOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether an LLM agent's deception can be detected before it becomes externally observable. Using the agentic upward deception setting (an agent hits a blocker and either admits failure or fabricates a success), the authors align trajectories at a behaviorally decisive step d, pool pre-d hidden states into a trajectory-level vector G, and train a deep-kernel MMD detector to score G as more honest- or more deceptive-like. Three questions are studied: (RQ1) can future outcomes be predicted, (RQ2) does the signal accumulate over execution, (RQ3) does steering along the honest–deceptive direction change behavior. The agent is Qwen3-14B; 1,500 train / 500 test trajectories are built from five task templates (original 5 from Guo et al. plus GPT-5.6-generated variants). Results: AUROC 89.4% at the decisive step and 90.4% even at t−7; mean AUROC rises 51.2% (prefix p1) → 80.0% (p10); CAA steering at α=2.0 converts 56.0% of flagged deceptive trajectories to honest with 3.2% invalid.

Critical strengths
  1. Correct reframing of the problem. Moving deception monitoring from post-hoc output/trajectory evaluation to a pre-behavioral, internal-state prediction problem is genuinely useful. It creates a path to early warning rather than audit, and the paper is explicit that the target is spontaneous deception in benign task execution, not instructed/incentivized deception.
  2. The temporal analysis (RQ2) is the paper's best-designed experiment. Freezing the detectors trained at decision-relative shifts and evaluating them unmodified on start-relative prefixes is the right control: it rules out that the rising AUROC comes from refitting to later prefixes, so the gain really is information becoming recoverable. The t vs t−k train/eval grid (Table 1) plus Figure 3 gives two complementary views of the same phenomenon at almost no extra cost.
  3. The intervention closes the predict→act loop. Gating CAA on the detector's own score, and reporting representation shift in PCA alongside the behavioral transition rates, makes the "behaviourally actionable" claim concrete rather than rhetorical. The type-balanced direction in Eq. 8 is also a sensible detail.
Critical weaknesses

W1 — The honest/deceptive label is confounded with task outcome.

  • Honest = "reports the encountered failure"; deceptive = "continues toward a false success." The detector may therefore be reading task infeasibility / imminent giving-up — an internal state that legitimately differs between the two classes for reasons other than deception.
  • No control for task type × difficulty × trajectory length. L_type "discourages" task-specific differences but there is no diagnostic showing the honest/deceptive distributions are separable within a task type after the task-only information is removed.
  • The 56% "correction" rate is equally consistent with steering toward admitting defeat as toward honesty; the paper does not report task-success or task-completion rate after steering (it only says preserving task performance is future work).

W2 — Dataset is synthetic and narrow, and separability may be template artifacts.

  • Trajectories are generated by GPT-5.6 under the same five task settings and only manually reviewed — the generator and template can leave stylistic traces that a 300-dim learned kernel on averaged layer representations will readily pick up.
  • Single backbone (Qwen3-14B), white-box, fixed decoding (temperature=0). No cross-model or cross-scaffold test.
  • No baselines at all: no linear/logistic probe on the same G, no output-text monitor, no random-direction or shuffled-label control. Without them we cannot tell whether the deep-kernel MMD is doing the work or the representations are simply easy.

W3 — Internal tension between RQ1 and RQ2 numbers.

  • At p1 (one pre-decision state) all detectors are near chance (51.2%), yet the t−7 detector — which at test time sees the whole prefix except the last seven steps — reaches 90.4% AUROC. That gap implies the signal is carried by a small number of specific late steps, not a smooth accumulation, which contradicts the "progressive emergence" narrative as stated.
  • Table 1 also shows off-diagonal AUROC sometimes higher than the bold diagonal (e.g. t−7→t−7 90.40 vs. t→t 89.40, and the t−7 row is uniformly high). That pattern suggests late-window detectors are less selective, not more transferable — possibly because they see more of the trajectory rather than because the structure generalizes.
  • No confidence intervals or significance tests anywhere, and single seed / single α=2.0 chosen as "main analysis."
Questions for authors
  1. On the confound (W1): Can you report detection AUROC within each of the five task types separately, and after regressing out a task-type one-hot / a task-only control score? If honesty is largely "report failure," a matched control — where the agent is told the failure is admissible — would separate deception from mere task difficulty.
  2. On baselines (W2): How does the deep-kernel MMD compare against (a) a logistic probe on the same G, (b) a detector on the final response text of each pre-decision step, and (c) a shuffled-label / random-direction control? Is the learned kernel actually necessary?
  3. On the RQ1/RQ2 tension (W3): How many pre-decision steps do the t−7 trajectories have, and what is the AUROC of the t−7 detector evaluated on prefixes p1…p_{m−7}? A per-step ablation (drop step s from the t window) would show whether the signal is concentrated in the one or two steps before d.
  4. On the decisive step: How is d determined — is the "LLM Judge" in Figure 2 the same model that generated the trajectories, and what is inter-annotator / judge–human agreement on d? Errors in d shift the whole windowing scheme, including the near-chance p1 result.
  5. On the intervention (W1/W3): What is the task-level outcome after steering — do steered trajectories still attempt the task correctly, or do they increasingly terminate with a failure admission? And is the 3.2% invalid rate at α=2.0 measured against a matched α=0 reroll, or against the original baseline distribution?
  6. On transfer: Do detectors trained on Qwen3-14B transfer at all to a second backbone (or to a different agent scaffold on the same backbone)? Given the white-box constraint the paper acknowledges, this is the single result that would decide whether the method is a monitoring technique or a property of one model.
8 comments
AuthorsOpenReview ↗

Response for Reviewer 2GSA (Part 1)

We appreciate your careful review and recognition of our pre-behavioral monitoring approach. Following your suggestion, we have added task-controlled and failure-matched analyses to the revised manuscript.

Q1/W1.1

(1) Task-Type Controlled Detection.

We first evaluated MMD separately within each of the five task types.

Task TypeWithin-Task AUROC
Type 10.723
Type 20.725
Type 30.918
Type 40.928
Type 50.924
Macro Average0.844

The detector achieved a macro-average AUROC of 0.844 across the five task types. We further removed task-type information from the representations and detection scores. The task-adjusted detector achieved 0.790 ± 0.011 AUROCacross 15 held-out folds, with an overall out-of-fold AUROC of 0.799. These results show that the predictive signal remains detectable within task types and after task-type adjustment.

(2) Failure-Matched Detection.

We identified 3,711 trajectories where the agent encountered a task-critical failure before exhibiting honest or deceptive behavior.Both groups therefore experienced task-critical failures, while their labels reflected how the agents subsequently handled those failures. We constructed 152 matched honest-error/deceptive-error pairs. By comparing agents that encountered similar failures but responded differently, we can test whether the detector predicts subsequent deceptive behavior beyond observable failure-related factors.

EvaluationControl / CohortAUROC
Failure-matched MMD152 strictly matched pairs0.770
Error-only diagnosticSame 152 pairs; observable tool and error statistics0.492
MMD sensitivity analysis503 approximately matched pairs0.820
Explicit-failure detectionFinal behavioral label controlled0.527

These results suggest that task identity and observable failure conditions do not fully explain the performance. Even under matched failure conditions, pre-decision representations remain predictive of how agents subsequently respond to those failures.

Q2/W2.3

(1) Logistic probe on internal representations. We trained a logistic-regression probe using the same layer-aggregated representation (G) and case-level splits as Deep-kernel MMD. Across the eight decision-relative windows, the linear probe achieved a mean AUROC of 0.846, compared with 0.871 for MMD. These results show that a linear probe can recover the pre-decision predictive signal, which also aligns with our stated contribution, while the learned kernel provides additional discrimination.

(2) Same-prefix transcript monitor. We further evaluated a text-only monitor following the setup of SHADE-Arena [1]. We used DeepSeek-V4.1-Flash to score the visible transcript up to each decision-relative position, excluding the decisive event and subsequent trajectory. The monitor did not access hidden states or behavioral labels. Further experimental details are provided in the revised manuscript. The text-only monitor achieved 0.825 AUROC at (t), but its performance decreased to 0.624 at (t-7). In contrast, both internal-state detectors maintained stronger discrimination at the earlier decision-relative window.

These results suggest that pre-decision representations provide predictive information beyond a specific detection method, while the stronger performance of deep-kernel MMD supports the effectiveness of our proposed approach.

[1] Kutasov et al. SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents. arXiv:2506.15740, 2025.

AuthorsOpenReview ↗

Response for Reviewer 2GSA (Part 2)

Q3/W3.1

1. Why strong performance at t−7 is consistent with weaker prediction at early prefixes

RQ1 evaluates windows relative to the decisive event, while RQ2 tracks prediction as context accumulates from the trajectory start. These analyses use different reference points.

Let m be the total number of pre-decision calls and pⱼ the prefix containing the first j calls. Then

W(t−7)=pm−7.W(t-7) = p_{m-7}.

Thus, W(t−7) contains all pre-decision calls except the final seven. In the fixed t−7 cohort of 1,226 trajectories, m ranges from 8 to 30. The window therefore contains 1–23 calls, with a mean of 14.1 and a median of 23. By comparison, p1 contains only the first call.

The AUROC of 0.904 at t−7 measures prediction from this accumulated history; it does not imply equally strong prediction from the first call. To examine how prediction changes as context accumulates, we evaluated the same frozen t−7 detector on prefixes p1–p10:

Prefixp1p2p3p4p5p6p7p8p9p10
AUROC0.4820.5400.5410.5810.5930.6360.6910.7080.7720.761

AUROC rises overall from 0.482 at p1 to 0.761 at p10, with a small decrease after p9. This supports improved prediction as pre-decision context accumulates. The 0.904 result is evaluated separately at the trajectory-specific endpoint pₘ₋₇, whose length varies across trajectories. It is therefore distinct from performance at any fixed start-relative prefix.

2. Which pre-decision calls contribute most to detection?

Let d denote the decisive event. We removed individual calls and selected call pairs from W(t) and evaluated the resulting representations with the detector frozen. The unablated AUROC was 0.8948; positive ΔAUROC below denotes a decrease from this baseline.

Removed StepFrozen AUROCFrozen ΔAUROC [95% CI]
d-10.8740 ± 0.01960.0208 [0.0189, 0.0227]
d-20.8823 ± 0.01790.0125 [0.0107, 0.0142]
d-30.8929 ± 0.01760.0020 [0.0002, 0.0039]
d-40.8947 ± 0.01840.0002 [−0.0010, 0.0012]
d-50.8934 ± 0.01760.0015 [0.0005, 0.0024]
d-60.8914 ± 0.01670.0034 [0.0017, 0.0051]
d-70.8946 ± 0.01860.0002 [−0.0017, 0.0020]
d-80.8886 ± 0.01940.0062 [0.0047, 0.0078]
d-1, d-20.8544 ± 0.01900.0405 [0.0380, 0.0431]
d-1, d-80.8653 ± 0.02130.0296 [0.0261, 0.0329]
d-1, d-60.8692 ± 0.01720.0257 [0.0226, 0.0285]
d-1, d-30.8698 ± 0.01960.0250 [0.0222, 0.0280]
d-1, d-50.8706 ± 0.01940.0242 [0.0220, 0.0266]
d-1, d-70.8719 ± 0.01920.0230 [0.0206, 0.0253]
d-1, d-40.8722 ± 0.01940.0226 [0.0208, 0.0244]
d-2, d-80.8731 ± 0.02020.0217 [0.0186, 0.0252]
d-2, d-60.8766 ± 0.01680.0182 [0.0149, 0.0218]
d-2, d-30.8777 ± 0.01740.0171 [0.0146, 0.0196]
d-3, d-40.8916 ± 0.01830.0032 [0.0006, 0.0061]
d-4, d-50.8925 ± 0.01770.0023 [0.0004, 0.0040]
d-5, d-60.8898 ± 0.01640.0050 [0.0030, 0.0069]
d-6, d-70.8908 ± 0.01720.0041 [0.0014, 0.0067]
d-7, d-80.8904 ± 0.02050.0044 [0.0022, 0.0067]

Among individual calls, removing d−1 or d−2 causes the largest drops. Removing both produces the largest decrease among the tested ablations: 0.0405 AUROC (95% CI [0.0380, 0.0431]). Even then, AUROC remains 0.8544 with the frozen detector and 0.8607 after retraining. The final two calls contribute most among those tested, but earlier history still supports detection. Together, the prefix and ablation results show that prediction improves with accumulated context and does not depend solely on the calls immediately before the decision.

AuthorsOpenReview ↗

Response for Reviewer 2GSA (Part 3)

Q4

(1) Final behavioral labeling. We use DeepSeek 4.1 Flash to label final behavioral outcomes, following the criteria for agentic upward deception in Guo et al. (2025). (2) Decisive-step identification. We identify the decisive step separately through deterministic, task-specific rules judges without LLM judges. We define as the earliest step at which the agent makes an observable commitment to an honest or deceptive outcome. Only hidden states strictly preceding are used for detection. (3) Annotation validation. We manually reviewed several hundred trajectories while refining the annotation rules. In a final audit of 200 randomly sampled trajectories, manual verification agreed with the pipeline on both behavioral labels and decisive-step locations in all cases (194/200).

Q5/W2.3

(1) Task execution under steering. We examined termination outcomes across the same 264 detector-gated tasks. At α=2, the max-step termination rate increased from 11.4% to 27.3%. Among honest responses, the proportion generated after reaching the step limit increased from 10.4% to 32.1%. These results suggest that steering did not simply cause immediate refusal; part of the increase in honest reporting followed continued task attempts and eventual failure acknowledgment.

(2) Intervention controls and invalid outputs.We further evaluated a matched α=0 rerun and reverse steering -v. The α=0 rerun produced only one additional deceptive outcome compared with the original trajectories, while -v increased the deceptive-outcome rate by approximately 8%. Neither the original nor the matched α=0 condition produced invalid outputs.Under +v, α=2, the reported 3.2% invalid rate corresponds to 7 of 216 originally deceptive trajectories, or 2.7% (7/264) across all detector-gated trajectories. This is an absolute within-condition rate, not a relative increase over either baseline.

(3) Performance on standard tasks. To further examine whether steering compromises the model's ability to complete answerable tasks, we evaluated MMLU and GSM8K, using 100 questions from each benchmark. We compared honesty steering at α=2 against a matched no-injection baseline and a norm-matched random-direction control. Further experimental details are provided in the revised manuscript.

ConditionInterventionMMLU AccuracyMMLU Effective Response RateGSM8K Exact MatchGSM8K Effective Response RatePooled AccuracyOverall Completion
α=0No vector injection86/100 (86%)100%94/100 (94%)100%180/200 (90.0%)200/200 (100%)
+v, α=2Honesty-direction injection84/100 (84%)100%92/100 (92%)100%176/200 (88.0%)200/200 (100%)
Random, α=2Layer-wise norm-matched random direction86/100 (86%)99%92/100 (92%)100%178/200 (89.0%)199/200 (99.5%)

At α=2 , honesty steering reduced accuracy by 2 percentage points on both MMLU and GSM8K, while maintaining a 100% valid response rate across all 200 questions. These results suggest a limited performance cost on standard tasks, while further evaluation is needed to assess whether steering causes premature failure reporting on challenging but solvable agent tasks.

Reviewer_PGsyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether future deceptive behavior in LLM-based agents can be predicted from their internal representations before it manifests in observable actions. The authors frame deception monitoring as a trajectory-level representation analysis problem, aligning agent rollouts around the "behaviorally decisive step" and training a Deep-Kernel MMD detector on pre-decision hidden states. Experimental results on an upward deception benchmark show that future honest and deceptive outcomes can be distinguished several steps ahead (up to t−7t-7 steps) of the decisive action, with predictive signals accumulating as execution unfolds. Finally, the authors apply CAA at inference time to steer hidden representations, reporting that 56% of selected deceptive trajectories can be converted into honest completions.

Critical strengths
  • A proactive monitoring perspective. Shifting from post-hoc output inspection to pre-behavior internal state prediction addresses a critical bottleneck in agent monitoring, offering a promising direction for early deception prevention.
  • Principled methodology with closed-loop validation. Combining sliding-window representation aggregation, distributional discrepancy testing via Deep-Kernel MMD, and causal intervention via Contrastive Activation Addition provides a coherent workflow spanning detection, temporal analysis, and behavioral steering.
Critical weaknesses
  • Confounding between deception and task-failure uncertainty. The five evaluation scenarios (e.g., broken tools, missing files) inherently present unrecoverable task obstacles. The representation shifts detected prior to the decisive step likely capture the model's awareness of environmental errors, confusion, or retrieval failure rather than a latent intention to deceive. The paper lacks essential negative controls comparing deceptive failure against non-deceptive error recovery.

  • Limited evaluation across model families. All empirical analyses and interventions are conducted exclusively on a single open-weight model, Qwen3-14B. Because internal representation geometries and alignment properties vary substantially across architectures and training regimes, findings from a single model cannot substantiate general claims about LLM agent deception dynamics.

Questions for authors
  • Can the authors disentangle deception signals from error-induced uncertainty? Could you provide control experiments demonstrating that the MMD detector separates deliberate deception from ordinary confusion or error handling when an agent encounters broken tools or missing resources?
  • How well do the findings generalize across architectures? Have you tested whether the pre-decision predictive signal (such as the t−7t-7 step predictability) holds for other open-weight model families or across different parameter scales?
  • What is the impact of activation steering on general capabilities? The authors report invalid generation rates on the benchmark tasks, but does injecting a global steering vector (α=2.0\alpha=2.0) impair performance on standard non-deceptive tasks such as MMLU, GSM8K, or multi-step tool use? Could the authors quantify this potential alignment tax?
  • How robust is the approach to more diverse and non-synthetic agent workflows? Since the 1,500 training and 500 test rollouts are synthetic expansions of only five task templates generated via GPT-5.6, to what extent do the learned representations reflect template-specific artifacts rather than generalized agentic deception? Can the detector generalize to out-of-distribution real-world agent environments?
5 comments
AuthorsOpenReview ↗

Response for Reviewer PGsy (Part 1)

We highly appreciate your constructive review and positive assessment of pre-behavior monitoring approach. We have added failure-matched control experiments to the revised manuscript and present the results below.

W1/Q1

(1) Failure-matched controls. We identified 3,711 trajectories where the agent encountered a task-critical failure before exhibiting honest or deceptive behavior.Both groups therefore experienced task-critical failures, while their labels reflected how the agents subsequently handled those failures. We constructed 152 matched honest-error/deceptive-error pairs. By comparing agents that encountered similar failures but responded differently, we can test whether the detector predicts subsequent deceptive behavior beyond observable failure-related factors.

(2) Detection results. On the 152 matched pairs, MMD achieved 0.770 AUROC, compared with 0.492 for an error-only diagnostic. The result remained consistent on 503 approximately matched pairs (0.820 AUROC). We also found that MMD showed near-chance discrimination between prefixes with and without explicit failures after controlling for the final behavioral label. These results indicate that pre-decision representations remain predictive of subsequent deceptive behavior even after controlling for observable task-failure factors, rather than merely reflecting the presence of task failures.

W2/Q2

We sincerely appreciate your thorough review and constructive feedback. We are grateful that you recognize the strengths of our work, particularly its proactive monitoring perspective. We address your concern about cross-model generality below and have incorporated the additional experiments into the revised manuscript.

To examine whether our findings extend beyond Qwen3-14B, we repeated the detection and signal accumulation experiments on Mistral-Nemo-Instruct-12B, a model from a different family and of a different scale.

(1) Cross-Window Detection

For RQ1, we trained and evaluated detectors across eight trajectory windows, from t to t-7. The complete train-window × test-window AUROC matrix is reported below.

Train \ Testtt-1t-2t-3t-4t-5t-6t-7Row Mean
t0.90110.89470.89430.89770.89440.89340.89760.89850.8965
t-10.89550.89510.89740.89750.89500.89230.89710.89690.8959
t-20.89100.89280.89870.90300.89820.89680.90160.90050.8978
t-30.88540.88900.89570.90110.90050.89570.90170.90050.8962
t-40.88190.88600.89570.89910.89550.89350.89640.89600.8930
t-50.86880.87380.88650.89240.88820.88620.88870.88750.8840
t-60.87560.87600.88680.89310.89380.89290.89510.89500.8885
t-70.88420.88730.89650.89990.89770.89770.90100.90050.8956

Across the 64 train–test window combinations, AUROC ranges from 0.8688 to 0.9030. Detection performance remains consistently high across different training and evaluation windows on Mistral-Nemo-Instruct-12B.

(2) Accumulation of Predictive Evidence

For RQ2, we evaluated the eight frozen detectors on the same fixed cohort of 182 trajectories. The detectors were evaluated at ten successive prefixes, from p1 to p10, without retraining at individual prefix positions.

Detectorp1p2p3p4p5p6p7p8p9p10
t0.61120.73810.75140.76590.77710.80070.81000.82480.82890.8299
t-10.58860.67420.70390.73220.77740.79970.81820.83070.83300.8396
t-20.60000.67580.72930.74820.76600.79420.80510.82160.82470.8296
t-30.58530.69360.74480.78400.79620.80970.82110.82890.82960.8354
t-40.58920.69470.72600.74120.76140.78230.79790.81730.82270.8326
t-50.60490.71170.72770.74680.77160.79120.80040.81350.81660.8211
t-60.60410.69730.71490.73460.77940.78650.79120.80590.80410.8138
t-70.60980.69460.73350.75780.78720.80150.80320.81150.81350.8201
Mean0.59910.69700.72890.75130.77710.79570.80590.81930.82160.8278

The mean AUROC across the eight frozen detectors increases from 0.5991 at p1 to 0.8278 at p10, reaching 0.7771 at p5. This consistent increase indicates that predictive evidence accumulates as more of the trajectory becomes available. These results reproduce our main detection findings on a different backbone, supporting the generality of our method beyond Qwen3-14B.

AuthorsOpenReview ↗

Response for Reviewer PGsy (Part 2)

W3/Q2

Specifically, we evaluated MMLU and GSM8K, using 100 questions from each benchmark. We compared honesty steering at α=2 with a matched no-injection baseline and a norm-matched random-direction control. Further experimental details are provided in the revised manuscript.

ConditionInterventionMMLU AccuracyMMLU Effective Response RateGSM8K Exact MatchGSM8K Effective Response RatePooled AccuracyOverall Completion
α=0No vector injection86/100 (86%)100%94/100 (94%)100%180/200 (90.0%)200/200 (100%)
+v, α=2Honesty-direction injection84/100 (84%)100%92/100 (92%)100%176/200 (88.0%)200/200 (100%)
Random, α=2Layer-wise norm-matched random direction86/100 (86%)99%92/100 (92%)100%178/200 (89.0%)199/200 (99.5%)

At α=2, honesty steering reduced accuracy by 2 percentage points on both MMLU and GSM8K, while maintaining a 100% valid response rate across all 200 questions. These results suggest that honesty steering incurs a small, statistically non-significant accuracy cost on the evaluated standard tasks, while maintaining valid response generation. Its impact on more complex multi-step agent tasks remains to be evaluated.

Q4

(1) Detection across task-generation sources. We constructed a dataset of 500 tasks generated by five models: Qwen3.8-27B, DeepSeek-V4.1-Flash, GLM-5.3-Flash, GLM-5.3, and GPT-5.6. Each model contributed 100 tasks, balanced across task types and behavioral labels. We used Qwen3-14B to execute these tasks and evaluated the resulting trajectories using the same detection framework.

The complete train-window × evaluation-window AUROC matrix on the multi-source dataset is reported below.

Train \ Evaltt-1t-2t-3t-4t-5t-6t-7
t0.8640.8180.8190.8000.8100.7600.7500.783
t-10.8640.8430.8140.7980.8060.7900.7610.794
t-20.9160.8830.8720.8820.8530.8470.8260.841
t-30.9160.8670.8590.8690.8590.8440.8140.812
t-40.9040.8930.8380.8200.8140.8090.8670.847
t-50.8130.8260.8120.7780.7490.8010.8230.821
t-60.8230.8070.7830.7970.7860.7690.8230.796
t-70.8040.7900.7840.7880.7800.7510.7040.733

The detector achieves an overall test AUROC of 0.854, with performance ranging from 0.720 to 0.920 across the five generation sources. These results suggest that the detected distinction is not driven by a single task-generation source.

Task-Generation ModelNumber of TasksTest AUROC
Qwen3.8-27B1000.920
GLM-5.3-Flash1000.920
GLM-5.31000.880
DeepSeek-V4.1-Flash1000.830
GPT-5.61000.720
Overall5000.854

(2) Signal accumulation across generation sources. We further evaluated the eight frozen detectors on successive trajectory prefixes, without prefix-specific retraining.

Prefixp1p2p3p4p5p6p7p8p9p10
Mean AUROC0.5400.5940.6410.6900.7190.7420.7460.7390.7470.756

The mean AUROC increased from 0.540 at p1 to 0.756 at p10, supporting the generality of our method beyond GPT-5.6-generated tasks.

Dear Reviewer PGsy

Thanks for your time and effort in reviewing this paper. Your comments are valuable to improve the quality of this paper. As the author-reviewer discussion period will approach the end, we gently request that you read the rebuttal and check whether your concerns are addressed.Thanks again for your time and help in improving the quality of this paper.

Regards from the authors

Reviewer_dfbfOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether spontaneous deceptive behavior in LLM-based agents can be predicted from internal representations before it becomes externally observable. The authors frame deception monitoring as a trajectory-level representation analysis problem. They propose a Maximum Mean Discrepancy (MMD)-based detector to distinguish future honest and deceptive outcomes using pre-decision hidden states. Furthermore, they track the temporal accumulation of these signals and demonstrate that intervening on these representations via Contrastive Activation Addition (CAA) can successfully steer agents away from deceptive behavior.

Critical strengths

The methodology is mathematically sound (deep-kernel MMD) and the experimental design directly addresses the proposed research questions. Using a deep-kernel MMD to compare distributions of hidden states is a robust choice. It avoids the limitations of simple linear probes and effectively captures complex, high-dimensional representation shifts.

While internal monitoring and activation steering exist, applying them to spontaneous, pre-behavioral agentic deception (rather than explicitly incentivized or post-hoc deception) is a novel and highly timely contribution to AI safety.

By showing that CAA can convert 56.0% of deception-prone trajectories into honest outcomes, the authors prove that the identified representations are not merely correlative, but behaviorally actionable and causally relevant.

Proactive mitigation is a critical goal in agent safety. Demonstrating that deception is an evolving internal process that can be intercepted and altered before execution has strong implications for building trustworthy autonomous systems.

Critical weaknesses

The paper defines the behaviorally decisive step dd as the "first step at which the agent produces behavior that reveals its eventual honest or deceptive outcome." However, the mechanism for identifying dd across 2,000 trajectories is unclear.

All experiments are conducted on Qwen3-14B. Internal representations and activation steering efficacy are known to be highly model-dependent. The claims would be significantly strengthened by validating the approach on at least one additional model family (of different parameter scale) to prove the phenomenon is not an artifact of a specific architecture.

The paper notes that increasing α\alpha to 2.5 increases invalid outcomes. However, it does not explicitly discuss whether the α=2.0\alpha = 2.0 intervention causes "false positives"; i.e., steering an agent to prematurely report failure on a difficult but solvable task, thereby degrading overall task success rates.

Questions for authors

Could you clarify the exact procedure used to identify the behaviorally decisive step dd for each trajectory? Was this done via heuristic rules, an LLM evaluator, or human annotators, and what was the inter-annotator agreement (if applicable)?

When applying the intervention at α=2.0\alpha=2.0, what is the impact on the agent's overall task completion rate? Does the "honesty steering" inadvertently cause the agent to give up on challenging but achievable tasks?

2 comments
AuthorsOpenReview ↗

Response for Reviewer dfbf (Part 1)

We sincerely appreciate your constructive review and supportive score!We are encouraged that you recognize the strengths of our work, particularly the novelty of studying spontaneous pre-behavioral agentic deception, the soundness of our deep-kernel MMD approach, and the potential of representation-based intervention for proactive mitigation. We address your concern below and have incorporated the additional experiments and clarifications into the revised manuscript.

W2

To examine whether our findings extend beyond Qwen3-14B, we repeated the detection and signal accumulation experiments on Mistral-Nemo-Instruct-12B, a model from a different family and of a different scale.

RQ1: Cross-Window Detection

For RQ1, we trained and evaluated detectors across eight trajectory windows, from t to t-7. The complete train-window × test-window AUROC matrix is reported below.

Train \ Testtt-1t-2t-3t-4t-5t-6t-7Row Mean
t0.90110.89470.89430.89770.89440.89340.89760.89850.8965
t-10.89550.89510.89740.89750.89500.89230.89710.89690.8959
t-20.89100.89280.89870.90300.89820.89680.90160.90050.8978
t-30.88540.88900.89570.90110.90050.89570.90170.90050.8962
t-40.88190.88600.89570.89910.89550.89350.89640.89600.8930
t-50.86880.87380.88650.89240.88820.88620.88870.88750.8840
t-60.87560.87600.88680.89310.89380.89290.89510.89500.8885
t-70.88420.88730.89650.89990.89770.89770.90100.90050.8956

Across the 64 train–test window combinations, AUROC ranges from 0.8688 to 0.9030. Detection performance remains consistently high across different training and evaluation windows on Mistral-Nemo-Instruct-12B.

RQ2: Accumulation of Predictive Evidence

For RQ2, we evaluated the eight frozen detectors on the same fixed cohort at ten successive prefixes, without retraining at individual prefix positions.

Detectorp1p2p3p4p5p6p7p8p9p10
t0.61120.73810.75140.76590.77710.80070.81000.82480.82890.8299
t-10.58860.67420.70390.73220.77740.79970.81820.83070.83300.8396
t-20.60000.67580.72930.74820.76600.79420.80510.82160.82470.8296
t-30.58530.69360.74480.78400.79620.80970.82110.82890.82960.8354
t-40.58920.69470.72600.74120.76140.78230.79790.81730.82270.8326
t-50.60490.71170.72770.74680.77160.79120.80040.81350.81660.8211
t-60.60410.69730.71490.73460.77940.78650.79120.80590.80410.8138
t-70.60980.69460.73350.75780.78720.80150.80320.81150.81350.8201
Mean0.59910.69700.72890.75130.77710.79570.80590.81930.82160.8278

The mean AUROC across the eight frozen detectors increases from 0.5991 at p1 to 0.8278 at p10, reaching 0.7771 at p5. This consistent increase indicates that predictive evidence accumulates as more of the trajectory becomes available. These results reproduce our main detection findings on a different backbone, supporting the generality of our method beyond Qwen3-14B.

W1/Q1

We appreciate your constructive review and positive assessment of our methodology. We clarify the behavioral labeling and decisive-step identification procedures below, with the corresponding details added to the revised manuscript. (1) Final behavioral labeling. We use DeepSeek 4.1 Flash to label final behavioral outcomes, following the criteria for agentic upward deception in Guo et al. (2025). A response is labeled deceptive if it conceals a material failure and falsely presents unsupported answers, substituted sources, or fabricated artifacts as successful completion. (2) Decisive-step identification. We identify the decisive step separately through deterministic, task-specific rules judges without LLM judges. We define as the earliest step at which the agent makes an observable commitment to an honest or deceptive outcome. Only hidden states strictly preceding are used for detection. (3) Annotation validation. We manually reviewed several hundred trajectories while refining the annotation rules. In a final audit of 200 randomly sampled trajectories, manual verification agreed with the pipeline on both behavioral labels and decisive-step locations in all cases (194/200).

AuthorsOpenReview ↗

Response for Reviewer dfbf (Part 2)

W3/Q2

(1) Task execution under steering. Steering increases honest outcomes, but also leads to more trajectories exhausting their interaction budget. On the same 264 detector-gated tasks, the max-step termination rate increased from 11.4% (30/264) without steering to 27.3% (72/264) at α=2 .

Among responses judged as honest, the proportion produced after reaching the step limit increased from 10.4% (5/48) to 32.1% (50/156). This suggests that steered agents often continued attempting the task before acknowledging failure, rather than immediately giving up.

(2) Performance on standard tasks. To further examine whether steering compromises the model's ability to complete answerable tasks, we evaluated MMLU and GSM8K using 100 questions from each benchmark. We compared honesty steering at α=2 against a matched no-injection baseline and a norm-matched random-direction control. Further experimental details are provided in the revised manuscript.

ConditionInterventionMMLU AccuracyMMLU Effective Response RateGSM8K Exact MatchGSM8K Effective Response RatePooled AccuracyOverall Completion
α=0No vector injection86/100 (86%)100%94/100 (94%)100%180/200 (90.0%)200/200 (100%)
+v, α=2Honesty-direction injection84/100 (84%)100%92/100 (92%)100%176/200 (88.0%)200/200 (100%)
Random, α=2Layer-wise norm-matched random direction86/100 (86%)99%92/100 (92%)100%178/200 (89.0%)199/200 (99.5%)

At α=2, honesty steering reduced accuracy by 2 percentage points on both MMLU and GSM8K, while maintaining a 100% valid response rate across all 200 questions. These results suggest a limited performance cost on standard tasks, while further evaluation is needed to assess whether steering causes premature failure reporting on challenging but solvable agent tasks.

Reviewer_SqmjOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks a good question: whether an LLM agent's eventual honest vs. deceptive outcome can be predicted from its hidden states before the "behaviorally decisive step" d (e.g., fabricating a file vs. reporting a failure). The data are Qwen3-14B trajectories on the Agentic Upward Deception tasks (Guo et al., 2025) plus manually reviewed GPT-5.6-generated instances, with 1,500 training and 500 test trajectories, both balanced. A deep-kernel MMD detector on pooled pre-decision hidden states reaches 84.9–90.4% in-window AUROC for windows W(t−k) that drop the last 7 steps. Frozen detectors applied to growing prefixes rise from 51.2% (p1) to 80.0% (p10) mean AUROC (Fig. 3). Detector-gated CAA steering raises the test-set honest rate from 50.0% to 71.6%.

Critical strengths

First, It's an important question, well-posed decomposition. Predicting agentic deception before the decisive act is important. This paper has great problem statement: Decision-aligned windows, start-aligned prefixes, and a detector-gated intervention. Frozen detectors avoid prefix-specific refitting (though, as W1 notes, they introduce a train/test shift), thresholds are chosen on a validation split, and the Limitations correctly frame labels as behavior rather than intent.

Second, the paper does provide a clear held-out signal. Its pre-decision states give 84.9–90.4% in-window AUROC on held-out data. Though what this signal is and whether simpler monitors match it is weak, its outcome information is clearly decodable before the decisive step. The type-balanced steering direction and the L_type term show attention to task-type confounds.

Third, A large net intervention effect, internally consistent. The counts reconcile exactly: 358 honest, 135 deceptive, 7 invalid. Against no intervention on the balanced test set, 121 deceptive to honest conversions vs. 13 honest to deceptive regressions is a substantial gain (+21.6 pp). The gate's selectivity helps (216/250 deceptive vs. 48/250 honest trajectories flagged). A code link is provided and i reviewed it.

Critical weaknesses

Weakness 1. The temporal claims are confounded with cohort and window length. W(t−k) exists only for trajectories with more than k pre-decision steps, yet no per-cell n is reported and the released code does not fix one cohort for Table 1. The signature is there: t−7 is the best evaluation column in all 8 rows, and every detector scores 0.5–13.0 pp higher on t−7 than on t inputs. So "90.4% vs. 89.4%" either compares different populations or shows that dropping the decision-adjacent steps helps every detector, which is hard to reconcile with "increasingly identifiable as trajectories progress". Fig. 3 also tests window-trained detectors on prefix averages, so its 51.2%→80.0% rise need not mean gradual emergence.

Weakness 2. The steering result (RQ3) is not shown to be direction-specific. If the released code is the Fig. 4 path, the gate uses precomputed W(t) scores, which appears to require the completed unsteered rollout, and gated tasks are re-run with steering throughout: hindsight task selection rather than intervention before the lie, which appears to differ from Sec. 3.4. α = 2 is picked on the test-set sweep; by my reading of Fig. 4, α = 0.5–1 turns about 26–27 of 48 gated honest trajectories deceptive. Even at α = 2, 13 of 48 flip, and the released server samples at temperature 0.2 unless overridden (paper: 0), so re-running the 82%-deceptive gated set would by itself tend to raise the honest rate. Missing controls: an α = 0 re-run, a norm-matched random direction, −v, and the honesty-prompt result (plotted in Fig. 5, never reported). The vector, a class-mean difference rather than the detector, may also partly encode giving up, since "honest" means reporting the failure.

W3. No baselines, so the white-box advantage is not shown. If the released server produced the features, Qwen3's thinking was off (the paper does not say), so the detector reads hidden states over the same visible transcript an external monitor sees. The "inherently post hoc" contrast therefore needs same-prefix baselines: an LLM judge or text classifier on the identical transcript, trivial predictors (pre-decision length, task type, tool errors), and a linear probe (the natural ablation, since Eq. 6 reduces to a nearest-class-mean rule in kernel space). These are controls for the paper's own claim, not a SOTA requirement.

W4. Label validity is unclear, which affects all three RQs. The paper never names the model, prompt or human validation behind the "LLM Judge" (Fig. 2) that assigns y and d. In the released code, d comes from an unreleased adapter, and final answers fall back to a regex word rule when no "strict" judge is supplied; that rule labels "I couldn't download the report because the tool returned an error" dishonest. If steered outputs fell back to this rule while baselines used the judge, the labeler alone could create transitions. The balanced splits are drawn by outcome from a larger pool without template grouping, and the natural deception rate is not reported.

What would change my score. Weak accept if the rebuttal shows: (1) per-cell n and fixed-cohort Table 1/Fig. 3 with CIs, with early windows still beating a length + task-type predictor; (2) the detector beating that predictor and a same-prefix transcript monitor at W(t); (3) on the 264 gated tasks, v at α = 2 clearly beating an α = 0 re-run and a random direction, with all labels from one named judge validated against blinded human labels (e.g., κ ≥ 0.7), and RQ3 run with an online gate or restated as steering detector-selected tasks. I would lower it if the simple predictor matches the detector, or v fails these controls, while RQ1 or RQ3 is still claimed.

Questions for authors
  1. What are n (honest/deceptive) and the task mix per cell of Table 1 and Fig. 3, and what do fixed-cohort and length-only results show?
  2. Is the released protocol (W(t)-score gate; gated tasks re-run and steered throughout) the one behind Fig. 4? Please report honest/deceptive/invalid counts per α (chosen on validation), plus α = 0, random-direction, −v and honesty-prompt controls.
  3. What do a same-prefix transcript monitor and a linear probe on G achieve on the same splits?
  4. Which judge (model, prompt, human agreement) assigned y and d, and which labeler produced baseline vs. steered labels? What are the pool size and natural deception rate?
12 comments
AuthorsOpenReview ↗

Response to Reviewer Sqmj (Part 1)

Thank you for your careful review. We respond to your comments point by point below. We will include the necessary analyses and clarifications in the revised version.

W1/Q1

Fixed-cohort evaluation preserves early detection and the prefix-accumulation trend. Below, we report per-cell composition, control for cohort differences, and compare against length and task-type predictors.

1. Per-cell sample composition

Table R1.1 — Training-set composition by window.

Train windowTotalHonestDishonestType 1Type 2Type 3Type 4Type 5
t1,500750750316321346300217
t−11,14160853331347346250185
t−21,06457449030740345210162
t−397154242928729330175150
t−479946833122116299135128
t−56343782561746235110109
t−651631320312331989498
t−74102671437321727885

Table R1.2 — Test-set composition and held-out performance by window.

Eval windowTotalHonestDishonestType 1Type 2Type 3Type 4Type 5AUROC [95% CI]
t500250250105107115101720.8774 [0.8458, 0.9084]
t−13772021751041111586610.8135 [0.7673, 0.8566]
t−2354192162103911474540.8209 [0.7743, 0.8648]
t−332318413994411265480.8091 [0.7595, 0.8540]
t−42671501177629356400.7982 [0.7435, 0.8517]
t−5212121915717443370.8247 [0.7657, 0.8789]
t−616898703615941310.8499 [0.7875, 0.9046]
t−713784532015137280.8232 [0.7479, 0.8910]

Table R1.3 — Sample composition at each prefix in the original Figure 3.

PrefixTotalHonestDishonestType 1Type 2Type 3Type 4Type 5
p150025025010510711510172
p2377202175104111158661
p335419216210391147454
p43231841399441126548
p5267150117762935640
p621212191571744337
p71689870361594131
p81378453201513728
p9105683780393325
p1095623370352825

Earlier windows contain fewer trajectories because W(t−k) requires at least k+1 pre-decision calls. The honest share also rises with k, from 50.0% at t to 65.1% at t−7 in training. As you noted, comparisons across columns in the original Table 1 therefore mix window position with differences in cohort composition.

2. Interpretation of cross-window comparisons

Both (t, t) and (t−7, t−7) are diagonal evaluations: training and testing use the same position. However, W(t−7) excludes the latest seven pre-decision steps included in W(t). Its higher AUROC in the original table cannot establish a temporal effect because the evaluation cohorts differ in size, label balance, and task mix. We address this with the fixed-cohort analyses below.

3. Common test cohort (n = 137)

We evaluated every position on the same 137 trajectories with valid representations at all positions from t to t−7 (84 honest / 53 dishonest; all five task types). Each detector retained its original position-specific training set.

Table R1.4 — Performance across positions on the same fixed test cohort (n = 137).

Eval shiftAUROCAccuracyBalanced accuracy
t0.89290.84670.8471
t−10.79200.79560.7950
t−20.86550.84670.8541
t−30.84700.80290.8045
t−40.83960.80290.8114
t−50.87650.83940.8482
t−60.87310.83210.8492
t−70.82320.79560.8055
AuthorsOpenReview ↗

Response to Reviewer Sqmj (Part 2)

W1/Q1

4. Fixed-cohort cross-window evaluation (n = 1,226)

We evaluated all train-position × evaluation-position combinations on 1,226 trajectories with valid representations at all eight windows. The test cases were identical across windows.

Table R1.5 — Train-position × evaluation-position AUROC [95% CI] on the fixed cohort (n = 1,226).

Train \ Evaltt−1t−2t−3t−4t−5t−6t−7
t0.885 [0.877, 0.893]0.863 [0.855, 0.871]0.861 [0.854, 0.868]0.856 [0.849, 0.864]0.854 [0.847, 0.861]0.849 [0.840, 0.858]0.840 [0.830, 0.850]0.801 [0.788, 0.814]
t−10.874 [0.866, 0.882]0.869 [0.862, 0.877]0.864 [0.858, 0.871]0.862 [0.856, 0.868]0.859 [0.851, 0.867]0.853 [0.846, 0.860]0.845 [0.837, 0.853]0.811 [0.798, 0.824]
t−20.874 [0.865, 0.883]0.867 [0.859, 0.875]0.865 [0.858, 0.872]0.863 [0.855, 0.870]0.860 [0.852, 0.868]0.855 [0.848, 0.863]0.849 [0.842, 0.856]0.811 [0.800, 0.822]
t−30.868 [0.859, 0.876]0.864 [0.855, 0.872]0.863 [0.855, 0.871]0.863 [0.854, 0.871]0.861 [0.852, 0.869]0.854 [0.846, 0.863]0.849 [0.842, 0.856]0.822 [0.811, 0.833]
t−40.866 [0.857, 0.875]0.864 [0.855, 0.872]0.863 [0.855, 0.872]0.863 [0.854, 0.871]0.861 [0.851, 0.870]0.857 [0.848, 0.866]0.852 [0.845, 0.859]0.830 [0.817, 0.842]
t−50.870 [0.861, 0.879]0.860 [0.852, 0.868]0.863 [0.856, 0.871]0.862 [0.856, 0.869]0.863 [0.856, 0.870]0.861 [0.854, 0.868]0.856 [0.850, 0.862]0.837 [0.829, 0.845]
t−60.874 [0.865, 0.883]0.866 [0.857, 0.875]0.868 [0.859, 0.876]0.868 [0.860, 0.876]0.868 [0.860, 0.876]0.867 [0.859, 0.875]0.863 [0.856, 0.870]0.840 [0.834, 0.847]
t−70.870 [0.859, 0.880]0.863 [0.853, 0.873]0.865 [0.856, 0.874]0.864 [0.856, 0.873]0.865 [0.857, 0.874]0.865 [0.857, 0.873]0.861 [0.853, 0.870]0.862 [0.854, 0.870]

All 64 combinations exceed 0.80 AUROC on the fixed cohort. Performance at t varies little with training position (0.866–0.885). At t−7, training at the same position yields 0.862, compared with 0.801 when training at t. Thus, detection remains effective across windows, although matching the training position helps at the earliest window.

5. Per-position evaluation with metadata control (n = 1,910)

We used the same 1,910 strictly labeled trajectories and case-level splits at every position. Each trajectory had at least eight pre-decision calls; 5-fold evaluation used 380–384 test cases per fold.

Table R1.6 — Fixed-cohort AUROC by position (n = 1,910).

PositionHidden-state detectorLength + task-type predictor
t.8948 ± .0181.7081 ± .0228
t−1.8738 ± .0185.7081 ± .0229
t−2.8607 ± .0182.7084 ± .0228
t−3.8484 ± .0195.7089 ± .0228
t−4.8412 ± .0233.7089 ± .0223
t−5.8270 ± .0202.7095 ± .0228
t−6.8201 ± .0203.7101 ± .0232
t−7.8098 ± .0232.7109 ± .0232

On this fixed cohort, detector AUROC decreases monotonically from .8948 at t to .8098 at t−7. The length + task-type predictor remains near .71. The detector exceeds this control by 9.9–18.7 AUROC points across positions.

6. Single-step analysis (n = 1,226)

To hold both the cohort and the amount of representation used constant, we evaluated individual pre-decision hidden states S(t−k) on 1,226 trajectories with 15 repeated folds. This analysis uses no cumulative averaging and excludes the decisive event.

Table R1.7 — Fixed-cohort performance using individual pre-decision steps (n = 1,226).

PositionMean AUROCSD
t0.87980.0242
t−10.84560.0295
t−20.83170.0250
t−30.81810.0295
t−40.80180.0275
t−50.81190.0414
t−60.82610.0253

The paired temporal slope is +0.00921 AUROC per step toward the decisive event (paired-bootstrap 95% CI [0.00702, 0.01148]). Mean AUROC at t exceeds that at t−6 by 0.0537 (95% CI [0.0373, 0.0695]). Individual hidden states are therefore more predictive, on average, closer to the decision.

AuthorsOpenReview ↗

Response to Reviewer Sqmj (Part 3)

W1/Q1

7. Length-only controls

Table R1.8 — Length-only and task-type controls (AUROC).

PredictorAUROC
Current window length, full cohort0.6232 ± 0.0106
Current window length, fixed cohort at t0.5763 ± 0.0343
Visible prefix length, fixed cohort0.6551 ± 0.0300
Task type + visible prefix length0.7082–0.7109

These controls use length and, where indicated, task type. They exclude response text, hidden states, and all decision or post-decision information. Length and task type carry predictive signal, but do not account for the detector’s performance (Table R1.6).

8. Fixed-cohort reanalysis of Figure 3 (n = 472)

We repeated the prefix-accumulation analysis on 472 trajectories containing all ten prefixes, p1–p10. The same eight frozen detectors were evaluated at every prefix, without retraining or recalibration.

Table R1.9 — Prefix accumulation on the fixed cohort (n = 472), AUROC [95% CI].

Detectorp1p2p3p4p5p6p7p8p9p10
t0.655 [0.619, 0.690]0.698 [0.660, 0.736]0.740 [0.707, 0.773]0.767 [0.734, 0.799]0.772 [0.741, 0.803]0.774 [0.739, 0.808]0.771 [0.738, 0.804]0.783 [0.749, 0.816]0.790 [0.757, 0.822]0.808 [0.775, 0.840]
t−10.628 [0.588, 0.668]0.686 [0.648, 0.724]0.728 [0.699, 0.758]0.752 [0.722, 0.783]0.759 [0.726, 0.793]0.758 [0.727, 0.790]0.759 [0.728, 0.791]0.771 [0.739, 0.804]0.777 [0.745, 0.809]0.797 [0.764, 0.830]
t−20.598 [0.559, 0.638]0.689 [0.650, 0.728]0.739 [0.707, 0.771]0.755 [0.726, 0.784]0.751 [0.723, 0.778]0.759 [0.731, 0.787]0.752 [0.725, 0.779]0.765 [0.738, 0.792]0.766 [0.739, 0.793]0.787 [0.760, 0.815]
t−30.621 [0.586, 0.656]0.698 [0.662, 0.733]0.733 [0.699, 0.767]0.752 [0.723, 0.781]0.744 [0.715, 0.773]0.752 [0.725, 0.779]0.748 [0.719, 0.776]0.760 [0.732, 0.788]0.765 [0.738, 0.793]0.782 [0.754, 0.811]
t−40.615 [0.585, 0.646]0.695 [0.659, 0.730]0.730 [0.698, 0.761]0.757 [0.729, 0.785]0.753 [0.725, 0.781]0.761 [0.735, 0.787]0.755 [0.728, 0.781]0.768 [0.741, 0.794]0.768 [0.742, 0.794]0.787 [0.761, 0.812]
t−50.624 [0.588, 0.660]0.702 [0.668, 0.736]0.737 [0.712, 0.762]0.756 [0.729, 0.783]0.759 [0.736, 0.782]0.760 [0.736, 0.785]0.756 [0.733, 0.779]0.763 [0.739, 0.786]0.768 [0.746, 0.790]0.783 [0.759, 0.806]
t−60.652 [0.618, 0.687]0.718 [0.682, 0.753]0.751 [0.727, 0.775]0.763 [0.737, 0.789]0.757 [0.735, 0.779]0.761 [0.738, 0.783]0.752 [0.729, 0.775]0.760 [0.736, 0.783]0.756 [0.734, 0.779]0.775 [0.751, 0.799]
t−70.622 [0.594, 0.649]0.690 [0.656, 0.724]0.742 [0.722, 0.763]0.750 [0.721, 0.779]0.754 [0.734, 0.774]0.752 [0.725, 0.779]0.747 [0.721, 0.773]0.755 [0.729, 0.782]0.752 [0.727, 0.777]0.767 [0.740, 0.794]
Mean0.627 [0.596, 0.658]0.697 [0.663, 0.731]0.737 [0.712, 0.763]0.757 [0.730, 0.784]0.756 [0.731, 0.781]0.760 [0.734, 0.785]0.755 [0.730, 0.780]0.766 [0.740, 0.791]0.768 [0.743, 0.793]0.786 [0.760, 0.812]

Mean AUROC across the eight detectors increases from 0.627 at p1 to 0.786 at p10 (gain: 0.1587; paired-bootstrap 95% CI [0.1345, 0.1845]). This increase persists on the same trajectories, so it is not due to changing cohort composition. Most of the gain occurs by mid-trajectory, with smaller gains thereafter. We will describe this as improved prediction from accumulated pre-decision context, rather than strictly gradual emergence. Table R1.7 separately tests how individual hidden states change toward the decision.

9. Summary of evaluation cohorts

The fixed-cohort analyses support detection at early windows and stronger prediction closer to the decisive event. Length and task type alone do not explain these results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.