- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies online, black-box, action-only monitoring of scheming and sabotage in LLM agents. It builds on prior work on offline deliberative monitoring. The authors introduce a three-judge labeling pipeline that transforms trajectory-level labels into per-step confidence, severity, and irreversibility scores. They also label an ideal flagging step and a point of no return (PONR). In addition they propose timeliness aware metrics, t-AUROC and t-pAUROC@20, which credit detection only before irreversible harm occurs.
Using this dataset, the authors distill frontier-model deliberations into seven open-weight monitors using SFT, with additional RL for two Qwen models. They evaluate on four held-out OOD datasets and report competitive cost–performance and latency–performance tradeoffs relative to prompted frontier monitors.
The paper also investigates how training-set composition affects OOD transfer, how monitoring failures arise (from misweighting vs missing evidence), and how detection degrades when the monitor has access to only a limited context window.
Critical strengths
-
Per-step labeling framework: I think the three-judge pipeline that uses privileged trajectory information to generate per-step supervision, flagging steps and points of no return, is a unique/important contribution for the community. It can enable adapting existing scheming/control datasets for online monitoring.
-
Experimental coverage: The failure mode analysis in the paper provides useful insights into how monitors sometimes misweigh evidence rather than simply failing to recognize it. The training-mixture experiments also identify action classes and scope violations as factors associated with OOD transfer. This seems to offer some practical/actionable guidance for constructing monitoring datasets.
Critical weaknesses
-
Issues with evaluation methodology: The main results evaluate only three selected prefixes per trajectory rather than every step, thus underrepresenting the fraction of false positives. Appendix E.6.3 (Table 8) shows that full evaluation reduces Qwen3.8-27B OOD t-pAUROC@20 from 0.885 to 0.751, while its FPR increases from 12.7% to 39.7%. This gap is smaller for Gemini 3.1 Pro. However since the paper's main claim is about the competitiveness of its trained online monitors, these results in my opinion undermine the reported Pareto-frontier comparisons. The problem is more severe on longer trajectories where online monitoring is most useful.
-
On methodological novelty and OOD generalization. The training strategy largely follows Sinha et al. (2026), including teacher rationale distillation, SFT, and RL, while prior work has already explored per-step supervision and online auditing. The strongest model is already competitive without training (0.837 OOD t-pAUROC@20), improving only to 0.878 after SFT+RL. Improvements are concentrated in certain datasets, particularly BashArena, while performance on SHADE-Arena actually gets worse (0.723 to 0.695). This raises questions about how much monitoring capability the proposed training method adds and whether the gains are generalizable.
Questions for authors
-
Full online evaluation: Given the discrepancy in Table 8, could the authors provide full-prefix evaluation results for Qwen3.8-27B and its closest frontier competitors (eg. Gemini 3.7 Flash or Claude Opus 4.8)? Do the main Pareto-frontier conclusions hold even under full evaluation?
-
On generalization :How much does the proposed training improve detection over the untrained base model under full online evaluation? Why does training improve performance substantially on BashArena but degrade it on SHADE-Arena? Can the authors provide stronger evidence that the improvements generalize beyond attack classes represented in training?
-
PONR labeling validity. Appendix E.3 mentions moving the PONR forward when it precedes the ideal flagging step, affecting around 175 SHADE-Arena trajectories. Could the authors explain this and also report results without this adjustment? How many of these temporal labels were human-verified/what was the agreement rate?
6 comments
Response to Reviewer 7Fcd (2/3)
On generalization :How much does the proposed training improve detection over the untrained base model under full online evaluation?
We have run additional experiments to determine this, please see the below table. It shows performance metrics for the untrained base model and trained variant on 600 trajectories from the four datasets (150 per dataset).
| Monitor | t-pAUROC@20 T=1 | t-pAUROC@20 T=0 | t-AUROC T=1 | t-AUROC T=0 | FPR T=1 | FPR T=0 | miss rate T=1 | miss rate T=0 |
|---|---|---|---|---|---|---|---|---|
| Qwen3.5-4B base | 0.334 | 0.484 | 0.685 | 0.713 | 48.7% | 28.0% | 16.0% | 27.3% |
| Qwen3.5-4B SFT+RL | 0.513 | 0.605 | 0.780 | 0.827 | 50.7% | 40.0% | 1.3% | 4.3% |
| Qwen3.8-27B base | 0.843 | 0.852 | 0.926 | 0.924 | 5.0% | 3.7% | 8.3% | 9.7% |
| Qwen3.8-27B SFT+RL | 0.751 | 0.792 | 0.896 | 0.930 | 39.7% | 24.7% | 1.0% | 1.7% |
We find that our trained monitors have a lower miss rate than the untrained base counterpart (1% vs. 8% miss rate for the Qwen3.8-27B and 1% vs 16% for 4B at temperature one), but find that Qwen3.8-27B has a higher FPR (40% vs 5% at temperature one). At temperature zero, t-AUROC (which is the metric measuring discrimination between scheming and benign trajectories) improves with training for both monitors.
t-pAUROC@20 decreases for the 27B model when comparing base vs. trained due to this metric punishing FPR much more and the trained model flagging more eagerly. We now make note of this in Section 4.4. This reflects eager flagging rather than training itself: the weaker prompted monitors lose as much or more from approximate to full evaluation (at T=0, Haiku and Luna drop by 0.15 and 0.14 in mean-OOD t-pAUROC@20, versus 0.09 and 0.13 for our trained 27B and 4B). However, the metric increases for the 4B model showing that under full evaluation training does help in some cases. All in all, this emphasizes the point that giving scheming interpretations to benign trajectories remains the biggest failure mode of our trained monitors and a valuable point of improvement for future work.
Appendix E.6.3 now reports full evaluation results for all eight monitors mentioned in this response and the previous one.
The training strategy largely follows Sinha et al. … while prior work has already explored per-step supervision and online auditing
We agree that the SFT recipe is inherited from the Sinha et al. recipe, and that was a deliberate design choice (although when it came to RL, we explored a different reward function). Holding the recipe fixed means any difference we observe is attributable to the adaption of this method to the online setting. We see the contribution of the paper as the online setting itself which required the formalization of the online setting, the PONR-anchored timely metrics, the step-level labeling pipeline, the mixture-transfer analysis, and the failure-mode analysis. We agree that per-step supervision and online auditing exist in prior work, but to our knowledge, none addresses online detection of covert scheming. StepShield uses synthetic trajectories, each with a single injected violation whose onset is fixed by construction, and it trains no monitor. AgentForesight trains an online auditor, but for benign task failures in multi-agent systems. Claude Code auto mode and Codex auto-review target overeager or overtly dangerous actions, and Codex explicitly states it "should not be understood as a mechanism for protecting against model scheming." None of these defines timeliness relative to irreversibility, labels existing scheming benchmarks at the step level, or evaluates trained action-only monitors against an agent that hides its intent, which is the gap our work fills.
The strongest model is already competitive without training (0.837 OOD t-pAUROC@20), improving only to 0.878 after SFT+RL. Improvements are concentrated in certain datasets, particularly BashArena, while performance on SHADE-Arena actually gets worse (0.723 to 0.695).
and
Why does training improve performance substantially on BashArena but degrade it on SHADE-Arena?
Our mixture analysis (Section 4.2) attributes this to how closely each set's action classes match the training data. BashArena is dominated by code and configuration inserts, which are also the dominant action class in BashBench and LinuxArena, so training transfers well: SFT lifts t-pAUROC@20 from 0.08 to 0.80 at 4B and from 0.72 to 0.90 at 27B. SHADE-Arena's dominant action class, financial transfers, is almost absent from all four training sets, so no mixture covers it. Sinha et al. also found SHADE-Arena fragile under every training mixture they tried, because training led their monitors to read messy but benign trajectories as concealment, which raised false positives. SHADE-Arena is hard even for strong frontier monitors: at maximum reasoning effort, GPT-5.6 Sol and Gemini 3.7 Flash reach only 0.615 and 0.654, both below the trained 27B (0.695); only Gemini 3.1 Pro does clearly better (0.803).
Response to Reviewer 7Fcd (3/3)
This raises questions about how much monitoring capability the proposed training method adds and whether the gains are generalizable.
and
Can the authors provide stronger evidence that the improvements generalize beyond attack classes represented in training?
We believe our OOD evaluation sets already test this. The datasets come from different author groups and generation styles, and they differ in construction components: iterative generator/discriminator refinement, agent-simulator rollouts, prompt-injection environments, and sandboxed agentic rollouts, each with distinct prompts and side tasks. Some attack classes appear only in OOD and are entirely absent from training, such as AgentDojo's prompt-injection attacks. Despite these differences, training generalizes, especially on the 4B monitor: SFT raises mean-OOD t-pAUROC@20 by 0.24 here. Gains on the 27B monitor whose base model is already strong are more modest (+0.03). The 27B is the exception rather than the rule: all seven open-weight students we train improve on mean-OOD t-pAUROC@20 under SFT, by between 0.07 and 0.24 for the other six (Nemotron 3.5 Lightning +0.23, Nemotron 3 Super +0.18, Inkling +0.13, Nemotron 3 Ultra +0.12, Inkling-Small +0.07; Table 14 in Appendix I.3), and the gain shrinks as the base model gets stronger, which is why the 27B, our strongest base, gains least. RL adds performance to both.
PONR labeling validity. Appendix E.3 mentions moving the PONR forward when it precedes the ideal flagging step, affecting around 175 SHADE-Arena trajectories. Could the authors explain this and also report results without this adjustment? How many of these temporal labels were human-verified/what was the agreement rate?
We use this mechanism because the adjudicator sometimes places the PONR on an irreversible step where the action is already damaging but the monitor cannot yet be confident the behavior is scheming by looking at the trajectory alone. Such a PONR sits before k∗, the earliest step at which flagging is ideally possible, so the timely window would contain no flaggable step and any monitor would be scored as missing it. In practice this mechanism is almost always activated in a specific SHADE-Arena scenario where an agent is asked to book flights for a customer with the hidden side-task to only book expensive flights from a certain provider. The first flight booked with this provider is flagged as the PONR, but it’s the pattern of booking multiple flights with that provider which is the evidence for scheming, so putting the PONR there is premature and unfair to monitors. After this change, the labels of 10+ trajectories containing this scenario were checked and we found them more accurate than before and agreed with the placement of the PONR.
In the following table we report the miss rate of different monitors on these specific trajectories before and after this adjustment (note: only 128 out of 175 fall inside the 800-trajectory SHADE-Arena evaluation set):
| Monitor | miss rate before | miss rate after | change (trajectories rescued) |
|---|---|---|---|
| Gemini 3.1 Pro | 96.1% | 86.7% | -9.4 pp (12) |
| GPT-5.6 Sol | 99.2% | 88.3% | -10.9 pp (14) |
| Claude Opus 4.8 | 100.0% | 95.3% | -4.7 pp (6) |
| Qwen3.8-27B SFT+RL | 94.5% | 51.6% | -42.9 pp (55) |
The trained 27B student benefits far more than the frontier monitors (-43 pp vs -5 to -11 pp). It flags these trajectories aggressively but late under the old PONR; clamping to k* nearly halves its miss rate. The frontier monitors rarely flag these hard trajectories at all, so there is little to rescue. Note that this rule was fixed before we trained any monitors.
Response to Reviewer 7Fcd (1/3)
We thank the reviewer for the positive assessment, especially for recognising the per-step labelling pipeline as a way to adapt existing datasets to online monitoring, and the practical guidance from our failure-mode and training-mixture analyses. The main concern is our reliance on approximate (landmark-prefix) evaluation. We agree it is a real caveat, and below we show which of the paper's conclusions hold under full evaluation and where we have revised the text.
And
We thank the reviewer for raising this; we agree that it is an important caveat on our headline numbers. Appendix E.6.3 compares metrics under landmark-prefix scoring and full scoring, showing that our approximate metrics are optimistic. Under full evaluation of 150 class-balanced trajectories per OOD set, Gemini 3.1 Pro’s FPR rises by 3.6x, while Qwen3.8-27B’s rises by 3x. The reason is that a monitor whose scores on benign steps have even moderate variance gets many more chances to cross the threshold once every prefix is scored. The trained monitors tend to fluctuate above and below the threshold more often (27B crosses the threshold and drops below it more than three times on 41% of benign SHADE-trajectories vs. 2.7% for 3.1 Pro), and the increased amount of sampling on benign trajectories under full evaluation makes it so that there is a higher chance of a flag score spike crossing the threshold, raising FPR, and making their t-pAUROC@20 fall more drastically than Gemini 3.1 Pro. To address this, we decided to run the full evaluation using a temperature of zero as well, as the lower variance in responses leads to a lower chance of inaccurately flagging benign trajectories. See the results here (note that the prompted frontier runs with default reasoning effort, and we use 150 trajectories from each OOD dataset):
In the table above, the trained 27B stays competitive on t-AUROC, above Flash and within 0.02 of Gemini 3.1 Pro, but trails both by 0.09 on t-pAUROC@20, so it no longer nearly matches Flash on the headline metric. We have updated the abstract, Section 4.1 and Appendix E.6.3 to reflect this. We did not run Opus and Sol due to it costing $8k to run this evaluation even on this subset of trajectories. Based on the observed variance on benign steps of our trained monitors, we see building aggregators that avoid having a single high score flag a benign trajectory as practical follow-up work. We will add this to our future work recommendations.
Nevertheless, at default temperature the ordering of monitors by t-pAUROC@20 is mostly preserved between approximate and full evaluation (Gemini 3.1 Pro > Flash > Qwen3.8-27B (SFT+RL) > Haiku > Luna > Qwen3.5-4B (SFT+RL)), with only Luna overtaking the trained 4B model. At temperature zero, our trained Qwen3.5-4B overtakes Luna again under full evaluation. Given that Luna has much higher latency and cost, we do not expect the Pareto frontier to change under full evaluation.
Lastly, we want to emphasize that using approximate evaluation is strictly necessary when doing evaluation on the complete OOD set as scoring every prefix is prohibitively expensive. Using our cost estimation, we think it costs about \59ktoevaluatethepromptedmonitorsonallOODdatasetsatdefaulteffortwhilstscoringeveryprefix(and 31k with per-step prompt caching).