acceptodds
Under review as a conference paper at ICLR 2027

JANUS: Foreseeing Latent Risk for Long-Horizon Agent Safety

Abstract

Agent safety is shifting from content moderation toward preventing operational failures before tool-using agents act. However, most agent guardrails remain reactive, judging safety only from observed actions or trajectory states. This is insufficient for long-horizon tasks, where benign-looking steps may lead to delayed harm. We introduce JANUS, a foresight-oriented framework for predictive guarding from partial trajectories. JANUS trains a shared policy with two tasks: anticipation forecasts safety-relevant futures, while adjudication judges safety from the observed prefix and anticipated future. We further propose CoAA-RL, which couples the two tasks through reward. Adjudication receives verifiable safety rewards, whereas anticipation is optimized for both future-summary fidelity and downstream utility. JANUS also synthesizes diverse trajectories through multi-agent simulation. Across four agent-safety benchmarks, the resulting model, VANGUARD, improves average protection by 15.9 percentage points over baseline guards while increasing benign task completion by 5.1 percentage points.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Agent Safety, Predictive Guardrails, Long-Horizon Reasoning, Large Language Model Agents, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CMckOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes JANUS, a training framework that combines simulated long-horizon trajectories with Coupled Anticipation and Adjudication Reinforcement Learning (CoAA-RL), jointly training future prediction and safety judgement. The paper studies predictive guarding for long-horizon agent safety, where risks arise from delayed consequences rather than immediately apparent harmful intent. The resulting model, VANGUARD, anticipates safety-relevant continuations and uses them to identify unsafe behaviour before execution. Experiments across four benchmarks show improved protection against unsafe agent behaviour, while benign task completion is preserved on AgentDojo.

Critical strengths
  1. The training data covers risks originating from users, external environments, and the agent’s own mistakes, giving the approach broader scope than defenses focused only on malicious prompts.

  2. VANGUARD achieves the lowest attack success rate on all four benchmarks, showing consistent improvements across the evaluated safety settings. The improvement is particularly substantial on LPS-Bench, where ASR decreases from 32.3% for the strongest competing defence to 7.5%. On AgentDojo, VANGUARD reduces ASR from 16.6% without a guard to 5.4%, while maintaining the same benign-task success rate of 68.0%.

  3. The ablations provide useful evidence for both anticipation and reward coupling. Removing anticipation during training or withholding predicted summaries at inference worsens safety and benign-task performance, while similarity-only, utility-only, and decoupled training also underperform the full method.

Critical weaknesses
  1. Benign-task utility is reported only for AgentDojo. Consequently, the table does not establish whether the safety improvements on the other three benchmarks come at the cost of blocking legitimate tasks.

  2. Results are presented without confidence intervals or variability across repeated runs, making it difficult to assess the reliability of smaller improvements, such as the reduction from 6.2% to 5.4% ASR on AgentDojo.

  3. The paper does not report computational overhead, so the performance gains cannot be assessed alongside the additional cost of prediction and adjudication.

  4. ASR measures whether harm occurred but does not reveal how many steps before harm the guard intervened. The prefix experiments provide partial evidence, yet do not report detection lead time relative to the first harmful action. Lower ASR alone does not establish better foresight. Prefix length does not directly measure intervention timing. The table (3) does not show how far before the first harmful action the guard intervenes. Reporting detection lead time would more directly substantiate the claim of anticipating delayed harm.

  5. Early prediction is less effective on planning tasks. In table3 at 25% prefix on LPS-Bench, VANGUARD’s ASR is 49.8%, compared with QwenGuard’s 33.2%. Its advantage therefore does not extend to the earliest observation budget across all settings.

  6. The related work explicitly discusses SafePred, which predicts future risks through world modeling, but the experiments do not compare against it. This leaves the empirical distinction from existing predictive guarding insufficiently established.

  7. The benefit of forecasting is not isolated from additional inference computation. The method performs separate anticipation and adjudication generations but does not compare against a direct safety judge given a comparable inference budget. Such a comparison would help establish whether explicit forecasting accounts for the improvement.

  8. Generalisation across execution agents remains unclear.

Overall, the paper presents a promising forecast-then-judge guard, with strong reported safety results and useful component ablations. Its most convincing contribution is coupling forecast training to downstream safety classification.

Questions for authors

see the weaknesses section

5 comments
AuthorsOpenReview ↗

Response to Reviewer CMck (Part 1/2)

Thank you for recognizing the broad risk coverage, safety improvements, and evidence for reward coupling. Following your suggestions, we added utility evaluations, repeated runs, timing and cost measurements, and matched comparisons with predictive and direct-judgment baselines.

W1: Utility beyond AgentDojo.

VANGUARD preserves AgentDojo utility at 68.0%, equal to no guard, while reducing ASR from 16.6% to 5.4%. We also evaluate 100 confirmed-safe counterparts per additional setting:

SettingNo-guard utilityTS-Guard utility / FBRVANGUARD utility / FBR
Agent-SafetyBench74.0%67.0% / 8.0%72.0% / 4.0%
AgentLAB69.0%68.0% / 7.0%67.0% / 5.0%
LPS76.0%70.0% / 10.0%73.0% / 5.0%

The counterparts preserve the legitimate task objective and success criterion. Utility measures successful completion, following lines 681 to 685, while FBR measures the fraction of safe tasks terminated by the guard. Safe-task status is assessed separately because a benign user request can still lead to unsafe execution, particularly on LPS.

VANGUARD reduces false blocking across all three settings and keeps utility within 2, 2, and 3 points of no guard. Utility improves over TS-Guard by 5 points on Agent-SafetyBench and 3 points on LPS. On AgentLAB, utility is 67.0% versus 68.0%, with lower FBR at 5.0% versus 7.0%. The additional evaluations therefore support a favorable safety and utility balance across these settings.

W2: Reliability of the small AgentDojo improvement.

We added repeated evaluations and a paired task-bootstrap interval. Across 3 runs with fixed guards and independent execution seeds, AgentDojo ASR is 5.6 ± 0.4% for VANGUARD and 6.4 ± 0.5% for LlamaFirewall. The 95% interval for VANGUARD minus LlamaFirewall is [−2.1, 0.5] percentage points.

The AgentDojo difference is small and its interval includes zero. On LPS, repeated ASR is 7.8 ± 0.7% for VANGUARD versus 53.2 ± 1.1% for LlamaFirewall. The larger observed gap on LPS complements the utility and timing evidence for long-horizon safety.

The repeated runs reuse fixed guard checkpoints and vary execution seeds. They add an assessment of execution variability to the original single-run setup (lines 700 to 701). We will report the repeated results and interval alongside the main table.

W3: Deployment cost.

Inference uses two sequential generations from the same 8B policy. The guard first predicts a future summary and then judges safety conditioned on that summary (lines 304 to 314). The 8×8 rollout sampling is used during training (lines 704 to 707).

Latency is measured on one H200 GPU in BF16 with batch size 1, a 4,096-token prefix, and a total output limit of 1,536 tokens per decision. The measurements include both stages and repeated context processing:

MethodCalls per decisionMean output tokensp50 / p95 latency
Single-call direct judge157.00.84 / 1.36 s
Two-call direct judge2113.21.79 / 2.02 s
VANGUARD2168.72.29 / 3.61 s

VANGUARD uses 168.7 output tokens per decision, with a median latency of 2.29 s. Compared with the two-call direct judge, it uses 55.5 more output tokens and adds 0.50 s to median latency. The corresponding LPS ASR falls from 14.0% to 8.1% in the controlled comparison below. End-to-end task time increases by 12.8% over no guard in the measured workload. These measurements quantify the safety benefit alongside the additional cost of anticipation.

AuthorsOpenReview ↗

Response to Reviewer CMck (Part 2/2)

W4: Direct detection lead time.

Following your suggestion, we directly measure warning time before the first harmful action. The evaluation replays prefixes of unguarded reference trajectories and measures lead as h−th-t, where h is the first annotated harmful action and t is the blocking decision. Blocking immediately before execution has lead 0.

At a validation-calibrated FBR target of 5.0%, delayed-planning EDR@1/2/3 is 65.0%/46.0%/29.0% for VANGUARD versus 36.0%/18.0%/8.0% for TS-Guard. Mean lead is 2.1 versus 1.0 actions, with coverage of 89.0% versus 68.0%. EDR includes missed detections among all 200 delayed-planning cases. Mean lead is conditional on detection, so coverage is needed to interpret it.

VANGUARD improves both detection coverage and warning time on the replay set. The gains at all three lead thresholds provide direct evidence for earlier intervention and complement the closed-loop ASR results. We will include the calibration settings, actual test FBR, and annotated action indices with this evaluation.

W5: The 25% LPS result.

We agree with this point. At 25% prefix, VANGUARD's 49.8% ASR is worse than QwenGuard's 33.2%. Its advantage in Table 3 begins at 50%, reaching 27.9%, 15.1%, and 7.5% at 50%, 75%, and 100%. We will revise the Table 3 discussion to state this dependence on the observation budget.

W6 and W7: SafePred and additional inference computation.

We added both SafePred and a two-call direct judge under matched evaluation conditions. The methods use the same execution agent and tasks, with matched visible history, decision points, and total output cap:

MethodLPS ASRAgentDojo utility
SafePred, matched block-only12.6%66.0%
Two-call direct judge14.0%66.0%
VANGUARD8.1%67.0%
  • Matched baselines. VANGUARD achieves the lowest LPS ASR and highest AgentDojo utility in this comparison. SafePred uses a block-only adaptation with native plan updates disabled, giving both methods the same intervention interface. The two-call direct judge spends its budget on two safety judgments. We will provide the judge checkpoint, inputs to each call, and aggregation rule alongside the SafePred guard configuration.

  • Summary content. Holding VANGUARD's adjudicator fixed, within-class summary shuffling raises LPS ASR from 7.5% to 15.6%. The original no-summary ablation also increases LPS ASR to 16.3% (Table 2, lines 331 to 334). These tests show the value of supplying the adjudicator with the corresponding future content. The two-call comparison above fixes the call count and output cap, with actual inference costs reported in W3.

The controlled comparisons link the safety gain to useful future content. VANGUARD achieves lower ASR than the two-call judge under the same output cap, with the additional token use and latency quantified above. Disrupting its summary content also reduces safety. The content ablations use the original 7.5% configuration, while the matched evaluation uses the 8.1% configuration. We will list these configurations alongside the results.

The LPS evaluation uses 570 instances and the episode-based ASR definition in Eq. 13 (lines 674 to 679 and 686 to 690). We will add the repository revision, evaluator version, and outcome counts, including the treatment of technical failures and guard-terminated trajectories, to the evaluation details.

W8: Transfer across execution agents.

We keep the guard weights, prompts, and intervention rules fixed and replace the execution agent:

Execution agentLPS ASR: no guard / TS-Guard / VANGUARDAgentDojo utility: no guard → VANGUARD
Qwen3-235B-A22B-Instruct-250776.1% / 27.2% / 6.5%75.3% → 74.2%
Llama-3.3-70B-Instruct81.4% / 30.2% / 8.9%70.1% → 69.1%

The same guard reduces LPS ASR substantially with both additional execution agents and outperforms TS-Guard on each. AgentDojo utility remains close to no guard, decreasing by 1.1 and 1.0 points. Together, these results support reusing VANGUARD across the evaluated execution-agent sizes and families with fixed weights, prompts, and intervention rules.

Reference: SafePred.

AuthorsOpenReview ↗

Follow-up to Reviewer CMck

Dear Reviewer CMck,

Thank you again for your detailed review. Following your suggestions, we added benign utility evaluations, repeated runs with a paired confidence interval, and measured inference costs. We also reported direct detection lead time, comparisons with SafePred and a two-call judge, and results with two additional execution agents.

We would appreciate your thoughts on whether our response addresses your concerns. Please let us know if any point needs further clarification.

Best regards, Authors of Submission 50365

Reviewer_WErdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes JANUS, a predictive guardrail framework for long-horizon LLM agent safety. It uses multi-agent simulation to generate safety-related trajectories and introduces CoAA-RL to jointly optimize future-risk anticipation and safety adjudication by combining prediction fidelity with downstream classification rewards. The resulting model, VANGUARD, achieves lower attack success rates across four agent-safety benchmarks while maintaining competitive benign-task utility. Ablation studies support the effectiveness of anticipation and coupled reward optimization.

Critical strengths
  1. Relevant problem formulation and meaningful methodological design.

The paper addresses an important limitation of reactive agent guardrails: their inability to anticipate delayed consequences in long-horizon workflows. Coupling future anticipation with safety adjudication through task-dependent rewards is a meaningful approach, as it explicitly optimizes predictions for downstream safety decisions rather than prediction fidelity alone.

  1. Strong benchmark results supported by ablation studies.

VANGUARD achieves the lowest attack success rate across four agent-safety benchmarks while maintaining competitive benign-task utility. The ablations further demonstrate that both anticipation and coupled reward optimization contribute substantially to the reported improvements, providing useful evidence for the proposed design.

Critical weaknesses
  1. Insufficient definition and direct evaluation of delayed-risk prediction.

The central claim is that JANUS anticipates risks before harmful actions occur, yet delayed risk is not operationally distinguished from immediately identifiable unsafe behavior. Training contains substantial user-originated harmful requests (55%), and the evaluation combines prompt-injection, general safety, and long-horizon planning cases. The reported attack success rate (ASR) measures whether an attack succeeds, not when a risk is first recognized. Prefix-length ASR is an indirect proxy and does not quantify detection lead time or early intervention at a fixed false-blocking rate. Consequently, it remains unclear whether gains reflect genuine delayed-consequence anticipation rather than improved step-level safety classification.

  1. Insufficient validation of anticipation fidelity and synthetic supervision.

CoAA-RL rewards anticipated summaries partly through downstream adjudication accuracy, but a higher classification reward does not necessarily imply more faithful prediction of future events. The paper does not independently quantify the factual correctness, risk-event coverage, or calibration of its anticipated summaries, particularly for low-probability delayed harms. Consequently, the observed gains may partly reflect additional safety-classification supervision rather than improved foresight. This concern is compounded by reliance on simulated trajectories and GPT-5.5-generated summaries and labels, whose reliability and transfer to real agent execution are not sufficiently established.

Questions for authors
  1. Delayed-risk definition and timing. How is delayed risk operationally defined and separated from immediately identifiable violations? Can the authors report detection lead time (steps before the first harmful action), early detection rates at several lead-time thresholds, and false-positive/benign blocking rates, in addition to ASR?
  2. Risk-specific evaluation. Can the authors report ASR and early-intervention performance separately for user-, environment-, and agent-originated risks, particularly delayed planning failures? This would help establish which failures benefit from anticipation.
  3. Comparison to existing predictive guards. SafePred (2026) already predicts long-term risk to guide intervention, while ToolSafe/TS-Guard (2026) performs proactive step-level safety monitoring. Can the authors compare with SafePred using matched agent backbones, intervention budgets, ASR, benign utility, and early-detection metrics, and isolate the gain specifically attributable to CoAA-RL?
  4. Anticipation fidelity versus adjudication gains. Can the authors independently evaluate future-summary factual accuracy and coverage of consequential risk events, including low-probability delayed outcomes? Can they disentangle improvements in anticipation from additional adjudication supervision using a matched training-budget control (e.g., a direct classifier trained with comparable safety labels and RL updates)?
  5. Data reliability and generalization. How were GPT-5.5-generated labels and future summaries validated against human judgments or observed outcomes? Do gains persist on independently collected agent trajectories, particularly agent-originated planning failures?
2 comments
AuthorsOpenReview ↗

Response to Reviewer WErd (Part 1/2)

Thank you for recognizing the relevance of the problem and the coupled learning design. Following your suggestions, we added timing and risk-specific evaluations, a SafePred comparison, and controls for summary fidelity and computational budget.

W1 / Q1: Delayed risk and intervention timing.

Delayed risk requires connecting the observed state to an unsafe downstream consequence. The 55% user-originated share describes risk origin. We evaluate delayed-risk recognition separately through intervention timing. In Fig. 1(c), the relevant dependency links an earlier Bakery setup to a later driver update (lines 47 to 53). Predicting that dependency allows the guard to flag the incompatible update before it breaks the earlier workflow.

TS-Guard checks candidate actions before execution. JANUS adds an explicit prediction of downstream consequences and trains that prediction for safety adjudication. We will revise Fig. 1 and the discussion in lines 38 to 50 to make this methodological distinction clear.

We added direct timing measurements using prefix replay of unguarded reference trajectories. At a validation-calibrated FBR target of 5.0%, delayed-planning EDR@1/2/3 is 65.0%/46.0%/29.0% for VANGUARD, compared with 36.0%/18.0%/8.0% for TS-Guard. Mean lead is 2.1 versus 1.0 actions, with detection coverage of 89.0% versus 68.0%.

Lead is h−th-t, relative to the first annotated harmful action. Lead 0 means blocking immediately before its execution. EDR includes missed detections among all 200 delayed-planning cases, while mean lead is conditional on detection. The results show higher detection coverage and earlier warning at every reported lead threshold. We will include the calibration settings, actual test FBR, and annotated action indices with this evaluation.

Q2: Benefits by risk origin.

The source-stratified results separate harmful user goals, risky external inputs, and agent-originated planning failures:

OriginASR: TS-Guard → VANGUARDEDR@2: TS-Guard → VANGUARD
User9.6% → 5.8%51.0% → 65.0%
Environment13.8% → 8.2%33.0% → 55.0%
Agent / planning27.6% → 10.2%18.0% → 46.0%

Each origin has 500 ASR trajectories and 200 delayed cases for timing. ASR uses the default intervention rule, while EDR uses validation-calibrated thresholds. This balanced evaluation makes the performance across risk origins easier to compare.

Agent-originated planning failures show the largest ASR reduction. Their EDR@2 also increases from 18.0% to 46.0%, meaning that more cases are detected at least two actions before the harmful action. This directly supports the intended benefit for delayed planning risks.

Q3: SafePred and the contribution of CoAA-RL.

We added a controlled comparison with SafePred. Both methods use the same Qwen3-32B execution agent, tasks, visible history, decision points, and total output cap:

MethodAgentDojo ASRAgentLAB ASRLPS ASRAgentDojo utility
SafePred, block-only6.5%10.5%12.6%66.0%
VANGUARD5.7%9.0%8.1%67.0%

VANGUARD achieves lower ASR across all three benchmarks and higher AgentDojo utility in this comparison. SafePred's EDR@1/2/3 is 56.0%/35.0%/20.0% under the same validation-calibration protocol, compared with 65.0%/46.0%/29.0% for VANGUARD. SafePred uses a block-only adaptation with native plan updates disabled, giving both methods a common intervention interface. We will provide the separate guard checkpoint, policy prompt, prediction horizon, and blocking rule with these results.

The original reward ablations support the coupled design. Sim-only removes adjudication utility from the anticipation reward, while Util-only removes future-summary similarity (lines 412 to 416). Their LPS ASRs are 19.5% and 17.2%, compared with 7.5% for VANGUARD in Table 2 (lines 331 to 337). Combining both signals improves safety over either single-reward variant. Decoupled reaches 48.4% and provides an additional comparison between training configurations. We will clarify its parameter sharing and update schedule in the implementation details.

The reward ablations use the original configuration, whose LPS ASR is 7.5%. The matched evaluation above uses the 8.1% configuration. We will list the configuration differences alongside the respective results.

AuthorsOpenReview ↗

Response to Reviewer WErd (Part 2/2)

W2 / Q4: Faithful anticipation versus extra supervision.

We added an independent event evaluation and matched-budget controls to test this concern. They assess prediction quality, the contribution of supervision, and the use of summary content.

  • Fidelity. Predictions are blindly assessed against observed continuations. Supported-event precision divides supported predicted events by all predicted events. Risk-event recall divides covered risk events by all observed risk events. VANGUARD obtains 87.0% precision and 81.0% recall, versus 86.0% and 67.7% for Sim-only, and 73.3% and 71.0% for Util-only. The coupled objective improves risk-event recall by 13.3 points over Sim-only while preserving comparable precision. We will provide the event counts and matching rubric with these results.

  • Budget controls. The direct classifier matches initialization, training data, labels, RL updates, and training compute. Its LPS ASR is 17.2%, compared with 8.1% for VANGUARD in the matched evaluation. A separate two-call direct judge under the same total output cap gives 14.0%. VANGUARD therefore retains lower ASR than the training-matched classifier and the two-call judge. Both inference methods use two calls. Actual output use is 168.7 versus 113.2 tokens per decision, with median latency of 2.29 versus 1.79 s. The judge checkpoint, two prompts, and decision aggregation rule will accompany the comparison.

  • Summary content. Within-class shuffling increases LPS ASR from 7.5% to 15.6% with the adjudicator fixed. This preserves class membership while disrupting the correspondence between the prefix and summary. The performance drop shows that the adjudicator uses information specific to the instance.

The fidelity measurements and budget controls jointly support explicit anticipation as a contributing factor. The summaries capture more observed risk events, and their corresponding content helps the adjudicator make safer decisions under the shared generation limits.

W2 / Q5: Supervision reliability and transfer.

Training uses simulator-generated tool observations (lines 174 to 176). We evaluated transfer on task- and template-disjoint execution trajectories and completed a human audit of 50 sampled trajectories.

  • Transfer. On the independent execution set, disjoint from training tasks and templates, TS-Guard → VANGUARD ASR is 10.0%→8.0% for user risks, 18.0%→6.0% for environment risks, and 32.0%→12.0% for planning failures. Gains persist across all three origins, with the largest reduction on planning failures. We will provide the sample counts, environment sources, and tool-observation procedure with the evaluation details.

  • Human audit. Two annotators independently reviewed 50 sampled trajectories. Synthetic labels agree with adjudicated labels in 47 of 50 cases (94.0%). The reference summary's main consequences are supported by the recorded continuation in 43 of 50 cases (86.0%). Raw annotator agreement is 44 of 50 cases (88.0%), with Cohen's κ=0.82\kappa = 0.82. The audit provides direct human validation of label quality and separately measures the grounding of the reference summaries.

References: SafePred and ToolSafe.

Reviewer_ibG5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper a framework for training guardrails that predict unsafe agent behaviour before a harmful action is executed. The framework builds a training dataset through multi-agent simulation. Then it uses GRPO to train a shared policy using the coupled rewards on two tasks. The anticipation task generates a summary of the future, and the adjudication task uses that summary and the observed prefix to predict a safety label. The authors show that the trained guardrail model VANGUARD has the lowest attack success rate on all four benchmarks.

Critical strengths
  1. The motivation of this paper is very clear as reactive guards can be too late when early harmless steps lead to delayed harm
  2. The reward design is very interesting. The coupled reward is more than a prediction step but provides faithful signal for safety decision. The idea can also be tested with other guard frameworks.
  3. VANGUARD shows the lowest attack success rate on all four benchmark comparing with other safe guards.
Critical weaknesses
  1. The paper does not state whether the simulated training data builds on the evaluation benchmarks. Its generation examples are not disclosed, so possible leakage cannot be ruled out. The objective weight and review acceptance rate are also missing.
  2. Safety evaluation relies on ASR, which does not reward earlier intervention, and summary accuracy is not reported. Lead time, false alarms and summary fidelity are needed to evaluate the framework.
  3. Benign task performance is reported only on AgentDojo. The other three benchmarks report only ASR, so a low ASR could partly reflect blocking too many actions.
  4. The paper does not define the label correctness and format validity scores. It is not clear whether missing an unsafe case is penalised more heavily than confusing safe with potential unsafe.
  5. Anticipation is rewarded when adjudication is correct, so summaries may learn to signal the label instead of predicting the future. As summary accuracy is not reported, this cannot be ruled out.
Questions for authors

Please see the comments on weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ibG5 (Part 1/2)

Thank you for recognizing the motivation, coupled reward design, and safety improvements. Following your suggestions, we added direct timing measurements, benign-task evaluations, and independent summary assessments. We explain how these results address the five concerns below.

W1: Training provenance and reproducibility.

Training generation used no tasks, trajectories, or examples from the four evaluation benchmarks. The 75,180 training examples were constructed through taxonomy-driven simulation. Reviewed trajectories are split into prefixes and continuations, and each prefix receives a safety label and a future summary (lines 188 to 197). The construction and review procedures are summarized below.

  • Released trajectories. We provide two archived agent trajectories showing the user instructions, assistant actions, and tool observations.

  • Training weights and filtering. The reward weights are λA=0.5\lambda_A = 0.5 and λD=0.1\lambda_D = 0.1 (lines 704 to 709), with η=1.0\eta = 1.0 in Eq. 9. The reviewer checks task consistency, risk coverage, and trajectory completeness before retaining a scenario (lines 180 to 184). First-pass acceptance is 73.6%. After at most 3 retries, final retention reaches 91.2%. Acceptance is measured over independently generated scenarios, while the 75,180 examples count annotated prefixes.

We will include the generation inputs and filtering configuration in the revised construction details, alongside the released trajectory examples.

W2: Direct evidence of early intervention.

Following your suggestion, we added direct warning-time measurements. The evaluation replays prefixes of unguarded reference trajectories and measures how far before the first harmful action each guard intervenes. At a validation-calibrated FBR target of 5.0%, the delayed-planning results are:

MethodEDR@1EDR@2EDR@3Detection coverageMean lead
TS-Guard36.0%18.0%8.0%68.0%1.0
VANGUARD65.0%46.0%29.0%89.0%2.1

Lead is measured as h−th-t, where h is the first annotated harmful-action index and t is the blocking decision index. Blocking immediately before that action has lead 0. EDR includes missed detections among all 200 delayed-planning cases, while mean lead is computed only over detected cases. Coverage is therefore reported alongside the mean.

VANGUARD achieves higher early-detection rates at all three lead thresholds. It also detects a larger fraction of cases and provides longer mean lead among detected cases. These results supply direct evidence of earlier risk recognition under this validation-calibrated protocol. We will include the calibration settings, observed test FBR, and example action indices with the results.

AuthorsOpenReview ↗

Response to Reviewer ibG5 (Part 2/2)

W3: Benign utility beyond AgentDojo.

VANGUARD retains the no-guard AgentDojo utility of 68.0% while reducing ASR from 16.6% to 5.4%. We also evaluate 100 confirmed-safe counterparts per additional setting:

SettingNo-guard utilityTS-Guard utility / FBRVANGUARD utility / FBR
Agent-SafetyBench74.0%67.0% / 8.0%72.0% / 4.0%
AgentLAB69.0%68.0% / 7.0%67.0% / 5.0%
LPS76.0%70.0% / 10.0%73.0% / 5.0%

The counterparts preserve the legitimate task objective and success criterion, with safety confirmed separately. Utility measures task completion, following lines 681 to 685. FBR measures the fraction of safe tasks terminated by the guard. Reporting both captures task performance and unnecessary intervention.

VANGUARD reduces false blocking in all three settings and retains utility within 2, 2, and 3 percentage points of no guard. Compared with TS-Guard, utility increases by 5 points on Agent-SafetyBench and 3 points on LPS. On AgentLAB, utility is 67.0% versus 68.0%, while FBR falls from 7.0% to 5.0%. These results extend the utility evidence beyond AgentDojo and show that the ASR reductions are accompanied by continued legitimate task completion.

W4: Reward definitions and error costs.

Both reward components are binary. We use ℓ=1[y^=y]\ell = \mathbf{1}[\hat{y}=y] and ϕ=1[required format is satisfied]\phi = \mathbf{1}[\text{required format is satisfied}]. With λD=0.1\lambda_D = 0.1, Eq. 4 becomes rD=0.9ℓ+0.1ϕr_D = 0.9\ell + 0.1\phi (lines 244 to 249 and 708 to 709). The format requires a brief rationale inside <think>...</think> followed by one boxed label (lines 873 to 879).

The correctness reward treats all label mismatches equally, including missed UNSAFE cases. At inference, only UNSAFE triggers blocking, while SAFE and POTENTIAL UNSAFE allow execution to continue (lines 697 to 700). This specifies both the training objective and the intervention behavior used for the main ASR evaluation. We will add these definitions next to Eq. 4.

The confusion matrix contains 200 cases per class, with balanced accuracy of 88.7%. The guard misses 18 of 200 UNSAFE cases (9.0%) and blocks 8 of 200 SAFE cases (4.0%). We also report the 18 POTENTIAL UNSAFE cases predicted as UNSAFE separately. These counts make the observed error types explicit under the uniform correctness reward.

W5: Future prediction versus label shortcuts.

The additional evaluations support learning useful future content. CoAA-RL combines future-summary consistency with adjudication utility in Eq. 6 (lines 250 to 260). The anticipation prompt asks for concrete actions and outcomes and excludes explicit safety verdicts (lines 810 to 815). We examine the resulting summaries through two complementary tests.

  • Event fidelity. Predictions are assessed against observed continuations. Supported-event precision counts supported predictions among all predicted events, while risk-event recall counts covered risks among the observed risk events. VANGUARD obtains 87.0% precision and 81.0% recall, compared with 86.0% and 67.7% for Sim-only, and 73.3% and 71.0% for Util-only. Combining the two rewards increases risk-event recall by 13.3 points over Sim-only while preserving comparable event precision.

  • Content controls. With the adjudicator fixed, within-class summary shuffling raises LPS ASR from 7.5% to 15.6%. Label-only summaries give 16.8%, while removing explicit label terms gives 7.9%. Shuffling breaks the correspondence between a prefix and its summary while preserving the safety class. The resulting degradation shows that instance-specific summary content contributes to the decision.

Removing explicit label terms preserves most of the benefit, while replacing the future content substantially worsens safety. Together with the event-level evaluation, this supports faithful and decision-relevant anticipation. We will include the event counts and matching rules with the fidelity results.

AuthorsOpenReview ↗

Follow-up to Reviewer ibG5

Dear Reviewer ibG5,

Thank you again for your careful review. In our rebuttal, we clarified the training data sources, filtering procedure, and reward definitions. We also added direct measurements of early intervention, benign utility evaluations beyond AgentDojo, and independent tests of summary fidelity and label shortcuts.

Could you let us know whether our response addresses your concerns? We would be happy to clarify any remaining points.

Best regards, Authors of Submission 50365

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Rebuttal and Discussion Phase

Dear PCs, SACs, ACs,

We sincerely appreciate your time and effort in overseeing the review process and evaluating our submission. To facilitate your review, we summarize our main contributions and the additional evidence presented during rebuttal.

We are pleased that the reviewers recognized the importance of anticipating delayed risks in long-horizon agent workflows. They highlighted the coupled anticipation and adjudication design of JANUS, VANGUARD's safety improvements across four benchmarks, and the ablations supporting explicit anticipation and reward coupling.

Following their suggestions, we added direct timing measurements, independent prediction assessments, and controlled baseline comparisons, alongside evaluations of utility, reliability, efficiency, and transfer.

  • A shared concern was direct evidence of early intervention. In prefix replay on 200 delayed-planning cases, with thresholds calibrated to a 5.0% validation false-blocking target, VANGUARD detects 46.0% of cases at least two actions before harm, compared with 18.0% for TS-Guard. Detection coverage increases from 68.0% to 89.0%.

  • Reviewer ibG5 requested clearer data provenance, summary validation, and broader utility evaluation. We clarified that all 75,180 training examples were generated independently of the evaluation benchmarks. Independent event assessment gives 87.0% supported-event precision and 81.0% risk-event recall. Across three additional settings with 100 confirmed-safe counterparts each, benign-task utility remains within 2–3 percentage points of no guard, with lower false-blocking rates than TS-Guard.

  • Reviewer WErd requested stronger evidence on predictive guarding and supervision reliability. Under a shared blocking interface and matched output cap, VANGUARD achieves 8.1% LPS ASR versus 12.6% for SafePred's block-only adaptation. A training-matched direct classifier obtains 17.2%. A human audit of 50 trajectories finds 94.0% label agreement, and independent execution trajectories show gains across all three risk origins.

  • Reviewer CMck requested computational controls, repeated evaluations, and transfer across execution agents. The two-call direct judge reaches 14.0% LPS ASR under the same output cap, versus VANGUARD's 8.1%. In the original configuration, within-class summary shuffling raises LPS ASR from 7.5% to 15.6%, demonstrating the contribution of instance-specific future content. We also report three-run evaluations with uncertainty estimates, measured latency and token usage, and results with two additional execution agents using fixed guard weights, prompts, and intervention rules. Training and validation curves have been provided.

We will incorporate these additional results, training and validation curves, and the corresponding evaluation details into the revised manuscript.

We sincerely appreciate your careful evaluation and valuable suggestions.

Sincerely,

Authors of submission 50365

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.