acceptodds
Under review as a conference paper at ICLR 2027

Prediction Is Not All You Need: TraceGuard for Action-Oriented Safety Evaluation of Long-Horizon Agents

Abstract

Tool-using agents can reach unsafe states through long-horizon trajectories in which no single early action is obviously harmful. A natural response is to predict failures before they occur. However, a warning is only useful while an effective intervention remains available and meaningful task progress can still be recovered. We introduce TraceGuard, a verifier-grounded framework for action-oriened safety evaluation. We first examine whether pre-failure trajectory signals yield deployable early warning. Although development-time analyses reveal apparent separability several decisions before failure, a frozen predictor does not generalize to held-out trajectories. This exposes a distinction between predictability, intervention actionability, and recoverability, and motivates a different question: what can still be done effectively? TraceGuard evaluates intervention and recovery frontiers relative to the first verifier-confirmed violation. Across more than 300 collected security trajectories, causal intervention experiments reveal strongly action-dependent safety windows: terminating execution remains effective substantially earlier than blocking only the current action, while various interventions incur different results for task utility. Recovery experiments further show that checkpoint rollback preserves substantially more valid work than full reset, whereas finer localized rollback does not consistently outperform restoration to the last safe checkpoint. These results show that predictive performance alone does not characterize an agent's usable safety margin. TraceGuard instead characterizes the safety, utility, and recovery options available as execution unfolds, providing safety profiles that can inform monitoring policies, tool permissions, and agent configuration.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.