HorizonGuard: Runtime Risk Forecasting for LLM Agents

LLM agents increasingly execute long-horizon tool-use trajectories, where safety risks can emerge from user instructions, environment observations, or the agent's own decisions. Existing guard models typically assess isolated prompts, final responses, or completed trajectories, making them ill-suited to intervene before an unsafe action is committed. We introduce HorizonGuard, a runtime risk forecasting model that conditions on the task and the observed trajectory prefix to predict a monotonic cumulative risk curve over the next steps. This formulation naturally enables a sequential first-trigger intervention policy while exposing the forecast horizon and decision threshold as deployment-time controls, allowing a single model to flexibly trade off early warning against false alarms without retraining. To train HorizonGuard, we construct HorizonRisk, a step-level dataset generated through controlled risk injection into real agent trajectories. HorizonRisk covers diverse risk sources, risk types, and execution states, with annotations for both the first risk exposure and the eventual unsafe commitment. We further introduce a runtime-oriented sequential evaluation protocol that measures intervention quality throughout execution rather than after trajectory completion. Experiments across multiple agent benchmarks demonstrate accurate runtime risk forecasting under both loose and strict labeling criteria, robust generalization across diverse trajectory states, and effective adaptation to different deployment risk preferences. These results establish HorizonGuard as a configurable runtime guardrail for proactively detecting and interrupting unsafe agent behavior, advancing agent safety from retrospective trajectory classification to forward-looking runtime risk forecasting.