- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies multi-step safety risks in tool-using LLM agents, where the safety of an action depends on previous execution history. The authors propose TRACE-GUARD, a stateful runtime guard that maintains a compact safety state and evaluates candidate state transitions before tool execution. They further introduce RMSBench, an executable benchmark for evaluating runtime safety with transition-level evidence. Experiments show that TRACE-GUARD improves over action-local defenses and can distinguish safe and unsafe cases with identical final actions but different histories.
Critical strengths
- Clear formulation of history-dependent runtime safety. The paper identifies an important limitation of action-local safety checking and formulates multi-step agent safety as a state-dependent runtime enforcement problem. The proposed state-transition view provides an interpretable mechanism for tracking accumulated safety-relevant conditions and making pre-dispatch decisions.
- Strong controlled evaluation for analyzing the proposed mechanism. The introduced controlled-history experiments are well designed: they keep the final action unchanged while varying only the preceding trajectory, directly testing whether the guard can capture history-dependent risks. The transition-level evidence collected in RMSBench also provides more diagnostic analysis than only reporting final task success or failure.
Critical weaknesses
- Limited evidence for generalization beyond manually specified safety policies. The proposed approach relies on a manually designed abstraction of safety-relevant states and transitions. While this design is effective for capturing predefined multi-step risks, it remains unclear how well the approach scales to open-ended agent environments with diverse and evolving safety requirements. In particular, the current evaluation focuses on a fixed rulebook and benchmark-specific scenarios, where the safety states and transition rules can be explicitly specified. The broader claims in the abstract and introduction appear somewhat general, but the paper does not provide sufficient evidence that the proposed state-transition framework can generalize beyond such controlled policy spaces. It would be valuable to evaluate how much effort is required to extend the rulebook to unseen environments or policies, and whether the approach remains effective under such settings.
- Missing comparison with stronger history-aware runtime guards. The experiments convincingly show that maintaining safety state is better than purely action-local checking. However, this comparison does not fully establish the advantage of the proposed structured state representation over other history-aware approaches. For example, a strong baseline that directly conditions on the full execution trajectory (or a learned trajectory summarization mechanism) could potentially capture similar history-dependent risks without requiring manually specified state abstractions. Since the main contribution is not simply that "history matters", but rather that explicit structured state transitions are preferable, a comparison against such trajectory-aware guards would be important to validate the proposed design choice.
Questions for authors
- Evaluation on existing multi-step safety benchmarks. Given the close connection between TRACE-GUARD and existing multi-step tool-use safety benchmarks such as STAC, why did the authors not evaluate TRACE-GUARD on these established benchmarks?
- Scalability of state abstraction. The current implementation uses a finite set of manually designed safety states and transitions. How do the authors envision constructing and maintaining such abstractions for large-scale open-world agents with many tools, resources, and interacting objects?
Thank you for recognizing the formulation of history-dependent runtime safety and the controlled evaluation that fixes the final action while varying prior history. Your comments raise two important questions: what explicit state contributes, and how policy rules can scale.
On explicit state and history-aware guards.
Our starting point is that access to history and consistent enforcement of safety conditions are different properties. Under No Defense, the agent processes task history through its native context mechanism, yet ASR remains high. Making history available for task execution therefore does not necessarily make cross-step safety conditions stable decision constraints.
TRACE-GUARD represents these conditions as explicit policy state and checks candidate transitions before dispatch. Across 30 controlled pairs, the final action and deterministic predicate are fixed while the executed prefix changes. The stateful guard correctly distinguishes all 30 pairs; the stateless guard has no pair with both sides correct. Disabling semantic translation allows this experiment to directly test state-conditioned rule selection.
The explicit representation also makes it possible to inspect which prior condition a decision depends on, which rule matched, and which successor state was considered. For example, an archive precondition before deletion can be expressed as an inspectable cross-step policy. This is a concrete design property demonstrated by the implementation and experiments.
A dedicated LLM guard that assesses the full history is a meaningful comparator. No Defense motivates the problem, while the controlled-history experiment tests the state mechanism; neither substitutes for a direct comparison with that guard.
Q1. How should RMSBench be understood relative to existing multi-step safety benchmarks?
RMSBench is designed to connect tasks with execution environments, task-specific before/after checks, and runtime decision records. This allows us to answer both whether the specified task criterion was satisfied and whether the monitor explicitly intervened.
The execution and recording protocol distinguishes harmful-task non-completion, explicit action blocking, and approved actions that fail to produce the target outcome. The reported results should be interpreted under that protocol and its task criteria, including the corresponding scope of proxy or incomplete checks. This evaluation should not be treated as an independent result under an original external benchmark’s protocol.
Q2. How can manually specified states and rules scale to open environments?
Our subsequent research direction has two connected components.
The first is rule templating. Rules can be organized around operation categories, resource attributes, preconditions, and risk transitions, allowing different tools to share policy semantics. For example, deletion of files, database entries, and cloud objects could share checks on target identity, authorization scope, and protective preconditions while using different tool mappings. Each mapping would still require validation against the environment’s semantics.
The second is a self-evolving rule module. New trajectories could reveal conditions missing from existing templates and suggest candidate rules. Independent validation, benign-task regression checks, and versioned updates would determine which candidates enter the rulebook. This offers a direction for reducing manual maintenance while preserving rule provenance and inspectable decisions.
Both directions benefit from resource-scoped state and execution feedback. Refining “archiving occurred” into “the archive of object A was successfully completed,” for example, would better support interleaved operations on multiple objects. The present work provides an implementation and controlled mechanism evidence under fixed policies; template transfer, self-evolving updates, and maintenance costs are subjects for subsequent research.