- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper trains a step-level safety monitor that judges each proposed agent action before it executes, with the goal of intervening not only correctly but in time. The key observation is that existing labels say whether a trajectory or step is unsafe, but not when the risk first becomes visible or until when intervention can still prevent harm. The authors address this by annotating every step with a four-axis violation taxonomy and an intervention window (signal s, boundary b, last preventable step h), and extending the corpus through multi-agent red-teaming with matched benign twins. They then propose ASPO, which combines stage-aligned policy optimization (SAPO) — scoring each decision by its stage, comparing it only against responses sampled at the same prefix, and balancing safe and intervention steps — with adaptive feedback self-distillation (AFSD), where the model acts as its own teacher using step annotations on intervention steps and rules mined from its own false alarms on safe steps. The resulting monitor, STEPSAFE, achieves J = 50.6 (4B) and 53.2 (8B) across five agent-safety benchmarks, outperforming open-source guardrails such as StepGuard (46.2).
Critical strengths
The reformulation from "whether an action is unsafe" to "when intervention becomes warranted" is well motivated, and the intervention window (s, b, h) is a clean and reusable formalization of timeliness.
SAPO's prefix-local credit assignment directly targets known failure modes of trajectory-level and naive step-level rewards, and the ablations (Table 3) isolate the contribution of each design choice convincingly.
The false-alarm rubric is an elegant answer to the lack of supervision on safe steps: rules mined from the monitor's own false alarms, admitted only if they clear no known violations, including the unsafe counterparts of matched benign twins.
The evaluation is fairly comprehensive: five benchmarks, eight published monitors plus API models, and ablations covering credit estimators, feedback components, model scale, seed variance, and operating points.
Critical weaknesses
The training annotations rely largely on GPT-5.6 Luna, and no human verification of label accuracy is reported. This may introduce annotator bias into training. To my knowledge, some of the safe/unsafe labels in the training data are inaccurate, so human verification would be important.
The paper does not include a case study. It is difficult for readers to see how StepSafe operates — how it prevents harm and provides early warnings. Some good examples would make the method much easier to follow.
Figure 3 may be somewhat inaccurate or biased, because the benchmarks authors utilize havde different accuracy and quality. I would suggest drawing the Pareto frontier separately for each benchmark. The ASSE benchmark in particular has apparent quality issues: in ASSE-Security, many agent responses appear to be API error messages such as "OpenAI STATUS error 500: (e.response)", while ATBench and R-Judge look considerably cleaner.
Figures 1 and 2 are somewhat hard to follow and generated by AI. I would suggest redrawing them in a revised version to improve readability.
Questions for authors
The raw data formats of these benchmarks appear to differ considerably. Could the authors clarify how the trajectories from different benchmarks were converted into a unified format, and how the evaluation was kept consistent across them?
Thank you for your response. My main reason for raising the third weakness was that inaccuracies in the benchmark could introduce substantial measurement error, making it difficult to draw reliable conclusions about model performance from the benchmark scores. Overall, most of my concerns have been addressed, I have raised my score to 3.