- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces HazardAuditor, an execution-grounded safety framework that normalizes diverse execution logs from heterogeneous computer-use agents into a unified behavioral representation to detect runtime risks. To prevent variable-length explanations from dominating model training, the authors propose Guard Policy Optimization (GuardPO), which normalizes the rationale and verdict regions to directly optimize the sequence-level safety decision.
Critical strengths
-
Comprehensive experiments with consistent performance improvement: The paper evaluates HazardAuditor across five distinct benchmarks (CUA-EXEC, AgentHazard, ATBench, R-Judge, and ASSE-Safety) and four heterogeneous agent frameworks (Claude Code, Codex, Hermes, and OpenClaw). Across these diverse settings, the proposed framework demonstrates robust and consistent gains, notably improving accuracy by up to 16.5 percentage points over the strongest prior guard model on the execution-grounded diagnostic.
-
Sufficient baseline comparison with the latest model variants: The authors establish a rigorous evaluation setting by comparing against a comprehensive suite of various types of models. This includes new decision models such as Jev and general-purpose safety guards (e.g., Llama Guard 3, Qwen3Guard), and recent trajectory-aware agent guardrails (e.g., AgentDoG, ShieldAgent, and BraveGuard),
Critical weaknesses
-
Incremental Novelty of Guard Policy Optimization (GuardPO): While GuardPO effectively mitigates the rationale length-bias issue in generative guards, its core algorithmic novelty is highly incremental compared to established policy optimization formulations. The method fundamentally relies on the existing Clipped Importance Sampling Policy Optimization (CISPO) framework to handle token-level policy clipping and sequence-level advantage broadcasting. The primary structural modification is restricted to applying a separate length-normalization across the rationale and verdict regions before loss aggregation. From a policy gradient derivation perspective, this acts as a domain-specific masking and weighted averaging heuristic for the loss implementation rather than a fundamental theoretical advancement in reinforcement learning.
-
Simplistic Reward Formulation and Advantage Allocation: The sequence-level advantage calculation relies on a highly simplistic, deterministic reward structure (+1 for correct, -1 for incorrect, -1.25 for malformed) based purely on the terminal verdict. Because this single sequence advantage is uniformly broadcast to every token in the response, the optimization process does not explicitly evaluate or shape the quality, logical soundness, or factual grounding of the intermediate reasoning steps. Exploring relative advantage calculations such as comparing multiple reasoning trajectories against one another for the same prompt or incorporating dense rewards would have provided a much more robust and rigorous validation of the proposed policy optimization approach.
Questions for authors
Do you have any ablation on the reward design?
6 comments
rebuttal2
Q3: Beyond GuardPO, what is the substantive contribution of the execution infrastructure?
A3: HazardAuditor combines two complementary contributions: execution-grounded supervision and decision-aligned optimization. The infrastructure goes beyond format conversion by establishing an adaptive red-teaming and executable verification pipeline.
Safety-aligned agents frequently refuse static harmful requests. Our Red Agent therefore follows an observe–adapt–act loop, reformulating requests, decomposing objectives, escalating gradually, and changing injection points in response to refusals. We evaluate benign operation, direct attacks through user messages, and indirect attacks additionally injected through tool observations.
Table 3: Adaptive red-teaming coverage and attack success rates.
| Evaluation Condition | Attack Surface | Verified ASR |
|---|---|---|
| Benign operation | No adversarial intervention | N/A |
| Direct attack | Adaptive user-message injection | 90.6% |
| Direct + indirect attack | User messages and injected tool observations | 93.9% |
| Increment from indirect surface | Additional compromised observations | +3.3 pp |
Representative attack strategies achieve ASRs of 88.5% for task decomposition, 87.9% for obfuscation, 85.5% for instruction injection, and 83.7% for jailbreaks. ASR measures attack success against target agents, not HazardAuditor's detection performance.
Importantly, attack generation and outcome verification are separated. Executable verifiers inspect MCP records and sandbox state rather than relying on LLM judgments alone. They reject 7.30% of Control-LLM-positive outcomes, and 99.88% of verifier programs reference execution logs or environment-state evidence. Behavioral labels additionally capture unsafe attempts even when the intended environmental effect is not achieved, consistent with our original labeling protocol.
The infrastructure covers 126 risk categories, 77 attack techniques, 30 environment categories, and 71 implemented tool APIs across four heterogeneous agent frameworks. Thus, its contribution lies in systematically generating adaptive attacks and producing independently verifiable, execution-grounded behavioral supervision, rather than merely unifying existing logs.
We sincerely thank the reviewer again for the constructive feedback. We hope the additional ablations clarify the specific contributions of GuardPO, while the infrastructure analysis demonstrates the broader methodological contribution of HazardAuditor. We would be happy to address any remaining questions and provide further clarification. If these concerns have been adequately resolved, we respectfully hope the reviewer might reconsider the overall assessment of our submission.
We greatly appreciate your time and constructive feedback. Should you have any questions regarding our rebuttals, please feel free to contact us, we would be more than happy to discuss them with you and make further revisions as needed.
Rebuttal 1
We sincerely thank the reviewer for the thoughtful assessment and for recognizing the execution-grounded motivation, comprehensive evaluation, and decision-level optimization of HazardAuditor. We appreciate the insightful comments on the novelty of GuardPO and its reward formulation. We provide the following clarifications and additional experimental evidence.
Q1: GuardPO appears algorithmically incremental over CISPO, with its primary modification being region-wise loss normalization. How does it establish a distinct contribution?
A1: We agree that GuardPO adopts the clipped importance-weighting principle of CISPO for token-level policy updates. Our contribution is a decision-aligned optimization objective addressing a structural mismatch in generative safety guards. Each response represents one safety decision, whereas conventional token-level aggregation allows longer rationales to exert disproportionate influence.
GuardPO addresses this through sequence-level outcome advantages and separate response-level normalization of rationale and verdict losses. To isolate these contributions, we compare standard GRPO, which uses group-relative advantages and conventional token-level aggregation, against GuardPO and its normalization ablations under the same SFT initialization and training budget.
Table 1: GuardPO ablation on CUA-EXEC.
Standard GRPO improves over SFT by only 0.77 percentage points, whereas GuardPO improves over GRPO by 9.61 points. Removing rationale normalization reduces accuracy by 3.26 points, while removing verdict normalization leads to a 9.55-point reduction. These results demonstrate that the gains are not simply attributable to reinforcement learning, but arise substantially from decision-aligned loss aggregation.
Q2: The reward formulation is simplistic, and the sequence-level advantage is uniformly broadcast to all response tokens. Do you have any ablation on the reward design? Why not use dense rewards or relative advantages?
A2: Our reward design follows the operational objective of a safety guard. Unlike an agent policy that optimizes sequential environment interactions, HazardAuditor performs a classification task and must produce a correct binary safety verdict. We therefore use deterministic terminal rewards rather than introducing additional learned reward models or LLM judges.
Independently verifiable process-level reward for individual reasoning steps are not available. Dense rewards would require such supervision and could incentivize plausible-sounding explanations without improving final decision correctness. We instead use SFT to establish evidence-grounded reasoning behavior and GuardPO to directly optimize the terminal safety decision.
We further examine reward sensitivity by comparing alternative deterministic reward assignments under the same optimization objective.
Table 2: Reward-design ablation on CUA-EXEC.
Removing the malformed-output penalty reduces accuracy by 6.62 points. Treating malformed outputs as ordinary errors achieves 89.91%, while increasing the penalty to -1.5 achieves 90.35%. Our original setting performs best among the evaluated variants, suggesting that a moderate format penalty provides a more effective balance.
Regarding relative advantages, Table 1 already provides a direct comparison with standard GRPO, which computes group-relative advantages across multiple rollouts for the same prompt. Its substantially lower accuracy further supports our decision-level optimization design. Uniformly broadcasting the sequence advantage is deliberate because the reward evaluates one complete safety decision, not independently verifiable reasoning tokens. Region-level normalization prevents rationale length from determining each response's optimization weight.