- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents AACE, which checks agent actions before execution and learns new constraints from failures. Proposed constraints are tested for violations and unnecessary blocking before being added to a persistent rule bank. Experiments in negotiation and procurement show fewer observed violations, although improved valid task completion is not statistically established in the main constraint-learning experiments.
Critical strengths
The paper addresses a practical problem: adding safety rules can also block legitimate actions. In Table 2, the validated Bank reduces Qwen attack violations from 57.0% to 31.1%, with no observed false blocks on benign episodes. Adding all candidates instead causes 13.3% benign false blocks. This provides useful evidence for checking rules before deployment.
The evaluation separates action enforcement from recovery after rejection. The recovery ablation and matched ToolGuard comparison make this distinction concrete. I also appreciate the explicit reporting of unsuccessful candidates, uncertainty intervals, and the lack of established gains in valid task completion.
Critical weaknesses
The experiments do not isolate the benefit of the proposed constraint-selection procedure. The main comparison is between three validated rules and all 26 candidates, which changes both selection and Bank size. A simpler selection method retaining three rules could potentially achieve similar results. The Curation Gate accepts every candidate that reaches it, so its additional value is also untested. ToolGuard is a useful static baseline, but it does not address the adaptive contribution; the distinction from adaptive guardrails such as AGrail (Luo et al., ACL 2025) also needs discussion.
The evidence for constraint learning remains narrow and dependent on the semantic evaluator. One learning run produces three privacy rules, with no admitted role or time-wasting rules. Both the gate and judge use Qwen-plus, including when Claude executes the task. Blinding the judge helps, but does not rule out shared semantic errors; Section 4.2 already reports an inconsistent judgment in a case with no intervention. An independent audit is needed to establish how much the main results depend on these labels. RetailBench provides independent deterministic evaluation, but only for one duplicate-procurement pattern.
Questions for authors
What evidence distinguishes the verification procedure from simply retaining a small Bank? A comparison with a simple selection baseline constrained to three rules would directly affect my assessment of the contribution.
Do the violation and false-block reductions remain under an independent human audit or a separately validated evaluator? Please include safe steps blocked within attack dialogues, not only benign episodes.
What does the Curation Gate contribute beyond empirical verification in the evaluated setting, and how does the update procedure differ from existing adaptive guardrails such as AGrail?