- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
1.The authors propose DARP, which turns structured denial feedback into typed alternatives, compares their recoverable value within the legal set, and limits capability use among near-maximal choices. State-bound certification carries the selected action through dispatch before the agent replans from the observed return.
Critical strengths
1.The results are compeling: DARP reduces mean capability exposure from 10.20 to 7.44. Replacing exact values with a frozen Qwen scorer yields 90.94% safe resolution and zero unauthorized dispatch.
2.The studied topic is valuable and offers real-world impact.
3.Metrics are multi-facet and evaluates the phenomenon well.
Critical weaknesses
1.The authors can better explain the terms used in Section3 Formulas, e.g., write the full names of each term in Formula 1. Also, Cap(a) is the semantic capability needed to execute a. So why not define it exe(a)?
2.The Figure 1looks a bit junior, with much important information omitted.
3.The details about benchmark construction are largely omitted, e.g., how many authors participate in the annotation?
4.Some parts of writing is hard to understand. e.g., line 186-187.
5.For the correct stop rate, authors can record agents stop on which steps, which can help better measure token wastes.
Questions for authors
See above.
Response to Reviewer T44e
Thank you for your review.
Q1. ... Formula 1 ... Cap(a) ...
Answer: The revised Eq. (1) names all six fields: operation, arguments, preconditions, effects, required semantic capabilities, and requested provider scopes. Cap(a) is the authority an action requires. Exec runs the action and returns its observation. For a permission request, Req(a) records the scopes being sought.
Q2. ... Figure 1 ... information ...
Answer: The redrawn Figure 1 shows the six stages of the recovery loop and the objects passed between them. It highlights how equal value and lower exposure determine at∗ in a recorded Gmail decision, then follows that action through verification, dispatch, and replanning. Table 14 gives the legality, value, and exposure of each candidate.
Q3. ... benchmark construction ... authors ... annotation ...
Answer: The revision identifies two authors who constructed or reviewed the templates, gold labels, and evaluation rules in Section 4 and Appendix I (Benchmark construction and example, p. 17). Appendix C specifies that two authors independently annotated each of the same 80 blinded cases, with randomized candidate order and with method identities, selected actions, scores, and evaluator labels withheld.
Q4. ... hard to understand ...
Answer: The revised typed-recovery paragraph after Eq. (9) states the candidate processing rules. The schema validates the arguments, and equivalent candidates are merged by a semantic fingerprint that omits the candidate ID. A non-repeatable action is retained if a modeled outcome changes a fact, adds a capability, or terminates the episode. Safe stop is registered explicitly.
Q5. ... correct stop rate ... steps ...
Answer: Table 23 reports correct stops, the post-denial step, and premature stops on recoverable tasks. Exact-value and Qwen-estimated DARP correctly stop at the first decision on all 40 unrecoverable tasks. Self-Revision and Direct Retry make their correct stops at the same step, on 37 and 38 of the 40 tasks. Appendix I (Stopping behavior, p. 22) reports 775.45 incremental scoring tokens per unrecoverable task for Qwen-estimated DARP with cached proposals, separately from the table. The stopping analysis makes no end-to-end efficiency comparison.
Q6. ... See above ...
Answer: The revised Section 3, Figure 1, benchmark and human-audit protocols, and stopping analysis address Q1–Q5 above. Is there a specific assumption or step in the soundness argument that needs further clarification?