- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes EdgeGuard-Agent, a framework that treats agent actions as state-transition edges to be verified, repaired, or blocked. It distinguishes missing user information from facts recoverable through tools and checks grounding, execution conditions, and safety before acting. The same auditor also supports trajectory repair and policy learning through SFT and GRPO. Evaluation covers three stateful tool-use benchmarks, with additional component ablations and cross-model comparisons.
Critical strengths
The shared formulation for runtime protection and training supervision is well motivated. Distinguishing user-owned information from environment-recoverable facts addresses a practical problem: asking more questions does not necessarily resolve uncertainty correctly. Local repair preserves valid earlier decisions, while re-executing repaired actions avoids inconsistent downstream observations in training data. These choices form a coherent design, supported by improvements in task success and write safety across the evaluated environments.
Critical weaknesses
The policy-learning experiments do not isolate the contribution of EdgeGuard-Agent supervision. Table 4 compares an untrained model with EdgeGuard-Agent-curated SFT and subsequent GRPO, but lacks standard SFT and task-reward-only GRPO controls under matched training budgets. There is also no training ablation separating trajectory filtering, causal repair, and edge-level rewards. Consequently, the reported improvement from 27.69% to 40.42% demonstrates that the full training pipeline helps, but does not establish how much comes from the proposed supervision rather than additional training. This leaves a central contribution less convincingly validated than the runtime method.
Questions for authors
-
Do you have comparisons with standard SFT and task-reward-only GRPO using the same initialization and comparable data and training budgets?
-
Can you isolate the effects of causal trajectory repair and edge-level rewards in the policy-learning results?
Follow-up on Our Latest Response and Remaining Questions
Thank you again for the time and care you have devoted to reviewing our paper and engaging in this discussion. We appreciate your willingness to revisit the evidence, and your detailed questions have helped us make the experimental support much clearer.
We would like to briefly follow up on our 18 November morning response. We clarified how the original training settings were checked against server configurations and checkpoint parameters, and reported three-seed results for all training methods, including the full methods. Causal replay and edge rewards improve over their respective standard controls by 3.73 and 2.84 TSR points on average, with positive differences at every seed. The new paired intervals also support the improvement over SABER (+9.33 points; 95% CI [1.4, 17.0]) and the contribution of local repair (+12.27; [4.2, 20.0]). We additionally provided the scenario breakdown, exact corrected-benchmark version, and annotator agreement you requested.
We understand that the contribution itself remains important to your assessment. The scenario results locate the largest gains in missing-capability and missing-evidence tasks, while the training controls provide evidence for causal replay and edge supervision beyond ordinary training. We hope these findings help clarify the value of the shared runtime-and-learning formulation.
Could you let us know whether this latest response resolves your remaining concerns, or whether there is any specific issue we can still clarify? If the additional evidence has addressed those concerns, would you consider whether a more favorable score is now warranted?
Thank you once again for your sustained, careful reading and constructive suggestions. We have learned a great deal from this exchange and sincerely appreciate your effort.