- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper studies credit assignment for tool agents when both the assistant and the user can act on the environment. It measures progress through state-field matches against targets obtained from reference actions, separating assistant-side and user-side progress. Delegated Credit Estimation (DCE) further discounts user-side credit when the same progress recurs after replacing the assistant's message with a neutral acknowledgement. These signals supplement terminal GRPO rewards. Experiments across multiple domains evaluate policy performance, with additional replay analyses of the credit correction.
Critical strengths
-
User-side execution is a practical aspect of tool use. Crediting useful user actions and asking whether they depend on the current assistant instruction is a sensible problem to study. The neutral-message replay design is intuitive.
-
The paper includes useful controls for user-side supervision and validates its recurrence estimates using separate replays. These analyses provide evidence that contextual DCE identifies progress that also occurs under a neutral message.
Critical weaknesses
-
The empirical contribution is limited to in-domain benchmark optimization. The training data come from the same benchmark task framework, with part of the Retail data synthesized from failed AReaL training trajectories. For Telecom, the authors generate new customer profiles but instantiate tasks from the official non-test collection. It is unclear how much training and evaluation overlap in problem types, failure patterns, and their combinations. This leaves a concern that the gains largely reflect adaptation to familiar task patterns. The baseline already has high single-trial success, while DCE's additional gains over factual delegated supervision are modest and confined to repeated-trial success. I do not find these results sufficient to establish a significant contribution.
-
The process reward is narrowly suited to tasks with explicit reference target states. Exact field matching fits Telecom configuration tasks, but it is unclear how it would provide meaningful progress signals for numerical goals where partial progress matters, tasks with multiple valid end states, or artifact creation. The central user-side credit mechanism is also tested only in Telecom; the other domains do not provide user-side state rewards. The problem is practical, but the design and evidence do not establish applicability to user-side execution more generally.
-
Online Contextual DCE adds substantial extra user-turn simulation, but its benefit under a matched compute budget is not established. The comparisons use the same training steps rather than the same total computation, and replay counts are not translated into token or runtime costs. It is unclear whether spending that budget on more ordinary rollouts or longer training would perform as well or better. Given the modest additional policy gain, this comparison matters for judging whether DCE is worthwhile.
Questions for authors
-
How much do Telecom training and evaluation tasks overlap in problem types and failure-pattern combinations? Is there evidence of transfer to held-out task patterns beyond changing customer profiles?
-
What evidence supports the reward design beyond tasks with an explicit reference target state, such as tasks with multiple valid outcomes or artifact outputs?
-
What is the additional token or runtime cost of Online Contextual DCE, and how does it compare with spending the same budget on more ordinary rollouts or longer training?
Author Response: Task Transfer, Method Scope, and Computational Cost
We thank the reviewer for recognizing the importance of user-side execution and for the questions on transfer, scope, and computational cost. Factual denotes Direct + Delegated. Page references are to the revised manuscript.
Q1: Training–evaluation overlap and transfer to unseen patterns
We audited all 1,300 Telecom training instances against the 114 evaluation tasks. Each task pattern pairs a problem type with its fault set, excluding customer identity and persona.
110/114 tasks (96.5%) have unseen fault combinations. The four shared fault sets occur under different problem types. Training excludes the official base evaluation split and uses new customer profiles; the audit establishes separation beyond profile changes.
G2 contains 81 tasks with unseen combinations of familiar faults within known problem types. G4 contains 29 service_issue tasks, a problem type absent from training. Scores are means over three evaluations of each checkpoint (%):
Factual and Online improve Pass⁵ over Terminal by 8.7 and 7.4 points on unseen combinations. The overall +2.1-point DCE gain combines different group effects: Factual leads on G2, while DCE gains +2.5 Pass¹ and +11.5 Pass⁵ points over Factual on the unseen G4 problem type. These results show transfer to unseen structures within the same tool environment (§5.4, pp. 9–10; Appendix G, pp. 28–30).
Q2: Scope of progress measurement and user-side credit
Our method separates role-specific progress measurement from recurrence-adjusted user-side credit. Reference-field matching supplies the progress score in our implementation, allowing different action sequences to reach the same target. DCE tests whether an observed user-side progress event recurs after replacing the current assistant message, using restorable context and a common event criterion across branches.
Telecom provides the benchmark's user-side tools that modify environment state. Airline and Retail have no mutable user-side component (§3.1, pp. 3–4). This determines where delegated credit can be evaluated in tau2-bench. On Telecom, adding delegated supervision raises Pass⁵ from 80.7 to 82.7 with direct-progress and risk settings fixed.
Task-specific scoring extensions include:
Extending DCE also requires defining observable user-side progress events. For artifacts, an event could be satisfaction of a functional requirement across different valid outputs.
ProCredit demonstrates acceptance-check progress rewards on AppWorld and ToolSandbox; FACA learns from user reactions and transfers to shared-workspace tasks. These provide examples of progress and user-feedback supervision (§2, pp. 2–3; Appendix F.5, p. 28).
The trained DCE policy also transfers to a second user simulator: +5.0 Pass⁵ points over Factual with Qwen3.8-27B, without retraining (Table 20, p. 32).
Q3: Replay cost and additional-training control
With the same ten actor/rollout GPUs, the aligned rollout-0–119 window takes 56.60 h for Online versus 46.84/46.01 h for Terminal Partial-State/Matched Global-Discount (119 intervals each): +9.76–10.59 h (+20.8–23.0%). These no-replay runs calibrate the budget because the historical Factual run shared its simulator endpoint with other workloads.
We continued Factual for 30 updates (120→150) under its original training setup. This took 11.38 h, exceeding the calibrated overhead. Evaluation uses the original Telecom protocol. Mean ± evaluation SD (%):
Online DCE exceeds continued Factual on all five metrics (Pass⁵ 84.8 vs 82.2). Thus, 30 additional updates did not recover its gain at the calibrated runtime budget (Appendix I, pp. 33–34; Table 23, p. 34).