- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents TRACE, a framework that uses an analysis agent to compare successful and failed trajectories and identify recurring capability gaps. It then generates a dedicated training environment for each selected capability, trains separate LoRA adapters through reinforcement learning, and learns a router to combine them. Experiments show improvements over the evaluated baselines.
Critical strengths
- The paper addresses a useful problem of turning failed agent trajectories into targeted training environments for reinforcement learning.
- The method is clearly described, and the experiments are extensive. It clearly explains the full pipeline from capability discovery and environment generation to adapter training and routing, and is easy to follow.
Critical weaknesses
-
Although the analysis is repeated multiple times, the novelty and robustness of using an analysis agent to discover failure patterns remain unclear. The identified capabilities may be task specific, and their organization is relatively flat. Human review of the discovered capabilities and experiments testing whether they transfer across tasks and domains would make the approach more convincing. A comparison with hierarchical capabilities, such as general capabilities and domain specific capabilities, would also be useful.
-
It is unclear whether TRACE continues to improve when the number of capability adapters increases beyond four, and whether the router can still route effectively when more capabilities are introduced. Under a limited adapter budget, it would also be interesting to consider replacing adapters for capabilities that are no longer useful. A comparison between predefined capabilities and evolving capabilities could help evaluate this possibility.
-
Some details needed to assess reproducibility and efficiency remain unclear, particularly the source and split of the router training data. The budget comparison should also account for environment generation, candidate filtering, and router training rather than relying mainly on rollout counts.
Questions for authors
See Weaknesses.
3 comments
Dear Reviewer eEfN,
Thank you again for your thoughtful feedback. As the rebuttal period draws to a close, we wanted to follow up on our response and highlight two new results relevant to your concerns about capability discovery:
- The controlled ablation shows that contrastive diagnosis improves over failure-only diagnosis by 5.5 percentage points, coming within 2.4 points of the human-written reference but with less time spent.
- The ToolSandbox evaluation provides preliminary evidence of transfer beyond τ²-Bench without retraining. We also clarified the router’s training data and the distinction between adaptation rollout counts and additional pipeline costs. We acknowledge that scaling beyond four adapters and evolving capability sets remain open questions, and will limit our claims accordingly.
If any concern remains unresolved, we would appreciate the opportunity to address it before the discussion ends. If these results and clarifications address your main concerns, we would be grateful if you could consider raising your score in light of the additional evidence.
Thank you for the detailed responses and additional experiments. But my main concerns remain: 1) TRACE only gets one more perfect score than GRPO on a relatively small task set, so the gain is quite modest 2) while I understand capping at 4 adapters as a trade-off, I am still concerned about how subjective the capability division is, and whether we need to redefine their number and scope for new benchmarks. Overall, I would like to maintain my original rating.
Thank you for the suggestions on human validation, transfer, and how TRACE might work with a larger capability set.
W1: Capability diagnosis, human review, and transfer
TRACE turns recurring differences between successful and failed trajectories into executable training environments. We use a fixed capability dictionary across labeling runs and retain capabilities that consistently meet the coverage and contrastive-gap thresholds. For the ablation, we synthesize 4 capability environments per method on τ²-Bench (matching TRACE) and hold the rest of the pipeline fixed. Each method supplies capabilities at the same granularity as TRACE, a description plus example trajectories, so the comparison isolates the source of the capabilities.
τ²-Bench (Pass Rate)
Contrastive diagnosis improves the overall pass rate by 5.5 percentage points over failure-only diagnosis and comes within 2.4 points of the human-written reference. This auxiliary ablation uses task-level classifier routing. The main paper’s 48.2% overall pass rate uses the trained token-level MoE.
For the human-written reference, we spent about 5–8 hours reviewing all the rollouts used for diagnosis and identifying capability targets. The analysis agents completed this step substantially faster.
TRACE selected the capability targets automatically in the main experiments. We reviewed the generated environments for reuse of original scenarios and checked verifier decisions. Two authors independently audited 50 rollouts per environment and agreed with the verifier’s binary success judgment on all 200 audited rollouts. Measuring agreement between human and model capability labels would require a separate audit.
We evaluate agents adapted for τ²-Bench on ToolSandbox without retraining:
This comparison provides preliminary evidence of transfer to a different tool-use environment. We agree that comparing a flat capability set with a hierarchy of shared and domain-specific capabilities would help establish when that transfer occurs. The current experiments use a flat set.
W2: Adapter scaling and replacement
Our τ²-Bench scaling experiment covers up to four adapters. The submitted router selects one adapter per token, and its choice can change across an interaction. This keeps one capability adapter active per token, while a larger adapter bank would still increase storage and expand the choices the gate must learn to make.
The current experiments keep the chosen capability set fixed. The behavior of the router beyond four adapters and the value of replacing adapters as new deficits emerge remain open questions. We will scope the scaling claim to the range we evaluated.
Furthermore, given the limited task pools available for diagnosis in τ²-Bench and SWE-bench Verified, expanding the capability set through increasingly fine-grained subdivisions could produce capabilities supported by only a few tasks. We aimed to avoid this fragmentation and focus on reusable capabilities. This motivates our choice of capability granularity.
W3: Router data and efficiency
We train the router on all trajectories collected for adapter training in the synthetic environments, including successful and unsuccessful rollouts. This pool contains no original benchmark trajectories. We freeze the base model and adapters and optimize only the token-level gates with response-token cross-entropy and a load-balancing penalty. Reusing the synthetic pool requires no additional environment rollouts for router-data collection.
The 5,120 rollouts count adapter training across all capabilities. The approximately 2,500 episodes per environment in Appendix C belong to a separate run checking diversity and overlap. We will correct the wording in Appendix C and distinguish the adaptation rollout comparison from the additional costs of discovery, environment generation, candidate filtering, validation, and router optimization.
Should any important concern remain, we would be grateful if the reviewer could bring it to our attention during the discussion period. We would be happy to incorporate these clarifications and results into the paper and submit a revised PDF. If our response addresses the reviewer’s concerns, we would appreciate it if the reviewer would consider raising their score in light of the additional evidence.