acceptodds
Under review as a conference paper at ICLR 2027

BehaviorPilot: Behavior-Guided Data Synthesis for Tool-Using Agents

Abstract

Training tool-using agents requires diverse interaction data and reliable execution feedback. Recent approaches adopt scenario-centric synthesis, generating environments and tasks across domains and tool configurations. However, scenario diversity does not ensure broad behavioral coverage: tasks from different scenarios may repeatedly exercise the same interaction patterns while leaving others underrepresented. To enable explicit control over behavioral coverage, we introduce tool-use behaviors as a representation of recurring interaction patterns and propose BehaviorPilot, a behavior-guided data synthesis framework. BehaviorPilot integrates three modules: behavior taxonomy induction to build a hierarchical behavioral representation from open-source trajectories for task synthesis and model diagnosis; behavior-conditioned data synthesis to generate executable tasks targeting selected behavior combinations and retain rollouts that pass task verification and cover the target behaviors; and iterative behavior-guided refinement to target model weaknesses by allocating additional synthesis budget in proportion to per-behavior failure rates on a fixed, behavior-balanced validation set. Applied to Qwen3-8B, BehaviorPilot achieves the best benchmark-average scores among the evaluated baselines on VitaBench, BFCL V4 multi-turn, and \tau^2-Bench. Notably, the resulting 8B model surpasses Qwen3-32B in overall performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Tool-Using Agents, Synthetic Training Data, Behavior-Guided Data Synthesis

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_XN7pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents BehaviorPilot, which generates training tasks for tool-using agents around an explicit behavior taxonomy and uses failure rates to guide further synthesis. Experiments on three benchmarks show that the trained model outperforms the evaluated baselines.

Critical strengths
  1. The paper addresses a practical limitation of scenario-based synthesis. Explicit behavior targets help reduce repetitive interaction patterns and improve coverage of underrepresented skills. The results support the value of behavior-guided synthesis. BehaviorPilot leads on all three benchmark averages, and its 4k-task dataset outperforms a 16k-task behavior-agnostic dataset
  2. The synthesis pipeline includes useful quality checks. Tasks and trajectories undergo execution and behavior-coverage checks, supported by a separate QA process and concrete verifier examples.
Critical weaknesses
  1. The methodological novelty is limited. Environment and task generation have been explored in AgentGen [1] and EnvScaler [2], while Agent-World [3] already uses failure diagnosis to guide further task synthesis. BehaviorPilot’s main addition is an explicit behavior taxonomy for organizing this process, rather than a new synthesis or refinement paradigm.
  2. It is unclear whether the gains from iterative refinement come from targeting weak behaviors or simply adding more training data. Table 2 increases the dataset from approximately 4k to 10k tasks, but do not compares this approach with adding the same number of tasks evenly across behaviors.
  3. A failed task does not necessarily identify which behavior the model struggles with. For example, a task targeting error recovery may fail because the model supplies an incorrect argument before reaching the recovery step. The method nevertheless uses the task’s failure to increase the training budget for its designated behavior, without evaluating whether this assignment matches the actual cause of failure; the QA checks in Appendix A.2 assess task–behavior alignment, not failure attribution.
  4. The caption of Table 2 refers to “Task,” whereas the corresponding column is labeled “Tasks.” Please make this consistent.
  5. Section 3.5 states that each behavior receives an equal number of validation tasks, but does not report that number, making it difficult to assess how reliably the failure rates are estimated.

[1] Hu et al. AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation.

[2] Song et al. EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis.

[3] Dong et al. Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence.

Questions for authors

See weakness.

3 comments
AuthorsOpenReview ↗

Reply to Reviewer XN7p (Part 1)

Thank you for your detailed and constructive review. We appreciate your recognition that explicit behavior targets reduce repetitive interaction patterns and improve coverage of underrepresented skills, that BehaviorPilot leads on all three benchmark averages with a 4k-task dataset outperforming a 16k-task behavior-agnostic one, and that our execution and behavior-coverage checks and independent QA process are useful. We also thank you for your questions, which helped us clarify our positioning and address the remaining concerns. We respond below with our positioning against prior methods, matched-budget baselines, failure-attribution analysis, and validation-set details.

W1 (Distinction from prior work)

“The methodological novelty is limited. Environment and task generation have been explored in AgentGen [1] and EnvScaler [2], while Agent-World [3] already uses failure diagnosis ... BehaviorPilot’s main addition is an explicit behavior taxonomy ... rather than a new synthesis or refinement paradigm.”

Thank you for highlighting these related works. They have demonstrated the value of environment/task synthesis, executable verification, and failure-guided expansion, but primarily organize data around environments, scenarios, or task difficulty. BehaviorPilot’s core contribution is controllable task synthesis for behavior coverage, with both generation and refinement specifically designed to support behavior guidance.

During generation, AgentGen expands environment diversity and adjusts task difficulty through Bi-Evol. EnvScaler generates scenarios from environment states, tools, and rules, then checks task completion. Agent-World generates executable tasks through tool-dependency graphs and programmatic synthesis. Although we share some operations, controlling tool calls or final outcomes does not ensure target behavior coverage: tool graphs describe dependencies and call order, while an agent may complete a task without exercising the intended behavior. BehaviorPilot therefore specifies target behavior combinations before generation and checks whether both reference solutions and actual training trajectories cover them. This directly controls which behaviors enter training, rather than merely adding labels to generated tasks.

For refinement, Agent-World analyzes failure traces, error distributions, and environment metadata to suggest tasks for specific environments. These suggestions do not directly specify how many tasks to generate for each behavior across environments. BehaviorPilot instead uses a behavior-balanced validation set to measure each behavior’s failure rate and determine its next synthesis quota. Thus, results from different environments jointly guide additional training data toward shared behavioral weaknesses.

As shown in Table A, we analyzed behavior frequencies in EnvScaler, Agent-World (our reproduction), and BehaviorPilot data. EnvScaler and Agent-World concentrate 84.23% and 50.73% in their top three behaviors, respectively, with near-median frequencies of only 0.00% and 0.22%–0.36%, and bottom-three frequencies of 0.00% for both. In contrast, BehaviorPilot’s top-three share is 39.11%, while its near-median and bottom-three frequencies reach 0.67%–0.76% and 0.06%–0.10%. These results show that BehaviorPilot produces training data with more balanced behavior coverage, reducing concentration on frequent behaviors while improving coverage of low-frequency ones.

Table A. Frequencies of each method’s three most frequent, three least frequent, and three near-median behaviors.

MethodTop 3 behaviorsBottom 3 behaviorsNear-median 3 behaviors
EnvScaler53.08% / 18.36% / 12.79%0.00% / 0.00% / 0.00%0.00% / 0.00% / 0.00%
Agent-World21.41% / 16.85% / 12.47%0.00% / 0.00% / 0.00%0.22% / 0.34% / 0.36%
BehaviorPilot15.48% / 12.04% / 11.59%0.06% / 0.09% / 0.10%0.67% / 0.69% / 0.76%
AuthorsOpenReview ↗

Reply to Reviewer XN7p (Part 2)

W2 (Failure-guided refinement versus uniform expansion)

It is unclear whether the gains from iterative refinement come from targeting weak behaviors or simply adding more training data. ... adding the same number of tasks evenly across behaviors.

Thank you for suggesting this control. To evaluate the contribution of failure-rate-based budget allocation, we added two baselines across all three rounds. All three strategies retain behavior-conditioned task synthesis, start from the same initial model and training corpus (~4k tasks), and add approximately 2k tasks per round. Behavior-random randomly assigns synthesis quotas across behaviors, while Behavior-uniform assigns equal quotas to all behaviors.

As shown in Table B, BehaviorPilot achieves the highest benchmark averages in all three rounds, outperforming Behavior-uniform by 4.8 points on BFCL and 1.8 points on τ²-Bench at approximately 10k tasks. Behavior-random and Behavior-uniform show limited gains from further expansion, as neither prioritizes the model’s weak behaviors. In contrast, BehaviorPilot allocates more tasks to behaviors with higher validation failure rates. It improves both benchmark averages in every round and outperforms both baselines at the same task counts.

Table B. Synthesis-budget allocation strategies at matched cumulative task counts. Bold indicates the best result within each refinement round.

RoundCumulative tasksStrategyBFCL Avg.τ²-Bench Avg.
Initial~4k—48.544.8
+1~6kBehavior-random47.945.9
+1~6kBehavior-uniform48.540.4
+1~6kBehaviorPilot49.046.4
+2~8kBehavior-random47.546.1
+2~8kBehavior-uniform47.443.4
+2~8kBehaviorPilot49.947.4
+3~10kBehavior-random47.847.0
+3~10kBehavior-uniform47.548.9
+3~10kBehaviorPilot52.350.7

W3 (Failures unrelated to the target behavior)

A failed task does not necessarily identify which behavior the model struggles with. ... The method nevertheless uses the task’s failure to increase the training budget for its designated behavior ... the QA checks in Appendix A.2 assess task–behavior alignment, not failure attribution.

Thank you for highlighting this concern. With 50 validation tasks per behavior, occasional errors unrelated to the target behavior affect only a small fraction of these tasks. If such errors occur frequently, they can also cause failures on tasks that directly test the corresponding behavior, leading us to generate more training data for that weakness. Therefore, this design helps us identify recurring weaknesses and generate training data to address them.

W4 (Table 2 caption typo)

“The caption of Table 2 refers to ‘Task,’ whereas the corresponding column is labeled ‘Tasks.’ Please make this consistent.”

Thank you for pointing this out. We have changed “Task” to “Tasks” in the caption to match the column header.

W5 (Validation-set size)

“Section 3.5 states that each behavior receives an equal number of validation tasks, but does not report that number.”

Thank you for pointing out this omission. Our fixed validation set contains 50 tasks per behavior across 100 behaviors, totaling 5,000 tasks. Each task has a single designated target behavior and is held out from training. For reference, BFCL V4 Multi-Turn, τ²-Bench, and VitaBench contain 800, 278, and 400 tasks, respectively. Our 5,000-task validation set provides a substantial evaluation budget with balanced coverage across all 100 behaviors.

Dear Reviewer XN7p,

We sincerely appreciate your careful review of our paper and your constructive feedback, which is helpful in improving our work.

As the author-reviewer discussion period will close shortly, we kindly ask you to take a look at our rebuttal and consider whether we have addressed the concerns you raised. Many thanks again for your time and assistance in improving the paper.

Regards from the authors.

Reviewer_fPi5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BehaviorPilot represents recurring tool-use patterns with a hierarchical behavior taxonomy and uses behavior combinations to guide executable task synthesis. Verified, behavior-covering trajectories support SFT, while task verifiers support RL. A fixed behavior-balanced validation set identifies weaknesses and allocates subsequent synthesis budget. The resulting Qwen3-8B model improves performance on three tool-use benchmarks.

Critical strengths
  1. The actionable contribution is a change in what synthetic data generation tries to cover: recurring interaction behaviors rather than domain labels alone. This gives practitioners a concrete way to identify redundant training experiences and request missing capabilities. Although capability coverage and targeted synthesis are established ideas, their operationalization in executable tool tasks is potentially useful.
  2. Table 3 directly supports the core generation strategy: with 100 environments and approximately 4,000 tasks, behavior-guided SFT scores 48.5/44.8 on BFCL/τ²-Bench versus 42.4/22.6 without behavioral guidance. Counts alone do not guarantee fully matched generation costs, but this is more informative than the final model’s improvement over its untrained base and supports a contribution even without the iterative refinement module.
Critical weaknesses
  1. The reliability and portability of the proposed coverage measure remain insufficiently established. Independent checks of behavior labels, overlap, annotator sensitivity, source-trajectory provenance, and full generation/training settings are needed to determine how readily others can reproduce and use the approach.
  2. Claims beyond the initial generation strategy need narrowing. Agent-World already diagnoses weaknesses and synthesizes targeted tasks. Table 2 does not isolate failure-rate allocation from additional data, and the SFT-only versus final SFT+RL checkpoint relationship is unclear. These gaps limit the claimed novelty of the adaptive loop; they do not erase the direct evidence for behavior-conditioned generation in Table 3.
Questions for authors
  1. Does Table 3 hold teacher, filtering, training-token exposure, and other generation choices fixed? Please provide the source inventory and an independent coverage audit sufficient to interpret this central comparison.
  2. Can the authors clarify the checkpoints and data behind Tables 1 and 2, and either support the specific benefit of failure-rate allocation with an equal-budget control or limit the corresponding claim?
3 comments
AuthorsOpenReview ↗

Reply to Reviewer fPi5 (Part 1)

Thank you for your detailed and thoughtful review. We appreciate your recognition that Table 3 directly supports our core behavior-conditioned generation strategy, and that shifting synthetic-data coverage from domain labels to recurring interaction behaviors offers a concrete way to identify redundant experiences and address missing capabilities. We also thank you for your comments, which helped us strengthen the reliability and reproducibility of the approach. We respond below with reliability and portability evidence, the fixed controls behind Table 3, checkpoint and data alignment, and matched-budget allocation baselines.

W1, Q1 (Reliability, portability, and reproducibility)

“Independent checks of behavior labels, overlap, annotator sensitivity, source-trajectory provenance, and full generation/training settings are needed to determine how readily others can reproduce and use the approach.”

Thank you for highlighting these concerns. We provide additional evidence and implementation details to clarify the reliability, portability, and reproducibility of our approach.

Trajectory sources. We used four open-source datasets containing high-quality, diverse tool-use trajectories: Nemotron-SFT-Agentic-v2, mcp-universe-trajectories, open-thoughts/AgentTrove, and open-thoughts/OpenThoughts-Agent-v1-SFT.

Reliability. We use DeepSeek-V4-Pro-Preview (the synthesis model used in the paper) in a fresh process to reassess primary labels against task content. Agreement is approximately 95% before correction and 99% after correction. Independent reannotation of labels and coverage decisions reports 87.1% label overlap and 1.1% annotator sensitivity. Separately, all audited task–solution pairs pass execution verification after correction. These checks provide quantitative evidence of annotation consistency and task execution validity.

Portability. Our taxonomy defines behaviors independently of specific environment instances. For a new environment, we identify supported behaviors from its PRD, tools, and data, then apply the same coverage and quota-allocation rules. This shared design supports transfer across environments without redesigning the taxonomy or allocation mechanism.

Reproducibility. Full SFT and GRPO hyperparameters and training budgets are included in Appendix D (Training Settings). And we will release the taxonomy, synthesized environments, tasks, prompts, data splits, and trained models.

Q1 (Controls for the Table 3 comparison)

Does Table 3 hold teacher, filtering, training-token exposure, and other generation choices fixed? ... Please provide the source inventory and an independent coverage audit ...

Thank you for requesting this clarification. Yes. In the comparison with 100 environments and approximately 4k tasks, we hold the trajectory teacher, filtering procedures, training-token exposure, and other generation settings fixed, varying only behavior conditioning. The trajectory sources and independent label/coverage audit are provided in our response to W1.

W2 (Distinction from Agent-World in failure-guided task synthesis)

“Agent-World already diagnoses weaknesses and synthesizes targeted tasks.”

Thank you for raising this comparison. BehaviorPilot’s core contribution to refinement is behavior-level targeting and quota allocation, which uses per-behavior validation failure rates to determine how many new tasks to generate for each behavior across environments. In contrast, Agent-World produces environment-specific task suggestions without directly specifying per-behavior quotas, making the behavior distribution of newly added training data harder to control.

As shown in Table A, we compared behavior frequencies in Agent-World (our reproduction) and BehaviorPilot data. Agent-World’s top-three share is 50.73%, compared with 39.11% for BehaviorPilot. Its near-median and bottom-three frequencies are 0.22%–0.36% and 0.00%, versus 0.67%–0.76% and 0.06%–0.10% for BehaviorPilot. These results show more balanced behavior coverage in BehaviorPilot-generated data, with less concentration on frequent behaviors and better coverage of low-frequency ones.

Table A. Frequencies of each method’s three most frequent, three least frequent, and three near-median behaviors.

MethodTop 3 behaviorsBottom 3 behaviorsNear-median 3 behaviors
Agent-World21.41% / 16.85% / 12.47%0.00% / 0.00% / 0.00%0.22% / 0.34% / 0.36%
BehaviorPilot15.48% / 12.04% / 11.59%0.06% / 0.09% / 0.10%0.67% / 0.69% / 0.76%
AuthorsOpenReview ↗

Reply to Reviewer fPi5 (Part 2)

W2, Q2 (Training data and SFT/RL results)

“…the SFT-only versus final SFT+RL checkpoint relationship is unclear.”

“Can the authors clarify the checkpoints and data behind Tables 1 and 2…?”

Thank you for pointing out this ambiguity. The SFT dataset consists of 9,000+ verified trajectories collected across the initial synthesis and three refinement rounds in Table 2 (~10k tasks in total). The “+3 rounds” SFT checkpoint in Table 2 is the starting checkpoint for RL. Using this checkpoint, we collect four rollouts per SFT training task and retain tasks solved in some but not all attempts. We then train with GRPO from the same checkpoint on the selected tasks to obtain BehaviorPilot-8B the final model reported in Table 1. As shown in Table B, the main gains on these two benchmarks are already achieved through SFT on BehaviorPilot-generated data. Subsequent GRPO improves BFCL but reduces τ²-Bench performance. We acknowledge this regression as a limitation of the current training pipeline and will further investigate multi-turn RL.

Table B. Performance before and after RL.

Model / checkpointBFCL Avg.τ²-Bench Avg.
Before RL: SFT checkpoint, Table 2 (+3 rounds)52.350.7
After RL: BehaviorPilot-8B, Table 153.348.1

W2, Q2 (Failure-rate allocation versus equal-budget control)

“…either support the specific benefit of failure-rate allocation with an equal-budget control?”

Thank you for suggesting this control. To evaluate the contribution of failure-rate-based budget allocation, we added two baselines across all three rounds. All three strategies retain behavior-conditioned task synthesis, start from the same initial model and training corpus (~4k tasks), and add approximately 2k tasks per round. Behavior-random randomly assigns synthesis quotas across behaviors, while Behavior-uniform assigns equal quotas to all behaviors.

As shown in Table C, BehaviorPilot achieves the highest benchmark averages in all three rounds. At approximately 10k tasks, it outperforms Behavior-uniform by 4.8 points on BFCL and 1.8 points on τ²-Bench. These results support the specific benefit of failure-rate-based allocation over uniform or random expansion at matched task counts.

Table C. Synthesis-budget allocation strategies at matched cumulative task counts. Bold indicates the best result within each refinement round.

RoundCumulative tasksStrategyBFCL Avg.τ²-Bench Avg.
Initial~4k—48.544.8
+1~6kBehavior-random47.945.9
+1~6kBehavior-uniform48.540.4
+1~6kBehaviorPilot49.046.4
+2~8kBehavior-random47.546.1
+2~8kBehavior-uniform47.443.4
+2~8kBehaviorPilot49.947.4
+3~10kBehavior-random47.847.0
+3~10kBehavior-uniform47.548.9
+3~10kBehaviorPilot52.350.7

Dear Reviewer fPi5,

Thank you for the time and effort you have devoted to reviewing our paper. We greatly appreciate your helpful comments and suggestions for improving its quality.

With the author-reviewer discussion period coming to an end, we would be grateful if you could read our rebuttal and let us know whether it addresses your concerns. Thank you again for your time and valuable feedback.

Regards from the authors.

Reviewer_brQfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BehaviorPilot organizes synthetic tool-use training data around an induced taxonomy of agent behaviors. It constructs executable environments and verified task-solution pairs conditioned on selected behaviors, then uses aggregate failure rates on a separate validation set to guide further synthesis. A Qwen3-8B agent trained with SFT and RL is evaluated on VitaBench, BFCL V4 Multi-Turn, and tau-squared-Bench. The main result is an overall score of 40.6, compared with 27.8 for the base model and 33.0 for EnvScaler-8B. The clearest evidence comes from comparing the two synthesis methods with the same numbers of environments and tasks. The additional benefit of failure-guided refinement is harder to identify.

Critical strengths
  1. Table 3 provides an important controlled comparison. With 100 environments and approximately 4,000 tasks in both conditions, behavior-guided synthesis reaches 48.5 on BFCL and 44.8 on tau-squared-Bench, versus 42.4 and 22.6 for behavior-agnostic synthesis. Simply expanding the latter to 400 environments and 16,000 tasks does not close the gap. This makes a convincing case that behavioral coverage matters, even when the alternative has more data.

  2. The method includes executable verification and checks that generated solutions cover the requested behaviors. The validation loop receives aggregate behavior-level statistics rather than validation tasks or answers, and the paper states that public benchmarks are reserved for final evaluation. The main gains also extend across three benchmarks. These choices make the proposal practically relevant and reduce concerns about direct benchmark-driven synthesis.

Critical weaknesses
  1. The refinement experiment adds data as it targets failures, so the source of its gains is unclear. In Table 2, the corpus grows from roughly 4,000 to 10,000 tasks; BFCL rises from 48.5 to 52.3 and tau-squared-Bench from 44.8 to 50.7. The missing comparison is a behavior-balanced corpus expanded uniformly to the same final size. Table 3 controls data volume for initial synthesis, but does not test this adaptive allocation rule. Also, the ablations use SFT alone, whereas the headline agent includes RL, leaving the contributions of refinement and RL difficult to separate.

  2. The behavior labels need a quantitative reliability check. The taxonomy drives generation, validation allocation, and coverage analysis, yet its source trajectory corpora are not clearly specified. Appendix A.2 checks primary-behavior alignment in a fresh model process, which is a useful safeguard, but reports no audit error rates or agreement measurements. This matters particularly for tasks assigned one primary label despite involving several behaviors. The SFT and RL hyperparameters are also needed for reproduction.

Questions for authors
  1. At approximately 10,000 tasks, how does failure-guided refinement compare with uniform allocation over the same behavior taxonomy, using the same generation and training budgets? This is the control most likely to change my view of the adaptive component.

  2. What exactly is the relationship between the more than 9,000 verified trajectories used for the headline agent and the task sets in Table 2? Please provide the main agent's SFT-only result on the same data and clarify which data are used for RL. The difference between the main tau-squared-Bench score of 48.1 and Table 2's final SFT score of 50.7 is hard to interpret without this alignment.

  3. Which open-source trajectory datasets were used to induce the taxonomy? Beyond the fresh-process QA in Appendix A.2, can you report a stratified audit of the labels and behavior-coverage decisions, including agreement, error rates, and the model used for QA?

  4. Please give the behavior-validation set size and construction details, along with the SFT and GRPO hyperparameters and training budgets. Can the taxonomy, environments, prompts, and splits be released together?

3 comments
AuthorsOpenReview ↗

Reply to Reviewer brQf (Part 1)

Thank you for your detailed and constructive review. We appreciate your recognition that the Table 3 controlled comparison makes a convincing case for behavioral coverage even when the alternative has more data, and that our executable verification, behavior-coverage checks, aggregate validation statistics, and reservation of public benchmarks for final evaluation reduce concerns about direct benchmark-driven synthesis. We also thank you for your questions, which helped us sharpen the controlled comparisons and clarify the training pipeline. We respond below with matched-budget comparisons, SFT/RL checkpoint alignment, reliability audits, and the training details in Appendix D.

W1, Q1 (Failure-guided refinement versus uniform expansion)

“At approximately 10,000 tasks, how does failure-guided refinement compare with uniform allocation over the same behavior taxonomy, using the same generation and training budgets?”

Thank you for suggesting this control. To evaluate the contribution of failure-rate-based budget allocation, we added two baselines across all three rounds. All three strategies retain behavior-conditioned task synthesis, start from the same initial model and training corpus (~4k tasks), and add approximately 2k tasks per round. Behavior-random randomly assigns synthesis quotas across behaviors, while Behavior-uniform assigns equal quotas to all behaviors.

As shown in Table A, BehaviorPilot achieves the highest benchmark averages in all three rounds, outperforming Behavior-uniform by 4.8 points on BFCL and 1.8 points on τ²-Bench at approximately 10k tasks. Behavior-random and Behavior-uniform show limited gains from further expansion, as neither prioritizes the model’s weak behaviors. In contrast, BehaviorPilot allocates more tasks to behaviors with higher validation failure rates. It improves both benchmark averages in every round and outperforms both baselines at the same task counts.

Table A. Synthesis-budget allocation strategies at matched cumulative task counts. Bold indicates the best result within each refinement round.

RoundCumulative tasksStrategyBFCL Avg.τ²-Bench Avg.
Initial~4k—48.544.8
+1~6kBehavior-random47.945.9
+1~6kBehavior-uniform48.540.4
+1~6kBehaviorPilot49.046.4
+2~8kBehavior-random47.546.1
+2~8kBehavior-uniform47.443.4
+2~8kBehaviorPilot49.947.4
+3~10kBehavior-random47.847.0
+3~10kBehavior-uniform47.548.9
+3~10kBehaviorPilot52.350.7

W1, Q2 (Training data and SFT/RL results)

“What exactly is the relationship between the more than 9,000 verified trajectories used for the headline agent and the task sets in Table 2? Please provide the main agent's SFT-only result on the same data and clarify which data are used for RL.”

Thank you for pointing out this ambiguity. The SFT dataset consists of 9,000+ verified trajectories collected across the initial synthesis and three refinement rounds in Table 2 (~10k tasks in total). The “+3 rounds” SFT checkpoint in Table 2 is the headline agent’s checkpoint before RL. Using this checkpoint, we collect four rollouts per SFT training task and retain tasks solved in some but not all attempts. We then train with GRPO from the same checkpoint on the selected tasks to obtain BehaviorPilot-8B, the final model reported in Table 1. As shown in Table B, the main gains on these two benchmarks are already achieved through SFT on BehaviorPilot-generated data. Subsequent GRPO improves BFCL but reduces τ²-Bench performance, explaining the difference between the pre-RL score of 50.7 and the final score of 48.1. We acknowledge this regression as a limitation of the current training pipeline and will further investigate multi-turn RL.

Table B. Performance before and after RL.

Model / checkpointBFCL Avg.τ²-Bench Avg.
Before RL: SFT checkpoint, Table 2 (+3 rounds)52.350.7
After RL: BehaviorPilot-8B, Table 153.348.1
AuthorsOpenReview ↗

Reply to Reviewer brQf (Part 2)

W2, Q3 (Taxonomy sources and behavior-label reliability)

“Which open-source trajectory datasets were used to induce the taxonomy? Beyond the fresh-process QA in Appendix A.2, can you report a stratified audit of the labels and behavior-coverage decisions, including agreement, error rates, and the model used for QA?”

Thank you for pointing out these missing details. We provide the trajectory sources and audit results below.

Taxonomy sources. We used four open-source datasets containing high-quality, diverse tool-use trajectories: Nemotron-SFT-Agentic-v2, mcp-universe-trajectories, open-thoughts/AgentTrove, and open-thoughts/OpenThoughts-Agent-v1-SFT.

Label reliability. We conduct quantitative audits to assess the consistency of behavior labels and coverage decisions. We use DeepSeek-V4-Pro-Preview (the synthesis model used in the paper) in a fresh process to reassess primary labels against task content. Agreement is approximately 95% before correction and 99% after correction. Independent reannotation of labels and coverage decisions reports 87.1% label overlap and 1.1% annotator sensitivity. All audited task–solution pairs pass execution verification after correction (0.00% execution errors). These results support label and coverage consistency, alongside task execution validity.

W2, Q4 (Validation set, training settings, and reproducibility)

“Please give the behavior-validation set size and construction details, along with the SFT and GRPO hyperparameters and training budgets. Can the taxonomy, environments, prompts, and splits be released together?”

Thank you for pointing out these missing details. We provide the validation, training, and release details below.

Validation set. The fixed validation set contains 5,000 tasks covering 100 behaviors, with 50 tasks per behavior. We use the same synthesis pipeline described in the paper, but each validation task targets a single behavior. These tasks share the training environments but are held out from training. Only aggregate behavior-level failure rates are returned to the synthesis pipeline.

Training settings. We have added the full SFT and GRPO hyperparameters and training budgets to Appendix D (Training Settings). And we will release the taxonomy, synthesized environments, tasks, prompts, data splits, and trained models.

Dear Reviewer brQf,

Thanks for your time and effort in reviewing this paper. Your comments are valuable to improve the quality of this paper.

As the author-reviewer discussion period is drawing to a close, we gently request that you read the rebuttal and check whether your concerns are addressed. Thanks again for your time and help in improving the quality of this paper.

Regards from the authors.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response — Thanks to All Reviewers for Their Constructive Feedback

We would like to thank all reviewers for their careful reading and constructive feedback. We are encouraged by your recognition of the value of explicit behavior targets for achieving balanced behavioral coverage and improving tool-use performance.

Reviewer brQf highlighted the convincing evidence that behavioral coverage matters beyond data volume and appreciated our safeguards against benchmark-driven synthesis. Reviewer fPi5 recognized the practical value of behavior-guided generation for identifying redundant training experiences and addressing missing capabilities. Reviewer XN7p acknowledged the benefits of explicit behavior targets for covering underrepresented skills, the strong benchmark results, and the execution and behavior-coverage checks supported by separate QA.

During the rebuttal period, we conducted additional experiments and revised the manuscript to address the reviewers' concerns:

  1. Failure-rate allocation versus additional data (All reviewers). We added random and uniform allocation baselines, starting from the same ~4k-task corpus and adding ~2k behavior-conditioned tasks per round under matched generation and training budgets. BehaviorPilot delivers larger overall gains and sustained improvements across all three rounds, whereas random and uniform allocation largely plateau after the initial corpus, with little benefit from adding more tasks. At ~10k tasks, BehaviorPilot outperforms Behavior-uniform by 4.8 points on BFCL and 1.8 points on τ²-Bench. These matched-budget results support the specific benefit of failure-rate-based allocation rather than additional data alone.

  2. Comparisons with prior work (Reviewers fPi5 and XN7p). We clarified BehaviorPilot’s explicit behavioral control: generation specifies target behaviors and verifies coverage in both reference solutions and training trajectories, while refinement assigns failure-based behavioral quotas across environments rather than environment-specific task suggestions. Coverage analysis shows lower top-three concentration than Agent-World and EnvScaler, with better coverage of low-frequency behaviors. This supports controllable behavior coverage in both generation and refinement, beyond environment diversity or failure-guided expansion alone.

  3. Taxonomy reliability, portability, and reproducibility (Reviewers brQf and fPi5). We specified the four trajectory sources and reported label and execution audits, with approximately 95% agreement before correction, 99% after correction, and 87.1% label overlap. We also clarified taxonomy reuse across environments and added training settings and budgets to Appendix D. To support reproducibility, we will release the taxonomy, environments, tasks, prompts, splits, and models.

  4. Comparison settings and SFT/RL results (Reviewers brQf and fPi5). We confirmed that Table 3 holds the teacher, filtering, training-token exposure, and other generation settings fixed, varying only behavior conditioning. We also clarified the data and checkpoints behind Tables 1 and 2 and reported results before and after RL, showing that the main gains are already achieved through SFT on BehaviorPilot-generated data. We will further explore multi-turn RL methods to improve performance across benchmarks.

  5. Validation design and failure attribution (Reviewers brQf and XN7p). We clarified that the fixed validation set contains 5,000 tasks, with 50 per behavior. We also explained how evaluating each behavior across multiple tasks reduces the influence of occasional off-target errors when identifying recurring weaknesses.

We believe these additional experiments and clarifications have strengthened the empirical support for BehaviorPilot and provided a clearer understanding of its contributions and limitations. We will incorporate the feedback into the final manuscript to further improve its clarity and completeness.

Once again, we sincerely thank all reviewers and the Area Chair for their valuable feedback, time, and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.