acceptodds
Under review as a conference paper at ICLR 2027

RarePlay: Exploring Scarce Action Spaces in Tool Graphs via Self-Evolving Synthesis

Abstract

Large language models (LLMs) have demonstrated strong function-calling capabilities, yet their performance on long-horizon tool-use tasks remains heavily dependent on the quality and diversity of training trajectories. Existing synthetic data generation methods typically rely on static tool graphs or multi-agent collaboration, which can produce homogeneous trajectories, incur substantial computational overhead, and may struggle to adapt task difficulty to the evolving capabilities of the model. We introduce RarePlay, a self-evolving framework for synthesizing diverse and progressively challenging function-calling data. RarePlay constructs a state transition graph from the current model’s behavior and introduces Rarity-Guided Exploration (RGE) to prioritize actions that are underrepresented under the current policy while remaining likely to lead to successful outcomes. Sampled trajectories are converted into training tasks through inverse task synthesis and further filtered using rubric-based rejection sampling to improve logical consistency and trajectory quality. By iteratively updating both the model and the state graph, RarePlay incorporates newly explored states and transitions, providing a richer foundation for sampling longer paths and supporting an easy-to-hard curriculum guided by a predefined hop-count schedule. Experiments on BFCL-v4 and AutomationBench demonstrate that RarePlay improves function-calling performance and achieves state-of-the-art results under comparable training-data budgets.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agent, tool using, graph

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ftoeOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RarePlay synthesizes multi-turn tool-use data from state-transition graphs built from current-policy rollouts. Its Rarity-Guided Exploration (RGE) sampler combines task success, low visit counts, tool diversity, and a cross-domain penalty. Teacher-generated tasks and trajectories undergo rubric-based filtering before iterative supervised fine-tuning under a predefined hop-count schedule. The method improves BFCL performance for 4B and 14B models and reports additional gains on AutomationBench.

Critical strengths
  1. The comparisons support the synthesis pipeline’s effectiveness. With matched training-example counts and the same 4B backbone, RarePlay achieves 30.00 on BFCL versus 24.25 for ToolMind. The 14B model improves from 28.50 to 36.00, while AutomationBench provides an additional evaluation setting (§5.3, Table 2).
  2. The component ablations provide useful evidence. Removing RGE or rejection sampling reduces BFCL performance from 30.00 to 23.38 and 24.75, respectively. Removing RGE also lowers trajectory retention from 60% to 17.2% (§5.4, Table 3), supporting the value of guided sampling and filtering.
Critical weaknesses
  1. The rare-behavior explanation lacks direct validation. The “w/o RGE” ablation replaces all four terms in Eq. 1 with random sampling, so it cannot isolate rarity from success preference or the other components. More importantly, the paper does not quantify whether targeted low-visit behaviors survive teacher regeneration and filtering. The rubric is designed to preserve these behaviors, but this does not establish their actual retention. The end-to-end gains therefore support the overall pipeline more directly than its targeted-coverage explanation.
  2. The filtering rule does not explicitly enforce all essential task outcomes. Section 4.3 combines behavioral coverage and result correctness into a weighted score, accepting trajectories that score above 80 (Appendix D). A trajectory could therefore fail a mandatory outcome worth 10 points and still pass with 90. It is unclear whether a separate hard success check prevents this. This is a potential gap in the stated rule, not an observed failure in the training data.
Questions for authors
  1. Behavior retention: Using saved graph counts, sampled paths, and teacher trajectories, could the authors quantify how much targeted low-visit behavior remains after regeneration and filtering? Please define the coverage measure, account for path length and task success, and compare with existing random-sampling records where available. This would test whether the intended behaviors enter the training data, not establish their independent causal effect on accuracy. No additional multi-round training is requested.
  2. Filtering correctness: Are mandatory task outcomes checked separately from the aggregate rubric score? If not, could the authors audit a small sample of accepted trajectories, such as 30–50, against actual task requirements and recorded sandbox states? Please describe the sampling procedure and report unmet essential outcomes rather than relying solely on the original rubric scores.
7 comments

We sincerely thank the reviewer for their thoughtful and constructive comments. We appreciate the opportunity to clarify our methodology and provide additional experimental evidence to address the concerns raised.

1.The rare-behavior explanation lacks direct validation. The “w/o RGE” ablation replaces all four terms in Eq. 1 with random sampling, so it cannot isolate rarity from success preference or the other components.

We thank the reviewer for this valuable suggestion. To better understand the contribution of each component in RGE, we conducted additional ablation experiments on qwen3-4B by removing each term individually while keeping the others unchanged. The results are presented below.

AblationBaseM.ParamM.FuncLongAvg.
w/o Rarity45.016.022.025.527.12
w/o Success43.518.024.025.527.75
w/o Diversity45.517.022.526.527.88
w/o Transition Penalty47.517.525.026.029.00
Full RGE47.016.028.029.030.00

The results show that ​all four components contribute positively to the overall performance of RGE​. Removing the rarity and success terms leads to the largest average performance drops, highlighting the importance of prioritizing underexplored yet successful transitions. The diversity term encourages diverse tool usage and contributes to generating more informative trajectories, while the transition penalty helps maintain coherent state transitions. Both components play complementary roles in improving the overall effectiveness of RGE.

These findings demonstrate that the four components play complementary roles, with rarity and success being particularly important for effective exploration.

2.Behavior retention: Using saved graph counts, sampled paths, and teacher trajectories, could the authors quantify how much targeted low-visit behavior remains after regeneration and filtering? Please define the coverage measure, account for path length and task success, and compare with existing random-sampling records where available. This would test whether the intended behaviors enter the training data, not establish their independent causal effect on accuracy. No additional multi-round training is requested.

We thank the reviewer for this constructive suggestion. We conducted an additional analysis using the saved state-transition graphs, sampled paths, and teacher-generated trajectories to examine whether RGE-targeted rare behaviors are preserved throughout the synthesis pipeline.

We identify rare transitions based on their traversal counts in the policy-derived state graph. We measure coverage as the proportion of targeted rare transitions represented in the generated trajectories, and density as the proportion of rare transitions among all transitions in the final training corpus. The density measure normalizes for trajectory length, while the final-corpus analysis includes only trajectories that pass the rubric-based filtering process.

Our analysis yields three key findings:

  1. Guided sampling: RGE achieves 2.5× higher coverage of policy-rare transitions than random sampling.
  2. Teacher regeneration: Approximately 63% of targeted rare transitions are successfully preserved in the teacher-generated trajectories.
  3. Final training corpus: After rubric-based filtering, the resulting corpus exhibits 3.4× higher density of rare transitions than the random-sampling baseline.

These results demonstrate that RGE's preference for underexplored behaviors is effectively propagated through teacher regeneration and filtering, leading to substantially greater representation of targeted rare behaviors in the final training data.

AuthorsOpenReview ↗

Response to reviewer ftoe(2)

3.Filtering correctness: Are mandatory task outcomes checked separately from the aggregate rubric score? If not, could the authors audit a small sample of accepted trajectories, such as 30–50, against actual task requirements and recorded sandbox states? Please describe the sampling procedure and report unmet essential outcomes rather than relying solely on the original rubric scores.

We thank the reviewer for this valuable suggestion. To further assess the correctness of accepted trajectories, we randomly sampled 30 trajectories and manually verified their final sandbox states against the original task requirements. Among them, 26 trajectories (86.7%) satisfied all mandatory outcomes, while 4 (13.3%) did not fully complete the corresponding tasks.

We would like to clarify that ​RarePlay does not treat complete task success as the sole criterion for determining the value of a training trajectory*. Our primary objective is to expose the student model to underexplored yet meaningful tool-use behaviors, rather than exclusively collecting trajectories that successfully complete every task requirement. As described in Section 4.3, our rubric-based filtering jointly considers the preservation of targeted intermediate behaviors and final-state consistency.

Consequently, trajectories that do not fully satisfy all task-level requirements may still contain valid and informative tool interactions that provide useful supervision for SFT. This distinction is particularly relevant to RarePlay, as its rarity-guided exploration mechanism is designed to improve the coverage of underrepresented tool-use behaviors. The aggregate rubric therefore reflects both behavioral coverage and task-level correctness, rather than imposing complete task success as an absolute prerequisite for acceptance.

Dear Reviewer,

We hope our responses have addressed your concerns. We would appreciate it if you could take another look and let us know if you have any further questions.

Thank you!

Reviewer_QQSCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes RarePlay, a self-evolving framework for synthesizing function-calling training data. It organizes the policy model’s rollouts into a state-transition graph, samples paths through a rarity-guided exploration function that favors rare yet successful behaviors, converts sampled paths into tasks via inverse task synthesis, and filters teacher trajectories with rubric-based rejection sampling. The framework iterates over multiple rounds with an increasing hop-count schedule, and experiments on BFCL-v4 and AutomationBench show improvements over base models and existing synthesis baselines under matched training-data budgets.

Critical strengths
  1. The motivation is clear. The paper identifies the limitation of static tool-graph synthesis methods, which neither define verifiable environments nor cover infrequent behaviors.
  2. The technical design is novel. Replacing tool-level graphs with a state-transition graph preserves execution states and enables verifiable task synthesis.
  3. The experiments are comprehensive. Experiments on BFCL-v4 and AutomationBench show effectiveness of the proposed model.
  4. The paper is rigorous in structure and easy to follow.
Critical weaknesses
  1. The rubrics are generated and scored by the same closed-source teacher model family. The paper provides no validation of whether gLLM scores align with human judgment.
  2. The RGE function contains four terms, but the ablation only removes the entire function in favor of random sampling. The contribution of each design choice is unclear.
  3. The pipeline relies on closed-source teachers for path synthesis, rubric generation, trajectory generation, and rubric evaluation, and each round requires MK_rollout policy rollouts plus teacher calls. The paper reports training efficiency of the final model but not the cost of data synthesis itself, such as the number of teacher calls, API cost, or wall-clock time per round, nor a comparison of synthesis cost against ToolMind.
  4. The captions of Table 3 and Table 4 are nearly identical and both say “Ablation study on BFCL with the 4B model”.
  5. Error writing:
  • “a observation set” in the caption of Figure 2.
  • “RarePlay outperform those....” at line 107.
Questions for authors
  1. Please report AutomationBench results for ToolACE-4B and ToolMind-4B, or explain why they are omitted.
  2. How reliable is the rubric-based filtering? Please provide any human evaluation or agreement analysis between gLLM scores and human judgments on a sample of trajectories?
  3. What are the values of M and K_rollout, the source and scale of the internally pre-synthesized observation tasks?
  4. What is the total cost of the synthesis pipeline, and how does it compare with the cost of ToolMind under the same data budget?
  5. Please explain why previous traversal counts and success statistics cannot be accumulated, and whether this rebuild affects the stability of RGE across rounds.
3 comments
AuthorsOpenReview ↗

Response to reviewer QQSC(1)

We sincerely thank the reviewer for their thoughtful and constructive comments. We appreciate the opportunity to clarify our methodology and provide additional experimental evidence to address the concerns raised.

1. What is the total cost of the synthesis pipeline, and how does it compare with the cost of ToolMind under the same data budget?

We thank the reviewer for this valuable question. We provide a breakdown of the teacher side synthesis costs for RarePlay and ToolMind, normalized by the number of generated trajectories for a consistent comparison.

Metric (per trajectory)RarePlayToolMind
Input tokens142K57K
Output tokens1.1K4.7K
Total tokens143.1K61.7K
BFCL Avg. (4B)30.0024.25

RarePlay incurs higher input-token costs, primarily due to repeated tool schemas and conversation histories. However, ​its output-token consumption is approximately 4.3× lower than ToolMind's, as executable environments replace LLM-simulated tool responses and enable deterministic state-based verification. Furthermore, RarePlay achieves a ​5.75 point improvement on BFCL​, demonstrating stronger downstream performance despite higher input-token consumption.

2.What are the values of M and K_rollout, the source and scale of the internally pre-synthesized observation tasks?

We thank the reviewer for this question. In our experiments, we set M = 2,000 and Krollout=8K_rollout = 8, corresponding to 2,000 observation tasks and 8 policy rollouts per task. The internally pre-synthesized observation tasks are collected from our previous synthesis pipelines, including tool-definition-graph-based synthesis and multi-agent collaborative synthesis​. These tasks serve as the initial observation set for constructing the state-transition graph.

3.Please explain why previous traversal counts and success statistics cannot be accumulated, and whether this rebuild affects the stability of RGE across rounds.

We thank the reviewer for this insightful question. We clarify that historical traversal counts and success statistics can technically be accumulated. However, RarePlay intentionally rebuilds the state-transition graph at each round to reflect the current policy's behavior distribution.

Specifically, RGE aims to identify rare yet successful transitions under the current policy, rather than across all historical policies. Since the policy evolves during training, accumulating statistics from earlier rounds would mix observations from different policy distributions, potentially introducing stale frequency estimates and biasing exploration toward behaviors that are no longer underrepresented.

We acknowledge that rebuilding the graph may introduce some statistical fluctuations, as traversal counts and success estimates are reset each round. Nevertheless, this design prioritizes adaptability to the evolving policy distribution.

4. Please report AutomationBench results for ToolACE-4B and ToolMind-4B, or explain why they are omitted.

We thank the reviewer for this question. We would like to clarify that ​AutomationBench was not yet available when ToolACE and ToolMind were introduced​, and their data synthesis pipelines were primarily designed for function-calling tasks such as those evaluated by BFCL. In contrast, AutomationBench focuses on more complex, cross-application workflows involving long-horizon tool interactions.

Therefore, directly evaluating these baselines on AutomationBench without adapting their synthesis pipelines may introduce a mismatch between their original training objectives and the evaluation setting, potentially leading to an unfair comparison. For this reason, we report their performance on BFCL, which provides a more appropriate setting for comparing these synthesis methods.

5. How reliable is the rubric-based filtering? Please provide any human evaluation or agreement analysis between gLLM scores and human judgments on a sample of trajectories?

We thank the reviewer for this valuable suggestion. To evaluate the reliability of rubric-based filtering, we randomly sampled 30 trajectories and conducted independent human evaluations across multiple rubric criteria for each trajectory. We then compared the human judgments with the corresponding gLLM evaluations at the rubric level. The results show a ​91.3% agreement rate, indicating strong consistency between automated rubric-based evaluations and human judgments. This provides empirical evidence supporting the reliability of our rubric-based filtering mechanism.

AuthorsOpenReview ↗

Response to reviewer QQSC(2)

6.The RGE function contains four terms, but the ablation only removes the entire function in favor of random sampling. The contribution of each design choice is unclear.

We thank the reviewer for this valuable suggestion. To better understand the contribution of each component in RGE, we conducted additional ablation experiments on qwen3-4B by removing each term individually while keeping the others unchanged. The results are presented below.

AblationBaseM.ParamM.FuncLongAvg.
w/o Rarity45.016.022.025.527.12
w/o Success43.518.024.025.527.75
w/o Diversity45.517.022.526.527.88
w/o Transition Penalty47.517.525.026.029.00
Full RGE47.016.028.029.030.00

The results show that ​all four components contribute positively to the overall performance of RGE​. Removing the rarity and success terms leads to the largest average performance drops, highlighting the importance of prioritizing underexplored yet successful transitions. The diversity term encourages diverse tool usage and contributes to generating more informative trajectories, while the transition penalty helps maintain coherent state transitions. Both components play complementary roles in improving the overall effectiveness of RGE.

These findings demonstrate that the four components play complementary roles, with rarity and success being particularly important for effective exploration.

7.Typographical Errors and Caption Corrections

We thank the reviewer for pointing out these issues. We will correct the typographical errors and revise the table captions accordingly in the revised manuscript.

Dear Reviewer,

We hope our responses have addressed your concerns. We would appreciate it if you could take another look and let us know if you have any further questions.

Thank you!

Reviewer_kE64OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RarePlay, a self-evolving framework for synthesizing diverse and progressively challenging function-calling data. RarePlay first constructs a state transition graph from the current model’s behavior and introduces Rarity-Guided Exploration (RGE) to prioritize actions that are underrepresented under the current policy while remaining likely to lead to successful outcomes. Sampled trajectories are converted into training tasks through inverse task synthesis and further filtered using rubric-based rejection sampling to improve logical consistency and trajectory quality. RarePlay further incorporates newly explored states and transitions by iteratively updating both the model and the state graph. Experiments on BFCL-v4 and AutomationBench demonstrate that RarePlay improves function-calling performance and achieves state-of-the-art results under comparable training-data budgets.

Critical strengths

The key novelty of this work is interesting: replacing static tool-definition graphs with a state-transition graph rebuilt from the current policy's rollouts. It makes synthesized tasks state-verifiable rather than tool-sequence-shaped, and lets the task distribution track the evolving model. The design is directly supported by the round ablation (Table 4: 26.88 →\to 27.50 →\to 30.00 average BFCL accuracy) and by retained trajectory lengths growing across rounds on both benchmarks (Fig. 4), which is concrete evidence for the intended easy-to-hard curriculum.

Critical weaknesses
  1. The training corpus is a frontier-teacher product end to end, and the pipeline never separates the method from the teachers. Per Table 5, GLM-5.3 synthesizes the tasks, writes the rubrics, generates the BFCL trajectories (Kimi-K3 for AutomationBench), and performs the rubric-based scoring that decides which trajectories survive rejection sampling. Neither GLM-5.3 nor Kimi-K3 appears in Table 1, so the reader cannot locate the trained 4B/14B models relative to their own teachers, and the teacher-side budget (tokens or API calls for synthesis, rollouts, and rubric evaluation) is never reported while baselines are matched only on trajectory count (Sec. 5.2). As with any 4K-trajectory corpus distilled from frontier closed models, an unknown share of the +8.13/+7.50 average gains (Table 2) measures distillation rather than rarity-guided selection; since the synthesis baselines used their own different teachers, even the matched-budget comparison does not isolate the contribution. Without the teachers' own scores and a cost accounting table, the headline attribution is not established.

  2. The empirical results are too thin to support the proposed method. The baselines only contain two prior synthesis pipelines (TOOLACE-4B: 13.25; TOOLMIND-4B: 24.25, Table 2) and a single closed-source point, GPT-5.2 (28.13 BFCL average). The frontier model comparison should contain the current generation of frontier agent models (e.g., GPT-5.5/5.6, Claude Sonnet 4.6, GLM-5.2, Kimi-K3, MiniMax-M2.7). Moreover, recently there have been many works on data synthesis for tool-use agents [1, 2, 3], yet none of them is mentioned in this work. In addition, the w/o-RGE ablation changes both the sampling rule and the surviving data volume (17.2% vs. 60% retention, Table 3), so the 6.62-point gap (23.38 vs. 30.00) supports "RGE yields more acceptable trajectories" at least as strongly as "RGE-selected data trains better."

References

[1] ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training. ICML 2026.

[2] Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning. ICML 2026.

[3] EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis. ACL 2026 Findings.

Questions for authors
  1. (Clarification) What do GLM-5.3 and Kimi-K3 themselves score on BFCL-v4 and AutomationBench under your protocol, and what was the total teacher-side budget (tokens/API calls across task synthesis, trajectory generation, and rubric evaluation) for the RarePlay branch versus the TOOLACE/TOOLMIND branches under the matched 4K-trajectory budget?

  2. (Clarification) The rubric scorer and the trajectory generator are the same model family (GLM-5.3, Table 5). How accurate is g_LLM against human judgment on a sampled set of trajectories, and how sensitive are the main results to the rubric threshold λ_r = 80 (Table 6)? A small human-audited agreement study would break the circularity.

  3. (Clarification) Please report a data-count-matched w/o-RGE ablation: given the retention difference (17.2% vs. 60%, Table 3), how much of the 6.62-point average gap survives when both configurations train on the same number of accepted trajectories drawn from pools of the same size?

  4. (Clarification) How does RarePlay's state-graph synthesis relate to recent tool-use data synthesis works such as ScaleEnv [1], Agent World Model [2], and EnvScaler [3]? Please differentiate the mechanism (policy-derived state-transition graphs plus rarity-guided exploration) from these pipelines and clarify whether a direct empirical comparison is feasible on your benchmarks.

  5. (Clarification) Could you add at least one current-generation frontier model (e.g., GPT-5.5, Claude Sonnet 4.6, GLM-5.2, Kimi-K3 — the very teacher you already use — or MiniMax-M2.7) as a reference point in Table 1? Since GPT-5.2 has no AutomationBench numbers, the "surpassing the closed-source baseline" claim currently rests on one column of one benchmark against a stale snapshot.

  6. (Clarification) Are the Table 2 results stable across seeds? Given the per-category swings (e.g., M.Param −4.0 for the 4B model), would one or two flipped evaluation items change any subset-level conclusion?

5 comments
AuthorsOpenReview ↗

Response to reviewer kE64(1)

We sincerely thank the reviewer for their thoughtful and constructive comments. We appreciate the opportunity to clarify our methodology and provide additional experimental evidence to address the concerns raised.

1.What do GLM-5.3 and Kimi-K3 themselves score on BFCL-v4 and AutomationBench under your protocol, and what was the total teacher-side budget (tokens/API calls across task synthesis, trajectory generation, and rubric evaluation) for the RarePlay branch versus the TOOLACE/TOOLMIND branches under the matched 4K-trajectory budget?

We thank the reviewer for this insightful question. To address this concern, we first evaluated Kimi-K3 on the benchmarks under our evaluation protocol. The results are summarized below.

ModelBaseM.FuncM.ParamLongBFCL Avg.AutomationBench Pass
Kimi-K356.039.537.056.547.2524%
RarePlay-14B49.531.033.030.536.0013%

As shown in the table, Kimi-K3 achieves better overall performance than RarePlay-14B. However, it is worth noting that Kimi-K3 has approximately 2.5 trillion parameters, whereas RarePlay-14B is built upon a substantially smaller 14B Qwen3 backbone, containing only ​0.56% of Kimi-K3's parameter count​. Despite this considerable difference in model scale, RarePlay-14B achieves ​76.2% of Kimi-K3's performance on BFCL-v4 multi-turn​, demonstrating that our generated QA trajectories contain rich and informative supervision signals that enable a much smaller student model to effectively learn the teacher's tool-use behavior.

Regarding the teacher-side computational budget, we provide a comparison of the synthesis costs between RarePlay and ToolMind, normalized by the number of generated trajectories.

Metric (per trajectory)RarePlayToolMind
Input tokens142K57K
Output tokens1.1K4.7K
Total tokens143.1K61.7K
BFCL Avg. (4B)30.0024.25

We report the number of API calls across the three main stages of RarePlay's data synthesis pipeline, as summarized below.

StageAPI Calls
Task Synthesis≈11,500
Trajectory Generation≈109,000
Rubric Evaluation≈14,000
Total≈134,500

RarePlay incurs higher input-token costs, primarily due to repeated tool schemas and conversation histories. However, its output-token consumption is approximately ​4.3× lower than ToolMind's​, as executable environments replace LLM-simulated tool responses and enable deterministic state-based verification. Moreover, RarePlay achieves a ​5.75 point improvement on BFCL​, demonstrating stronger downstream performance despite higher input-token consumption.

Importantly, the teacher models used in RarePlay, including Kimi-K3 and GLM-5.3, have publicly available weights and can be deployed locally. In our implementation, one complete round of the data-generation pipeline takes approximately six hours. These results highlight a practical trade-off between synthesis costs and downstream effectiveness, demonstrating that RarePlay can effectively distill advanced tool-use capabilities into a compact student model.

AuthorsOpenReview ↗

Response to reviewer kE64(2)

2.The rubric scorer and the trajectory generator are the same model family (GLM-5.3, Table 5). How accurate is g_LLM against human judgment on a sampled set of trajectories, and how sensitive are the main results to the rubric threshold λ_r = 80 (Table 6)? A small human-audited agreement study would break the circularity.

We thank the reviewer for this valuable suggestion. To evaluate the reliability of rubric-based filtering, we randomly sampled 30 trajectories and conducted independent human evaluations across multiple rubric criteria for each trajectory. We then compared the human judgments with the corresponding gLLM evaluations at the rubric level. The results show a ​91.3% agreement rate, indicating strong consistency between automated rubric-based evaluations and human judgments. This provides empirical evidence supporting the reliability of our rubric-based filtering mechanism.

To further justify our choice of the rubric threshold λr=80\lambda_r=80, we examined trajectory retention rates under different threshold settings. The results are summarized below.

Threshold λr\lambda_rRetention Rate
7577.2%
8060.0%
8546.8%
9031.8%
958.0%

As shown in the table, increasing the threshold progressively reduces the number of retained trajectories. While lower thresholds retain more data, they impose less stringent quality requirements, whereas higher thresholds substantially reduce the available training data. We therefore adopt λr=80\lambda_r=80 as a practical compromise between filtering strictness and data availability, retaining 60% of generated trajectories while excluding lower-scoring samples.

3.Please report a data-count-matched w/o-RGE ablation: given the retention difference (17.2% vs. 60%, Table 3), how much of the 6.62-point average gap survives when both configurations train on the same number of accepted trajectories drawn from pools of the same size?

We thank the reviewer for this valuable suggestion. We would like to clarify that ​our original ablation study already adopts a data-count-matched setting​. Specifically, both the full RarePlay framework and the w/o-RGE variant are trained on exactly the same number of accepted trajectories. Given the different retention rates , we generate additional candidate tasks for the w/o-RGE variant until it reaches the same number of accepted trajectories as the full framework.

Therefore, ​the reported 6.62-point average performance gap is already measured under matched training data volumes​, rather than being attributable to differences in the number of training trajectories. Notably, the w/o-RGE variant is allowed a larger candidate generation budget to compensate for its lower retention rate, yet it still underperforms the full framework.

4.How does RarePlay's state-graph synthesis relate to recent tool-use data synthesis works such as ScaleEnv [1], Agent World Model [2], and EnvScaler [3]? Please differentiate the mechanism (policy-derived state-transition graphs plus rarity-guided exploration) from these pipelines and clarify whether a direct empirical comparison is feasible on your benchmarks.

We thank the reviewer for highlighting these relevant works. We believe RarePlay is largely orthogonal and complementary to ScaleEnv, Agent World Model, and EnvScaler. These approaches primarily focus on ​environment synthesis​, constructing diverse and executable tool-interaction environments. In contrast, RarePlay focuses on ​adaptive task and trajectory synthesis within existing environments​. Specifically, RarePlay constructs state-transition graphs from the current policy's rollouts and employs Rarity-Guided Exploration to prioritize underexplored yet successful transitions, enabling task synthesis that adapts to the evolving policy.

Since these works primarily differ in their environment construction mechanisms, a direct comparison on BFCL-v4 and AutomationBench would require adapting their synthesized environments and task-generation pipelines to our benchmark settings, rather than simply replacing RarePlay's synthesis strategy. Nevertheless, these approaches are complementary, and their generated environments could potentially serve as inputs to RarePlay's rarity-guided synthesis framework. We will clarify this distinction and discuss these works in the revised manuscript.

AuthorsOpenReview ↗

Response to reviewer kE64(3)

5.Could you add at least one current-generation frontier model (e.g., GPT-5.5, Claude Sonnet 4.6, GLM-5.2, Kimi-K3 — the very teacher you already use — or MiniMax-M2.7) as a reference point in Table 1? Since GPT-5.2 has no AutomationBench numbers, the "surpassing the closed-source baseline" claim currently rests on one column of one benchmark against a stale snapshot.

We thank the reviewer for this valuable suggestion. As detailed in our response to Question 1, we have supplemented our evaluation with results from current-generation frontier models, including Kimi-K3, on both BFCL-v4 and AutomationBench.

Notably, while RarePlay-14B still trails these frontier models in absolute performance, its substantially smaller parameter size and considerable improvements over the corresponding base model highlight the effectiveness of our approach in enhancing tool-use capabilities. We will incorporate these additional reference results into Table 1 and revise the corresponding claims to ensure a more comprehensive and fair comparison.

6.Are the Table 2 results stable across seeds? Given the per-category swings (e.g., M.Param −4.0 for the 4B model), would one or two flipped evaluation items change any subset-level conclusion?

We thank the reviewer for raising this concern. To assess the stability of our results, we conducted additional evaluations using two random seeds (43 and 48) on 4B model, alongside the original seed (42) reported in Table 2.

CategorySeed 42Seed 43Seed 48
Base47.046.043.0
M.Param16.019.021.5
M.Func28.023.527.5
Long29.029.026.0
Average30.0029.3829.50

The results demonstrate ​strong stability in overall performance​, with average accuracies of 30.00%, 29.38%, and 29.50% across the three seeds, respectively. The maximum difference is only ​0.625 percentage points.

We acknowledge that individual categories exhibit greater variability, particularly M.Param and M.Func, due to their relatively small evaluation sets (200 instances each). A change in one or two predictions can therefore affect the category-level scores by 0.5–1.0 percentage points. Nevertheless, the consistently close aggregate results suggest that our overall findings are robust across the evaluated seeds.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.