- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RarePlay synthesizes multi-turn tool-use data from state-transition graphs built from current-policy rollouts. Its Rarity-Guided Exploration (RGE) sampler combines task success, low visit counts, tool diversity, and a cross-domain penalty. Teacher-generated tasks and trajectories undergo rubric-based filtering before iterative supervised fine-tuning under a predefined hop-count schedule. The method improves BFCL performance for 4B and 14B models and reports additional gains on AutomationBench.
Critical strengths
- The comparisons support the synthesis pipeline’s effectiveness. With matched training-example counts and the same 4B backbone, RarePlay achieves 30.00 on BFCL versus 24.25 for ToolMind. The 14B model improves from 28.50 to 36.00, while AutomationBench provides an additional evaluation setting (§5.3, Table 2).
- The component ablations provide useful evidence. Removing RGE or rejection sampling reduces BFCL performance from 30.00 to 23.38 and 24.75, respectively. Removing RGE also lowers trajectory retention from 60% to 17.2% (§5.4, Table 3), supporting the value of guided sampling and filtering.
Critical weaknesses
- The rare-behavior explanation lacks direct validation. The “w/o RGE” ablation replaces all four terms in Eq. 1 with random sampling, so it cannot isolate rarity from success preference or the other components. More importantly, the paper does not quantify whether targeted low-visit behaviors survive teacher regeneration and filtering. The rubric is designed to preserve these behaviors, but this does not establish their actual retention. The end-to-end gains therefore support the overall pipeline more directly than its targeted-coverage explanation.
- The filtering rule does not explicitly enforce all essential task outcomes. Section 4.3 combines behavioral coverage and result correctness into a weighted score, accepting trajectories that score above 80 (Appendix D). A trajectory could therefore fail a mandatory outcome worth 10 points and still pass with 90. It is unclear whether a separate hard success check prevents this. This is a potential gap in the stated rule, not an observed failure in the training data.
Questions for authors
- Behavior retention: Using saved graph counts, sampled paths, and teacher trajectories, could the authors quantify how much targeted low-visit behavior remains after regeneration and filtering? Please define the coverage measure, account for path length and task success, and compare with existing random-sampling records where available. This would test whether the intended behaviors enter the training data, not establish their independent causal effect on accuracy. No additional multi-round training is requested.
- Filtering correctness: Are mandatory task outcomes checked separately from the aggregate rubric score? If not, could the authors audit a small sample of accepted trajectories, such as 30–50, against actual task requirements and recorded sandbox states? Please describe the sampling procedure and report unmet essential outcomes rather than relying solely on the original rubric scores.
7 comments
Response to reviewer ftoe(2)
3.Filtering correctness: Are mandatory task outcomes checked separately from the aggregate rubric score? If not, could the authors audit a small sample of accepted trajectories, such as 30–50, against actual task requirements and recorded sandbox states? Please describe the sampling procedure and report unmet essential outcomes rather than relying solely on the original rubric scores.
We thank the reviewer for this valuable suggestion. To further assess the correctness of accepted trajectories, we randomly sampled 30 trajectories and manually verified their final sandbox states against the original task requirements. Among them, 26 trajectories (86.7%) satisfied all mandatory outcomes, while 4 (13.3%) did not fully complete the corresponding tasks.
We would like to clarify that RarePlay does not treat complete task success as the sole criterion for determining the value of a training trajectory*. Our primary objective is to expose the student model to underexplored yet meaningful tool-use behaviors, rather than exclusively collecting trajectories that successfully complete every task requirement. As described in Section 4.3, our rubric-based filtering jointly considers the preservation of targeted intermediate behaviors and final-state consistency.
Consequently, trajectories that do not fully satisfy all task-level requirements may still contain valid and informative tool interactions that provide useful supervision for SFT. This distinction is particularly relevant to RarePlay, as its rarity-guided exploration mechanism is designed to improve the coverage of underrepresented tool-use behaviors. The aggregate rubric therefore reflects both behavioral coverage and task-level correctness, rather than imposing complete task success as an absolute prerequisite for acceptance.
Dear Reviewer,
We hope our responses have addressed your concerns. We would appreciate it if you could take another look and let us know if you have any further questions.
Thank you!
We sincerely thank the reviewer for their thoughtful and constructive comments. We appreciate the opportunity to clarify our methodology and provide additional experimental evidence to address the concerns raised.
1.The rare-behavior explanation lacks direct validation. The “w/o RGE” ablation replaces all four terms in Eq. 1 with random sampling, so it cannot isolate rarity from success preference or the other components.
We thank the reviewer for this valuable suggestion. To better understand the contribution of each component in RGE, we conducted additional ablation experiments on qwen3-4B by removing each term individually while keeping the others unchanged. The results are presented below.
The results show that all four components contribute positively to the overall performance of RGE. Removing the rarity and success terms leads to the largest average performance drops, highlighting the importance of prioritizing underexplored yet successful transitions. The diversity term encourages diverse tool usage and contributes to generating more informative trajectories, while the transition penalty helps maintain coherent state transitions. Both components play complementary roles in improving the overall effectiveness of RGE.
These findings demonstrate that the four components play complementary roles, with rarity and success being particularly important for effective exploration.
2.Behavior retention: Using saved graph counts, sampled paths, and teacher trajectories, could the authors quantify how much targeted low-visit behavior remains after regeneration and filtering? Please define the coverage measure, account for path length and task success, and compare with existing random-sampling records where available. This would test whether the intended behaviors enter the training data, not establish their independent causal effect on accuracy. No additional multi-round training is requested.
We thank the reviewer for this constructive suggestion. We conducted an additional analysis using the saved state-transition graphs, sampled paths, and teacher-generated trajectories to examine whether RGE-targeted rare behaviors are preserved throughout the synthesis pipeline.
We identify rare transitions based on their traversal counts in the policy-derived state graph. We measure coverage as the proportion of targeted rare transitions represented in the generated trajectories, and density as the proportion of rare transitions among all transitions in the final training corpus. The density measure normalizes for trajectory length, while the final-corpus analysis includes only trajectories that pass the rubric-based filtering process.
Our analysis yields three key findings:
These results demonstrate that RGE's preference for underexplored behaviors is effectively propagated through teacher regeneration and filtering, leading to substantially greater representation of targeted rare behaviors in the final training data.