- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how to select high value training data from trajectories across different tasks and proposes ACE-Sampler, a hierarchical random sampling framework. The method measures data diversity through modeling at both the trajectory and turn levels. It then combines importance ranking, validity filtering, and difficulty estimation to select training samples in a hierarchical manner. Experimental results show that ACE-Sampler achieves better overall performance than random sampling across three tool use benchmarks under a fixed budget of training iterations.
Critical strengths
-
The paper addresses the problem of data sampling across different tasks and has practical significance. Unlike traditional sampling methods based on semantic similarity, the proposed approach also considers the distinctive characteristics of tool calls. This makes it better suited to agent tasks involving multiple turns.
-
The paper proposes a hierarchical stochastic sampling framework that combines behavioral diversity at the trajectory level with importance at the turn level. It also incorporates sample validity filtering and model aware difficulty estimation to account for multiple key factors in data selection. Through hierarchical sampling, the framework can capture diverse behavioral patterns across trajectories while prioritizing turns with greater learning value within each trajectory.
-
The paper evaluates ACE-Sampler on three benchmarks using three different models. It also compares ACE-Sampler with various data sampling methods based on quality, difficulty, and diversity. Overall, the experimental evaluation is relatively comprehensive.
Critical weaknesses
-
The main contribution of the paper is the hierarchical stochastic sampling framework, which performs data sampling at both the trajectory and turn levels. However, the paper lacks ablation studies that examine the effectiveness of this mechanism. For example, it does not evaluate the performance of using only turn level sampling or only trajectory level sampling.
-
Table 5 shows that the final 5,000 selected training turns are heavily concentrated in a few data sources. This raises the question of whether the performance gains mainly result from the sampler favoring data sources with higher quality or better alignment with the target benchmarks. Could these gains be driven by such a preference rather than by the proposed behavioral diversity modeling and hierarchical sampling strategy?
-
ACE-Sampler shows only marginal improvements over the strongest baselines on Qwen3-1.7B and Qwen3-8B, with almost negligible performance gains.
Questions for authors
-
The paper emphasizes that its behavioral representation captures action ordering and dependencies, but computes similarity using Jaccard similarity between factor sets. Could the authors clarify how action ordering and dependency information are encoded in these sets?
-
In Algorithm 1, the variable k is incremented at both lines 19 and 24, which seems inconsistent with its definition in the main text as the number of selected turns from the current trajectory.
3 comments
Q1: Could the authors clarify how action ordering and dependency information are encoded in these sets?
A: Each turn in a trajectory is annotated with two factors: turn function, which describes the type of action performed, and interaction relation, which captures its dependency on previous interactions. For example, turn-function labels include "acquire_information" and "request_clarification", while interaction-relation labels include "uses_earlier_tool_result" and "uses_user_followup". The complete vocabulary is provided in Appendix A.3 (Annotation Prompt).
At the trajectory level, the turn-function and interaction-relation labels are aggregated across all turns to form the turn-function pattern and dependency pattern, respectively. These trajectory-level factors capture the overall action and dependency patterns within a trajectory, while the corresponding turn-level annotations preserve the information associated with individual actions.
Q2: In Algorithm 1, the variable k is incremented at both lines 19 and 24.
A: Thanks for pointing that out. It is a typo and line 24 should be removed. We have revised it in the revision.
Dear Reviewer,
As the discussion period is approaching its end, we kindly invite you to review our responses and let us know whether your concerns have been adequately addressed. If so, we would greatly appreciate your consideration of updating your score. Otherwise, we would be happy to continue the discussion and clarify any remaining questions.
Thank you for your time and valuable feedback!
We thank the reviewer for the constructed reviews! We address the corresponding questions below and all suggested additional experiments have been added in the revision, highlighted with blue color.
W1: The paper lacks ablation studies that examine the effectiveness of hierarchical sampling.
A: Thank you for pointing this out. We explored several method variants during the development of our final approach. Due to space limitations, we did not include all of them in the main paper, instead focusing on the key ablations concerning accuracy, complexity, and diversity. Here, we present the additional ablation on the hierarchical sampling mechanism. The results are as follows:
Here, "Only Trajectory-level" selects trajectories using the same trajectory-level sampling method, but randomly samples at most three turns within each selected trajectory. "Only Turn-level" directly selects turns from all candidate turns based on turn diversity, importance ordering, and complexity scores.
As can be seen, both trajectory-level-only and turn-level-only sampling yield lower performance than the full method, particularly trajectory-level-only sampling, which performs substantially worse due to random turn selection. The results validate the effectiveness of hierarchical sampling.
W2: There is a question of whether the performance gains mainly result from the sampler favoring data sources with higher quality or better alignment with the target benchmarks.
A: Thank you for raising this important question. To address this concern, we conducted additional experiments. According to Table 5, EnvFactory accounts for the largest proportion of the selected data. Therefore, we conducted two additional experiments by applying our method and random sampling exclusively to the EnvFactory dataset. To further examine the effect of data source composition, we constructed another dataset with exactly the same source-wise sampling quotas as ACE-Sampler in Table 5, but randomly selected turns within each source quota. The results are shown below (using Qwen3-4B as the base model):
Based on these results, we draw the following observations:
EnvFactory appears to have higher average data quality, as random sampling exclusively from EnvFactory outperforms random sampling across all sources.
The improvements of our method cannot be explained solely by its preference for EnvFactory or its source composition. Both applying our method exclusively to EnvFactory and random sampling with identical source quotas yield lower performance than the full ACE-Sampler. These results suggest that our method effectively selects informative samples across the seven datasets, beyond simply favoring particular data sources.
Regarding potential alignment with the target benchmarks, our sampling procedure does not explicitly use any information from the target benchmarks to guide data selection. Therefore, the observed improvements are more likely attributable to the effectiveness of the proposed sampling mechanism rather than explicit benchmark-specific alignment.
W3: ACE-Sampler shows only marginal improvements over the strongest baselines on Qwen3-1.7B and Qwen3-8B.
A: The comparison should be evaluated across model scales and benchmarks rather than by the strongest result of an individual baseline at one scale. A single-scale advantage does not demonstrate that a selector generalizes across learners or evaluation settings. For example, ATLaS is close to our method on Qwen3-1.7B, but its performance is substantially weaker on Qwen3-4B and Qwen3-8B. Similarly, D3 is particularly strong on Qwen3-8B, while its results are weaker at the other two model scales. Neither baseline exhibits a consistent advantage across the benchmarks.
In contrast, our method provides the most stable cross-scale behavior. It achieves the best aggregate Overall score for all three base models, and it leads (or the second best) the average results for each benchmark at every model scale. Thus, the meaningful result is not a marginal improvement at one model size, but the consistent top-ranked aggregate performance across all model scales and the three benchmarks.