- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces Skill2Env, a framework for automatically converting public Agent Skills into verifiable reinforcement learning (RL) environments. The authors argue that existing agentic RL environments primarily focus on technical workflows, such as software engineering, and lack broad coverage of real-world user needs and practitioner-authored workflows. To address this limitation, the paper leverages public Agent Skills as a source of domain expertise and task generation guidance. Through workflow decomposition, task diversification, environment construction, and automated verification, Skill2Env converts approximately 3,400 Skills into 7,971 executable training tasks spanning 13 domains. Experimental results show that RL training on Skill2Env improves the model's agentic terminal-use performance. In particular, the FlashREINFORCE variant achieves improvements of 9.0 and 7.1 percentage points on Terminal-Bench 2.1 and S2EBench, respectively.
Critical strengths
- The paper addresses an important and practically relevant problem. Scaling diverse and verifiable training environments is a significant challenge in agentic RL, and the proposed direction is relevant to improving general agentic capabilities.
- The environment generation pipeline is relatively comprehensive and has practical engineering value. The framework covers multiple stages of task generation and verification, and the paper provides detailed descriptions of its implementation and engineering design.
- The paper constructs a relatively large-scale and diverse training dataset. The resulting dataset contains nearly 8,000 executable tasks spanning 13 domains, providing a potentially useful resource for agentic RL research.
Critical weaknesses
-
Lack of critical controlled comparisons to validate the effectiveness of the core approach. The central claim of the paper is that leveraging practitioner-authored Agent Skills enables the generation of RL environments that better reflect real-world user needs and incorporate domain expertise. However, the authors do not provide a direct comparison between Skill-guided environment synthesis and conventional task generation approaches that do not rely on Skills. Consequently, the current experiments do not sufficiently establish the unique contribution of using Agent Skills as the source of training environments.
-
Insufficient independent validation of environment quality despite the large dataset size. Although the authors report generating nearly 8,000 training tasks and filtering them through automated checks, these checks primarily establish basic executability and consistency between the reference solution and programmatic tests. Passing such checks does not necessarily guarantee that the generated tasks are representative of real-world use cases, have appropriate difficulty, or are equipped with comprehensive and reliable evaluation criteria. Furthermore, the main RL experiments use only approximately 2,400 randomly sampled training instances, rather than the full dataset. The paper does not provide systematic scaling experiments to investigate how training performance changes with increasing dataset size. This limits the evidence supporting the value of the full generated corpus.
-
Limited experimental setup and insufficient evidence of generalization. The RL experiments are primarily conducted on a single base model, Qwen3.8-27B, without validation across different model families or parameter scales. In addition, the authors' private benchmark, S2EBench, contains only 79 tasks and is constructed from the same SkillHub ecosystem as the training data. Although these tasks are manually verified, the limited benchmark size and shared source ecosystem raise concerns about the representativeness of the evaluation and the generalizability of the reported improvements.
-
Insufficient and non-independent validation of the effectiveness of Skill-derived rubrics. Skill-derived behavioral rubrics are an important component of the proposed approach, but the corresponding experimental evidence is not fully convincing. First, the Outcome + Rubric RL checkpoint performs worse than the outcome-only RL checkpoint on the evaluated benchmarks, suggesting that incorporating rubric rewards does not provide a direct benefit in terms of objective task completion performance. Second, the reported improvements in behavioral alignment rely primarily on evaluations by an LLM judge rather than independent human assessments. The same judge is used for both rubric-based reward calibration during training and the subsequent pairwise Skill-alignment evaluation, potentially introducing evaluation bias. Furthermore, the paper does not provide a direct pairwise comparison of behavioral quality between Outcome-only RL and Outcome + Rubric RL. It also lacks systematic ablations on the rubric reward weight and alternative reward combination strategies. As a result, the current evidence is insufficient to fully establish the practical benefits of rubric-based supervision or characterize its trade-offs with outcome-based performance.
-
The presentation emphasizes system implementation and empirical results without sufficiently highlighting the central research question. In particular, the abstract primarily summarizes the scale of data generation, the RL training setup, and benchmark improvements, without clearly articulating the fundamental distinctions between the proposed approach and existing methods or the core research hypothesis being investigated. Similarly, much of the main text is devoted to describing the data generation pipeline, infrastructure, and implementation details, while relatively limited attention is given to in-depth experimental analyses of the key design choices.
Questions for authors
See above
2 comments
Thank you for the rebuttal and the additional experiments. I appreciate the new evidence on data scale, the independent evaluation of rubric-based results, and the direct comparison between Outcome RL and Outcome+Rubric RL.
However, one central concern remains unresolved. I raised the comparison against Skill-free generation because I wanted to verify whether the method’s core contribution indeed comes from practitioner-written Skills, as emphasized in the paper. However, the two additional datasets introduced in the rebuttal do not isolate the contribution of Skills themselves, as the datasets use different generation pipelines. Practitioner-written Skills serve as the upstream input to Skill2Env, but most of the subsequent environment-generation and verification pipeline does not inherently depend on Skills. I would like to understand whether the performance gains come from the Skills themselves or from Skill2Env's environment-generation pipeline.
We thank the reviewer for the detailed review. We ran new experiments for W1, W2 and W4 (Tables A and B in general comments).
W1. No comparison against Skill-free generation. Table A (in general comments) adds this comparison. Endless-Terminals and TMax-15K are two existing datasets synthesized without Skills. With the same checkpoint, recipe and budget, they improve TB 2.1 by 2.2 and 4.4 points and Skill2Env by 9.0. The gap on S2EBench is similar (−2.2 and +3.4 against +7.1). These two datasets also use different generators from ours, so Table A compares the datasets as a whole. To isolate the effect of the Skill itself, we would need to re-generate at least 2.4k tasks with the same pipeline but without Skill input and run RL again. We could not finish this in the rebuttal period and will state it as a limitation.
W2. Environment quality and scale. We agree that Oracle/NOP only shows a task is executable and consistent with its tests. For difficulty, GLM-5.3 averages 0.74 reward over 15,968 trajectories with a median of 19 model calls (Sec. 3.4), so the tasks are solvable but not trivial for a strong open model. For usefulness, the most direct check is downstream: in Table A, 2.4k Skill2Env tasks train a better agent than 2.4k tasks from either alternative. For scale, we added a run on the full corpus (Table A). S2EBench rises from 40.5 to 43.4 and TB 2.1 stays at the 2.4k level (57.8 against 58.4). This matches the training reward curve, which slows down after about 2,000 tasks. So the full corpus keeps helping on the broader benchmark while TB 2.1 saturates early.
W3. One model; same-source benchmark. The gain does hold under two different RL recipes (DPPO and FlashREINFORCE, Table 1). For S2EBench, it shares the pipeline with the training data but its Skills do not. We held out 100 Skills before training data synthesis, and the 79 tasks were generated from them and reviewed by 4 researchers. As for the size, 79 tasks is rather common for modern terminal-agent benchmarks. We also report widely-used TerminalBench 2.1 (89 tasks, of similar size as ours), and the gain there reflect a transferred general agentic ability. We choose Qwen3.8-27B as base model because it's commonly considered one of the strongest local LLMs, and it's a size widely used in existing data research. We did not add other model variants due to limitation of resource during rebuttal. We will state the single-model scope as a limitation.
W4. Rubric evidence. We agree that the rubric reward does not improve task completion, and the paper reports it this way (Sec. 5.2). What it changes is how closely the agent follows the Skill. Table B (in general comments) fills the two gaps the reviewer names. For independence, we re-judged with Fable 5.2, a model from another family that is used nowhere in generation or training, and it agrees with GPT-6 Astra on every row. For the direct comparison, both judges prefer the Outcome+Rubric checkpoint over the Outcome-only one (50.5 to 33.0 and 47.0 to 30.0).
W5. Presentation. Thanks for the suggestion. We include rich description for data pipelines and infrastrcutre details since they are the fundamental "methods" to reproduce experiments and artifacts in our approach. We will adopt the suggestion to rewrite the abstract and introduction around the research question: whether Skills written by practitioners are a excellent source of terminal agent RL environments.
Overall, we hope our explanation and the new results address the soundness concerns. High-quality RL data is scarce in the open community. Under the same recipe, Skill2Env gives the largest gain among the open terminal RL datasets we compared (Table A), which is crucial as base models become stronger and saturated to older datasets. We release SkillHub, full 7971 terminal agent RL dataset, and the synthesis pipeline with ablation results so researchers can continue post-training research on stronger baselines.