- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces ARISE. ARISE uses evidence from rollouts to dynamically create and maintain a set of rubrics that provide additional training feedback, rewarding partial progress. Rubrics also have paired skills that help the agent acquire the capabilities needed for success on them. Finally, ARISE also introduces an adaptive task sampling mechanism that favors tasks with applicable rubrics where the agent’ performance is not too low or high. Their experiments on SkillsBench and TerminalBench show benefits from the method.
Critical strengths
-
The presentation is good.
-
The idea of using an LLM to analyze past rollouts in order to generate rubrics, using those rubrics for additional feedback, dynamically adding and removing active rubrics as the agent capabilities evolve, and pairing rubrics with skills to help the acquisition of those capabilities all make sense to me. This part of the method seems reasonable and interesting.
-
The chosen methods the experiments compare to and the ablations seem appropriate. The additional results are also useful.
Critical weaknesses
-
The method relies on a stronger external model as the rubric judge and reflection model, which could limit the applicability of the method to frontier training. Do you think the method would work if the trained model was used for all roles?
-
Results seem to use a single training run for each method and ablation. Moreover, evaluations use only one benchmark for ablations and two for the main comparisons. While these decisions can be reasonable due to compute costs, some of the observed differences (mainly within the ablations) are relatively small, making the evidence for the contribution of each component less convincing.
Questions for authors
Of the components of the method, I found the task sampling component the least convincing.
- The method relies on predefined task types and capability groups. How much work did it take to develop these? Do you consider this a limitation of the method?
- Task selection seems to depend purely on performance on capabilities (rubrics) and not on performance on the tasks themselves. Is this desired? Why?
- A task seems to be more likely to be sampled when it belongs to several types and has applicable rubrics that cover several capabilities. Is this the case? Is this desired? Could this lead to a feedback loop where certain tasks are sampled more frequently, making the reflection model see more trajectories from them and potentially generate more rubrics targeting their behaviors, further increasing their sampling probability?
How was the hyperparameter selection procedure carried out for the method and the baselines? I did not find this information in the paper.
Secondary feedback and questions:
-
When a rubric's performance exceeds the low threshold, the corresponding skill is removed from the model’s context. Is that performance estimated from training rollouts that include skill guidance when the skill is active? If so, could removing the skill cause the performance to drop again into the region where the skill is added? Is there anything preventing this potential oscillation?
-
If you have the results, I think it would be nice to also have available the performance as training progresses for the ablations other than outcome only RL (Figure 3).
-
While the writing is generally good, I think section 3.2.2 could be clearer.
5 comments
Response to Reviewer bRt5 (Part 2 / 3)
Q1: Of the components of the method, I found the task sampling component the least convincing.
- The method relies on predefined task types and capability groups. How much work did it take to develop these? Do you consider this a limitation of the method?
- Task selection seems to depend purely on performance on capabilities (rubrics) and not on performance on the tasks themselves. Is this desired? Why?
- A task seems to be more likely to be sampled when it belongs to several types and has applicable rubrics that cover several capabilities. Is this the case? Is this desired? Could this lead to a feedback loop where certain tasks are sampled more frequently, making the reflection model see more trajectories from them and potentially generate more rubrics targeting their behaviors, further increasing their sampling probability?
Thank you for these thoughtful questions. We clarify the setup cost, sampling rationale, and potential feedback loop below.
1. Task categorization requires little setup effort, and capability groups are reusable. We consider the need to adapt the task taxonomy a practical limitation when transferring to new environments. However, our automated construction and annotation procedure takes approximately two hours on our corpus, reducing this setup cost. Moreover, the five general capability categories can be reused, and adapting the taxonomy does not require redesigning the sampler. We have added the construction and annotation procedure to Appendix C.2.
2. Rubric-based sampling tracks evolving capabilities beyond binary task outcomes. This choice is intentional: intermediate behaviors may improve while the final task outcome remains unchanged. Moreover, in a large training corpus, individual tasks may be revisited infrequently, so estimates based only on each task's outcomes can lag behind changes in the policy's capabilities. To address this, we aggregate rubric evidence by task type and capability, allowing observations from related tasks to update shared estimates and help the sampler respond to these changes. Consistent with this rationale, replacing our sampler with outcome-based VADE sampling (w/ VADE Sampling in Figure 3(a)) reduces performance on both benchmarks. Nevertheless, we agree that combining rubric feedback and task success rates for sampling is a reasonable extension, and we leave its investigation to future work.
3. Multiple associations increase sampling opportunities, but we constrain concentration and examine the proposed feedback loop. A task associated with more eligible task-type–capability pairs can indeed be selected through more paths. We consider this tendency reasonable because tasks spanning more workflows and capabilities offer diverse training opportunities and may therefore have greater learning value. To limit concentration, however, tasks cannot repeat within a batch, and each pair receives at most eight adaptive selections per batch, as specified in Appendix B.5. In addition, coverage-driven discovery prioritizes insufficiently evaluated tasks, while rubric retirement removes mastered criteria. Beyond these design constraints, we also examined whether frequently sampled tasks dominate rubric generation by measuring the association between each task's rollout count and the number of times its evidence is cited by newly generated rubrics. The resulting weak correlation (Spearman's ) indicates little association between sampling frequency and rubric source citations.
Q2: How was the hyperparameter selection procedure carried out for the method and the baselines? I did not find this information in the paper.
Thank you for pointing out this omission. ARISE and all baselines use the same validation split and learning-rate search space. We hold out 10% of the 1,728-task corpus across multiple domains and select each method's learning rate from {1e-6, 2e-6, 5e-6} based on its validation performance. Since each training run requires approximately two days on 32 H800 GPUs, we limit the search to learning rate and adopt the original papers' settings for other baseline-specific hyperparameters. Other ARISE-specific hyperparameters were fixed heuristically based on the mechanism design. We apologize for this omission. We have added details of the validation split to Appendix C.1 and the hyperparameter selection procedure to Appendix D.3.
Response to Reviewer bRt5 (Part 3 / 3)
Q3: When a rubric's performance exceeds the low threshold, the corresponding skill is removed from the model’s context. Is that performance estimated from training rollouts that include skill guidance when the skill is active? If so, could removing the skill cause the performance to drop again into the region where the skill is added? Is there anything preventing this potential oscillation?
Thank you for raising this concern.
1. Active skills are evaluated using rollouts that include their guidance. Yes, when a skill is active, its rubric pass rate is estimated from training rollouts containing that skill. After the skill is hidden, the rubric continues to evaluate subsequent behavior, and the skill can be reactivated if its pass rate returns to the activation region.
2. We observed no skill reactivation during training. Moreover, the case studies in Appendix F show that rubric pass rates improve after skill guidance is removed. The artifact-specification skill is hidden at step 30, after which its rubric pass rate rises from 34.3% to 66.0% at step 150. Similarly, the tool-response skill is hidden at step 60, and its rubric pass rate rises from 43.2% to 97.2% at step 100.
3. Periodic updates reduce rapid switching. Skill activation and hiding are decided every 10 training steps, as specified in Appendix B.2, using accumulated rubric evaluations to reduce sensitivity to individual rollout fluctuations. Together with the absence of observed reactivation, this indicates stable skill visibility during training.
Q4: If you have the results, I think it would be nice to also have available the performance as training progresses for the ablations other than outcome only RL (Figure 3).
Thank you for this helpful suggestion. We additionally report the training progress of all four component ablations on SkillsBench v1.1. As shown in the table below, ARISE outperforms every ablation at steps 50, 100, and 150, showing a consistent advantage across the evaluated training stages. Pass rates are averaged over three evaluations at each checkpoint. We have added these checkpoint results to Appendix L of the revised paper.
| Setting | Step 0 (%) | Step 50 (%) | Step 100 (%) | Step 150 (%) |
|---|---|---|---|---|
| Outcome-only RL | 23.4 | 26.4 | 32.2 | 34.5 |
| w/ Frozen Rubrics | 23.4 | 28.7 | 34.5 | 39.9 |
| w/o Skill Injection | 23.4 | 29.9 | 35.2 | 41.4 |
| w/o Adaptive Sampling | 23.4 | 29.1 | 37.2 | 42.5 |
| w/ VADE Sampling | 23.4 | 31.8 | 37.9 | 43.7 |
| ARISE | 23.4 | 34.5 | 41.4 | 45.6 |
Q5: While the writing is generally good, I think section 3.2.2 could be clearer.
Thank you for this helpful suggestion. We have refined Section 3.2.2 to make the sampling procedure easier to follow. Specifically, we clarify how task-type tags are constructed and assigned, improve the transitions between the existing equations, and provide clearer pointers to the implementation details in Appendices B.5 and C.2.
Response to Reviewer bRt5 (Part 1 / 3)
Thank you for recognizing the rationale behind dynamically evolving rubrics and pairing them with skills, as well as the clear presentation and appropriate baselines and ablations. We appreciate your detailed feedback and address each point below.
Thank you for this important question. Our additional experiment shows that ARISE remains effective without a stronger external judge or reflection model. We used Qwen3.5-27B for all three roles, keeping the judge and reflection model fixed at the initial checkpoint while updating only the policy. As shown in the table below, this variant approaches the performance of the original ARISE and outperforms both outcome-only RL and OnlineRubrics on both benchmarks, even though OnlineRubrics elicits criteria by comparing policy responses with reference responses from DeepSeek-V4-Flash. We have added these results to Appendix J.
Thank you for this valuable suggestion. Following your feedback, we added experiments with three training seeds and extended the ablation evaluation to Terminal-Bench v2.1.
1. ARISE's advantage over the baselines and strongest ablation persists across training seeds. Following your suggestion, we conducted additional experiments with three training seeds for ARISE, outcome-only RL, OnlineRubrics, and the strongest component ablation, ARISE with outcome-based VADE sampling (w/ VADE Sampling). Each checkpoint is evaluated three times on SkillsBench v1.1, and we report the mean and sample standard deviation across the three seed-level averages. As shown in the table below, ARISE maintains a higher mean score than both baselines and the strongest ablation, supporting its robustness to training variability and the contribution of capability-based sampling. We have added these results to Appendix E.2.
2. The component ablations show consistent benefits on both benchmarks. To assess whether the component contributions extend beyond SkillsBench, we additionally evaluated all ablation checkpoints on Terminal-Bench v2.1. As shown in the table below, full ARISE outperforms every ablation on both benchmarks, supporting the contributions of dynamic rubric evolution, skill guidance, and capability-based sampling. Moreover, replacing our sampler with VADE sampling reduces performance on both benchmarks, providing further evidence for the value of sampling based on behavioral feedback. We have updated Figure 3(a) in the main paper to include the Terminal-Bench ablations.