- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper uses GRPO to train a single LLM agent on ALFWorld, WebShop and DBBench at the same time. For every rollout, an LLM judge assigns rubric labels drawn from one common vocabulary, and these labels serve to choose prompt groups, to give context to a self-distillation teacher, and to guide the training prompts of the following round. Across two Qwen backbones, RISED reaches the best overall pass rate.
Critical strengths
- Because all environments use one common rubric vocabulary, rollouts collected in different environments can be compared directly, and a single labeling pass feeds selection, distillation and steering at once (Algorithm 1).
Critical weaknesses
-
The work offers little novelty, since each of its components closely matches an earlier method. Skill-SD (arXiv 2604.10674) already proposes the distillation objective together with a teacher that receives summaries built from rollouts, RuPI (arXiv 2608.02948) already gives rubrics to the teacher as privileged context, and the selection score takes the MMR form previously applied to embeddings (Sec. 2.4). The only new element is choosing samples by their rubric profiles.
-
No experiment measures the effect of rubric-based selection by itself, and this is the only new component. RISED improves on the strongest baseline by 1.2 and 0.6 points (Table 2), and I did not find a variant in which distillation and steering stay in place while Eq. 1 is swapped for variance-based or random selection. The component ablation is run with a single seed (Sec. 5).
-
RISED is compared with the baselines on unequal terms, since it gets knowledge from stronger models. Gemini-2.5-Pro writes the rubric descriptions from a set of rollouts that contains successful ones by Claude-Opus-4.6 (App. A.1), and these descriptions are inserted into both the training prompts and the teacher context (Sec. 3), whereas GRPO and the variance selectors have access to the task prompt alone.
-
The method is difficult to understand, since the main text has no diagram of it. Every one of the four figures in the main text plots training curves, and the overview and the pseudocode are placed only in the appendix (Figure 5, Algorithm 1).
Questions for authors
- What fraction of the self-distillation signal goes to the zero-variance groups that are given as its motivation? The paper motivates distillation with the 62.9% of candidate groups whose rewards are all the same (Sec. 2.4), yet a group with no positive rubric gets no teacher context (App. E.6), and the judge produces positive rubrics only for successful rollouts. Could the authors report, among the selected zero-variance groups, how many are all-success and how many are all-failure, and what fraction of rollouts have a non-empty teacher context?
4 comments
RISED is compared with the baselines on unequal terms, since it gets knowledge from stronger models. Gemini-2.5-Pro writes the rubric descriptions from a set of rollouts that contains successful ones by Claude-Opus-4.6 (App. A.1), and these descriptions are inserted into both the training prompts and the teacher context (Sec. 3), whereas GRPO and the variance selectors have access to the task prompt alone.
All methods are evaluated on held-out tasks using the same original task prompts. RISED uses rubric guidance and privileged teacher context during training only. This follows the self-distillation setup of the two works cited in the review: Skill-SD uses an auxiliary model, Seed1.8, to summarize agent rollouts into skills. RuPI uses task-specific rubrics supplied by its datasets, including physician-authored rubrics on HealthBench. In each case the policy-derived teacher produces the token-level targets and the external information supplies privileged context (Wang et al., 2026, and Bablani et al., 2026). RISED likewise uses privileged context during training and evaluates the policy on the original task generations.
To compare with a baseline that also has access to this vocabulary during training, we have included All-Rubric SD, which conditions its teacher on annotations from the same vocabulary. RISED achieves higher overall validation scores than All-Rubric SD on both backbones, supporting the benefit of how RISED uses the shared annotations.
The method is difficult to understand, since the main text has no diagram of it. Every one of the four figures in the main text plots training curves, and the overview and the pseudocode are placed only in the appendix (Figure 5, Algorithm 1).
We agree that readers should be able to see a full RISED pipeline in the main text. In the revision, we moved the full training procedure (Algorithm 1) and the pipeline diagram (Figure 1) into the main text. The performance curves report held-out validation pass rates. RISED and the baselines are evaluated using the same original task prompts, with rubric guidance restricted to training. We make this evaluation protocol explicit in the figure captions (Sec. 3 and Fig. 2).
What fraction of the self-distillation signal goes to the zero-variance groups that are given as its motivation? The paper motivates distillation with the 62.9% of candidate groups whose rewards are all the same (Sec. 2.4), yet a group with no positive rubric gets no teacher context (App. E.6), and the judge produces positive rubrics only for successful rollouts. Could the authors report, among the selected zero-variance groups, how many are all-success and how many are all-failure, and what fraction of rollouts have a non-empty teacher context?
Across 50 sampled training steps from one Qwen2.5-3B RISED run, 62.4% of selected prompt groups had identical rewards. Among those groups, 45.4% were all-success, 31.3% were all-failure, and 23.3% had equal partial rewards. 70.6% of rollouts in these groups had positive teacher context. Coverage was 100% for all-success and equal-partial-reward groups and 6.1% for all-failure groups. Of all selected prompt groups, 18.3% had identical rewards and no positive teacher context for any rollout. Negative tags can still guide subsequent rollouts (App. C.10, Table 12).
Overall, the revision brings the pipeline diagram and Algorithm 1 into the main text, clarifies the component ablations in Sec. 5, adds the variance-plus-feedback control in App. C.9, and reports teacher-context coverage in App. C.10 and Table 12.
Thank you for the detailed response and the new experiments. The coverage statistics answer my question, and moving the method overview into the main text improves readability. However, the new control in App. C.9 shows that rubric-based selection, the only new component, adds 0.41 points over Round-robin Variance with distillation and steering kept fixed, from a single run, which is within the variance of the baselines. The novelty concern also remains
Thank you for acknowledging that the coverage analysis answers your question and that the revised overview improves readability. We would like to clarify the characterization of rubric-based selection as “the only new component.”
RISED adopts Skill-SD’s loss and further designs how rollout-derived rubrics supply supervision in joint multi-environment training. We assign positive and negative rubrics distinct roles: demonstrated behaviours condition the teacher, while violations guide subsequent rollouts. App. C.3 evaluates these choices with rubric selection retained. Over the last ten evaluations, all-rubric teacher conditioning scores 49.68%, positive-only conditioning without steering scores 53.44%, and full RISED averages 53.75%. Although full RISED and positive-only conditioning reach similar late-training performance, the figure shows faster early learning with negative-rubric steering, most clearly on DBBench. These comparisons support using positive rubrics for teacher conditioning and negative rubrics for rollout guidance.
The combination of selection and distillation follows from the training problem. Shared rubric profiles make behaviours comparable across environments for selection, and the same annotations can supply teacher context when selected groups have identical rewards. Our contribution therefore includes the design and evaluation of reusing shared rubric annotations for selection and supervision.
The new comparison is a selector-replacement ablation: it replaces rubric ranking with Round-robin Variance while retaining RISED’s positive-rubric teacher conditioning and negative-rubric steering. It assesses the incremental effect of ranking within a pipeline that already incorporates our supervision design, which App. C.3 evaluates separately.
We hope these clarifications help you reassess the scope and novelty of our contribution.
Thank you for your detailed feedback and for recognizing our design of the rubrics twice. We have addressed the weaknesses and questions as follows:
RISED is designed for joint training across heterogeneous, multi-turn environments. Each update draws on trajectories from several environments. This exposes two limitations of scalar rewards, which motivate our two contributions.
Concurrent RuPI (Bablani et al., 2026) uses task rubrics for open-ended generation. All-Rubric SD adapts its teacher conditioning to interactive, multi-turn, multi-environment RL using RISED’s rollout-derived vocabulary. Its final-checkpoint gains over GRPO-64 on both backbones support the use of rollout-derived rubrics for distillation in this setting (Sec. 4.2 and Table 2).
To our knowledge, RISED is the first to reuse the same rollout annotations for online prompt-group selection and for conditioning an on-policy self-distillation teacher in joint multi-environment agent training.
To make the effect of rubric-based selection by itself clearer, we added GRPO-64 to Fig. 3 (Fig. 2 in the original submission). The purple curve keeps rubric-based selection and disables distillation and steering. Over the last ten evaluations, it averages 51.87%, compared with 48.15% for GRPO-64 and 50.14% in a separate random-subsampling run with distillation and steering disabled. Both selection runs target 128 candidates and update on 64 (Sec. 5 describes the component ablation). The green curve trains on the standard 64-group batch with rubric ranking disabled and retains distillation and steering. Its last-ten average is 51.27%, compared with 53.75% for full RISED (Fig. 3 and Sec. 5).
To further isolate the contribution of rubric-based ranking, we replaced it with Round-robin Variance while holding distillation and steering fixed. On Qwen2.5-3B, the hybrid averaged 53.34% over the last ten evaluations in one completed run. Round-robin Variance alone averaged 52.30%, and full RISED averaged 53.75%, an observed lead of 0.41 percentage points over the hybrid (App. C.9).
↓ Continued below ↓