- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes SPLICE to address the all-fail group problem in GRPO. A teacher model selects splice points in failed trajectories and generates bounded options, which are evaluated through student continuations to construct return-weighted local supervision. SPLICE is evaluated on two long-horizon agent tasks and outperforms the evaluated baselines. Ablation studies further examine the effectiveness of its key components.
Critical strengths
- The paper addresses the all-fail group problem in GRPO and proposes a method to recover learning signals.
- The method combines bounded teacher options with student continuations to provide local supervision.
- The proposed method outperforms the evaluated baselines on two agentic tasks.
Critical weaknesses
- SPLICE may be less effective on trajectories with multiple bottlenecks, since a useful local intervention can still be followed by failure at a later bottleneck.
- Under a limited continuation budget, options with low but nonzero success probabilities may be incorrectly discarded.
- The ablations do not separate direct teacher completions from learner-assisted recoveries, leaving the contribution of learner-conditioned evaluation unclear. When the teacher completes the task within an option, the benefit may come from supervised learning on that successful option.
Questions for authors
- The criteria and mechanism used by the teacher to select splice points need further clarification.
- A comparison using student-generated options from the same splice states, under a matched rollout budget, would help establish the benefit of teacher guidance.