- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Sleep-Time Training (STT), which combines supervised fine-tuning on corrected demonstrations with group-relative reinforcement learning over original and iteratively repaired trajectories to improve agents from deployment experience during user-idle periods.
Critical strengths
-
This paper addresses an important problem: helping deployed agents learn from past interactions and improve on future tasks.
-
This paper introduces STT-Bench to evaluate improvement across days, transfer to unseen tasks, and adherence to user preferences.
Critical weaknesses
-
“Sleep-time” and “user-idle periods” describe a practical deployment setting, but the underlying approach appears to be periodic training between interaction sessions. It is unclear whether this setting introduces a distinct research contribution.
-
The method appears to distill knowledge and behavior from a stronger model through SFT demonstrations, trajectory corrections, and GRPO reward judgments. The contribution beyond this teacher-guided training pipeline is unclear.
-
Evaluation is limited to the proposed STT-Bench, which uses a single synthetic persona and simulated tools. Results on established agent benchmarks would help assess whether the improvements generalize beyond this setting.
Questions for authors
- How does STT differ from simply distilling a stronger model’s knowledge through SFT/GRPO? A more explicit comparison with existing approaches would help clarify the contribution.
- All experiments use STT-Bench. It would be helpful to include results on established benchmarks to see whether the gains extend beyond the proposed setting.
- On page 4, the paper states that 'C', 'P', and the overall score are reported separately, but I could only find the overall scores. Please include the separate results, especially 'P', so that improvements in preference adherence can be assessed directly.
- The SFT trajectory correction stage uses deterministic task and preference checks. Why are these checks not also used to provide rewards during GRPO? Have the authors compared these two reward sources?