- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SimpleHRL, which uses a prompt-induced plan/execution structure during GRPO training to improve OOD generalization. The authors report gains on cross-domain benchmarks and attribute them to more transferable reasoning strategies and improved credit assignment.
Critical strengths
- The method is simple and easy to implement.
- The paper evaluates OOD generalization across multiple task domains and includes mechanistic analyses.
- This paper is easy to follow and overall well-presented
Critical weaknesses
- SimpleHRL generates substantially longer responses (2,531 vs. 1,525 tokens). The long-CoT baseline only increases length at inference time, which does not control for the effects of longer trajectories during RL training. Length-matched training, compute-matched comparisons, and truncation statistics are needed.
- More task-agnostic plan embeddings may simply reflect the generic language enforced by the prompt, rather than genuinely transferable reasoning strategies. The paper lacks interventions demonstrating that these strategies causally improve OOD performance.
- Experiments use only Qwen3-4B and Qwen3-8B. The results may depend on these models' existing structured-reasoning capabilities. Experiments on more model families would be helpful.
Questions for authors
Please see weaknesses