- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
ARETE trains compact language model agents through two forms of regulation: choosing between planning, executing, and reflecting on tasks, and adjusting practice tasks by utilizing the difference between independent and scaffolded performance. According to experimental reports from TextCraft and ScienceWorld, the success rate is higher and there are fewer ineffective operations compared to the selected baseline.
Critical strengths
1.Separating whether the reasoning mode is appropriate from whether it is executed correctly can address a specific weakness in outcome-only training.
2.The ablation of components also indicates that both reasoning regulation and adaptive practice contribute, with the former generating greater benefits.
Critical weaknesses
1.Lack of the closest baseline. RLVMR has combined structured reasoning behavior with verifiable process rewards. Although discussed in the paper, it was not evaluated, resulting in the benefits of the ARETE additional mechanism remaining unresolved.
2.The replicated GRPO baseline is worth carefully examining: Qwen2.5-3B only achieved 64% on TextCraft, while the proportions in AgentGym-RL and HPO papers were 75% and 81%, respectively. These settings may not be exactly the same, but the unexplained differences weaken the claimed improvement.
3.Matching the number of policy rollouts does not establish equal training resources. ARETE also uses teacher assisted demonstrations, scaffolded tasks, task generation, and simulator validation; These comparisons do not clearly separate algorithm benefits from these additional resources.
4.The broader generalization evidence is still limited to two environments. The provided text neither reports the differences between training seeds nor provides uncertainty estimates, making it difficult to assess the reliability of the benefits.
Questions for authors
1.How does ARETE compare to RLVMR under the same backbone, initialization, task split, and training budget?
2.What is the reason for the lower TextCraft GRPO results compared to previous reports? Has the original implementation and training plan been replicated?
3.Will the benefits continue when the baseline receives equal demonstration and scaffolded training data? Please also report the total training cost and results for multiple seeds.