- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an agent-guided framework for policy learning and skill composition to improve generalization to out-of-distribution settings and long-horizon tasks. Given demonstrations, the agent segment (i.e., skills) and proposes multiple policy variations to improve generalization. Each policy is trained and evaluated, with its design, description, and evaluation results stored for future use. During inference, the agent selects and combines appropriate policies based on the task and current state. Experiments on simulated manipulation tasks demonstrate improved out-of-distribution generalization and performance on long-horizon tasks.
Critical strengths
- The paper demonstrates that agents can automatically design policies that substantially improve performance. This is an impactful finding, as manually designing such policies can be inefficient and challenging. Furthermore, the proposed skill composition framework demonstrates that these policies can be combined to solve long-horizon tasks rather than only individual skills.
- The paper clearly explains each component and its role step by step, providing a useful foundation for future research. The ablation studies help clarify the contribution of each component, while the detailed prompts support reproducibility and further research on agent-driven policy learning.
Critical weaknesses
Scalability under limited resources: The proposed method requires training multiple policies, which can be computationally expensive, especially when using large models such as VLAs. Furthermore, policy verification requires additional rollouts, which can be costly in real-world settings. How could the proposed framework scale under limited computational and physical resources?
Questions for authors
- How does the framework determine whether each policy has converged? Different policy designs and representations may require different training budgets, making convergence difficult to assess and compare fairly.
- Could the authors provide concrete examples of the "variations it is expected to tolerate" the prior description ? Does it have prior knowledge of the expected evaluation environments or distribution shifts? Also, understanding what 'known limitation' was hard for me. Concrete examples of the generated interface would help readers better understand how this information is determined and used.