- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces ATLAS, a framework for LLM-guided hierarchical reinforcement learning that introduces an explicit world-law layer between language-model reasoning and skill learning. An LLM constructs and revises environmental laws in BC+, while independently constructed symbolic and environment-level tests constrain their acceptance. A deterministic compiler extracts skill prerequisites, effects, resource costs, and dependencies, which are grounded through separately trained RL policies and a curriculum that revisits downstream skills as prerequisites improve. Execution trajectories and controlled interventions support further law acquisition and revision. Experiments demonstrate improved agreement with a fixed behavioral test suite, successful execution of long prerequisite chains in Craftax-Classic, and an illustrative acquisition of unfamiliar rules in Mars.
Critical strengths
- The separation between world modeling and skill specification is well motivated. ATLAS maintains a shared BC+ theory and deterministically derives skill prerequisites, effects, resource consumption, and dependencies. This provides a coherent way to reuse environmental knowledge across skills and identify which specifications need updating when a law changes. In particular, retaining the supporting laws for each specification gives the proposed revision process a useful structure.
Critical weaknesses
- The experiments do not isolate the benefit of the central architectural choice. The main argument is that learning world laws and deriving specifications deterministically is more reliable than having an LLM generate specifications directly. However, there is no comparison that holds the surrounding system fixed while changing the source of the specifications.
- The reported results have several evaluation concerns. In Craftax, ATLAS reports the best evaluation within a training task rather than a separate final evaluation, and its results come from a single training run, whereas CODE-SHARP reports averages across five runs. The equivalence of their survival settings also needs clarification. In Mars, ATLAS and the baseline are evaluated under different world configurations and procedures, making the reported performance improvement difficult to interpret. These inconsistencies weaken the empirical evidence for ATLAS's superiority.
Questions for authors
-
Does the world-law representation improve over direct specification generation under matched conditions?
-
Do the principal performance gains survive a matched evaluation protocol?
Dear Reviewer 97xq,
Thank you again for your time and constructive feedback.
As approximately two days remain in the author–reviewer discussion period, we would be most grateful if you could take a moment to review our rebuttal. In it, we have addressed your concerns through additional analyses, experiments, and clarifications.
We would be happy to clarify any remaining questions and welcome your further feedback.
Thank you for your time and consideration.
Best regards, The Authors