- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how action abstraction affects LLM agents in long-horizon environments. It introduces CodeHack, a fixed library of hand-written Python skills for NetHack and MiniHack, and compares agents using primitive actions, skills, or both within a shared runtime. The main contribution is a systematic empirical comparison of these interfaces in terms of performance, inference cost, and learning. Validation includes MiniHack tasks, a 14-model zero-shot evaluation on NetHack, PPO training on two models, and additional SFT and skill-removal experiments. The results show that supplied skills improve game progression and reduce inference cost, while skill-based controllers achieve larger gains over the evaluated RL budget. Mixed control retains most of these benefits and can help when skills are missing or insufficient, although it incurs higher inference costs than skill-only control.
Critical strengths
-
The paper compares primitive-only, skill-only, and mixed control within standardized execution propocol / run-time, the scale is for multiple model families and zero-shot and learning settings. This solidly provides a reasonably controlled of how differnt action interface affects agent performance.
-
The eval part takes account game progression + token usage + inference cost. Demonstrating substantial practical benefits from the supplied skills. The mixed-control and skill-removal experiments also provide fair evidence on the benefits and costs of primitive actions when the skill library is insufficient.
Critical weaknesses
I lean toward weak reject primarily because I don't find the work to be novel / interesting enough for the bar of ICLR. I enocurage the authors to clarify their contributions on top of related work. What are some deeply new insights that is relevant for the community and industry, yet existing literature did not cover? Note that the benefits of skills to LLM agents is a well-studied topic. To recommend acceptance on such topic, the paper has to introduce some important new insights.
-
Limited novelty. Prior work such as Voyager and NetPlay explored LLM control through executable skills. NetPlay also support skill interruption and low-level input interfaces in NetHack. The idea is not brand-new. The comparison and fine-tuning experiments provide additional evidence, but it feels to me that the paper does not clearly establish a substantial new insight beyond the expected benefits of supplying reliable skills.
-
Insufficient ablations to explain the improvements. The comparison bundles abstraction with expert knowledge and local control encoded in the skills. I see that the shared runtime helps control implementation differences, yet the experiments do not distinguish the sources of improvement. I feel comparisons with simpler action chunks, or a simple rule-based controller using the same skill library, would help explain what the abstraction / LLM controller contribute.
-
Limited eval of fallback under incomplete skill coverage. The skill-removal experiments use only one model; The detailed evidence for useful fallback consists of two GPT-5 trajs. Also, removing the navigation family removes staircase skills, so part of the skill-only performance drop follows from losing essential actions. Note that: these results demonstrate that primitive can help, but do not establish how reliably it do so.
Questions for authors
-
How much of the observed improvement comes from temporal abstraction, versus the expert knowledge and control logic encoded in the hand-written skills? Do you have controlled ablations that help distinguish these contributions?
-
Across evaluation episodes, how often does switching to primitives successfully resolve a failed skill? How much additional inference cost does this incur?
5 comments
Rebuttal (continued)
We also agree that removing the navigation family eliminates essential skills such as staircase navigation. However, this is precisely one of the situations we aim to investigate: when an essential behavior is no longer available through the skill library, can access to primitives help the agent compensate? Our results suggest that it can, although we agree that further experiments across more models and a more systematic analysis of fallback success rates would be valuable to establish its reliability.
Across evaluation episodes, how often does switching to primitives successfully resolve a failed skill? How much additional inference cost does this incur?
Quantifying fallback success rate and its incurred cost is challenging as switching to primitives does not indicate a failed skill. Agents also could switch preemptively making it difficult to identify when fallback occurs and when it succeeds. We agree that this is an important limitation of our current evaluation and will acknowledge it in the revised version of the manuscript.
We hope our responses clarify the novel insights of our study and that the planned ablations will further strengthen the empirical evidence. We would appreciate the reviewer's thoughts on whether these clarifications address their main concerns, particularly regarding the novelty and significance of our findings, and whether they would consider revising their assessment in light of our responses.
Thanks for the detailed response. I appreciate the clarification of the paper's intended contributions, particularly the trade-off between flexibility and efficiency and the quantitative evidence from the skill-removal experiments.
However, I think my main concern regarding novelty remains. While the identified trade-offs are useful empirical observations, I am not fully convinced that they constitute substantially new insights beyond existing work on skill-based agents and hierarchical control. The observations that additional action flexibility may increase decision complexity, and that primitive actions can compensate for incomplete skill coverage, seem largely consistent with existing intuitions.
I appreciate the proposed additional ablations and believe they could strengthen the paper, but would need to see the results before reassessing the experimental concerns.
That said, my assessment of weak rejection is unchanged
We thank the reviewer for their thoughtful and constructive feedback. We appreciate the recognition of the value of our study and agree that we could sharpen the presentation of our insights and implications of our findings. Below, we clarify our contributions and address the specific concerns raised.
We agree that code skills are not new, and that our paper could better highlight its key findings. While it may be intuitive that access to reliable skills improves zero-shot performance, it is considerably less obvious where skills land on the performance-cost frontier (RF3), how the benefits of skills translate to learning (RF5), and how they interact with the availability of primitive actions (RF4).
In particular, our experiments reveal several non-obvious insights: (1) Additional flexibility does not automatically improve performance: mixed control does not consistently outperform skill-only control (despite having a more expressive action space) and incurs higher inference costs (RF3). (2) Nevertheless, restricting flexibility can be harmful when skills are insufficient: access to primitives can help agents compensate for incomplete skill coverage (RF4). This tension highlights why neither skill-only nor mixed control is universally preferable, and why the design of action interfaces deserves systematic investigation. Moreover, if both excessive and insufficient flexibility have costs, learning when to operate at each level of abstraction becomes an important research direction. Our RL results (Appendix D.1) provide initial evidence in this direction, showing that training changes how agents distribute their actions across abstraction levels.
While Voyager and NetPlay demonstrate the usefulness of executable skills, neither systematically investigates these tradeoffs across different action interfaces and learning settings.
We thank the reviewer for these suggestions and agree that additional ablations would help clarify the sources of improvement. We are currently working on the action chunking, where the agent generates a sequence of primitive actions per model call, and a rule-based controller using the same skill library. We hope these experiments will help address the reviewer's concerns, and would appreciate their confirmation that this action-chunking setup aligns with what they envisioned.
We recognize that these baselines may not fully disentangle the effects of temporal abstraction from the knowledge encoded in the skills, but believe they will provide useful additional evidence about the sources of improvement.
We agree that our experiments do not fully establish how reliably agents can fall back to primitive actions under incomplete skill coverage. However, we would like to clarify that our evidence for the benefits of primitive fallback is not limited to the two GPT-5 trajectories. In Appendix C.1, we systematically remove five different skill families and evaluate both skill-only and mixed control over 64 episodes per condition. These experiments show that mixed control is generally less affected by missing skill families, providing quantitative evidence that access to primitives can help compensate for incomplete skill coverage. The two GPT-5 trajectories in Appendix C.2 serve as qualitative illustrations of how this fallback can occur in practice, rather than as evidence of its reliability. (continued on next post)