acceptodds
Under review as a conference paper at ICLR 2027

Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents

Abstract

Language agents struggle to act and learn in environments that require long sequences of low-level actions. Code-based abstractions can make these agents more productive by letting them invoke reusable skills instead of repeatedly selecting individual actions. The code handles recurring local decisions, while the language model decides which skills to use and how to combine them. Yet abstractions are leaky, and situations beyond a skill's capabilities may require a return to primitive actions. Motivated by this tradeoff between productivity and flexibility, we systematically study how code-based action abstraction affects the performance, inference cost, and learning of language agents. We study this in NetHack, a challenging, long-horizon game environment, using CodeHack, our library of code-based skills with natural-language descriptions. We use this library to compare agents restricted to primitives with those using semantic skills alone or in combination with primitives. We evaluate these agents in three settings: zero-shot prompting, supervised fine-tuning, and reinforcement learning. Across a broad zero-shot evaluation on NetHack, we find that compared with primitives, skills nearly triple game progression, while reducing inference cost per episode by 86%. Combining skills with primitives retains much of this benefit while preserving a path back down to low-level actions. Finally, in RL, we find that skill-based agents learn significantly faster than agents acting on primitives, achieving a 7.2x larger average gain in dungeon level over the same training budget. These results show that a supplied skill library can improve performance, efficiency, and learning, while retaining primitives provides flexibility when the library is insufficient. We release CodeHack together with training and evaluation code.

Primary area:
reinforcement learning
Keywords:
reinforcement learning, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eEoUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how action abstraction affects LLM agents in long-horizon environments. It introduces CodeHack, a fixed library of hand-written Python skills for NetHack and MiniHack, and compares agents using primitive actions, skills, or both within a shared runtime. The main contribution is a systematic empirical comparison of these interfaces in terms of performance, inference cost, and learning. Validation includes MiniHack tasks, a 14-model zero-shot evaluation on NetHack, PPO training on two models, and additional SFT and skill-removal experiments. The results show that supplied skills improve game progression and reduce inference cost, while skill-based controllers achieve larger gains over the evaluated RL budget. Mixed control retains most of these benefits and can help when skills are missing or insufficient, although it incurs higher inference costs than skill-only control.

Critical strengths
  1. The paper compares primitive-only, skill-only, and mixed control within standardized execution propocol / run-time, the scale is for multiple model families and zero-shot and learning settings. This solidly provides a reasonably controlled of how differnt action interface affects agent performance.

  2. The eval part takes account game progression + token usage + inference cost. Demonstrating substantial practical benefits from the supplied skills. The mixed-control and skill-removal experiments also provide fair evidence on the benefits and costs of primitive actions when the skill library is insufficient.

Critical weaknesses

I lean toward weak reject primarily because I don't find the work to be novel / interesting enough for the bar of ICLR. I enocurage the authors to clarify their contributions on top of related work. What are some deeply new insights that is relevant for the community and industry, yet existing literature did not cover? Note that the benefits of skills to LLM agents is a well-studied topic. To recommend acceptance on such topic, the paper has to introduce some important new insights.

  1. Limited novelty. Prior work such as Voyager and NetPlay explored LLM control through executable skills. NetPlay also support skill interruption and low-level input interfaces in NetHack. The idea is not brand-new. The comparison and fine-tuning experiments provide additional evidence, but it feels to me that the paper does not clearly establish a substantial new insight beyond the expected benefits of supplying reliable skills.

  2. Insufficient ablations to explain the improvements. The comparison bundles abstraction with expert knowledge and local control encoded in the skills. I see that the shared runtime helps control implementation differences, yet the experiments do not distinguish the sources of improvement. I feel comparisons with simpler action chunks, or a simple rule-based controller using the same skill library, would help explain what the abstraction / LLM controller contribute.

  3. Limited eval of fallback under incomplete skill coverage. The skill-removal experiments use only one model; The detailed evidence for useful fallback consists of two GPT-5 trajs. Also, removing the navigation family removes staircase skills, so part of the skill-only performance drop follows from losing essential actions. Note that: these results demonstrate that primitive can help, but do not establish how reliably it do so.

Questions for authors
  1. How much of the observed improvement comes from temporal abstraction, versus the expert knowledge and control logic encoded in the hand-written skills? Do you have controlled ablations that help distinguish these contributions?

  2. Across evaluation episodes, how often does switching to primitives successfully resolve a failed skill? How much additional inference cost does this incur?

5 comments

We thank the reviewer for their thoughtful and constructive feedback. We appreciate the recognition of the value of our study and agree that we could sharpen the presentation of our insights and implications of our findings. Below, we clarify our contributions and address the specific concerns raised.

Limited novelty. Prior work such as Voyager and NetPlay explored LLM control through executable skills. NetPlay also support skill interruption and low-level input interfaces in NetHack. The idea is not brand-new. The comparison and fine-tuning experiments provide additional evidence, but it feels to me that the paper does not clearly establish a substantial new insight beyond the expected benefits of supplying reliable skills.

We agree that code skills are not new, and that our paper could better highlight its key findings. While it may be intuitive that access to reliable skills improves zero-shot performance, it is considerably less obvious where skills land on the performance-cost frontier (RF3), how the benefits of skills translate to learning (RF5), and how they interact with the availability of primitive actions (RF4).

In particular, our experiments reveal several non-obvious insights: (1) Additional flexibility does not automatically improve performance: mixed control does not consistently outperform skill-only control (despite having a more expressive action space) and incurs higher inference costs (RF3). (2) Nevertheless, restricting flexibility can be harmful when skills are insufficient: access to primitives can help agents compensate for incomplete skill coverage (RF4). This tension highlights why neither skill-only nor mixed control is universally preferable, and why the design of action interfaces deserves systematic investigation. Moreover, if both excessive and insufficient flexibility have costs, learning when to operate at each level of abstraction becomes an important research direction. Our RL results (Appendix D.1) provide initial evidence in this direction, showing that training changes how agents distribute their actions across abstraction levels.

While Voyager and NetPlay demonstrate the usefulness of executable skills, neither systematically investigates these tradeoffs across different action interfaces and learning settings.

Insufficient ablations to explain the improvements. The comparison bundles abstraction with expert knowledge and local control encoded in the skills. I see that the shared runtime helps control implementation differences, yet the experiments do not distinguish the sources of improvement. I feel comparisons with simpler action chunks, or a simple rule-based controller using the same skill library, would help explain what the abstraction / LLM controller contributes.

We thank the reviewer for these suggestions and agree that additional ablations would help clarify the sources of improvement. We are currently working on the action chunking, where the agent generates a sequence of primitive actions per model call, and a rule-based controller using the same skill library. We hope these experiments will help address the reviewer's concerns, and would appreciate their confirmation that this action-chunking setup aligns with what they envisioned.

We recognize that these baselines may not fully disentangle the effects of temporal abstraction from the knowledge encoded in the skills, but believe they will provide useful additional evidence about the sources of improvement.

Limited eval of fallback under incomplete skill coverage. The skill-removal experiments use only one model; The detailed evidence for useful fallback consists of two GPT-5 trajs. Also, removing the navigation family removes staircase skills, so part of the skill-only performance drop follows from losing essential actions. Note that: these results demonstrate that primitive can help, but do not establish how reliably it do so.

We agree that our experiments do not fully establish how reliably agents can fall back to primitive actions under incomplete skill coverage. However, we would like to clarify that our evidence for the benefits of primitive fallback is not limited to the two GPT-5 trajectories. In Appendix C.1, we systematically remove five different skill families and evaluate both skill-only and mixed control over 64 episodes per condition. These experiments show that mixed control is generally less affected by missing skill families, providing quantitative evidence that access to primitives can help compensate for incomplete skill coverage. The two GPT-5 trajectories in Appendix C.2 serve as qualitative illustrations of how this fallback can occur in practice, rather than as evidence of its reliability. (continued on next post)

AuthorsOpenReview ↗

Rebuttal (continued)

We also agree that removing the navigation family eliminates essential skills such as staircase navigation. However, this is precisely one of the situations we aim to investigate: when an essential behavior is no longer available through the skill library, can access to primitives help the agent compensate? Our results suggest that it can, although we agree that further experiments across more models and a more systematic analysis of fallback success rates would be valuable to establish its reliability.

Across evaluation episodes, how often does switching to primitives successfully resolve a failed skill? How much additional inference cost does this incur?

Quantifying fallback success rate and its incurred cost is challenging as switching to primitives does not indicate a failed skill. Agents also could switch preemptively making it difficult to identify when fallback occurs and when it succeeds. We agree that this is an important limitation of our current evaluation and will acknowledge it in the revised version of the manuscript.

We hope our responses clarify the novel insights of our study and that the planned ablations will further strengthen the empirical evidence. We would appreciate the reviewer's thoughts on whether these clarifications address their main concerns, particularly regarding the novelty and significance of our findings, and whether they would consider revising their assessment in light of our responses.

Thanks for the detailed response. I appreciate the clarification of the paper's intended contributions, particularly the trade-off between flexibility and efficiency and the quantitative evidence from the skill-removal experiments.

However, I think my main concern regarding novelty remains. While the identified trade-offs are useful empirical observations, I am not fully convinced that they constitute substantially new insights beyond existing work on skill-based agents and hierarchical control. The observations that additional action flexibility may increase decision complexity, and that primitive actions can compensate for incomplete skill coverage, seem largely consistent with existing intuitions.

I appreciate the proposed additional ablations and believe they could strengthen the paper, but would need to see the results before reassessing the experimental concerns.

That said, my assessment of weak rejection is unchanged

Reviewer_phwUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CodeHack, a fixed library of executable skills, and studies primitive-only, skill-only, and mixed control for language agents in NetHack and MiniHack. Across zero-shot evaluation, supervised fine-tuning, and reinforcement learning, supplied skills improve performance and reduce inference cost. Mixed control preserves most of these benefits while allowing primitive fallback when the library is insufficient.

Critical strengths
  • The paper offers a systematic empirical comparison across 14 models and multiple learning regimes, using a shared runtime and observation format across action interfaces. The consistent performance gains, together with token and inference-cost measurements, provide useful evidence for designing language-agent action interfaces.

  • The analysis goes beyond demonstrating that skills help. Skill-family removal experiments and concrete fallback trajectories examine the tradeoff between abstraction and flexibility. The finding that mixed control largely preserves performance but increases inference cost is practically informative, and the paper appropriately acknowledges that its agents remain far from solving NetHack.

Critical weaknesses
  • The experiments establish the value of the supplied CodeHack library more clearly than the mechanisms underlying that value. The 78 hand-engineered skills incorporate navigation, tactical decisions, and recovery knowledge, whereas the primitive baseline emits one command per model call. Without intermediate controls, such as primitive-action batching or simpler macros, it remains difficult to separate temporal abstraction and reduced invocation overhead from the benefits of embedded domain expertise.

  • The RL comparison matches controller decisions and gradient updates, but skills can consume substantially more primitive transitions per decision. The wall-clock curves support practical training efficiency, yet the reported 7.2× gain does not establish improved environment-sample efficiency. Primitive-transition counts and clearer definitions of episode limits and duration-dependent discounting would make this result easier to interpret and reproduce.

Questions for authors

How much of the performance–cost advantage remains against a baseline that generates short sequences of primitive actions per model call, without access to the hand-engineered skill policies?

Can you report RL learning curves against primitive environment transitions and clarify whether episode/no-progress limits count primitive steps or controller decisions? How are variable skill durations handled in discounting and advantage estimation?

4 comments

We thank the reviewer for their positive assessment of our work and constructive feedback. We appreciate their suggestions for strengthening the experimental evaluation and clarifying the interpretation of our results, which we address below.

The experiments establish the value of the supplied CodeHack library more clearly than the mechanisms underlying that value. The 78 hand-engineered skills incorporate navigation, tactical decisions, and recovery knowledge, whereas the primitive baseline emits one command per model call. Without intermediate controls, such as primitive-action batching or simpler macros, it remains difficult to separate temporal abstraction and reduced invocation overhead from the benefits of embedded domain expertise.

We agree that our code-based skills incorporate domain-specific knowledge, and we explicitly acknowledge this in Appendix A.1. Importantly, our goal is not to isolate the benefits of temporal abstraction independently of the knowledge encoded in the skills, but rather to study the benefits and tradeoffs of providing language agents with a fixed library of reusable, temporally extended behaviors.

The RL comparison matches controller decisions and gradient updates, but skills can consume substantially more primitive transitions per decision. The wall-clock curves support practical training efficiency, yet the reported 7.2× gain does not establish improved environment-sample efficiency. Primitive-transition counts and clearer definitions of episode limits and duration-dependent discounting would make this result easier to interpret and reproduce.

We agree that matching controller decisions does not control for env steps, and therefore our results should not be interpreted as demonstrating improved environment sample efficiency. However, environment samples are not the bottleneck in our setting (LLM inference and training is). Additionally, the skill-based controller does not make direct use of these intermediate primitive transitions (only the start state/end state, and the aggregated discounted reward), and controlling environment samples would result in different numbers of controller decisions and gradient updates. To provide a more complete picture of learning efficiency, we will also add learning curves against primitive environment steps in the revised version of the paper.

We cap the episodes at 5000 primitive steps, with additional no-progress timeout of 150 primitive steps. Regarding discounting, rewards are discounted within each skill, and subsequent discount exponents are adjusted by the number of primitive steps executed. However it turned out not to matter because we found gamma=1.0 to work the best, which effectively turns off the discounting. We will clarify these details in the revised paper.

How much of the performance–cost advantage remains against a baseline that generates short sequences of primitive actions per model call, without access to the hand-engineered skill policies?

We are currently implementing an action-chunking baseline and plan to evaluate it in the zero-shot setting using Gemma-4-31B. Would the reviewer consider this a useful initial comparison, given the limited rebuttal period?

We hope our clarifications and planned revisions address the reviewer's remaining concerns, and would welcome any further feedback.

Thank you for the clarifications. The primitive-step episode limits, γ=1 setting, and distinction between practical training efficiency and environment-sample efficiency help address my concerns.

The proposed zero-shot action-chunking experiment with Gemma-4-31B would be a useful initial comparison. Please report progression and inference cost under the same observations and primitive-step budget, and clarify the chunk length and interruption behavior. The planned learning curves against primitive environment transitions would also strengthen the RL analysis. These additions would help interpret the gains, while the contribution of domain knowledge encoded in the skills remains a limitation.

We thank the reviewer for continuing the discussion and for their constructive suggestions. We have now completed the action-chunking experiment, as well as an additional rule-based controller baseline, and updated the paper with new experiments.

New results: Action-chunking baseline. Following the reviewer’s suggestion, we evaluated zero-shot action chunking with Gemma-4-31B and GPT-OSS-120B on NetHack. The models predict sequences of primitive actions per call, with execution interrupted when a new monster appears, HP decreases, the dungeon level changes, or an action is invalid. We evaluated chunk lengths of 2, 5, 10, and 20 on 16 seeds, using a 5,000-step horizon. The primitive-only baselines are the results from Table 4 in the paper. Actions/call is the number of actions executed divided by the number of model calls in an episode, averaged across episodes.

ModelSettingScoreProgression (%)Max depthActions/call
Gemma-4-31BPrimitives (chunk 1)171.11 ± 16.011.45 ± 0.121.05 ± 0.031.00
Chunk 295.31 ± 23.560.95 ± 0.251.06 ± 0.061.99 ± 0.00
Chunk 5140.94 ± 29.631.15 ± 0.261.06 ± 0.064.85 ± 0.03
Chunk 10114.00 ± 18.691.31 ± 0.231.06 ± 0.069.40 ± 0.05
Chunk 2077.69 ± 17.710.83 ± 0.241.12 ± 0.1216.59 ± 0.85
GPT-OSS-120BPrimitives (chunk 1)152.97 ± 14.461.39 ± 0.111.28 ± 0.081.00
Chunk 235.44 ± 10.140.21 ± 0.151.06 ± 0.061.97 ± 0.01
Chunk 546.69 ± 12.810.44 ± 0.201.06 ± 0.064.87 ± 0.01
Chunk 1013.56 ± 4.580.00 ± 0.001.00 ± 0.009.34 ± 0.23
Chunk 206.38 ± 2.720.00 ± 0.001.00 ± 0.0016.79 ± 1.03

Action chunking allows multiple actions per model call but achieves lower mean performance than primitive-only control. While larger chunks reduce the number of model calls, they also require the model to predict actions without observing their intermediate consequences, potentially wasting actions and resources. Action chunking results are provided in Appendix C.3.

Learning curves against primitive environment steps. We have added learning curves against cumulative primitive environment steps for both Llama-3.1-8B-Instruct and Qwen-3.5-4B (Figs. 20 and 23 in Appendix D).

The performance gap between skill-based and primitive-only control is smaller when accounting for the additional environment transitions consumed by skills. This is somewhat expected since skills get to perform less gradient updates for the same number of environment steps compared to primitives. Nevertheless, skill-based agents still achieve a 4.16x larger gain in dungeon level (averaged across models) than primitive-only agents when compared at matched environment-step budgets. These results suggest that the benefits of skills are not solely attributable to an increased number of environment interactions.

We appreciate the reviewer's suggestions, which have helped strengthen the experimental evaluation. We hope the additional results and revisions address the remaining concerns, and would welcome the reviewer's reconsideration of their assessment in light of this new evidence.

I appreciate authors' feedback. The feedbacks help to address my concerns. I will keep the weak accept recommendation.

Reviewer_k8CzOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents CodeHack, a code-based skill library designed for NetHack/MiniHack problems. The paper proposes using code-based skills as a high level abstraction to solve the two benchmarks and writes around 80 skills and forms a library. Results show that under skill, primitive, and mixed setups, using the skills helps the models achieve better results in Zero-Shot setups. The paper further shows that training with these skills achieves good performance on these benchmarks, especially with RL or SFT+RL.

Critical strengths
  1. Using code as a high-level abstraction for better low-level control seems to be a good idea.

  2. The paper considers both a training-free approach and a training-based approach. For the training-free approach, the paper tests on several different models with their skills. For the training-based approach, the paper considers RL, SFT, and RL+SFT, which is comprehensive.

Critical weaknesses
  1. The overall contribution is more like an engineering solution, with limited novelty/generalizability/broader impact. First, the main contribution of the paper appears to be hand-crafting a skill library for a specific environment (From the paper, it appears that MiniHack/NetHack are very similar to each other). It is questionable whether the takeaways from this paper can be generalized to broader (skill-based) agentic environments. Furthermore, after reading the appendix, one can notice that lots of environment priors are injected into the skill library. Hence, it is unclear whether the gains are actually from the skill abstraction or from lots of engineered priors that are fed into the agent.

  2. The presentation of the main method is unclear. The main method Section (Section 3) only contains descriptive paragraphs about the library CodeHack. There are lots of important details missing here. For example, readers need to know what a skill look like, and how the agent is using those skill to interact with the environment. Moreover, a training-based approach is a very important methodological part of the paper. However, the training-based approach is only briefly mentioned in Section 4, and lots of important details on training are missing in the main paper, making it hard to understand how training is actually done. For example, how are skills integrated with PPO? Substantial effort should be put into clearly explaining the training procedure and how skills are involved in training.

Questions for authors
  1. Could you explain the metric 'progression' in more detail? It appears that regardless of the methods and training procedures, 'progression' is always low, with an upper bound of around 5%. Does that indicate even after substantial training/skill injection, current agents are still struggling to solve these benchmarks?

  2. The authors may want to revise their Figure 2 as many details are unclear/irrelevant to the main contribution of the paper. For example, how is the execution runtime part related to your method? Why can primitives be regarded as a fallback of skills? Does that mean skill is already a privileged representation compared to primitives?

  3. The comparison of purely skill-based control and mixed control seems more complex than the research findings claim. According to Figure 4, it seems that in zero-shot cases, weaker models achieve better results when control is mixed instead of skill-based, but for strong models like GPT-5, it is the inverse. Also, RL+SFT and SFT rows are missing the cases when control is mixed/primitive-based. A more thorough analysis of what control method achieves the best performance is required (both experiments and analysis).

4 comments

We thank the reviewer for their detailed feedback and for recognizing the breadth of our experimental evaluation. We respectfully disagree with their characterization of the work as primarily an engineering contribution with limited novelty and broader relevance, as we believe this assessment does not fully reflect the scientific questions and findings of our study. Below, we clarify our contributions and address the concerns regarding generalizability, engineered priors, and methodological clarity.

The overall contribution is more like an engineering solution, with limited novelty/generalizability/broader impact. First, the main contribution of the paper appears to be hand-crafting a skill library for a specific environment (From the paper, it appears that MiniHack/NetHack are very similar to each other).

We would like to clarify that our main contribution is not the skill library, but rather a systematic study of how action abstraction affects language agents. As outlined at the end of the introduction, the skill library serves as experimental infrastructure enabling controlled comparison of action interfaces across models and learning settings.

Importantly, our experiments reveal several non-obvious tradeoffs: additional flexibility does not automatically improve performance, as mixed control does not consistently outperform skill-only control and incurs additional costs; yet restricting flexibility can be harmful when skills are insufficient. This tension highlights why neither interface is universally preferable and motivates studying how agents select between abstraction levels. Moreover, our RL experiments show that skills substantially improve training efficiency, while learning also changes how agents distribute their actions across abstraction levels (Appendix D.1). We believe these findings offer insights into the design and training of skill-based agents beyond the specific CodeHack implementation.

It is questionable whether the takeaways from this paper can be generalized to broader (skill-based) agentic environments.

We agree that evaluation in other domains would strengthen the generalizability of our findings. Nevertheless the tradeoffs between different levels of abstraction are not specific to NetHack. Our findings of incomplete skill coverage and mixed control provide insights into balancing the efficiency and flexibility in designing agentic systems. We see additional empirical validation in other domains as future work.

Furthermore, after reading the appendix, one can notice that lots of environment priors are injected into the skill library. Hence, it is unclear whether the gains are actually from the skill abstraction or from lots of engineered priors that are fed into the agent.

We agree that our skills incorporate domain-specific knowledge, and we explicitly acknowledge this in Appendix A.1. Our goal is not to isolate the benefits of temporal abstraction independently of the knowledge encoded in the skills, but to study the benefits and tradeoffs of providing language agents with a fixed library of high quality skills. This setting allows us to investigate how the availability of high-level skills affects agent performance, efficiency, and learning, including the tradeoffs between skill-only and mixed control.

The presentation of the main method is unclear. The main method Section (Section 3) only contains descriptive paragraphs about the library CodeHack. There are lots of important details missing here. For example, readers need to know what a skill look like, and how the agent is using those skill to interact with the environment. Moreover, a training-based approach is a very important methodological part of the paper. However, the training-based approach is only briefly mentioned in Section 4, and lots of important details on training are missing in the main paper, making it hard to understand how training is actually done. For example, how are skills integrated with PPO? Substantial effort should be put into clearly explaining the training procedure and how skills are involved in training.

We agree with the reviewer that Section 3 could benefit from additional details, and are updating the main text to include more details. Nevertheless, we note that several of the requested implementation details are already provided in the appendix. In particular, Appendix A.2 describes how skills are executed through a call-and-return mechanism, Appendices A.3–A.4 further describe the internal state representation and interruption mechanisms, and Appendix B.3 provides the prompt templates. (continued on next post)

AuthorsOpenReview ↗

Rebuttal (continued)

Regarding training, the integration of skills with PPO is relatively straightforward: skills can be viewed as an alternative action space, where each skill invocation corresponds to a single transition at the controller level. When a skill is invoked, it executes a sequence of primitive actions, and the rewards accumulated over these intermediate steps are aggregated into a single reward (with discount factor γ = 1 in our experiments). The resulting transition is then treated like any other transition in PPO: value estimates and GAE advantages are computed only at the states where the controller makes decisions, without considering the intermediate states visited during skill execution. Thus, the standard PPO training procedure can be applied directly to the resulting higher-level decision process. Appendix B.4 provides further implementation details.

Could you explain the metric 'progression' in more detail?

We have provided a detailed description of the NetHack progress metric in Appendix B.1, which we briefly summarize below for convenience:

The NLE-progress metric introduced in BALROG (Paglieri et al., 2025) aims to measure meaningful progress toward winning NetHack rather than relying on the in-game score. The metric is derived from human-played NetHack trajectories and maps the highest dungeon level and experience level reached in an episode to their corresponding probabilities of eventual ascension. The final progress score is the maximum of these two values, with 0% corresponding to the starting levels and 100% to successfully ascending (winning) the game.

It appears that regardless of the methods and training procedures, 'progression' is always low, with an upper bound of around 5%. Does that indicate even after substantial training/skill injection, current agents are still struggling to solve these benchmarks?

Correct, NetHack remains extremely challenging for current agents, as also reflected in the BALROG leaderboard https://balrogai.com/. However, it is worth noting that progression is a calibrated measure of advancement, not a percentage of the game completed. Therefore, small numerical values can still correspond to meaningful differences in how far the agents progress.

The authors may want to revise their Figure 2 as many details are unclear/irrelevant to the main contribution of the paper. For example, how is the execution runtime part related to your method? Why can primitives be regarded as a fallback of skills? Does that mean skill is already a privileged representation compared to primitives?

We thank the reviewer for this feedback and will update Figure 2 alongside Section 3. The execution runtime provides shared infrastructure for all three action interfaces, rather than additional capabilities exclusive to skills. Primitives are not inherently fallback actions, but can serve this role under mixed control when the available skills are insufficient. Skills provide abstractions over sequences of primitive actions, rather than a privileged interface to the underlying environment. We will clarify these distinctions in the revised figure and text.

The comparison of purely skill-based control and mixed control seems more complex than the research findings claim. According to Figure 4, it seems that in zero-shot cases, weaker models achieve better results when control is mixed instead of skill-based, but for strong models like GPT-5, it is the inverse.

We thank the reviewer for this sharp observation. While the relative performance of skill-only and mixed control varies for models, we do not observe a clear relationship with model size. Table 4 shows that the differences are often small relative to the standard errors. We therefore believe the available evidence is insufficient to draw a clear conclusion about which interface performs better depending on the model size. We will clarify this observation and the uncertainty surrounding these comparisons in the revised manuscript.

Also, RL+SFT and SFT rows are missing the cases when control is mixed/primitive-based. A more thorough analysis of what control method achieves the best performance is required (both experiments and analysis)

Regarding the missing SFT comparisons, our experiments focus on skill-based demonstrations because the teacher (Gemma-4-31B) performs poorly with primitive control. Training on primitive trajectories would require collecting a new dataset, introducing additional confounding factors.

We hope our clarifications and planned revisions address the reviewer's concerns regarding our contributions and methodology. We would appreciate their feedback on whether these changes resolve the issues underlying the presentation and soundness assessments, and whether they would consider revisiting their overall assessment accordingly.

Thank the authors for the response. However, several major concerns have not been addressed:

The findings of the paper is limited to the CodeHack/NetHack benchmark considering the following:

  • The paper does not provide any experiments beyond the two very similar benchmarks.
  • The main technical contribution, the library, contains only handcrafted skills and has lots of injected priors. (Reviewer phwU also points out this)

In light of these two weaknesses, the paper appears to be an engineered approach on skills for agents that has limited novelty, which I agree with reviewer eEoU. Furthermore, the author's claim, which states that the paper is 'a systematic study of how action abstraction affects language agents', is an overclaim of the paper's contribution, and the author also does not address my concern on the generalizability of the method to other domains.

Second, the authors' rebuttal does not provide additional takeaways on the comparison of purely skill-based control and mixed control, as well as any experiments/analysis on RL+SFT and SFT experiments when control is mixed/primitive-based.

Third, currently I don't see a revision of the paper despite the authors' claim that they will update it.

Considering this, I don't think the current rebuttal would change my overall assessment on this paper.

We thank the reviewer for their continued feedback and for helping us clarify the scope of our contribution.

Scope and generalizability. We agree that evaluating beyond NetHack and MiniHack would strengthen the evidence for generalizability, and we acknowledge this limitation in the paper. The two benchmarks nevertheless provide different tasks, allowing us to investigate specific failure modes (MiniHack) as well as agent performance in a more open-ended setting (NetHack). While our experiments cannot establish that the same results will hold in other domains, they provide evidence about the tradeoffs between action interfaces across a range of models and learning settings.

Our characterization of the work as a systematic study refers to the comparisons within this setting. We evaluate 14 models across three action interfaces in the zero-shot setting, train two models with RL across these interfaces, and conduct a targeted experiment on the effect of SFT before RL. This provides a systematic comparison across models, interfaces, and learning settings, which was also appreciated by the other two reviewers. We believe these comparisons provide useful evidence about the tradeoffs between action interfaces across models and learning settings

Skill-only, mixed control, and SFT comparisons. We agree that SFT and SFT+RL comparisons across interfaces would be valuable. Our experiments instead focus on the potential benefits of SFT priming before the RL with skills. Collecting demonstrations for the additional comparisons and running additional SFT and RL training on top would require more time than was available during rebuttal. We have clarified the scope of these results and noted this limitation in the discussion.

We also appreciate the request for clearer takeaways on skill-only versus mixed control. As the reviewer notes, their relative performance varies across models, though the results do not establish a clear trend. Averaged across models, mixed control retains 95% of skill-only progression, while increasing inference cost per episode by 2.3x. Thus, mixed control can preserve much of the benefit of skills while offering access to primitives, but this additional flexibility does not consistently improve performance and can come at a higher cost. Our skill-removal experiments further suggest that access to primitives can help when skill coverage is incomplete, although they do not establish how reliably individual fallback attempts succeed.

Revised manuscript. We thank the reviewer for helping us improve the clarity of the paper. We spent some time carefully reviewing the paper and have now updated it, including a completely new Figure 2 and revised Sections 3 and 4 to clarify the method and training procedure. We have also added experimental results for action chunking and a rule-based policy.

We thank the reviewer again for their feedback, which has helped us improve the paper. We hope the revised manuscript and the clarifications above address their concerns, and we would appreciate their assessment of the changes and reconsideration of the paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their effort and informative feedback. We believe that it will lead to a substantially better version of our paper.

We are pleased that the reviewers appreciated our comprehensive and systematic empirical comparison, the practical benefits of the supplied skills, and the insights into designing language-agent action interfaces. At the same time, reviewers pointed out some areas for improvement. We are grateful for this feedback as well and acknowledge certain shortcomings. We have made substantial efforts to amend these. Below is a short summary of our changes, and we present more details in responses to the reviewers:

  • [phwU, eEoU] We evaluated an action-chunking baseline to investigate whether batching primitive actions provides benefits comparable to code skills.
  • [eEoU] We developed and evaluated a rule-based controller using the same skill library.
  • [phwU] We added RL learning curves against primitive environment steps.
  • [k8Cz] We redesigned Figure 2 and revised Sections 3 and 4 to clarify the method and training procedure.
  • [k8Cz, eEoU] We expanded the discussion of generalizability, interface comparisons, and the limitations of our fallback analysis.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.