acceptodds
Under review as a conference paper at ICLR 2027

NetHack+: A Configurable Environment for Long-Horizon Agent Research

Abstract

NetHack remains a challenging benchmark for long-horizon decision-making despite extensive research employing reinforcement learning, imitation learning, and language-model agents. A substantial difficulty in NetHack derives from the lack of infrastructure surrounding the game. As a result, we present NetHack+, an upgraded game engine with 42 knobs for varying game difficulty, 14× faster throughput than NetHack Learning Environment (NLE), and integrations into Gym, PufferLib, and Prime Intellect's Verifiers. We accomplish this by refactoring NLE to allow saving and restoring game state, configurable game mechanics, and custom observation and action interfaces embedded in modern LLM harnesses (Prime Agent and Claude Code). We also package a new metric, Bal-min, to reward balanced progression in dungeon depth and experience, motivated by successful human runs and designed to limit rewards for agents that descend without gaining experience. Experimentally, interface improvements and continual harness optimization yield modest gains in Bal-min, reaching approximately 1.5× baseline performance. To demonstrate the utility of checkpointing, we introduce Prime-Agent-Explore, an orchestrator-based search algorithm that selects saved checkpoints and guides subsequent exploration. Using 30 attempts with full vision, Prime-Agent-Explore reaches 11.38% Bal-min and 34.66% Bal-max, improving over the single-life baseline by 8.6× and 3.5×, respectively. With fog-of-war, Prime-Agent-Explore reaches 6.03% Bal-min and 30.93% Bal-max, exceeding the previously reported LLM Bal-min baseline by 3.3×. We also evaluate Prime-Agent-Explore on MiniHack, Crafter, and TextWorld, providing state-of-the-art performance across all three benchmarks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
nethack, agent benchmark, planning, continual harness, test-time-compute, long-horizon planning, language model agents, agent evaluation, checkpointed exploration, agent harnesses

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gE2HOpenReview ↗
Rating
2
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces NetHack+, a configurable engine with checkpointing, flexible agent interfaces, and faster simulation. It also proposes BALmin to measure balanced progression and Prime-Agent-Explore for orchestrator-guided checkpoint search. Experiments show improved progression in NetHack and three additional game environments.

Critical strengths
  1. Configurable mechanics and checkpointing enable controlled studies of agent failures and search strategies.

  2. Removing checkpoint selection or guidance reduces performance, supporting the value of both components.

Critical weaknesses
  1. The experiments do not sufficiently distinguish long-term planning from local survival, game knowledge, and interface improvements. BALmin’s relationship to eventual success is also not established.

  2. The headline gains compare multi-attempt checkpoint search with single-life baselines, making it difficult to separate methodological benefits from additional computation and retries.

Questions for authors
  1. What evidence shows that improved NetHack performance reflects better long-horizon planning rather than better local decisions or game knowledge?

  2. Compared with more open-ended environments such as Minecraft, what distinctive advantages does NetHack offer for evaluating long-horizon agents? This asks for clarification of benchmark scope, rather than a new experiment.

  3. Does Prime-Agent-Explore outperform independent restarts or simpler checkpoint strategies under matched computational budgets?

  4. Is BALmin more predictive of eventual ascension than BALmax, beyond encouraging balanced depth and experience?

3 comments
AuthorsOpenReview ↗

Author Review

Dear Reviewer gE2H,

Thank you for the time and effort put into the review. We appreciate your recognition that configurable mechanics and checkpointing enable controlled studies of agent failures and search strategies, and that our ablations support the value of both checkpoint selection and guidance.

We understand the main concerns to be (1) whether the gains reflect long-horizon planning rather than local decisions or game knowledge, (2) BALmin's relationship to eventual success, and (3) separating PAE's benefit from additional computation and retries. We have run new experiments and revised the paper (changes highlighted in yellow). Together with the discussion below, does this fully address your concerns? We look forward to continuing the discussion!

The experiments do not sufficiently distinguish long-term planning from local survival, game knowledge, and interface improvements. [...] What evidence shows that improved NetHack performance reflects better long-horizon planning rather than better local decisions or game knowledge? (W1, Q1)

Our harness adds navigation skills and a structured text encoding of observations, so that low-level control and perception do not become the bottleneck and higher-level decisions can be isolated. We then measured the remaining low-level confounders with the single-life probes of Section 5.1, for example simplifying interactions such as unlocking doors (control) and providing streamed observations (perception). None changes BALmin significantly (unlocked doors p = 0.31, ASCII map p = 0.16, JSON map p = 0.78).

In contrast, PAE, a much stronger search and exploration mechanism, produces the largest improvement of any intervention: with GLM-5.2 it reaches 11.70% BALmin at 30 attempts, against 4.15% for the best of 30 independent lives (combined response). Within PAE, letting the model choose which earlier state to continue from, a decision about the whole trajectory, raises BALmin from 2.45% to 4.21% over a fixed rule (table below).

PAE also benefits from additional search and from chances to recover from mistakes. The small effects of control and perception, against the much larger gains from PAE, therefore indicate that higher-level planning and exploration remain the major bottleneck for current agents in NetHack.

BALmin's relationship to eventual success is also not established. [...] Is BALmin more predictive of eventual ascension than BALmax, beyond encouraging balanced depth and experience? (W1, Q4)

We address this in the combined response above, in the section "BALmin vs BALmax".

The headline gains compare multi-attempt checkpoint search with single-life baselines [...] Does Prime-Agent-Explore outperform independent restarts or simpler checkpoint strategies under matched computational budgets? (W2, Q3)

For independent restarts, we address this in the combined response above, in the section "Evaluating NetHack", which compares PAE@k with Best@k. For simpler checkpoint strategies, Table 6c compares Best@15 with PAE@15 and its ablations, all with GLM-5.2 at 15 attempts:

MethodSeedsBALmin
Best@150–24.15%
No Selection + No Guidance0–42.45%
No Selection + Guidance0–42.81%
Selection + No Guidance0–44.21%
PAE@150–49.53%

The strategies with no selection resume the most recent checkpoint, and the orchestrator models in evaluations with no guidance do not hand over a plan to the subagent. We find that Best@15 exceeds both ablations with no selection. Selection without guidance is on par with Best@15 (4.21% versus 4.15%). We thus find that simpler checkpoint strategies at best match Best@15, whereas PAE@15, which combines selection with guidance, exceeds Best@15 by 2.3×.

AuthorsOpenReview ↗

Author Review (cont. 2/2)

Compared with more open-ended environments such as Minecraft, what distinctive advantages does NetHack offer for evaluating long-horizon agents? (Q2)

NetHack differs from Minecraft in three ways that matter for evaluating long-horizon agents. First, information is restricted: potions, scrolls and wands have randomized appearances in every game, so the agent must identify them anew, which is not always possible and often relies on indirect evidence such as shop prices [1]. Second, later encounters depend greatly on earlier decisions, because the monsters that are generated depend on the hero's experience level and depth [2]. Third, NetHack is a roguelike, so a single mistake can end the game. In Minecraft, recipes and progression are fixed across worlds and mistakes are recoverable.

As a result, LLM agents in Minecraft can accumulate skills into reusable libraries of executable code [3] and have completed the game [4]. NetHack remains far from solved even when retries are allowed. When we provide 30 checkpointed attempts, PAE only reaches 11.70% BALmin with GLM-5.2, and single-attempt agents reach only about 2%, leaving substantial headroom as a benchmark.

References

[1] NetHack Wiki. "Identification." nethackwiki.com/wiki/Identification

[2] NetHack Wiki. "Monster creation." nethackwiki.com/wiki/Monster_creation

[3] Wang, G. et al., 2023. "Voyager: An Open-Ended Embodied Agent with Large Language Models." arXiv:2305.16291

[4] Faldor, M., Singh, S., Raileanu, R., 2026. "A team of Gemini 4 Argon agents beats Minecraft." arXiv:2610.22319

I have lowered my confidence from 2 to 1 to reflect my unce rtainty about parts of my assessment. I hope this helps the AC weigh my review.

Reviewer_hjNbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper makes 3 contributions: First, it builds NetHack+, and improves the bottlenecked infrastructure of the game NetHack. It has 14x simulation speed, more access for environment control, ability to save, restore, branch, parallelize games, and more friendly agent MCP protocols. Second, it introduces BALmin metric, which takes minimum of dungeon level and xp ascension-probability scores instead of maximum, to slow down agents to reach dungeon level that are too deep for its xp level. Third, it proposes PAE, it is a Go-Explore agent that can constantly save and restore from previous checkpoints for next attempts. With single-life and harness interventions (decision-making gates, revive and rollback, unlocked doors, streaming observations) and continual harness, it has slight improvement compared to the baseline; With 30 attempts, it reaches BALmin score of 11.38% with full vision, and 6.03% with fog of war, 3.3x of the previous SOTA of BALmin score 1.84%

Critical strengths

S1. Improvement of infrastructure. NetHack+ engine with fast save/restore/branching features and controllable knobs enables search-based methods and detailed ablations that is not possible in NLE.

S2. PAE is able to raise BALmin score to 6.77%, vs baseline 1.84%, with multiple attempts in NetHack.

S3. Detailed analysis and ablations. Section 5.1 attributes effects of decision-making gates, revival + rollback, unlocking doors, and streamed observations, and fog of war. Additional ablation is done with harness optimization.

S4. The paper compares multiple models with different harnesses, on 3 different environment settings with token cost, which demonstrates clearly the performance-cost tradeoff.

Critical weaknesses

W1. PAE has multiple life attempts with rollback. Previous SOTA baselines and human are in single life setting, so the comparison is unfair. In PAE, after each death, the orchestrator can restore from an earlier checkpoint. Test-time search methods usually use a simulator to search and plan, and after certain rollouts, it commits an action, that can never be undone; PAE, however, can undo every mistakes, to fix mistakes or retry dangerous fights, and learn from future environment feedbacks. This is save-scumming, heavily discussed in NetHack community and is considered against the spirit of the game. The paper also has inconsistent standards, e.g. force revive is considered to "fundamentally changes the game rather than the harness", but PAE is essentially doing the same thing.

W2. Section 6 states that "behavioral changes allow BALmin to increase, yet decrease BALmax (Table 3), validating the hypothesis that BALmax over-indexes on short-term strategies while BALmin does not." However, the claim is validated circularly. The interventions include descent gate planning "when an agent attempts to descend while underleveled", and harness "never descend unless XL increased since last descent". These are basically instructing the agent to increase the xp level in trade of dungeon depth level, therefore increase BALmin and decrease BALmax. The results show the interventions changes the xp/depth tradeoff, but not necessarily means the tradeoff is desired. A possible validation could be estimating P(ascension | Dlvl, XL) jointly instead of using adhoc min or max functions, and better heuristics could rise from that.

W3. The 14x throughput, environment parallelism and O(1) checkpointing have no hardware/software details to show what modules lead to major improvements, and there is no metric comparison such as simulation speed, latency, equivalence against vanilla NLE. Table 1 reports 42 knobs (8/4/30), but Table 8 lists only 35 live knobs (8/4/23).

Questions for authors
  1. On PAE's evaluation protocol: (a) Since any death can be undone by restoring from a checkpoint, it resembles save-scumming in human play. How do the authors justify comparing these scores with single-life human games and single-attempt BALROG results? (b) Table 2 excludes forced revive because "it changes the game rather than the harness". What makes PAE different which also continues play after death? The authors also state that PAE avoids information leakage, but in Appendix E the orchestrator can read the information about previous death. Could the authors specify what information flows across attempts?
  2. Does BALmin predict eventual ascension or later progress better than BALmax? A joint distribution suggested in W2 could be helpful.
  3. How are the 14x throughput, environment parallelism claims and O(1) checkpointing measured? And how is the engine validated to ensure the same behavior as the original NLE?
2 comments
AuthorsOpenReview ↗

Author Response

Dear Reviewer,

Thank you for the detailed review and constructive feedback on the paper. We appreciate your recognition that the NetHack+ infrastructure enables search-based methods and detailed ablations that are not possible in NLE (S1, S3), that PAE raises BALmin with multiple attempts (S2), and that our comparison across models and harnesses clearly demonstrates the performance-cost tradeoff (S4).

We understand the main concerns to be (1) the fairness of comparing multi-attempt PAE with single-life baselines and human play, (2) the circularity of the evidence for BALmin over BALmax, and (3) the details behind the engine claims. We have run new experiments and revised the paper (changes highlighted in yellow). Together with the discussion below, does this fully address your concerns? We look forward to continuing the discussion!

W1. PAE has multiple life attempts with rollback. Previous SOTA baselines and human are in single life setting, so the comparison is unfair. [...] (W1, Q1)

We address these points in the combined response above, in the section "Evaluating NetHack".

On Q1(b), both PAE and Rollback (including forced revive) restore earlier states. In PAE, each attempt starts in a new context and only the orchestrator's plan, the lessons from earlier attempts, and the table of saved checkpoints carry over. In Rollback, the model's context still contains the ground-truth actions that followed the checkpoint: for example, we have observed a model using Rollback to simulate fighting a certain monster with different sequences of actions. We design PAE to not offer enough granular information to permit this.

W2. [...] the claim is validated circularly. [...] Does BALmin predict eventual ascension or later progress better than BALmax? (W2, Q2)

We provide additional reasoning on how we arrived at BALmin in the combined response above, in the section "BALmin vs BALmax". On the joint estimator suggested in W2, we fit P(ascension | Dlvl, XL) on the same human games with a logistic regression on splines of Dlvl and XL with an interaction term, scored by five-fold cross-validation. It improves AUROC by at most 0.015 over BALmax and 0.027 over BALmin (e.g., 0.818 versus 0.805 and 0.792 using Dlvl and XL observed at 5k turns), so the choice of aggregation accounts for little of the predictive power, and we thus keep the progression mappings of BALROG, which keeps our scores most comparable with prior work. The simplicity of BALROG allows us to easily identify reward-hacking behaviors.

W3. The 14x throughput, environment parallelism and O(1) checkpointing have no hardware/software details [...] how is the engine validated to ensure the same behavior as the original NLE? (W3, Q3)

We benchmarked training a 4M-parameter neural network in the PufferLib library using NetHack+ versus NLE 1.3.0 on an RTX 6000, with N = 4,096 parallel games on T = 16 threads. We observed 400k steps/second using NetHack+ versus 30k steps/second with NLE, a 13.3× throughput gain.

Memory improvements. NLE loads a private copy of the NetHack library for each environment (3.8 MB), whereas NetHack+ loads one copy and keeps one state block per environment (0.33 MB).

Checkpointing. On an AMD EPYC 9655 (16 cores, 63 GB of memory), restoring a checkpoint takes 28 µs after 1k turns and 44 µs after 50k turns on one dungeon level, and 2.2 ms once 20 levels have been visited. NLE can only return to an earlier state by replaying the game from its seed, which takes 11 ms at 1k turns and 1.8 s at 50k turns.

Equivalence to NLE. We store 200-step sequences from NLE 1.3.0 across 2,000 seeds as gold standards. Tracing these games, NetHack+ provides byte-identical glyphs, characters, colors and status on 1,981 seeds, and an identical map on all 2,000. The 19 failure cases differ in depth, level number and score for a single frame after a trap-door fall, due to a purposeful rendering refresh that we add to NetHack+ to support configurable vision toggles.

Knob count. We have corrected the paper to 35 knobs (8/4/23), as listed in Table 9.

The authors addressed my concern of infrastructure and implementation details. However, I still have concerns about PAE multilife setting, and novelty of BALmin. Thus I will keep my score.

Reviewer_ANDoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper refactors nethack environment cleverly so that it now has O(1) save/restore branching, allowing thousands of envs per process. Also it allows to set difficulty knobs. The authos use the engine to break down LLM-agent failures, into: planning, control and perception. They also propose BALmin as a replacement for BALmax, and introduce Prime-Agent-Explore, a checkpoint-based serch agent with SOTA performance.

Critical strengths
  • The engine itself seems to be a real contribution. Cheap save/restore, massive parallelsim, multiple knobs, etc. can improve usability of NetHack env.
Critical weaknesses
  • It seems explore gets 30-100 checkpointed retries; isn't it basically a savescumming? While forced revive is excluded as it "changes the game", keeping it in the table, comparing it against single-life baselines and human players seems very unfair.
  • one of the motivation of the paper is that BALROG's harness limits the models' performances (line 119). This one hasn't clearly shown in the experiments, there are only mixed performances of baselines of multiple knobs turned, not isolating the effect of harness itself.
  • except the results with explore, other results have largely overlapping confidence intervals; while they do not seem statistically significant, the paper seems to overstate the mean improvements.
  • From the definitions itself, increasing Balmin will decrease Balmax; I don't think it is an evidence that justifies Balmin, because we can also comeup with another set of interventions that can increase Balmax in a cost of Balmin, although it may seem counterintuitive to usual players. There should be an evidence that according to human data, Balmin correlates with a number of different metrics on human's progress, whereas Balmax does not.
  • The introduced algorithm of 'explore', seems to be very close to Go-Explore, but there is no discussion on how they are different and it is not compared in the experiments.
Questions for authors
  • How does explore agent with 30 attempts compare with the agent with 30 lives?
  • Why restoring checkpoint is not considered changing the game?
  • Can you give some motivating experiments on why BARLOG harness is bad?
2 comments
AuthorsOpenReview ↗

Author Response

Dear Reviewer ANDo,

Thank you for the careful review and the suggestions for improving the paper. We appreciate your recognition of the engine itself as a real contribution, whose cheap save/restore, massive parallelism and knobs can improve the usability of NetHack as an environment. We understand the main concerns to be (1) the fairness of comparing multi-attempt PAE with single-life baselines, (2) whether the experiments isolate the effect of the BALROG harness, and (3) the justification for BALmin over BALmax. We have run new experiments to address these and have revised the paper (changes highlighted in yellow). Together with the discussion below, does this fully address your concerns? We look forward to continuing the discussion!

It seems explore gets 30-100 checkpointed retries; isn't it basically a savescumming? [...] How does explore agent with 30 attempts compare with the agent with 30 lives? Why restoring checkpoint is not considered changing the game? (W1, Q1, Q2)

We address savescumming in the combined response above, in the section "Evaluating NetHack". Our new draft compares PAE@k with Best@k to explicitly match compute.

On Q2, both PAE and Rollback (including forced revive) restore earlier states. In PAE, each attempt starts in a new context and only the orchestrator's plan, the lessons from earlier attempts, and the table of saved checkpoints carry over. In Rollback, the model's context still contains the ground-truth actions that followed the checkpoint: for example, we have observed a model using Rollback to simulate fighting a certain monster with different sequences of actions. We design PAE to not offer enough granular information to permit this.

one of the motivation of the paper is that BALROG's harness limits the models' performances (line 119). This one hasn't clearly shown in the experiments [...] except the results with explore, other results have largely overlapping confidence intervals [...] Can you give some motivating experiments on why BARLOG harness is bad? (W2, W3, Q3)

We address this in the combined response above, in the section "Evaluating NetHack": we ran BALROG's harness against ours within our game engine, and our harness scores higher for all three models, with a 7–92× lower cost per game. As a result of decreasing evaluation costs due to our improved harness, we use n = 15 games (5 seeds with 3 games per seed) for the majority of our experiments, compared with n = 5 in prior work with language models such as BALROG [1].

From the definitions itself, increasing Balmin will decrease Balmax; I don't think it is an evidence that justifies Balmin [...] (W4)

We address this in the combined response above, in the section "BALmin vs BALmax".

The introduced algorithm of 'explore', seems to be very close to Go-Explore, but there is no discussion on how they are different and it is not compared in the experiments. (W5)

PAE follows Go-Explore's framework [2] for solving games, but differs significantly in implementation. Go-Explore aims to:

  1. remember states that have previously been visited;
  2. first return to a promising state (without exploration), then explore from it;
  3. solve simulated environments by exploiting any available means (including by introducing determinism), then robustify (create a policy that can reliably perform the solution) via imitation learning.

We provide a language-model algorithm that completes steps 1 and 2. The behavior-cloning experiment in the combined response section "Evaluating NetHack" is an initial exploration of how to begin bootstrapping imitation learning on our higher-quality data.

References

[1] Paglieri, D. et al., 2025. "BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games." ICLR

[2] Ecoffet, A. et al., 2021. "First Return, Then Explore." Nature

Thank you for the response. Some concerns are addressed, but the algorithmic contribution remains modest. In my opinion, the value as a benchmark paper is also limited: I will keep my score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Combined Response to All Reviewers (1/2)

We thank all reviewers for their time and thoughtful feedback, and for recognizing the contributions of the NetHack+ engine, with its efficient save/restore and controllable knobs, as well as the nethack-plus package, which runs plug-and-play language-model evaluations with any harness in fixed-skill or modifiable-skill settings. We emphasize that our goal is to enable future research on NetHack by creating a shared platform with modernized infrastructure, metrics, and capabilities and demonstrating their value.

Since all three reviewers asked about (1) comparing multi-attempt PAE with single-life baselines and (2) whether BALmin is better justified than BALmax, we provide a combined response below.

Evaluating NetHack

PAE is a search algorithm that provides the agent with an environment simulator, as with classical tree search techniques in board games and Go-Explore [1], made possible by our new capability to efficiently save and restore game states (Section 3). PAE is thus not directly comparable to single-life evaluations. To improve clarity on our contributions, we split our evaluations into two types: 1) single-life evaluations demonstrating harness improvements over BALROG, and 2) multi-life evaluations.

Single-life evaluations: BALROG harness versus ours. We test models with BALROG's harness versus ours within NetHack+, with full vision on both. Our harness scores higher for all three models, with a 7–92× lower cost per game.

ModelMedian BALmin, BALROG harnessMedian BALmin, our harnessMedian BALmax, BALROG harnessMedian BALmax, our harnessMean cost per game (USD), BALROG / ours
DeepSeek V4.1 Flash1.81%2.25%2.27%6.66%0.99 / 0.03
GLM-5.21.75%1.97%2.42%5.90%82.61 / 0.90
Gemini 3.1 Pro1.75%2.42%2.42%12.56%75.49 / 10.87

Multi-life evaluations: Best@k versus PAE@k. PAE leverages a simulator to move the Pareto frontier of performance against the number of attempts in NetHack, as seen in the following table. To separate the benefits of PAE from those of simply having more attempts, we compare PAE@k (PAE with k attempts) with Best@k, a naive baseline that runs k independent lives and takes the best-scoring one. We find PAE@k exceeds Best@k in BALmin at every k, from 1.6× at k = 5 to 2.8× at k = 30.

ModelkSeedsBest@k (BALmin)PAE@k (BALmin)
GLM-5.250–22.84%4.61%
GLM-5.2100–23.26%7.45%
GLM-5.2300–24.15%11.70%

Behavior cloning on PAE data. We then provide preliminary evidence that solutions found with a simulator yield high-quality data that can be distilled into policies that do not use simulators. We fine-tune a behavior-cloning Mamba model (167M parameters) from Tuyls et al. [2] on 1M transitions. In the baseline, we use 1M transitions of AutoAscend data. In our experimental condition, we substitute 11% of data with games from PAE traces (concatenated from attempts constituting the best result). Substituting PAE data raises mean BALmin from 0.37% to 0.91% and median BALmin from 0.00% to 1.54%. Substituting the same share with single-life language-model games (LLM) instead reaches 0.66% avg BALmin and 0.00% median BALmin.

Fine-tuning dataMean BALminMedian BALminMean BALmaxMedian BALmax
No fine-tuning0.15%0.00%1.83%2.08%
AutoAscend, 1M transitions0.37%0.00%3.04%2.91%
AutoAscend 895k + LLM 112k0.66%0.00%2.98%2.91%
AutoAscend 895k + PAE 112k0.91%1.54%3.20%2.91%
AuthorsOpenReview ↗

Combined Response to All Reviewers (cont. 2/2)

BALmin vs BALmax

Our claims are: (1) language models consistently have a higher dungeon-level score than experience-level score, which is associated with a lower ascension probability in human games, and BALmin avoids rewarding this behavior while BALmax rewards it. (2) BALmin predicts winning as well as BALmax, within 0.013 AUROC at every turn count.

(1) Depth running ahead of experience. Among human games from NLD, only 5.6% spend more than 20% of their time in the first ten dungeon levels with a dungeon-level score that is 3% or more above their experience-level score, versus 33% of language-model games with our harness. Furthermore, human games that spend more than 20% of this time with this 3-point lead have a win rate of 3.9%, versus 13.1% for human games that spend the same share with the experience-level score ahead instead.

(2) Predicting ascension. Both BALmin and BALmax predict eventual ascension well. BALmin stays within 0.013 AUROC of BALmax at every turn count, and the difference is not significant at 1k, 2k or 20k turns.

Turn TAUROC BALmaxAUROC BALminBALmin − BALmax [95% CI]
1k0.5520.552−0.000 [−0.013, +0.011]
2k0.6350.630−0.005 [−0.017, +0.007]
5k0.8050.792−0.013 [−0.023, −0.005]
10k0.9060.899−0.007 [−0.015, −0.000]
20k0.9710.967−0.004 [−0.009, +0.000]

We therefore present BALmin as a more conservative metric than BALmax that cannot be increased by descending alone while remaining predictive of ascension.

References

[1] Ecoffet, A. et al., 2021. "First Return, Then Explore." Nature

[2] Tuyls, J. et al., 2024. "Scaling Laws for Imitation Learning in Single-Agent Games." TMLR

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.