- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper connects LLMs to several roguelike games to explore the ability of LLMs to optimize for long term objectives. The paper explores both providing the LLMs with a decision harness as well as using the LLM providers own coding harnesses. The paper draws links between management actions and long term optimization.
Critical strengths
- It was interesting to see how different models and in different harnesses performed on these type of games. The paper is a good starting point for research questions about long term optimization in LLMs.
Critical weaknesses
- Several issues with the presentation. Metrics are not clearly defined (or at all) or are explained in terms of game mechanics without clearly defining the game mechanics. Overall the paper seems to assume the reader has familiarity with the games. Figure 2 has clear issues with the buy and remove icons being swapped and overall being confusing. Figure 4 is also very confusing with weird axes scales. The appendix writing style is low quality and seems to contain some sections that feel like they are verbatim LLM output. Several data points such as progression in Figure 1 are needlessly presented with different scales making progression across games hard to compare. Many citations also are not the published versions of works or are referenced not using the lead authors name when they should be (eg using github handles instead of known names). The research questions and findings seem disconnected and the paper does not clearly connect them.
- Several issues with the methodology. The human baseline uses an expert human but the LLMs are playing the games blind. Either the human should be blind or the LLMs should be given in context examples of successful and unsuccessful trajectories. It makes sense in this case that any strong learning agent would not be able to perform as well globally as locally as it is more likely to have encountered similar local situations during training. Many of the results are confounded by aspects that cannot be controlled for such as what training data/environments closed models use, however its is known that recent post trained models are trained on long term objectives with reinforcement learning and the opportunity to try and identify the effect of this on long term optimization by controlling for base vs post trained models or different reasoning levels is missed. Several claims seem to be overstated or are actually attributes of the environment rather than the LLMs. For example, finding 2 and 6 seem more likely to be a property of the environment rather than the LLMs and any optimal agent would likely have similar results.
Questions for authors
- How strong is a simple RL baseline of these games? Could such models trained with short term and long term objectives not be more useful as a baseline of short and long term oracles?
- Are baseline boss-success rates computed over all started runs or only runs reaching each boss? How does the comparison with deck-forced runs isolate resource management from earlier combat failures and resource attrition?
- Why is the resources not considered part of the state? Is there some mechanism that prevents this from being the case?
7 comments
Evaluation Metrics (2/5)
W1.2: The problem of unclear indicator definitions.
Thank you for raising this question. The detailed definition and calculation of the indicators are shown in the table below. We will optimize them in the revised version.
| Primary Dimension | Secondary Dimension | What It Measures | Computation |
|---|---|---|---|
| Overall Progress | — | How far the agent advances throughout the game | Uses game-native progression metrics: cumulative Floor in STS2, Ante in Balatro, and Rent in LBaL. |
| Execution | Efficiency | How effectively the current resource system converts available resources into useful outcomes | . : number of local tasks; : total effective output obtained in the -th task; : amount of resources consumed in the -th task. |
| Execution | Task Cost | The action cost required to complete local tasks | . : number of local tasks; : number of actions required to complete the -th task. |
| Execution | Burst | The peak output capacity of the current resource system | . : number of local tasks; : the -th highest single-action output in the -th task, with outputs sorted in descending order. |
| Active Management | Intervention Frequency | How frequently the agent proactively adjusts its resource configuration | . : number of resource decision points at which the agent performs an intervention; : total number of available resource decision points. |
| Active Management | Management Diversity | The diversity of resource management mechanisms used by the agent | . : number of distinct management action types used in a trajectory; : total number of available management action types (6 in STS2, 8 in Balatro, and 4 in LBaL). |
The following table summarizes the game-specific definitions of the variables used in the evaluation metrics above:
| Variable | STS2 | Balatro | LBaL |
|---|---|---|---|
| Total damage dealt in a combat | Total score obtained in a blind | Total gold obtained in a rent stage | |
| Total energy spent in a combat | Number of hands played in a blind | Number of spins performed in a rent stage | |
| Number of combat turns | Number of hands played per blind | Number of spins per rent payment | |
| -th highest single-turn damage | -th highest single-hand score | -th highest single-spin reward |
For Execution, the three sub-metrics are first computed as trajectory-level raw scores for each individual trajectory. We then average these raw scores across all complete trajectories for each model. Each sub-metric is subsequently normalized by dividing by its 95th percentile , calculated over all trajectories within the corresponding game environment; Task Cost is inversely normalized. The final Execution score is the equally weighted average of the three normalized sub-metrics.
For Active Management, the two sub-metrics are first computed for each individual trajectory and combined into a trajectory-level score using: ABM = ×100. We then average the trajectory-level scores across all complete trajectories for each model. Finally, the model-level score is normalized by the P95 of trajectory-level ABM scores across all trajectories within the corresponding game environment, yielding the Manage score.
Research Questions and Presentation (3/5)
W1.3: The research questions and findings seem disconnected and the paper does not clearly connect them.
The three research questions in our paper are directly addressed by the reported findings, but we agree that their correspondence could be made clearer. The mapping is as follows:
-
RQ1: Can current LLM agents effectively perform Long-Horizon Resource Optimization? Addressed by Findings 1 and 2, which evaluate agents' overall long-horizon performance and reveal the gap between task progression and successful completion.
-
RQ2: What capabilities affect agents' Long-Horizon Resource Optimization? Addressed by Findings 5 and 6, which use controlled ablations to examine the respective roles of in-combat execution and active resource management.
-
RQ3: What behavioral patterns do agents exhibit? Addressed by Findings 3 and 4, together with the Case Study in Section 3.5, which analyze agents' resource-management behaviors, differences across agent configurations, and the contrast between human and agent decision-making over long horizons.
We will make these connections more explicit in the revised version to help readers follow how the experimental findings answer the research questions.
W1.4: Other Presentation Issues (Appendix, Figures, Data Scales, and Citations)
Appendix and Figures
Thank you very much for pointing out these issues. In the revised version, we will revisit both figures, correct the Buy and Remove icon assignments in Figure 2, and adjust the axis presentation in Figure 4 to improve clarity and readability. We will also carefully review and polish the Appendix to improve its overall writing quality.
Data Scales
For cross-game progression, we did report each game's native progression metrics and will improve this presentation in the revised version. We also report normalized average progression in the Prog. Avg column of Table 1, which expresses progress as a percentage of each game's win threshold.
Citations
Thank you very much for pointing out the issues with citation formats and publication versions. For engineering reference projects such as BalatroBot, BalatroLLM, and sts2-cli, which do not have corresponding academic papers, we used their GitHub handles as citation identifiers in the original manuscript. In the revised version, we will verify the project authors' information and standardize these citations. We will also comprehensively review the cited versions and prioritize conference or journal versions for formally published works. In particular, we will update the citations for CivRealm, Voyager, PTCG-Bench, TRIP-Bench, UltraHorizon, and DeepPlanning accordingly.
We thank you again for these helpful suggestions, which have helped us identify areas for improvement in figure presentation, writing quality, and citation practices. We will carefully address each of these issues in the revised version.
Game Mechanics (1/5)
Dear reviewer p8Jb:
Thank you for recognizing our model comparisons and its value as a starting point for long-term optimization research. We have carefully prepared responses to address your concerns.
Thank you for raising this concern. We will clarify the descriptions of the game mechanics already included in the paper and provide a more detailed introduction to each of the three games.
Existing Overview of the Game Mechanics. Figure 1 already provides an intuitive abstraction of the game mechanics: players start with an initial set of resources and must sequentially face increasingly difficult challenges. As they progress, they continuously acquire, skip, remove, and upgrade components in their resource pools. Different components may exhibit complementary, synergistic, or conflicting relationships. Ultimately, game performance depends on whether players can continuously manage and optimize their resources to construct a synergistic build capable of meeting increasingly demanding challenges.
Detailed Introductions to the Three Games.
Slay the Spire 2: Players explore a branching map, progressing through successive floors by engaging in combat, encountering events, and visiting shops. They strengthen their builds by acquiring, upgrading, and removing cards, as well as collecting relics and potions, with the ultimate goal of defeating the Bosses at each stage. The core gameplay revolves around constructing and continuously optimizing a resource system consisting of cards, relics, potions, and other components.
Balatro: Players form different poker hands by combining playing cards and enhance their scoring capabilities using Jokers, Tarot cards, Planet cards, and other components. At each stage, they must meet an increasingly high score target within a limited number of hands and discards. The core gameplay revolves around managing resources such as the deck, Jokers, consumables, and money to construct a synergistic system that can consistently generate high scores.
Luck be a Landlord: Players generate coins through a slot machine to pay increasingly expensive rent. In each round, they select from a randomly offered set of symbols to add to the slot machine, and use items to enhance symbol effects and modify the reward mechanisms. The core gameplay revolves around constructing and continuously optimizing a resource system consisting of symbols and items to sustain long-term coin generation.
Thank you again for raising this concern. In the revised manuscript, we will add a detailed introduction to the game mechanics of all three games in the Appendix.