acceptodds
Withdrawn from ICLR 2027

Large Language Model-Based Game Agents Adjust Their Play to Graded Instructions for Competing Priorities

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Automated playtesting may require agents to emulate human players at different skill levels, whereas game agents used as non-player characters may need to exhibit a consistent persona. These uses may call for different balances between playing strength and other desired behaviors. For example, in chess and shogi, preserving a win and favoring piece captures need not lead to the same move. We therefore investigate whether and how large language model (LLM)-based game agents respond to graded natural-language instructions specifying the relative importance of these goals. We test game agents built on three LLMs in Animal Shogi, Breakthrough, and Rush Hour on states where following the requested style costs playing strength. We vary relative priority over five levels using numerical, modal, and adverbial instructions. Across all tested models and games, agents exhibit a graded behavioral response, with style-matching actions becoming more frequent overall as the requested style is assigned higher priority.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, LLM Agents, Instruction Following, Game AI, Play Style

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Ff3eOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies whether LLM-based game agents can adjust their behavior according to graded natural-language priorities between playing strength and a requested play style. The authors evaluate Claude Sonnet 4.5, Gemini 2.5 Flash, and GPT-5 mini on Animal Shogi, Breakthrough, and Rush Hour. For each game, they construct positions where following the requested style conflicts with the game objective, and vary the relative priority of the two objectives using numerical, modal, and adverbial instructions.

Critical strengths
  1. The paper is clearly written and the experimental setup is easy to follow. The distinction between playing strength and style adherence is well motivated, and the three instruction formulations are presented clearly.

  2. The experimental design is careful. The use of exact solvers to construct positions where style-following genuinely conflicts with the game objective makes the main behavioral measurement easier to interpret.

  3. The empirical evaluation includes a substantial number of robustness checks. In addition to the main three-game and three-model evaluation, the paper includes repeated sampling at fixed positions, competence conditioning, an open-weight model replication, instruction-formulation comparisons, position-selection sensitivity, and sequential Rush Hour experiments.

Critical weaknesses

W1. The practical significance of the observed style–strength trade-off is limited by the relatively low gameplay competence of the evaluated LLM agents.

The paper demonstrates that increasing style priority shifts action choices away from playing strength. However, the agents' performance ceiling is already fairly limited even when style is assigned zero priority. For example, at 0% style priority, win-retention rates in Animal Shogi are only 0.502, 0.403, and 0.549 for Claude Sonnet 4.5, Gemini 2.5 Flash, and GPT-5 mini, respectively, with even lower exact-optimal-action rates.

This makes the practical meaning of the resulting trade-off less clear. A useful controllable game agent would ideally start from a strong policy and then selectively sacrifice performance to satisfy a requested style. Prior work on strength adjustment has demonstrated this paradigm in much stronger game-playing systems, e.g., by adjusting strong MCTS-based Go agents toward designated human skill levels. In contrast, the present experiments primarily demonstrate controllability within a relatively low-competence regime. Although the competence-conditioned analysis shows that graded responsiveness is not solely caused by gameplay failures, it does not establish whether the same controllability remains meaningful for substantially stronger game-playing agents.

W2. The evaluation does not clearly establish what is uniquely gained by using an LLM rather than an existing preference-conditioned or strength-adjustable game agent.

Multi-objective and preference-conditioned game agents can already adapt their behavior according to desired objective weights, and prior work has demonstrated fine-grained strength adjustment in challenging games such as Go and chess. The main potential advantage of an LLM-based approach is therefore its ability to interpret flexible natural-language specifications without task-specific retraining.

However, the current evaluation uses a small set of predefined, programmatically measurable action-level styles and fixed instruction templates. These settings do not strongly exercise the open-ended linguistic flexibility that would distinguish an LLM-based agent from a conventional conditional policy. Experiments with unseen style descriptions, compositional preferences, or user-specified styles not explicitly anticipated by the evaluation would make the motivation for using an LLM considerably stronger.

W3. The priority grid is too coarse to support strong claims about smooth or fine-grained controllability.

The numerical experiments evaluate only five priority levels: 0%, 25%, 50%, 75%, and 100%. The reported curves connect these observed conditions but do not estimate a continuous response function. The results therefore convincingly establish an ordered or graded response, but they do not show that behavior varies smoothly or predictably between these widely spaced points.

This distinction is important if the intended contribution is fine-grained control over the style–strength trade-off. A much denser priority grid, ideally together with held-out intermediate values, would allow the authors to test whether the response genuinely interpolates between specified priorities rather than exhibiting threshold-like transitions that appear smooth only after connecting a small number of observations. This issue is even more pronounced for the modal and adverbial formulations, whose levels do not have a natural quantitative spacing.

W4. The action-parsing fallback may introduce an unquantified implementation bias into the single-decision results.

When no legal action identifier can be parsed from a model response, the implementation selects the first action in the supplied legal-action list. However, the reported validity audit only verifies that the stored actions are legal and does not separately report how often this fallback was triggered in the single-decision experiments. Therefore, the reported zero invalid-action rate does not rule out parser-induced changes to the measured action distribution.

This could matter if the ordering of legal actions is not randomized, since fallback events could systematically favor particular action types and thereby affect style-match or objective-retention rates. The paper should report the fallback frequency for each model and condition, clarify how legal actions are ordered, and ideally verify that the main results are unchanged when fallback cases are excluded or when legal-action order is randomized.

These are important becuase LLMs are not the best in the game domain. Therefore gaming can become both good benchmarks and diagnosis for the current mechanism. However, with limited ability, we are just not there yet to discuss the style of gaming and decrease the performance on LLMs. This is a quite centric problem when formulating the idea of the paper. I do think a better way to discuss is to at least mantain the ability while having different styles. I will consider the rebuttal but now I don't think this is really helpful for the community just yet. This paper is borderline between a score of 1 and 2. The writing itself is not that bad, but based on my current assessment, I lean more toward a 1. I will carefully consider the rebuttal and see how many of these concerns can be adequately addressed before deciding whether the score should be raised.

Questions for authors

Questions for Authors

  1. How should the practical value of the demonstrated style–strength trade-off be interpreted given the relatively low gameplay competence at the strongest-performance endpoint?
    Even at 0% style priority, the evaluated agents fail to preserve solver-established wins relatively often in some settings. Do the authors expect the same controllability to persist when starting from a substantially stronger game-playing policy, where reducing performance would represent a trade-off from a genuinely high-performance baseline?

  2. What is the main advantage of using an LLM for this problem compared with existing preference-conditioned or strength-adjustable game agents?
    The current styles are predefined, simple, and programmatically measurable. If the intended advantage is zero-shot natural-language flexibility, could the authors evaluate unseen style descriptions, compositional preferences, or styles that were not explicitly anticipated when constructing the benchmark?

  3. Would the observed graded response remain smooth under a substantially denser numerical priority grid?
    The current numerical evaluation uses only five levels: 0%, 25%, 50%, 75%, and 100%. These results establish an ordered response, but a coarse grid may hide threshold-like behavior between adjacent levels. Could the authors evaluate additional intermediate values, preferably including held-out priorities, to determine whether the behavior genuinely interpolates smoothly?

  4. Does the observed controllability extend to long-horizon competitive gameplay rather than primarily isolated decisions?
    The complete-episode evaluation is performed only in Rush Hour, which is a single-player puzzle. Could the authors evaluate complete Animal Shogi or Breakthrough games to determine whether a requested style remains coherent across multiple decisions while interacting with an opponent?

  5. How representative are the conflict-selected positions of the situations encountered during ordinary gameplay?
    The main evaluation deliberately selects states where every style-matching action conflicts with the game objective, while the appendix indicates that such conflicts occur in only a minority of style-available randomly sampled positions. Could the authors quantify the resulting style–strength trade-off on unconditioned gameplay states or full episodes, rather than primarily on states selected to maximize identifiability of the conflict?

6.How often was the first-legal-action fallback triggered in the single-decision experiments, and how were legal actions ordered in the prompt?
The current validity audit establishes that all stored actions are legal, but it does not separately identify parser fallback events. Could the authors report fallback rates by model and condition and verify that the main results remain unchanged after excluding such cases or randomizing the legal-action order?

Reviewer_jQzyOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether LLM-based game agents adjust their action choices in response to graded instructions specifying the relative priorities of playing strength and a requested play style. The authors evaluate three LLMs in Animal Shogi, Breakthrough, and Rush Hour using five priority levels expressed through numerical percentages, modal verbs, and adverbs. The main evaluation uses solver-verified positions where no style-matching action preserves a forced win in the two-player games or maintains an optimal solution in Rush Hour. Across all three instruction types, higher style priority generally leads to more style-matching actions, with numerical instructions producing the most consistent ordering across priority levels.

Critical strengths

The paper demonstrates that game-agent behavior can be adjusted through different forms of natural-language priority instructions without additional training. Numerical priorities, modal verbs, and adverbs generally shift action choices in the intended direction.

Critical weaknesses

My main concern is with the experimental setup.

  • The evaluated styles are closer to explicit action-selection rules than meaningful play styles. For example, “Giraffe” specifies which piece to move, and “Rear-advance” instructs the agent to move a least-advanced pawn. These tasks require little reasoning to translate a high-level preference into a strategy. Evaluating styles that guide multiple decisions, such as exploration or collection, would better demonstrate meaningful style control.

  • The findings offer limited insight beyond increased compliance with more strongly emphasized instructions. The main evaluation deliberately selects positions where a trade-off exists between style matching and playing strength. The observed style-performance trade-off reflects this choice of evaluation positions. While this design isolates priority responsiveness, it remains unclear what the results show about style control beyond these constrained choices. I therefore find the contribution of the current experiments limited. To strengthen the contribution, the authors could, for example, evaluate whether agents follow a style when it is compatible with strong play and avoid unnecessary performance loss when the objectives conflict.

Questions for authors

Can the authors provide an example where following the requested style requires planning across multiple decisions, rather than selecting actions with a specified local property? This would help clarify whether the findings extend to broader play styles.

Reviewer_bEYPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates LLM agent responsiveness to graded natural language instructions with specified priorities over three game environments (Animal Shogi, Breakthrough and Rush Hour).

The setting being automated game play as a game development requirement, for example playtesting that simulates human players at different skill levels, or NPCs with a consistent persona. The agents should be controllable to guide the balance between their strength and style of play.

The core claimed contribution is a structured evaluation of three LLMs (Claude Sonnet 4.5, Gemini 2.5 Flash and GPT-5 mini) over three games, five priority levels and three instruction formulations, along with the finding that style-matching becomes more frequent as style receives higher priority.

Priorities over a requested style and the game objective are given to the LLM through a structured system message. Weighting or prioritization over the two is provided as a percentage totalling 100%; there are five levels.

As alternatives to the percentages, the authors also express the same five levels using modal verbs (‘must’ vs ‘can’) and adverbs (‘strongly’ vs ‘slightly’), giving three separate instruction formulations.

Verbs and adverbs are grouped into sets based on priority: where the priority is high the sets contain authoritative words such as ‘must’ / ‘very strongly’, and the opposite for low priority, e.g. ‘may’ / ‘slightly’, with the lowest level stating that no priority is assigned.

The authors evaluate on conflict positions identified using exact solvers to avoid confounds where style-matching and objective matching agree on the same action.

Results show that increasing the instructed priority on style results in more frequent style-matching and less game objective actions across the game scenarios. The authors report that numerical conditioning gives the largest and most consistently ordered response.

Critical strengths

The authors’ experimental design is well thought through. Evaluating only conflict positions identified by an exact solver makes the tradeoff between following style and winning the game measurable. If the style action had also won, it would be impossible to tell which instruction drove the choice.

With that said, the design does not address every confound. For example, the model may not know the correct game winning action, and the style instruction could simply be biasing an otherwise random decision. Still, the evaluation design is backed by a strong set of controls, including a baseline without style instructions, repeated sampling at fixed positions, and a series of fixed win instructions.

Critical weaknesses

The main result is expected, and as such I think the contribution is too small and limited for ICLR venue. When an instruction-tuned model is told to prioritise a specified style or move and actively not try to win the game, it is expected that style matching increases. Style matching is correct in sign, but the more interesting result is that the rate does not match the stated percentage, and the mismatch depends on both the model and the game.

For example, at 50% style priority Claude follows the style about 60% of the time in Animal Shogi (Table 2) but rarely in Rush Hour (Figure 8). To achieve controllable agents in game play, it seems one would have to tune the commanded rate per model and per game to get a graded agent at some desired level. For example, to reach 50% style matching, Claude might need a setting between 25% and 50% in Animal Shogi but above 75% in Rush Hour.

The authors do not claim that stated percentages should be matched, but without this the practical value for controllable game agents is unclear. It would strengthen the paper to show whether a desired style rate can be reached reliably for each model and game.

Questions for authors

n/a

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.