acceptodds
Under review as a conference paper at ICLR 2027

RepThink: Self-Evolving Representations for LLM Minimax Reasoning in Adversarial Games

Abstract

LLMs possess extensive knowledge of games, yet often struggle to understand concrete states and reason about adversarial consequences. We address two coupled failures: failing to identify decision-relevant relations in game states and failing to compare actions by their worst-case consequences. A correct executable simulator alone is insufficient, because accurately simulated states may still be difficult for the LLM player to interpret for decision-making. We introduce RepThink, which self-evolves a representation skill that transforms game states into relational evidence and explains its meaning and applicability. The LLM player reuses this skill across current and simulated states within a MAX–MIN procedure, alternating between the two players’ objectives and comparing the worst examined continuations. After games, an evaluator agent diagnoses decision traces, revises the skill, and validates revisions by replaying the same positions. Across five board games, continued evolution improves performance under fixed-depth minimax reasoning. Against Stockfish limited to Elo 1320, RepThink with depth-two reasoning raises DeepSeek’s Chess win rate from 12% with Base ReAct to 42%. Our theory distinguishes evidence-interpretation errors from MAX–MIN reasoning errors and derives directional bounds on their effects on decision quality.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language model, self-evolving agents, environment representations, minimax reasoning, two-player games

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_JmWNOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The work proposes a skill evolution method through the MiniMax framework, where the agent tries to maximize the reward from the current state, and the same agent plays the opponent’s reply and picks the one that is worst for itself (adversarial simulation). Once the games are done, an evaluator agent audits the logs and improves the skill and the harness around the agent.

Critical strengths
  1. The work addresses a practical problem of harness evolution. Adversarial review, in general, seems to improve results across many domains.
  2. The paper reports clear improvements in chess with Rep-think, and it is intuitive how the methods help avoid clear blunders.
Critical weaknesses
  1. Frontier models are not evaluated. The work only evaluates DeepSeek v4 Flash and MiniMax 2.5. Ideally, we want the models to use test-time compute to simulate before taking actions. So, evaluating frontier models with adversarial reviews would help us identify whether separate simulation improves performance or not.
  2. Lack of novelty. Simulating a step before taking a decision is a well-known method. It is hard to pinpoint how the Mini-Max framework is different from adversarial simulation. Appendix E provides evaluator instructions, but the complete player and baseline prompts are not provided, which makes it difficult to understand how the authors are injecting the MiniMax narrative.
  3. Minimal information is provided about the evaluation protocol of baselines. In this line of work, it is very important to compare different methods with tuned hyperparameters. Also, trade-offs in inference compute between Rep-think and other variants are not discussed.
  4. The paper is hard to follow. Figure 2 has overlapping text and arrows.
Questions for authors
  1. How are MiniMax simulations different from normal adversarial simulations?
  2. I see adversarial simulations as a form of test-time scaling method. Instead of scaling depth, can we scale the width during each play? i .e., sample n times -> then do adversarial review for all the moves; this can be one agent given all moves to choose the best move or different agents doing adversarial simulation for each unique move, then choose the best possible move. Compare compute-matched depth vs. width methods.
Reviewer_mf7tOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces a new game playing multi-agent system, RepThink, which involves two parts, which are supposed to tackle two failure modes of LLMs playing games: wrong state interpretation and wrong evaluation of the moves in the future. Firstly, while playing the game the Game-Agent is forced to follow minmax algorithm structure, LLM decides which branches should be selected, how to propagate the value etc. The states during such LLM “MAX-MIN” procedure are evaluated with the skill (python code is run + skill in md). Secondly, the skill is evaluated and changed by the Evaluator Agent once per few games. Furthermore, authors perform theoretical analysis of their solution and propose bounds on the error of their method. Lastly, they evaluate the solution on multiple games against MCTS, and a larger experiment in chess against Stockfish 1300, showing their method can outperform (with different strength) other LLM-agents. They follow with an ablation experiment, testing the usefulness of the depth level and skill evolution.

Critical strengths
  1. The experiments are performed on many known games and chosen benchmarks seem to be thorough.
  2. The paper proposes an interesting combination of using known solutions (MAX-MIN) and LLMs. Using LLM as the guide in which branches should be expanded seems like a good heuristic, whereas, using structure that is known to work in solving sequential problems. If the experimental results were convincing, that would be an interesting result.
  3. The idea to evolve a skill of an agent (including python code) based on the feedback is interesting.
  4. Figures are clean and help clarify what the paper is about.
Critical weaknesses
  1. I am not sure of the goal of the paper. If the goal is to play board games with LLM Agent as well as possible, then solutions such as asking the large model to implement MCTS seem to be enough, and making smaller LLM part of the minimax seems to be unnecessary. If the goal is to have a framework which is applicable to real world problems, such as the ones you mention in Conclusions, then I would like the authors to demonstrate the transfer. Because right now after reading the paper I am not convinced that it transfers easily. E.g. how would the python skill apply in the domain where states are not easy to represent or are textual description? Or how would the method be expanded beyond zero-sum two-player games?
  2. The Game-Agent and Evaluator Agent use two different LLM models, Game-Agent has a model that is small (MiniMax-M2.5 or DeepSeek-V4-Flash, both about ~250B parameters) and Evaluator a frontier model from 2026 (GPT 5.4 with unknown number of parameters, but likely between 2 and 6T). It is not stated in the paper what model is used for baselines, but I assume the smaller model. It does not look like a fair match and it is not justified why such a strong model was needed but only for the Evaluator. I believe that comparing methods with access to different strength of the LLMs is not fair and a bit misleading.
  3. Section 5: The theoretical results section is not easy to follow. Besides, from my understanding of the theoretical results, they are not very meaningful. More specifically, Proposition 5.1 says that the Game-Agent’s judgment error is never larger than the sum of errors coming from “information lost by the evidence” and “LLM’s deviation from the reference interpretation”. But what else can influence the judgment error? I don’t see the reason why this is not holding trivially. Theorem 5.2 bounds the regret of playing a, by the sum of five different terms, without any meaningful interpretation of either what this means or whether it is a meaningful bound. Moreover, can \alpha and \beta_\hat{a} be called an approximate regret? If so, having the regret of playing action a upper bounded by the approximate regret of playing action a plus sum of 4 different terms does not sound meaningful.
  4. There are no experiments regarding Section 5, e.g. showing the usefulness of the bounds from Theorems in practice.
  5. The experiments section lacks some commentary or explanation on the results. The results section mostly describes with high detail exactly what I can find in the results tables. E.g. RepThink has the largest improvement in Chess and Generalized TTT, why is that?
  6. Majority of the results lack standard deviation, only Table 1 presents SD but only for RepThink round 10.
  7. The failure modes from Figure 1 are not justified by experiments. The authors mention it but then do not show that this failure mode indeed happens a lot nor they show what are other failure modes, e.g. what later the RepThink can’t solve. I also would like to see if RepThink actually solves the presented failure modes.
  8. The paper contains a lot of phrases that in my opinion lack the substance (aligned with LLM like texts). Besides, the sentences don’t read well, there are a lot of “;” signs etc. E.g. “Python changes can expose missing relations; Markdown changes can clarify their interpretation and use under the fixed MAX–MIN rule.”, why could python change expose that some relation is missing? And then how a Markdown change can clarify the interpretation of missing relation?
  9. In ablation experiment Representation only (D=0) seems to be almost as good as D>0. Furthermore, it looks like increasing depth (the whole MAX-MIN procedure) does not improve significantly, nor monotonically. The ablation without skill but with MAX-MIN barely improves on the ReAct baseline. It seems that the main advantage of the method comes from using the skill (python + markdown), which is generated by the Evaluator model. From Table 1 and round 1 and round 10 comparison, it looks like the evolution of the skill helps a lot. However, I would like to see more checkpoints to see if the improvement is monotonic or just increases immediately and then plateaus.
Questions for authors
  1. I would like to see experiments on RepThink with the same model as Evaluator and Game-Agent. Additionally, results from more rounds. Could you present such results (with SD)? Or alternatively, results of baselines using GPT 5.4?
  2. Could you specify what models do the baselines use?
  3. How is RepThink’s skill initiated (round 1)? Is it generated by an Evaluator Agent?
  4. Could you please elaborate on the results from section 5 and perhaps explain them more intuitively? Can you explain why they are not trivially true?
  5. Could you compare your work to prompt optimization literature in more detail? Specifically, do other works evolve full skills?
  6. In 4.2, you state that python does not run MCTS nor Stockfish. Did you check that the produced code does not perform MCTS? Same with “The skill exposes threats and consequences but never ranks actions”. Do you mean you forbid it in the prompt or that you have checked that it did not happen?
  7. Are “relations” observations about the state? Why are they called relations? Can you provide what kind of relations RepThink created for the states?
Reviewer_wnb5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Paper Summary

This paper aims to make a frozen LLM play two-player board games better without training it. The authors point out two common LLM mistakes in games: misreading the board state, and not assuming the opponent will make the best reply.

Method. RepThink gives the game-playing LLM a “representation skill”: a Python script that turns a board state into relations (e.g., attacks, threats, forks, connectivity), and a Markdown file that explains how to use them. The LLM uses this skill on both the real board and imagined future boards while doing fixed-depth minimax reasoning (by default, its own move followed by the opponent’s reply). The Python code does not score or rank moves; the LLM makes all judgments. After each batch of games, a stronger LLM (GPT-5.4) reads the game logs, finds mistakes, edits the skill, and checks the edit by replaying the same positions. The game-playing LLMs (MiniMax-M2.5, DeepSeek-V4-Flash), the rules, and the reasoning procedure stay fixed.

Claimed contributions. (1) Evolving code-based state representations for an LLM that makes its own decisions, instead of feeding a code world model to an external planner. (2) An error analysis that splits decision error into missed moves, missed opponent replies, misread states, and wrong min/max comparisons. (3) Experiments across five games.

Validation. The method is tested on Tic-Tac-Toe, Connect-Four, Generalized Tic-Tac-Toe, Hex-7, and Chess, against custom MCTS opponents with 300 or 1,000 simulations. Further evolution (round 1 to round 10) improves scores in nine of ten settings, with the largest gain in Chess (27.0 to 89.7 for MiniMax). Ablations compare language-only minimax, representation only, and different search depths. In Chess against Stockfish limited to Elo 1320, DeepSeek’s win rate rises from 12% to 42%, mostly from fewer big blunders.

Critical strengths
  1. Transfer to an external, calibrated opponent (Chess vs. Stockfish-1320). The skill was evolved against MCTS, not Stockfish, yet DeepSeek’s win rate rises from 12% to 42%, with clear drops in severe errors (1.9% to 1.2%) and gift rate (4.1% to 0.8%), audited by a stronger Stockfish.
  2. Clear ablation showing the representation is the key ingredient. In Chess, language-only minimax scores 4.0, representation only scores 67.0, and both together score 97.0 (similar patterns in other games).
Critical weaknesses

Unclear motivation, limited novelty, and unclear positioning. The paper does not explain why an LLM should be used as the game-playing agent. If the goal is playing strength, standard game AI methods (e.g., AlphaZero-style search with learned evaluation, or Stockfish) are far stronger and cheaper. Common reasons to use LLMs in games are interpretability, generality across games without per-game engineering, or studying LLM reasoning itself. The paper does not evaluate interpretability. Its generality is weakened because each game still gets its own evolved Python feature extractor. It also does not report the cost of many LLM calls per move.

The method itself follows the same loop as many recent self-improving agent methods (GEPA, Trace2Skill, AutoHarness, Reflexion): play, let a stronger LLM read traces, edit an artifact, replay. Here the edited artifact is a state representation. The two failure types (misreading the state, not assuming the opponent’s best reply) are well-known LLM mistakes in games. The fix uses standard minimax and textbook features. The “theory” restates the standard split of search errors (coverage, evaluation, backup) with simple proofs and is not connected to any measured quantity.

Questions for authors

Q1. Why use an LLM as the game-playing agent? If the goal is playing strength, AlphaZero-style methods or Stockfish are much stronger and cheaper. If the goal is interpretability or generality, can you evaluate these directly?

Q2. Are the MCTS opponents your own implementation? Please release the code and calibrate their strength (e.g., Elo, or results against a standard implementation such as OpenSpiel MCTS). Why is the root action chosen by mean return instead of visit count?

Q3. How do MCTS-300 and MCTS-1000 compare head-to-head in each game? Can both LLMs be evaluated against the same opponents?

Q4. Since the skill is evolved and evaluated against the same opponent, how does the evolved skill perform against held-out opponents in all five games (e.g., different MCTS settings, alpha-beta engines, multiple Stockfish levels)?

Q5. What are the results when GPT-5.4 is the Game-Agent, with Base ReAct and with RepThink? What happens if the Game-Agent also serves as its own Evaluator (e.g., DeepSeek evolving its own skill)?

Q6. Why not compare against fine-tuning methods such as Dionisopoulos et al. (2026) or MARSHAL, which the paper cites?

Q7. How was the round-0 skill created, and by whom? Can you show the final evolved Python and Markdown for Chess and Connect-Four, and summarize what was added in each round?

Q8. How much of the gain comes from simple checks such as immediate wins, immediate losses, and hanging pieces?

Q9. What are the results when the LLM plays second?

Q10. Can you run baselines, the round-1 checkpoint, ablations, and the Stockfish experiments with multiple seeds and report confidence intervals and significance tests?

Q11. What is the cost per move (tokens, tool calls) and the total GPT-5.4 cost of evolution? How does RepThink with DeepSeek compare with GPT-5.4 playing directly at similar cost?

Q12. Can you estimate any of the theoretical quantities (alpha, beta, epsilon, eta), for example by asking the agent to output explicit leaf scores or by labeling a sample of decisions by error type before and after evolution?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.