- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper argues that LLM game agents are evaluated mainly by win rate against a population of opponents, and that this does not measure worst-case security. A high payoff against a pool says nothing about how much an opponent who adapts to the agent's weaknesses can win. The proposed measure is exploitability, normalized by the absolute game value |v*|.
Critical strengths
-
The paper moves the evaluation of LLM game agents from win rate against a population to exact worst-case exploitability. Win rate against a pool measures how much an agent profits from the opponents it happens to meet, while exploitability measures how much an opponent who adapts to the agent can take from it. The same concern applies to any leaderboard ranked by pairwise results against a sampled population, so I expect the audit protocol to be reused independently of the training method.
-
The paper separates the requested policy from the expressed policy and reports its experiments completely. Because an LLM shares parameters across information sets, an update that targets the action probabilities at one information set can change the probabilities that the model plays at others. The paper makes this gap an explicit quantity and measures it throughout training. Every run stays in the reported denominators, and the paper discloses its negative results. This makes the strength of each claim easy to judge.
Critical weaknesses
- The method and the evidence apply only to games small enough to solve exactly, which is much narrower than the paper's thesis about win-rate evaluation, and the most direct baseline is missing.
The experiments use games with 6 to 270 information sets per seat, and each training step extracts the full policy and computes an exact best response to it. At this size the exact Nash equilibrium is cheap to compute and already serves as the fallback target, so the method in effect stores a known near-equilibrium table in one LoRA adapter per game, seat, and seed. The paper also gives no route, even for evaluation alone, to games where exact best responses are intractable, and those are the settings where its broader thesis matters.
- The bounds are correct, yet they neither produce the reported results nor select the training objective.
Theorem 1 and Corollary 1 describe iterative training, while the final method replaces iterative RM+ training with a fixed target because Figure 3 shows that iterative training is unstable. In the fixed-target stage, Theorem 1 only supplies an upper bound B_T on the target's exploitability, which the linear program already computes exactly, and the main text never reports the measured size of the three terms.
Proposition 1 shows only that target-reach weights give the tightest bound of this form, which matters little when the bound is vacuous, and I could not find an ablation that compares the two weightings. The 94.4% stable sub-value rate therefore rests on Proposition 2, where an exact audit with rollback keeps every accepted policy below the ceiling. That guarantee is sound but nearly tautological, and it is available only because exact evaluation is cheap in these games.
- How the figures are presented can be improved.
Questions for authors
- The thesis concerns win-rate evaluation in general, so the paper would be much stronger if its evaluation reached a game where exact best responses are intractable. An approximate best response trained with RL, for example, gives a lower bound on exploitability in a larger game, and one such experiment would support the broader claim more than further ablations on small games.
Many thanks for your review!
We also prove that an audit with acceptance ceiling τc and underestimation at most ε for every candidate gives E≤τc+ε for every accepted policy. With the 0.99∣v∣ ceiling, uniform underestimation below 0.01∣v∣ preserves sub-value certification. Our rational verifier checks feasibility and payoff witnesses using exact Fraction arithmetic, independently of floating-point solver tolerances.