acceptodds
Under review as a conference paper at ICLR 2027

PokaiTrainer: Scaling Equilibrium Search to Competitive Pokémon VGC

Abstract

Decision-time equilibrium search carried poker to superhuman play, but it has so far relied on tractable subgames: a handful of actions per decision, chance confined to card deals, one player moving at a time. Competitive Pokémon in its official doubles format (VGC) breaks all three assumptions at once. Both players act simultaneously from joint menus in the hundreds, each joint action resolves to hundreds of stochastic outcomes, and the opponent's reserves and stat allocations are hidden. No prior Pokémon agent performs equilibrium search, and whether it scales to this regime was open; we show that it does, and report what it took. PokaiEngine, our Rust battle engine, enumerates a joint action's full weighted outcome distribution in one pass, at % parity with Pokémon Showdown and a fraction of the cost of sampling it. PokaiTrainer adapts Student of Games to this scale and trains it by self-play over hundreds of human teams. Each decision is solved by counterfactual regret minimization as a Bayesian matrix game over public belief states, subgames grow under an explicit compute budget, and value targets are harvested from the interior of every solve and grounded by realized outcomes. The strength is in the search. The network's policy alone loses even to a shallow heuristic search. PokaiTrainer is, to our knowledge, the first VGC agent rated on the live Showdown ladder. Under open team sheets it wins 59% of 150 best-of-three sets against a human field averaging Elo, holds a 1350-1400 Elo band, and at its peak reached 1492 Elo, entering the format's top 500.

Primary area:
reinforcement learning
Keywords:
equilibrium search, decision-time search, imperfect-information games, simultaneous-move games, stochastic games, counterfactual regret minimization, public belief states, self-play, Student of Games, Pokémon VGC, game AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_R4tMOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies whether the equilibrium search method can be scaled to competitive Pokémon VGC. To do this, they first propose a Rust battle engine, PokaiEngine, to enumerate a joint action’s full weighted outcome distribution in one pass. Then, they propose PokaiTrainer, which adapts Student of Games to the competitive Pokémon setting and trains it by self-play over hundreds of human teams. Finally, they conduct experiments, and the results show that on the live Showdown ladder, it wins 59% of 150 best-of-three sets against a human field averaging ∼1320 Elo, holds a 1350–1400 Elo band, and at its peak reached 1492 Elo, entering the format’s top 500.

Critical strengths

The main contribution for this paper is the PokaiEngine, which can enumerate the full weighted outcome distribution of a joint action in one pass. The engine makes the used method feasible since it can also enumerate all chance nodes.

Critical weaknesses

The method seems to be incremental, based on Student of Game and special for Pokémon, and the paper provides no evidence that its adaptations generalize, or even that they are necessary in Pokémon. Specifically, there is no comparison with the original Student of Game or ReBeL to demonstrate that these adaptations are necessary.

This paper is hard to understand, and the presentation of this paper could be improved. This limits the paper’s accessibility to the broader audience.

Questions for authors
  1. To assess the benefit of the proposed adaptations, could the authors provide the results of the vanilla Student of Games or explain why the baseline is missing?

  2. For the experiments on the live Showdown ladder, was any other baseline agent evaluated under the same conditions? With only testing PokaiTrainer, it is difficult to determine the performance attribution.

4 comments

Thanks for the review.

Q1. Neither vanilla SoG nor vanilla ReBeL runs on VGC as is, which is why there's no such row. Run on a standard Pokémon simulator, they'd have to sample chance outcomes, at ~500× the cost of our exact enumeration, and solve full joint-action menus of ~10⁴ cells per node. We did swap in some of their components one at a time. SoG's node-count expansion matches our strength at 19% more search time. ReBeL's exhaustive fixed-depth subgame wins only 43% against our expansion, and its fictitious-play solver trails CFR. These comparisons will move to the main text. We agree the training-side changes aren't shown to be necessary, and the revision presents them as design choices rather than contributions. On generalization, we agree the adaptations are specific to VGC, and our reply to F2N8 (Q1) covers what we think could carry over.

Q2. We didn't. No other VGC agent had reported a ladder rating when we ran ours, and we chose not to add more bots, since a 75-set run measures win rate only to about ±11 points, and we wanted to keep our footprint on the public ladder small. Attribution comes from controlled comparisons instead, against earlier training rounds, heuristic search, a PPO policy, the behavior-cloned parent, and a non-equilibrium solver.

On the presentation, other than the internal run names, which parts were hardest to follow? We'd love to fix them if possible.

Thanks again for the review. We've uploaded the revised PDF, and much of it targets the presentation issues you raised. §4 now opens with a table defining the formulation and a diagram of one turn decision, Table 3 shows where we depart from SoG and ReBeL, and the terminology is more consistent throughout. We'd really like to know whether it reads more easily now, and which parts are still hard to follow. Happy to answer anything else before the discussion closes.

Thanks for the authors' response.

These responses partially addressed my concerns. The remaining concern is whether the adaptations can generalize to other games. If not, this paper is likely to adapt SoG to the specific Pokemon game. Furthermore, the authors agree that the training-side changes aren't shown to be necessary. Therefore, I would maintain my score for the current version.

For the presentation, the main issue is the Pokemon game terminology. Some VGC terms are used without explanation, which may influence understanding of the game. Additionally, Figure 1 is still too abstract to understand.

Thanks for the reply and for the constructive feedback on the presentation! We've uploaded a revision that adds a glossary of the VGC terms we use, including how the stats work (App. F), and made Figure 1 more concrete, with HP numbers for one example attack and a caption that explains each chance event. Hopefully that helps.

On generalization, we agree the adaptations are shaped by VGC, and we see this mainly as an empirical paper. It shows that SoG-style equilibrium search scales to a game where chance follows nearly every action, with hundreds of outcomes per joint action every turn. In SoG's own domains chance appears only in poker's card deals, which come between betting rounds, so a subgame can stop before a deal instead of branching on its outcomes. A VGC subgame branches after every joint action inside it, so the outcomes compound with depth. The piece we expect to carry over is the enumeration engine, for games where chance follows most actions and its outcomes are public, such as turn-based RPG combat.

Reviewer_F2N8OpenReview ↗
Rating
3
Confidence
4
Informativeness
2(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper presents PokaiEngine, a battle engine that efficiently enumerates weighted stochastic outcomes, and PokaiTrainer, a Pokémon VGC agent based on decision-time equilibrium search. The agent adapts Student of Games by combining CFR over restricted Bayesian matrix games, budgeted search expansion, learned continuation values, and inference over hidden opponent configurations. It is trained using human replays and self-play, and evaluated through engine comparisons, agent matchups, and live games against human players on the Showdown ladder.

Critical strengths
  1. The engine supports efficient evaluation of stochastic outcomes. Comparisons against sampling and checks against Showdown provide useful evidence about its speed and correctness.
  2. Evaluation against human players on the live ladder provides evidence beyond self-play and internal agent comparisons.
  3. The paper openly reports limitations, including weak policy-only performance, diminishing training gains, and persistent errors in the learned value function.
Critical weaknesses
  1. I appreciate the engineering and experimental effort, but I have difficulty identifying a clear technical contribution beyond building a Pokemon environment and integrating existing learning and search methods. The authors explicitly describe the search as an adaptation of Student of Games rather than a new algorithm. Simultaneous moves, hidden information, and stochastic transitions are established features of extensive-form games. Their combination makes computation demanding, but the paper needs to explain more clearly which adaptations provide new insight or capabilities beyond this particular application.
  2. The equilibrium-quality measurements do not establish low exploitability in the full game. The reported NashConv values concern restricted subgames and do not account for excluded actions, chance pruning, or errors in learned continuation values. The local best response also uses the agent's own payoff matrices, so it can miss deviations omitted or misvalued by that model. No full-game exploitability curve is provided to show how equilibrium quality changes during training.
  3. The policy used in some evaluations differs from the solved equilibrium mixture. Internal matchups often use argmax actions, while ladder play changes the mixture's temperature. The favorable local best-response result for the full mixture therefore does not directly establish robustness of the policy used on the ladder.
  4. Many training adaptations do not show a clear benefit in the reported ablations, which the authors acknowledge are often within evaluation noise. Together with the single training seed, this makes it difficult to identify which components are necessary and which empirical findings are reliable enough to generalize beyond this implementation.
Questions for authors
  1. Could the authors clarify which adaptations they consider the main technical contributions, and which findings might be useful beyond VGC? This would help me better understand the broader significance of the work.
  2. Would it be feasible to supplement the equilibrium-quality measurements with a responder that considers actions outside the agent's shortlists and evaluates continuations independently of its value estimates? An estimate across training checkpoints would also help clarify how equilibrium quality develops. If this is computationally prohibitive, a discussion of the limitations would be helpful.
  3. Could the authors discuss how using the full solved mixture versus the ladder's temperature-adjusted policy affects playing strength and robustness to an adapting opponent?
  4. Given that several ablation results fall within evaluation noise, could the authors clarify which adaptations have the strongest empirical support? Any available evidence about sensitivity to the training seed would also be helpful.
3 comments

Thank you so much for the thorough review!

Q1. We agree the formalism isn't new, and our novelty beyond Pokemon is limited. Our contributions are the engine and the adaptations that make search run at VGC's scale (exact chance enumeration, spreads as mixtures inside each world, belief-ranked joint shortlists, and compute-priced expansion). Each exists because a plain version doesn't fit at this size (hundreds of joint actions per side, hundreds of chance outcomes per joint action, and ~10⁷ possible stat spreads per Pokémon). Beyond VGC, the part most likely to carry over is building an enumeration engine for games where chance follows most actions and its outcomes are public, such as turn-based RPG combat. The revision will make this more explicit.

Q2 and Q3. Great catch on the equilibrium quality measurements, and thank you. Our original best response reused the agent's own payoff matrices so it couldn't find much. The new responder as you suggested, can play any legal action, and optionally value every action with a separately trained network instead of the agent's own. Each cell is 512 games (256 for the shortlist column), and 50% means unexploited.

Agent playsShortlist BRFull-menu BRFull-menu BR, separate network
full mix49.654.751.4
ladder (T=0.5)52.366.652.3
argmax77.783.473.0

The full-menu gains come from joint actions off the agent's shortlist. The separate network is from an earlier, weaker run of ours, so it isn't fully independent. T=0.5 is ~3pp stronger than the full mix (49.0% vs 45.8%, both against our argmax policy) but more exploitable, which we accept on the ladder since opponents never see our distribution, and our win rate doesn't drop later in a Bo3 set (51.3% in game 1, 50.5% in game 2, 61.8% in game 3).

An exact best response would need the agent's strategy at every point in the game, which our agent only computes on demand by searching, so we track exploitability across training with the full-menu responder against the full mix instead (512 games each, about ±4pp).

Training round8121620242832
Full-menu BR win %59.856.258.761.756.457.854.7

Exploitability stays between 55% and 62% across training. Round 32 is the lowest, but the downward trend is within noise, so we can't say training steadily reduces it. We'll narrow the intro's robustness claim to what these measurements show.

Q4. The enabling adaptations are supported by cost rather than win rate. Exact enumeration is ~500× cheaper than sampling, full joint menus cost 1.84× for no gain, and SoG's node-count expansion matches ours at 19% more search time. In win rate, ranking the opponent's shortlist by belief is the clearest effect (+16pp). Grounding, interior rows and augmentation show no measurable effect.

For seeds, we trained four more to round 5 and played every pair 4,096 games. They differ by ~5 Elo (sd), against ~90 Elo gained over those 5 rounds, so ablation noise comes mostly from evaluation games rather than seeds.

Thank you for the clarifications and additional experiments. I will maintain my initial positive review and increase the informativeness and soundness scores.

Thank you again for the constructive feedback!

Reviewer_KrN4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PokaiTrainer, an equilibrium-search agent for competitive Pokémon VGC. Building on ReBeL and Student of Games, the authors develop a branching battle engine, adapt CFR to simultaneous-move Bayesian subgames, and train policy and value networks through search-based self-play.

Critical strengths
  1. The branching simulator and compute-budgeted search are thoughtfully designed. The engine validation and performance comparisons are convincing.
Critical weaknesses
  1. The claimed departures from SoG show no measurable effect in ablations.
  2. How should we interpret the gap between improvements in internal search evaluations and the nearly unchanged ladder performance?
  3. Only the CFR agent was evaluated against humans, so ladder strength cannot be attributed to the equilibrium property.
Questions for authors

Same as weaknesses.

4 comments

Thanks for the review.

W1. Agree for the training choices (grounding, interior rows, augmentation), they're within noise as App. E.4 reports, but we listed them alongside the necessary adaptations, so they might have read as contributions. The revision will present them as design choices. The departures that matter are the ones VGC forces, and the evidence for them is cost. Exact chance enumeration is ~500× cheaper than sampling, and full joint-action menus cost 1.84× for no gain. We also tested compute-priced expansion directly. Swapping in SoG's node-count expansion gives the same strength at 19% more search time.

W2. The scaled search shape beats our native shape 58% internally but showed no ladder edge, and to be honest we don't have a concrete explanation. We did rule out two causes. The edge holds at the ladder's sampling temperature (58% at T=0.5), and the ladder agent used its full search budget on 97% of decisions (not cut short by time). Our guess is that with the same network on both sides, the deeper search wins reliably by seeing the same lines but slightly further, while humans play differently enough that the extra depth matters less. Each run's 75 sets also have a lot of variance, both from outcome randomness and from team/opponent matchups, so the results can't rule out a smaller gain.

W3. Agreed, and we don't attribute ladder strength to the equilibrium. A non-equilibrium solver plays our agent at parity (51.7%), so the strength comes from search. The equilibrium is there for robustness. A best response that sees the agent's strategy and can play any legal action wins 82.0% against the non-equilibrium agent's mix, but only 54.7% against ours (66.6% against the sharper policy we sample on the ladder). Humans in a Bo3 do adapt to an opponent's tendencies, especially its opening strategy. If they were exploiting our agent, our win rate would drop after game 1, but it doesn't (51.3% in game 1, 50.5% in game 2, 61.8% in game 3).

Thank you for the response and the additional experiments.

My assessment is unchanged. Since the paper claims no algorithmic novelty, its case rests on the empirical findings, which remain weaker than presented. The response confirms that ladder strength comes from search, not the equilibrium, and the deployed ladder policy (T=0.5) is exploited at 66.6% by an unrestricted best response, so the robustness argument does not hold for the deployed agent. In addition, the 58% internal gain from deeper search does not appear on the ladder.

Thanks again for the review. We've uploaded the revised PDF. Following your W1, §4 now separates the departures VGC forces from training choices that tested within noise (Table 3), and the SoG and ReBeL component swaps and the decoupled UCT results are in §5.2. We'd love to hear whether the revision addresses your concerns, and especially whether the paper reads more clearly now. Happy to answer anything else before the discussion closes.

Oops, sorry, our last comment went out before we saw your reply (we hadn't refreshed the page). Thanks for the reply.

Our ladder is best-of-three, which is why we treat the equilibrium as necessary rather than optional. Within a set the opponent sees our play and can adapt, and a deterministic policy repeats the same openings. In our measurements the deterministic version of the agent loses 78–83% to a best response, against 55% for the solved mix. On the ladder the win rate does not fall from game one to game two (51.3% vs 50.5% on the same sets), which is what we'd hope to see. The T=0.5 policy we deploy is a deliberate middle ground. It plays 3pp stronger than the solved mix, and while a responder that sees its full distribution can exploit it, ladder opponents never see that distribution, and a Bo3 is too short to learn it from play (unlike a poker session of hundreds of hands).

We reported the missing ladder gain from deeper search because we found it genuinely puzzling. None of the paper's claims depend on it.

We'd also like to highlight that, to our knowledge, PokaiTrainer was the strongest VGC agent reported at the time of submission, peaking top 500 on the live ladder.

Comments on the paper

2 comments

Thanks to all three reviewers. We're still working on the paper revision, but general updates:

  • We've separated the adaptations VGC requires from training choices that measured within noise (all reviewers)
  • We added a best response agent that can play outside the agent's shortlist, and narrowed our robustness claim to match (F2N8, KrN4)
  • We trained four more seeds for 5 rounds. They differ by ~5 Elo, against ~90 Elo gained over those 5 rounds and ~240 over the full run (F2N8)
  • We're adding a diagram of one decision and a notation table, and removing internal run names from the main text (R4tM)
AuthorsOpenReview ↗

11.18 Updated Revision

We've uploaded the revised PDF. Thanks again to all three reviewers. The main changes are:

  • The intro numbers our contributions, and §4 separates the adaptations VGC forces from training choices that measured within noise (Table 3) (all reviewers)
  • §4 adds a table defining the problem formulation and a diagram of one turn decision, with the full notation in App. B.1 (R4tM)
  • §5.2 adds the SoG and ReBeL component swaps and a decoupled UCT baseline. It matches our agent when both play their top action, but sampled from its visit distribution it wins only 16.8%, and best responses beat it 74–84% (KrN4, R4tM)
  • Table 6 adds best responses that can play any legal action and the exploitability estimate across training checkpoints, and App. E.4 adds the seed study (F2N8, KrN4)
  • Terminology is more consistent, the only internal agent name left in the main text is one defined reference agent, and the appendix is cleaned up (R4tM)

One clarification on the ~500× figure in our earlier replies. It compares exact enumeration against sampling in Showdown. Within our own engine, enumeration is ~19× cheaper than sampling, and the rest comes from the engine being faster.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.