acceptodds
Under review as a conference paper at ICLR 2027

Game-Theoretic Search in Physical Systems with an Application to the Sport of Curling

Abstract

Game-theoretic reinforcement-learning systems have achieved strong results in board and card games, but extending them to physical sports requires reasoning over continuous actions, stochastic execution, and nontrivial physics. We introduce a search-and-distillation framework for the sport of mixed doubles curling, using a physics simulator with iterative empirical-game optimization. A graph transformer jointly represents shot selection and position value: a Gaussian-mixture policy models distinct continuous shot modes, a distributional value head predicts end-score outcomes, and an auxiliary latent-dynamics objective encourages world-model-like physics understanding. Starting from human match data, we iteratively compute a meta-Nash mixture over existing policies, search for an approximate best response against that mixture, and distill accepted search decisions and realized returns into the next policy. Under human-like imperfect execution noise, the final policy beats open-weight language-model agents by +0.96 points per end. Against frontier agentic models, which are becoming powerful across diverse games, our policy wins by +0.19 points per end. In blind paired comparisons on 30 held-out positions, two U.S. national (Olympic) curling team staff members judged the policy's shots comparably to recorded human actions, selecting the policy as more optimal in 26 of 60 judgments (43%). These results demonstrate a practical framework for combining empirical-game reasoning, continuous-action search, and physical simulation in stochastic sports.

Primary area:
reinforcement learning
Keywords:
Reinforcement learning, game theory, empirical games, meta-Nash, search, policy distillation, continuous control, stochastic control, model-based planning, physics simulation, physical systems, graph neural networks, curling, sports analytics, multi-agent decision-making

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zDcWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose a game-theoretic search-and-distillation framework for mixed doubles curling, which they treat as a stochastic, continuous-action, physics-based game. Specifically, a graph transformer encodes stones, landmark points and game context as a graph with handcrafted geometric edge features. Human match records contain board positions but no release parameters, so the authors recover human actions with a Cross-Entropy Method inverse solver, and use the results to initialize the policy and value heads. Training then runs three PSRO-style rounds. Each round computes a meta-Nash mixture over the current policy population and searches for an approximate best response with KR-UCT against a JAX physics simulator under Student-t execution noise. Search proposals are accepted only when a paired gate confirms a gain, and the accepted decisions are distilled into the next policy.

Critical strengths
  • S1: Curling combines alternating zero-sum play, continuous actions, stochastic execution and nontrivial collision physics.
  • S2: Search runs on the explicit simulator rather than a learned model.
  • S3: Execution noise is part of the model, where noise is used when ranking candidates and when executing the throw.
  • S4: Blind, randomized paired judgments by U.S. Olympic curling staff are rare and valuable in game-AI papers, and the free-text feedback gives qualitative insight.
  • S5: The free-guard-zone rule, power play, simulator constants, action bounds, noise scales, search hyper-parameters and compute are all documented, so the setup could be rebuilt.
Critical weaknesses
  • W1: The game-theoretic evidence is thin. Only three response rounds were run. The first two restricted games were essentially pure, and per-round gains are small.
  • W2: No relevant algorithmic baselines, such as plain KR-UCT with no learned policy/value, human-prior policy in the same simulator, and self-play/NFSP agent.
  • W3: LLM agents could be a weak and confounded main baseline.
  • W4: Ablations do not support all design requirements.
  • W5: The inverse solver uses a different physics preset from the one used for play and evaluation, so the human priors are recovered in one physics and deployed in another.
  • W6: The method combines known components with relatively limited novelty. The contribution is mostly system integration for one domain, and nothing shows the framework generalizes to other physical games.
Questions for authors
  • Q1: Play and evaluation use different physics parameters from the inverse solver. How sensitive are the recovered human actions, and the downstream policy, to this mismatch?
  • Q2: Can the ablations be measured in playing strength with uncertainty, rather than differences in the third or fourth decimal place of held-out NLL and MSE? If neither auxiliary objective changes play, why keep them as core design requirements?
  • Q3: Can the policy be conditioned on end index, score and ends remaining so it can handle match-level strategy and power-play timing? Have you tried the framework on any other physical game to support the generality implied by the title?
Reviewer_LMc1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a search-and-distillation system for mixed-doubles curling, treating it as a stochastic, continuous-action, two-player game governed by physical dynamics. Its contribution is combining a graph-transformer representation, a multimodal continuous-shot policy, a probabilistic score-value model, physics-based tree search under execution noise, and three rounds of approximate best-response training against an empirical meta-Nash population initialized from human match data. In simulation, the final policy beats earlier response policies and the preceding meta-Nash mixture. It also beats open-weight LLMs, performs comparably to frontier LLMs, and receives no significant preference difference from human expert shots, though these latter evaluations are small. Deployment-time search provides a large performance gain, while the auxiliary learning objectives show only marginal benefits.

Critical strengths
  • The paper presents a clear, coherent end-to-end system for a difficult continuous-action, stochastic physical game. This could have the potential to extend to other complex strategic settings that are rooted in physical systems.
  • The deployment-search ablation is interesting: search improves performance by 1.24±0.111.24 \pm 0.11 points per end. The authors also report uncertainty and limitations very openly and clearly.
Critical weaknesses
  • Methodological novelty: The system combines established components, without sufficient justification that this integration constitutes a broadly useful methodological contribution outside of curling.

  • Simulator dependence: Training, search, and most evaluation use one fixed simulator that simplifies curl realism and omits sweeping, changing ice, and player-specific execution. This makes it unclear whether the reported results and strengths would translate to realistic curling.

  • Weak game-theoretic evaluation: Only three response rounds are used, the resulting meta-Nash solutions are nearly pure, and there's a significant amount of space dedicated to analyzing π3\pi_3 evaluated against policies from its training rounds. I believe this does not constitute clear evidence of robustness or low exploitability against unseen strategies.

  • Potentially unbalanced/inconclusive comparisons: The authors' system receives significant simulator training and many test-time simulations, but the LLM agents receive at most two previews. Even then, the frontier-model and full-match-weighted advantages are not clearly statistically significant (0.19±0.210.19 \pm 0.21).

  • Lack of evidence of human-play quality: The human experts selected π3\pi_3's action in only 26 of 60 comparisons. The result constitutes “no detected preference,” not equivalent to expert or human-level play.

  • Curling fidelity: Evaluating individual ends only omits some of the score-dependent strategy and power-play decisions that develop across a full multi-end match. This may limit the significance of the application and its conclusions.

Questions for authors
  • As clarification, could the authors describe the broader applicability of the specific novel elements of their method? And specific technical analysis/evidence of why their methodological contribution combining the graph-transformer representation, a multimodal continuous-shot policy, probabilistic score-value model, and physics-based tree search provides advantages over existing systems?

  • As a suggestion for future work, I would be curious how, along the lines of robustness/exploitability, pi_3$ performs against an optimized, separately trained best response or additional response rounds, rather than primarily against prior policies from its training?

  • For clarification and as a suggestion for future work: do the authors think advantages are consistent under simulator changes such as different curl, ice, sweeping, or execution parameters, and against simpler planning baselines given the same simulation budget?

Reviewer_V5ehOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents an alpha-go/mu-zero style learning for the game of curling. The learner is able to perform well and experiments with expert human curling players suggest many of the shots are on par with human experts. The program is compared against straight AI bots, which also perform well.

Critical strengths

This paper presents a complete architecture for planning and learning in a game with a continuous actions. This has been a challenge in past work, most of which has just studied a subset of the game. (Eg hammer shots)

Research papers often run things in simulation but do not validate against human experts. This paper runs an evaluation using what appear to be Olympic-caliber players (it isn’t clear who exactly on the staff was doing the evaluation), which produces positive feedback in most cases.

Simulating curling is difficult, because the physics of the blocks interacting with the ice is not completely well known. It appears that a suitably strong simulation was used to ensure play has physical realism, although the paper notes that may be a weakness in a few settings.

Critical weaknesses

If I understand correctly, the paper only evaluates positions from expert play. It isn’t clear how this is contributing to the paper. There is significant sampling of possible actions from these positions, and the MCTS search itself is expensive, so data collection for training takes about 2 days, while the final training was relatively fast. But this raises the question: What if you train from scratch (alpha-zero) instead of starting from expert positions?

The appendix mentions that rollouts are used, but doesn’t describe the rollout policy.

In some ways this paper mixes between alpha-go which started with human data and mu-zero which learned a latent representation. The paper does not contain an ablation study justifying the need for learning the grounded representation, although the text mentions that this helps ground the simulation.

The paper does not give details about the meta-Nash approach - only referring to a paper by Lanctot as reference. I would have liked to have seen more here about how Nash equilibria are being approached.

The margins in Table 1 against the other π_1 and π_2 are relatively small, and don’t appear to be within confidence bounds.

The paper reports that giving the same problem to LLMs can achieve similar results. It is worthwhile to have this knowledge as a comparison, but it begs the question of exactly what is going on. The appendix discusses this, but I found the discussion (5.2) somewhat opaque. Is there a better summary of what is causing the LLMs to do so well?

Questions for authors

In line 87 what approach is the previous meta-Nash referring to?

What if you train from scratch (alpha-zero style) instead of starting from expert positions?

Have you measured the impact of learning the latent state?

Can you provide more details about how meta-Nash is achieved?

Comments on the paper

1 comment
Area_Chair_yKASOpenReview ↗

Please Engage With Authors

Dear Reviewers,

We're halfway through the discussion stage. The authors have provided their responses to your reviews. If the responses have not addressed the concerns raised in your reviews, please raise and allow the authors to respond again. If yes, please still respond to the authors to reflect that.

Many thanks, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.