- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies pursuit policies for Scotland Yard as a two-player zero-sum one-sided partially observable stochastic game, where the five pursuers share observations and only the evader has private location information. The authors propose SWIFT, a self-play framework that trains role-specific graph networks on a single public-history search graph shared by both roles. The construction combines information-consistent search, a learned location prior that reweights sampled worlds during backup, an autoregressive joint-action policy, and self-play distillation of search targets. On an adapted 199-station, 22-round version of the game, the authors report that the trained pursuers win at least 98% of games against six self-trained evaders, even without test-time search.
Critical strengths
-
The paper identifies a clean structural property of one-sided public-observation POSGs: because the pursuers have no private information, every public history corresponds to exactly one pursuer information set, so the per-information-set trees required by ISMCTS and self-determinization collapse into a single public-history graph.
-
The empirical results are strong within the paper's protocol: SWIFT wins at least 98% against all six self-trained evaders without test-time search, and the belief-weighting and encoder ablations are consistent with the claimed mechanism.
-
The efficiency contrast is potentially significant: SWIFT is trained on a single RTX 5090 for five hours.
Critical weaknesses
-
The presentation method of the paper needs improvement. The paper should not aim at Scotland Yard as the solution, but rather at the pursuit game where "only one side is partially observable". Scotland Yard is too narrow in game domain.
-
No external anchor for the central claim. All six evaders are trained by the authors, and no published agent participates; PimBot, the standard strong agent in this game, is never included in the comparison, even though the paper builds on its single-tree determinization and location categorization. The Introduction describes SoG only through an 11-node map, although SoG was trained on the full 199-station game and won 55% head-to-head against PimBot with 10M simulations. This omits the most natural external comparison and understates a competitor's contribution.
-
The headline claim is broader than the evidence. "At least 98% against every evader we test" is not supported for the human evader. The 98% figure is also specific to the five-pursuer game; with three pursuers the same method wins about 23% against its final evader.
-
The baseline comparisons confound algorithm and training opponent. The PPO-type baselines are trained only against Greedy, which is nearly solved from the first checkpoint, while P2SRO is a 12-hour self-play baseline and SWIFT gets five hours. There is no no-learning ISMCTS and no search-vs-search comparison at equal simulation budget.
-
No exploitability or best-response measure is reported. Every learned opponent is produced by the authors' own training runs, and every strength comparison is a win rate against fixed opponents or a rating from cross-play within one pool, so none of them bounds how much a stronger evader could gain against the trained pursuers; even the P2SRO evader, the closest to a best-response opponent, is trained against its own population.
Questions for authors
-
Can you evaluate SWIFT head-to-head against PimBot with roles swapped, under the classic rules or an adaptation to your 22-round rules, and report the result of SoG vs PimBot?
-
Can you isolate the AlphaZe** gap with a 2×2 ablation (per-world vs. public-history search × with/without KataGo refinements) within a single rating pool, and report the L1 distance or a reliability diagram between qP and the exact belief b?
-
Can you restrict "every evader we test" to the six agent evaders, state the five-pursuer condition in the abstract and conclusion, and describe the human evaluation as a single-participant exploratory study?
-
Can you add a MAPPO self-play baseline trained for five hours, compare P2SRO at equal wall-clock time, and report search-vs-search at equal simulation budget?
-
Can you report exploitability on the 11-node map used by SoG, or the gain of a best-response evader trained against the final pursuers?
5 comments
Response to Reviewer EEEB (2/3)
Weakness 3 and Question 3: the scope of the claims
Weakness 3. The headline claim is broader than the evidence. "At least 98% against every evader we test" is not supported for the human evader. The 98% figure is also specific to the five-pursuer game; with three pursuers the same method wins about 23% against its final evader.
Question 3. Can you restrict "every evader we test" to the six agent evaders, state the five-pursuer condition in the abstract and conclusion, and describe the human evaluation as a single-participant exploratory study?
We made all three changes. (i) "Every evader we test" now refers to the learned, rule-based and search-based evader agents we evaluate, that is, the six of the submission and PimBot (Abstract, §1, §6). (ii) The abstract, §1 and the conclusion state the result for the five-pursuer game. In the three-pursuer game (App. D.5), SWIFT wins about 29% against its final evader in a new run; the 23% of the submission came from an earlier run. P2SRO, now also trained in this game for about 14 hours, wins about 28% against the same evader and less often than SWIFT against every heuristic evader. (iii) The human evaluation is described as a single-participant exploratory study (App. D.1), and the main text draws no conclusion from it.
Question 2: the 2×2 ablation and the calibration of q_P
Question 2. Can you isolate the AlphaZe** gap with a 2×2 ablation (per-world vs. public-history search × with/without KataGo refinements) within a single rating pool, and report the L1 distance or a reliability diagram between qP and the exact belief b?
2×2 ablation. We ran the 2×2 within one rating pool. Public-history search beats per-world search whether the KataGo tricks are switched on or off. The tricks, in turn, help only when combined with public-history search. Pursuer ratings after 5 h, in the pool formed by the four runs (this response only):
| Pursuer search | KataGo tricks on | KataGo tricks off |
|---|---|---|
| Public-history (shared pursuer nodes) | 2022 | 1639 |
| Per-world (separate pursuer trees) | 1438 | 1461 |
Public-history search is worth 584 points when the tricks are on and 178 when they are off. The tricks add 383 points with shared nodes and change little (−23) with per-world trees. In the public-history runs, the evader searches from a single root; in the per-world runs, it keeps one root per possible location. Since a single root weakens the pursuer (General Response, item 1a), this choice does not favor public-history search. For AlphaZe** itself, we rated the final networks of SWIFT, its ablations and both AlphaZe** variants in another pool. AlphaZe** is 784 points below SWIFT (1255 vs. 2039) and loses another 255 points when the tricks are added (1000). Without the tricks on either side, SWIFT still leads by about 400 points (1653 vs. 1255), so the tricks do not explain the gap. One possible explanation is that the KataGo tricks strengthen AlphaZe**'s search within individual sampled worlds, making it favor actions that rely more heavily on world-specific perfect information. This may exacerbate the effects of strategy fusion, resulting in a worse averaged root policy.
Calibration of q_P. We report both (new part of App. D.4, Figure 9). The location prior is much closer to the exact belief than uniform weighting. Over 1,024 games between the final networks, we compute the exact belief b of the evader network by Bayesian filtering of the public history (5,344 states, 147,765 state–location pairs). The mean L1 distance between q_P and b grows from about 0.23 one hidden move after a reveal to about 0.61 after four, whereas uniform weighting stays at 0.71–0.86. The mixture used in the search lies in between. q_P is well calibrated up to about 0.35 and overconfident above that: when it predicts about 0.95, the evader is there in about 83% of cases. The mixture keeps every possible location at no less than half its uniform weight, which limits the effect of this overconfidence on the search.
Response to Reviewer EEEB (3/3)
Weakness 4 and Question 4: training opponents, equal training time and search baselines
Weakness 4. The baseline comparisons confound algorithm and training opponent. The PPO-type baselines are trained only against Greedy, which is nearly solved from the first checkpoint, while P2SRO is a 12-hour self-play baseline and SWIFT gets five hours. There is no no-learning ISMCTS and no search-vs-search comparison at equal simulation budget.
Question 4. Can you add a MAPPO self-play baseline trained for five hours, compare P2SRO at equal wall-clock time, and report search-vs-search at equal simulation budget?
A learning opponent alone does not explain SWIFT's advantage. We agree that the PPO-type baselines confound the learner with the Greedy training opponent, and §5 now states which baselines train against learning evaders and which against Greedy. A MAPPO self-play baseline would test whether a learning opponent alone produces strong pursuers, and three comparisons in the paper already answer this. P2SRO trains the same PPO-GNN learner against a population of learning evaders: against the SWIFT evader, it wins about 61% after five hours and 85.7% after twelve, compared with 66.2% for PPO-GNN and 98.2% for SWIFT (Table 1; App. D.2). AlphaZe** uses SWIFT's networks, training pipeline and training time against its own learning evader and wins only 30.5% against the SWIFT evader. Every SWIFT ablation is also trained by self-play, yet separate pursuer trees alone cost 590 rating points (General Response, item 1). The information-consistent search, rather than the learning opponent, makes the difference.
At equal wall-clock time, P2SRO stays far behind SWIFT. At five hours, the P2SRO pursuer wins about 61% of games against the final SWIFT evader, compared with 98.2% for SWIFT (App. D.2, Figure 6).
The search budget does not explain SWIFT's strength. As a search baseline without learning, we added PimBot. It is the published single-tree determinization MCTS for Scotland Yard, and its shared pursuer tree follows the same principle as ISMCTS. With 10⁴ simulations per decision, it wins 85.7% on average against the learned evaders, compared with 98.5% for SWIFT without search. We did not run search against search at equal budget, because the paper's claim concerns the networks that training produces, which play without search. Three further results show that the budget does not explain the pursuers' strength. First, the SWIFT pursuer without search wins at least 98% of games against PimBot with up to 10⁶ simulations per decision. Second, AlphaZe**'s pursuers spend up to 500 simulations per move (100 in each of up to five worlds), compared with about 300 for SWIFT, and end far weaker. Third, the 2×2 above compares the two search structures at the same budget.
Weakness 5 and Question 5: exploitability
Weakness 5. No exploitability or best-response measure is reported. Every learned opponent is produced by the authors' own training runs, and every strength comparison is a win rate against fixed opponents or a rating from cross-play within one pool, so none of them bounds how much a stronger evader could gain against the trained pursuers; even the P2SRO evader, the closest to a best-response opponent, is trained against its own population.
Question 5. Can you report exploitability on the 11-node map used by SoG, or the gain of a best-response evader trained against the final pursuers?
We report the gain of a best-response evader, and the SWIFT pursuer is much harder to exploit than every learned baseline. A PPO evader trained from scratch against the frozen final SWIFT pursuer, on the full 199-station map, wins 7% of games after half an hour, 12% after one hour and 17% after two. Against the learned baseline pursuers, the same learner wins 53.3–99.6% after half an hour and 68.4–99.6% after two hours (General Response, item 2; revised §5.1, Figure 2; App. D.8). The fixed evaders of Table 1 win at most 1.8% against SWIFT, so adaptation helps the evader, but it still wins far less against SWIFT than against any learned baseline pursuer. We chose this option because it evaluates the very pursuer reported in the paper, in the game it was trained for. Exploitability on the small map of Student of Games would describe a different policy trained for a different game.
We hope these results address your concerns, and we are glad to discuss any remaining point.
Response to Reviewer EEEB (1/3)
Thank you for the review. We added the experiments that bear most directly on the main claim (PimBot in both roles, best-response evaders, the 2×2 ablation and the calibration of the location prior) and narrowed the claims as you suggested. All new results support the paper's central claim.
Weakness 1: framing beyond Scotland Yard
We agree and now frame the paper around this class of games. The Introduction now starts from pursuit-evasion games with one-sided partial observability, in which the pursuers share public information and the evader observes the full state. Adapted Scotland Yard is presented as the testbed (§1, paragraph 1), and §3 introduces it as our instance of this class. The search construction itself relies only on this information structure: the shared public-history pursuer nodes, the evader roots for every possible location and Proposition 1 need only that the pursuers hold no private information and that the evader knows the true state.
Weakness 2 and Question 1: PimBot as an external anchor, and Student of Games
We evaluated SWIFT against PimBot with roles swapped, under an adaptation to our 22-round rules. Averaged over the two roles, SWIFT without search wins more than half of the games at every PimBot budget from 10 to 10⁶ simulations per decision. This average falls from 100% at the smallest budget to 54% at the largest (table in the General Response, item 3; new App. D.9, Figure 15). As pursuer, SWIFT wins at least 98% at every budget. As evader, it wins 28% against PimBot with 10⁴ simulations and 10% with 10⁶, in line with the pursuers' structural advantage in this game. In Table 1 (10⁴ simulations per decision, 512 games per pairing), PimBot is the strongest baseline pursuer, with an average win rate of 85.7% against the learned evaders (SWIFT: 98.5%), and the SWIFT evader is its hardest opponent.
We reimplemented PimBot following Nijssen & Winands (2012) with all components of the original (General Response, item 3). Since it plays our rules, its handling of double moves is not needed, and we re-estimated the location-category weights from 1000 self-play games under our rules. As you requested, §1 and §2 now report the result of Student of Games: it beats PimBot with 10⁷ simulations per search in 55% of 2000 games, half of them with roles swapped (Schmid et al., 2023).