- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper asks if the chess-playing policy networks encode information about how hard chess positions are for humans, without having been trained to predict difficulty. Paper also addresses if the networks use difficulty information when choosing moves. The authors first fit linear ridge probes on frozen activations of seven networks to predict puzzle ratings. They report the increment over Stockfish search features, the network policy entropy and the player's rating. There are 4 main findings: 1) difficulty can be decoded beyond engine features 2) the signal aggress across all networks after residualization 3) the thinking time is also decodable but is a separate signal from difficulty 4) difficulty isn't used for move selection.
Critical strengths
I found the controls to be very strong, with a three tier control ladder and a solution aware control, with about 80% of the signal surviving. In addition, the ablation design is very rigorous and detailed, including 50 variance-matched control directions, a second architecture, policy-jacobian positive control, etc.
Critical weaknesses
The biggest weakness is probably the disproportionate generality of the paper claim given that all tests are done in a single domain only. I understand that authors selected chess based on meeting 4 criteria, but there are probably other options that do as well (maybe Go?), or at least partially do. Furthermore, there are some concerns regarding interpretation and robustness of results - agreement across networks residual readouts probably should not be immediately interpreted as a difficulty signal (even with all board structure removal steps), and small effect sizes and lack of replication on held out months make the results less convincing.
Questions for authors
-
Networks' activations range from 256 to 2048 dimensions, but every probe uses the same penalty. So wider layers give the probe more to work with. Can this explain why some models (Leela-CF) show larger increments? Would the ranking hold if all models had the same number of dimensions, or if the penalty was model-adjusted?
-
The depth comparison result (which the main claim relies on) uses a subsample of all puzzles, and reports no confidence intervals - could you report the intervals for decline in engine-only R^2 and for the probe increments, and full data set results?
-
In ablation studies, after you remove the difficulty direction, a new probe still recovers most of the difficulty signal - so the information is still int he network just along other directions. Couldn't this be interpreted as testing whether this one direction is used, not whether difficulty information is used? Could you try iteratively refitting and removing probe directions until difficulty is no longer decodable, and test whether the network behavior changes then?
2 comments
(2 of 2): one domain, effect sizes, later months, depth, iterative removal
In short. We agree that every test is in chess and that the effects are modest. Residual agreement is no longer offered as evidence. Three later months carry over, but had been inspected before. A fresh month met rules fixed in advance. The depth result now uses all puzzles, with intervals, and no claim rests on it. Iterative removal settles neither use nor non-use.
"all tests are done in a single domain only"
Agreed: every test is in chess, and the paper's title and abstract now say so. We have not tested Go. It would allow the same comparison, but it lacks an openly available problem rating calibrated on many solvers (Section 9).
"agreement across networks residual readouts"
Agreed: it is no longer offered as evidence of a difficulty signal. The split that replaces it (Section 5) measures overlap in rating prediction, not a partition of features inside the networks.
"small effect sizes"
Agreed: they are modest. A readout lowers the error of an engine-based rating estimate by 13.5 to 42.0 points, from about 340 (Table 1). Beyond the full control set, the readouts of Maia-2, Maia-3-79M and Leela-CF explain 2.8%, 8.7% and 13.2% of the variance the controls leave (Table 5). This is an effect size, not a rating method (Section 7).
"lack of replication on held out months"
We tested September 2021, a month this study had not used. Before reading its ratings, we fixed rules from the January 2019 results, and the month met all of them. Two rules go beyond signs and shares. Leela-CF's gain had to reach 0.80 of Maia-3-79M's; it is 1.41 times as large, against 1.13 on January 2019. Maia-3-79M's history shift had to exceed Leela-CF's by 0.04 nats on the matched score; it does by 0.085. The paper's leading score subtracts a further baseline. Computed afterwards and outside the rules, it gives 0.061 nats [0.050, 0.071]. The other eight rules are easier; a random-weight network meets two. Every gain is smaller than on January 2019, and Maia-2's training window covers this month. We changed the month from March 2021 before reading any September rating: one March rating had been printed before the plan was frozen. The rules stayed the same (Appendix A.16).
Three later months, which had been inspected before, show robustness but not a fresh test. Fitted once and applied unchanged, the self-play readout raises the share of puzzle pairs ordered correctly from 70 to 76% to 78 to 82% (Section 7). Its addition is 8.1 [7.7, 8.4], 5.4 [5.3, 5.5] and 5.5 [5.2, 5.8] percentage points in the three months. These intervals resample the target month's puzzles and hold the fit fixed (Table 4).
"could you report the intervals for decline in engine-only R^2 and for the probe increments, and full data set results?"
Yes, and the main claim no longer relies on it. On all 26,352 puzzles, the seven-feature engine summary predicts the rating worse at depth 20 than at depth 12, by 0.0319 [0.0278, 0.0359] in R². Readout gains over the depth-12 summary are 0.0714 [0.0666, 0.0764] (Maia-2), 0.1591 [0.1526, 0.1662] (Maia-3-79M) and 0.2239 [0.2163, 0.2322] (Leela-CF). Over the depth-20 summary they are larger only because the engine baseline falls; engine plus readout predicts no better. So we withdraw the growth of the gain with depth. The decline is consistent with how Lichess selects puzzles, and the main text no longer claims it (Appendix A.14).
"Could you try iteratively refitting and removing probe directions until difficulty is no longer decodable, and test whether the network behavior changes then?"
Yes: removing one direction tests only that direction. We ran iterative removal in Maia-2. Behavior changes, but not selectively, so it settles neither use nor non-use. Non-use is no longer claimed. Removing raw-rating directions makes the solution move more likely than matched random edits do. For the part beyond the engine, the stopping rule fixed in advance was never met. A rule chosen afterwards is met only at 50 to 64 directions. At 64, the solution becomes less likely than under every matched edit. Neither removal is selective: the network's own entropy and solution probability become less readable too (Section 8).
In the paper: Sections 5, 7, 8 and 9; Appendix A.6; Appendix A.14; Appendix A.16.
Thank you. Your feedback was detailed, thoughtful and thorough. We are sorry that this reply comes late in the discussion period: we wanted to do justice to every point by revising the paper thoroughly rather than send a rushed reply. We have addressed every point of your review in the revised paper, which is substantially rewritten for clarity, rigor and completeness. We would be grateful if you would read the revised paper and reconsider your rating in its light.
(1 of 2): width, penalty and robustness; no ranking is claimed
In short. No fixed ranking holds. Read the same way, which large network leads depends on how many components are read, so we claim no order. Changing the penalty moves no gain in the puzzle rating by more than 0.007. Every conclusion holds over eleven fold partitions. Table 5 states what each claim rests on and when its analysis was fixed.
"Would the ranking hold if all models had the same number of dimensions, or if the penalty was model-adjusted?"
No ranking holds, and the paper no longer claims one. We now read every network the same way: one common control block and the same number of principal components. The common block is the 23 engine and puzzle features plus the entropies of Maia-2, Maia-3-79M and Leela-CF.
Read so, Maia-3-79M and a Leela Chess Zero network of its size, whose training is not published, carry comparable amounts. Maia-3-79M leads at 32 and 64 components, the other network at 256 and above. Leela-CF minus Maia-3-79M is −0.0054 [−0.0102, −0.0010] at 32 components and positive, with its interval above zero, from 128 up. We claim no order.
The comparison network need not be the larger one: the network of Maia-3-79M's size removes 82% of Maia-3-79M's gain beyond the first control set.
On the penalty: at full width, with the penalty fixed, scaled with the layer's width, or tuned inside the training folds, no gain in the puzzle rating moves by more than 0.007 (Appendix A.2).
"Can this explain why some models (Leela-CF) show larger increments?" Not the readout's width: Leela-CF and Maia-3-79M are both read at 1,024 dimensions. How many components are read does matter: with 8 components Leela-CF keeps 25% of its full-width gain, against 64% for Maia-3-79M (Appendix A.2). As for network size, these data cannot separate size from accuracy. Across seven Leela Chess Zero networks, the gain rises with size and with first-move accuracy alike (Appendix D.1). At one fixed size, a self-play network's gain beyond the first control set still rises during training, from 0.015 to 0.035. Beyond the larger control sets, that rise is 0.005 or less.
Soundness. Table 5 (Section 9) gives each claim's effect and when its analysis was fixed. Every row is marked as fixed after related results were seen; only the first-set gain dates from the submission; the further month's rules were fixed in advance, and the table says so. The intervals resample source games and hold the fitted predictions fixed, so they leave out refitting. In three reruns of the engine, the gains moved by at most 0.003. Over ten further fold partitions, each gain stays within 0.005 of its value, and every conclusion holds in all eleven. The self-play comparison holds at every control level on all eleven (Section 5). Maia-2's shares depend on the partition, so the paper gives them as ranges. Choosing the layer inside each training fold picks the same layer in every fold for Maia-2, Maia-3-79M and Leela-CF. A separate re-implementation occasionally chose a neighboring layer. For Maia-3-79M the runner-up layer is nearly tied: it gives 0.1108 against 0.1113 (Appendix D.8).
The controls you found strong. You noted "about 80% of the signal surviving" a solution-aware control. The controls are wider now, and less survives them. In the three networks tested, about 40 to 60% of each gain remains beyond the second control set. Beyond the full set, about a quarter to a half remains, now in all eight networks (24 to 45% on our split, 21 to 45% over three fold partitions; Section 4).
In the paper: Sections 3, 5 and 9; Table 5; Appendices A.2 and D.
Continued in part 2 of 2.