The main weaknesses of the paper are listed below.
W1.) Poor quality of writing.
First, the language lacks scientific rigor. For example, some of the verbs
the authors use are "earn", "buys", "travels" and similar,
while they also use imprecise expressions like "few but real handles".
More importantly, Section 4 is presented through a series of (loosely connected)
paragraphs, each introducing a test in isolation but none explaining what gap each
one closes and/or why this is the right set of tests to run in the first place.
Some of the key objects are also used before being defined: for instance,
the notion of candidate is formalised only in D2, but D1 already
relies on it (it is even in the title of the paragraph). Similarly,
the central notion of handle is never explicitly defined in the main body
of the paper. The main text suggests a handle is a candidate that
passes D2, but the glossary gives a different definition, in terms
of one that passes both D2 and D3.
W2.) The restriction of concepts to facts is barely motivated.
That only concepts expressible as binary facts are graded finds as
only (briefly mentioned) justification a decidability argument.
However, the paper should discuss what this choice excludes and how
this impacts the presented results. Also, (cf. W1) the definition of
a fact appears in Section 3 only while it is needed earlier.
W3.) Many thresholds are set arbitrarily and no sensitivity analysis is provided.
Beating the value-blind fake by 2 game-clustered standard errors,
bands defined through 10 random directions,
a candidate pass under "40 random pushes of equal size by three of
their standard deviations", the REAL candidate firing on at least 6000
held-out scoring positions (and similar others) are arbitrarily set.
All the hand-picked thresholds should be motivated and their impact on the results should be discussed
by including a sensitivity analysis under different thresholds.
W4.) The rule's sensitivity is never measured.
The authors state that "no direction known in advance to be a handle
was sent through D2 and D3". It is appropriate to ask
what would happen if a direction is engineered to drive the
policy on a fact.
W5.) The findings are weaker than what the presentation suggests.
Appendix L shows that REAL handles are poor detectors of their assigned
facts with median precision of 0.23, 0.05 and 0.03. The handles
associated with forced-wait are the largest group and they
fire mostly on stall positions (median 0.88–0.94), where the mover
has no advancing move. This property refers to the board alone and
needs no solved table, which was the main motivation for the paper.
Also, the named handles fall
on facts being properties of the board alone. Finally, the claim that
"carries causal handles for the concepts that players of the game
name" is not currently supported by the results and should be
narrowed down according to what the paper is actually able to quantify.
W6.) The scope is quite limited, especially for an experimental only paper.
The paper is an experimental one, and it makes claims about methods and
validators (e.g. "the shuffled-label control certifies everything").
These claims are only supported by a single domain and a single architecture,
which limits the generalizability of the results. The authors should
test at least one more domain (for example, chess endgames
with tablebases, other solved games) and at least one more architecture
(for example, a transformer). Without any of these, it is unclear
whether the measured rates and lessons are properties of the validators
or of the particular network and pipeline object of the study.
W7.) The reported effect sizes are small.
Appendix J shows that the D3 effect of REAL move-side features has a
median of 0.016, 0.016, 0.014, but the game-clustered standard error
is 0.013, 0.014, 0.013. The effect is thus comparable to
its standard error.
Given the severity of several of the above weaknesses,
I recommend a weak reject decision for this paper.
Response to Reviewer Y6zK
Thank you for the review, and for the two references, both of which we have added.
W1, correlation versus causation. You raise the possibility that the activation patterns we find are correlated with a strategy but not causally related to it. That is the right worry about a concept-discovery result, and it is the one the protocol is built around, so we think the paper failed to make it visible rather than failed to address it. Of the three tests a feature must pass, only the first, D1, is correlational, and it sets no bar and declares no handle. D2 is an intervention: we overwrite one coordinate of the feature with its value at a matched position where the fact is false, re-run the network, and score the change in the network's play against 40 random pushes of the same size and a value-blind fake. D3 is a second intervention on the same pairs: the swap from a fact-false twin must cost more than the swap from a fact-true twin, beyond a band of random directions, so that the change depends on the fact's content and not on the swap itself. A feature is called a causal handle only if it passes both. We have now said this in Section 4's opening paragraph, before any test is described (p. 4), and a planted handle sent through the same rule gives the two interventions the positive control they lacked (new Appendix R, pp. 45–46). The claim this supports is a handle on play on the fact's positions; we do not claim that the network computes the strategy, and the limitations say why (p. 10).
W2, Jenner et al. (2024) and Geiger et al. (2021). Thank you for both. We added Jenner et al. to the games paragraph of Section 2 (p. 3): activation patching on a chess network shows that it represents a puzzle's future moves and that its choice of the solution move depends on them, with the puzzle's annotated line as the yardstick. We added Geiger et al. at D2's definition (p. 5): the coordinate swap is an interchange intervention restricted to one coordinate, and Appendix I now names it alongside the activation-patching term it had used.
The vocabulary and the abstract. You ask that at least the abstract be readable and self-contained. We rewrote it in plain words with every claim and number kept (p. 1), so a reader meets no coined term before its meaning. In the main text we replaced coined terms with plain phrases wherever one existed; your example "race class" now reads "set aside if its fact is a value class (who is winning) or the negation of a fact" (Section 4, p. 6), and Section 4's new opening paragraph defines candidate, causal handle and noise count before they are used. The glossary (Appendix A) remains for the terms the appendices need.
Thank you again; both points improved the paper, and we are glad to discuss either further.