- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies automated circuit discovery for GNN force field models. They define a behaviour, a set of chemically meaningful pertubations on small local substructures (stretching a bond, or opening an angle) that change the model output by a certain margin. They then find parts of the computational graph of the model, the units, that explain a certain fraction of this model output change in an intervention on most of the molecules. They then try to find a concrete small circuit across different molecules that is both sufficient and necessary for this change. They find such small circuits across three different force field models and can relate selected examples to concrete pyhsical properties/laws.
Critical strengths
S1: They find small circuits in trained force field models and are able to interprete some of them in a case study. The relation to concrete physical laws/properties seems interesting. However, I am lacking expertise to really judge how meaningful these explanations are and how much insight they provide.
S2: The motivation to apply circuit discovery (which is mostly used for LLMs) to force field models make sense. And the authors convincingly argue that this application is not straightforward as molecules contain continuous position features, force field models use equivariant transformations, and it is not obvious how to define meaningful perturbations.
Critical weaknesses
W1: Lack of clarity: The paper is very hard to read and understand. While I acknoledge that part of this might be caused by my lack of expeertise in circuit discovery, I still belief that there is a significant lack of clarity:
- The paper would benefit from a background section on circuit discovery that then forms the basis of the subsequent discussion and makes clear what parts of the pipeline are just taken over from already existing circuit discovery pipelines and what parts are new/adapted to the force field setting.
- The definitions are not well-organized and not clear enough.
- Many definitions contain terms that weren't defined previously or aren't sufficeintly explained: "An interface is a declared finite set of nodes of the computation graph of fθ at one boundary" (ll. 167). What does boundary mean here? "An intervention on a subset A ⊆ VI at strength α ∈ [0, 1] evaluates the model on G with replaced by for c ∈ A and recomputes every downstream node; when the written value comes from a molecule in a different orientation it is transported through the unit’s group action." What does "different orientation" mean here? What does "written value" refer to here?
- The same symbol is defined in different ways. In Definition 2 v(A) denotes the model output after intervening on A (it explicitly says ), whereas in Definition 3 seems to denote the change in model output (it explicitly says now ).
This lack of clarity makes it very difficult to understand the precise method of the paper (which also seems very complicated and involves many different stages).
Questions for authors
- Can you more clearly describe the different stages of the algorithm?
- For how many of the discovered circuits can you find concrete interpretable connections to phyical properties/laws? What concrete insight or practical consequences follow from these connections?
6 comments
Response to reviewer 2N8u, part 2 of 2
Q2. How many circuits connect to physical properties, and what follows?
"For how many of the discovered circuits can you find concrete interpretable connections ... What concrete insight or practical consequences follow from these connections?"
Beyond the case studies, we evaluated all 58 circuits using strict chemical criteria that we defined before we ran the sweep, informed by the seven case studies (Section 4.3, Appendix G.3).
- 55 of 58 act as harmonic springs: When we displace the coordinate in either direction within a molecule, the energy carried by the circuit curves upward, mirroring the harmonic term of a classical force field (Hooke's law).
- 36 of 58 exhibit a restoring trend across molecules: Over molecules where the bond or angle starts at different lengths, the circuit's force consistently points back toward a single, shared equilibrium.
- 15 of 58 pass all four criteria: restoring across molecules, zero-force value within 5° or 0.05 Å of its own component's, textbook order between hybridisation classes, and zero-force value within 5° or 0.05 Å of the textbook value. For calibration, the full model passes the three criteria that apply to it (the first, third and fourth) for only 21 of the 58 circuits, and 5 circuits pass where the full model fails.
- They recover chemistry from scratch: For example, MACE's H–C–C circuit places sp$^3$ and sp$^2$ carbon at 109° and 120° (textbook 109.5° and 120°), and Orb's two C=C circuits place the bond at 1.30 and 1.32 Å and its two C=O circuits at 1.19 and 1.22 Å (textbook 1.33 to 1.34 and 1.21 to 1.22 Å). No model was given hybridisation or bond order; they receive elements and coordinates only.
We draw four consequences from this:
- First, the models rebuild classical chemistry internally. Networks trained only on energies and forces organically organize their computation by standard chemical categories. A circuit is the exact same small set of units across every molecule in a family, and its component carries roughly a fifth of the total response (medians of 22%, 18%, and 24% in MACE, Orb, and NequIP).
- Second, the explanations concern correct behaviour. New DFT calculations at 693 perturbed geometries agree with the models' energy changes at (Appendix A.1). So, the circuits explain behaviours that the models get right.
- Third, a circuit shows where in the network a property is set. In MACE's C=C bond, one component carries 73% of the energy response and 37% of the stiffness, while the rest of the network shifts the bond length to its final value. If a developer spots a bond-length error, the circuit tells them exactly which part of the network to examine (something accuracy metrics cannot do!)
- Fourth, we can compare architectures by what they compute. Most MACE circuits sit in the receiver atom's own features and readout, while Orb and NequIP circuits reach back along the messages from the moving atom. The three reach similar accuracy by different internal routes.
We hope the rewritten definitions, the stage table and the worked example make the method clear, and that the sweep over all circuits answers your second question. We are glad to clarify anything further during the discussion period.
Thank you for your detailed answers and the new results.
The interpretations of the circuits look interesting and it is very good that you can find clear interpretations for most of them. (However, due to my lack of expertise it is hard for me to accurately judge their significance.)
Additionally, I have gone through the revisions in the definition in the updated paper. The specific examples that I orginally listed have largely been fixed. However I still have strong concerns about the clarity of the paper. To illustrate this, I have gone (only) over Definitions 1 & 2 and listed the points that remains unclear (to me). While the indended meaning can sometimes be inferred from the context, this makes reading just very hard. And in other cases one has to guess what the authors mean.
-
A perturbation specification is said to be parametrized by "target coordinate, a displaced atom, a step size , and fixed geometric constraints" (l.156). This is quite vague. What does target coordinate mean here? The new coordinate of the displaced atom? But what is then the influence of the step size? What are the fixed geometric constraints? Are we only considering bond stretching and angle bends or are ther other "local geometric transformations"?
-
"Finally, a pair constitutes a behavior of on if valid, range-constrained occurrences of appear in at least a minimum number of molecules ". Valid-and range-constrained is never defined for occurences of , just for applications of . Also what happens if there are multiple occurences of in . Then is not uniquely defined.
-
A unit is defined as a node in the computational graph (l.175) but is later to be said to be e.g., a set of edge messages or an aggregated group (l.177). How does the computational graph look like? Wouldn't an aggregated group correspond to multiple nodes? Similary, we then define as the activation of unit . How is the activation of an aggregated group then defined here? Is it a vector of the individual activations?
-
One condition of to be called an admissible interface is that , isn't this always true for any interface? Similarly, isn't also always true as an interface is defined to cut through every directed path? So does there exist any non-admissible interface?
-
The definition of a recorruing term is not clear to me:
-
What does "molecuels in family " (l.192) mean? is defined to be a typed (sub)graph). Is molecules containing meant here?
-
Also it is not clear to me how is defined here? Is it allowed to depend on the input graph? E.g. can it be something like "all messages to the central crabon atom in the occurence of in . So does the set consist of roles instead of concrete nodes in a computational graph (as l.175 would suggest).
-
While each point on its own might not be that significant or might possibly seem clear from the context, the amount of those imprecisions makes this paper very hard to follow. For this reason, I maintain my original score.
Response to reviewer 2N8u, part 1 of 2
We thank the reviewer for their critical review and for listing exactly which terms were unclear. We have fixed each one in the revised PDF (changes are highlighted in blue).
W1a. Background on circuit discovery.
The introduction now clarifies that circuit discovery was developed on language models (as the reviewer notes), and Section 3 opens with a table detailing what we borrowed from existing literature and what are our contributions. Briefly, standard circuit discovery (e.g., ACDC; Conmy et al., 2023) isolates a behavior on discrete inputs, copies activations between clean and corrupted runs, and prunes the computation graph to the smallest part that reproduces the behavior.
W1b. "Boundary".
A boundary is a cut through the network at one depth, a set of activations that every path from the input to the output crosses. If we set all of them to their perturbed values, we get the perturbed output. Definition 2 now clarifies this.
W1c. "Written value" and "different orientation".
The written value is the activation we put into a unit during an intervention, ac(G)+αΔac: its clean unperturbed value plus a fraction α of the change that the perturbation caused. Equivariant activations are vectors and tensors expressed in a coordinate frame. When the change comes from another site or another molecule, as in our transfer tests, we first have to rotate it into the receiving frame, and that rotation is what "transported through the group action" means. We have rewritten the sentence in order to clarify this to readers.
W1d. v(A).
We thank the reviewer for the observation regarding the mismatch and clarify here: we inadvertently used the symbol in two different ways. We have corrected the text so v(A) consistently denotes the change in output under the intervention. This means v(∅)=0 across every definition, and admissibility now reads v(∅)=0 and v(VI)=ΔyB. Because the appendix and our codebase already used this exact meaning, no results change.
Q1. Describing the stages.
We show here a summary of the table which is now detailed in Section 3, to describe the different stages of the algorithm:
Appendix B.1 follows one perturbation, a C=C stretch in MACE, through all five stages, as an example.