- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors propose CrystalSAE, a framework for interpreting the internal representation of DiffCSP, a diffusion model that generates crystal structures based on composition, using a sparse autoencoder (SAE: an autoencoder that re-expresses internal activation vectors as the sum of a very small number of learned dictionary directions). CrystalSAE trains a layer-wise SAE on the differences (residual updates) that each layer of DiffCSP’s denoising network, CSPNet (a 6-layer graph neural network), adds to the internal representation.
For 17 concepts, including lattice constants (the lengths of three sides, a, b, and c, and three angles, α, β, and γ), volume per atom, density, the seven crystal systems, coordination number (CN: the number of nearest neighbors; CN=4 and CN=6). Using only the training split of the standard crystal generation benchmark MP-20, the authors select one SAE feature most strongly correlated with each of these 17 concepts, fix its orientation, and demonstrate on the validation split that all 17 pairs correspond statistically significantly to their respective concepts. The authors also present exploratory visualizations of the distribution of these features by layer and diffusion time, as well as for 68 space groups.
The authors investigate how crystals generated from the composition of the test split change when the directions of the selected features are added to the internal representation as an intervention. The authors report that significant changes occur in seven quantities, a, b, c, α, β, γ, and density, depending on the amount added; that the remaining 10 concepts (7 crystal systems, CN=4/6, and volume per atom) do not meet the intervention’s pass criteria; and that the dose-response curve is smoother than that based on the orientation of individual neurons.
Critical strengths
-
This paper employs a validation procedure that strictly separates feature selection from evaluation. The authors predefined 17 concepts, fixed the features and their orientations solely in the training split, then performed substitution tests and multiple comparison corrections (Benjamini–Hochberg method) in the validation split, reserving the test split exclusively for the intervention. The 10 concepts that did not meet the intervention criteria are also disclosed in Appendix D, and the authors explicitly state that they do not treat the success of the intervention as causal sufficiency. This procedure mitigates the selection bias, commonly seen in interpretive studies using SAE, where features that appear promising in hindsight are chosen to explain results; provided that a comparison group is included, it provides a reliable foundation for the results.
-
To the best of the reviewers’ knowledge, there are no previous examples of using SAE to examine the internal workings of a crystal formation model (among diffusion models that generate structures, there is FoldSAE, which focuses on proteins). This paper sets crystal-specific concepts, crystal system and coordination, as predefined evaluation targets (while space groups are used for exploratory visualization), and verifies whether the structural agreement rate (the proportion of generated structures that match the ground truth) is maintained when the SAE reconstruction is fed back into the generator (Figure 3(b)). The decision to accept this paper is supported by the fact that the authors did not merely transplant SAE research from other fields but presented the first framework for this subject matter that can serve as a benchmark for subsequent research.
Critical weaknesses
-
The correspondence with the concepts presented in this paper cannot be distinguished from the reading of model inputs. DiffCSP’s CSPNet (Jiao et al., NeurIPS 2023. cited in §3.1 of this paper) directly receives the inner product matrix of the lattice in the message computation of each layer, and node features begin with the embedding of compositions (atom types). Although the main text does not specify the time steps of the internal representations used to verify the correspondence, the supplementary code, by default, collects the internal representations used to measure the correspondence from inputs to the ground-truth crystal with uniformly random diffusion time noise added. Since the eight continuous features can be calculated from and the composition, and the crystal system can also be largely predicted from the lattice metric, it is possible that, for samples with low noise, the internal representation could produce a similar level of correlation simply by reading the input directly. The authors evaluate the strength of the model using effect sizes rather than p-values (for continuous features, the proportion of variance explained by a single feature, , ranges from 0.13 to 0.50; for crystal systems and coordination, the AUPRC metric for binary classification ranges from 0.19 to 0.74), rather than p-values to assess strength. However, since there is no comparison with values obtained from the input alone, readers cannot determine whether these values are large or small (whether SAE features contain more information than simple probes is also an issue in language models: Kantamneni et al., ICML 2025). This correspondence is the very basis for the second contribution in §1 (correspondence with 17 concepts), and unless the authors demonstrate an improvement over a model based solely on input, it cannot be said that DiffCSP incorporates crystallographic features internally.
-
The results of the intervention described in this paper (steering: manipulating the output by adding direction to the internal representation) do not demonstrate control over the targeted concepts. First, the authors do not demonstrate specificity for directions of uniform magnitude. The comparison in Figure 7 consists solely of a single neuron selected during training (according to the supplement’s README) and a baseline referred to in the main text as “random directions”; the caption for Figure 7 collectively refers to both as “the neuron-based baselines,” and in the supplement’s code, the latter is also a single neuron selected via a seed. In both cases, the direction and magnitude of the perturbation do not match those of SAE (comparisons with simple benchmarks are also required for language model steering: Wu et al., AxBench, ICML 2025). In Figure 6(c), there are even lines where the direction of b changes by three angles greater than b itself. Section 4.3 acknowledges this as lattice coupling, but if coupling is allowed, it is inconsistent with the authors’ practice of counting seven instances per concept as passing. Second, readers cannot verify whether the intervention worked in the intended direction. The authors do not specify the direction of each feature in the main text, and the pass criteria (§3.3) do not address whether the change is in the intended direction. Section 4.3 explicitly states, “positive steering increases a, b, and c, while decreasing α, β, and γ” (the caption for Figure 6(b) conveys the same meaning), and in Figure 6(d), the density is also decreasing in the direction of the steering. If the authors added the concepts in the direction of increasing them, as indicated by the code, this would be the opposite of their intended effect; if they added them without considering the direction, this would constitute an omission in the report. The third contribution in §1 of this paper (significant effects on the seven metrics) cannot, as it stands, be distinguished from the general effect that adding something to the internal representation changes the lattice.
Questions for authors
-
[Clarification] From which state at which diffusion time (a noise-added correct crystal or a trajectory in the process of being generated) were the internal representations used by the authors to map SAE learning to concepts collected? (The default in the supplement’s
scripts/collect_residuals.pyis a uniformly random time for each crystal.) Can the authors show the / AUPRC on the validation data for each time step ? If the correspondences are concentrated around small values, reviewers believe they cannot distinguish this from simply reading out the input lattice. (Critical Weaknesses 1) -
[Clarification] Under the authors’ setup, what is the value on the validation data when each concept is predicted solely from the input at the same time step ( or , composition, number of atoms), and what is the additional contribution of the SAE features after controlling for this? Please disclose the results from
scripts/probe_baselines.pyin the supplement if available. If the SAE features clearly outperform the baseline using only the input, the reviewers will reconsider their assessment of Critical Weakness 1. Additionally, for CN=4, the primary metric AUPRC is 0.222, and the secondary metric AUROC (where 0.5 is the chance level) is only 0.533. According to Table S1, the positive examples for CN=4 in the validation split are approximately 17.6% (16,426/93,286) on a site-by-site basis, so the chance level for AUPRC is also roughly 0.18. The authors should explain the rationale for counting these as concepts with confirmed correspondence. (Critical Weaknesses 1) -
[Clarification] Do the authors have results evaluating an intervention that involved drawing a large number of random directions aligned in magnitude with the SAE directions (see
scripts/run_norm_matched_control.pyin the supplements) using the same acceptance criteria? If the effects of the 17 directions do not fall within the upper range of that distribution, the reviewer believes the claim regarding the intervention would be weakened. In Figure 7, what is the average number of neurons for the “Random” and “Neuron” categories, and how did the authors select them? (While the main text refers to “Random” as “random directions,” the “random-neuron” in the supplement’sscripts/run_neuron_control.pyuses a single neuron selected by a seed.) Furthermore, while the intervention in the main text consists of three levels, (Equation (4)), the curves in Figure 7 are plotted using five levels ranging from 0 to 2. The README in the supplement states that five levels are used for comparison, but this is not mentioned in the main text. Did the authors generate the SAE separately using the same five levels, and what is the relationship between that and the pass criterion (three levels)? (Critical Weaknesses 2) -
[Clarification] The authors should provide a table showing the direction (sign of the correlation) for each of the 17 features during training and the sign of the changes observed following the intervention. Although Equation (4) does not include a term for direction, did the authors actually multiply by the direction and then sum the results? In Figure 6(d), are α, β, γ, and density decreasing as intended, or are they decreasing in the opposite direction? Does the vertical axis in Figure 7 (Normalized target z) preserve the sign (the curves for α, β, and γ are plotted upward)? If changes in the intended direction are added to the pass criteria (signs must match for both strengths, the value must be equal to or greater than that of the stronger one, and the result must be significant after correction), how many entries would still pass? As noted in §3.2, feature sharing is permitted; however, since feature 213 in the L1 layer (the second layer counting from L0) is shared by β and γ, the seven passing cases effectively correspond to six directions. Furthermore, α and the cubic crystal share the same L1-F368; thus, the same intervention results in a pass for α but not for the cubic crystal. Shouldn’t the number of passes be counted excluding this overlap? Furthermore, while the effect sizes for a, b, and c are small, approximately 0.02–0.10 (corresponding standardized effect size ) as read from Figure 6(d), what is the basis for considering these changes to be crystallographically significant? (Critical Weaknesses 2)
-
[Clarification] Figure 6(d) depicts tetragonal, triclinic, CN=4, and CN=6 systems, as well as volume per atom, in colors indicating “pass,” yet the main text in §4.3 and Appendix D classify these as “fail.” Furthermore, upon visual inspection of Figure 6(d), the point for a at η=2 appears slightly lower than the point at η=1 (approximately 0.022 and 0.027, The difference is smaller than the error bars, and it is not clear from the figure whether this satisfies the criterion in §3.3 (where the larger of the two values must be equal to or greater than the other). The authors should provide a table showing which is correct, along with the average changes, t-values, and values corrected for multiple comparisons (q-values) for each concept at the two strength levels. If the main text is correct, a correction to the figure would suffice; however, if the figure is correct, the discussion regarding the number of passes and failures in §4.3 and Appendix D would change. Furthermore, if the table reveals that any one of the seven does not meet the criterion, the reviewers will further strengthen their assessment of Critical Weakness 2. (Critical Weakness 2) There are two additional points I would like to confirm.
- Did the authors generate the feature assignments in Tables S2–S3 using only the publicly released training split in
scripts/discover_features.pyand a fixed checkpoint at 40 epochs? (The supplementary filedocs/MANUSCRIPT_ALIGNMENT.mdmentions the possibility that an older checkpoint selected from the validation split was used.) If they used a checkpoint selected from the validation split, then the validation on the validation data cannot strictly be considered validation on data not used for selection, and reviewers would lower the evaluation of Soundness. (Critical Strengths 1, Soundness) - How does the authors’ claim in Figure 6(e), that Top-K-style SAEs (which use only the top K features for each input) are stronger than other types of SAEs, align with Section 5 of the supplement’s
docs/KNOWN_LIMITATIONS.md, which states that since the results are not included, the main text “should not rely on a strong Top-K-superiority claim”?
- Did the authors generate the feature assignments in Tables S2–S3 using only the publicly released training split in
-
[Proposal] Do features of a similar magnitude for the same concept appear even in SAEs trained by the authors using different seeds (it is known that SAE features are highly seed-dependent: Paulo & Belrose, ICLR 2026)? Furthermore, by comparing how crystal system and space group features change in models that explicitly handle space groups, such as DiffCSP++ (Jiao et al., ICLR 2024; cited in Section 2 of this paper), this would yield insights specific to crystals. For discrete and local concepts such as crystal systems and coordination, it would be worthwhile to test the intervention of multiple features (subspaces) listed in Appendix D, as well as interventions limited to specific time steps or atomic species. (This suggestion does not affect the evaluation.)
-
[Proposal] In addition to the input-only criterion outlined by the authors in “Questions for Authors 2,” the authors propose applying the same procedure, selecting the most relevant feature from the training data, to the coordinates of the raw internal representation (neurons; 6 layers × 512 = 3,072 candidates), principal component analysis (PCA) components, and values applied to a SAE trained on a randomly initialized DiffCSP (6 layers × 1,024 candidates), as well as the values (upper bounds) of linear probes across the entire internal representation, the significance of using SAE becomes clear (it has been reported that SAE features appear interpretable even in randomly initialized models: Heap et al., arxiv 2025) . Since the evaluation criteria rely solely on the input for “Questions For Authors 2,” this proposal does not affect the evaluation.