acceptodds
Withdrawn from ICLR 2027

CrystalSAE: Interpreting and Steering Crystal Diffusion Models through Sparse Autoencoders

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Crystal diffusion models perform well in crystal structure prediction, but their internal representations remain poorly understood. To improve model interpretability and inform controllable generation, we propose CrystalSAE, a sparse autoencoder (SAE) framework for identifying and probing crystallographic features in crystal diffusion models. Specifically, we apply SAEs to the layer-wise residual updates of DiffCSP, a typical crystal diffusion model, and relate the resulting feature activations to crystallographic concepts. We then intervene along selected feature directions to examine their influence on generation. Extensive experiments on MP-20 dataset confirm that sparse features selected on training data retain statistically significant associations with 17 crystallographic concepts on held-out data. Crystal visualizations further illustrate associations between selected features and 68 space groups. Moreover, our intervention experiments show that stronger perturbations of selected features produce larger average changes in the lattice parameters and density of generated crystals. These findings reveal crystallographically aligned features within DiffCSP and suggest a path toward more interpretable and controllable crystal generation.

Primary area:
interpretability and explainable AI
Keywords:
sparse autoencoders, crystal structure prediction, diffusion models, mechanistic interpretability, concept discovery, causal steering

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_57NXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose CrystalSAE, a framework for interpreting the internal representation of DiffCSP, a diffusion model that generates crystal structures based on composition, using a sparse autoencoder (SAE: an autoencoder that re-expresses internal activation vectors as the sum of a very small number of learned dictionary directions). CrystalSAE trains a layer-wise SAE on the differences (residual updates) that each layer of DiffCSP’s denoising network, CSPNet (a 6-layer graph neural network), adds to the internal representation.

For 17 concepts, including lattice constants (the lengths of three sides, a, b, and c, and three angles, α, β, and γ), volume per atom, density, the seven crystal systems, coordination number (CN: the number of nearest neighbors; CN=4 and CN=6). Using only the training split of the standard crystal generation benchmark MP-20, the authors select one SAE feature most strongly correlated with each of these 17 concepts, fix its orientation, and demonstrate on the validation split that all 17 pairs correspond statistically significantly to their respective concepts. The authors also present exploratory visualizations of the distribution of these features by layer and diffusion time, as well as for 68 space groups.

The authors investigate how crystals generated from the composition of the test split change when the directions of the selected features are added to the internal representation as an intervention. The authors report that significant changes occur in seven quantities, a, b, c, α, β, γ, and density, depending on the amount added; that the remaining 10 concepts (7 crystal systems, CN=4/6, and volume per atom) do not meet the intervention’s pass criteria; and that the dose-response curve is smoother than that based on the orientation of individual neurons.

Critical strengths
  1. This paper employs a validation procedure that strictly separates feature selection from evaluation. The authors predefined 17 concepts, fixed the features and their orientations solely in the training split, then performed substitution tests and multiple comparison corrections (Benjamini–Hochberg method) in the validation split, reserving the test split exclusively for the intervention. The 10 concepts that did not meet the intervention criteria are also disclosed in Appendix D, and the authors explicitly state that they do not treat the success of the intervention as causal sufficiency. This procedure mitigates the selection bias, commonly seen in interpretive studies using SAE, where features that appear promising in hindsight are chosen to explain results; provided that a comparison group is included, it provides a reliable foundation for the results.

  2. To the best of the reviewers’ knowledge, there are no previous examples of using SAE to examine the internal workings of a crystal formation model (among diffusion models that generate structures, there is FoldSAE, which focuses on proteins). This paper sets crystal-specific concepts, crystal system and coordination, as predefined evaluation targets (while space groups are used for exploratory visualization), and verifies whether the structural agreement rate (the proportion of generated structures that match the ground truth) is maintained when the SAE reconstruction is fed back into the generator (Figure 3(b)). The decision to accept this paper is supported by the fact that the authors did not merely transplant SAE research from other fields but presented the first framework for this subject matter that can serve as a benchmark for subsequent research.

Critical weaknesses
  1. The correspondence with the concepts presented in this paper cannot be distinguished from the reading of model inputs. DiffCSP’s CSPNet (Jiao et al., NeurIPS 2023. cited in §3.1 of this paper) directly receives the inner product matrix Lt⊤LtL_t^\top L_t of the lattice LtL_t in the message computation of each layer, and node features begin with the embedding of compositions (atom types). Although the main text does not specify the time steps of the internal representations used to verify the correspondence, the supplementary code, by default, collects the internal representations used to measure the correspondence from inputs to the ground-truth crystal with uniformly random diffusion time tt noise added. Since the eight continuous features can be calculated from L⊤LL^\top L and the composition, and the crystal system can also be largely predicted from the lattice metric, it is possible that, for samples with low noise, the internal representation could produce a similar level of correlation simply by reading the input directly. The authors evaluate the strength of the model using effect sizes rather than p-values (for continuous features, the proportion of variance explained by a single feature, R2R^2, ranges from 0.13 to 0.50; for crystal systems and coordination, the AUPRC metric for binary classification ranges from 0.19 to 0.74), rather than p-values to assess strength. However, since there is no comparison with values obtained from the input alone, readers cannot determine whether these values are large or small (whether SAE features contain more information than simple probes is also an issue in language models: Kantamneni et al., ICML 2025). This correspondence is the very basis for the second contribution in §1 (correspondence with 17 concepts), and unless the authors demonstrate an improvement over a model based solely on input, it cannot be said that DiffCSP incorporates crystallographic features internally.

  2. The results of the intervention described in this paper (steering: manipulating the output by adding direction to the internal representation) do not demonstrate control over the targeted concepts. First, the authors do not demonstrate specificity for directions of uniform magnitude. The comparison in Figure 7 consists solely of a single neuron selected during training (according to the supplement’s README) and a baseline referred to in the main text as “random directions”; the caption for Figure 7 collectively refers to both as “the neuron-based baselines,” and in the supplement’s code, the latter is also a single neuron selected via a seed. In both cases, the direction and magnitude of the perturbation do not match those of SAE (comparisons with simple benchmarks are also required for language model steering: Wu et al., AxBench, ICML 2025). In Figure 6(c), there are even lines where the direction of b changes by three angles greater than b itself. Section 4.3 acknowledges this as lattice coupling, but if coupling is allowed, it is inconsistent with the authors’ practice of counting seven instances per concept as passing. Second, readers cannot verify whether the intervention worked in the intended direction. The authors do not specify the direction of each feature in the main text, and the pass criteria (§3.3) do not address whether the change is in the intended direction. Section 4.3 explicitly states, “positive steering increases a, b, and c, while decreasing α, β, and γ” (the caption for Figure 6(b) conveys the same meaning), and in Figure 6(d), the density is also decreasing in the direction of the steering. If the authors added the concepts in the direction of increasing them, as indicated by the code, this would be the opposite of their intended effect; if they added them without considering the direction, this would constitute an omission in the report. The third contribution in §1 of this paper (significant effects on the seven metrics) cannot, as it stands, be distinguished from the general effect that adding something to the internal representation changes the lattice.

Questions for authors
  1. [Clarification] From which state at which diffusion time (a noise-added correct crystal or a trajectory in the process of being generated) were the internal representations used by the authors to map SAE learning to concepts collected? (The default in the supplement’s scripts/collect_residuals.py is a uniformly random time for each crystal.) Can the authors show the R2R^2 / AUPRC on the validation data for each time step tt? If the correspondences are concentrated around small tt values, reviewers believe they cannot distinguish this from simply reading out the input lattice. (Critical Weaknesses 1)

  2. [Clarification] Under the authors’ setup, what is the value on the validation data when each concept is predicted solely from the input at the same time step (LtL_t or Lt⊤LtL_t^\top L_t, composition, number of atoms), and what is the additional contribution of the SAE features after controlling for this? Please disclose the results from scripts/probe_baselines.py in the supplement if available. If the SAE features clearly outperform the baseline using only the input, the reviewers will reconsider their assessment of Critical Weakness 1. Additionally, for CN=4, the primary metric AUPRC is 0.222, and the secondary metric AUROC (where 0.5 is the chance level) is only 0.533. According to Table S1, the positive examples for CN=4 in the validation split are approximately 17.6% (16,426/93,286) on a site-by-site basis, so the chance level for AUPRC is also roughly 0.18. The authors should explain the rationale for counting these as concepts with confirmed correspondence. (Critical Weaknesses 1)

  3. [Clarification] Do the authors have results evaluating an intervention that involved drawing a large number of random directions aligned in magnitude with the SAE directions (see scripts/run_norm_matched_control.py in the supplements) using the same acceptance criteria? If the effects of the 17 directions do not fall within the upper range of that distribution, the reviewer believes the claim regarding the intervention would be weakened. In Figure 7, what is the average number of neurons for the “Random” and “Neuron” categories, and how did the authors select them? (While the main text refers to “Random” as “random directions,” the “random-neuron” in the supplement’s scripts/run_neuron_control.py uses a single neuron selected by a seed.) Furthermore, while the intervention in the main text consists of three levels, η∈{0,1,2}\eta\in\left\lbrace 0,1,2\right\rbrace (Equation (4)), the curves in Figure 7 are plotted using five levels ranging from 0 to 2. The README in the supplement states that five levels are used for comparison, but this is not mentioned in the main text. Did the authors generate the SAE separately using the same five levels, and what is the relationship between that and the pass criterion (three levels)? (Critical Weaknesses 2)

  4. [Clarification] The authors should provide a table showing the direction (sign of the correlation) for each of the 17 features during training and the sign of the changes observed following the intervention. Although Equation (4) does not include a term for direction, did the authors actually multiply by the direction and then sum the results? In Figure 6(d), are α, β, γ, and density decreasing as intended, or are they decreasing in the opposite direction? Does the vertical axis in Figure 7 (Normalized target z) preserve the sign (the curves for α, β, and γ are plotted upward)? If changes in the intended direction are added to the pass criteria (signs must match for both strengths, the value must be equal to or greater than that of the stronger one, and the result must be significant after correction), how many entries would still pass? As noted in §3.2, feature sharing is permitted; however, since feature 213 in the L1 layer (the second layer counting from L0) is shared by β and γ, the seven passing cases effectively correspond to six directions. Furthermore, α and the cubic crystal share the same L1-F368; thus, the same intervention results in a pass for α but not for the cubic crystal. Shouldn’t the number of passes be counted excluding this overlap? Furthermore, while the effect sizes for a, b, and c are small, approximately 0.02–0.10 (corresponding standardized effect size dzd_z) as read from Figure 6(d), what is the basis for considering these changes to be crystallographically significant? (Critical Weaknesses 2)

  5. [Clarification] Figure 6(d) depicts tetragonal, triclinic, CN=4, and CN=6 systems, as well as volume per atom, in colors indicating “pass,” yet the main text in §4.3 and Appendix D classify these as “fail.” Furthermore, upon visual inspection of Figure 6(d), the point for a at η=2 appears slightly lower than the point at η=1 (approximately 0.022 and 0.027, The difference is smaller than the error bars, and it is not clear from the figure whether this satisfies the criterion in §3.3 (where the larger of the two values must be equal to or greater than the other). The authors should provide a table showing which is correct, along with the average changes, t-values, and values corrected for multiple comparisons (q-values) for each concept at the two strength levels. If the main text is correct, a correction to the figure would suffice; however, if the figure is correct, the discussion regarding the number of passes and failures in §4.3 and Appendix D would change. Furthermore, if the table reveals that any one of the seven does not meet the criterion, the reviewers will further strengthen their assessment of Critical Weakness 2. (Critical Weakness 2) There are two additional points I would like to confirm.

    • Did the authors generate the feature assignments in Tables S2–S3 using only the publicly released training split in scripts/discover_features.py and a fixed checkpoint at 40 epochs? (The supplementary file docs/MANUSCRIPT_ALIGNMENT.md mentions the possibility that an older checkpoint selected from the validation split was used.) If they used a checkpoint selected from the validation split, then the validation on the validation data cannot strictly be considered validation on data not used for selection, and reviewers would lower the evaluation of Soundness. (Critical Strengths 1, Soundness)
    • How does the authors’ claim in Figure 6(e), that Top-K-style SAEs (which use only the top K features for each input) are stronger than other types of SAEs, align with Section 5 of the supplement’s docs/KNOWN_LIMITATIONS.md, which states that since the results are not included, the main text “should not rely on a strong Top-K-superiority claim”?
  6. [Proposal] Do features of a similar magnitude for the same concept appear even in SAEs trained by the authors using different seeds (it is known that SAE features are highly seed-dependent: Paulo & Belrose, ICLR 2026)? Furthermore, by comparing how crystal system and space group features change in models that explicitly handle space groups, such as DiffCSP++ (Jiao et al., ICLR 2024; cited in Section 2 of this paper), this would yield insights specific to crystals. For discrete and local concepts such as crystal systems and coordination, it would be worthwhile to test the intervention of multiple features (subspaces) listed in Appendix D, as well as interventions limited to specific time steps or atomic species. (This suggestion does not affect the evaluation.)

  7. [Proposal] In addition to the input-only criterion outlined by the authors in “Questions for Authors 2,” the authors propose applying the same procedure, selecting the most relevant feature from the training data, to the coordinates of the raw internal representation (neurons; 6 layers × 512 = 3,072 candidates), principal component analysis (PCA) components, and values applied to a SAE trained on a randomly initialized DiffCSP (6 layers × 1,024 candidates), as well as the values (upper bounds) of linear probes across the entire internal representation, the significance of using SAE becomes clear (it has been reported that SAE features appear interpretable even in randomly initialized models: Heap et al., arxiv 2025) . Since the evaluation criteria rely solely on the input for “Questions For Authors 2,” this proposal does not affect the evaluation.

Reviewer_CeTFOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies crystal structure prediction using a crystal diffusion model. In this work, the authors propose a CrystalSAE, which is an interpretable crystal diffusion model using sparse autoencoders that excels at predicting crystal structures. In this work the authors use a frozen crystal diffusion model and apply sparse autoencoders to interpret crystal generative models.

Critical strengths
  • This work proposed an interpretable crystal generation model using sparse autoencoders, combining feature discovery with crystallographic analysis.
  • This work indentifies features associated with seventeen crystallographic concepts on MP-20. Also, the seventeen associations remain statistically significant on the held-out test dataset after multiple-testing correction.
Critical weaknesses
  • The core architecture (top-K SAEs on denoiser residual updates followed by steering along decoder directions) follows Surkov et al. (2024) [1], and the crystal-specific changes are limited to concept definitions and evaluation. This limited the novelty of their framework, as they used the above paper idea and used it for the crystal domain.

  • In this work the authors apply CrystalSAE only for DiffCSP on the MP-20 dataset. But here is not well discussed that aligned sparse features and the steering effects are also observed in newer generation models like MatterGen, FlowMM, or OMatG, etc. This approach differs from the general claim of the paper on crystal diffusion models. Please perform the above experiments to establish the generalizability of the claim.

  • In this work results are only for DiffCSP on MP-20, but this work does not check the validity, novelty, or stability on the steered crystals and does not perform controllable generation of new crystals.

  • Furthermore, this paper lacks an ablation study of their proposed framework. Here this work misses the ablation importance of each component of their CrystalSAE, factorizing residual updates instead of hidden states, the diffusion timesteps and layers at which steering is applied, and the stability of the discovered features across SAE training seeds.

[1] Surkov V, Wendler C, Mari A, Terekhov M, Deschenaux J, West R, Gulcehre C, Bau D. One-step is enough: Sparse autoencoders for text-to-image diffusion models. arXiv preprint arXiv:2410.22366. 2024 Oct 28.

Questions for authors

Please see the weakness

Reviewer_ZZsYOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper explores whether the internal representations of DiffCSP capture meaningful information about crystal structures and whether such information can be used to influence generation. To investigate this, the authors train sparse autoencoders on intermediate model activations and examine how the extracted features relate to different structural properties. Their analysis suggests that certain features encode information about lattice geometry, symmetry, and atomic coordination, with these relationships remaining consistent on held-out data. The authors also modify selected internal features during sampling and observe corresponding changes in the resulting crystal structures. Overall, the study provides an initial investigation into how crystal diffusion models organize structural information internally and how these representations might support more controllable generation.

Critical strengths

The paper explores the interpretability of crystal structure prediction models by identifying meaningful features within their learned representations. This is an interesting and worthwhile research direction, as understanding how these models capture crystallographic information could provide valuable insights into their underlying generation mechanisms. In the long run, such insights may help improve the interpretability, reliability, and controllability of crystal generative models, potentially benefiting their applications in materials discovery.

Critical weaknesses

(1) Inaccurate distinction between crystal structure prediction and de novo crystal generation. The paper does not clearly distinguish between crystal structure prediction (CSP) and de novo crystal generation. In the related work section, several methods, such as SymmCD and WyFormer, are discussed in the context of CSP, although they primarily focus on de novo generation rather than conventional CSP. More broadly, the paper frequently treats these two tasks interchangeably, despite their different objectives and evaluation settings. This conceptual ambiguity raises concerns about the accuracy of the literature review and the paper's understanding of the crystal generation research landscape.

(2) Limited scope and insufficient generalizability. The scope of this work is rather limited, as the experiments focus exclusively on DiffCSP and the crystal structure prediction (CSP) task, without considering de novo crystal generation. Although DiffCSP is a representative crystal diffusion model, numerous crystal generative models with different architectures and modeling paradigms have been proposed in recent years. Evaluating only a single model is insufficient to demonstrate whether the identified interpretable features reflect general crystallographic knowledge or are specific to DiffCSP. Moreover, de novo generation is an important application of crystal generative models, particularly for discovering novel materials. Without extending the analysis to other representative models and generation tasks, the contributions remain narrow, and the generalizability of the findings is not sufficiently established.

(3) Unclear practical significance of the identified interpretable features. The reviewer finds the practical purpose of identifying these interpretable features insufficiently motivated. While the paper demonstrates associations between learned features and crystallographic properties, as well as structural changes resulting from feature interventions, it remains unclear what meaningful insights these findings provide into the underlying generation mechanism. Simply showing that perturbing certain features changes lattice parameters or other structural properties does not necessarily establish their practical usefulness. More importantly, a key motivation for improving model interpretability should be to better understand and potentially improve crystal generation. However, the paper does not investigate whether these features can help improve CSP accuracy, structural validity, or generation quality, nor does it demonstrate effective control toward desired structural targets. Without a clearer connection between interpretability and downstream improvements, the significance of the identified features remains questionable.

Questions for authors

What is the practical value of these interpretable features? Could they be leveraged to improve CSP performance or enable more controllable crystal generation? More importantly, what are the key insights and conclusions drawn from identifying these features?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.