- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper presents a workflow for identifying individual SAE features and feature combinations whose ablation preferentially affects a biological category of interest. Applied to GroEL/GroES clients, it identifies features in ESMC representations and evaluates their structural effects in held-out clients and matched controls.
Critical strengths
- The protocol is well motivated and methodologically reasonable.
- Its use of held-out evaluation, matched controls, and stability screening strengthens the evidence beyond post hoc biological interpretation.
- Considering combinations of features alongside individual features is valuable.
- GroEL/GroES dependence provides a biologically well-motivated case study.
Critical weaknesses
Enthusiasm is tempered by the limited scope of evaluation: a single biological category (mechanism) and perturbation (structural change) are examined. The results therefore provide limited evidence for the broader applicability of the workflow.
The analysis also begins with a substantial prefilter, retaining only 78 features from a 16,384-feature(?) SAE. Although restricting attention to commonly activated, interpretable features is reasonable, the contribution of this filtering step and the sensitivity of the results to its criteria remain unclear. Broader evaluation would clarify the method’s value and generality.
Questions for authors
- How many distinct SAE features are activated across the discovery cohort before filtering, and how many remain after each filtering criterion?
- How sensitive are the findings to the activation thresholds and the exclusion of features with generic descriptions?
- Can the workflow demonstrate comparable selectivity for another biological category or using a different intervention or output measure?
2 comments
While I think the method is reasonable and possibly useful, even considering the thermostability evaluation, I still do not think this is appropriately scoped for an ICLR main conference paper, although reasonable people may disagree. I will be maintaining my score.
Dear Reviewer R8sL,
We appreciate the reviewer’s thoughtful feedback on SAE feature selection for discovery and about the empirical/domain scope, and are glad that they found our workflow and application of interaction discovery to SAEs and biology well motivated and having the potential to strengthen discovery and biological interpretation. Please find our response below.
SAE features
Filtering down SAE features keeps the model interpretable and the discoveries to be more impactful. The SAE begins with 16,384 features, where 10,245 activate on at least one residue of the 25 discovery clients. For the discovery to find meaningful terms/interactions that generalize to held-out proteins, ablating features must be impactful enough. Hence, we filter out SAE features that do not activate in at least 5% of a discovery proteins’ residues, and also are shared across at least 20 (80%) of discovery proteins. This leaves us with 86 SAE features. Finally, we remove 8 SAE features that do not have an interpretable description, for a total of 78 SAE features to do discovery over. Additionally, there are severe computational and algorithmic constraints on the discovery SAE feature set: ProxySPEX [1] recommends scaling the number of masks at a rate of O(nlogn), where n is the number of features for discovery, which means expanding the size of the set requires substantially more masks. Due to these computational constraints, we felt that these reasonable thresholds (5%, 80%) created a generalizable, and computationally reasonable, discovery set (and size).
For other discoveries that are more focused on individual/localized discovery (e.g. understanding a targeted portion of a protein of interest) which is less about holdout generalization, expanding the SAE feature set could be useful. We will include a discussion about this in the final draft.
Activation thresholds
There is no threshold for activation strength for SAE feature selection. We consider an SAE feature active on a residue if it is >0 and we only select features for discovery based on coverage with respect to how prominent it is activated across a protein (5% threshold), and if it is shared across proteins (appears in >= 20 discovery proteins). The sparsity property of SAEs is particularly appealing for ProxySPEX due to its masking nature (zero-ing out features) in interaction discovery, which provides a nice inductive bias that does not push the ESMC representations off manifold when we do masked ablations for discovery. Analyzing the terms/interactions after including generic SAE features, and manually providing an interpretation from 1) the resulting structural effects and 2) the descriptions of the labeled SAE features is potential for future work, and will be discussed in the final draft.
Another biological category, intervention, or outcome measure
Please see our response to Reviewer tg98 (A second setting: thermostability.) for an additional domain/application. For GroEL/ES, we also evaluate our selected terms on 10 Class III- proteins (Fujiwara et al., 2010 [2]), which bind GroEL but do not require it to fold, a stricter control than the SAE-matched set. Effects are larger in clients than in both control groups (Figure 7, Appendix A.2); please see our response to Reviewer tg98 for the details.
We focus on intervening on SAE features as this paper is motivated by interpretability and scientific (hypothesis) discovery, so the interpretability of the features is compelling. SAE features have broadly been understood and interpreted marginally (one at a time), so discovering interactions between SAE features in biological folding models is novel, and, as we propose, can provide nuanced insights into scientific behavior for proteins, in this case. Although our workflow is principled and generalizable to non-SAE/interaction discovery, we do not focus on it here. Regarding output measure, Figures 9-11 and Table 2 (Appendix A.4) report the client contrast on raw TM-score change and two RMSD metrics. It is positive on all four in both settings, despite the terms not being discovered with RMSD.
[1] Butler et al. “ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMs.” NeurIPS, 2025.
[2] Fujiwara et al. “A systematic survey of in vivo obligate chaperonin-dependent substrates.” EMBO J., 2010.