- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This submission introduces EvoCircuit, a framework for protein optimization where the candidate selection is additionally guided by the proximity to sparse features of high-scoring experimentally validated sequences extracted from cross-layer transcoders of ESM-2. The authors evaluate their method on 7 protein fitness landscapes through retrospective active-learning simulations.
Critical strengths
The idea to incorporate sparse features extracted from pretrained pLMs into protein optimization is novel, and the paper is nicely written and easy to follow.
Critical weaknesses
The paper has following weakness:
W1 Contribution is limited
I do not understand why the authors split their contributions the way they do, namely into (i) the introduction of EvoCircuit, (ii) its evaluation, and (iii) ablation studies. I would expect that introducing a framework naturally entails evaluating it and conducting ablations; the latter two do not seem to constitute standalone contributions. This gives the impression that the main contribution is a somewhat hand-wavy implementation of the idea of incorporating sparse features into candidate selection, which, in my view, is rather incremental. In such cases, I would be more convinced by an extensive experimental evaluation demonstrating strong performance against a diverse set of baselines. However, I find the baseline comparisons limited and the gains achieved by EvoCircuit modest.
W2 Evaluation is weak and limited to fitness landscapes with fixed mutational sites
Having said that the idea of incorporating sparse features extracted from pretrained pLMs into protein optimization is novel, I am not convinced it's a good and effective one, and experiments on fitness landscapes in the paper do little to change my mind. Given that the proposed method uses sparse features to guide candidate retrieval rather than relying solely on surrogate predictions, it would be beneficial to demonstrate gains on fitness landscapes which don't cover 90%+ of the variant space, where surrogate models tend to be misspecified and having such additional signal would be extremely valuable. In the experiments conducted by the authors, I remain unconvinced that the proposed approach provides substantial benefits, particularly given the modest gains achieved by EvoCircuit.
W3 Authors did not provide code, and there is no reproducibility statement
Questions for authors
Q1: Could the authors conduct some experiments on fitness landscapes without fixed mutational sites? See for example landscapes used in [1-3]
Q2: What's the diversity of sequences returned by your method, and how does it change given the number of different anchors?
Q3: How does the performance depends on the number of selected features?
I understand this requests a lot of experiments, but given the limited in my eyes contribution of this submission, I believe this is the only angle which could make this work potentially relevant for the community. In particular, I will consider raising my score if Q1 will be adequately addressed.
References
[1] Kirjner et al., Improving protein optimization with smoothed fitness landscapes, ICLR 2024
[2] Kmicikiewicz et al., ProSpero: Active Learning for Robust Protein Design Beyond Wild-Type Neighborhoods, NeurIPS 2025
[3] Ren et al., Proximal Exploration for Model-guided Protein Sequence Design, ICML 2022
4 comments
Thank you for providing this additional analyses. The variable-site experiments are a step in the right direction, and do strengthen the paper. However, my previous concerns about value added by EvoCircuit remain not fully addressed.
On contribution
My original comment was meant to ask for a clearer demonstration of what EvoCircuit contributes and why that contribution matters. The revised introduction does describe the contribution more precisely in my view, mainly through empirical performance of the proposed method. As such, the strength of EvoCircuit performance remains central to my assessment.
On performance
In my view, the new experiments do not establish how competetive/usefull EvoCircuit is against more commonly used for variable-site protein search methods. Greedy acqusition is a useful control, but I give very little weight to gains over TS. With sparse coverage on those types of landscapes combined with small measurement budget, TS cannot reliably guide selection because it's stuck in its exploratory stage, and cannot start exploiting. In my eyes the paper still lacks a convincing comparison against more established methods used on variable-site landscapes (say, PEX, AdaLead, GFNs, etc), or demonstration of a setting where the value added by EvoCircuit is crucial for the performance (say, distribution shift or some other setting), where other methods fail. Additionally, here the gains are even harder to assess, because Table 4 (and 3 also) lack standards deviations / confidence intervals
Taken together, I appreciate the effort made by the authors, however given the current evidence I will be keeping my current score of weak rejection
Analysis of EvoCircuit’s Added Value
Thank you for encouraging stronger comparisons with established methods on variable-site landscapes and a clearer analysis of EvoCircuit’s added value.
Overall, EvoCircuit can complement existing acquisition policies with activity-associated CLT features. Our additional experiments identify both the candidates this signal recovers and the features influencing their selection. We also report SDs and 95% CIs for each method’s mean in Tables 3 and 4 (Appendix N).
[Comparison with established methods]
We compared and combined EvoCircuit with PEX, which you suggested as an established method for variable-site landscapes (Tables L.1–L.2). Initial observations and assay budgets were matched across four libraries and six predictors. Adding EvoCircuit increases mean top-1% discoveries in 21/24 combinations and discovery AUC in 15/24. On GFP, TabPFN—the strongest unaugmented PEX configuration by mean AUC—improves from 5.2±2.9 to 7.0±3.1 top-1% discoveries (paired-difference 95% CI [0.5, 3.1]).
[What CLT-guided selection contributes]
EvoCircuit supplies additional selection evidence for high-activity candidates insufficiently prioritized by the existing predictor and policy. With observed measurements and predictor scores held fixed, we compare the numbers of top-10% variants selected exclusively with EvoCircuit and exclusively by the baseline policy.
EVOLVEpro acknowledges that greedy selection may overlook fitness-landscape regions [1]; our RF/ESM-2 Pab1 comparison exhibits positive net recovery beyond its top-ranked candidates, alongside campaign discoveries increasing from 44.2 to 48.5. FolDE reports that top-N batches concentrate on previously successful mutation sites [2]. With our FolDE/ESM-2 adaptation, CLT retrieval recovers high-activity candidates outside the top-ranked set, providing another selection basis even when the predictor already uses pLM information. ALDE finds that global prediction accuracy and calibration need not track optimization success [3]. Our DNN/XGB results likewise identify benefit through actual candidate recovery: with Greedy and the FolDE/DNN/XGB-derived predictors, low-order variants (≤2 mutations) account for most additional top-10% discoveries in all nine GFP/cgreGFP/AAV combinations.
PEX explicitly prioritizes high-fitness candidates with fewer mutations [4]. Adding EvoCircuit to PEX/RF increases mean higher-order top-10% discoveries (≥3 mutations) across GFP, cgreGFP, and AAV. These results support EvoCircuit as an orthogonal acquisition approach that can yield additional performance gains when combined with existing predictors and search policies (Figures M.1–M.2).
[Interpreting acquisition decisions and their significance]
These selections can be traced to the features used by acquisition. In a GFP example, an activity-associated feature admits a top-1% candidate omitted by PEX; removing that feature from retrieval excludes the candidate (Appendix L). Following InterPLM’s interpretability workflow [5], we examine feature activations in relation to biological annotations. In GCK, we inspect local correspondence with the annotated substrate-binding residue N204 and separately verify the feature’s influence on candidate selection (Appendix J, Tables J.1–J.2).
Although this does not establish a biochemical mechanism, this connection makes the feature evidence behind acquisition decisions inspectable and provides a basis for formulating concrete hypotheses for follow-up experiments under limited assay budgets.
EvoCircuit therefore offers a framework that connects improved candidate discovery with feature-level understanding of acquisition decisions in protein optimization.
[References]
[1] Jiang et al., Rapid in silico directed evolution by a protein language model with EVOLVEpro, Science, 2025.
[2] Roberts et al., Low-N Protein Activity Optimization with FolDE, arXiv:2510.24053, 2025.
[3] Yang et al., Active learning-assisted directed evolution, Nature Communications, 2025.
[4] Ren et al., Proximal Exploration for Model-guided Protein Sequence Design, ICML, 2022.
[5] Simon and Zou, InterPLM: discovering interpretable features in protein language models via sparse autoencoders, Nature Methods, 2025.
Thank you for the additional experiments and the clarification. Revised Appendix J is much more interesting. Regardless of the outcome here, I think demonstrations and connections like this are directions that could make this work potentially exciting for the community. I will change my score to 3.
Expanded Evaluation Beyond Fixed-Site Benchmarks
Thank you for your careful review and constructive suggestions.
[W1 — Clarifying the contribution]
We revised the contribution statement in the Introduction to emphasize the acquisition framework and empirical findings, rather than evaluation and ablation as standalone contributions. It now highlights predictor- and landscape-dependent discovery gains across eleven landscapes (Sections 5.1–5.2) and feature-level analysis linking activation patterns to recommendations through feature removal (Section 5.4). Section 5.3 separately provides matched fixed-state evidence for the contribution of activity-based feature selection.
[W2 / Q1 — Evaluation beyond fixed-site benchmarks]
Following your suggestion and the references you shared, we added Pab1, GFP, cgreGFP, and AAV to evaluate libraries with variable mutation-site subsets (Sections 4.1 and 5.2, Table 4). Pab1 contains single and double mutants; the other three also include higher-order variants. GFP and cgreGFP vary mutation sites across their sequences, while AAV varies subsets within a 28-residue region.
Each comparison uses the same predictor configuration, ten paired seeds, 64 shared uniform-random initial measurements, and 112 additional variant selections. Across the full measured pools, EvoCircuit increases mean top-10% discoveries in 17 of 20 comparisons against greedy acquisition. On cgreGFP/FolDE, discoveries increase from 20.5 to 29.9 (+45.9%), and on AAV/XGB from 48.2 to 55.7 (+15.6%).
Unlike its mixed fixed-site results, FolDE with EvoCircuit improves all three discovery metrics—top-10% count, top-1% count, and discovery AUC—on GFP, cgreGFP, and AAV. For DNN and XGB, EvoCircuit likewise increases the mean values of all three discovery metrics on these libraries. While gains remain setting-dependent, these results show discovery benefits beyond near-complete combinations of a few fixed sites.
[Q2 — Discovery coverage and anchor count]
Appendix J measures distinct-hit coverage across repeated campaigns rather than within-campaign sequence distance. On TrpB4/RF, the union of distinct above-parent variants across 20 campaigns increases from 302 to 352 (+16.6%), with nearly unchanged mean per-campaign discoveries (greedy: 64.6; EvoCircuit: 64.4). Across the four added libraries, the 10-campaign union of top-10% variants increases in 16 of 20 matched comparisons. These results show broader discovery coverage, including less repeated discovery in the TrpB4/RF setting.
We also tested A = 4, 16, and 32 anchors with RF on GB1 and TrpB4, fixing K = 8, shortlist size M = 128, and the direct predictor/feature-guided allocation at 8+8. Each setting used 10 paired seeds, the parent plus 63 shared random initial variants, and 112 additional variant selections. Increasing A distributed the fixed shortlist budget across more high-activity neighborhoods, without producing a clear increase in sequence diversity (normalized Hamming distance) in these tests. These anchor- and feature-count analyses are reported here rather than in the current revision.
[Q3 — Sensitivity to the number of selected features]
In the same RF study, we varied the maximum number of selected features K while keeping A = 16 anchors. Mean above-parent discovery AUCs are:
An exploratory DNN/one-hot follow-up on GFP and AAV used 10 paired seeds, 64 shared uniform-random initial measurements, and 112 additional variant selections. Increasing K from 8 to 32 raises top-10% discovery AUC from 17.6 to 21.5 on AAV (paired gain 3.9; pointwise 95% CI [1.9, 6.1]) and from 10.3 to 12.6 on GFP (paired-difference 95% CI [-0.1, 4.8]). Among the sensitivity contrasts reported here, only the AAV/DNN increase is statistically significant after Holm correction within the respective four-contrast families.
We retain K = 8 as a common reference, not a claimed optimum. The response to K varies by setting, and the AAV/DNN result shows that a larger feature set can further improve acquisition.
[W3 — Implementation and reproducibility]
We plan to release the implementation and feature-exploration demo with the camera-ready version. Appendices A, B, and D specify the architecture, acquisition settings, and predictor configurations; Appendix L illustrates our demo. These details and the planned code release are intended to support independent reproduction of the experiments and inspection of candidate-selection decisions.
Taken together, the expanded evaluation shows that EvoCircuit can complement existing predictors in discovering high-activity variants beyond the original fixed-site benchmarks. We welcome further discussion of these results and their interpretation.