- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses an practical problem: efficiently selecting promising protein variants when the combinatorial search space is large and experimental evaluations are expensive. This paper introduces BaLiSq, a Bayesian linear regression framework for protein sequence optimization that constructs a Gaussian prior over substitution weights from a positive-definite kernel defined on single-mutant variants. Building on BaLiSq, the authors propose BaLiSqO, a Bayesian optimization method based on Thompson sampling over mutation-constrained combinatorial protein spaces. The paper also derives a closed-form information-gain expression and evaluates predictive performance on ProteinGym multi-mutant assays, as well as optimization performance on simulated landscapes involving FoldX, RaSP, and a ProGen2-derived score.
Critical strengths
S1. The paper uses a kernel on single-mutant variants to construct a prior over substitution weights, preserving a connection to GP regression in the single-mutant setting while obtaining an additive representation for multi-mutant sequences. This offers a practical compromise between incorporating kernel-derived information and maintaining a tractable representation of large combinatorial spaces.
S2. The sampled linear objective can be optimized efficiently under a mutation-count constraint, avoiding explicit enumeration of the entire candidate space. The treatment of previously selected variants and the closed-form information-gain result further develop the framework into a usable Bayesian optimization approach rather than merely a predictive model.
Critical weaknesses
W1. The equivalence to GP regression on single mutants is an important characterization, but by itself it does not establish a substantial methodological advance. Likewise, additive sequence models and Thompson sampling are established ideas. The most distinctive contribution appears to be their particular combination: kernel-induced Bayesian substitution weights, structured posterior covariance on multi-mutants, and tractable constrained acquisition optimization.
The paper would be stronger if it more clearly isolated the benefit of this combination relative to the closest existing approaches. The current results suggest that the method can be useful in low-data regimes and competitive in data-rich regimes, but do not fully establish which of its design choices account for the observed gains or when those gains are most reliable.
W2. The computational objectives provide controlled and reproducible test cases, but their evidential scope is limited. In particular, additive FoldX/RaSP settings cannot establish robustness to arbitrary substitution-specific interactions, and a ProGen2-derived score is not equivalent to experimentally measured protein fitness.
The paper acknowledges that its additive predictive mean does not model arbitrary epistasis. This is a reasonable design choice in some low-data regimes, but it also defines a meaningful boundary on the method's applicability. The current evidence does not establish that the reported optimization advantage generalizes to experimentally measured landscapes dominated by specific interactions.
Questions for authors
Q1. Could the authors more explicitly distinguish the contribution of BaLiSq from existing kernel-based Bayesian regression and additive sequence models? In particular, which experiments demonstrate that the combination of kernel-induced weight covariance and tractable Thompson sampling provides an advantage beyond the additive mean representation alone?
Q2. Can the authors provide evidence on an available experimentally measured multi-mutant landscape with meaningful epistasis, or otherwise clarify the scope of the conclusions supported by the current computational objectives? It would be particularly helpful to distinguish the intended low-data use case from settings where specific higher-order interactions dominate.
2 comments
W2+Q2
We thank the reviewer for highlighting this limitation. We have added a dedicated evaluation on the experimentally measured four-site GB1 landscape (Figure C6), comparing BaLiSq with the ALDE DNN ensemble and MULTI-evolve across training-set sizes.
In five-fold cross-validation with more than 100,000 training variants per fold (Figure C5), neither neural-network baseline consistently outperforms BaLiSq, showing that greater model flexibility does not automatically translate into better predictions on an experimental landscape with meaningful epistasis, even with substantial training data.
We agree that in silico objectives such as additive FoldX/RaSP- or ProGen2-derived scores are not equivalent to experimentally measured protein fitness. Ideally, we would validate optimization experimentally, but we unfortunately do not have access to wet-lab facilities. A comparable wet-lab benchmark would however require independent campaigns for each method, repeated across starting proteins and runs resulting in hundreds of experimental campaigns per batch (23 wild types x 6 baselines x 3 repeats for statistical inference). Even with access to wet-lab facilities, this would require substantial experimental resources and time. We therefore use computational oracles as rational alternatives for controlled optimization comparisons.
We chose to use an additive version of FoldX, because of its computational complexity, but are now rerunning the study with the non-additive (epistatic) version of FoldX Stability and will include the results in the camera ready version.
As complementary evidence, Figure 2 evaluates the underlying base BaLiSq model on real DMS datasets with experimentally measured fitness values. While BaLiSq does not explicitly model substitution-specific interactions, its predictive performance shows that it can still accurately predict experimental multi-mutant fitness in the evaluated low-data settings.
We appreciate this suggestion and have clarified the distinction between experimental wet-lab validation and computational optimization benchmarks in the discussion of the manuscript and will add the non-additive (epistatic) version of FoldX in the camera ready version to strengthen the paper with an additional epistatic oracle.
We thank the reviewer for their careful efforts and constructive feedback. We will upload the revised manuscript before the revision deadline.
W1+Q1
We agree with the reviewer that a clearer understanding of what drives the observed gains would strengthen the study. In response to this suggestion, we have now added additional ablations and baselines to the study to isolate the benefits of our approach.
For the optimization part we have:
Figure 2 and C1 shows that BaLiSq performs better (in terms of prediction) than competing methods when we have less observations than the number of possible substitutions in the design space. Figure C1 shows how much of this is due to the prior mean and how much is the kernel.
We expect that once the training data grows substantially above the design space, non-additive models will be able to model the interaction effects and get better predictions than an additive model. Our method focuses on the low-data regime in a large design space that characterizes many experimental settings. Here batch sizes are usually 96 and only a limited number of batches are evaluated. Figure 2 shows that in this regime it is hard to learn more than additive effects. Though there might be epistatic effects, it would require much more data to model it and our point is, that if you do not have that much data, then an additive model can offer a useful balance between predictive performance and computational tractability.