acceptodds
Under review as a conference paper at ICLR 2027

Feedback-Calibrated Protein Optimization with Batch-Aligned Tail Arbitration

Abstract

Protein optimization aims to discover high-fitness sequences under a limited experimental budget. Existing machine-learning methods use task-specific predictors, biological priors, or ranking-aware objectives to guide which variants are tested in the next experimental round. However, these methods cannot adapt to shifts in the reliability of predictive evidence as measurements accumulate and ensure the correct ranking of key high-fitness candidates. To address these challenges, we propose Batch-Aligned Tail Arbitration (BATA), which uses experimental feedback to adaptively combine prior-informed and task-specific rankings for next-batch selection, with calibration focused on the batch-aligned high-fitness region. Across measured GB1, PABP, and TrpB landscapes, BATA achieves the best mean task rank (1.67) in final best fitness after 480 measurements. Controlled comparisons further show task-dependent gains from high-fitness calibration and batch alignment. Our work introduces feedback-calibrated predictor arbitration, where experimental feedback dynamically determines how predictive evidence guides next-batch selection, opening a new direction for protein optimization.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
protein optimization, protein engineering, active learning, directed evolution, feedback calibration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_W6QtOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses protein optimization under a limited experimental budget, in the common setting where measurements arrive in sequential batches, and each completed batch is used to train a model that selects the variants to assay next.

The authors' central premise is that a selection model for this setting should combine two components: (i) a prior-informed predictor that encodes knowledge available before any task-specific data is collected, and (ii) a task-specific predictor trained on the experimental measurements gathered so far. Crucially, they argue that the relative influence of these two components should not be fixed in advance but re-estimated at every round.

To this end, they propose Batch-Aligned Tail Arbitration (BATA), a framework that recalibrates a single mixing weight w ∈ [0, 1] to balance the prior-informed and task-specific predictors, updating w at each round as new batch measurements become available.

BATA is evaluated primarily on three datasets—GB1, PABP, and TrpB—with supplementary results also on HIS7 and GRB2. Performance is reported using two metrics, Final@480 and Query-AUC. The headline result is that BATA achieves the best mean Final@480 rank averaged across the three main tasks.

Critical strengths
  1. Well-motivated problem. The paper clearly explains the motivation and establishes the importance of the problem, identifying a specific gap that the proposed method aims to address.

  2. Well-structured related work. The related work places the contribution in the context of existing approaches and clearly explains how the proposed method differs from prior work.

  3. Strong and up-to-date baselines. The evaluation includes recent, competitive methods (ALDE, EVOLVEpro, and REAP100), providing a relevant basis for comparison.

Critical weaknesses
  1. Limited number of evaluation tasks for the headline results. The main results are evaluated on only three protein tasks, and BATA does not win on all three. This is a narrow basis for the paper's central claims. The evaluation would be substantially more convincing with additional targets — the authors already use HIS7 and GRB2 elsewhere (e.g., Figure A6), so extending the headline experiment to those seems feasible. Further targets could be drawn from ProteinGym (Notin et al., 2023) or Li et al. (2025).

  2. The headline results themselves are not clearly convincing. Reading Figure 2: on GB1, BATA ranks third; on PABP, the confidence intervals appear to overlap across methods; on TrpB, BATA wins, but it is not possible to tell from the figure whether the margin over competing methods is statistically significant. As presented, the three tasks do not clearly establish an advantage for the method.

  3. No statistical significance reported for the main results. Table 1 reports mean ± standard deviation, and Figure 2 reports pointwise 95% bootstrap confidence intervals without significance testing, making the comparisons hard to interpret.

  4. Task-specific predictors differ across tasks, with no guidance for new targets. Per Table A1, the predictor setup varies by task: one-hot + ridge + Thompson on GB1, ESM2 + single RF + greedy on PABP, and ESM2 + five forests + Thompson on TrpB. It is unclear how a practitioner with a new protein should choose a setup — the paper does not provide a selection rule or default recommendation. This weakens the method's practical applicability.

  5. (Minor) Inconsistent number of initialization runs, unexplained. The headline results use 35 initializations, while the initialization-sensitivity results in Table A3 use 35 and 70, without explanation for the difference.

  6. (Minor) Reporting only top fitness underuses the measured data. Each batch measures 96 variants, but the paper reports only the top fitness, which characterizes a single variant. Additional metrics for the 96 measured variants per batch (e.g., mean, recall) would provide more information about the method's behavior.

References: Pascal Notin, Aaron W. Kollasch, Daniel Ritter, Lood Van Niekerk, Steffanie Paul, Hansen Spinner, Nathan Rollins, Ada Shaw, Ruben Weitzman, Jonathan Frazer, Mafalda Dias, Dinko Franceschi, Rose Orenbuch, Yarin Gal, and Debora S. Marks. ProteinGym: Large- Scale Benchmarks for Protein Design and Fitness Prediction, December 2023. URL http://biorxiv.org/lookup/doi/10.1101/2023.12.07.570727.

Francesca-Zhoufan Li, Jason Yang, Kadina E. Johnston, Emre Gürsoy, Yisong Yue, and Frances H. Arnold. Evaluation of machine learning-assisted directed evolution across diverse combinatorial landscapes. Cell Systems, 16(9):101387, September 2025. ISSN 24054712. doi: 10.1016/j.cels.2025.101387. URL https://linkinghub.elsevier.com/ retrieve/pii/S2405471225002200.

Questions for authors
  1. Why is the main evaluation limited to three tasks? Can you run the headline experiment on additional targets—e.g., HIS7 and GRB2 (already used in the supplementary material) —and/or additional proteins from ProteinGym or Li et al. (2025) to make the aggregate rank more robust?

  2. Can you report paired significance tests for the main results (Table 1, Figure 2)?

  3. For a new protein with no established dataset, how should a user choose the representation, task-specific model, and selection rule — is there a recommended default?

  4. What accounts for the different number of initializations between the headline results (35) and the initialization-sensitivity results (35 and 70)?

  5. Final@480 and Query-AUC both reduce a batch to its single best variant. Can you report distributional metrics over the 96 variants selected per batch (mean/median fitness, precision@k)?

======== Thanks to the authors for providing their response. However, it does not seem the authors have updated the submission with highlighted changes. From the responses it is hard to recover if there are any updates to the paper. I tried to check it but could not easily find it. In general, it would be great if the authors provide more analysis of the obtained results. At the moment is compact and hard to disentangle what is going on.

The response addresses some of the concerns (hence the score is updated to 3), but the major issues about an overall limited number of targets/landscapes and not fully convincing/conclusive results remain.

3 comments
AuthorsOpenReview ↗

Response to Reviewer W6Qt

Thank you for separating endpoint performance, statistical uncertainty, practical configuration, and batch-level quality. Part 1 of 2 answers C1–C3; part 2 answers C4–C6.

C1 / Q1 — Additional targets. Following your suggestion, we will add HIS7 and GRB2 to the headline evaluation. Both have complete five-method campaigns (24 matched initializations per task, identical initial sets across methods, 480 measurements each). On Final@480, BATA ranks third on HIS7 and second on GRB2. We will report each with its own sample size and provenance beside the Main-35 table in the revised paper. Evaluating further proteins from ProteinGym or Li et al. (2025) requires a fixed protocol of its own.

C2 — Task-level comparisons. Overlapping pointwise bands do not determine the uncertainty of a paired difference, so we report the paired differences directly (Table R1; BATA minus baseline, n = 35 matched initializations; Holm-adjusted p). On GB1, BATA trails ALDE on Final@480 (−0.0835, p = 0.017). On PABP, BATA improves over ALDE, REAP100 and RF (p < .001 each) but not over EVOLVEpro (+0.0494, p = 0.804). On TrpB, none of the four Final@480 comparisons is significant after correction. We will add these differences to Table 1 and Figure 2, and state the aggregate result as a mean rank over the three tasks.

C3 / Q2 — Paired significance tests. Table R1 gives them for all Main-35 comparisons: 20,000-resample paired bootstrap intervals and two-sided sign-flip tests (100,000 draws) on the BATA-minus-baseline difference, with Holm correction over the 12 comparisons for each metric. We will add the table to the paper and the method to Appendix A.4. These are paired analyses of the evaluated tasks and initializations.

Table R1. Paired differences, BATA minus baseline: mean [95% bootstrap CI] and Holm-adjusted p.

TaskBaselineΔ Final@480pΔ Query-AUCp
GB1ALDE−0.0835 [−0.1313, −0.0360]0.017−0.0473 [−0.0732, −0.0210]0.011
GB1EVOLVEpro+0.1169 [+0.0763, +0.1584]<.001+0.0760 [+0.0415, +0.1096]0.001
GB1REAP100−0.0446 [−0.0932, +0.0043]0.528−0.0085 [−0.0379, +0.0205]1.000
GB1RF+0.0219 [−0.0267, +0.0714]0.804+0.0037 [−0.0207, +0.0265]1.000
PABPALDE+0.5986 [+0.4437, +0.7523]<.001+0.2927 [+0.1999, +0.3913]<.001
PABPEVOLVEpro+0.0494 [−0.1423, +0.2433]0.804−0.0593 [−0.1956, +0.0778]1.000
PABPREAP100+0.4209 [+0.2585, +0.5794]<.001+0.1720 [+0.0771, +0.2768]0.010
PABPRF+0.4321 [+0.2994, +0.5667]<.001+0.1339 [+0.0519, +0.2240]0.025
TrpBALDE+0.0251 [−0.0091, +0.0612]0.703+0.0094 [−0.0129, +0.0318]1.000
TrpBEVOLVEpro+0.0227 [−0.0051, +0.0514]0.639+0.0204 [+0.0008, +0.0390]0.236
TrpBREAP100+0.0351 [+0.0001, +0.0718]0.476+0.0357 [+0.0138, +0.0583]0.028
TrpBRF+0.0275 [−0.0094, +0.0670]0.703+0.0335 [+0.0113, +0.0564]0.045

C4 / Q3 — Configuration for a new protein. Table A1 lists the configuration used for each task. We will add to Appendix A.2 the inputs each configuration requires, and the PABP recipe applied unchanged to GRB2 (DCA Ridge5 prior, ESM2-650M random forest with 100 trees, greedy selection, with GRB2-specific features) as a reproducible starting point. Any choice among representations, model sizes, or selectors should rely on labels already available to the practitioner under a fixed validation rule.

C5 / Q4 — Cohorts of 35 and 70. Main-35 combines 20 initialization sets with a 15-set expansion. The 70-initialization bank for GB1 and TrpB is a separate seed list, not an extension of Main-35, and PABP keeps its original 35. We report the cohorts separately; in Figure 3 the lines connect cohort-level summaries, not paired differences. We will label the cohort and sample size in every table and caption, and document both constructions in Appendix A.3 (see also our reply to Reviewer x2Jk, Q2).

C6 / Q5 — Batch-level metrics. Final@480 and Query-AUC describe discovery of the best variant; batch mean, median and top-10 mean (the mean of the ten highest-fitness variants in each batch, which we added as a tail metric following your request for distributional measures) describe the selected batch. We computed them over the 2,100 newly selected batches of the 525 Main-35 campaigns (the shared initial batch excluded; the unit of replication is the campaign). Table R2 gives all five methods and Table R3 the paired top-10 differences. Relative to ALDE, BATA's batch mean and median are comparable on PABP and TrpB and lower on GB1; its top-10 mean is higher on PABP, comparable on TrpB, and lower on GB1. We will add these tables, with the paired differences, to the appendix. We do not report precision@k, because it requires a relevance set defined for each candidate pool.

Table R2. Batch fitness over the four newly selected batches: mean / median / top-10 mean, averaged over 35 campaigns (native fitness units of each task).

MethodGB1PABPTrpB
BATA0.2065 / 0.1728 / 0.54210.9563 / 0.9692 / 1.39670.4011 / 0.4254 / 0.7163
ALDE0.2898 / 0.2867 / 0.61010.9475 / 0.9644 / 1.32420.4035 / 0.4046 / 0.7050
EVOLVEpro-650M0.1910 / 0.1640 / 0.48210.9449 / 0.9628 / 1.38150.3551 / 0.3660 / 0.6837
REAP100-650M0.2880 / 0.2817 / 0.58920.9721 / 0.9793 / 1.40200.4073 / 0.4280 / 0.6918
RF0.2071 / 0.1712 / 0.53020.9148 / 0.9406 / 1.37290.3136 / 0.2957 / 0.6720

Table R3. Top-10 batch mean, BATA minus baseline: mean paired difference [95% bootstrap CI], 35 matched initializations, 20,000 resamples, no multiplicity correction.

BaselineGB1PABPTrpB
ALDE−0.0680 [−0.0854, −0.0501]+0.0725 [+0.0456, +0.0979]+0.0114 [−0.0041, +0.0264]
EVOLVEpro-650M+0.0601 [+0.0338, +0.0881]+0.0152 [−0.0108, +0.0428]+0.0327 [+0.0179, +0.0482]
REAP100-650M−0.0470 [−0.0689, −0.0246]−0.0054 [−0.0253, +0.0138]+0.0245 [+0.0044, +0.0458]
RF+0.0120 [−0.0039, +0.0272]+0.0238 [+0.0023, +0.0455]+0.0443 [+0.0234, +0.0656]
Reviewer_W6QtOpenReview ↗

Major issues about an overall limited number of targets/landscapes and not fully conclusive results remain

I thank the authors for providing the response. However, it does not seem the authors have updated the submission with the highlighted changes. From the responses it is hard to recover if there are any updates to the paper. I tried to check it but could not easily find it.

The response in the discussion addresses some of the concerns (hence the score is updated from strong reject to weak reject), but the major issues about an overall limited number of targets/landscapes and not fully conclusive results remain. It would be also great if the authors provide more analysis of the obtained results.

Reviewer_x2JkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Batch-Aligned Tail Arbitration (BATA), a method for sequential protein optimization that adaptively combines a DCA-based predictor with a task-specific predictor. It learns a mixture weight from out-of-fold predictions by minimizing squared percentile-rank error over high-fitness measured variants, using logarithmic discounting and a cutoff equal to the next batch size. Across three measured protein landscapes, BATA achieves the best average final-fitness rank in the main evaluation, although its advantages depend on the task, initialization cohort, and predictor configuration. Additional experiments examine calibration objectives, predictor capacity, and switching strategies.

Critical strengths
  • A simple, interpretable approach to a meaningful optimization problem. BATA addresses the practical possibility that complementary predictors change in usefulness as measurements accumulate. Its single mixture parameter, rank normalization, and closed-form calibration make the method straightforward to inspect.
  • Useful controlled experiments and transparent reporting of limitations. The paper uses matched initializations and paired comparisons, and reports unfavorable results rather than presenting uniform superiority. In particular, the same-capacity TrpB comparison supports an improvement over equal-weight mixing, while the stronger task-only and switching experiments clarify limitations of the proposed mechanism.
Critical weaknesses
  • The paper should clarify what is new about its predictor-combination method relative to existing approaches. In Sections 4.1–4.3, BATA uses out-of-fold predictions to learn a weight (w) that combines the prior-informed and task-specific predictors, minimizing the weighted squared rank error in Eq. 5. This resembles stacking, an established method that learns how to combine predictors using held-out predictions. Ranking-weighted Gaussian process ensembles also use ranking performance on accumulated observations to adjust model contributions during optimization. BATA’s distinctive choices appear to be calibrating on the highest-fitness measured variants, applying logarithmic discount weights, and setting the calibration cutoff to the next batch size, (k_t=\min(B,n_t)). These choices may be useful, but the paper should distinguish their contribution from the broader, established idea of learning predictor weights from feedback.
  • The experiments do not yet clearly show how much BATA’s main design choices contribute to its performance. In Section 5.5, comparing calibration cutoffs of 96 and 192 while keeping the experimental batch size at 96 tests whether focusing on fewer high-fitness measured variants helps. However, it does not establish that the cutoff should generally equal the batch size, which is a central design choice of BATA.
Questions for authors
  • Why does BATA combine ensemble predictions differently during calibration and candidate selection? Section 4.1 averages the ensemble members’ predictions before converting them to ranks and learning the mixture weight. Section 4.3 instead ranks each member’s predictions separately, then uses these ranks to select candidates. Averaging predictions before ranking can produce a different ordering from ranking each member separately. Please explain why a weight learned using the first procedure is appropriate for the second, and whether this difference materially affects the selected candidates or optimization results.
  • How were the initial measured variants and the two evaluation cohorts chosen? Sections 5.1 and 5.3 describe Main-35 and a separate sensitivity evaluation with 70 initializations for GB1 and TrpB. Were the initial 96 variants sampled uniformly from each candidate pool, or selected using restrictions such as fitness or distance from a reference sequence? Were both cohorts generated using the same sampling procedure, and how was Main-35 designated as the primary evaluation?
3 comments
AuthorsOpenReview ↗

Response to Reviewer x2Jk

Thank you for the detailed questions on BATA's novelty, design choices, and evaluation protocol. Part 1 of 2 answers C1, C2 and Q2; part 2 answers Q1 and gives the new aggregation comparison.

C1 — Relation to stacking and ranking-weighted ensembles. Learning predictor weights from held-out feedback is an established idea, and BATA builds on it. The three choices you identify (calibrating on the highest-fitness measured variants, applying logarithmic discount weights, and setting the cutoff to the next batch size, k_t = min(B, n_t)) are the contribution we claim. Together they define the loss in Eq. (5), whose weight has the closed form in Eq. (6) and is refitted after each batch. Section 5.5 isolates these choices with the rest of the pipeline fixed: G-Rank uses all measured variants, T-Uniform-96 removes the DCG discount, and T-DCG-192 changes the cutoff. T-DCG-96 improves Final@480 over G-Rank by 0.0256 on TrpB and over T-Uniform-96 by 0.1719 on PABP. We will state this contribution, and its relationship to stacking and ranking-weighted Gaussian process ensembles, in the abstract, the contribution statements, and related work.

C2 — Cutoff and batch size. The comparison in Section 5.5 fixes B = 96 and varies the cutoff between 96 and 192, so it evaluates tail emphasis at this batch size. On TrpB, T-DCG-96 improves Final@480 over T-DCG-192 by 0.0115; on GB1 the difference is −0.0006. How the best cutoff depends on B lies outside the tested range, and Section 5.5 will state this scope.

Q2 — Initialization sets and cohorts. (a) Each Main-35 set contains 96 variants drawn uniformly without replacement from the task's candidate pool, without access to fitness labels and without a fitness or distance criterion; exact duplicate sets were excluded, and all methods share the same sets within each task. (b) The two cohorts were not generated by the same procedure. The 70-initialization bank for GB1 and TrpB is an official seed list with documented duplicate repair, and it shares no initial set with Main-35. (c) Main-35 combines 20 initialization sets with a 15-set expansion and covers all three tasks; the 70-initialization bank covers GB1 and TrpB, and PABP retains its 35 runs. We report the two cohorts separately and do not pool them. We will document both constructions in Appendix A.3.

Q1 — Aggregation in calibration and selection. The two steps take different inputs. Calibration (Section 4.1) fits one scalar weight from one prior-informed and one task-specific rank vector (Eq. 3), so the five-fold OOF predictions of each predictor's ensemble are averaged before ranking. Selection (Section 4.3) ranks each refitted member over the unmeasured pool, because Thompson-style selection on GB1 and TrpB samples one member for each choice, and PABP selects greedily by mean combined score. As you note, ranking the averaged prediction and ranking each member can order variants differently, so the weight is learned on the first ranks and applied to the second (Eq. 7).

The archived campaigns retain only member-averaged OOF predictions, so we ran the alternatives as new controlled comparisons: calibration on member percentile ranks averaged without re-ranking (arm 2) and with re-ranking (arm 2′), with folds, seeds, selection rule and budget unchanged, on 24 matched initializations per task (72 new campaigns per arm; the original-method arm reuses verified existing runs). Table R4 gives all 18 pre-specified paired comparisons (three contrasts, three tasks, two metrics; 20,000-resample bootstrap, no multiplicity correction). Every 95% interval includes zero, so these runs do not show a reliable difference between the aggregation orders. We will add the comparison to the appendix, describe both steps and the reason for each in Sections 4.1 and 4.3, and state the OOF predictions as held-out predictions of each variant.

Table R4. Paired differences in the aggregation comparison (24 matched initializations per task): mean [95% bootstrap CI].

TaskContrastΔ Final@480Δ Query-AUC
GB1original − arm 2−0.0258 [−0.0684, +0.0097]−0.0041 [−0.0168, +0.0079]
GB1original − arm 2′+0.0019 [−0.0334, +0.0432]+0.0016 [−0.0104, +0.0139]
GB1arm 2 − arm 2′+0.0277 [−0.0071, +0.0654]+0.0057 [−0.0058, +0.0174]
PABPoriginal − arm 2+0.0252 [−0.1483, +0.1951]−0.0193 [−0.0970, +0.0601]
PABPoriginal − arm 2′−0.0116 [−0.1349, +0.1064]+0.0037 [−0.0422, +0.0438]
PABParm 2 − arm 2′−0.0367 [−0.2389, +0.1672]+0.0230 [−0.0565, +0.1035]
TrpBoriginal − arm 2+0.0022 [−0.0101, +0.0138]−0.0054 [−0.0136, +0.0019]
TrpBoriginal − arm 2′+0.0073 [−0.0067, +0.0229]+0.0030 [−0.0040, +0.0108]
TrpBarm 2 − arm 2′+0.0052 [−0.0073, +0.0183]+0.0084 [−0.0015, +0.0187]

I thank the authors for their clarifications and additional experiments. I maintain my score of 2 (weak rejection) because the evidence for the batch-alignment component remains limited. Comparing calibration cutoffs of 96 and 192 while fixing the batch size at 96 cannot distinguish whether the benefit comes from using fewer high-fitness observations or specifically from matching the cutoff to the batch size. Although the authors acknowledge this limitation, batch alignment remains a central part of the contribution that distinguishes BATA from existing predictor-combination approaches. The current evidence does not yet sufficiently establish this component’s added value to change my recommendation.

Reviewer_rcs8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Batch-Aligned Tail Arbitration (BATA), a method for protein optimization under limited experimental budgets that adaptively combines an evolutionary prior-informed predictor with a task-specific fitness predictor to select promising sequences from a candidate pool. At each experimental round, BATA uses out-of-fold predictions to learn a mixture of the two predictors by minimizing a weighted ranking loss over the highest-fitness measured sequences. The loss places greater emphasis on top-performing variants and uses a cutoff matched to the next experimental batch size. The learned mixture is then used to rerank the remaining candidate pool for experimental selection. Experiments across three measured protein fitness landscapes (GB1, PABP, and TrpB) show that BATA achieves the best average task ranking in final fitness compared with several established baselines, although performance varies across tasks and initialization cohorts. Ablations suggest that focusing calibration on high-fitness candidates can improve optimization, while comparisons with task-specific-only predictors indicate that the benefits of adaptive combination are less conclusive and depend on the underlying predictor configuration.

Critical strengths

Simple and practical approach to combining complementary predictors. The paper proposes a straightforward method for combining evolutionary prior-informed and task-specific predictions during sequential protein optimization. The approach is computationally simple and can be applied to different underlying predictors without modifying their architectures. Although the methodology builds on established ensemble-learning techniques, it provides a potentially useful heuristic for incorporating multiple sources of predictive information into experimental batch selection.

Competitive empirical results with useful ablations. BATA achieves the best average task ranking in final fitness across three protein landscapes, although its performance varies across tasks and initialization cohorts. The authors conduct meaningful controlled comparisons of different calibration objectives, including global ranking losses, uniform weighting of high-fitness candidates, and different tail-weighting cutoffs. They also examine how predictor weights evolve with experimental feedback and compare against fixed-weight combinations and task-specific-only predictors. These experiments help characterize the behavior of the method and provide some evidence for the value of emphasizing high-fitness candidates during calibration.

Critical weaknesses

Incremental methodology with limited new insight. The proposed method largely combines established techniques from ensemble learning and learning to rank. Learning predictor mixtures from out-of-fold predictions is closely related to classical stacking, while emphasizing high-ranked candidates through discounted weighting is also well established. Moreover, prior work has investigated combining evolutionary and assay-supervised information for protein fitness prediction (e.g., Hsu et al., Learning Protein Fitness Models from Evolutionary and Assay-Labeled Data, Nature Biotechnology, 2022, which is not cited). BATA's most distinctive element appears to be its particular tail-weighted calibration objective, especially matching the cutoff to the experimental batch size. However, the ablations do not establish why this alignment is important beyond simply emphasizing high-fitness examples. Furthermore, the controlled comparisons in Appendix C show that its advantage over the task-specific predictor alone is not statistically conclusive.

The evaluation is also restricted to reranking fixed candidate pools, whereas recent approaches such as SGPO (Yang et al., NeurIPS 2025), VLGPO (Bogensperger et al., ICML 2025), and LatProtRL (Lee et al., ICML 2024) explore generative optimization beyond predefined libraries. While fixed-pool optimization is a legitimate setting, its narrower scope places greater importance on establishing the specific benefits of the proposed calibration mechanism. Given the substantial prior work on these ingredients, I would be interested in more targeted analyses that isolate the contribution of batch-aligned calibration and clarify when and why adaptive arbitration improves optimization.

Questions for authors

My questions are in the weaknesses. I am open to raising my score depending on the reviewers' response.

1 comment
AuthorsOpenReview ↗

Response to Reviewer rcs8

Thank you for your comments on BATA's contribution, calibration design, and empirical scope.

W1 — Contribution and related work. OOF ensemble weighting, the combination of evolutionary and assay information, and ranking discounts all have precedents. BATA's contribution is the high-fitness rank-calibration rule in Eqs. (3)–(6): it fits a convex mixture of five-fold OOF percentile-rank predictions by minimizing discounted squared rank error on the measured high-fitness region, and refits the mixture after each batch to guide the next selection. Following your suggestion, we will cite Hsu et al., who add an evolutionary density feature to a ridge-regression model trained on assay-labeled data, and relate BATA's rule to stacking and ranking-weighted ensembles in the abstract, contribution statements, and related work.

W2 — Batch-aligned calibration. Our experiment evaluates the calibration objective at B=96, so the comparison covers the tested cutoffs at this batch size. Across 24 matched campaigns per task, T-DCG-96 improves mean Final@480 over G-Rank by 0.0256 and over T-DCG-192 by 0.0115 on TrpB, and over T-Uniform-96 by 0.1719 on PABP. On GB1, its mean difference from T-DCG-192 is −0.0006. These results show task-dependent effects of the tested tail objectives and cutoffs, and Section 5.5 will state this scope.

W3 — Comparison with the task-specific predictor. On TrpB, BATA's mean Final@480 gain is 0.0345306 over the same-capacity task-specific predictor and 0.020349 over equal-weight rank combination, with intervals reported in Appendix C.2. The first control addresses the value of adding prior-informed predictions, and the second the value of learning the mixture. Section 5.6 and Appendix C.2 will present the two conclusions separately.

W4 — Scope and mechanism. BATA addresses sequential selection from finite candidate libraries under a 480-measurement budget. Within this setting, the controlled comparisons indicate where calibration helps: learned weighting improves on equal-weight combination on TrpB, and DCG-weighted calibration at the batch-sized cutoff gives the best aggregate rank at B=96 (1.33 for both Final@480 and Query-AUC). The learned weight calibrates a fixed pair of predictors: in an independent test on ten new initializations per task, a weight-threshold switching rule did not improve on BATA (mean rank 2.20 versus 1.40; Appendix C.3, Table A5). We will clarify the scope relative to generative optimization and organize the mechanistic discussion around these controlled comparisons in Sections 2 and 6.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their detailed comments. Several points were raised by more than one reviewer, and we address them here; reviewer-specific questions are answered in the individual replies.

Contribution. Reviewers rcs8 and x2Jk asked how BATA differs from stacking, ranking-weighted ensembles, and prior work combining evolutionary and assay information (Hsu et al.). Learning predictor weights from held-out feedback, combining evolutionary and assay information, and discounting by rank each have precedents. BATA's contribution is a specific calibration rule for sequential selection under a limited measurement budget. It fits a convex mixture of five-fold OOF percentile-rank predictions by minimizing DCG-discounted squared rank error on the measured high-fitness region, with the cutoff set to the next batch size, and refits the mixture after each batch (Eqs. 3–6).

Evidence. (1) Scope: BATA addresses sequential selection from finite candidate libraries under a 480-measurement budget. Section 5.5 isolates the three design choices with the rest of the pipeline fixed; it is run at B = 96, so it evaluates the tested objectives and cutoffs at this batch size, and the effects are task-dependent. (2) Design consistency: calibration averages member predictions before ranking, while selection ranks each member. In a controlled comparison of two alternative aggregation orders on 24 matched initializations per task, all 18 paired 95% intervals include zero; the intervals are wide, so the comparison does not establish equivalence (see our reply to Reviewer x2Jk, Q1). (3) Statistics: we report paired bootstrap intervals, sign-flip tests and Holm-adjusted p-values for all Main-35 comparisons (n = 35 matched initializations per task); the task-level results are in our reply to Reviewer W6Qt. On TrpB, the same-capacity task-specific and equal-weight controls (Appendix C.2) separate the value of prior-informed predictions from the value of learning the mixture. (4) Tasks and cohorts: HIS7 and GRB2 have complete five-method campaigns (24 matched initializations each), and we report them with their own provenance. Main-35 (20 sets plus a 15-set expansion) and the 70-initialization bank for GB1 and TrpB are separate cohorts, reported separately. (5) Batch-level quality: we computed batch mean, median and top-10 mean over all newly selected batches, which complement the best-variant endpoints.

Revision. We will cite Hsu et al. and relate BATA's rule to stacking and ranking-weighted ensembles in the abstract, contribution statements and related work; state the scope of the cutoff comparison in Section 5.5; describe the calibration and selection aggregation in Sections 4.1 and 4.3 and add the aggregation comparison to the appendix; add paired differences to Table 1 and the methods to Appendix A.4; add HIS7 and GRB2 beside the Main-35 table; and document the initialization cohorts in Appendix A.3, the configuration recipe in Appendix A.2, and batch-level metrics in the appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.