- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies protein design when only a limited number of experiments can be run and candidate sequences must satisfy known requirements. The authors propose EvoGFN, a GFlowNet that starts from an existing sequence and changes it one position at a time. Each change is checked against the requirements, so candidates that fail them are not generated. A fitness predictor trained on available observations guides candidate generation. The evaluation uses synthetic tasks, existing GB1 and TrpB measurements, and GB1 with an added predicted stability requirement. The authors also test untrained generators, larger candidate pools, and other methods with the same feasibility checks to understand which parts of the approach produce the gains.
Critical strengths
- Gains are obvious and cannot be explained simply by baselines producing more infeasible candidates. Under the mild stability requirement in Table 2, EvoGFN achieves a normalized best fitness of 0.795, compared with 0.690 for PEX and 0.685 for PEX-ensemble, despite their feasibility rates already being 99.6% and 99.9%. EvoGFN also outperforms masked-ALDE and masked-MLDE, which both achieve 100% feasibility. This is meaningful evidence for the effectiveness of the complete workflow.
- The analysis of the method’s limitations is useful and includes targeted controls. The paper goes beyond reporting best fitness: it explains how local checks can exclude valid final sequences and studies completion-aware alternatives. The untrained-generator, larger-pool, and exact-decoding controls also show an effort to understand where the gains come from rather than only presenting favorable rankings. These analyses help establish the method’s scope.
Critical weaknesses
- The additional benefit of GFlowNet training is not sufficiently isolated, leaving the main methodological contribution unclear. The mild-gate results in Table 2 establish an advantage for the complete workflow, but do not directly show whether the gain comes from learning the generator, the candidate distribution, the number of candidates, or the prediction and selection steps. The masked-ALDE and masked-MLDE baselines control feasibility, but they do not differ from EvoGFN only in whether its generator is trained.
- Table 3 provides a more direct trained-versus-untrained comparison, but its GB1 and TrpB tasks have no stability constraint and therefore do not directly explain Table 2. It seems that predictor corruption affects both generator training and candidate screening, and the experiment cannot separate the two effects.
- The budget accounting in some main comparisons may overstate the practical assay-efficiency advantage. EvoGFN’s feasibility checks are treated as computation, whereas infeasible candidates from several baselines consume assay slots without returning fitness labels. Since the rule is known in advance, it is unclear why these methods cannot discard invalid candidates computationally before spending an assay. This has substantial consequences: ordinary MLDE never obtains enough usable data to start training on the three main constrained tasks.
Questions for authors
- Can every method apply the same feasibility check before an assay? Do the main advantages in Figure 1 remain when every method can prefilter invalid candidates, with computational costs recorded separately?
- How do EvoGFN, PEX-ensemble, and the two masked baselines in Table 2 differ in candidate generation, prediction models, candidate counts, and final selection, and how these differences support the central methodological claim?
Summary of the revision for the Area Chair and reviewers
Dear Area Chair and reviewers,
Thank you for the comments, which helped clarify the framing and strengthen the evaluation. The revision tests whether incorporating a supplied feasibility rule into a learned generator improves the fitness reached under a fixed assay budget when competing methods can apply the same check before an assay. Computational screening and assay use are counted separately. The main findings are as follows.
Meaning of the biological constraint. In GB1, fitness is measured binding, while feasibility means passing a separately supplied predicted-stability threshold. A variant can bind well and fail that threshold. This setting isolates how a generator uses an available acceptance rule. Without measured stability labels across the library, we use a simulated, ThermoMPNN-based gate and explicitly distinguish compliance with it from evidence that a protein folds or expresses. Corrupting the mild gate with false rejections, false acceptances, or both leaves EvoGFN's normalised best between 0.79 and 0.82, against 0.795 with the reference gate (Section 3.3); this tests sensitivity to those perturbations, without validating the gate's biological accuracy.
Discussion and scope. The prefiltered comparisons address the concern that gains arise solely from charging assays for computationally identifiable failures. They do not settle how much further the filtered baselines could improve with more search, or with q6p5's proposed combination of zero-label training and free rejection, which remains untested. Ef6q noted on 15 November that the additional experiments largely address the original concerns and asked what drives the difference from PEX-ensemble. Appendix F.4 now describes that workflow, and our closing reply distinguishes the shared predictor architecture from the generation and selection differences that remain. We did not receive a reply from wms8 to our 9 November comment and have summarised the completed additions in a closing reply.
The abstract and introduction have been rewritten, the broad "leads every published pipeline" statement removed, and the campaign steps and appendix explanations clarified. We hope the revised comparisons and controls help the committee assess the contribution at the scope supported by the results.
Thank you for your time.