- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes Factorized LLM Search (FLS), a blackbox framework for protein sequence optimization that explicitly separates evaluation allocation from mutation generation. An explicit controller allocates evaluation resources across promising measured variants and feasible local edit components, while a frozen LLM proposes specific amino-acid substitutions. Experiments show the effectiveness of FLS compared with related baselines. Additional ablations and trajectory analyses provide some insights into the respective roles of LLM-based mutation generation and explicit evaluation allocation.
Critical strengths
-
This work is well-motivated and clearly-presented. The paper identifies the conflation of evaluation allocation and mutation generation as a potential limitation of direct LLM-based optimization. Separating these decisions provides a reasonable framework for enhancing the LLM-guided search.
-
The proposed FLS has good interpretability. By introducing an explicit controller while keeping the LLM frozen, FLS makes the allocation process more transparent and facilitates the analysis of search behavior.
-
The empirical evaluation verifies the effectiveness of FLS in protein sequence optimization.
Critical weaknesses
-
The novelty and main insight are not clearly presented. The paper does not sufficiently clarify what new algorithmic mechanisms or theoretical insights the proposed factorization introduces compared with conventional evolutionary search frameworks. The authors should more explicitly identify the fundamental differences and explain how the factorization goes beyond merely reorganizing existing components.
-
The evaluation and analysis are not sufficient. The paper lacks comprehensive evaluations of the individual controller components.
Questions for authors
Please refer to the weaknesses pointed above.
2 comments
Response to Reviewer cvy3 [2/2]
W2. How much do the individual controller and search components contribute to FLS's performance?
To assess the full set of design choices, we compare complete FLS with variants of its controller and candidate-generation components under matched initialization and evaluation budgets. FLS achieves higher mean endpoint fitness scores on at least three of the four tasks in each component comparison. The following controls specify what changes in each variant:
- Position control: let the LLM choose the source while the program retains parent, edit-action, and chemistry-class control.
- LLM proposals: replace local LLM proposals with random legal edits within the assigned tasks.
- Parent selection: replace fitness-based selection with uniform selection from measured variants.
- Chemistry scheduling: let the LLM choose the chemistry class instead of following the programmed schedule.
- Recombination: replace the archive-recombination candidate with an additional local LLM proposal, preserving the evaluation budget.
Table W2-1 reports the best observed fitness score after 500 evaluations, as mean ± sample SD over ten runs per method and task.
Table W2-1. Component comparisons.
| Method | avGFP | Ube4b | GB1 | Bgl3 |
|---|---|---|---|---|
| Complete FLS | 3.990 ± 0.013 | 6.938 ± 0.400 | 3.471 ± 0.000 | 2.110 ± 0.117 |
| Without position control | 3.964 ± 0.014 | 6.436 ± 1.024 | 2.744 ± 0.025 | 1.885 ± 0.127 |
| Random local proposals | 3.992 ± 0.016 | 5.701 ± 1.016 | 3.258 ± 0.371 | 1.873 ± 0.129 |
| Uniform parent selection | 3.955 ± 0.012 | 6.427 ± 1.005 | 2.771 ± 0.245 | 2.198 ± 0.180 |
| LLM-chosen chemistry class | 3.985 ± 0.014 | 6.691 ± 0.468 | 2.814 ± 0.457 | 2.210 ± 0.192 |
| Without recombination | 3.988 ± 0.023 | 5.460 ± 1.432 | 3.137 ± 0.543 | 1.095 ± 0.465 |
The component effects vary across landscapes: the largest position-control gain occurs on GB1, while LLM proposals and recombination provide their clearest mean endpoint fitness gains on Ube4b and Bgl3. Recombination leverages the mutation pool built up during FLS search, combining previously measured mutations into new candidates. Its effectiveness highlights the value of the mutation options discovered by the other FLS components. Together with the remaining controller comparisons, these results show contributions from multiple FLS components, with their relative impact depending on the landscape.
Thank you again for your thoughtful comments!
Response to Reviewer cvy3 [1/2]
Thank you for prompting a more explicit explanation of the factorization insight and the contributions of its components. Following your comments, we provide:
Thank you for your suggestion to clarify FLS’s contribution relative to conventional evolutionary search. Our contribution is to identify a mismatch between fitness variation across position sets and LLM search behavior, and to demonstrate the benefit of separating edit-task allocation from candidate generation. Below, we explain this insight and its implications for positional exploration and endpoint fitness.
Fitness variation across position sets motivates separate allocation
Our central insight is a mismatch between fitness variation across mutation-position sets and the search-budget allocation implicit in LLM proposals. When the LLM jointly proposes mutation positions and residue identities, the allocation of evaluations across positions is implicit in its proposal distribution.
To quantify fitness variation across position sets, we group measured variants by their exact sets of positions mutated relative to the wild type, regardless of residue identities, and retain sets evaluated at least twice. The share of total fitness variance explained by differences between group mean fitness scores is 54–83%. We also measure position-set reuse: the fraction of evaluated candidates whose exact mutated-position set appeared earlier in the same run. For Direct LLM Sampling, 70–91% of candidates reuse such sets while varying residue identities. Both ranges report the minimum and maximum across the four tasks.
To reduce this mismatch, FLS assigns edit tasks programmatically before LLM candidate generation. The analyses below show that FLS explores more positions and achieves higher mean endpoint fitness scores than Direct LLM Sampling.
Relation to conventional evolutionary search. Fitness-based selection determines which individuals survive. In LLM-guided evolutionary methods such as the evaluated MOLLEO baseline, the LLM still chooses where to edit when proposing offspring. FLS makes within-parent task allocation explicit before asking the LLM for a concrete mutation.
Evaluation metrics.
Results. FLS achieves broader positional exploration and higher mean endpoint fitness scores than Direct LLM Sampling on all four tasks. Table W1-1 reports Direct LLM Sampling → FLS after 500 evaluations. Positions visited and reference-position coverage summarize the trajectory analysis over ten runs per method. Endpoint fitness values reproduce the original paper's comparison as mean ± sample SD.
Table W1-1. Position exploration and fitness.
Together, the analyses support factorized search as a way to reduce this mismatch: FLS visits more positions and achieves higher mean endpoint fitness scores on all four evaluated tasks.