acceptodds
Under review as a conference paper at ICLR 2027

RosettaSearch: Multi-Objective Inference-Time Search for Protein Sequence Design

Abstract

Sequences produced by state-of-the-art backbone-conditioned design models such as LigandMPNN often fail to fold into the target structure when checked with an independent structure predictor. We introduce RosettaSearch, an inference-time, multi-objective search that repairs such sequences without any model retraining. RosettaSearch uses an LLM as a generative optimizer inside a priority-based parallel search. At each step, the LLM proposes targeted edits after reading structured feedback from RosettaFold3: global rewards (pLDDT, TM-score, C-RMSD), residue-level annotations of problem regions, and text constraints that discourage reward hacking. On 400 LigandMPNN designs for native PDB monomers, RosettaSearch improves fidelity metrics by 18% to 68% and raises the design success rate 2.5 (7.9% to 20.5%). The gains persist under two independent predictors not used in the search (Chai-1 and Boltz-2), and a random-mutation baseline with identical feedback and oracle budget shows no improvement. On 275 de novo Dayhoff backbones, RosettaSearch raises the success rate of ProteinMPNN designs from 72% to 90%, and it rescues sequences for difficult to design backbones on which ProteinMPNN fails. RosettaSearch can also optimize a physics-based Rosetta energy term, and a small policy fine-tuned with a MultiMax reinforcement-learning reward improves the search's success rate. In a preliminary wet-lab test, we find that a designed sequence for a de novo backbone shows a single cooperative thermal unfolding transition at 89 C. To our knowledge, this is the first large-scale demonstration that LLMs can serve as effective generative optimizers for backbone-conditioned protein sequence design.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
multi-objective optimization, protein design, backbone conditioned sequence design

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_MriPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents RosettaSearch, an inference-time protein sequence optimization framework that uses an LLM to iteratively revise sequences based on structural feedback from RosettaFold3. The search combines global fidelity metrics, residue-level error information, and multi-objective priority search, and does not require retraining the underlying protein design model. The method is evaluated on LigandMPNN and ProteinMPNN designs across PDB monomers and Dayhoff backbones, with additional checks using Chai-1 and Boltz-2, random-mutation baselines, a fine-tuned open-weight LLM, and a small wet-lab stability experiment.

Critical strengths
  1. Using a general-purpose LLM as an inference-time optimizer for protein sequence design is an interesting direction. Rather than replacing ProteinMPNN or LigandMPNN, RosettaSearch treats their outputs as starting points and lets the LLM iteratively revise sequences using structural feedback. This makes the method flexible to new objectives without retraining a task-specific protein model.

  2. The paper does more than show gains under the same search oracle. Improvements are also checked with Chai-1 and Boltz-2, and the method is tested on both PDB monomers and de novo Dayhoff backbones. These additional evaluations make the results more convincing than if they relied only on RosettaFold3 feedback.

Critical weaknesses
  1. The evaluation does not show whether the LLM-based optimizer is competitive with specialized iterative protein-design or sequence-refinement methods. RosettaSearch is mainly compared with the initial ProteinMPNN/LigandMPNN designs and random or BLOSUM-based mutation baselines. Since the main claim is that an LLM can serve as a generative optimizer, comparisons with dedicated iterative refinement methods would be important to establish whether the LLM route offers a real advantage beyond simply allowing repeated oracle-guided updates. This is especially relevant given that RosettaSearch uses many expensive structure-prediction calls per protein.

  2. It is unclear how much of the improvement comes from protein-specific reasoning by the LLM rather than sequence priors, memorization, and black-box search with a strong structure oracle. In the PDB experiments, the native sequence is explicitly provided in the LLM context, and the authors note that the model can simply return the native sequence during optimization. The Dayhoff experiment without a reference sequence is a useful control, but it does not rule out pretraining overlap or homolog memorization. A stronger contamination or homology analysis, together with results on the PDB setting without native-sequence context, would make the claimed role of LLM reasoning much more convincing.

Questions for authors
  1. In the PDB monomer experiments, the native sequence is included in the LLM context, and the paper notes that the model may return the native sequence during optimization. How much of the reported gain remains if the native sequence is removed from the prompt? This would materially affect how I interpret the role of the LLM as an optimizer.

  2. Was any analysis performed to estimate possible pretraining overlap or sequence homology between the evaluated proteins and the LLM’s training data? The Dayhoff setting removes the explicit reference sequence, but it does not by itself rule out memorization of related proteins or sequence families.

  3. Why are specialized iterative protein-design or sequence-refinement methods not included as baselines? Since RosettaSearch uses repeated structure-oracle calls to revise sequences, comparison against dedicated iterative refiners under a matched oracle or compute budget would be important for assessing whether the LLM-based route provides a meaningful advantage.

1 comment
AuthorsOpenReview ↗

There is no native sequence in context

We sincerely apologize for editing issues that caused some of the text from an older version of the manuscript to remain. All our current results stated in the paper DO NOT provide the native sequence in context to the LLM. The earlier versions of this work provided additional context information to aid generation, which we found to cause some reward hacking.

Reviewer_R4KoOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RosettaSearch, an inference-time multi-objective optimization framework for backbone-conditioned protein sequence design. The proposed method employs large language models as generative optimizers to iteratively refine sequences produced by existing design models, such as ProteinMPNN and LigandMPNN, using structural feedback from RosettaFold3. Specifically, it combines global structural metrics (pLDDT, TM-score, and Cα-RMSD), residue-level feedback, and constraint-aware guidance within a priority-based parallel search framework. The authors further explore multi-objective reinforcement learning through a MultiMax reward to improve the sequence-editing policy. Experiments on approximately 400 PDB monomer designs demonstrate that RosettaSearch improves design success rates from 7.9% to 20.5%, with additional improvements confirmed by independent structure predictors, including Chai-1 and Boltz-2. On 275 de novo Dayhoff backbones, the method increases the success rate of ProteinMPNN designs from 72.4% to 89.5%. Preliminary experimental validation also reports a cooperative thermal unfolding transition for one designed protein. Overall, the paper demonstrates the potential of LLM-guided, feedback-driven search for improving protein sequence design without retraining the underlying design models.

Critical strengths
  1. Important and practical research problem. The paper addresses a key limitation of existing protein inverse folding models by introducing inference-time optimization with structural feedback, enabling iterative improvement of generated protein sequences without retraining the underlying design models.

  2. Well-motivated multi-objective optimization framework. RosettaSearch combines global structural metrics, residue-level feedback, and priority-based parallel search to guide sequence refinement. The integration of multiple objectives and localized feedback provides a reasonable approach to improving structural fidelity.

  3. Extensive experimental evaluation. The authors demonstrate substantial improvements on PDB monomers and de novo Dayhoff backbones, with additional validation using independent structure predictors, including Chai-1 and Boltz-2. The paper also includes search baselines, reinforcement learning experiments, and preliminary wet-lab validation.

Critical weaknesses
  1. Insufficient comparison with closely related work. ProtInvTree (NeurIPS 2025) has already explored search-based optimization for protein inverse folding using MCTS and structural feedback. The authors should discuss this closely related work and clarify the incremental contributions of RosettaSearch, particularly regarding its search strategy, feedback mechanism, and optimization effectiveness. A direct comparison under matched evaluation settings and computational budgets would substantially strengthen the novelty claims.

  2. Lack of strong non-LLM search baselines. The current experiments mainly compare against random and BLOSUM62-weighted mutations, which are relatively weak optimization baselines. Comparisons with MCTS, evolutionary search, or protein-model-guided optimization would help establish whether LLM-based sequence editing provides additional benefits beyond feedback-driven search itself.

  3. High computational cost. RosettaSearch requires repeated structure predictions, typically involving approximately 75 RosettaFold3 evaluations per protein. More comprehensive performance–compute comparisons are needed to establish its efficiency relative to alternative search strategies and increased sampling from existing design models.

  4. Limited experimental validation. The wet-lab evaluation is restricted to ten designs for a single de novo backbone, with one showing a clear cooperative thermal unfolding transition. This provides preliminary evidence of stability but does not establish that the designed sequences adopt their intended structures. More extensive experimental validation, including comparisons with the initial designs, would strengthen the practical conclusions.

  5. Insufficient justification of the MultiMax reward. The proposed MultiMax objective rewards the largest improvement among multiple structural metrics, potentially allowing substantial degradation in other objectives. Further comparisons with alternative multi-objective optimization strategies and analysis of the resulting trade-offs would strengthen this component.

Questions for authors

See Weaknesses.

1 comment
AuthorsOpenReview ↗

Comparison to baselines

MCTS RosettaSearch is built on the PrioritySearch algorithm in OpenTrace. It keeps a buffer of candidate sequences and runs a best-first search. Each round selects the top candidates by priority, expands each with several LLM proposals, scores the proposals with RF3, and writes the results back into the buffer. In the paper, priority is the mean reward of a candidate’s rollouts. The implementation also supports a UCB priority, mean + c·√(ln T / n)·(R_max − R_min), clipped to the reward range. This is the same exploration/exploitation rule used in MCTS, applied to a flat buffer instead of a tree. We used mean reward since we find it to be marginally better in our preliminary experiments.

Evolutionary algorithms We evaluated two gradient-free baselines — simulated annealing (SA) and a genetic algorithm (GA) — under the same evaluation budget for RF3 per protein, the same pLDDT / TM-score / Cα-RMSD feedback, the same reward, the same starting sequence (the best LigandMPNN design; no native sequence is used) and evaluation criteria. Only the proposal step differs: candidates come from BLOSUM62-weighted substitutions at the positions the feedback flags with probability ∝exp⁡(BLOSUM62(a,a′)/T)\propto \exp(\text{BLOSUM62}(a,a')/T), T=2.0T=2.0, skipping already-evaluated sequences. SA uses Metropolis acceptance with geometric cooling (T0T_0 = median ∣Δreward∣|\Delta\text{reward}| over 4 calibration proposals, Tend/T0=0.05T_{\text{end}}/T_0=0.05) and restarts from the best sequence after 8 consecutive rejections. GA maintains a population of 8 with 2 elites, size-3 tournament selection, and uniform crossover at probability 0.7 followed by the same mutation operator.

We explore them under the same edit regimes as well: moderate, aggressive and drastic. We observe the following success rates:

MethodStep sizeSuccess-1Success-2
SAmoderate11.0%4.0%
SAaggressive9.0%1.0%
SAdrastic7.0%2.0%
GAmoderate13.0%6.0%
GAaggressive9.0%2.0%
GAdrastic7.0%3.0%

Pooled across step-size regimes results:

MethodSuccess-1Success-2
SA12.0%5.0%
GA13.0%7.0%
Reviewer_eVD6OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RosettaSearch, an LLM-based inference-time method for protein sequence optimization that iteratively modifies existing sequences using structural feedback from RosettaFold3. It combines residue-level feedback, multi-objective rewards, and priority search to improve the structural fidelity of designed sequences without additional training. The method is evaluated on approximately 400 PDB monomers and 275 Dayhoff de novo backbones, with independent validation using Chai-1 and Boltz-2. The results show improvements in pLDDT, TM-score, and design success rate over the initial ProteinMPNN/LigandMPNN designs and random mutation baselines.

Critical strengths
  1. The paper applies LLM-based generative optimization to protein sequence design, providing an inference-time sequence refinement method that uses structural feedback without additional training.

  2. The method shows improvements in structural fidelity and design success rate on both PDB monomers and Dayhoff de novo backbones. The improvements are also partially supported by independent evaluations using Chai-1 and Boltz-2.

Critical weaknesses
  1. The motivation for using a general-purpose LLM for protein sequence optimization is not convincing. The feedback provided to the LLM mainly consists of global structure scores and low-quality residue regions, rather than complex natural language feedback. Identifying problematic regions could be done with simple rules, while a protein language model could propose amino acid substitutions. The paper does not show why an LLM would be better than such a pipeline. Moreover, Appendix A.10–A.13 report frequent errors in LLM reasoning, with only 56% of the evaluated reasoning cases considered valid by Gemini-3.

  2. The baselines are not sufficient. The paper mainly compares against ProteinMPNN, LigandMPNN, random mutations, and BLOSUM62-weighted mutations, but lacks comparisons with stronger folding-guided sequence optimization methods, such as protein hallucination, AlphaDesign, and RL-based methods like ProteinZero. These methods can also optimize protein sequences using structural feedback, so the current experiments do not clearly demonstrate the advantage of LLM-based search over existing approaches.

Questions for authors
  1. Why use a general-purpose LLM instead of a protein-specific model for sequence optimization? Since the structural feedback already identifies problematic residue regions, a simple rule-based selection method combined with a protein language model could also propose mutations. Could the authors compare against such a pipeline using the same RF3 feedback and evaluation budget to show the necessity of the LLM?

  2. Could the authors compare RosettaSearch with stronger folding-guided optimization methods, such as protein hallucination, AlphaDesign, or ProteinZero? It would be helpful to see comparisons under the same initialization, structure prediction oracle, and evaluation budget. Also, while the paper evaluates ESMFold-guided search, the experimental settings are not directly matched with RF3-guided search. Could the authors provide a more controlled comparison between these oracles?

2 comments
AuthorsOpenReview ↗

Comparison to baselines

ProteinZero. We could not include ProteinZero (Wang et al., 2025) as a baseline on our benchmark because of protein length. ProteinZero is an online reinforcement-learning method that fine-tunes an inverse-folding model, and its public release provides two model families. The original version builds on InstructPLM, whose architecture restricts the sequence length it can process. The newer ProteinMPNN-based checkpoints are, according to the authors, intended for backbones of up to 100 residues, and their behavior beyond that range is uncharacterized. Our validation set of 415 monomers has a median length of 258 residues and a maximum of 1,016, and only 36 of the 415 (8.7%) have 100 residues or fewer. Applying the checkpoints outside their stated range would not be a fair comparison, and restricting the evaluation to the 36 short proteins would not support a conclusion about the benchmark as a whole. ProteinZero is also a training-time method, whereas the key results from RosettaSearch involve inference time updates without updates to the model.

AuthorsOpenReview ↗

Protein Language Model (ESM2) instead of LLM

ESM2 as Generator

ESM2 baseline implementation. The ESM2 baseline is an ablation of RosettaSearch in which only the proposal operator is replaced; the search algorithm, the structural evaluator RF3 are the same. Each run starts from the same LigandMPNN sequence and the same priority search (3 exploration candidates expanded per iteration, 2 proposals per candidate, heap memory of 20, mean-score exploration priority, best candidate always retained and re-validated each iteration). Every proposal is folded with RosettaFold3 and scored with the same multi-objective reward over TM-score, pLDDT, and Cα-RMSD (w_TM = 5.0, w_pLDDT = 1.0, w_RMSD = 5.0), under a hard budget of RF3 evaluations per protein (max 75 iterations), with a per-protein deterministic seed. The proposal step works as follows. The same RosettaFold3 feedback text that is given to the LLM (low-pLDDT regions, TM-score, Cα-RMSD) is parsed into a set of flagged residue positions; if no region is flagged, all positions are considered. Exactly min(k, n_flagged) of those positions are selected uniformly at random without replacement, where k is the step size (moderate k = 5, aggressive k = 10, drastic k = 20; under the unconstrained regime the count is drawn uniformly from [1, n_flagged]); ligand-contacting and other immutable positions are excluded. The selected positions are masked and filled by ESM2-650M (facebook/esm2_t33_650M_UR50D) via iterative unmasking: up to 8 forward passes, remaining positions filled in random order, each sampled from the top-10 amino-acid logits at temperature 1.0, conditioned on the residues already committed. The baseline therefore uses the structural feedback only to localize edits and contributes no learned preference over which positions to edit, isolating the contribution of the protein language model's residue prior from that of the LLM's reasoning over the feedback.

Step sizeSuccess-1Success-2
Moderate (k=5)93 (22.4%)38 (9.2%)
Aggressive (k=10)86 (20.7%)35 (8.4%)
Drastic (k=20)12 (9.5%)1 (0.8%)

Success-1: pLDDT ≥ 0.8 and TM-score ≥ 0.8. Success-2: additionally Cα-RMSD ≤ 1.5 Å. Counts are proteins for which any evaluated sequence in the trajectory meets the criterion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.