acceptodds
Under review as a conference paper at ICLR 2027

Trellis: Agent-Orchestrated Design and Evolution of Biological Sequence-Based Property Predictors

Abstract

Predicting biological properties such as expression, immunogenicity, stability, and binding affinity of proteins is a critical step in many drug discovery pipelines. New experimental datasets and protein sequence representations create opportunities to improve these predictors, but updating them requires repeated engineering of features, architectures, and training procedures. To address this, we introduce TRELLIS, an agent-orchestrated framework for developing biological sequence based property predictors. Large language model (LLM) agents write and execute code in isolated environments, and a Monte Carlo Tree Search explores diverse combinations of sequence representations and model architectures. Across FLIP2, PeptiVerse, and major histocompatibility complex (MHC) class I pathway tasks, a multi turn tool calling agent produced predictors with scores above the listed literature references on eight tasks. Tree search improved test performance over the best Qwen3.8-27B baseline on seven of eight selected tasks. We also evaluated sequential updates of an MHC class I elution predictor using Immune Epitope Database (IEDB) release-year holdouts. We release our code, which accepts any labeled sequence-property dataset without task-specific modification at https: //anonymous.4open.science/r/trellis-B12C.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
LLM agents, automated machine learning, biological sequence property prediction, Monte Carlo tree search

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_1YLKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose TRELLIS, an agent-based framework for automatically reviewing biological sequence-based property predictors. The framework uses LLM coding agents to generate and execute model training code, while MCTS explores different combinations of sequence representations and predictor architectures. The authors evaluate TRELLIS on FLIP protein fitness benchmarks, PeptiVerse peptide property prediction tasks, and MHC class I pathway prediction tasks. The single-agent framework achieves scores above the reported literature references on several tasks, while MCTS improves performance over the single-agent approach on 7/8 selected tasks.

Critical strengths

The authors evaluate the framework on a relatively broad range of protein and peptide property prediction tasks, demonstrating its applicability across different datasets and prediction objectives, with improved predictive performance on several tasks.

Critical weaknesses
  1. Limited methodological novelty. I find this paper to be mainly a combination of existing tree search methods, coding agents, and protein foundation models. The resulting predictors mostly rely on existing representations combined with conventional prediction heads. Therefore, I am not fully convinced of the methodological novelty of this work.

  2. Unclear motivation and framework design. I am not entirely convinced by the use of the term "agent-orchestrated" in the title, since a substantial part of the framework relies on MCTS rather than being directly orchestrated by LLMs. I am not entirely convinced of the necessity of such a relatively complex framework. Since most resulting predictors rely on pretrained representations combined with conventional prediction heads, it would be useful to examine whether simpler model selection and hyperparameter tuning strategies could provide comparable improvements. Moreover, the reported performance gains are relatively modest, and it remains unclear how much of the predictive performance comes from the pretrained representations rather than the proposed model development framework. More comparisons with simpler approaches would help justify the additional complexity.

  3. Unclear baseline comparisons. The specific baseline model names corresponding to the "Paper" scores in Table 1 are not clearly provided, making it difficult to assess how competitive the proposed framework is against existing methods.

  4. Writing and presentation. I found some parts of the paper difficult to follow, particularly regarding the nature and positioning of the proposed work. For example, the Related Work section covers several broad research directions but could be more focused on distinguishing this framework from closely related approaches.

Questions for authors
  1. Why is MCTS necessary for this framework? Why not allow the LLM orchestrator to directly decide the next predictor configurations based on previous experimental results? Have the authors compared MCTS with simpler search strategies under the same computational budget?

  2. What are the specific baseline models corresponding to the "Paper" scores in Table 1? Are these results directly comparable under the same data splits and evaluation protocols?

  3. What is the rationale behind the selection of Qwen3.8-27B and GLM-5.3? How would the framework perform with other widely used LLMs, such as GPT, Gemini, or Claude? Would stronger coding models reduce the benefit of the proposed MCTS framework?

2 comments

We thank the reviewer for these constructive comments. Several points overlap with Reviewer 1 so below contains some similarities in our response.

Motivation and methodological novelty.

We agree that trellis builds on existing components. Our contribution is primarily an application framework for biological sequence property prediction. Biological sequence properties often require different featurizations and more frequently need to be retrained as new assay data comes in or include a new protein representation model. We argue that as LLM agents are increasingly used in protein design, a more flexible framework where the agent is not given a rigid set of features to explore to automatically pick the best feature and instead given just access to a code execution sandbox is beneficial. Our primary motivation behind tree search in our work, is to force such exploration of different representations early on. Once an architecture and representation are chosen, hyperparameter tuning can, in principle be performed even deterministically.

Baseline models corresponding to the "Paper" scores

The “Paper” column reports the best published scores from the source benchmarks used in Table 1. primarily FLIP2 and PeptiVerse and MUNIS for elution. Those papers contain human-designed ml models with different protein language models or classifical sequence features with conventional heads, rather than a single shared architecture. We evaluate trellis on the same train/test splits and report same metrics. In the revision, we can clarify the name of the specific reference model next to each Paper entry

Model Selection Choice

We selected Qwen3.8-27B and GLM-5.3 as moderately sized, reproducible open models suited to a setting where predictors may need repeated retraining as new assay data arrive, under practical compute budgets. The framework is llm model-agnostic: the framework can use other LLMs easily. We are providing comparison with Claude and also BioAutoML approach below.

DatasetRef. (Table 1 Paper)Trellis baseline agent 95% CITrellis MCTS 95% CIClaude Sonnet 4.5 95% CI
Amylase0.6170.462 [ ]0.305 [−0.049, 0.658]0.252 [0.232, 0.271]
IRED0.2110.190 [0.116, 0.265]0.158 [0.095, 0.222]0.195 [0.191, 0.198]
NucB0.7230.407 [0.379, 0.434]0.447 [0.343, 0.551]0.348
PDZ30.5120.418 [0.197, 0.638]0.491 [0.479, 0.502]0.187 [−0.125, 0.499]
Half-life SMILES0.5420.293 [0.140, 0.447]0.136 [−0.112, 0.383]0.164 [−0.074, 0.401]
Hemolysis SMILES0.5300.402 [0.235, 0.568]0.538 [0.399, 0.677]0.439 [0.376, 0.502]
Nonfouling SMILES0.7350.655 [0.618, 0.691]0.676 [0.605, 0.747]0.687 [0.678, 0.695]
Caco-2 SMILES0.7990.578 [0.202, 0.955]0.554 [0.390, 0.718]0.507 [0.321, 0.694]
DatasetRef. (Table 1 Paper)Trellis MCTS 95% CIBioAutoML 95% CI
Amylase0.6170.305 [−0.049, 0.658]0.269 [0.224, 0.313]
IRED0.2110.158 [0.095, 0.222]0.192 [0.188, 0.196]
NucB0.7230.447 [0.343, 0.551]0.428 [0.420, 0.436]
PDZ30.5120.491 [0.479, 0.502]0.445 [0.426, 0.464]
Half-life SMILES0.5420.136 [−0.112, 0.383]−0.229 [−0.362, −0.095]
Hemolysis SMILES0.5300.538 [0.399, 0.677]0.522 [0.480, 0.564]
Nonfouling SMILES0.7350.676 [0.605, 0.747]0.696 [0.689, 0.704]
Caco-2 SMILES0.7990.554 [0.390, 0.718]0.689 [0.636, 0.741]

Traces for all these run are available as html viewer at https://anonymous.4open.science/r/trellis-B12C/reviewer2_responses/index.html

Reviewer_1YLKOpenReview ↗

Comment

Thank you to the authors for their responses and additional experiments. I have carefully reviewed the rebuttal and the comments from other reviewers. However, my main concerns regarding the limited methodological novelty and the necessity of the proposed MCTS-based framework remain unresolved. Therefore, I have decided to maintain my original rating.

Reviewer_HhgwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work proposes an agentic framework for designing and training models that predict protein properties from sequence. The design consists of choosing protein representation, model architecture and training procedure. The paper tests three distinct setups. First, the authors use a single agent to design a predictor and evaluate it on a range of tasks from three benchmarks. On several tasks, the agent proposes a solution that substantially improves over the published reference scores. Second, the authors propose a more computationally expensive Monte Carlo tree search where a diversity judge prioritizes diversity of the proposed solutions. This tree search improves scores on 7 out of 8 tasks in comparison to the single-agent baseline. Third, the authors show a setup for sequential updates of the predictor when the training database grows. In most of the experiments, the proposed agentic framework does not produce solutions better than published references, but in some cases the improvements are substantial.

Critical strengths

S1: This framework can be useful for creating baseline implementations. The proposed approach is evaluated on multiple tasks, and in general, the solutions are not better than the published reference scores. On a few tasks, however, the improvements are substantial. This approach can potentially reveal that some benchmarks can be improved via architecture optimization and representation choices. If there exists such a straightforward improvement for a benchmark, it should be done automatically, so I support the vision of the paper.

Critical weaknesses

W1: The analysis of the proposed methods is not sufficient. There are three method contributions: the single-agent baseline, the Monte Carlo tree search, and the sequential update loop, but none is analyzed properly to justify its design. The results are presented, but not interpreted and there are no ablations of the critical components. It is difficult to evaluate the contribution of this paper in this form, but if authors address this in detail, I am willing to raise my score. Following analysis and ablations should be considered:

Single-agent:

  • Add information about why some runs fail. If the agent runs out of time, explain why. For example, was the agent unable to produce a valid code? Or was the agent waiting for some evaluations to finish?
  • Add a breakdown of how much time was spent on different parts of the task (writing initial code, interaction with tools, debugging, waiting until code executes).
  • The observation that a single agent retains early representation choice (Line 151) motivates the tree search and the diversity judge. Quantify how often this happens or provide an example of such a trajectory.
  • Why is the variance of the successful runs so high (Table 1, RhoMax)?

Monte Carlo Tree search:

  • Show trajectories of best validation score over the course of the run of tree search.
  • Explain failures. Some nodes failed to produce a valid solution, while others were rejected by the diversity judge. Add the numbers of failures and rejections to the main manuscript and elaborate.
  • Evaluate the contribution of the diversity judge (Line 370). Run tree search with and without a diversity judge and quantify the diversity of the architectures and the achieved scores.
  • Analyze the cost of tree search and compare it to the single-agent.

Sequential update loop:

  • Add the score of the original predictor on the new IEDB 2025 test set and evaluate if the new predictor trained on more data is better than the old one.

W2: Add comparison of the tree-search to the single-agent baseline with the same compute budget. Because the final performance of the tree-search remains well below the published reference, I suspect the tree-search is not working properly / it does not optimize enough. Comparison to a fair baseline would help to resolve this concern.

  • The single-agent baseline ends after submitting the first code that successfully executes and produces a scorable solution. Meanwhile, tree-search has 16 nodes, therefore more submission attempts.

  • It is not clear whether the single agent can see the validation set, whereas the tree search has access to it. Can authors adapt the single-agent so that it can see the validation scores and refine its model based on that feedback?

  • Part of the tree search is run in parallel, so runtime is not directly comparable between single-agent and tree search.

W3: The writing of the paper needs to be significantly improved, especially the Method (requires more detail) and Results (requires interpretation) sections. In particular:

  • It is not clear how the methods are implemented. For example, the single agent selects architecture and representation choices from a predefined list, but this is not specified in the Method section, only in the appendix prompt listings. Many other implementation choices are not described and have to be searched for in the code or in the appendix code snippets.
  • There are many parameters and terms that lack sufficient explanation. For example, what is the probability of the shift to a new architecture in the tree search? Does the agent propose architectures from scratch? Which representations are available to the agent? What is the role of the lookup_hint tool?
  • It is not clear how the single-agent runs are set up. From my current understanding, an agent tries to train a model for 6 hours, where the main issue is that the code does not execute. After the first accepted submission, this agent does not try to optimize, and its trajectory ends (Line 146). Is the predictor training included in the 6h budget? Does the agent see the validation split?
Questions for authors

Clarification requests:

Q1: Please clarify my questions about the evaluation protocol:

  • The method is evaluated on a subset of the FLIP2 benchmark, but it does not include all tasks (missing TrpB) and all data splits. How were the tasks and splits chosen?
  • In Section 4.2, 8 tasks were selected for tree search. These are the tasks where the single-agent baseline produced worse results than the published reference. In Table 1, there are 14 rows (on FLIP2 and Peptiverse) where the Qwen single-agent is worse than the reference. How were the 8 tasks selected?

Q2: Are there any transferable findings about protein property prediction? Please report which representation families were tried and how they scored across tasks. For example, is some sequence representation very suitable for some type of task?

Suggestions for future work:

Q3: It seems like the agents often struggle to produce a valid solution. Would it be possible to start from the published reference solution and let the agents improve it?

5 comments

We thank the reviewer for their extremely helpful comments and suggestions for further analysis. Thank you for thoughtful questions that helped us iterate on our ideas. Addressed the questions below in sections.

W1: Single-agent

Add information about why some runs fail. If the agent runs out of time, explain why. For example, was the agent unable to produce a valid code? Or was the agent waiting for some evaluations to finish?

A run ends when the agent either reaches the turn limit or submits a solution.py that is accepted. Acceptance requires that solution.py execute end-to-end in the sandbox (and produce valid predictions). Submissions that cannot run end-to-end are rejected and do not count as a completed solution. In practice, nearly all unsuccessful runs fall into two cases: (1) the agent reaches the maximum number of turns without an accepted submit, or (2) a submitted solution.py fails at execution—most often from calling a non-existent library API or from tensor/shape mismatches in the model. HTML trace viewer - https://anonymous.4open.science/r/trellis-B12C/reviewer1_responses/index.html

Add a breakdown of how much time was spent on different parts of the task (writing initial code, interaction with tools, debugging, waiting until code executes).

Across 8 replicates on 4 datasets, this is the breakdown. Note: since this is hard to do by hand, we had an LLM judge classify each turn into one of the activity categories

Activity%What this row is
Feature extraction / embedding (sandbox)55.0%Computing inputs to the mode like from ESM-2, RDKit probes, ESMC, Morgan/ECFP, ChemBERTa, PeptideCLM, SaProt
Training & validation (sandbox)35.3%Fitting and scoring heads: torch MLP/CNN, cross-validation, logistic regression, ridge, tree ensemble etc
Submission (sandbox)5.6%submit_answer re-running solution.py end-to-end
EDA / env setup (sandbox)2.9%CSV lookup (shape / value counts), cache and CUDA probes, inspect embedding cache
Writing code (LLM)0.9%LLM drafting and refining code between tool calls
Debugging (LLM after failure)0.3%LLM repair turns after a failed execute_code / submit_answer
Interaction with tools≈0%lookup_hint / lookup_common_errors catalog calls

The observation that a single agent retains early representation choice (Line 151) motivates the tree search and the diversity judge. Quantify how often this happens or provide an example of such a trajectory.

Count
Locked (featurizations tried set size = 1)16 / 25 (64%)
Explored (featurizations tried set size ≥ 2)9 / 25 (36%)

Examples are provided at https://anonymous.4open.science/r/trellis-B12C/example_reviewer2_asked/ci8-elution-r2-134a487d.html

Add the score of the original predictor on the new IEDB 2025 test set and evaluate if the new predictor trained on more data is better than the old one

We reran the pipeline to obtain more replicates for elution (n = 4). The first dataset is the pre-2025 dataset from [ 1], and we used the train/test splits defined in that work. Because this dataset is large and the agent is allowed to train on all of the data if it chooses to, the agent is initially given only a partial train and test set to enable a realistic compute time and cost, subsampled to match the allele distribution and to balance the binary label ratio. Once the agent submits a solution, that solution is rerun on the full dataset and scored on the full test set. validation set agent sees is a random split of the train set, it never sees the held out test set.

To test the self-update loop, we added IEDB records published after the January 2025 cutoff through 2026, and split them into train and test sets (iedb_2026_train, iedb_2026_test) by clustering on peptide sequence. The model trained on the earlier dataset is referred to as m1_old_2025 - best one out of all replicates based on its performance on the test set; it reaches an average precision of 0.9287 on iedb_2026_test. For the update step, the agent was given the m1_old_2025 weights, the solution submitted in the previous loop, and iedb_2026_train. Across four replicates, the updated models reached average precisions of 0.9396, 0.9289, 0.9287, and 0.9407 on iedb_2026_test (mean 0.9345).The third replicate failed because it hit timeout.

  1. Wohlwend, J., Nathan, A., Shalon, N. et al. Deep learning enhances the prediction of HLA class I-presented CD8+ T cell epitopes in foreign pathogens. Nat Mach Intell 7, 232–243 (2025). https://doi.org/10.1038/s42256-024-00971-y

W2: Add comparison of the tree-search to the single-agent baseline with the same compute budget. Because the final performance of the tree-search remains well below the published reference, I suspect the tree-search is not working properly / it does not optimize enough. Comparison to a fair baseline would help to resolve this concern.

The single-agent baseline ends after submitting the first code that successfully executes and produces a scorable solution. Meanwhile, tree-search has 16 nodes, therefore more submission attempts. It was unclear how to properly restrict the budget between mcts and the single agent, but to make a more fair comparison, instead of ending the trajectory as soon as single agent submits a solution, we allow the agent to submit more solutions and refine if it wants to before we hit max turns

DatasetPaperSingle Agent Multiple Submit Best [min, max]Scores reported in manuscript Best [min, max]MCTS Best [min, max]
NucB0.7230.492 [0.427, 0.492]0.424 [0.394, 0.424]0.544 [0.366, 0.544]
PDZ30.5120.499 [0.494, 0.499]0.494 [0.279, 0.494]0.502 [0.483, 0.502]
Hemolysis0.4400.502 [0.469, 0.502]0.440 [0.363, 0.440]0.609 [0.459, 0.609]
Nonfouling0.7350.673 [0.661, 0.673]0.624 [0.624, 0.624]0.706 [0.599, 0.706]

It is not clear whether the single agent can see the validation set, whereas the tree search has access to it. Can authors adapt the single-agent so that it can see the validation scores and refine its model based on that feedback? The agent is allowed to see the validation scores. The loop ends only when solution.py is submitted.

Reviewer_YbAYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents an agentic framework for performing AutoML within biological tasks. The agents plan, design, execute and validate ML model development experiments, store the information in a history, and reiterate to refine the method. The paper describes minor improvement in a set of benchmarks and performs the experiments using two different LLMs to embody the agents.

Critical strengths
  1. The introduction of an agentic workflow to perform AutoML it is sound and could lead to an acceleration of modelling within the domain enabling non-technical researchers to build their own custom models.
  2. The set of benchmarks include multiple different tasks and two input types molecular graphs in the form of SMILES and amino acid sequences.
  3. The diversity evaluation of the plan is a good way of avoiding the system from getting stuck in local optimisation minima, which a priori could be an improvement over other AutoML strategies, like Bayesian Optimisation which may be more prone to get stuck.
Critical weaknesses
  1. The method does not consistently overperform the existing baselines. Even when it does the improvements are marginal. It would be necessary to perform statistical testing to see if the results from the method are significantly better (CI 95%) than the paper reference. Examples of tight margins are: Hydro 0.308->0.392; Nonfouling AA 0.736->0.742; Solubility AA 0.749-> 0.763; Elution default 0.952 -> 0.957. Moreover, the improvements are spread across both language models. Therefore the method is not a consistent improvement, given that the choice of LLM itself cannot be optimised a priori. Otherwise it should be considered at the validation stage, rather than post-hoc at test time.
  2. The paper acknowledges the existence of AutoML methods for biological sequences like BioAutoML, however it does not compare against them under similar budgets.
Questions for authors

NA

1 comment

We thank the reviewer for their helpful comments. And provide a comparison with the published reference scores and with existing methods such as BioAutoML.

95% confidence intervals for the datasets mentioned (n=8)

DatasetRef. (Table 1 Paper)Baseline 95% CI (Qwen)
Hydro (low→high, Spearman)0.3080.259 [0.181, 0.338]
Nonfouling (SMILES, F1)0.7350.655 [0.618, 0.691]
Solubility (SMILES, F1)0.6680.703 [0.674, 0.732]
Elution (default, AP)0.9520.915 []

Any CI intervals with [] is indicating that under same compute replicate budget, some runs failed due to a combination of agent never submitting the solution, exhausting the turn limit or getting stuck in a code error loop. We believe these should be counted towards performance so we did not ensure parity in number of data points across the datasets

We also included CI intervals for MCTS for the datasets in Table 2

DatasetRef. (Table 1 Paper)Baseline Agent 95% CIMCTS 95% CI
Amylase (Spearman)0.6170.462 [ ]0.305 [−0.049, 0.658]
IRED (Spearman)0.2110.190 [0.116, 0.265]0.158 [0.095, 0.222]
NucB (Spearman)0.7230.407 [0.379, 0.434]0.447 [0.343, 0.551]
PDZ3 (Spearman)0.5120.418 [0.197, 0.638]0.491 [0.479, 0.502]
Half-life SMILES (Spearman)0.5420.293 [0.140, 0.447]0.136 [−0.112, 0.383]
Hemolysis SMILES (F1)0.5300.402 [0.235, 0.568]0.538 [0.399, 0.677]
Nonfouling SMILES (F1)0.7350.655 [0.618, 0.691]0.676 [0.605, 0.747]
Caco-2 SMILES (Spearman)0.7990.578 [0.202, 0.955]0.554 [0.390, 0.718]

Comparison to existing frameworks

BioAutoML

DatasetRef. (Table 1 Paper)TRELLIS 95% CIBioAutoML 95% CI
Amylase (Spearman)0.6170.305 [−0.049, 0.658]0.269 [0.224, 0.313]
IRED (Spearman)0.2110.158 [0.095, 0.222]0.192 [0.188, 0.196]
NucB (Spearman)0.7230.447 [0.343, 0.551]0.428 [0.420, 0.436]
PDZ3 (Spearman)0.5120.491 [0.479, 0.502]0.445 [0.426, 0.464]
Half-life SMILES (Spearman)0.5420.136 [−0.112, 0.383]−0.229 [−0.362, −0.095]
Hemolysis SMILES (F1)0.5300.538 [0.399, 0.677]0.522 [0.480, 0.564]
Nonfouling SMILES (F1)0.7350.676 [0.605, 0.747]0.696 [0.689, 0.704]
Caco-2 SMILES (Spearman)0.7990.554 [0.390, 0.718]0.689 [0.636, 0.741]

Traces for all these run are available as html viewer at https://anonymous.4open.science/r/trellis-B12C/reviewer1_responses/index.html

Across these benchmarks, TRELLIS improves on both the published reference scores and BioAutoML on some datasets and falls short on a few others. We would like to use this opportunity to position our intention more precisely. Our main claim is not that the agent consistently outperforms a human-optimized pipeline, but that this architecture enables an agent to generate sequences, optimize properties, and update its predictors as new assay data arrive. The mixed results are consistent with this framing and reflect a deliberate trade-off: unlike a fixed feature-engineering pipeline, an agent with sandbox access can install libraries for new protein language models as they are released and construct dataset-specific features, which we view as the primary advantage of this design. We also note that this loop remains feasible with a comparatively small model (Qwen3-xxB), a choice motivated by the compute budget required to train predictors, particularly where they must be retrained as new assay data become available. Our motivation for MCTS is to encourage diverse exploration of sequence features, since different protein properties may benefit from different featurizations.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

We thank the reviewers for recognizing the potential of automated predictor development for biological research and for the questions that prompted our additional experiments. During rebuttal, we added repeated-run analyses, BioAutoML and Claude comparisons, a stronger single-agent baseline, and experiments on updating predictors as new labels become available.

Importance of automating predictor development and updating

As new assay data and sequence representations become available, researchers repeatedly need to construct features, select architectures, write training code, and evaluate updated predictors. With Trellis, we automate these steps through coding agents that execute experiments in isolated environments. Our tree search formulation encourages exploration of different representations and architectures, and we further retain the resulting code and experimental history for subsequent updates.

We evaluated this approach across protein fitness, peptide properties, and MHC class I prediction. Our practical contribution is the ability to develop task-specific predictors from labeled datasets and revise the modeling procedure when additional data or representations become available. During rebuttal, we tested this capability against conventional AutoML, another coding model, and a single agent allowed to revise its solutions.

New experiments for each reviewer

  • Reviewer YbAY requested uncertainty estimates and a comparison with BioAutoML. We repeated the experiments, reported 95% confidence intervals, and documented unsuccessful runs in the execution traces. Across eight tasks, the Qwen-based Trellis search achieved higher reported mean scores than BioAutoML on five tasks. On PDZ3, for example, we achieved Spearman correlation of 0.491 [0.479, 0.502] versus 0.445 [0.426, 0.464]. We evaluated the same Qwen-based setup across these tasks, allowing readers to assess performance without selecting the language model separately for each reported result.

  • Reviewer Hhgw asked us to explain execution behavior and strengthen the comparison with a single agent. We examined failed submissions, execution costs, and representation choices. In 16 of 25 inspected runs, the single agent used only one featurization, motivating broader exploration early in model development. We then allowed the single agent to make multiple submissions and refine its models using validation feedback. Tree search achieved higher best-observed scores on all four tested tasks. For example, the best NucB correlation was 0.544 versus 0.492, and the best hemolysis F1 was 0.609 versus 0.502. We also clarified validation access, task selection, and execution requirements, and reran sequential predictor updates using newly released IEDB records.

  • Reviewer 1YLK questioned the practical contribution and asked about stronger coding models and simpler alternatives. We added Claude Sonnet 4.5 alongside the BioAutoML comparison. The Qwen-based Trellis search achieved higher reported means on five of eight tasks than the Claude single-agent configuration. We also clarified the roles of the coding agent and tree search and identified the source benchmarks behind the published reference scores. To test repeated updating directly, we added a TrpB experiment with simulated assay feedback. Starting from 203 oracle-labeled examples, the agent trained a predictor, proposed sequences, received model-based fitness scores, and retrained on the accumulated labels. Three of four trajectories reached five rounds, including two that returned final solutions. In those three trajectories, the highest oracle fitness increased from 0.479 in the first round to 1.100–1.186 in later rounds. This demonstrates an executable cycle of prediction, proposal, feedback, and predictor updating.

Case for acceptance

Trellis automates a recurring part of biological research, from building an initial predictor to revising it as new data arrive. Across the additional experiments, we produced competitive predictors relative to BioAutoML, improved the best observed results over a single agent that could revise its submissions, and completed repeated updates in a simulated experimental loop. We also documented the actual executions so researchers can inspect the modeling choices and reuse the resulting code.

We believe this combination of automated model development, exploration across representations, and adaptation to new labels makes a substantive contribution to biological machine learning. We respectfully ask the Area Chair and reviewers to consider the expanded evidence, reconsider the initial scores, and support acceptance of Trellis.

Thank you so much,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.