acceptodds
Under review as a conference paper at ICLR 2027

Beyond Retrieval and Fine-tuning: Learning from Scientific Execution Experience for Self-Improving Agents

Abstract

Scientific execution experience often intertwines transferable strategies with context-specific constraints. Neither imitating successful trajectories nor accumulating external experience alone therefore guarantees effective transfer. We introduce BioMaster, a framework that learns from the same scientific execution experience through two complementary pathways: parameter learning and external experience reuse. BioMaster post-trains its model on filtered execution trajectories and distills task-specific methods into structured Research Units(RUs), which encode applicability conditions, execution dependencies, and verification evidence to support retrieval and adaptation at inference time. Failure diagnosis further guides targeted task construction and subsequent model updates. With the base model held fixed, BioMaster achieves the highest scores among the evaluated systems on STELLA Simple Set, Biomni-Eval1, and BioMystery. With the agent harness held fixed, its post-trained model improves BioMystery accuracy by 9.26 percentage points over the base model. Component analyses further support complementarity between parameter learning and experience reuse. Together, these findings highlight the value of jointly internalizing scientific experience into model parameters and structuring it as reusable methodological knowledge.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Scientific agents, Agent post-training

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_ACq7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BioMaster, a framework that uses scientific execution experience for both model post-training and external experience reuse. Validated trajectories provide supervised training data, while structured Research Units (RUs) preserve methods, applicability conditions, dependencies, and verification evidence for retrieval. The framework also uses failure diagnosis to guide additional task construction. Experiments compare complete systems under a fixed backbone and different models under a fixed agent harness. A component study on 203 tasks suggests that post-training increases the benefit of using a fixed RU library.

Critical strengths
  1. The interaction between post-training and experience reuse is interesting. With the harness and RU library fixed, RU access changes performance by -1.31 percentage points for the base model and +4.27 points for the post-trained model. This comparison provides useful evidence that the value of external experience depends on the capabilities of the model using it.

  2. The RU representation addresses practical requirements of scientific workflows. Retaining input conditions, execution dependencies, validation evidence, and known failures makes the stored experience more useful for adaptation and verification. The system-level and model-level evaluations also help distinguish improvements from the execution framework and those from model training.

Critical weaknesses
  1. The evidence for failure-guided updating is unclear. In Table 21, random expansion outperforms failure-diagnosis expansion both without RUs (59.44 vs. 57.31) and with RUs (63.71 vs. 61.74). Moreover, the random-expansion scores exactly match the final v2 scores in Figure 4, although v2 is described as a failure-guided update. This makes it difficult to determine which checkpoint supports the main conclusions and whether failure-guided task construction provides an advantage over ordinary data expansion.

  2. The separation between training, RU construction, failure diagnosis, and final evaluation is insufficiently specified. Appendix A.3 explains how the 203 evaluation tasks are sampled, but does not clearly establish their exclusion from training trajectories and RU sources. It is also unclear whether the tasks used for failure diagnosis are separate from the final test set. These details are necessary to assess whether the reported gains demonstrate transfer to unseen tasks.

Questions for authors
  1. Which checkpoint is reported as BioMaster-A1-v2 in Figure 4: the failure-diagnosis version or the random-expansion version? How should the results in Table 21 be reconciled with the failure-guided update described in the main text?

  2. Were all 203 evaluation tasks and their execution records excluded from both SFT data and RU construction? Please clarify how related tasks from the same paper or workflow were handled, and whether failure diagnosis used a separate development set.

  3. Please clarify the training sequence. Section 3.3 and Table 15 report 6,264 Stage-2 trajectories, while Table 18 reports 4,264. Section C.4 states that v2 starts from v1, but Table 19 lists the Stage-1 checkpoint as its initialization. Which configuration was actually used?

4 comments
AuthorsOpenReview ↗

Corrected v2 Attribution and Training Configuration

Thank you for your careful review. Your comments have helped us clarify the checkpoint mapping, data flow, and training procedure. We address Q1 and Q3 below, followed by Q2 in our second reply.

Q1. Is the v2 model in Figure 4 the random-expansion or failure-guided variant?

Figure 4 reports BioMaster-A1-v2-r, the failure-diagnosis-guided variant. The suffix r denotes reflection-guided expansion; d denotes direct random expansion without failure diagnosis. Some appendix descriptions and plot labels reversed this mapping, causing the failure-guided results to be interpreted as random-expansion results. The correct mapping is:

CheckpointExpansion strategyRU offRU on
BioMaster-A1-v2-dDirect random expansion57.3161.74
BioMaster-A1-v2-r (final v2)Failure-diagnosis-guided expansion59.4463.71

Failure-guided expansion exceeds random expansion by 2.13 pp without RU and 1.97 pp with RU. Figure 4's scores of 59.44 and 63.71 both belong to v2-r, consistent with the final model described in the main text. The correction concerns strategy labels and model attribution; the numerical scores are unchanged.

Both variants start with the same 4,264 initial scientific trajectories, independently add 2,000 trajectories, and use full-parameter SFT on 6,264 trajectories from the same Stage-1 checkpoint. They are separate training branches. v1 provides diagnostic feedback, not their initial parameters.

The additional data differ in task selection. Random expansion does not target v1's failures. Failure-guided expansion uses v1's performance on the 203 component tasks to select workflow clusters requiring further training. Only the selected cluster IDs are passed to task construction; the original evaluation prompts, answers, and execution traces are not supplied. Newly collected trajectories are validated before training.

The corrected results therefore show higher mean component scores for failure-guided expansion in both RU conditions, at matched trajectory count, initialization, and parameter-update method, within this workflow-feedback setting.

We will correct Figure 4, original Table 21, Appendix C.4, and the associated plots, explicitly stating BioMaster-A1-v2 = BioMaster-A1-v2-r, so that model names, training datasets, and scores correspond consistently.

Q3. How should the 4,264/6,264 trajectory counts and v2 initialization be reconciled?

4,264 is the initial scientific dataset used for v1; 6,264 is the expanded dataset used for each v2 variant. All three models initialize from the same Stage-1 checkpoint.

1. Dataset versions and counting convention

Counts refer to filtered execution trajectories actually included in scientific-task SFT:

Scientific-task modelInitial trajectoriesAdditional trajectoriesTotal training trajectoriesInitialization
BioMaster-A1-v14,264None4,264Shared Stage-1 checkpoint
BioMaster-A1-v2-d4,2642,000 random-expansion trajectories6,264Shared Stage-1 checkpoint
BioMaster-A1-v2-r (final v2)4,2642,000 failure-diagnosis-guided trajectories6,264Shared Stage-1 checkpoint

Each v2 branch retains all 4,264 initial trajectories and combines them with its own 2,000 additional trajectories for full-parameter SFT. The branches use different expansion data; their additions are not cumulative. Stage 1's 17,332 Harness-adaptation execution examples are counted separately. The two scientific-data counts therefore use the same convention, with the difference corresponding to 2,000 additional trajectories.

2. Feedback sequence and parameter initialization

v1's workflow-level feedback guides task construction for the diagnosis-guided branch. Once the expanded dataset is ready, v2 restarts scientific-task training from the shared Stage-1 checkpoint.

Thus, v1 precedes v2 in the data-construction process, while their parameter training starts from the same checkpoint. The original wording “continued training from v1” is inaccurate. “Stage 3” denotes failure feedback, data expansion, and retraining, rather than a third consecutive optimization stage starting from v1.

The v1-to-v2 comparison captures the overall effect of the expanded dataset. Q1's comparison of v2-r and v2-d, both using 6,264 trajectories and the same initialization, further compares the two expansion strategies.

3. Manuscript corrections

We will use 4,264 consistently for v1 in Section 3.3, Table 15, and related descriptions, and reserve 6,264 for the expanded v2 datasets. Table 18's count of 4,264 and Table 19's Stage-1 initialization match the actual configuration; contradictory descriptions of continued training in Appendix C.4 will be corrected.

AuthorsOpenReview ↗

How are the 203 tasks separated from training and RU construction, and used for failure diagnosis?

Q2. How are the 203 tasks separated from training and RU construction, and used for failure diagnosis?

We clarify both the use of evaluation content and the workflow-level feedback passed to task construction. Performance on the 203 tasks identifies workflow clusters requiring improvement, but their prompts, answers, and execution traces are not supplied to new-task construction, SFT, or RU construction.

1. How failure feedback enters task construction

There are two steps. First, v1 executes the 203 component tasks, and its performance identifies workflow categories needing additional training experience. The task-construction module then receives only the selected cluster IDs to specify the categories for new tasks and experience collection.

The IDs indicate which workflows should receive more experience. New tasks must still be constructed, executed, and validated before their records become SFT or RU-extraction material.

StageMaterial usedRelationship to the 203 component tasks
SFTValidated scientific execution trajectories from training tasksExcludes these 203 tasks, their answers, and execution records
RU constructionMethods, applicability conditions, and validation experience extracted from source execution recordsNo RUs are extracted from these tasks' answers or execution traces
Failure diagnosis and task expansionv1's performance on the 203 tasks and the workflow cluster IDs selected accordinglyNew-task construction receives only selected cluster IDs, not original task prompts, reference answers, or execution traces
Component evaluationThe same 203 tasks and their evaluation materialsCompares model and RU configurations within the workflow-feedback setting

The 203 tasks therefore inform the direction of expansion, while the material used to update parameters and construct RUs comes from training-task executions. We will explicitly describe this dual role of workflow feedback and component evaluation, together with the information passed between stages.

2. Workflow coverage and task provenance

Appendix A.3's “one task per selected cluster” sampling controls component-evaluation coverage and weighting; it does not hold out entire workflow clusters. Our partition permits training and evaluation tasks to share general workflow types.

A scientific task can combine cell annotation, differential expression, and pathway enrichment, and the same method can address different biological questions. For example, comparing monocytes from infected versus healthy donors and comparing CD8+ T cells before versus after treatment can both use differential expression. Each still requires analysis and validation using its own data, target population, and comparison design. RUs provide methodological guidance; task-specific results depend on the task's inputs and execution.

For tasks linked through the same paper, data entity, or derivation, we will examine input instances, scientific objectives, required outputs, and associated execution records. The supplement will list all 203 tasks' IDs and prompts, source papers or original source IDs, input-data and sample identifiers, parent-task and derivation relationships, RU sources, and workflow memberships, specifying what related tasks share.

We will also present representative evaluation tasks alongside the closest training tasks and RU source records to distinguish shared methods, common sources, and reuse of specific problems or solutions. Different task IDs or wording do not establish independence, and shared workflows do not by themselves establish task duplication.

3. Gains on other public task sets

Beyond the 203-task component comparison, Figure 4(a) reports v1 and final-v2 results on three public benchmarks:

Public benchmarkBioMaster-A1-v1BioMaster-A1-v2-rImprovement (pp)
STELLA Simple Set83.5385.19+1.66
Biomni-Eval147.1153.66+6.55
BioMystery49.6350.37+0.74
Equal-weight average60.0963.07+2.98

Final v2 improves over v1 on all three benchmarks, raising the mean by 2.98 pp. The observed gains therefore extend beyond the 203 tasks used for workflow selection. Component results characterize model and RU performance within the feedback setting; the public benchmarks provide additional evidence of the training update's benefits on other scientific task sets.

We will align Appendix A.3, Appendix C.4, and the configuration tables so that task provenance, workflow feedback, new-experience construction, and evaluation can be traced consistently.

We hope these clarifications and corrected results address your concerns and would appreciate your reconsideration of the assessment and score. We welcome further discussion and are happy to revise the manuscript in response to any remaining questions or suggestions.

AuthorsOpenReview ↗

Follow-up on Failure-Guided Expansion Across Three Public Benchmarks

To evaluate generalization on tasks separate from the 203-task feedback set, we added three direct comparisons between failure-guided and random expansion, one on each public benchmark.

We used the fixed-Harness model-level protocol from Figure 3. Both variants started from the same Stage-1 checkpoint and added 2,000 trajectories, collected with the same teacher, to the same initial training set.

BenchmarkRandom expansion (v2-d)Failure-guided expansion (v2-r)v2-r minus v2-d
STELLA Simple Set84.5185.19+0.68 pp
Biomni-Eval151.9653.66+1.69 pp
BioMystery v1147.4150.37+2.96 pp

Failure-guided expansion scored higher on all three benchmarks, with the largest difference on BioMystery. The Biomni-Eval1 difference is calculated before rounding.

Prior work has used model failures to guide data generation [1, 2], compared failure-driven and random sampling at a matched data budget [3], and evaluated failure-guided training on a separate benchmark [4]. In BioMaster, the 203 component tasks inform which workflow clusters to expand. The task generator receives only the selected cluster IDs. The original task prompts, reference answers, and execution traces are excluded from new task construction, supervised fine-tuning, and RU extraction (Appendix A.3). This lets us use workflow-level feedback while keeping the original evaluation materials out of the training pipeline.

We will include these comparisons and clarify the feedback protocol in the revised manuscript. We would appreciate hearing whether the new results address your concerns, and we would be happy to discuss any remaining questions.

References

  1. He et al. (2023). Targeted Data Generation: Finding and Fixing Model Weaknesses. ACL.
  2. Stan et al. (2026). Learning from Reasoning Failures via Synthetic Data Generation. AAAI.
  3. Hao et al. (2026). HardGen: Generating Hard Samples for Tool-use Agents. EMNLP.
  4. Kang et al. (2026). TRACE: Capability-Targeted Agentic Training. ICML 2026 AIWILD Workshop.
AuthorsOpenReview ↗

Looking forward to further discussion

Dear Reviewer,

We are looking forward to your feedback and would greatly appreciate any further questions or concerns you may have.

If there are additional issues that require clarification or further experiments, an earlier response would help us better address them within the remaining rebuttal period.

Thank you again for your time and consideration.

Reviewer_QEpEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BioMaster, which uses experience from executing scientific tasks both for model post-training and for constructing structured Research Units (RUs). It also uses failure diagnosis to add tasks and update experience. The experiments compare complete systems, models under a fixed execution framework, and the combined effects of training and RU reuse. The main performance gains are observed on BioMystery.

Critical strengths

1.RUs record applicability conditions, dependencies, validation evidence, and known failures in addition to successful steps. This is practically useful for reusing experience in scientific tasks.

2.The component experiments with a fixed framework and RU library are fairly convincing: RUs change the base model's performance by −1.31 percentage points and the trained model's performance by +4.27 percentage points. This indicates that model capability affects how effectively the experience library is used (Figure 4).

Critical weaknesses

1.The advantage of failure-driven updates is not yet supported. In Appendix Table 21, with RUs enabled, expansion based on failure diagnosis scores 61.74, compared with 63.71 for random expansion; it also performs worse with RUs disabled. The main text should explain this result and specify which expansion strategy the final model uses, rather than attributing gains from additional data to failure diagnosis.

2.The separation between training and evaluation in the component experiments is unclear. Appendix A.3 states that 203 tasks are sampled across clusters, but also explicitly notes that this sampling does not itself determine whether tasks are assigned to training or RU construction. The paper needs to clarify whether evaluation tasks or trajectories from the same sources enter training, the RU library, or the failure-repair process before cross-task transfer can be assessed.

3.Inconsistencies in the training procedure affect reproducibility. Section 3.3 and Table 15 report 6,264 scientific trajectories for v1, whereas Table 18 reports 4,264. Appendix C.4 states that v2 continues training from v1, but Table 19 lists initialization from Stage 1. These discrepancies also affect the interpretation of what drives the improvement from v1 to v2.

Questions for authors

1.How exactly are the 203 evaluation tasks separated from training trajectories, RU sources, and failure-diagnosis tasks? Was direct reuse arising from the same source papers or workflows excluded?

2.Which row in Table 21 corresponds to the final v2 model in Figure 4? Why does random expansion outperform expansion based on failure diagnosis, and what data and initial checkpoint were actually used for the final model?

6 comments
AuthorsOpenReview ↗

Corrected Checkpoint Mapping and Failure-Guided Gains

Thank you for your careful reading and constructive feedback. Your comments have helped us improve the manuscript and clarify several confusing descriptions. We address your concerns below with corrected checkpoint labels, model comparisons, and explicit data and training configurations.

Q1. Why does Table 21 appear to favor random expansion, and which model is reported in Figure 4?

Figure 4 reports BioMaster-A1-v2-r, the failure-diagnosis-guided variant. The suffix r denotes reflection-guided expansion; d denotes direct random expansion without failure diagnosis. The correct configurations and results are as follows.

1. Training data and initialization

All three scientific-task models use full-parameter SFT from the same Stage-1 checkpoint. Stage 1 establishes tool-use and Harness interaction skills. Subsequent training uses complete scientific execution trajectories covering method selection, multi-step tool operations, intermediate-result inspection, and error recovery to develop reasoning and long-horizon execution.

ModelScientific-task training dataConstruction and characteristics of additional dataInitialization
BioMaster-A1-v14,264 initial trajectoriesInitial scientific execution data covering complete analysis, execution, and verificationShared Stage-1 checkpoint
BioMaster-A1-v2-d4,264 initial + 2,000 random-expansion trajectories = 6,264Direct random expansion without targeting failures diagnosed in v1Shared Stage-1 checkpoint
BioMaster-A1-v2-r (final v2)4,264 initial + 2,000 diagnosis-guided trajectories = 6,264Tasks targeting weaknesses identified in v1: candidate comparison, precise operations, database semantics, and result checkingShared Stage-1 checkpoint

Failure-guided expansion changes how additional training tasks are selected and constructed. We identify weak capabilities from v1's failures, construct targeted tasks, and collect and validate complete execution trajectories. The resulting data retain the full scientific execution process while targeting observed weaknesses.

The two v2 models are independent branches, each adding 2,000 trajectories to the initial 4,264. v1 supplies diagnostic feedback for v2-r; both v2 models initialize from Stage 1. The comparison therefore matches trajectory count, initialization, and parameter-update method.

2. Corrected results and model differences

Results on the same 203-task component evaluation set are shown below.

ModelRU offChange vs. v1RU onChange vs. v1
BioMaster-A1-v158.95—60.92—
BioMaster-A1-v2-d: random expansion57.31−1.6461.74+0.82
BioMaster-A1-v2-r: failure-guided expansion, final v259.44+0.4963.71+2.79

Relative to v1, random expansion improves the RU-on score by 0.82 pp but reduces the RU-off score by 1.64 pp. Failure-guided expansion improves both scores, by 0.49 and 2.79 pp, and achieves the highest mean in both conditions. Increasing trajectory count alone therefore does not consistently improve performance; the selection of additional tasks matters.

The direct comparison is v2-r versus v2-d: at the same trajectory count and initialization, failure-guided expansion improves scores by 2.13 pp without RU and 1.97 pp with RU. This supports the added value of diagnosis-guided data construction beyond increasing dataset size. Figure 4's scores of 59.44 and 63.71 both belong to v2-r.

We have corrected the strategy labels and checkpoint descriptions in Figure 4, original Table 21, Appendix C.4, and the associated plots, explicitly stating BioMaster-A1-v2 = BioMaster-A1-v2-r. The reported scores are unchanged.

AuthorsOpenReview ↗

Task Separation and Training Configuration

Q2. Are the 203 evaluation tasks separated from training and RU sources, including related papers and workflows?

All 203 tasks, their answers, and execution traces were excluded from SFT and RU construction. Only workflow cluster IDs selected from v1's performance are passed to task construction; prompts, answers, and execution traces are excluded.

1. Data-use boundaries

StageData or information usedRelationship to the 203 component evaluation tasks
SFTValidated scientific execution trajectories from training tasksExcludes all 203 tasks, their answers, and execution traces
RU constructionReusable methods, applicability conditions, and validation experience extracted from source execution recordsNo RUs are extracted from these tasks' answers or execution traces
Failure diagnosis and data expansionv1's performance on the 203 tasksOnly selected workflow cluster IDs reach task construction; prompts, reference answers, and execution traces are excluded
Component evaluationThe 203 tasks and their evaluation materialsMeasures performance after workflow-level feedback from this set

2. Shared workflows do not imply task reuse

Tasks can combine cell-type annotation, differential expression, and pathway enrichment. Shared workflows or clusters do not imply reuse of complete tasks or their solutions.

For example, differential expression in monocytes from infected versus healthy donors and in CD8+ T cells before versus after treatment requires different inputs, targets, designs, and outputs. RUs guide methods; each task still requires adapted analysis, execution, and verification.

3. Source relationships

We will provide a provenance and split manifest for all 203 tasks: task IDs and prompts, source papers or original source IDs, input-data and sample identifiers, parent-task and derivation relationships, RU sources, and workflow memberships.

We will check shared-paper/data links, inputs, objectives, target outputs, and derived traces for direct reuse. Different IDs or wording do not establish independence; shared workflows do not establish duplication. Appendix A.3 will document these boundaries.

4. Gains beyond the 203 component tasks

In Figure 4(a), final v2 improves over v1 on all three public benchmarks evaluated separately from the 203-task component set:

BenchmarkBioMaster-A1-v1BioMaster-A1-v2-rGain (pp)
STELLA Simple Set83.5385.19+1.66
Biomni-Eval147.1153.66+6.55
BioMystery49.6350.37+0.74
Equal-weight average60.0963.07+2.98

The +2.98 pp mean gain shows that improvements extend beyond the 203 tasks used for workflow selection.

Q3. What are the actual v1 dataset size and v2 initialization?

v1 uses 4,264 scientific trajectories; final v2 uses 6,264. Both initialize from the same Stage-1 checkpoint. Counts refer to filtered trajectories included in each SFT dataset.

1. Data and initialization

CheckpointInitializationTraining dataParameter update
Stage 1Qwen3.6-35B-A3B17,332 Harness-adaptation execution examplesLoRA SFT
BioMaster-A1-v1Shared Stage-1 checkpoint4,264 initial scientific-task trajectoriesFull-parameter SFT
BioMaster-A1-v2-dShared Stage-1 checkpoint4,264 initial + 2,000 random-expansion trajectories = 6,264Full-parameter SFT
BioMaster-A1-v2-r (final v2)Shared Stage-1 checkpoint4,264 initial + 2,000 failure-guided trajectories = 6,264Full-parameter SFT

Each v2 branch retains all 4,264 initial trajectories and independently adds 2,000. The two expansions are not cumulative.

2. Diagnostic feedback versus parameter initialization

v1's performance on the 203 tasks selects workflow clusters; only these IDs are passed to task construction. We combine 2,000 new trajectories with the initial data and restart full-parameter SFT from Stage 1.

v1 provides feedback, not v2's initial parameters. “Stage 3” means diagnosis, expansion, and retraining, not continued optimization from v1. We will correct the original wording.

3. Shared v1 and final-v2 training hyperparameters

Settingv1 and final v2-r
OptimizerDistributed Fused AdamW
Peak / minimum learning rate5×10−65\times10^{-6} / 5×10−75\times10^{-7}
Schedule / warm-upCosine / 5%
Global batch size32
Gradient accumulation steps4
Maximum sequence length131,072 tokens
Maximum training epochs3
Random seed42

We will align Sections 3.3-3.4, Tables 15, 18, and 19, and Appendix C.4, explicitly identifying final BioMaster-A1-v2 as v2-r.

Thank you again for your feedback. We would appreciate your reconsideration of the assessment and score in light of these clarifications. We welcome further questions and would be happy to revise the manuscript based on your suggestions.

Thank you for the response. My concerns about the training configuration and the apparent contradiction in Table 21 are addressed.

The evaluation concern is only partially addressed. Since the 203 tasks guided data expansion, improvements on those same tasks cannot be treated as fully independent evidence of generalization. In particular, the comparison may favor failure-guided expansion because it targets weaknesses identified on the evaluation set, making its advantage over random expansion less conclusive. The separate public-benchmark results support the method's broader usefulness, but do not directly establish the advantage of failure-guided over random expansion.

Reviewer_kLh4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BioMaster, a framework that turns execution trajectories of a strong teacher model into two things: training data for post-training a smaller open model, and Research Units (RUs), structured method records that the agent retrieves at inference time. The authors evaluate on three biology agent benchmarks and claim that post-training improves the base model, and that RUs help more after post-training, which shows that parameter learning and experience reuse are complementary.

Critical strengths

The RU design is well structured, and the worked example makes it clear what an RU contains. The experiments are repeated several times without best-of selection, and the comparison with concise traces and runtime memory is a useful control.

Critical weaknesses

My main concern is that the gain probably comes from the teacher, not from the agent improving itself. All training trajectories are generated by Kimi K3 and filtered by checkers, and the student is trained on them with SFT. This is distillation, and the teacher is still ahead on one benchmark. The paper never compares with plain distillation from the same teacher without the Harness and RUs, with a weaker teacher, or with teacher-free post-training such as GRPO. The tasks already have checkers, so a verifiable reward is available. Without these, I cannot tell whether the improvement comes from the proposed design.

The complementarity claim is not cleanly supported. The training trajectories already contain retrieved RUs as context, so the "post-training only" model has seen RUs during training. The interaction may just come from matching training and inference formats. I also think the evidence is thin, since the gain is concentrated on one benchmark, another one drops, and there is no significance test.

The failure-guided update is not supported by the appendix. The random-expansion variant matches the main-text v2 numbers, while the failure-diagnosis variant is lower. Which data was actually used for the reported v2 model?

(minor) I could not find the train-test overlap checks that the paper says are in the appendix, the Stage 2 data size is inconsistent between the main text and the appendix, and the Evo configuration is never clearly described.

Questions for authors

Please see weaknesses.

5 comments
AuthorsOpenReview ↗

Contributions Beyond Teacher Distillation

Thank you for your thoughtful review and constructive suggestions. They have helped us clarify the method and strengthen its experimental support. We address your concerns below with existing controls, additional experiments, and corrections to the manuscript.

Q1. Are the improvements attributable only to teacher distillation?

We agree that SFT on validated teacher-generated trajectories includes distillation. Our contribution concerns how scientific executions are organized, converted into reusable RUs, and selected for further training through failure feedback. The following controls establish the roles of these designs.

1. Harness gains with model parameters fixed. With RUs disabled:

ModelStandard execution environmentBioMaster HarnessHarness gain
Base50.0854.02+3.94
Final v255.3459.44+4.10

Each row uses identical model parameters. The untuned Base gains 3.94 pp from the Harness, showing that execution support itself is useful. The trained v2 also outperforms Base in both environments, so its gains extend beyond the BioMaster execution environment.

2. Experience gains with the v2 checkpoint and Harness fixed. Existing representation controls and the additional independent-source RU experiment give:

Inference-time experiencev2 scoreGain over no experience
No external experience59.44—
Concise execution traces55.99−3.45
Runtime memory60.59+1.15
Original structured RUs63.71+4.27
Independent-source relevant RUs63.22+3.78

Full RUs outperform runtime memory by 3.12 pp and concise traces by 7.72 pp. Thus, how experience is organized and supplied affects its usefulness; adding experience does not automatically improve performance. Independent-source RUs also improve the same checkpoint by 3.78 pp, only 0.49 pp below the original library. These comparisons demonstrate the additional value of inference-time experience access with student parameters fixed.

3. Expansion strategy matters with the teacher, trajectory count, and initialization matched. Appendix C.4.3 compares two branches using Kimi K3 trajectories and full-parameter SFT:

CheckpointInitial trajectoriesAdditional trajectoriesTotalInitializationRU offRU on
v2-d4,2642,000 random6,264Shared Stage 157.3161.74
v2-r (final v2)4,2642,000 failure-guided6,264Shared Stage 159.4463.71

On the 203 component tasks used for workflow feedback, failure-guided expansion exceeds random expansion by 2.13 pp without RU and 1.97 pp with RU. The teacher, trajectory count, initialization, and update method are matched; the expansion strategy changes. These results show that selecting and constructing experience matters even with the teacher fixed.

4. Expanded training also improves external benchmarks. Figure 4(a) reports:

Benchmarkv1Final v2-rChange
STELLA Simple Set83.5385.19+1.66
Biomni-Eval147.1153.66+6.55
BioMystery49.6350.37+0.74
Equal-weight average60.0963.07+2.98

Final v2 improves all three benchmarks over v1, extending the gains beyond the 203 tasks used to select workflows.

Together, these controls support contributions from execution support, experience reuse, and feedback-guided data construction. The additional 4B training comparisons in Q2 examine two trajectory teachers. We will describe the training accurately as teacher-assisted experience construction and supervised updating, with the above controls clarifying what each component contributes.

AuthorsOpenReview ↗

Teacher Dependence and RU Complementarity

Q2. Does the method depend on a strong teacher?

We added experiments with a Qwen3.5-4B student, using trajectories generated by Kimi K3 or Qwen3.6-35B-A3B under the ordinary and BioMaster training schemes. Every configuration is evaluated on the same 203 tasks, with three independent executions per task.

Training conditionOrdinary schemeBioMaster scheme
Qwen3.5-4B baseline14.4516.42
Kimi K3 → Qwen3.5-4B19.2120.03
Qwen3.6-35B-A3B → Qwen3.5-4B17.4118.39

With BioMaster training, Kimi trajectories improve the baseline from 16.42 to 20.03 (+3.61 pp); Qwen trajectories improve it to 18.39 (+1.97 pp). The benefit therefore extends to the Qwen3.6-35B→Qwen3.5-4B combination.

BioMaster also has higher final scores than the ordinary scheme under both teachers, by 0.82 and 0.99 pp, respectively, calculated before rounding. Because the schemes have different baselines, these final-score differences compare the full configurations. Kimi still yields higher final performance, indicating that teacher choice matters, while positive gains are not confined to Kimi-generated trajectories.

Q3. What about student-generated experience and GRPO?

We are currently running the student-generated-experience and GRPO comparisons and expect to complete them by November 16. We will provide the results and experimental settings in a follow-up response.

Q4. Is complementarity merely training–inference format matching?

“Post-training only” means that an RU-trained model has retrieval disabled at inference. It does not mean that the model never encountered RUs during training. We will rename this condition and report training exposure separately from inference-time retrieval.

1. Stage 1 already teaches RU interaction; scientific-task training adds further gains.

Stage 1 uses short, simple Harness-adaptation tasks. Scientific SFT uses complete trajectories involving method selection, multi-step execution, intermediate checks, and result verification.

CheckpointTraining samples containing RU callsRU offRelevant RU onRU gain
BaseNo BioMaster post-training54.0252.71−1.31
Stage 15,779 / 17,332 (33.34%)56.8158.62+1.81
v1All 4,264 scientific trajectories58.9560.92+1.97
Final v2All 6,264 scientific trajectories59.4463.71+4.27

We agree that early interface and interaction adaptation contributes. Nevertheless, final v2 improves over Stage 1 by 2.63 pp without RU and 5.09 pp with RU, adding 2.46 pp to the RU gain. For v1, the corresponding additional RU gain is only 0.16 pp. Both v1 and v2 initialize from the same Stage-1 checkpoint.

These results show that scientific-task training improves both task performance without retrieval and the overall benefit obtained from RUs beyond the Stage-1 checkpoint.

2. Content relevance matters under the same RU interface.

We preserve the retrieval-call rules and replace returned content only when retrieval occurs, matching quantity, format, and length as closely as possible. Gains are relative to each model’s own RU-off score; interaction is the v2 gain minus the Base gain.

RU conditionBaseBase gainv2v2 gainInteraction
Off54.02—59.44——
Original relevant52.71−1.3163.71+4.27+5.58
Random52.38−1.6459.61+0.17+1.81
Clearly irrelevant52.55−1.4757.64−1.80−0.33
Independent-source relevant53.37−0.6563.22+3.78+4.43

For v2, original relevant RUs outperform random and irrelevant RUs by 4.10 and 6.07 pp. A familiar interface does not make arbitrary content equally useful. Independent-source RUs yield +3.78 pp, only 0.49 pp below the original library, showing that the benefit extends to new sources.

Random RUs leave v2 near its baseline, but clearly irrelevant RUs reduce it by 1.80 pp; we therefore distinguish useful-content gains from resistance to interference.

Together, these results show an early contribution from interface adaptation and a larger relevant-RU benefit in final v2. The additional benefit depends on experience relevance and extends to an independent library, supporting the value of scientific-task training beyond the initial Harness adaptation.

AuthorsOpenReview ↗

Benchmark Trade-offs, RU Representation, and Failure-guided Expansion

Q5. How do you explain the decline on one external benchmark?

Final v2 improves the aggregate result and exceeds v1 on all three benchmarks. STELLA remains below Base, but expanded training recovers part of the initial decline. The matched checkpoint comparison in Figure 4(a) is:

BenchmarkBasev1Final v2v2 − v1
STELLA Simple Set87.6083.5385.19+1.66
Biomni-Eval150.9647.1153.66+6.55
BioMystery41.1149.6350.37+0.74
Equal-weight average59.8960.0963.07+2.98

The average rises from 59.89 to 63.07. Relative to v1, improvements cover all three benchmarks, with the largest gain on Biomni-Eval1. Expanded training preserves and extends BioMystery gains while recovering performance elsewhere.

For STELLA, the initial decline is 4.07 pp, from 87.60 to 83.53. Retraining from the shared Stage-1 checkpoint on the expanded dataset raises the score to 85.19, recovering approximately 40.8% of that decline.

A plausible explanation is the difference between the training emphasis and benchmark requirements. Scientific execution trajectories emphasize method selection, multi-step operations, verification, and error recovery, whereas the STELLA protocol scores the correctness of selected options. Adaptation can change answer strategies and knowledge use; better execution does not necessarily translate into more accurate option selection. The remaining gap points to a need for better retention of existing capabilities. We will present both the aggregate improvements and this benchmark-specific trade-off.

Q6. What does the evidence show about structured RUs?

RUs organize a reusable scientific method's objective, inputs, steps, applicability conditions, and validation evidence. Existing experiments compare complete experience representations under the same Harness:

Experience representationBaseFinal v2v2 gain over no experience
No external experience54.0259.44—
Concise execution traces50.9055.99−3.45
Runtime memory51.7260.59+1.15
Full structured RUs52.7163.71+4.27

Full RUs improve v2 by 4.27 pp over no experience, 3.12 pp over runtime memory, and 7.72 pp over concise traces. Concise traces preserve historical operations that the model must interpret and adapt. Runtime memory supplies environment and operational details, including recorded recovery. Full RUs additionally specify input requirements, applicability, and validation evidence, helping the model decide whether a method applies, how to adapt it, and how to check its results.

Q7. Why does Table 21 appear to favor random expansion, and which model is final v2?

Final v2 is BioMaster-A1-v2-r: r denotes reflection-guided expansion; d denotes direct random expansion. Some appendix explanations and plot labels reversed this mapping. The correct correspondence is:

CheckpointScientific training dataInitializationRU offRU on
v14,264 initial trajectoriesShared Stage 158.9560.92
v2-d4,264 initial + 2,000 random = 6,264Shared Stage 157.3161.74
v2-r (final v2)4,264 initial + 2,000 failure-guided = 6,264Shared Stage 159.4463.71

Thus, Figure 4's 59.44 and 63.71 belong to v2-r. The correction concerns labels and checkpoint correspondence; the numerical scores remain unchanged.

Both v2 variants are independent full-parameter SFT branches retaining the initial 4,264 trajectories and adding 2,000. v1 supplies workflow feedback, not initialization parameters. Failure-guided expansion exceeds random expansion by 2.13 pp without RU and 1.97 pp with RU, at the same trajectory count and initialization. Relative to v1, its gains are +0.49/+2.79 pp, versus −1.64/+0.82 pp for random expansion. Adding the same number of trajectories therefore produces different outcomes depending on their selection.

The Harness, tools, and scoring protocol are matched, and RU-on comparisons use the same library snapshot. The advantage appears both without RU and with that fixed library. The 203-task workflow-feedback setting is clarified in Q8. We will align Table 21, Appendix C.4, and the plots, explicitly marking BioMaster-A1-v2 = BioMaster-A1-v2-r.

Reviewer_u2kcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BioMaster turns bioinformatics research execution runs into filtered trajectories for fine-tuning Qwen3.6-35B and structured ResearchUnits that agents retrieve when inference. A failure-guided loop creates new tasks for weak areas. The main claim of this paper is that training raises the value of Research Units. Across 203 tasks, the RU effect increases 5.58 points interaction. With Claude Opus 4.6 fixed, BioMaster leads on STELLA, Biotin-Eval1 and the overall averages. With the harness fixed, the post-trained model improves BioMystery by 9.26 points over the base model.

Critical strengths

This paper proposes a new view of how agents can learn from scientific execution experience, specially for the bioinformatics research. The author proposes that fine-tuning and retrieval can complement each other, and how much an agent gains from stored experience depends on how well the model can use it. The author also turn this idea into a direct test, measuring how much the benefit of a fixed RU library changes after training.

The paper proposes a new learning process build on a single set of execution runs. Instead of building training data and knowledge base separately, they make the same runs into both trajectories for fine-tuning and model and reusable methods for agent to retrieve during the work.

The author proposes a more usable format for storing experience of the agent. The research unit records what a method does, what it needs, when it applies and how it was verified. This can help the agent to decide whether a past method fits the new task.

Critical weaknesses

The main weakness is that the results don't appear to show that failure-guided learning helps. The reported v2 scores exactly match the random-expansion run, while the failure-diagnosis run scored lower.

The description of how the model is trained is not very clear. Stage 2 uses a different number of trajectories in the main text and in Table 18, and v2 starts from v1 in one place but from the Stage 1 checkpoint in another.

The claim in the paper that training and RUs work better together appears to be stronger than the evidence we can observe in the results. The RU benefit already appears at Stage 1, which is meant to teach tool-call syntax and harness interaction, so the model could have already learned how to operate the RU interface.

Questions for authors

Is the v2 reported in Figure 4 the random-expansion checkpoint or the failure-diagnosis checkpoint? How does this affect the failure-guided claim in this paper?

Does the interaction still appear with a randomly shuffled RU library? Does it still appear with an RU library from a separate source?

Are any of the 203 tasks or their clusters used for training or RU construction? How did the authors check for overlap between the training tasks or RU source and the evaluation tasks?

4 comments
AuthorsOpenReview ↗

Failure-Guided expansion improves performance, and RU gains transfer to a new library

Thank you for your thoughtful and constructive feedback. Your suggestions have helped us strengthen the manuscript and clarify points of confusion. We address your concerns below with corrections and additional results.

Q1. Final v2 and failure-guided expansion

Figure 4 reports BioMaster-A1-v2-r, our final model using reflection-guided expansion based on failure diagnosis. The suffix r denotes reflection-guided expansion, while d denotes direct random expansion without failure diagnosis. The correct mapping and results are:

CheckpointExpansion strategyRU offRU on
v2-dDirect random expansion57.3161.74
v2-rFailure-diagnosis-guided expansion59.4463.71

With the correct mapping, failure-guided expansion outperforms direct random expansion by 2.13 pp without RU and 1.97 pp with RU. This comparison therefore evaluates the two expansion strategies with matched trajectory counts and initialization, supporting the benefit of using failure diagnosis to guide additional data collection. We have corrected the corresponding descriptions and labels in new manuscript.

Q2. Training data and initialization

CheckpointInitializationTraining dataParameter update
Stage 1Qwen3.6-35B-A3B17,332 Harness-adaptation examplesLoRA
v1Shared Stage-1 checkpoint4,264 scientific trajectoriesFull-parameter SFT
v2-dShared Stage-1 checkpoint4,264 + 2,000 random = 6,264 trajectoriesFull-parameter SFT
v2-r (final v2)Shared Stage-1 checkpoint4,264 + 2,000 diagnosis-guided = 6,264 trajectoriesFull-parameter SFT

Stage 1 uses short, simple tasks for tool and Harness adaptation. Stage 2 uses long scientific trajectories to train multi-step reasoning and long-horizon execution. v1 and v2 share Stage-1 initialization; v1's failure diagnoses guide v2-r's data expansion.

Q3. Stage 1 and training-RU complementarity

Stage 1 already includes RU interactions. We agree that its contribution belongs in the explanation of complementarity.

AspectStage 1Stage 2: initial scientific training (v1)
Training dataSimple synthetic bioinformatics task executionsValidated complete scientific-task trajectories
Training samples17,3324,264
Samples containing RU calls5,7794,264
RU-call coverage33.34%100%
Parameter updateLoRAFull-parameter SFT
Training focusTool-call syntax, arguments, Harness interactionMethod selection, multi-step execution, verification, error correction

RU-call coverage counts samples with at least one RU call. Both stages use masked autoregressive training, with different data, RU exposure and parameter updates.

Using the same Harness and fixed RU library:

CheckpointRU offRU onRU gain (pp)
Base54.0252.71-1.31
Stage 156.8158.62+1.81
v158.9560.92+1.97
v2-r (final v2)59.4463.71+4.27

"RU off" refers to inference. Stage 1 yields a 1.81 pp RU gain. Stage 2 training on the expanded scientific-task dataset raises it to 4.27 pp in final v2 (+2.46 pp). Compared with Stage 1, v2 improves by 2.63 pp without RU and 5.09 pp with RU. Stage 1 provides the foundation; Stage 2 delivers the stronger task performance and RU benefits of the final model.

Q4. Random, irrelevant and independent-source RUs

Shuffling record order barely changes retrieval. We replace returned content only when retrieval is invoked, retaining call rules/budgets and matching RU count, format and length as closely as possible. Random RUs may be relevant by chance; irrelevant RUs exclude clearly relevant methods.

For RU condition cc, the interaction is:

Ic=(Sv2,c−Sv2,off)−(SBase,c−SBase,off).I_c=(S_{\mathrm{v2},c}-S_{\mathrm{v2},\mathrm{off}})-(S_{\mathrm{Base},c}-S_{\mathrm{Base},\mathrm{off}}).

Inference conditionBasev2Base RU gainv2 RU gainInteraction
RU off54.0259.44------
Original relevant RU52.7163.71-1.31+4.27+5.58
Random RU52.3859.61-1.64+0.17+1.81
Irrelevant RU52.5557.64-1.47-1.80-0.33
Independent-source relevant RU53.3763.22-0.65+3.78+4.43

These controls show that RU benefits transfer beyond the original library: independent-source RUs give v2 a 3.78 pp gain and a +4.43 pp training-RU interaction. Content relevance also matters: original and independent RUs outperform random RUs for v2 by 4.10 and 3.61 pp.

Random RUs mainly reveal reduced interference: v2 changes by +0.17 pp versus -1.64 pp for Base. Clearly irrelevant RUs still hurt both models. Together, these controls strengthen the evidence for training-RU complementarity: the positive interaction transfers to an independent library, and relevant experience provides greater benefits than random RU content.

AuthorsOpenReview ↗

All 203 Evaluation Tasks Are Held Out from Training, RU Construction.

Q5. Were the 203 tasks or their workflow clusters used for training or RU construction?

All 203 final evaluation tasks, their answers and execution traces were excluded from SFT and RU construction.

A scientific task can combine several workflows, such as cell-type annotation, differential expression and pathway enrichment. Clusters organize related workflow types, while each task specifies its scientific objective, inputs, target population, comparison design and outputs.

Our partition separates complete task instances while allowing shared workflows. Two tasks can both include differential expression while using different samples, addressing different questions and combining different additional analyses. Task overlap is assessed from descriptions, inputs, objectives and associated records.

As shown in the table below, this is an example of a shared workflow clustering cluster but with two different partitions for two tasks. QC: quality control; DE: differential expression.

DimensionTask A: Training/RU sourceTask B: Evaluation
TaskInfection-related monocyte transcriptional changes and pathwaysPre/post-treatment CD8+ T-cell states and clonal changes
Full objectiveCompare immune-cell composition in infected vs healthy donors; identify monocyte differentially expressed genes and associated functional pathways.Compare CD8+ subset composition and expression states pre/post treatment; track within-patient TCR clonal changes and their associations with cell states.
Source and inputsStudy A: peripheral-blood single-cell gene-expression count matrix; donor, sample, group and batch metadata.Study B: tumor single-cell gene-expression count matrix; TCR annotations linked by cell barcodes; patient, time-point and batch metadata.
Sample design12 infected and 12 healthy donors; donor-level between-group comparison.10 patients, each sampled before and after treatment; within-patient paired comparison.
Workflows(1) Single-cell QC and cell-type annotation; (2) immune-cell composition comparison; (3) monocyte DE; (4) pathway enrichment.(1) Single-cell QC, cell-type annotation and RNA-TCR matching; (2) CD8+ subset composition comparison; (3) paired DE and pathway enrichment; (4) TCR clonotypes and frequencies; (5) associations between clonal changes and cell states.
Shared workflowsCell-type annotation; cell composition comparison; DE; pathway enrichment.Cell-type annotation; cell composition comparison; DE; pathway enrichment.
Statistical designSubset monocytes; aggregate expression counts by donor; compare infected and healthy groups.Subset CD8+ cells or specified subtypes; aggregate expression counts by patient/time point for paired comparisons; track clonal frequencies within each patient.
OutputsCell-type annotation, composition and monocyte DE-gene tables; pathway enrichment results; analysis report.CD8+ subset composition, paired DE-gene and clonal-change tables; pathway enrichment and clone-state association results; analysis report.
Transferable knowledgeMetadata organization; target-cell restriction; DE; multiple-testing correction; result validation.Adapt and reuse these methods for new cell types, the paired-patient design and input data.
Independent contentStudy A's task prompt, input instances, comparison targets, results and execution traces.Study B's task prompt, input instances, comparison targets, results and execution traces.

Both tasks share annotation, differential expression and pathway analysis, with different objectives, inputs, designs and outputs. Task B also includes TCR and cross-modality analyses.

Task A's RUs can guide differential analysis or validation. Task B still requires its own data, targets, paired-patient design, execution and verification. This is the intended reuse of methods, while evaluation answers and traces remain separate.

We will clarify workflow coverage, cluster-based sampling and task-level data separation in the revision, and provide source and split records for all 203 evaluation tasks. The actual task comparisons will include task IDs, input-data identifiers and workflow memberships so readers can inspect which methods are shared and which task contents are kept separate.

Our evaluation tests whether models can apply learned or retrieved workflows to unseen scientific tasks.

Thank you again for your valuable feedback. We have addressed your comments through the clarifications, corrections, and additional results above. We welcome further discussion and would be happy to make additional revisions in response to any remaining questions or suggestions.

AuthorsOpenReview ↗

Additional Experiments Beyond the 203-Task Feedback Set

Thank you for your feedback on the evaluation. To test generalization on tasks separate from the 203-task feedback set, we added three direct comparisons between failure-guided and random expansion, one on each public benchmark.

We followed the fixed-Harness model-level protocol in Figure 3. Both variants started from the same Stage-1 checkpoint and added 2,000 trajectories to the same initial training set, using the same teacher for trajectory collection.

BenchmarkRandom expansion (v2-d)Failure-guided expansion (v2-r)v2-r minus v2-d
STELLA Simple Set84.5185.19+0.68 pp
Biomni-Eval151.9653.66+1.69 pp
BioMystery v1147.4150.37+2.96 pp

Failure-guided expansion scored higher on all three benchmarks, with the largest difference on BioMystery. The Biomni-Eval1 difference is calculated before rounding.

The 203 tasks informed which workflow clusters to expand. The task generator received only the selected cluster IDs. The original task prompts, reference answers, and execution traces were excluded from new task construction, supervised fine-tuning, and RU extraction (Appendix A.3). Prior work has used model failures to guide data generation [1, 2], compared failure-driven and random sampling at a matched data budget [3], and evaluated failure-guided training on a separate benchmark [4].

We will add these results to the revised manuscript and clarify how the 203 tasks were used. We would appreciate hearing whether these comparisons address your concerns and would be happy to discuss any remaining questions.

References

  1. He et al. (2023). Targeted Data Generation: Finding and Fixing Model Weaknesses. ACL.
  2. Stan et al. (2026). Learning from Reasoning Failures via Synthetic Data Generation. AAAI.
  3. Hao et al. (2026). HardGen: Generating Hard Samples for Tool-use Agents. EMNLP.
  4. Kang et al. (2026). TRACE: Capability-Targeted Agentic Training. ICML 2026 AIWILD Workshop.
AuthorsOpenReview ↗

Looking forward to further discussion

Dear Reviewer,

We are looking forward to your feedback and would greatly appreciate any further questions or concerns you may have.

If there are additional issues that require clarification or further experiments, an earlier response would help us better address them within the remaining rebuttal period.

Thank you again for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the discussion and manuscript revisions

Dear Area Chair and Reviewers,

Thank you for the time and effort you have devoted to reviewing our manuscript. We appreciate the reviewers’ recognition of the practical value of structured Research Units (RUs) and our controlled evaluation of scientific experience reuse. During the discussion, we clarified the training and evaluation protocols and completed additional experiments addressing the reviewers’ questions. We summarize the main contribution and updated evidence below.

1. Main contribution

BioMaster uses scientific execution experience in two complementary ways: validated trajectories support model post-training, while structured RUs preserve methods, applicability conditions, execution dependencies, and verification evidence for inference-time retrieval and reuse.

Our controlled evaluations quantify the benefits of the Harness, post-training, and structured RUs. On the 203-task component evaluation, the BioMaster Harness improves the unchanged Base model from 50.08 to 54.02 (+3.94 percentage points) with retrieval disabled. Within the same Harness, post-training improves performance over Base from 54.02 to 59.44 (+5.42 points), with retrieval disabled at inference. Enabling structured RUs for the final model further increases performance from 59.44 to 63.71 (+4.27 points). The complete configuration thus achieves a 13.63-point improvement over the untuned Base model in the standard execution environment, demonstrating the practical value of combining execution support, scientific post-training, and experience reuse.

2. Main concerns and additional evidence

ConcernClarification and additional evidence
Failure-guided learning beyond the feedback setWe completed direct comparisons against random expansion on three public benchmarks. Failure-guided expansion achieves higher mean scores on STELLA Simple Set (+0.68 pp), Biomni-Eval1 (+1.69 pp), and BioMystery v11 (+2.96 pp). Both variants use the same Harness, teacher, Stage-1 initialization, initial dataset, and number of additional trajectories.
Training configuration and checkpoint attributionWe clarified that final v2 is the failure-guided v2-r checkpoint. The initial scientific dataset contains 4,264 trajectories, and each expansion adds 2,000, yielding 6,264. All scientific-training runs initialize from the same Stage-1 checkpoint; v1 provides diagnostic feedback for v2-r.
Data boundaries and evaluation rolesWe explicitly identify the 203-task set as a workflow-feedback and component-evaluation set. Its prompts, reference answers, and execution traces are excluded from SFT, RU construction, and supplementary task generation. The new public-benchmark comparisons evaluate the expansion strategies outside this feedback set.
RU benefits beyond initial adaptationThe RU benefit increases from +1.81 pp after Stage 1 to +4.27 pp in final v2. Additional content controls examine random, irrelevant, and independent-source RUs. With the final model and Harness fixed, independent-source relevant RUs improve performance by +3.78 pp, supporting the usefulness of experience from an additional source.
Teacher dependenceSupplementary 4B experiments show gains of +3.61 pp with Kimi K3 and +1.97 pp with Qwen3.6-35B-A3B as teachers, relative to each configuration’s baseline.

Reviewer QEpE explicitly acknowledged: “My concerns about the training configuration and the apparent contradiction in Table 21 are addressed.” Their follow-up requested a direct comparison of expansion strategies outside the feedback set. We subsequently completed and reported this comparison on all three public benchmarks, directly addressing that request.

3. Manuscript revisions and request for assessment

The revisions clarify checkpoint attribution, trajectory counts, initialization, data boundaries, and RU exposure during training, and expand the analysis of experience reuse. These changes concern Sections 3.3–3.4 and 4.5, Appendix A.3, and Appendices C and E. The latest external-benchmark comparisons and GRPO results have been reported in the discussion and will also be incorporated into the manuscript.

BioMaster shows how scientific execution experience can be reused in model training and at inference time. Our experiments examine the gains from the Harness, post-training, and RUs, while the new public-benchmark results show that failure-guided expansion can improve performance beyond the feedback set. We hope other researchers can build on this work to develop and evaluate scientific agents that make better use of past execution experience. We would be grateful if you could consider these contributions and the additional results alongside the revised manuscript in your final assessment.

Thank you for your time and careful consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.