acceptodds
Under review as a conference paper at ICLR 2027

AutoBioLab: From Scientific Intent to Machine Executable Laboratory Automation

Abstract

While large language models (LLMs) can increasingly translate scientific requests into instrument commands, generating a sequence of executable commands does not guarantee the intended experimental state. We study this gap in automated liquid handling, where intermediate mixtures may be transferred, diluted, and reused across multiple steps, making later operations to depend on the material changes produced earlier. To bridge LLM reasoning with precise physical realities, we introduce AutoBioLab, an agentic system that fundamentally decouples the verification of scientific intent from robotic execution. By pairing LLM-driven planning with deterministic solvers, material state replay, separate scientific and robotic contracts, this agentic architecture convert abstract experimental designs into a verifiable physical state before any action is taken. Empirically, this decoupled approach more than doubles the end-to-end planning success rate across multiple state-of-the-art LLMs, raising the number of fully correct plans from 21.3 to 44.0 out of 48, which enhance the open-source model to approximate the SOTA performance of current commercial model. Furthermore, our runtime agent improves the successful recovery of human injected errors from 66.3% to 82.3%. Ultimately, AutoBioLab demonstrates that separating what an experiment should achieve from how a robot must move is essential for reliable, machine-executable laboratory automation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Scientific Agent, Laboratory Automation, Hardware-connected Agents, Model Hardware Standard, LLM Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_m93gOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AutoBioLab, a system that turns scientific requests into liquid-handling plans. An LLM proposes the experimental structure, deterministic solvers calculate quantities, and separate scientific and robotic checks guide repair. The main idea is that executable commands are not enough; the resulting volumes and concentrations must also match the request. The paper evaluates 48 planning tasks and a simulated fault-recovery suite. With Sonnet, planning success improves from 21.3 to 44.0 out of 48 tasks. Mechanism experiments support the benefit of joint quantity solving and requirement-aware checks. On 96 simulated fault scenarios per round, overall correct handling increases from 66.3% to 82.3%; this metric includes both successful recovery and appropriate handover. Robot and camera endpoints are simulated, so these results do not establish physical laboratory reliability.

Critical strengths

The paper targets experimental outcome correctness rather than command validity alone. Separately checking task-derived targets against replayed material states addresses a consequential failure mode: an agent can execute a self-consistent but incorrect experiment. This distinction gives the work practical relevance beyond improving command-generation success.

The paired solver experiment provides useful evidence for the central quantitative mechanism. Holding model replies fixed, joint solving improves complete-plan success from 2.50 to 9.25 out of 12 tasks. This is more informative than an end-to-end comparison alone because it directly tests whether accounting for coupled transfers corrects quantitative failures.

Critical weaknesses

The evaluator does not fully establish correctness relative to an independently fixed initial laboratory state. Appendix A.2 states that replay starts from the plan’s declared initial state, while source concentrations can be inferred from names or descriptions and reagent-specific inventory checks are incomplete without explicit bindings. It is unclear whether every reported success preserves the request’s initial conditions and reagent constraints. I am not claiming that any plan exploits this weakness, but an audit is necessary before treating the reported success rate as reliable evidence of scientific correctness.

The experiments do not sufficiently isolate the contribution of the proposed architecture. AutoBioLab receives more inference than direct-model baselines, and compiler behavior and repair budgets change across some ablation rounds. Solver-disabled conditions also retain prompts that delegate calculations to the missing solver. Without a frozen comparison against a capable agent using the same representation, compiler, and quantitative tools, the results cannot clearly distinguish the proposed mechanisms from additional computation and infrastructure. This also leaves the incremental contribution over related verification-and-repair systems insufficiently established.

Questions for authors

Q1: What checks ensure that plan-declared initial contents, source concentrations, and reagent identities match the original request? Can you report how many current successes remain when evaluation starts from a canonical, independently specified laboratory state?

Q2: Is material-state replay against task-derived targets repeated after every robotic command patch? Which command fields may be changed, and what prevents a robotically valid patch from changing the scientific outcome?

Q3: Which reported experiments use a fully frozen compiler, model configuration, and repair budget? Please verify the repetition counts and settings across Figure 3, Tables 11 and 13, and Appendix C.1.

Q4: Does the improvement remain against a tool-using agent given the same intermediate representation, compiler, solver access, and comparable inference budget? A controlled comparison would materially affect my assessment of the architectural contribution.

Q5: How many successful recovery episodes also satisfy the independent scientific task targets, rather than only matching the fault-free execution? Please also separate results on development plans from the eight later-introduced base plans.

Reviewer_PGKoOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper starts from the observation that generating a sequence of executable commands does not guarantee experimental state. AutoBioLab decouples the verification of scientific intent from robotic execution: an LLM proposes the experimental structure, deterministic solvers compute the quantities, and a robotic contract checks the compiled device commands. A ReAct-style controller handles runtime recovery with simulated robot and camera endpoints. On the BTB-48 benchmark, the authors report that this decoupled approach more than doubles the end-to-end planning success rate with Sonnet 5 and improves the handling of injected faults.

Critical strengths
  1. A well-posed problem. The paper starts from the observation that generating a sequence of executable commands does not guarantee the intended experimental state. And the paper shows that liquid handling is highly order-sensitive. This order dependence is a real difficulty for automated liquid handling.
  2. Separate scientific and robotic contracts. Because the two contracts operate on separate representations. I think this is a necessary step for bringing robots into the laboratory, since most current work on embodied agents in the laboratory only considers whether the robot completes its task.
Critical weaknesses
  1. Unclear benefit for stronger LLMs. On BTB-48, Opus 5, already solves almost all tasks. AutoBioLab itself is never run with Opus 5, neither in planning nor in runtime recovery. It is therefore unclear whether the framework still brings a gain as LLMs become stronger.

  2. Limited novelty. Letting the LLM propose the structure while external deterministic computation and checking handle the quantities follows the paradigm of PAL and LLM-Modulo. Laboratory systems such as AutoLabs and BioProAgent are already close to this design. These works are only discussed in the related work, without any experimental comparison, so it is hard to tell what's new about AutoBioLab.

  3. Rough writing and inconsistent statements. Some statements are inconsistent. Sec. 4.3 refers to Figure 2b for the module-group ablation, which is shown in Figure 2c. The caption of Figure 3d and the first sentence of Appendix B.4 describe three repair repetitions, while Table 13 reports two.

  4. No experiments on a physical robot. The paper presents itself as laboratory automation that separates scientific checking from robotic execution, and its contributions include physical error recovery and plans that satisfy robotics real-world requirements. However, no experiment is executed on a real liquid handler, and the paper shows no figure of the physical setup. Sec. 2.4 also mentions fixing a physical collision, but collisions are not among the bounds that are checked (Appendix A.2). The work therefore mainly focus on planning and material state replay in simulation, which does not fully match the contribution.

Questions for authors
  1. What is the performance of AutoBioLab with Opus 5, in planning and in runtime recovery? Does the framework still bring a gain for strong models that are already close to saturation on BTB-48?

  2. Can the authors compare AutoBioLab experimentally with related work that also let the LLM propose the structure and use external deterministic computation and checking, for example AutoLabs and BioProAgent on BTB-48?

  3. Can the authors add a baseline in which the LLM proposes the plan structure and the same deterministic compiler, including stroke splitting, lowers it to commands, but without solvers or scientific checks? This baseline would separate the contribution of the compiler from that of the scientific contract.

Reviewer_2sUMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AutoBioLab converts natural-language liquid-handling requests into plans whose material outcomes can be checked before execution. An LLM proposes the well layout and transfer structure. Deterministic solvers calculate quantities, including the extra liquid an upstream well needs to supply later transfers. A composition replay tracks the original liquids through each transfer. Targets extracted from the request guide quantity corrections and plan rewrites. The resulting plan is compiled into robot commands and checked for device and resource constraints.

The planning evaluation uses 48 tasks covering normalization, resource constraints, serial dilution, and multistage preparation. Across three collections, AutoBioLab with Sonnet 5 passes an average of 44.0 tasks, compared with 21.3 for direct Sonnet. Mechanism experiments replay fixed model replies with different solvers and apply different checks to copies of the same initial plans.

A simulated runtime maintains execution state and responds to injected faults. Each recovery round contains 96 scenarios, including cases requiring safe stopping or handover. With Sonnet, correct handling rises from 66.3% for the raw policy to 82.3% with AutoBioLab. The evaluation compares the resulting material state with a fault-free reference.

Critical strengths
  1. The system addresses a concrete planning problem: liquid prepared in one well may later be withdrawn to supply another, so satisfying each local dilution ratio can still leave incorrect final volumes. The fixed-reply experiment directly tests this dependency. Joint solving raises mean success from 2.50 to 9.25 of 12 tasks, while local solving leaves it at 2.50.

  2. The requirement-checking experiment tests whether plans satisfy targets extracted from the original request. Thirty-six initial plans are each shared across six solver/checker conditions. With solvers enabled, adding requirement checks to the basic checks repairs three additional task-runs without losses or repair-model calls. This gives concrete evidence for checking the requested outcome separately from the plan's declared outcome.

Critical weaknesses
  1. The title and opening frame the contribution as translating scientific intent into laboratory automation. The demonstrated task starts with specified liquid volumes and concentrations and produces plans for simulated liquid handling. The paper needs to explain how much of the broader problem this solves. The claim that separating scientific and robotic checks is essential also goes beyond the evidence from these particular system configurations.

  2. The added value over closely related planning systems remains unclear. The paper includes multiple models, Claude Code, and Coscientist, but leaves BioProAgent and AutoLabs to the related-work discussion. These systems already combine scientific tools or checks with correction. A focused comparison would help establish what the proposed quantity solving and state replay add. The substantial differences in inference budgets also limit what the overall comparisons tell us about the source of the gains.

  3. Planning and fault recovery are evaluated separately. Recovery success means matching the fault-free execution of a prepared plan, which can itself contain a planning error. The paper therefore leaves open how often its full workflow reaches the original requested outcome after a fault. With 48 planning tasks and recovery scenarios built from 14 plans, all evaluated in simulation, the evidence for practical laboratory reliability remains limited.

Questions for authors
  1. What capability does AutoBioLab add over the closest scientific planning systems, particularly BioProAgent or AutoLabs? A comparison on the same tasks would be useful. For the existing agent baselines, can success be compared at similar inference budgets?

  2. Can the planning and recovery stages be evaluated together, with the final state scored against the original task requirements? This would show how often the full workflow completes the requested experiment under injected faults.

  3. Which demonstrated result supports the broad claim about translating scientific intent into laboratory automation? Please clarify the contribution at the level of liquid-handling planning and explain its practical value for existing laboratory workflows.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.