- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper argues that the acquired evidence in sequential experiment selection influences not just the immediate discovery yield but also the follow-up batches in subsequent rounds. The proposed pipeline comprises an evidence-use decision step which highlights the guiding question for the next batch and selects relevant evidence from the whole record to substantiate it. The agent then compares the potential follow-ups based on cost, pre-requisites etc. and commits a batch for measurement. The method is evaluated on molecular hit discovery, protein library design, and two drug-combination settings, against numerical baselines and two agentic baselines based on the DeepSeek-V4.1-Flash backbone. EvoLab reports higher mean final discoveries than all baselines with lower token consumption.
Critical strengths
Following are the critical strengths of the paper:
- Well formulated problem statement: Realistic to have different costs for different actions with ordering dependence between potential actions. Not every past outcome is necessary to observe at each round, and thus focused evidence becomes important.
- The full information retrospective oracle analysis is interesting and sheds light on how the agent performance changes as rounds progress, highlighting issues with exploration/exploitation.
Critical weaknesses
- The paper presentation is very confusing:
- Terminology related to experimental tasks (panels, grids etc.) is used extensively in the method section without being introduced.
- The paper talks about selecting relevant evidence to pass into the context but section 3.2 mentions all acquired raw records are presented.
- Towards the end of the manuscript, authors differentiate two versions of the method, single loop and explicit-review, without talking about it in the text before.
- Issues with experiments:
- The results are only provided with DeepSeek-V4.1-Flash, where the questions of robustness to LLM choice remains unanswered.
- Section 4.3 digs deeper into breaking down the allocation trajectory without providing actual evidence of the hypotheses or the observations specified.
- Several captions specify the methods worked on different cohorts and model identifiers, which breaks the true paired comparison between methods and raises concerns about the conclusions drawn.
Questions for authors
Clarification Questions:
- Do the BioDiscoveryAgent and ReAct baselines receive exactly the same amount of information in the prompt as EvoLab?
- Table 1 is interesting as it shows how the agent performance compares to a full information oracle. Is there a reason why round 5 has such a high loss? Also, the table caption says the numbers are the lowest loss per round. Are the numbers across rounds corresponding to a single run for each method? Or do they correspond to different runs, wherever the lowest loss happened?
- In general, Figure 3 and Table 1 both mention that some numbers are with different ‘cohorts’. Please clarify what does it mean and why is it not exactly paired with the same seeds and model identifiers? If different seeds, could you provide results with the same seeds across multiple runs?
- Why are ablations done on the explicit-review variant and not the single-loop variant used in the main comparison? How do the two variants differ?
Follow-up Experiments:
- Is there any supporting evidence of the decompositions explaining agent behavior mentioned in Section 4.3?
- There is a lack of clarity about what actual prompts look like. Including a single round of LLM trace would have helped. For example, how is the evidence provided in the prompt, how many past records are selected at max, how many follow-ups are compared due to token constraints, is there 1 follow-up per question?
- Results on more LLMs will solidify the generality of the claims. Please provide results on at least 1 more LLM.