acceptodds
Under review as a conference paper at ICLR 2027

H2L-Bench: A Benchmark for Agentic Planning Under Uncertainty in Drug Discovery

Abstract

AI agents have made rapid progress on tool use, multi-step planning and open-ended reasoning, yet the benchmarks that drive this progress rarely capture what matters in scientific decision making: acting under uncertainty, choosing between costly experiments, and committing to an answer with limited information. We introduce H2L-Bench, a lightweight decision-theoretic sandbox that simulates the hit-to-lead phase of small-molecule drug discovery as a budget-constrained sequential-decision task with a mechanistic ground truth. An agent chooses among eight assays of varying cost and information content, then nominates one compound from a library. Winning requires the compound's hidden traits to clear multiple thresholds simultaneously, with assay noise and budget scaled across three difficulty tiers. As a principled reference we implement a Bayesian value-of-information (VoI) agent that maintains an analytical Gaussian posterior over each compound's latent traits and picks the assay that most increases the best compound's probability of winning per dollar spent. Comparing VoI to four frontier LLMs (Claude Haiku 4.5, Sonnet 5, Opus 5, and OpenAI GPT-5.4) at 50 seeds per cell, we find that VoI has the highest win rate on every tier, but in paired same-seed tests its lead over the best LLM on each tier is not significant after correction for multiple comparisons. Beyond the headline win rates, the environment lets us analyse where LLMs and a Bayesian agent diverge action-by-action: on a subset of seeds, LLMs recover the correct compound by committing budget to a confirmatory profile of one compound, or to an expensive singular assay, that a depth-1 per-dollar VoI ranking keeps deferring. An approximate two-step lookahead does not close this gap on fresh seeds, so the shortfall reflects VoI's per-dollar scoring rule rather than its planning horizon. Test-time scaffolds do not help: on the hard tier, chain-of-thought and tree-of-thoughts leave Sonnet 5's win rate statistically unchanged, and explicit planning lowers it (32% to 23%).

Primary area:
datasets and benchmarks
Keywords:
LLM agents, benchmarks, value of information, Bayesian experimental design, drug discovery

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wjMcOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces H2L-Bench, which is a test environment for sequential experimental decisions under uncertainty, focusing on small-molecule hit-to-lead discovery. In this task, an agent has a fixed budget to run 8 different assays on 24 compounds, and finally picks one compound that meets several hidden thresholds. The authors use an approximate Bayesian value-of-information (VoI) policy as a reference, and compare it with four LLMs, some heuristic/learned baselines, and a few inference-time scaffolds. The VoI method gets the highest win rate in each difficulty tier, but its advantage over the best LLM is not statistically significant after correction. Also, the paper argues that LLMs sometimes succeed by focusing confirmatory tests on good compounds, while the myopic, cost-normalized VoI policy spreads its budget too wide.

Critical strengths
  1. I think this is very relevant to practical drug discovery problems. By combining noisy observations, assay artifacts, different costs, and strict budgets, it captures real problems in scientific experimentation that normal benchmarks often miss.
  2. The Bayesian VoI policy provides a very useful baseline for comparison beyond just LLM rankings. The authors are also very honest about its privileged knowledge of the generative model, treating it as a reference instead of claiming it is an optimal policy.
Critical weaknesses
  1. The explanation for the mechanism is a bit too strong based on the current evidence. The task uses a fixed list of candidates and just asks the model to choose which experiments to run. It doesn't really test if the model can design new molecules or understand how molecular structures affect activity. Therefore, the main question is whether the model's success just comes from making early commitments when new evidence shows up. Although VoI performs worse and LLMs perform better in some cases, it doesn't prove this is the exact cause. Maybe LLMs just make stronger commitments because early evidence is clearer, not because their strategy for allocating experiments is actually better.

  2. The depth-2 comparison is not very clear for testing the planning horizon. Appendix I changes many things at the same time—such as which actions are removed, whether batch screening is available, how accurate the sampling is, and when the process stops. It also measures two-step planning using only the cost of the first action. Because of this, its worse performance doesn't clearly prove that cost normalization is why VoI fails. The problem could also come from limited lookahead or approximation choices.

  3. The PK case study does not support the explanation very well. Section 5.2 mentions that in_vivo_pk addresses a difficult axis in antibiotic-medium. However, Table 4 looks at potency, selectivity, and metabolic stability, while PK measures oral bioavailability and half-life. PK might help confirm basic performance, but the paper needs to fix this explanation and show evidence of how these results actually changed the final candidate selection.

Questions for authors
  1. For the depth-1 versus depth-2 comparison, could you use the same action sets, stopping rules, and numerical-accuracy standards in both conditions, and explain how the costs of follow-up evaluations are accounted for?
  2. In the antibiotic-medium experiment, which specific constraint did PK resolve, and did the resulting observations actually change the final nominated compound?
2 comments
AuthorsOpenReview ↗

Response to Reviewer wjMc (part 1 of 2)

Thank you for the thoughtful and careful review. All three weaknesses pointed to real problems, and we ran the experiments needed to address them. The revised PDF is uploaded; numbers below refer to it. We believe this has greatly strengthened the manuscript.

PK in antibiotic-medium. You are right, and our explanation was wrong. in_vivo_pk reads only oral bioavailability and half-life. In antibiotic-bacterial these are easy floors, not hard axes (Table 5, formerly Table 4), and the generator draws them independently of every other trait. So PK resolves no hard constraint: it checks two floors that no other assay reads.

To check this, we went through the 8 seeds behind Figure 3 (right), where GPT-5.4 won and VoI lost:

  • GPT-5.4 ran PK in 6 of the 8 games, each time on the compound it went on to nominate (the true winner), within its last four actions. No readout fell below a floor, and PK never changed the nominee. Two of the 8 games were won without PK.
  • VoI's nominees in these games failed hard axes, which PK does not measure, so running PK would not have changed any of VoI's 8 outcomes.
  • Over all 50 games in this cell, GPT-5.4 made 41 PK calls, 38 of them on its eventual nominee.

PK is therefore the last step of fully profiling the lead, not a decisive assay. The win rates are unaffected; only the explanation changes. We rewrote Strategy 2 in Section 5.2 as "complete the lead's profile", corrected the Figure 3 caption and label, and removed the claim from the abstract.

W2 / Q1: depth-2 comparison. We agree that the original comparison did not isolate the horizon. We ran five conditions that change one factor at a time, on 30 fresh seeds (200–229) in each of the two cells (ppi-hard and antibiotic-medium), 300 games in all (Appendix I, Tables 20–21):

ConditionWins (of 60)
C1: depth-1, as in the paper46
C2: depth-1 with depth-2's approximations (same action set without the batch screen, same shortlist and stopping rule, 16 imagined outcomes)34
C3: depth-2, as in the paper37
C4: C3 with depth-1's spend-down fallback37
C5: C3 with the follow-up priced: gain divided by cost(a₁) + E[cost(a₂*)]35

C2 and C3 share action sets, stopping rule and Monte Carlo accuracy, so they differ only in horizon, as Q1 asks. Paired exact McNemar tests, Holm-corrected over the four contrasts:

  • Approximations (C1 vs C2): 17 vs 5 discordant pairs, p = 0.017 (0.068 after correction).
  • Horizon (C2 vs C3): 12 vs 15, p = 0.70. The 95% interval for the difference is −12 to +22 points, so moderate effects in either direction are not ruled out.
  • Stopping rule (C3 vs C4): the same nomination in all 60 games.
  • Pricing the follow-up (C3 vs C5): 13 vs 11, p = 0.84.

On the second part of Q1: in the original depth-2 agent (C3), the follow-up must be affordable after the first action, but its cost does not enter the denominator. C5 charges it, and that does not help.

Depth-2's shortfall therefore comes from the approximations needed to make it tractable, not from the extra step, and nothing here shows that cost normalisation is why VoI loses. We removed that claim from the abstract, Section 4 and Appendix H. Appendix I now reports the controlled comparison, and states that depth-2 uses 16 imagined outcomes of each first action and 64 of each follow-up.

AuthorsOpenReview ↗

Response to Reviewer wjMc (part 2 of 2)

W1: mechanistic explanation. We agree that the trajectory analysis showed an association, not a cause. We ran two tests (Appendix O):

  • Is it the early evidence? For each hard-tier game of base Sonnet 5 and Opus 5 (250 each), we replayed the agent's own observations through VoI's posterior up to its first confirmatory assay, and measured how clear the evidence was at that point: the P(win) gap between the two leading compounds, and the leader's share of total P(win). In logistic regressions of winning with archetype fixed effects, commitment's coefficient does not change when either measure is added (Sonnet 5: +0.58 per SD in all three models; Opus 5: +0.70, +0.67 and +0.70; all p < 0.001), and neither clarity measure predicts winning (all p > 0.19; Table 27). Within every tercile of the P(win) gap, games with above-median commitment win more often.
  • Does imposing commitment help? We built a VoI variant that, from its first confirmatory choice on, runs confirmatory assays only on its top three compounds by P(win), and ran it against standard VoI on 250 fresh hard-tier games (seeds 200–249). It concentrated its budget as intended (0.58 of confirmatory spend on the top compound, the same as base Sonnet 5, spread over 2.8 compounds instead of 4.8) but did not win more often: 39.6% against 42.8% (p = 0.31; paired difference −3.2 points, approximate 95% CI −8.6 to +2.2; Table 28).

Commitment thus predicts the LLMs' wins after controlling for how clear the early evidence was, but imposing it on VoI does not help. We now describe it as a behavioural marker of the LLMs' successful games, not a cause, and revised the abstract, Section 5.3 and the conclusion accordingly. One explanation consistent with both results, which we have not tested, is that the LLMs concentrate their budget on compounds whose confirmatory results come back favourable, so that concentration follows good evidence rather than producing it.

We also agree that the task does not test molecule design or structure–activity reasoning. This was already stated in the limitations, but we have now also stated it in the introduction.

Thank you again for the specific questions; the controlled depth comparison in particular changed what the paper claims.

Reviewer_gKRtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces H2L-Bench, a synthetic environment that casts the hit-to-lead phase of drug discovery as a budget-constrained sequential decision task: an agent decide eight assays of different cost on the given candidate compounds, then nominates one. A depth-1 Bayesian value-of-information (VoI) agent that knows the generative model serves as the reference, and four frontier LLMs are compared. VoI has the highest win rate on every tier, but its lead over the best LLM is not significant after Holm correction.

Critical strengths

S1. The environment is fully specified (Table 1, Appendices A, E, F). So I think it could be reimplemented from the paper alone.

S2. The statistical analysis is solid and appropriately cautious. The paired exact McNemar tests with Holm correction (Table 8) don't show significant results, and the paper clearly states about that.

Critical weaknesses

W1. The reference comparison is asymmetric. VoI knows the trait priors, noise scales and artifact rates, while the LLMs receive only qualitative noise descriptions. VoI is also depth-1, so a near tie is hard to interpret as evidence of comparable decision-making ability.

W2. The work needs a clearer comparison with novelty of prior work. Without compound structure and with one winner in each medium or hard library, the task resembles budgeted decision making in prior work. Chen et al. [1] and BioDiscoveryAgent [2] also study sequential experiment selection but are not cited.

W3. The benchmark provides no repository link or code-release commitment. The reproducibility statement describes how to reimplement, but this makes independent reproduction and reuse harder.

References: [1] Chen et al., 2026, "Case-Guided Sequential Assay Planning in Drug Discovery", arXiv:2601.14710 [2] Roohani et al., 2025, "BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments", ICLR 2025

Questions for authors

Q1. Could you clarify the novelty of H2L-Bench relative to Chen et al. [1], who also plan sequences of drug-discovery assays?

Q2. Section 4 gives VoI the noise scales and priors. Would Opus 5's win rate change if its prompt included the same quantitative parameters?

Q3. The evaluation reaches 96 compounds. Could you assess scalability to larger libraries in hit rate, runtime, and model inference cost?

2 comments
AuthorsOpenReview ↗

Response to Reviewer gKRt (part 1 of 2)

Thank you for the review and the concrete questions. We ran new experiments for Q2 and Q3 and added the comparison with prior work. The revised PDF is uploaded; numbers below refer to it (Table 1 is new, so the submission's Table 8 is now Table 9).

W1 / Q2: the asymmetric comparison. We agree. VoI is a reference rather than a competitor (Section 4), and a gap that is not significant is not evidence that the LLMs decide as well as VoI does. We added two analyses.

Q2: Opus 5 with VoI's parameters (Appendix R). We appended to base Opus 5's system prompt the parameters VoI uses, generated from the environment's own constants: every trait prior and archetype override, the lipophilicity coupling, the artifact rate, the potency-to-IC50 map, the screen model and each assay's noise model. On 50 hard-tier games (seeds 0–9 in each archetype), paired with the base agent, it won 19, the same as without the parameters (10 vs 10 discordant games, p = 1.00); VoI won 22 (p = 0.65). The parameters did change how Opus 5 played: it ran fewer in-silico predictions (16.7 vs 23.5 per game) and more single-shot screens (1.6 vs 0.3) and dose-response assays (1.1 vs 0.4), closer to VoI's pattern, but it did not win more. With 50 games the 95% interval on the difference is about ±17 points, so this rules out only a large effect. With the parameters, Opus 5 reasons much longer, so we raised its output cap from 4,000 to 16,000 tokens to avoid truncated replies.

How large could VoI's lead be? (Table 10). For each tier and LLM we now report VoI's paired lead in win rate with a 95% confidence interval. Over the best LLM in each tier the lead is 6.4, 4.8 and 2.8 points (easy, medium, hard), and the data rule out leads larger than about 12 points; on the hard tier they also allow Opus 5 to be ahead by up to 5 points.

On VoI being depth-1: in a controlled comparison (Appendix I), adding one step of lookahead to VoI, with action set, stopping rule and Monte Carlo accuracy matched, changed little (37 vs 34 of 60 games, p = 0.70).

Q3: scalability (Appendix P, Table 29). Across the 8–96-compound sweep of Section 5.4 (kinase-oncology / medium, a budget of 35,000 dollars, 50 seeds per point):

  • Runtime. VoI's time per game grows roughly as N^1.7, from 13 s at 8 compounds to 15 minutes at 96 on one laptop core, and 45 minutes at 192.
  • Inference cost. LLM input tokens per game grow 4–10-fold from 8 to 96 compounds (Sonnet 5 56k to 232k, Opus 5 56k to 544k, GPT-5.4 39k to 322k), because the prompt describes every compound and each call re-sends the game history; output tokens grow 3–9-fold. A GPT-5.4 game at 96 compounds spends about 17 minutes waiting on the model.
  • Hit rate. VoI degrades slowest (54% at 96 compounds, against 16–28% for the LLMs; Figure 5). At 192 compounds VoI wins 28%. At a fixed budget, a larger library becomes mostly a screening problem, since only a few compounds can receive confirmatory assays, so every agent eventually degrades.
AuthorsOpenReview ↗

Response to Reviewer gKRt (part 2 of 2)

W2 / Q1: novelty relative to Chen et al. and BioDiscoveryAgent. Thank you for both references. Section 2 now cites and discusses them, and Table 1 compares them with H2L-Bench (reasons for each entry in Appendix N).

  • Chen et al. (IBMDP) is the closest prior work we know of, and shares our framing of cheap, weakly informative assays against expensive, decisive ones. It is a model-based RL method that plans the assay sequence for a single candidate compound, using a tempered Bayesian belief over similar historical compounds and ensemble MCTS. Cost enters the reward rather than a hard budget, and planning stops once uncertainty about the candidate is low enough. It is compared with a rule-based heuristic on real central nervous system (CNS) compounds, where the property being estimated is brain penetration, and with value iteration in a synthetic environment; no LLMs are evaluated. In H2L-Bench, by contrast, the agent spends a hard dollar budget across a library of 24 compounds and must nominate one; each assay has its own noise and artifact model with known parameters; and the purpose is to evaluate LLM agents against a Bayesian reference.
  • BioDiscoveryAgent has an LLM agent choose which genes to perturb over a fixed number of rounds of fixed size (e.g. 5 rounds of 128 genes), with tools such as literature and gene search. Experiments are not priced, the agent does not choose between experiment types, outcomes are looked up from completed screens, and the baselines are Bayesian-optimisation acquisition functions and random sampling; there is no reference policy. H2L-Bench compounds carry no structure or identity, so prior knowledge cannot help, and performance reflects how the agent uses the evidence it buys.

More generally, we agree that the task is structurally fixed-budget best-arm identification with constraints and heterogeneous measurement costs (Section 2). This is deliberate: a problem with known structure is what makes a Bayesian reference computable. Our contribution is not a new class of decision problem but a benchmark that measures how LLM agents handle it, relative to that reference. On the single winner: every one of the 250 medium and 250 hard libraries in the evaluation has exactly one compound that clears all thresholds (Appendix B, Table 6), because the generator resamples until at least one does, and under these priors a second is rare.

W3: code release. We will release the code, seeds, configurations and game transcripts upon acceptance.

Thank you again for the specific questions, which led to the two new analyses above, and which we believe strengthen the contribution.

Reviewer_XpY5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors introduce H2L-Bench, a benchmark measuring the hit-to-lead selection capabilities of LLMs. The authors also introduce a classical Bayesian value-of-information policy which maintains a Gaussian posterior over each compound's hidden assay results.

Critical strengths

None

Critical weaknesses

I question the novelty and utility of the H2L-Bench benchmark as a whole. What kind of LLM-specific capabilities does it measure beyond already-released benchmarks such as SMDD-Bench [1], TDC [2], and PMO [3]? The notion of exploring a set of chemicals/chemical space, using a limited number of potentially noisy assays/oracle scoring functions, and reasoning over molecular modifications is not novel and has been explored in previous benchmarks.

Furthermore, the agent scaffolds used are extremely outdated and no longer used:

  • "CoT asks the model to write its reasoning before the action" (line 224)
  • "Tree-of-Thoughts..." (line 225)
  • "Planner ..." (line 226)

Reasoning LLMs are trained to naturally output a reasoning trace before answering -- prompting via CoT is no longer necessary as the authors do not evaluate on base LLMs that would require such prompting.

Given the heavy analysis applied on tree of thoughts, chain of thoughts, and planner-style agent scaffolds, I do not believe it is possible to re-evaluate on more modern scaffolds such as Codex or Pi and rewrite the paper within such a short rebuttal period. Therefore, I recommend a clear reject.

Minor points: The overall writing style feels AI-generated and is very hard to understand.

[1] https://arxiv.org/abs/2605.21740 [2] https://tdcommons.ai/ [3] https://proceedings.neurips.cc/paper_files/paper/2022/file/8644353f7d307baaf29bc1e56fe8e0ec-Paper-Datasets_and_Benchmarks.pdf

Questions for authors

See above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer XpY5 (part 1 of 2)

Thank you for the review. We respond to the two main concerns, the agent scaffolds and novelty, and to the point about the writing. The revised PDF is uploaded; numbers below refer to it.

Agent scaffolds. We think there is a misunderstanding about which agents the main results use. The headline LLM results (Table 3, Figure 2, Sections 5.1, 5.2 and 5.4) do not use CoT, ToT or Planner prompting. Sonnet 5 and Opus 5 run with their native adaptive extended thinking, and GPT-5.4 with medium reasoning effort (Section 4; lines 210–223 of the submission). Each turn is one native reasoning step followed by one action, as you recommend. The scaffolds appear in a single subsection (5.3, details in Appendix K), which asks whether adding explicit structure on top of a reasoning model helps. Its chain-of-thought result supports your point: Sonnet 5 already reasons for about 930 tokens per turn, CoT's total output is about the same (990 tokens, about 150 of them visible), and its win rate is unchanged (33% vs 32%). ToT and Planner test something native reasoning does not provide, namely control structure: proposing and scoring several candidate actions, or committing to a multi-step plan in advance. Planner significantly lowers the win rate (32% to 23%, p = 0.033).

A current agent harness, run during the discussion period. You suggested re-evaluating with a modern harness such as Codex or Pi, and doubted this could be done within the rebuttal period. We ran it (Appendix Q). We exposed the game actions as tools to Claude Code, an off-the-shelf coding-agent harness, gave the model a sandboxed Python interpreter (numpy, scipy, pandas) that cannot see the benchmark code or the hidden scenario, and played Opus 5 on all 250 hard-tier games, paired by seed with the base agent and VoI:

Hard tier, 250 paired gamesWin rateExact McNemar p, against the harness
Opus 5 in Claude Code, with Python42.4%—
Opus 5, our agent loop (main results)40.0%0.50
VoI42.8%1.00

Opus 5 wins about as often in Claude Code as in our own agent loop (42.4% vs 40.0%, p = 0.50), and as often as VoI (42.8%). The model used Python in 89 of the 250 games, and in 13 of them wrote an explicit Bayesian update. The median code cell ran after 97% of the game's actions: the model used code to choose which compound to nominate, not which assay to run next. Our conclusions therefore do not depend on our own agent loop. We did not run Codex, which would need a separate OpenAI deployment, or Pi.

AuthorsOpenReview ↗

Response to Reviewer XpY5 (part 2 of 2)

Novelty relative to SMDD-Bench, TDC and PMO. We agree that these benchmarks share our motivation that measurements are scarce; SMDD-Bench explicitly treats its oracle calls as stand-ins for wet-lab assays. The difference is in what the agent decides and how success is defined:

  • SMDD-Bench evaluates LLM agents on molecule design tasks (pharmacophore identification, scaffold hopping, lead optimisation and others) that require chemical and 3D structural knowledge. The agent proposes molecules under separate call limits for two oracles (Boltz-2 and ADMET-AI), and the same oracles score its submission.
  • PMO measures the sample efficiency of molecular optimisation methods under a budget of 10,000 calls to a single oracle per task, which is also the objective.
  • TDC is a collection of datasets, learning tasks, oracle functions and leaderboards. Its one budgeted task, a docking benchmark, limits calls to a single docking oracle while generating molecules.

In all three, the agent decides which molecule to propose or query next, and the oracle that measures a molecule also defines success. None puts different measurements on a common budget, so none requires trading a cheap, noisy measurement against an expensive, more decisive one.

In H2L-Bench the library is fixed and compounds carry no chemical structure, so there are no molecular modifications to reason about; the agent decides which measurement to buy and when to stop. Its eight assays read different traits, cost from 50 to 8,000 dollars and differ in noise and artifact behaviour, all under one dollar budget. The assays are not the success criterion: a compound wins only if its hidden traits clear every threshold, so the agent must judge when noisy, possibly biased evidence is strong enough to commit to one compound. Because the generative model is fully specified, we can compute a Bayesian reference policy and compare LLM decisions with it action by action, which is what the analyses in Sections 5.2–5.4 rely on. We rewrote Section 2 accordingly and added Table 1, which compares H2L-Bench with SMDD-Bench, PMO, TDC, BioDiscoveryAgent, DiscoveryWorld, BoxingGym and IBMDP on seven properties, including whether measurements are priced, noisy and hidden and whether there is a reference policy. Appendix N gives the reason for each entry.

Writing. We take this criticism seriously. As the paper's AI use statement says, we used generative AI tools to help edit the manuscript, and the authors reviewed all of the text. If particular passages were hard to follow, we would be grateful to know which, and we will aim to improve the writing and clarity.

Thanks for the replies. I do not believe my concerns have been addressed whatsoever.

ToT and Planner test something native reasoning does not provide, namely control structure: proposing and scoring several candidate actions, or committing to a multi-step plan in advance. Planner significantly lowers the win rate (32% to 23%, p = 0.033).

Both methods are quite outdated. This does not address my previous point.

In H2L-Bench the library is fixed and compounds carry no chemical structure, so there are no molecular modifications to reason about; the agent decides which measurement to buy and when to stop. Its eight assays read different traits, cost from 50 to 8,000 dollars and differ in noise and artifact behaviour, all under one dollar budget. The assays are not the success criterion: a compound wins only if its hidden traits clear every threshold, so the agent must judge when noisy, possibly biased evidence is strong enough to commit to one compound. Because the generative model is fully specified, we can compute a Bayesian reference policy and compare LLM decisions with it action by action, which is what the analyses in Sections 5.2–5.4 rely on. We rewrote Section 2 accordingly and added Table 1, which compares H2L-Bench with SMDD-Bench, PMO, TDC, BioDiscoveryAgent, DiscoveryWorld, BoxingGym and IBMDP on seven properties, including whether measurements are priced, noisy and hidden and whether there is a reference policy. Appendix N gives the reason for each entry.

I also do not believe compounds containing no chemical structure, noisy measurements, hiding prices, etc. are sufficient novelty for a benchmark beyond what has already been released. I maintain my score of reject.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.