acceptodds
Withdrawn from ICLR 2027

Agentic Selection Without Feedback: An LLM Agent Designs Cancer Panels by Knowing What Not to Sequence

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Designing the target panel for a cell-free DNA cancer assay is a budgeted selection problem with no execution feedback. The panel fixes which genomic regions are sequenced, it dominates development cost, and its only test is a clinical study that arrives after the design is frozen. Current practice mines labeled case-control cohorts, so panels inherit the population they were trained on and can degrade in populations with a shifted cfDNA background, such as patients under with chronic liver disease, where labeled positives are scarce. We ask whether selection from prior information alone can compete with supervised design. We present an LLM agent that constructs a multi-cancer methylation panel for a chronic liver disease population from three inputs, the cancer literature, a reference cell-type methylation atlas, and unlabeled plasma from cancer-free donors in the target population. No tumor sample and no detector response is available at any design stage. On a titration benchmark with 700 background controls and a single fixed detector, the frozen panel reaches 0.7323 AUC against 0.7050 for the strongest commercial panel at matched sequencing cost, and exceeds all comparators under paired tests. Ablations suggests the gain is a function of the panel as a whole and not any specific markers. The agent's contribution is the systematic exclusion of weak regions, which requires prior knowledge but no outcome data. The agent were highly reproducible with any two of five independent runs share 86% to 92% of their selected CpGs. The agent provide unique region when substituting the target population shifts selection far beyond run-to-run variation (Jaccard index 0.73 vs. 0.92) supporting on demand per population panel design. These results suggest that, for budgeted decisions made without feedback, an agent's value lies in reliably rejecting poor candidates rather than in identifying the best ones.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Agentic Discovery, Liquid Biopsy, Cancer Screening

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rfFeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates a practically important problem: how to design cancer detection panels under a limited genomic coverage budget when no feedback from experimental execution is available. The authors propose an LLM-based agent for designing a multi-cancer DNA methylation detection panel for a chronic liver disease population. Instead of using cancer-positive samples or feedback from a downstream detector, the agent integrates three types of information: cancer-related literature, a reference DNA methylation atlas comprising 39 cell types, and unlabeled plasma data from cancer-negative donors in the target population. Through literature retrieval and biological claim extraction, construction of genomic methylation patterns, and detectability validation in the target population context, the agent iteratively generates and filters candidate regions and ultimately determines the panel under a given genomic coverage budget.

The authors evaluate the designed panel using an in-silico titration benchmark, in which public WGBS data from tumor and adjacent normal tissues are spiked at different tumor fractions into a cancer-negative plasma background from 700 donors with chronic liver disease and compared using a unified downstream detector and donor-disjoint cross-validation. In this experiment, the reference agent-designed panel achieved an AUC of 0.7323, higher than the commercial TWISTALLIANCE panel with comparable sequencing cost (AUC 0.7050), and outperformed other comparison methods. Further ablation experiments and stability analyses indicate that the agent's advantage does not mainly come from a few high-performance markers, but more likely from the systematic exclusion of weaker candidate regions. The authors also find that when run repeatedly on the same target population, the panels generated by the agent show high consistency, whereas changing the target population background leads to more pronounced changes in panel composition. Based on these results, the authors argue that in an open-loop scenario lacking positive samples and experimental feedback, an LLM agent can still use domain knowledge and unlabeled background data for effective budget-constrained selection, and that its main value may lie in reliably excluding candidate regions that are not worth spending sequencing budget on.

Critical strengths

It proposes a novel and practically significant research question. This paper focuses on budget-constrained panel design without cancer-positive samples or experimental feedback, distinguishing it from the usual closed-loop agent framework of “experiment–feedback–reoptimization.” This setting is highly realistic, especially for biomedical scenarios where positive samples are scarce, experimental costs are high, and panel design must be frozen in advance. The paper clearly formalizes this problem and explores how much of a role an LLM agent can play in such an open-loop setting, which is one of the paper’s most important conceptual contributions.

The experimental evaluation is fairly systematic and includes targeted analysis of the sources of the agent’s gains. The authors not only compare the agent-designed panel with commercial and academic panels, but also evaluate them using a unified downstream detector, donor-disjoint cross-validation, and the same simulated mixtures, thereby separating, to some extent, the contribution of the panel itself from the influence of the downstream detector. In addition, through multiple independent runs, input ablations, and subpanel analyses, the authors find that the performance improvement does not obviously depend on a few specific markers, but is more likely associated with the systematic exclusion of weaker candidate regions. This mechanistic analysis enables the paper not only to report performance improvements but also to attempt to explain why the agent can gain an advantage without positive-sample feedback.

Critical weaknesses
  1. The contribution of “agentic selection” is not cleanly isolated. The strongest non-agent baseline, DETERMINISTIC-MERGE, combines the ASSOCIATION-ONLY and DETECTABILITY-ONLY panels, but it does not operate on the same candidate pool generated by the LLM agent. Therefore, the agent’s improvement over this baseline (0.7323 vs. 0.7047 AUC) cannot distinguish whether the gain comes from the LLM’s reasoning/selection ability or from generating a different and potentially better candidate pool. Since demonstrating the value of agentic selection is one of the paper’s central claims, the absence of this control substantially weakens the causal interpretation of the main results.

  2. The claimed budget-matched comparison is unclear because several agent panels exceed the 2.3 Mb budget. The paper formulates the problem under a 2.3 Mb coverage constraint, yet several agent runs are reported at 2.37–2.46 Mb, with AGENT-ALD reaching 2.72 Mb and AGENT-NOATLAS reaching 3.03 Mb. This raises concerns about whether these comparisons are truly matched in sequencing/capture cost. Since budget-constrained selection is itself a central premise of the paper, this issue is particularly critical. The paper needs to clearly explain the discrepancy and demonstrate that all primary comparisons satisfy an equivalent effective cost constraint.

Questions for authors

Regarding the 2.3 Mb budget constraint: The paper defines 2.3 Mb as the panel design budget, but in Appendix A several agent-generated panels have actual sizes exceeding this budget—for example, Run 2/4 is 2.42 Mb and Run 5 is 2.46 Mb, while AGENT-ALD and AGENT-NOATLAS reach 2.72 Mb and 3.03 Mb, respectively. Can the authors clarify the specific definition of the 2.3 Mb budget here and why these panels are still considered to have comparable sequencing costs? If these panels in fact have a higher capture/sequencing footprint, does the paper’s main performance advantage still hold under a strictly identical budget?

Regarding the conclusion about population-specific panels: The authors replace the target plasma background from CLD with ALD, observe a clear change in panel composition, and on this basis argue that the agent can adapt to different target populations. However, is the current evaluation conducted on a completely independent held-out ALD population corresponding to the design stage? If not, how do the authors rule out the possibility that this change in panel composition does not necessarily translate into a corresponding performance improvement in the target population? This is very important for whether the conclusion of “population-specific panel design” holds.

Reviewer_ySemOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents an agentic system that designs a cfDNA methylation panel for cancer detection in patients with chronic liver disease (CLD). The main challenge is that the panel design receives no feedback and has no labeled cancer cases. The system uses three inputs to design the panel, cancer literature, a cell type methylation atlas, and unlabeled cancer free CLD plasma.

Critical strengths

1- The panel is designed without labeled positives from the target population. 2- They perform ablations to find and explain where the gain comes from. 3- The evaluation on CLD case is thorough and many tests and ablations are carefully done.

Critical weaknesses

1- Lack of novelty from ML perspective
2- Evaluation is limited to one disease condition (ie CLD). Testing on a general screening background and other populations are helpful. 3- The leakage filter removes retrieved papers that directly cite the evaluation cohorts, while findings can also leak through papers that cite those works indirectly. Additionally, there is a risk that the LLM memorized relevant findings during pre-training. 4- The paper does not provide or describe the release of code, prompts and agent logs.

Minor Weaknesses: The claim of "the agent outperform hand tuned model" is overstated, since their CLD specific panel is compared with general panels (such as TwistAlliance) that was not adapted to the CLD background.

Questions for authors

please see comments in the "Critical Weaknesses" section

Reviewer_bjviOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies cancer-panel design without positive samples or detector feedback during selection. An LLM agent combines cancer literature, a cell-type methylation atlas, and unlabeled plasma from the target population to construct and filter genomic regions under a sequencing budget. On the simulated plasma benchmark, the reference panel achieves an AUC of 0.7323, compared with 0.7050 for the strongest commercial comparator. The paper also examines repeatability, population-specific selection, and the contribution of each information source.

Critical strengths
  1. The evaluation uses the same mixtures and a fixed detector across panels, with paired comparisons, ablations, and repeated design runs. These controls make the panel comparisons easier to interpret.
  2. The reference panel improves on the commercial comparator in the reported benchmark. The repeatability and population-specific analyses also help assess whether the workflow could be useful in practice.
Critical weaknesses
  1. The paper demonstrates a useful design workflow, but its research contribution beyond this application remains unclear. The results establish pipeline effectiveness more clearly than a new selection principle or a distinct contribution from LLM-based selection.
  2. The DETERMINISTIC-MERGE baseline in Section 5.4 is useful, but it merges the single-source panels and ranks regions by CpG count. As the discussion acknowledges, candidate generation and selection remain confounded. The comparison therefore does not isolate the benefit of LLM-based selection or establish systematic exclusion as the main source of the gains.
  3. The evaluation uses simulated mixtures in one application setting. It supports a comparison under that protocol, but leaves open whether the advantage holds on real clinical samples or transfers to other selection tasks without feedback.
Questions for authors
  1. Could the authors compare against a stronger non-agent selector using the same candidate pool, information sources, sequencing budget, and a comparable compute budget?
  2. With the candidate pool held fixed, how much of the gain remains? This would help separate the value of LLM-based selection from the value of generating different candidates.
  3. Is any validation on real clinical samples available? If not, which features of the simulation are most likely to affect the relative ranking of the panels in practice?

Comments on the paper

1 comment

We thank the reviewers for their careful reading. We appreciated their recognition of the practical importance of designing a panel when cancer-positive samples are scarce, and of the care taken in the experimental design to evaluate competing panels.

The main contribution of this paper is to establish that LLM agents are capable of this panel design task. To our knowledge, this is the first demonstration that an LLM agent can select the complete set of genomic regions for a multi-cancer cell-free DNA methylation capture panel without using cancer-positive samples. Recent theoretical work formalizing liquid biopsy as a sparse measurement of a low-rank manifold showed that the choice of regions is the most important determinant of cancer test performance. We used our agents to design a panel, froze the design before evaluation, and showed that it outperformed a commercial pan-cancer panel of comparable size on an in silico benchmark. The practical advance is that scarce cancer-positive specimens from a subpopulation can be reserved for testing a panel whose targets have already been chosen for that population. This enables the application of liquid biopsy in more populations.

Many of the reviewers’ questions center on whether the panel’s benchmark performance carries over to patient plasma. The reviewers identified information leakage to agent and memorization during pretraining as possible influences on the benchmark results. Our clinical studies, in which we manufactured the agent-designed capture panel and tested it on patient plasma, address those concerns directly. We have reserved the results of the clinical studies for first publication in a medical journal. We are therefore withdrawing this submission without reporting those results here.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.