- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates a practically important problem: how to design cancer detection panels under a limited genomic coverage budget when no feedback from experimental execution is available. The authors propose an LLM-based agent for designing a multi-cancer DNA methylation detection panel for a chronic liver disease population. Instead of using cancer-positive samples or feedback from a downstream detector, the agent integrates three types of information: cancer-related literature, a reference DNA methylation atlas comprising 39 cell types, and unlabeled plasma data from cancer-negative donors in the target population. Through literature retrieval and biological claim extraction, construction of genomic methylation patterns, and detectability validation in the target population context, the agent iteratively generates and filters candidate regions and ultimately determines the panel under a given genomic coverage budget.
The authors evaluate the designed panel using an in-silico titration benchmark, in which public WGBS data from tumor and adjacent normal tissues are spiked at different tumor fractions into a cancer-negative plasma background from 700 donors with chronic liver disease and compared using a unified downstream detector and donor-disjoint cross-validation. In this experiment, the reference agent-designed panel achieved an AUC of 0.7323, higher than the commercial TWISTALLIANCE panel with comparable sequencing cost (AUC 0.7050), and outperformed other comparison methods. Further ablation experiments and stability analyses indicate that the agent's advantage does not mainly come from a few high-performance markers, but more likely from the systematic exclusion of weaker candidate regions. The authors also find that when run repeatedly on the same target population, the panels generated by the agent show high consistency, whereas changing the target population background leads to more pronounced changes in panel composition. Based on these results, the authors argue that in an open-loop scenario lacking positive samples and experimental feedback, an LLM agent can still use domain knowledge and unlabeled background data for effective budget-constrained selection, and that its main value may lie in reliably excluding candidate regions that are not worth spending sequencing budget on.
Critical strengths
It proposes a novel and practically significant research question. This paper focuses on budget-constrained panel design without cancer-positive samples or experimental feedback, distinguishing it from the usual closed-loop agent framework of “experiment–feedback–reoptimization.” This setting is highly realistic, especially for biomedical scenarios where positive samples are scarce, experimental costs are high, and panel design must be frozen in advance. The paper clearly formalizes this problem and explores how much of a role an LLM agent can play in such an open-loop setting, which is one of the paper’s most important conceptual contributions.
The experimental evaluation is fairly systematic and includes targeted analysis of the sources of the agent’s gains. The authors not only compare the agent-designed panel with commercial and academic panels, but also evaluate them using a unified downstream detector, donor-disjoint cross-validation, and the same simulated mixtures, thereby separating, to some extent, the contribution of the panel itself from the influence of the downstream detector. In addition, through multiple independent runs, input ablations, and subpanel analyses, the authors find that the performance improvement does not obviously depend on a few specific markers, but is more likely associated with the systematic exclusion of weaker candidate regions. This mechanistic analysis enables the paper not only to report performance improvements but also to attempt to explain why the agent can gain an advantage without positive-sample feedback.
Critical weaknesses
-
The contribution of “agentic selection” is not cleanly isolated. The strongest non-agent baseline, DETERMINISTIC-MERGE, combines the ASSOCIATION-ONLY and DETECTABILITY-ONLY panels, but it does not operate on the same candidate pool generated by the LLM agent. Therefore, the agent’s improvement over this baseline (0.7323 vs. 0.7047 AUC) cannot distinguish whether the gain comes from the LLM’s reasoning/selection ability or from generating a different and potentially better candidate pool. Since demonstrating the value of agentic selection is one of the paper’s central claims, the absence of this control substantially weakens the causal interpretation of the main results.
-
The claimed budget-matched comparison is unclear because several agent panels exceed the 2.3 Mb budget. The paper formulates the problem under a 2.3 Mb coverage constraint, yet several agent runs are reported at 2.37–2.46 Mb, with AGENT-ALD reaching 2.72 Mb and AGENT-NOATLAS reaching 3.03 Mb. This raises concerns about whether these comparisons are truly matched in sequencing/capture cost. Since budget-constrained selection is itself a central premise of the paper, this issue is particularly critical. The paper needs to clearly explain the discrepancy and demonstrate that all primary comparisons satisfy an equivalent effective cost constraint.
Questions for authors
Regarding the 2.3 Mb budget constraint: The paper defines 2.3 Mb as the panel design budget, but in Appendix A several agent-generated panels have actual sizes exceeding this budget—for example, Run 2/4 is 2.42 Mb and Run 5 is 2.46 Mb, while AGENT-ALD and AGENT-NOATLAS reach 2.72 Mb and 3.03 Mb, respectively. Can the authors clarify the specific definition of the 2.3 Mb budget here and why these panels are still considered to have comparable sequencing costs? If these panels in fact have a higher capture/sequencing footprint, does the paper’s main performance advantage still hold under a strictly identical budget?
Regarding the conclusion about population-specific panels: The authors replace the target plasma background from CLD with ALD, observe a clear change in panel composition, and on this basis argue that the agent can adapt to different target populations. However, is the current evaluation conducted on a completely independent held-out ALD population corresponding to the design stage? If not, how do the authors rule out the possibility that this change in panel composition does not necessarily translate into a corresponding performance improvement in the target population? This is very important for whether the conclusion of “population-specific panel design” holds.
We thank the reviewers for their careful reading. We appreciated their recognition of the practical importance of designing a panel when cancer-positive samples are scarce, and of the care taken in the experimental design to evaluate competing panels.
The main contribution of this paper is to establish that LLM agents are capable of this panel design task. To our knowledge, this is the first demonstration that an LLM agent can select the complete set of genomic regions for a multi-cancer cell-free DNA methylation capture panel without using cancer-positive samples. Recent theoretical work formalizing liquid biopsy as a sparse measurement of a low-rank manifold showed that the choice of regions is the most important determinant of cancer test performance. We used our agents to design a panel, froze the design before evaluation, and showed that it outperformed a commercial pan-cancer panel of comparable size on an in silico benchmark. The practical advance is that scarce cancer-positive specimens from a subpopulation can be reserved for testing a panel whose targets have already been chosen for that population. This enables the application of liquid biopsy in more populations.
Many of the reviewers’ questions center on whether the panel’s benchmark performance carries over to patient plasma. The reviewers identified information leakage to agent and memorization during pretraining as possible influences on the benchmark results. Our clinical studies, in which we manufactured the agent-designed capture panel and tested it on patient plasma, address those concerns directly. We have reserved the results of the clinical studies for first publication in a medical journal. We are therefore withdrawing this submission without reporting those results here.