acceptodds
Withdrawn from ICLR 2027

Verified Memory for Knowledge Reuse in Variant-to-Gene Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Agents that prioritise causal genes at genome-wide association (GWAS) loci are typically stateless: every locus is re-analysed from scratch, even when the same gene–trait relationship was already established in an earlier study. We present EviMem+CAVE, a memory layer for such agents that stores verified gene–trait claims in a four-state contract (CAVE) and retrieves prior loci by a numeric Evidence Fingerprint (EviMem). On a frozen benchmark of 844 loci (296 held-out test loci), the memory short-circuits 31.7% of within-ancestry test loci and 79.1% of cross-ancestry replication loci, answering them without an LLM call (top-1 precision 0.925 and 0.977), and on the within-ancestry test loci raises GPT-5.2 top-1 accuracy over the same agent without memory by +0.118 (95% CI +0.077 to +0.159). A mechanistic decomposition shows that the gain comes from verified reuse of recurrent gene–trait knowledge through the CAVE store rather than from fingerprint similarity, and that on novel gene–trait pairs we detect no improvement. We also show that a supervised baseline’s apparent lead is an artefact of how gold genes are inserted into candidate sets, and we report the evaluation protocol under which reuse can be measured separately from such artefacts

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
memory-augmented LLM agents, knowledge reuse, variant-to-gene prioritisation, genomics

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JmivOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EviMem+CAVE adds memory to an LLM-based gene-ranking system. It combines verified gene-trait lookups with retrieval of similar past cases. Tests on held-out and cross-ancestry loci show fewer LLM calls and higher accuracy, mainly through reuse of known relationships. The paper also identifies a benchmark shortcut caused by how correct genes are added to candidate lists.

Critical strengths
  • Practical savings. The system avoids 31.7% of LLM calls on within-ancestry test loci, with 92.5% accuracy on those cases. This is useful for repeated analyses involving known associations.
  • Useful benchmark analysis. The missingness-only baseline shows that models can exploit missing evidence to identify manually added correct genes. This exposes a shortcut that can make benchmark results misleading.
Critical weaknesses
  • Limited benefit beyond a simple lookup. The full system achieves 61.0% accuracy versus 61.1% for a gene-identity lookup in Table 1. Nearly all avoided calls come from direct CAVE lookup, while fingerprint retrieval and retrieved hints contribute little. The results therefore do not establish the value of the additional components.
  • The cleaner evaluation remains inconclusive. The natural-gold subset contains only 59 loci and uses evidence availability, rather than actual retrieval records, to identify naturally retrieved genes. The accuracy improvement over the memory-free agent is statistically inconclusive on this subset, leaving the benefit without the candidate-insertion artifact uncertain.
Questions for authors
  • How does the full system compare with a direct CAVE lookup using the same trait-name key and LLM fallback, but no fingerprint retrieval or hints, in accuracy and calls saved?
  • Can the natural-gold subset be checked against actual candidate-insertion records? Does the accuracy gain persist on confirmed naturally retrieved cases?
Reviewer_fAs7OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

L2G prioritization tries to rank candidate causal genes identified from GWAS associated with a trait. While LLMs could be utilized to infer gene-trait relationships and produce rankings, this paper asks whether the process could be improved by reusing verified claims. The authors thus propose a CAVE + EviMem framework. CAVE formats and stores gene-trait claims verified from the training data as four-state contracts. EviMem retrieves gene records with similar evidence to the candidate gene using a predefined numeric Evidence Fingerprint. At inference time, candidate genes could be retrieved from CAVE or EviMem would directly reuse validated gene-trait claims, instead of calling an LLM. The authors used a benchmark curated from Open Targets to test the proposed method. On both within-ancestry and cross-ancestry loci, the method reduced LLM calls and improved top-1 accuracy compared to LLMs without memory. Further analysis disclosed that nearly all gains were drawn from CAVE rather than EviMem, and no improvements were made on inferring gene-trait pairs unseen in the training data. The author also identified an evaluation artifact in the benchmark’s candidate curation process that leads to overestimating the performance of supervised baselines. The main contribution of the paper is therefore showing that reusing verified gene-trait claims could safely reduce LLM costs.

Critical strengths
  1. The research questions are reasonable. The question of whether reusing gene-trait claims could safely reduce LLM calls on L2G prioritization problem is practical for the engineering side. The designs of the four-state contract, the CAVE and EviMem framework, as well as the ablation analysis on those modules, also provide insights for how to properly reuse gene-trait relationships.
  2. Proper experiments are designed. Test results on both within-ancestry and cross-ancestry loci show that a certain portion of loci has corresponding gene-trait claims verified from the training data, which could potentially reduce LLM calls. Improvements on top-1 accuracy show that the reuse is safe and could benefit the inference on seen claims. Comparisons on CAVE and EviMem further investigate where the contribution comes from.
  3. The conclusion is honest. The paper restricts its contribution to reducing LLM calls and discloses its limitations on the effectiveness of EviMem. The paper also reveals the artifact in the gold-gene construction process.
Critical weaknesses
  1. Research questions are not profound, and research contributions are limited. The main research question is whether a known gene-trait pair could be reused safely to reduce LLM calls, which is trivial and provides limited insights into the actual L2G prioritization problem.
  2. Methods lack novelty. Most of the meaningful results are from CAVE, which appears to be a simple look-up memory. The other module EviMem is a simple similarity-based gate and only shows limited contributions.
  3. The delivery of key mechanisms is unclear. Besides VALIDATED, the definition and generation of other states in CAVE are not described clearly. The similarity threshold used in EviMem also seems to be intuitive with no explanations of how they are defined.
Questions for authors
  1. What is the main novelty of your methods? The method seems to be a simple claim retrieval memory and a similarity gate shown to be useless.
  2. How are PROVISIONAL, SUPPORTED, and REJECTED defined and created? The paper seems to only clearly explain the construction of VALIDATED but failed to clearly describe other states.
  3. How are the thresholds for EviMem similarity defined?
Reviewer_T9r9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that LLM-based locus-to-gene (L2G) prioritization agents unnecessarily repeat analyses of previously validated gene–trait associations. It introduces EviMem+CAVE, a memory framework that allows agents to reuse validated associations and avoid LLM calls.

The method is evaluated on GWAS loci, including a cross-ancestry replication panel, against memory-free LLM agents, simple gene-lookup baselines, XGBoost, and established L2G methods. While it improves accuracy over the memory-free agent and reduces LLM calls, the reported improvement is largely driven by recurrent gene–trait pairs. Its performance is comparable to a simple gene-identity lookup, with no detectable improvement on novel gene–trait associations.

Critical strengths

The main strength is the idea of reusing previously validated gene-trait associations instead of repeatedly performing gene prioritization from scratch. The experiments show that a simple lookup can be effective, suggesting that combining verified knowledge reuse with existing gene-prioritization methods could be a useful direction.

Critical weaknesses

The main weakness is that the paper does not evaluate the most natural baseline: a simple lookup of previously validated gene-trait associations, with an established L2G method used when no match is found. Instead, the framework relies on an LLM-based agent without establishing why an LLM is necessary. Since the proposed method performs comparably to a simple gene-identity lookup and shows no detectable improvement on novel gene-trait associations, it remains unclear what the additional components contribute beyond knowledge reuse itself.

Questions for authors
  1. Missing baseline: Can you compare EviMem+CAVE against a simpler approach that first looks up validated gene-trait associations and falls back to an established L2G method, such as Open Targets L2G, when no match is found? This seems the most natural baseline for assessing the contribution of the proposed framework. The authors can articulate if they disagree.

  2. Verification mechanism: Since the system starts with gene-trait associations already validated in the training data and does not validate new ones during evaluation, what does CAVE add beyond a simple lookup table of known associations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.