- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper uses LLM-guided program search to select subject- and variant-specific neighborhoods in genetic ancestry space. Local SNP effects estimated from labeled reference subjects are used to construct weights for prediction. The problem is important, but the method proposed in this paper is not practical for polygenic risk prediction commonly used. In addition, LLM is very narrowly used, so it is questionable whether LLM-based scientific discovery is convincingly shown.
Critical strengths
• The paper consider the important problem of disease-risk prediction in underrepresented populations. • It explores an interesting approach to adaptive borrowing across genetic backgrounds.
Critical weaknesses
- The contribution of LLM search is not established. Neighborhood selection could use cross-validated nearest neighbors, kernel bandwidth selection, or conventional optimization.
- The method requires individual level genotype-phenotype data for test sample prediction, so it cannot be used for most of use cases in polygenic risk prediction. Proper use case should be demonstrated.
- The authors used very small scale data for the demonstration. Scalability of the method to large dataset is not shown.
- The stability of local SNP-effect estimation is unclear. Restricting estimation to local neighborhoods can increase uncertainty in already noisy genetic effects.
Questions for authors
• Does LLM search outperform conventional neighborhood-selection methods under the same evaluator and a comparable search budget? • What are the distributions of selected neighborhood sizes, and how stable are local effect estimates and final predictions under resampling? • How many SNPs are used in each cohort? Can the method scale to genome-wide prediction, and how does it handle LD as the variant set expands? • For admixed individuals, global-PC proximity may not capture locus-specific ancestry similarity. How might this limitation affect neighborhood selection and prediction?
3 comments
New Results and Experiments Addressing Reviewers’ Questions and Comments (2)
How many SNPs are used in each cohort? how does it handle LD as the variant set expands?
Number of variants. Each cohort went through quality control (QC) before the evaluator saw it: candidate variants, then missingness filters, then LD pruning.
| Cohort | Candidate variants | Variants after QC (used) |
|---|---|---|
| MEC | 200 | 110 |
| OncoArray | 200 | 73 |
| AoU | 50 | 5 |
How LD is handled. LD pruning is a QC step before the evaluator. Within each chromosome, a variant is dropped if it is strongly correlated with a higher-ranked variant that is kept. The correlations are computed on all subjects of the cohort together, across ancestry groups, without labels. The discovered scheme then works on one variant at a time, so it runs unchanged on any variant set.
New experiment: larger variant sets. We reran the OncoArray QC with 2–5× more candidate variants and kept everything else fixed: the same subjects, the same splits, the same LD threshold and the same locked scheme. LD pruning kept 121–230 variants, and each larger set contains the original 73. AUC on the 63 held-out focal subjects, with 90% (both side) bootstrap CIs:
| Candidate variants | Variants after QC (used) | Discovered by LLM | Mixture Learning | Independent Learning | TL-GDES | TL-PR |
|---|---|---|---|---|---|---|
| 200 | 73 (original) | 0.808 [0.712, 0.890] | 0.738 [0.632, 0.837] | 0.725 [0.604, 0.841] | 0.747 [0.639, 0.849] | 0.760 [0.658, 0.853] |
| 400 | 121 | 0.823 [0.729, 0.905] | 0.785 [0.683, 0.876] | 0.721 [0.595, 0.835] | 0.796 [0.697, 0.881] | 0.796 [0.696, 0.884] |
| 600 | 152 | 0.778 [0.673, 0.875] | 0.763 [0.654, 0.864] | 0.681 [0.550, 0.802] | 0.775 [0.669, 0.868] | 0.772 [0.669, 0.870] |
| 800 | 198 | 0.792 [0.689, 0.881] | 0.766 [0.661, 0.864] | 0.597 [0.462, 0.723] | 0.779 [0.677, 0.866] | 0.779 [0.683, 0.873] |
| 1000 | 230 | 0.802 [0.711, 0.888] | 0.763 [0.659, 0.860] | 0.610 [0.481, 0.737] | 0.762 [0.652, 0.852] | 0.781 [0.684, 0.873] |
The discovered scheme stays first at every size and does not degrade as variants are added.
(Revised PDF: Appendix C, "Quality control and variant selection" and Table 4; new last paragraph of Section 4.2, "The advantage persists on larger variant sets"; Appendix D, Table 5.)
------------------------------
The authors used very small scale data for the demonstration. Scalability of the method to large dataset is not shown.
New: a second disease with a much larger validation cohort. We applied the full discovery process to lung cancer (focal group: Asian ancestry; see "Reviewer vof2: another disease / focal ancestry group"). The OncoArray lung cancer validation has 840 held-out focal subjects, more than 13× the 63 held-out focal subjects of the prostate OncoArray validation, and the locked scheme was also validated on All of Us. The discovered scheme ranks first on both validation cohorts, and all eight one-sided p-values are below 0.01, including when it must beat each baseline by at least δ = 0.01 AUC.
---------------------------------------
Can the method scale to genome-wide prediction
Our current evaluation focuses on panels of established prostate cancer risk variants, an established and clinically relevant prediction setting. Such panels have been evaluated prospectively for risk-guided screening (McHugh et al., 2025; DOI: 10.1056/NEJMoa2407934), and comparative studies found that the genome-wide methods evaluated did not outperform established-variant scores in their prostate cancer datasets (Yu et al., 2020; DOI: 10.1002/pros.23920; Darst et al., 2023; DOI: 10.1016/j.ajhg.2023.05.010). Genome-wide scaling is therefore not required for the prediction setting studied here, and we did not evaluate it.
Addressing Additional Comments
For admixed individuals, global-PC proximity may not capture locus-specific ancestry similarity. How might this limitation affect neighborhood selection and prediction?
Global PCs are an established representation of the genetic ancestry continuum (Ding et al., 2023, Nature; DOI: 10.1038/s41586-023-06079-4), and our scheme borrows along that continuum. We agree that for recently admixed individuals, subjects close in global-PC space may be less similar at particular loci, so the borrowed reference data there may be less informative and the benefit of borrowing smaller. The borrowing radius already varies per subject and per variant, so the formulation is a natural basis for a local-ancestry-aware distance (e.g., using inferred local ancestry at each variant). This requires extending the neighborhood definition and a separate validation, which is outside the scope of this revision.
(Revised PDF: new paragraph in Section 6, Limitations.)
-------------------------
The method requires individual-level genotype-phenotype data for test sample prediction, so it cannot be used for most use cases in polygenic risk prediction. A proper use case should be demonstrated.
Individual-level genotype–phenotype data are standard inputs for supervised genetic prediction, including ancestry-aware and transfer-learning approaches evaluated in large biobanks (Chen et al., 2026; DOI: 10.1038/s41592-026-03235-2; Wu et al., 2026; DOI: 10.1038/s41467-026-68696-7). Our method estimates predictive relationships from labeled reference individuals and uses the target individual's genotype and ancestry coordinates as prediction inputs. The target's phenotype is never used to construct its prediction. The intended use case is prediction within a biobank or authorized research environment maintaining a labeled reference cohort, as represented by our MEC, OncoArray, and All of Us experiments.
New Results and Experiments Addressing Reviewers’ Questions and Comments (1)
The LLM-discovered scheme beats non-LLM neighborhood search tuned on the same data, with the same evaluator and the same budget, on all three independent cohorts
What we did. We kept everything fixed and replaced only the LLM with a conventional grid search: the evaluator, the Discovery Cohort and its folds, the locking rule and the validation. Two neighborhood-borrowing rules were searched:
Each rule used the same 56 evaluator calls as the LLM search and was locked before any validation data were used.
Results. AUC on held-out focal subjects, with a 90% bootstrap CI for OncoArray. The p-value is from the one-sided paired test over all 112 held-out focal subjects.
About the non-significant p = 0.11 against the novelty-adaptive baseline.
(Revised PDF: Section 4.3, Figure 3, Appendix B.)
---------------------------------------------------------------
New experiment: the advantage of the discovered scheme survives perturbation of the reference cohort in 10 out of 10 resamples
What we did. Every scheme borrows from the reference portion, so its local effect estimates and predictions depend on which subjects that portion contains. We drew 10 bootstrap resamples of the OncoArray reference portion (23,030 subjects, 453 focal). Subjects were sampled with replacement within each ancestry group and case/control status, so group sizes and case counts were kept. For each resample we reran the locked scheme and the four established schemes from scratch: neighborhoods, local effect estimates, features, the risk model and the predictions. The calibration and held-out portions were not changed, so every resample is scored on the same 63 held-out focal subjects.
Results. Held-out AUC over the 10 resamples:
The discovered scheme ranked first in every resample, and its lowest AUC (0.772) is above the mean of every established scheme. No established scheme was consistently second (TL-PR 7 times, Mixture Learning 2, TL-GDES 1).
(Revised PDF: new paragraph in Section 4.2, "The advantage survives perturbation of the reference portion"; Appendix E, Table 6.)