- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
VANDAM incorporates sequence-derived molecular priors into genomic foundation models through local feature injection (LFI) and regional property prediction (RPP). RPP complements the native token objective during continual pre-training, while LFI is additionally used under functional supervision. Across four backbones and nine downstream tasks, the approach improves most matched comparisons. Component ablations, non-biological controls, and probing of held-out molecular descriptors provide additional evidence for its effectiveness.
Critical strengths
-
The evaluation is reasonably thorough. Matched continuation baselines across four backbones help distinguish the proposed objectives from simply training longer, while component and non-biological controls provide useful evidence beyond downstream gains alone.
-
The distinction between supplying molecular features and explicitly supervising their prediction is informative. The PT/MT ablations suggest that their utility depends on the training setting, and the held-out descriptor probes provide additional evidence that RPP changes the learned representations.
Critical weaknesses
-
The methodological novelty is incremental. UTR-LM already uses sequence-derived biophysical auxiliary targets, and MIXBend incorporates physicochemical features into DNA representation learning. The broader framing should more clearly distinguish these precedents from the specific contribution of regional supervision and its combination with LFI.
-
The matched baselines establish improvements over the selected backbones, but leave the broader competitive position unclear. Comparison with suitable functionally supervised models, such as Enformer [1], SPACE [2], Borzoi [3], or AlphaGenome [4], on compatible tasks would strengthen the evaluation, with differences in supervision, context length, and data overlap made explicit.
-
The explanation for the different roles of LFI and RPP in PT versus MT remains tentative. The central ablations use NTv3-8M, and masking removes approximately 47% of local feature vectors under MLM. This may also contribute to LFI's limited PT benefit; the alternative masking treatments in Appendix O are evaluated only in MT.
[1] Avsec et al. Effective gene expression prediction from sequence by integrating long-range interactions. Nature Methods, 2021.
[2] Yang et al. SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation Model. ICML, 2025.
[3] Linder et al. Predicting RNA-seq coverage from DNA sequence as a unifying model of gene regulation. Nature Genetics, 2025.
[4] Avsec et al. Advancing regulatory variant effect prediction with AlphaGenome. Nature, 2026.
Questions for authors
Does existing evidence distinguish the effect of functional supervision from feature sparsity caused by masking when explaining LFI's different benefits in PT and MT?
4 comments
Response to Reviewer ZYdz, part 2
Response to Weakness 2: Comparison with functionally supervised models
We thank the reviewer for this suggestion. We agree that comparison with functionally supervised genomic models provides important context beyond the matched-backbone comparisons. We have therefore expanded the evaluation to include Enformer, SPACE-large, Borzoi, and AlphaGenome, alongside DNABERT-2 and HyenaDNA-large with VANDAM-MT. The table reports the evaluated context lengths and separates tasks according to documented task or label overlap with the functionally supervised comparators.
We group tasks by documented overlap with at least one functionally supervised comparator. Task overlap includes supervision on the same prediction task, even with held-out evaluation examples.
The functionally supervised models achieve the highest scores on 11 of the 15 tasks, demonstrating stronger overall performance in this comparison. VANDAM-MT achieves the highest scores on regulatory variants (eQTL), ClinVar pathogenicity, ClinVar indel, and LOL-EVE causal eQTL. The specialist models also lead on several tasks without documented task or label overlap, so their advantage cannot be attributed solely to overlap. These results position VANDAM as a complementary strategy for improving existing genomic foundation models through molecular supervision, rather than as a replacement for functionally supervised models. Differences in supervision, context length, and training data mean that this comparison assesses broader competitive performance rather than isolating the effect of the VANDAM training objectives.
Benchmarking against functionally supervised models
Task or data overlap
| Model | AlphaGenome all-folds | AlphaGenome all-folds | SPACE-large | Enformer | Borzoi | Borzoi | DNABERT-2 VANDAM-MT | HyenaDNA-large VANDAM-MT |
|---|---|---|---|---|---|---|---|---|
| Sequence length | 16k | 128k | 8k | 196,608 bp | 8k | 512k | 2.5k | 8k |
| TF binding | 91.26 | 91.35 | 88.86 | 91.50 | 90.94 | 90.75 | 82.60 | 80.22 |
| Histone marks | 75.67 | 71.85 | 73.44 | 71.72 | 73.15 | 69.45 | 72.60 | 54.87 |
| CAGE expression | 67.00 | 22.81 | 62.78 | 69.74 | 65.97 | 11.84 | 36.60 | 32.37 |
| Regulatory variants (eQTL) | 69.11 | 66.15 | 69.55 | 66.05 | 69.92 | 65.11 | 71.80 | 70.25 |
Response to Reviewer ZYdz, part 3
No task or label overlap
| Model | AlphaGenome all-folds | AlphaGenome all-folds | SPACE-large | Enformer | Borzoi | Borzoi | DNABERT-2 VANDAM-MT | HyenaDNA-large VANDAM-MT |
|---|---|---|---|---|---|---|---|---|
| Sequence length | 16k | 128k | 8k | 196,608 bp | 8k | 512k | 2.5k | 8k |
| ClinVar pathogenicity | 66.42 | 65.15 | 61.77 | 61.14 | 66.43 | 61.67 | 69.70 | 76.49 |
| OMIM | 84.44 | 88.85 | 76.92 | 89.41 | 84.33 | 91.01 | 66.40 | 51.88 |
| TraitGym Mendelian | 73.21 | 73.96 | 71.92 | 73.35 | 73.16 | 71.75 | 57.50 | 53.86 |
| BRCA1 | 67.44 | 66.67 | 57.59 | 64.52 | 61.60 | 63.42 | 56.20 | 54.39 |
| Gene-expression variants | 70.72 | 70.99 | 70.17 | 70.24 | 69.82 | 69.93 | 69.40 | 65.47 |
| ClinVar indel | 51.19 | 47.90 | 50.07 | 48.25 | 52.68 | 46.88 | 72.10 | 74.42 |
| LOL-EVE causal eQTL | 53.12 | 53.53 | 51.55 | 53.29 | 52.15 | 53.45 | 55.50 | 54.09 |
| SongLab ClinVar | 50.24 | 51.11 | 53.83 | 52.10 | 54.85 | 53.79 | 52.80 | 53.69 |
| TraitGym complex | 64.28 | 62.87 | 61.92 | 61.87 | 63.21 | 62.00 | 52.20 | 49.18 |
| Variant Benchmarks coding | 70.12 | 69.95 | 64.74 | 64.86 | 64.26 | 64.60 | 63.50 | 63.92 |
| Variant Benchmarks non-coding | 93.78 | 93.91 | 83.55 | 86.48 | 87.63 | 86.98 | 72.80 | 66.88 |
Response to Weakness 3: LFI and RPP under PT and MT and alternative masking treatments
We clarify that the approximately 47% zeroing rate applies specifically to the single-nucleotide MLM backbones NTv3-8M and Caduceus, rather than to all backbones. In DNABERT-2, BPE tokens pool local descriptors across multiple nucleotides; with 12% of sequence tokens replaced by MASK tokens, only 13.2% of the pooled token-level feature vectors become entirely zero under the observed token-length distribution, on the multi-species dataset. HyenaDNA uses causal next-token prediction and does not remove local features through MLM masking. The potential effect of masking-induced feature sparsity must therefore be considered separately for each backbone.
Feature zeroing by tokenization and training objective
| Backbone | Tokenization | Training objective | Observed zeroed feature vectors |
|---|---|---|---|
| NTv3-8M | Single nucleotide | MLM | 47% |
| Caduceus | Single nucleotide | MLM | 47% |
| DNABERT-2 | Byte-pair encoding (BPE) | MLM | 13.2% |
| HyenaDNA | Single nucleotide | NTP | 0% |
To address the restriction of Appendix O to MT, we extended the mask-imputation evaluation to PT. The tables below compare zeroing, uniform imputation, and conditional imputation across all 15 tasks for NTv3-8M and Caduceus.
Response to Reviewer ZYdz, part 4
Mask imputation - PT
| Task | NTv3-8M — Uniform | NTv3-8M — Cond. | NTv3-8M — Zeroing (VANDAM) | Caduceus — Uniform | Caduceus — Cond. | Caduceus — Zeroing (VANDAM) |
|---|---|---|---|---|---|---|
| ClinVar pathogenicity | 62.83 | 59.21 | 60.91 | 63.65 | 62.72 | 62.34 |
| OMIM | 52.78 | 57.02 | 55.22 | 51.63 | 55.22 | 53.73 |
| TraitGym Mendelian | 52.29 | 55.55 | 58.68 | 54.28 | 57.16 | 57.77 |
| BRCA1 | 51.59 | 52.50 | 52.63 | 53.35 | 53.82 | 55.38 |
| TF binding | 81.79 | 81.87 | 80.66 | 79.16 | 75.27 | 75.60 |
| Regulatory variants (eQTL) | 68.19 | 68.47 | 68.70 | 66.73 | 65.20 | 65.22 |
| Gene-expression variants | 63.79 | 63.74 | 62.69 | 61.38 | 59.23 | 59.40 |
| Histone marks | 58.94 | 58.27 | 67.48 | 54.79 | 52.40 | 52.84 |
| CAGE expression | 34.28 | 32.17 | 32.42 | 26.62 | 22.16 | 22.06 |
| ClinVar indel | 54.33 | 52.57 | 55.43 | 55.29 | 55.55 | 55.44 |
| LOL-EVE causal eQTL | 55.55 | 54.89 | 51.34 | 53.17 | 53.03 | 52.12 |
| SongLab ClinVar | 54.29 | 54.40 | 54.33 | 53.43 | 53.50 | 53.29 |
| TraitGym complex | 51.63 | 51.52 | 50.94 | 50.76 | 50.67 | 50.64 |
| Variant Benchmarks coding | 62.71 | 61.30 | 62.19 | 57.34 | 58.00 | 57.61 |
| Variant Benchmarks non-coding | 61.45 | 61.71 | 61.50 | 59.33 | 60.55 | 60.59 |
| Mean AUROC | 59.44 | 59.50 | 60.19 | 58.16 | 58.02 | 58.00 |
Mask imputation - MT
| Task | NTv3-8M — Uniform | NTv3-8M — Cond. | NTv3-8M — Zeroing (VANDAM) | Caduceus — Uniform | Caduceus — Cond. | Caduceus — Zeroing (VANDAM) |
|---|---|---|---|---|---|---|
| ClinVar pathogenicity | 68.63 | 70.81 | 70.20 | 51.76 | 51.05 | 51.40 |
| OMIM | 58.12 | 55.95 | 57.40 | 51.11 | 51.59 | 53.10 |
| TraitGym Mendelian | 56.10 | 54.35 | 56.80 | 53.08 | 53.63 | 55.60 |
| BRCA1 | 59.39 | 61.16 | 54.60 | 53.13 | 49.92 | 52.50 |
| TF binding | 78.86 | 78.63 | 79.80 | 62.49 | 66.07 | 66.40 |
| Regulatory variants (eQTL) | 69.79 | 70.13 | 69.30 | 62.15 | 64.29 | 63.00 |
| Gene-expression variants | 70.71 | 70.03 | 63.10 | 58.41 | 58.38 | 59.90 |
| Histone marks | 70.97 | 71.39 | 69.30 | 49.94 | 50.06 | 55.40 |
| CAGE expression | 33.35 | 33.50 | 35.90 | 12.64 | 16.85 | 14.60 |
| ClinVar indel | 57.38 | 60.32 | 62.60 | 47.09 | 47.44 | 49.60 |
| LOL-EVE causal eQTL | 49.04 | 51.15 | 54.20 | 55.78 | 57.37 | 51.40 |
| SongLab ClinVar | 52.09 | 53.32 | 56.00 | 51.94 | 51.48 | 50.20 |
| TraitGym complex | 51.86 | 50.93 | 50.80 | 50.43 | 49.97 | 49.60 |
| Variant Benchmarks coding | 62.05 | 60.27 | 63.70 | 51.96 | 51.17 | 53.10 |
| Variant Benchmarks non-coding | 61.89 | 62.28 | 64.90 | 57.50 | 58.89 | 59.00 |
| Mean AUROC | 61.92 | 62.19 | 62.34 | 54.06 | 54.38 | 55.01 |
In MT, zeroing achieves the highest score on 8/15 tasks for each backbone and the highest mean AUROC across the 14 AUROC tasks: 62.34 for NTv3-8M and 55.01 for Caduceus, compared with 61.92 and 54.06 for uniform imputation and 62.19 and 54.38 for conditional imputation, respectively. These results support retaining zeroing as the default in MT.
In PT, RPP alone retains the highest mean AUROC on both backbones: 60.71 for NTv3-8M and 59.67 for Caduceus, compared with 60.19/59.44/59.50 and 58.00/58.16/58.02 for LFI with zeroing/uniform/conditional imputation, respectively. Thus, alternative masking treatments do not reverse the preference for RPP alone in PT, supporting the conclusion that LFI’s limited PT benefit is not explained solely by masking-induced feature sparsity.
Response to Reviewer ZYdz, part 1
Response to Question 1: Functional supervision versus masking-induced feature sparsity
We agree that the NTv3-8M ablation alone cannot distinguish the effect of functional supervision from masking-induced feature sparsity. To examine whether the same pattern occurs without this sparsity, we repeated the component and token-objective ablations with HyenaDNA-small. HyenaDNA uses causal next-token prediction, and its local descriptors are computed from right-aligned pentamers containing only visible bases. It therefore has no masking-induced feature removal, unlike the 47% zeroing of feature vectors under NTv3-8M’s MLM objective.
Component and token-objective ablations across PT and MT
The HyenaDNA ablation reproduces the configuration ranking observed for NTv3-8M: RPP alone achieves the highest mean AUROC in PT, whereas LFI + RPP achieves the highest mean AUROC in MT. Crucially, this pattern persists in HyenaDNA without masking-induced feature sparsity. These results therefore support our interpretation that the relative benefits of LFI and RPP depend on the training regime, and that the pattern observed in NTv3-8M is not solely attributable to feature removal under MLM. Removing the native token objective also reduces performance in both regimes, supporting molecular supervision as a complement to token prediction.
Response to Weakness 1: Methodological novelty and prior work
We thank the reviewer for encouraging a sharper discussion of the methodological contribution. We agree that sequence-derived biophysical supervision and physicochemical feature integration have important precedents, and that our framing should distinguish these established ideas from the specific contributions of VANDAM.
Our contribution builds on these ideas. UTR-LM focuses on RNA, incorporating secondary-structure and sequence-level minimum-free-energy prediction for mRNA 5′ untranslated regions. MIXBend combines DNA sequence and physicochemical representations for DNA bendability prediction. VANDAM introduces tile-level prediction of multiple DNA molecular properties during continual training of existing genomic foundation models and evaluates transfer across diverse downstream tasks. We also investigate how this regional supervision combines with local feature injection under self-supervised and functionally supervised training regimes.
To make these distinctions explicit, we will add the following paragraph to Related Work:
“Biophysical supervision and physicochemical feature integration have established precedents in sequence representation learning. UTR-LM uses secondary-structure and minimum-free-energy prediction as auxiliary objectives for learning representations of mRNA 5′ untranslated regions, while MIXBend combines DNA sequence and physicochemical representations for DNA bendability prediction. Building on these ideas, VANDAM introduces regional property prediction (RPP), which provides tile-level supervision of multiple DNA molecular properties during continual training of genomic foundation models. We investigate its combination with local feature injection (LFI) across self-supervised and functionally supervised training regimes, with transfer evaluated across diverse downstream tasks. The specific contribution is this regional supervision strategy and the study of its interaction with LFI, rather than the general use of biophysical priors.”