- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
IsoAlign continues DNA-model pretraining with paired transcripts and protein isoforms. DP-JEPA aligns DNA-derived predictions with frozen ESM2 protein representations. IM-JEPA uses visible isoforms from the same gene to predict masked isoform representations. Evaluation covers mutational effects, splice-related variants, TF binding, tissue-specific expression, and DNA–protein retrieval. Parent-model and continued-pretraining comparisons show gains on several tasks, with DP-JEPA generally more consistent than IM-JEPA. Scalar variant distances are evaluated directly by AUPRC; vector probes use chromosome-held-out logistic regression. TFLink/GTEx prediction uses both DNA and protein features. The results support protein-informed pretraining, but do not yet isolate the isoform-specific contribution or establish regulatory understanding.
Critical strengths
-
Protein-guided pretraining produces meaningful gains on some tasks. For example, NTv2 with DP-JEPA improves MFASS AUPRC under the L2 score from 0.04 to 0.17 (Table 1). Architecture-matched continued-pretraining controls help distinguish this benefit from simply training the DNA encoder longer.
-
Several downstream splits account for dependence between examples. DMS examples are grouped by amino-acid mutation, and GTEx is split by gene. The combination of unsupervised variant scores and supervised tasks also gives a broader view of representation utility than retrieval alone.
Critical weaknesses
-
The main missing comparison is protein-guided training with one canonical isoform per gene. Without it, the gains could come from protein alignment rather than alternative-isoform information. The “Canonical CDS” ablation changes the input format and does not answer this training-data question. This leaves the central isoform-aware contribution insufficiently supported.
-
Pretraining and model-selection boundaries need clearer reporting. The final G2P gene/isoform counts, split unit, and downstream gene or family overlap are not sufficiently documented. Appendix E calls the retrieval data a “held-out validation set” without clarifying whether it differs from the early-stopping set. Section 3 reports the better of two alignment configurations without specifying the selection data. These gaps make generalization difficult to assess; they do not establish leakage.
-
The sequence ablations do not justify the conclusion that IsoAlign learns “true regulatory syntax” (PDF p. 9). Each altered-input model is retrained. Successful retrieval from frameshifted sequences or UTRs could still rely on gene identity or homology. Figure 3 includes relevant isoform-swap and homolog-swap controls, but retraining on altered pairings does not test the original model's discrimination among same-gene isoforms. The evidence is stronger for sequence-dependent alignment than for regulatory understanding.
Questions for authors
-
Is there a capacity- and budget-matched control trained with one canonical isoform per gene? Please distinguish this from the existing “Canonical CDS” input-format ablation.
-
What are the final G2P gene/isoform counts, split units, and gene or protein-family overlaps with downstream benchmarks? Is retrieval evaluated separately from early stopping, and which data selected the better configuration in Section 3?
-
For the original correctly paired model, how well does retrieval work when candidates are restricted to isoforms of the same gene? This could use existing retrieval scores and would differ from retraining with swapped pairings in Figure 3. It would help assess how much alignment goes beyond gene identification.
3 comments
[W4, Q4] Fixed scoring rule: We agree that a scoring rule fixed independently of downstream labels is important for practical deployment. We therefore summarize performance using cosine distance as a single fixed scoring rule across both datasets. Cosine distance provides a scale-invariant measure of representation changes between reference and alternate sequences, without requiring task-specific fitting or score selection. Under this rule, DP-JEPA (CLIP) improves NTv2-500M from 0.32 to 0.63 AUPRC on ClinVar Splicing and from 0.04 to 0.08 on MFASS. Similarly, for AlphaGenome, DP-JEPA (CLIP) improves performance from 0.61 to 0.63 on ClinVar and from 0.29 to 0.36 on MFASS. These results demonstrate improvements under a consistent scoring rule, without selecting the better metric separately for each dataset. We also acknowledge that performance depends on the scoring rule and model configuration, and have qualified our claims accordingly in Sec. 5.2.
[W5, Q5] We clarify the gradient flow and trainable components for each objective below:
| Component | DP-JEPA | IM-JEPA |
|---|---|---|
| DNA encoder | Trained (lr 1×10⁻⁵) | Trained (lr 1×10⁻⁵) |
| Protein encoder | Trained (lr 5×10⁻⁷) | Frozen |
| Targets | Stop-gradient | Detached |
| Predictor | 4-layer Transformer, 8 heads, d=512 | Same |
| Projection head | 3-layer MLP | Not used |
In DP-JEPA, the protein pathway is optimized with a substantially lower learning rate than the DNA pathway (5e-7 vs. 1e-5), encouraging a relatively stable target representation, consistent with the asymmetric training strategy of VL-JEPA (Chen et al., 2026). We have clarified this in Sec. 4.1. The protein target representations are detached from the prediction loss, preventing gradients from this loss from updating the protein encoder or its projection head. We have corrected Eq. 4 (Sec. 4.2.1) in the revised manuscript to explicitly indicate this stop-gradient operation: zy=sg(y(pool(g(y)))). SigReg (Balestriero & LeCun, 2025) is used to discourage representation collapse (Sec. 4.2.2, Eq. 9). In IM-JEPA, the protein encoder is frozen, and masked isoform representations are predicted using an MSE objective Sec. 4.2.2, Eq. 8).
Thank you again for your thoughtful review. Following your suggestions, we have added within-gene retrieval on the original model and clarified that the canonical-only control differs only in its training data. We also documented the splits, the overlap with downstream benchmarks and model selection, and revised the “regulatory syntax” claim. As the discussion closes today, we wanted to check whether these responses address your concerns. If so, we would be grateful if you would consider updating your score.
We thank the reviewer for recognizing that our pretraining produces meaningful gains (e.g., MFASS L2 AUPRC 0.04→0.17), that architecture-matched controls separate this benefit from longer training, that our splits account for dependence between examples (GroupKFold for DMS, gene-level splits for GTEx), and that combining unsupervised variant scores with supervised tasks gives a broader view than retrieval alone.
[W1, Q1] Canonical-isoform-only control: The requested canonical-isoform-only training control is already included in our ablation analysis as “Canonical CDS” (Figure 3). We have clarified this in Section 5.4 of the manuscript. This configuration uses the same architecture, batch size, and training steps as DP-JEPA, but restricts training to one canonical isoform per gene instead of up to eight. Thus, the difference is in the training data rather than the input format. Multi-isoform training improves cross-gene MRR from 0.805 to 0.843 (+4.7%) and within-gene MRR from 0.823 to 0.884 (+7.4%). These results support the contribution of alternative-isoform information beyond protein-guided alignment alone.
[W2, Q2] Data splits and leakage across tasks: The G2P pretraining dataset contains 55,752 records covering 17,710 genes and 46,125 transcripts, split 80/10/10 at the gene level (14,168 / 1,771 / 1,771 genes). All isoforms of a gene stay in the same partition, and retrieval is evaluated only on held-out genes (Sec. 3, Appendix A.1). The partitions are not family-disjoint. However, the homolog-swap ablation (same protein family, different gene) reduces MRR from 0.843 to 0.326 (61.3% reduction), so retrieval does not rely only on protein-family identity (Sec. 5.4, Fig. 3).
Overlap with downstream benchmarks: DMS: TP53, CBS, PDE3A, and GDI1 appear in pretraining only as wild-type sequences, while evaluation uses unseen mutations (e.g., 7,448 TP53 variants). This follows standard VEP practice, where models are pretrained on reference sequences and evaluated on held-out variants (Riesselman et al., 2018; Frazer et al., 2021; Meier et al., 2021; Notin et al., 2023). Gene-level overlap is acknowledged in Sec. 3.
ClinVar and MFASS: We evaluate vector-based metrics using leave-one-chromosome-out (LOCO) cross-validation and scalar metrics using direct AUPRC (Sec. 5.2). Gene-level disjointness from G2P pretraining is not enforced, but evaluation variants are unseen during IsoAlign pretraining (Appendix A.3).
GTEx: We use an 80/10/10 gene-level split (15,671 / 1,958 / 1,958 genes), ensuring that transcripts from the same gene remain in the same partition (Sec. 5.3).
Checkpoint and configuration selection: We select checkpoints by validation loss and use retrieval metrics for monitoring only. This keeps model selection tied to the training objective instead of the retrieval benchmark we report. The “held-out validation set” in Appendix E is the validation split used for early stopping, and we have made this clear in the revised manuscript. We also chose the Section 3 configuration (architecture, embedding dimension, learning rate, temperature) by validation loss on the 1,771-gene validation split. Test genes were used only for a single final evaluation, after the configuration was fixed.
[W3, Q3] Sequence Ablation and Within-Gene Retrieval Performance: We agree that the retrained sequence ablations demonstrate which sequence features can support DNA–protein alignment, rather than establishing that the original model learns regulatory syntax. We have revised the Fig. 3 caption and Section 5.4 to state explicitly that these ablations are independently retrained models. To directly evaluate isoform discrimination without retraining, we restrict retrieval candidates to isoforms of the same gene using the original trained model and macro-average over genes. Within-gene retrieval achieves an MRR of 0.884 and R@1 of 78.3%, compared with 0.843 and 76.2% for cross-gene retrieval. Since gene identity is shared across all within-gene candidates, this evaluation provides evidence of isoform-level discrimination beyond gene identity alone. Furthermore, swapping protein isoforms within the same gene reduces within-gene MRR from 0.884 to 0.695 and R@1 from 78.3% to 46.8%, indicating sensitivity to the correct DNA–protein correspondence. We have revised the “true regulatory syntax” claim in the Conclusion to describe these findings more precisely as evidence of sequence-dependent alignment rather than demonstrated regulatory understanding.