- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces BioBLIP to investigate whether pretraining on variant annotation enables a multimodal model to transfer to genomic tasks without task-specific training. The model uses AlphaGenome, GenePT, and ESM-2 to obtain DNA, gene-neighborhood, and protein representations, respectively. Modality-specific Q-Formers and a Master Q-Former fuse these representations into soft prompts for a frozen Qwen language model. Training first aligns each modality with annotation text, then uses a structured variant-annotation generation loss to update the modality-specific Q-Formers, Master Q-Former, and projection layer, while keeping the biological encoders and Qwen frozen. In downstream evaluations, BioBLIP performs variant prioritization and target-gene prediction without additional fine-tuning, outperforming the paper’s Qwen3-32B text baseline on both tasks. The largest improvement occurs on the target-gene prediction subset where the target is not the gene closest to the variant, with accuracy increasing from 43.70% to 60.50%.
Critical strengths
-
BioBLIP achieves a significant improvement in identifying target genes that are not the closest to the variant. On this subset, BioBLIP outperforms the equally sized Qwen3-32B text baseline by 16.80 percentage points. The advantage persists on the stricter subset where each test variant is more than 1 kb from every training variant.
-
The paper demonstrates the feasibility of using variant-annotation supervision to integrate multimodal biological information and support cross-task transfer. A single model trained on variant annotation transfers to multi-candidate variant prioritization and target-gene selection without updating the LLM parameters or performing downstream fine-tuning. Connecting multiple biological encoders to a frozen LLM through hierarchical querying modules provides a useful design for reusing biological representations.
Critical weaknesses
-
BioBLIP and the text baseline receive different information and training, so the source of the gains is unclear.
- GenePT contains gene-function descriptions, while the text baseline mainly receives gene names and distances. BioBLIP also receives OpenTargets annotation training and uses generated annotations for variant prioritization. The gains may therefore come from additional knowledge or training, rather than multimodal representations alone.
-
The ablations do not isolate the sources of downstream gains.
- Joint embedding shuffling disrupts multiple information sources.
- Table 5 does not clarify whether reduced-modality models were retrained.
- The Master Q-Former ablation reports only annotation results with Qwen3-4B trained for 15 epochs.
- These results leave the downstream contributions of individual inputs and the fusion module unclear.
-
Gene-prediction accuracy alone does not support the paper’s stronger claims about mechanistic reasoning.
- Appendix G acknowledges correct gene predictions with incorrect intronic placements in the ACADL, CPT2, ASIP, and LCT examples.
- The main evaluation scores the final gene without systematically assessing the accuracy of the explanations.
-
Generalization across LLM backbones remains untested. Experiments only cover Qwen3-4B and Qwen3-32B, leaving it unclear whether BioBLIP’s downstream gains extend to other LLM families.
Questions for authors
See weaknesses.
5 comments
W2.1 Clarifying Embedding Permutation Experiment
W2.1 Joint embedding shuffling disrupts multiple information sources.
A2.1 Thanks for raising this. The purpose of the embedding permutation experiment is to answer the question: does the model use its biological inputs at all? This is not guaranteed. MIRAGE (Asadi et al., 2026) shows models scoring well on multimodal benchmarks while ignoring the images, and concurrent work (arXiv:2610.00898) finds most multimodal models don’t rely on their inputs. Permuting embeddings drops BioBLIP’s performance by 70–89% on all three tasks (Table 4), confirming that BioBLIP does rely on the multimodal data.
W2.2 Reduced Modality Ablations
W2.2 Table 5 does not clarify whether reduced-modality models were retrained.
A2.2 Table 5 shows ablations without retraining. The ablations were applied at inference to the trained full model, with removed modalities replaced by [noise]. This study measures what the fully-trained and deployed model depends on.
Retraining the models from scratch with fewer modalities shows less of a gap (see results below reported for Qwen3-4B backbone; sampling done with three random seeds). The same modality trends are clear in the retrained models. AlphaGenome modality contributes the most to Chromosome, Variant Type, and Variant Consequence. GenePT contributes the most to the nearest genes field. ESM protein embeddings contribute least overall - which is expected because these are mostly noncoding variants which do not affect the protein sequence.
We have qualified in the paper that the nonlinear effects observed with modalities is at inference-time. Full results for the retrained models are below, with the greatest gap bolded.
| Field | 3 modality | AFG ablation | ESM ablation | GenePT ablation |
|---|---|---|---|---|
| Chromosome | 90.1 ± 0.4 | 84.8 ± 0.3 | 86.8 ± 0.3 | 89.3 ± 0.3 |
| Variant Type | 89.7 ± 0.4 | 86.9 ± 0.3 | 89.2 ± 0.2 | 87.6 ± 0.6 |
| Variant Consequence | 67.7 ± 0.8 | 63.0 ± 0.9 | 67.2 ± 0.5 | 65.0 ± 0.6 |
| Nearest Genes | 64.1 ± 0.3 | 64.9 ± 0.3 | 65.4 ± 0.2 | 49.3 ± 0.3 |
Takeaways: The large gaps are due in part to inference-time ablations. With retrained reduced-modality models, the gaps are less, but the field-level contribution of each modality remains the same.
We have added these results and explanations to the Appendix.
W3 Factual Precision of Target Gene Reasoning Traces
Gene-prediction accuracy alone does not support the paper’s stronger claims about mechanistic reasoning.
A3. Thanks for the insightful comments. Such an analysis is indeed helpful.
We selected 100 random target gene identification traces, and analyzed them for the correctness of the claims using LLM-as-a-judge (Opus 5.5) and VEP Ensembl annotations and prior literature as evidence sources. We also hand checked a subset of 10 traces for agreement and found 100% alignment with the LLM judge's decisions.
Below, we demonstrate that on target gene identification (1) precision of the claims made is 90.5% overall (2) 27.5% of claims made are hypotheses. Hypotheses are defined as statements that need additional eQTL or experimental data to be verified. We have added this analysis to Appendix I.
| Stage | Precision | Correct claims | Hypotheses | Incorrect claims | Number of Claims |
|---|---|---|---|---|---|
| S1 Variant annotation | 82.10% | 82.10% | 0.00% | 17.90% | 285 |
| S2 Gene-level effect | 89.40% | 58.40% | 34.60% | 7.00% | 762 |
| S3 Cellular process | 94.40% | 69.80% | 26.10% | 4.10% | 510 |
| S4 Tissue-level effect | 92.90% | 66.70% | 28.30% | 5.10% | 435 |
| S5 Disease mechanism | 92.80% | 61.70% | 33.50% | 4.80% | 457 |
| OVERALL | 90.50% | 65.60% | 27.50% | 6.90% | 2449 |
Precision = (Correct claims) / (Incorrect claims + Correct claims)
Takeaways: BioBLIP provides 90.5% precision on 100 randomly selected traces, using LLM-as-a-judge and humans manually checking a subset of the traces. Precision is defined as the number of correct claims over the number of verifiable claims.
However, BioBLIP does not presume to guarantee step-by-step correctness in the traces produced. The primary contribution of the paper is a new multimodal architecture for zero-shot transfer in genomic tasks, and the step-by-step reasoning traces are useful because they allow an expert to audit them and investigate biological hypotheses raised. These are strengths over a predictive-only architecture.
Improving groundedness, faithfulness, and other properties of LLM traces is an active area of research with tools such as VERDICT (Zhou et al, 2026). We leave applying this research to BioBLIP as future work. We have updated this in the Discussion section of the paper.
(contd. below)
W4 Generalization across LLM backbones
A4. No component of BioBLIP is specific to the LLM. The LLM is frozen and receives soft tokens through a learned projection. We also provide results that performance improves across two model scales (4B and 32B, Table 1). Qwen3-32B is among the strongest open source LLMs for reasoning (Yang et al., 2025) and it would be nontrivial for the experiments to swap it out at this stage in terms of time and compute. We leave it to future work and have added it in the Discussion section of the paper.
Please let us know if you have any further questions or comments!
References
[1] Mohammad Asadi, Jack W O’Sullivan, Fang Cao, Tahoura Nedaee, Kamyar Fardi, Fei-Fei Li, Ehsan Adeli, and Euan Ashley. Mirage the illusion of visual understanding. arXiv preprint arXiv:2603.21687, 2026.
[2] Fang, Ada, et al. "When Do Biological Reasoning Models Use Their Biological Inputs?." arXiv preprint arXiv:2610.00898 (2026).
[3] Yang, An, et al. "Qwen3 technical report." arXiv preprint arXiv:2505.09388 (2025).
[4] Zikai Zhou, Yufei Jin, Yilin Xu, Yu-Chiang Wang, Chieh-Ju Chao, and Monica Lam. Accountable ai with grounded, faithful, consistent, actionable rationales: A case study in clinical trial matching with verdict. pp. 58987–59032, 2026.
We thank the reviewer for thorough and constructive comments. Based on reviewer’s valuable feedback we conducted a number of additional experiments and provided additional analyses, which further validate the efficacy of our BioBLIP architecture. We believe these together further strengthen our work.
Below please find responses to individual comments/questions:
W1. BioBLIP and the text baseline receive different information and training.
A1. Thanks for the comment. As mentioned in Section 2.2, the LLM backbone of BioBLIP is not trained. Neither is the LLM in the text baseline.
The main contribution of BioBLIP is a new multimodal architecture which is pretrained to align biological foundation model embeddings with variant annotations. We show that the model performs well at variant annotation task and then demonstrate zero-shot task transfer. Pretraining the Qformers on OpenTargets variant annotation is an important part of the contribution and should not be viewed as a confound. To control for training-test set leakage, we report results on held-out loci for both downstream tasks.
BioBLIP and the text baseline receive different information by necessity, because it is impossible to completely textify the latent information contained in a biological embedding such as GenePT or AlphaGenome. For example, GenePT was constructed with GPT-3.5 embeddings (Chen et al, 2024). A major purpose of BioBLIP's architecture is to make such embedded knowledge available to an LLM.
To construct a strong text baseline, we provide textified AlphaGenome predictions and the same nearest-gene identities and distances used by BioBLIP and show the results in Section 3.2.
Variant prioritization on Traitgym is already a task that benefits greatly from accurate variant annotations. BioBLIP produces more accurate variant annotations through its multimodal tokens. However, we conducted an additional experiment to identify whether multimodal tokens add any predictive capabilities beyond BioBLIP's improved variant annotations in this specific Traitgym benchmark.
Takeaways: this experiment suggests that BioBLIP's multimodal tokens add predictive information beyond its generated variant annotations. For difficult, poorly sampled regions of the genome, adding multimodal tokens plus the annotations provides 15.05% Top-1 accuracy improvement and 26.16% Top-2 accuracy improvement.
In this experiment, Qwen3-32B was used to rank the variants with exactly BioBLIP's generated variant annotations, but no multimodal tokens. We also report the results on a more challenging subset of n=51 groups with variants from poorly sampled regions of the genome in OpenTargets (>1kb from any other variant).
BioBLIP Improvement, Relative
W2. Isolating Sources of Downstream Performance Gains
A2. Thanks for this question. We conducted an additional experiment ablating the master Q-former and measuring performance on the downstream tasks. We trained a BioBLIP-style Qwen3-32B model with no master Q-former and the same training protocol.
Takeaways: The experiments demonstrate that the Master Q-former provides a large benefit in downstream task performance. This key component of the architecture provides ~22% improvement in accuracy on regulatory variant prioritization. It provides ~28% improvement in accuracy on target gene prediction on the challenging subset of non-closest gene cases. This Master Q-former is critical for zero-shot transfer performance.
Task 1: Regulatory Variant Prioritization
Task 2: Target Gene Prediction
We have added these results to Section 4 of the paper.
A2.3 Please see A2.
(contd. below)