- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces BioStitch, a framework that connects biological foundation models and LLMs through multi-level gated cross-attention. The framework supports biology-to-language tasks, including DNA variant-effect prediction, protein question answering, and single-cell annotation, as well as language-conditioned protein generation. The paper also reports reductions in training memory and prompt KV-cache usage for long biological inputs.
Critical strengths
- Proposes a potentially reusable interface supporting both biological foundation models and LLMs for efficient integration.
BioStitch aims to integrate biological foundation models and LLMs without requiring the biological input to be represented entirely as text. The unified formulation for both biology-to-language and language-to-biology tasks provides a reusable interface for future biological applications rather than a collection of task-specific pipelines.
- Demonstrates substantial improvements in a relatively rigorous DNA variant-effect evaluation.
The improvements in the DNA variant-effect experiments are impressive. BioStitch outperforms BioReason across four dataset–model configurations, with an average F1 improvement of 15.27 percentage points, under a strict held-out evaluation protocol. The authors also investigate potential confounding factors, including the original baseline's evaluation protocol and the effect of connector warm-up. This makes the DNA experiments more convincing than results based only on comparisons with published scores.
- Reports meaningful reductions in training memory and prompt KV-cache usage for long biological inputs.
BioStitch reduces the need to place long biological representations directly into the LLM's self-attention sequence. In the reported long-DNA setting, peak training memory decreases from 79.9 GB to 16.8 GB, while the LLM prompt KV cache decreases from 476.9 MB to 6.65 MB. These are meaningful efficiency gains for applications involving long biological sequences.
Critical weaknesses
- The distinction between BioStitch and existing cross-attention-based multimodal architectures needs to be established more clearly.
Cross-attention and gated cross-attention are established mechanisms, including in architectures such as Flamingo and Evolla. The main contribution of BioStitch appears to be its use of multi-level gated cross-attention to connect pretrained biological and language models, together with its analysis of source-layer selection and its application to both biology-to-language and language-to-biology tasks. However, the current results do not fully isolate the contributions of multi-level integration, gating, and source-layer selection. I encourage the authors to conduct controlled ablation experiments comparing single-layer cross-attention, multi-layer cross-attention without gates, and the full BioStitch architecture under matched training conditions. This would clarify whether the proposed design offers a substantive advantage over simpler alternatives and better establish the methodological contribution of the work.
- BioStitch’s median-source-layer choice is motivated by experiments on DNA-to-text tasks; it should not be interpreted as a universally validated optimum for every experiment.
The median layer is best understood as a practical default motivated by the DNA experiments, not an established optimum across all modalities. As DNA sequences, protein representations, and single-cell expression profiles have different structures and representational properties, a layer-selection strategy that works well for a DNA encoder may not be optimal for a protein model or a transcriptomic encoder. In the task of text-to-protein generation, BioStitch demonstrates that a shared gated cross-attention interface can be applied in both directions. However, the evidence that its layer-selection strategy is appropriate in both directions is weak. The reverse text-to-protein experiment supports the framework’s applicability, but does not by itself validate the generality of the median-source-layer principle.
- The paper presents four application settings, but the experimental comparisons are not equally controlled.
For DNA variant-effect prediction, the authors conduct a relatively rigorous comparison against BioReason using matched test sets and a strict held-out protocol. For protein question answering, however, the comparison with Protein2Text involves substantially different training resources and language models. Although BioStitch's performance is encouraging, this is not a controlled comparison of the architectures. The same distinction matters when interpreting comparisons with published systems in single-cell annotation and protein generation.
- The computational improvements are promising, but different efficiency metrics measure different aspects of the system.
For the reported long-DNA setting, BioStitch substantially reduces peak training memory and prompt KV-cache usage. These results support the claim that BioStitch can reduce particular training and prompt-state costs. They do not establish a general improvement in inference efficiency across all workloads. It would also be useful to report how these metrics scale with biological input length and the number of cross-attention layers. Such results would make the practical advantages and limitations of the architecture clearer.
Questions for authors
- Because gated cross-attention and intermediate-layer fusion have precedents in multimodal learning, what are the differences between BioStitch and the closest existing approaches, particularly Evolla?
- What is the essential architectural contribution of BioStitch? How much of the observed gain comes from gated cross-attention itself, as opposed to using intermediate biological representations or inserting cross-attention at multiple layers?
- The gains in DNA variant-effect prediction are substantial, while it raises a further question: What information does BioStitch extract from the biological representations that enables it to identify these additional pathogenic variants?
- What evidence supports the biological quality of generated proteins? Beyond text–sequence alignment, do the generated sequences exhibit improvements in independent structural or functional evaluations?
7 comments
Response to Reviewer aqS6 (Part 2/5)
Weakness W2: BioStitch’s median-source-layer choice is motivated by experiments on DNA-to-text tasks; it should not be interpreted as a universally validated optimum for every experiment.
Response:
Thank you for highlighting this distinction. We added a text-to-protein source-layer comparison, now reported in Appendix O.1 and referenced from Section 4.2. The revised manuscript treats intermediate extraction as a practical default and reports the metric-dependent generation results rather than a universal optimum. Following the original text-to-protein setup, we used Qwen3-1.7B as the frozen text source and ProGen2-base as the frozen protein receiver, training only the connector on the ProDVa Molinst-SwissProtCLAP corpus. Evaluation used PDFBench with the held-out Mol-Instructions protein-design prompts. We varied the Qwen extraction layer across 8, 14, 20, and 28 while keeping injection at ProGen2 blocks 5, 14, and 22, counted from 1. The backbone checkpoints, connector architecture, and training setup were held fixed across the four runs.
| Metric | L08 | L14 | L20 | L28 |
|---|---|---|---|---|
| ProTrek ×100 ↑ | 8.7192 | 8.8172 | 8.8345 | 8.1840 |
| Normal 4-way selection (%) ↑ | 39.2640 | 36.5566 | 36.6647 | 37.4509 |
| Normal 10-way selection (%) ↑ | 30.7636 | 27.5452 | 27.4469 | 28.4935 |
| Repeat (%) ↓ | 16.0883 | 16.6908 | 15.9695 | 16.3382 |
| Rep-2 (%) ↓ | 55.9237 | 56.3692 | 55.6901 | 56.3429 |
| Rep-5 (%) ↓ | 16.0265 | 16.7247 | 15.9021 | 16.6012 |
| ProGen2 PPL ↓ (decoder diagnostic) | 4.98754 | 4.95091 | 5.01639 | 4.98048 |
| ProtGPT2 PPL ↓ | 245.98177 | 246.78696 | 241.14229 | 247.34379 |
| RITA PPL ↓ | 6.06443 | 6.07637 | 6.06310 | 6.11106 |
The preferred source depth varies across metrics. The two intermediate settings, layers 14 and 20, obtain higher ProTrek scores than the final layer, and layer 20 has the lowest repetition measures and perplexity under ProtGPT2 and RITA. The shallower layer 8 performs best on the Normal selection metrics, while the middle layer 14 has the lowest ProGen2 perplexity, which serves as a decoder-likelihood diagnostic.
The reverse-direction results therefore support intermediate extraction as a practical choice for balancing text–protein alignment and sequence quality. They also provide a direct empirical basis for refining source depth according to the generation objective, extending the layer-selection analysis beyond DNA-to-text.
Weakness W3: The paper presents four application settings, but the experimental comparisons are not equally controlled.
Response:
Thank you for your suggestion. In the protein question-answering experiment, the systems share the official Protein2Text-QA test set, but use different training data and language models. The public release provides evaluation subsets rather than the complete training corpus used by the reference system. We therefore constructed our own adaptation set from the released IntroductionQA and DiscussionQA subsets, using substantially fewer training examples than the reference system.
To check for training–test overlap, we audited the training and test proteins by exact amino-acid sequence identity and protein name, finding no shared proteins under these criteria. The complete official test set was retained. The evaluation therefore compares systems trained with their respective resources on the same complete official test set.
We have made this comparison scope explicit in Sections 4.4–4.6 for protein question answering, single-cell annotation, and protein generation, distinguishing the published-system references from the separately controlled comparisons. The ProGen2 generation experiment separately includes a prefix control with the same pretrained backbones, training corpus, and training budget. For protein question answering, retraining the systems on a common training set with matched backbones and budgets would provide a more direct architectural comparison and is a useful direction for future work.
Response to Reviewer aqS6 (Part 3/5)
Weakness W4: The computational improvements are promising, but different efficiency metrics measure different aspects of the system.
Response:
Thank you for distinguishing these efficiency measures. To examine the scaling behavior more directly, we added measurements at biological input lengths of 512, 2048, and 4096 NT tokens, comparing BioReason with single-point and three-point BioStitch. We have included the sweep in Appendix O.3 and revised Section 4.7 to distinguish training memory, prompt self-KV, prefill, decoding, and end-to-end latency. The results are shown below:
| Biological tokens | Metric ↓ | BioReason | BioStitch single-point | BioStitch three-point |
|---|---|---|---|---|
| 512 | Peak training memory (GiB) | 15.253 | 12.775 | 13.462 |
| 512 | Training time per sample (ms) | 244.0 | 308.9 | 367.8 |
| 512 | Prefill (ms) | 58.3 | 60.1 | 64.8 |
| 512 | Time to first token (ms) | 86.3 | 111.1 | 116.4 |
| 512 | Decode (ms/token) | 51.73 | 52.92 | 54.68 |
| 512 | Total time for 64 generated tokens (s) | 3.346 | 3.450 | 3.563 |
| 512 | Peak inference memory (GiB) | 9.172 | 9.212 | 9.494 |
| 512 | Prompt self-KV (MiB) | 88.758 | 4.102 | 4.102 |
| 2048 | Peak training memory (GiB) | 23.463 | 12.816 | 13.571 |
| 2048 | Training time per sample (ms) | 354.7 | 285.0 | 292.5 |
| 2048 | Prefill (ms) | 88.3 | 59.6 | 63.0 |
| 2048 | Time to first token (ms) | 135.1 | 114.8 | 117.6 |
| 2048 | Decode (ms/token) | 51.02 | 52.09 | 54.23 |
| 2048 | Total time for 64 generated tokens (s) | 3.351 | 3.400 | 3.537 |
| 2048 | Peak inference memory (GiB) | 9.441 | 9.232 | 9.513 |
| 2048 | Prompt self-KV (MiB) | 340.758 | 4.102 | 4.102 |
| 4096 | Peak training memory (GiB) | 34.344 | 12.867 | 13.692 |
| 4096 | Training time per sample (ms) | 627.5 | 346.9 | 385.2 |
| 4096 | Prefill (ms) | 162.1 | 60.3 | 62.4 |
| 4096 | Time to first token (ms) | 283.4 | 189.1 | 191.9 |
| 4096 | Decode (ms/token) | 51.18 | 51.64 | 53.95 |
| 4096 | Total time for 64 generated tokens (s) | 3.509 | 3.445 | 3.593 |
| 4096 | Peak inference memory (GiB) | 10.035 | 9.316 | 9.539 |
| 4096 | Prompt self-KV (MiB) | 676.758 | 4.102 | 4.102 |
The strongest benefits appear in training memory and prompt self-attention KV storage as biological input length increases. At 4096 tokens, three-point BioStitch uses 13.692 GiB of peak training memory, compared with 34.344 GiB for BioReason, while prompt self-KV storage is 4.102 versus 676.758 MiB. This cache measurement covers the receiver's prompt self-attention states.
These measurements show that BioStitch keeps training-memory and prompt self-KV costs nearly constant as biological input length increases, while reducing prefill time and time to first token for longer inputs. Comparing single-point and three-point injection also quantifies the additional cost of multi-layer integration, directly addressing how efficiency scales with input length and the number of cross-attention layers.
Response to Reviewer aqS6 (Part 1/5)
We thank Reviewer aqS6 for the careful and constructive review. We appreciate the suggestions on architectural positioning, source-layer selection across task directions, the comparability of the experiments, and the interpretation of computational efficiency and biological results. We address these points below with additional experiments and analyses, which we have incorporated into the revised manuscript.
Response:
Thank you for your suggestion. BioStitch makes source extraction and receiver injection depths explicit design choices and applies a common internal gated cross-attention formulation to biological understanding and language-guided generation. We have clarified the architectural positioning in the Introduction and Related Work and added the component ablations to Section 4.2 and Appendix N.1.
To examine these choices, we retrained seven VE-NonSNV configurations using NT-v2-500M and Qwen3-1.7B with the same data splits, initialization, training seed, and two-stage training budget. The full model uses middle-layer NT features and injects them at shallow, middle, and deep Qwen layers, corresponding to source layer 15 and injection layers 6, 14, and 22. The no_gate variant retains all three connections but fixes the effective coefficient to 1. The single-point variants retain the middle source features and use only one injection point. Finally, source_low and source_high retain all three injection points but use shallow or deep NT features, corresponding to source layers 6 and 23. All configurations use the same final-checkpoint decoding and answer-scoring protocol.
The full configuration achieves the highest scores across all three metrics. Its pathogenic F1 is 2.609 percentage points above the fixed-gate variant and 2.068 points above the strongest single-point variant. Changing to shallow or deep source features gives F1 scores of 73.938% and 77.543%, respectively. These matched comparisons support the combination of learned gating, middle-layer source features, and multi-layer injection, with the full configuration outperforming each of the simpler alternatives considered.