- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents a way to predict holo structures from apo inputs. The work does this by using a protein language model conditioner and an MMDiT style flow trunk, and learns a vector field between apoo and holo structures. On the standard D3PM benchmark, the method significantly outperforms prior work.
Critical strengths
The paper's strongest claim is that the results are exceptionally strong, and a significant improvement over prior work.
Critical weaknesses
I have several methodological issues that I think make it difficult to immediately support acceptance:
- There's a significant body of work on this topic, and it's quite hard to understand what's driving the strong results. In paticular, works like [1] and [2] tackle similar (though non idetntical) problems. Historically, it's been challenging to directly push apo to holo structures, so I wonder what this method is doing differently and think that a rigorous ablation is important. To be clear, I don't mean listing the differences -- I think, to the paper's credit, this is quite clear -- I mean actually ablating each component to see what's enabling this.
- [1] found that there's significant overlap between the test set and Uniprot. Does this work use a correctly deduplicated split?
- My rough calculation estimates 365M parameters in this model (12 dual stream + 12 single stream layers). There are only 1200 apo-holo pairs. I find it very hard to imagine training a significant model with this mismatch, so I think this needs to be properly justified and explained.
If these questions were properly explored and answered, I would shift to acceptance.
[1] Zhang, Xuejin, et al. "Bending and binding: Predicting protein flexibility upon ligand interaction using diffusion models." NeurIPS 2023 Generative AI and Biology (GenBio) Workshop. 2023. [2] Corso, Gabriele, et al. "Composing unbalanced flows for flexible docking and relaxation." International Conference on Learning Representations. Vol. 2025. 2025.
Questions for authors
- What is driving the strong results?
- Why can a very small dataset train a relatively large transformer?
7 comments
What is driving the strong results?
“There's a significant body of work on this topic, and it's quite hard to understand what's driving the strong results.”
“Historically, it's been challenging to directly push apo to holo structures, so I wonder what this method is doing differently...”
“To be clear, I don't mean listing the differences ... I mean actually ablating each component to see what's enabling this.”
In short, the data point to the training recipe, above all the relative-displacement loss, to the sequence stream, and to added capacity only together with the language-model embedding. We agree that the different sources of the performance should be separated. We share your caution, since directly pushing apo structures to holo has long been difficult. Listing architectural differences is therefore not enough; we ablated the components directly and separately examined benchmark overlap and model capacity.
On the public split, the released configuration gives a median Cα RMSD of 0.809 Å; without the relative-displacement loss it gives 1.455 Å.
By contrast, changing the particular sequence representation does not worsen the median beyond the variation across retrainings. A fixed amino-acid-type vector gives 0.784 Å, ESM-2 gives 0.780 Å, ESM-C gives 0.757 Å, and removing the global token gives 0.772 Å. Five seeds of the released configuration span 0.761–0.836 Å.
Removing the sequence stream entirely gives 1.004 Å (Appendix A.14). Amino-acid identity remains available to that model through the structure stream’s one-hot node features, so this comparison removes the pretrained sequence representation and dual-stream interaction rather than all residue identity.
Section 4 then measures what the joint attention learns, beyond what accuracy alone shows. The revision moves the four reductions of the joint map, previously Figure A6 in the appendix, into the main text as Figure 5. Within the structure stream, received attention identifies moving residues more strongly than emitted attention, and this asymmetry is learned. After controlling for distance from the apo centroid, the partial rank correlation between received attention and total distance change is 0.046–0.145, exceeding matched untrained networks in all 9 trainings (p = 0.0039).
The Conclusion now adds one association on the protein-disjoint split: “pairs whose structure-stream attention marks the moving residues more closely show less off-target motion (Spearman −0.42 over 58 pairs, Holm-adjusted p = 0.035, Appendix A.9).” Appendix A.9 defines both quantities.
The public split also contains substantial overlap with training. Among 109 chain-sharing test pairs with a training partner, 105 have the same conformational change represented in training. Copying the displacement of the nearest training partner gives 0.438 Å median RMSD on those pairs, compared with 0.478 Å for A2Holo. Across the 123 test pairs that have a training partner (not the same set as the 123 UniProt-sharing pairs of Appendix A.9), A2Holo’s error correlates with the corresponding copying error at Spearman +0.868.
A same-size model that replaces joint attention with cross-attention, in which each stream attends only to the other, is one of the seven models in Appendix A.15, and A2Holo has the lowest median of the seven (Table A6). The sequence-stream ablation and the seven models are reported in Appendices A.14–A.15; the public-split training-component runs above are additional to the paper. The updated anonymous release links the checkpoints of the seven models (CHECKPOINTS.md), with their training options.
Model size, sequence information, and the small training set
“My rough calculation estimates 365M parameters in this model ... There are only 1200 apo-holo pairs.”
“Why can a very small dataset train a relatively large transformer?”
In short, the model does fit its training pairs closely, and in the 2×2 comparison below on the protein-disjoint split its extra capacity pays off only when the second stream carries the language-model embedding. Your estimate is close: A2Holo has 368.8M parameters (368,807,959), and the public split contains 1,291 training pairs.
The model fits that training set closely: across five seeds, the training median Cα RMSD is 0.157–0.212 Å with one Euler step. We therefore agree that training fit itself is not the informative question. The more relevant question is whether the additional capacity improves held-out prediction.
Appendix A.15 addresses this directly using a 2×2 comparison on the protein-disjoint split (one training per model; blocks are double-stream + single-stream). The structure-copy model keeps the 12+12 dual-stream architecture, but its second stream receives a copy of the structure input instead of the embedding, with no global token.
| Second-stream content | Smaller model | 12+12 model |
|---|---|---|
| Ankh3 embedding | 4+4, 127.9M: 5.362 Å | A2Holo, 368.8M: 4.998 Å |
| No LM information | 0+12, 113.7M: 5.235 Å | Structure-copy, 367.8M: 5.283 Å |
With language-model information, increasing from 4+4 to 12+12 lowers Cα RMSD in 37/58 protein-disjoint test pairs (p = 0.0037). Without language-model information, increasing from 0+12 to the size-matched 12+12 structure-copy model instead raises RMSD in 36/58 pairs (p = 8.4×10⁻⁴). The two effects differ in 38/58 pairs (p = 3.1×10⁻⁴).
Thus, in this 2×2 comparison, the larger dual-stream model pays off only when its second stream carries the language-model embedding.
The public-split depth experiments give the same caution against attributing performance simply to parameter count. The 4+4 model (127.9M parameters) does not differ significantly from 12+12: 12+12 has lower RMSD in 84/150 pairs and 4+4 in 66/150 (p = 0.087). Models with 8+8, 10+10, 14+14, and 16+16 blocks, spanning 248–489M parameters, give median RMSDs of 0.747–0.835 Å, comparable to the 0.761–0.836 Å range across five seeds of the released 12+12 model.
The released 12+12 configuration was fixed before these depth comparisons rather than selected after observing them.
We therefore do not argue that the dataset size is large relative to the parameter count.
Overall response and revision map
We thank Reviewer T8CD for a careful and specific review. Figure, table, and appendix numbers below refer to the revised manuscript.
We thank you for the summary.
Because the exceptionally strong headline performance is also the point that most requires explanation, the revision ablates its components and repeats the comparison on a protein-disjoint split.
We investigated your concerns along four axes: