- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes CrypticFlow, which is a hierarchical flow matching model for apo-to-holo protein generation. This method first generates coarse Ca anchors in Cartesian space using EGNN, then iteratively fills in residues in local internal coordinates. Finally, it also applies some hard rules to ensure geometry quality. In the D3PM test set benchmark, CrypticFlow achieves both strong holo state prediction and great local geometry preservation compared with other baseline methods.
Critical strengths
- Modeling the apo-to-holo states using a hierarchical approach, first generating coarse coordinates in Cartesian space, then middle residues with local internal coordinates, is a good way to address poor local geometric fidelity in purely Cartesian models and global error accumulation in chain-wide internal-coordinate generation.
- Well-designed experiments and ablations to show the effectiveness of some components, including but not limited to the ESM-3 conditioning, local internal coordinates, and different anchor strides, which makes the results more convincing.
Critical weaknesses
- The method is developed in the absence of ligands, which means it only works for the protein monomer apo-to-holo conformation change. In real SBDD, people may care more about the conformational change when proteins bind to different ligands. This is not an issue for the method proposed, but it can make the problem statement and application weaker.
- The model is trained and evaluated on the D3PM dataset. It would be interesting to show how it performs on an external validation set and on classical apo-to-holo targets to demonstrate its generalization ability.
Questions for authors
- Have you thought about doing an exposure-bias training to solve the potential train-inference mismatch, where during training it always sees the ground-truth coarse Ca structures, but in inference it would rely on its own generated structures? An example work is https://arxiv.org/pdf/2602.04883, which also does hierarchical protein generation, and they show that exposure-bias training is helpful for solving this problem.
- The midpoint flow is stochastic, so the model could in principle generate multiple holo-like conformations for the same apo input. Have the authors evaluated whether repeated samples form a meaningful conformational ensemble e.g., in terms of structural diversity or coverage of alternative holo states?
3 comments
Response to Reviewer LaNz — Q2. Do repeated samples form an ensemble?
We evaluated this. Using each of the five trained models reported in the paper (seeds 1, 3, 42, 88, 99), we generated 20 samples for each of the 150 test pairs. Values are mean ± s.d. over the five seeds, and RMSD is the direct Cα RMSD without superposition, as in the paper.
- Structural diversity. We define the diversity of a target as the mean pairwise Cα RMSD among its 20 samples. Over the 150 targets, the median is 0.020 ± 0.003 Å and the 90th percentile is 0.22 ± 0.05 Å. Compared with the median apo–holo RMSD of 4.46 Å on the test set, the samples are effectively the same structure.
- Cause. The top-level EGNN anchor flow is a deterministic ODE that starts from the apo structure, so all 20 samples share identical top-level anchors (0.0 Å difference). Stochasticity enters only through the midpoint flow, which generates the midpoints with these fixed anchors as parents. That is, once the EGNN has determined the top-level anchors, the overall shape of the structure is constrained, and the stochasticity of the midpoint stage alone does not appear to produce different conformations.
- Coverage. Here the oracle is the single sample, out of the 20, with the lowest Cα RMSD to the true holo structure (direct RMSD as defined in the paper: no superposition, over the commonly observed residues). Even so, it gives almost the same accuracy as a single sample.
- Local geometry and structural quality are also unchanged. Adjacent Cα–Cα distance validity has the same five-seed mean for both choices (within a seed they differ by at most one bond out of 54,509), and Cα angle validity is 0.050 pp higher for the oracle. MolProbity is 0.007 lower for the oracle. Scoring the same structures again shifts it by as much, so this is not a difference we would read as an improvement in quality.
| CrypticFlow Single sample | CrypticFlow Best of 20 (oracle) | |
|---|---|---|
| Median RMSD (Å) ↓ | 1.333 ± 0.056 | 1.330 ± 0.060 |
| Mean RMSD (Å) ↓ | 2.766 ± 0.041 | 2.756 ± 0.042 |
| < 2 Å (%) ↑ | 58.53 ± 0.73 | 58.80 ± 0.56 |
| Cα–Cα distance valid (%) ↑ | 99.58 ± 0.02 | 99.58 ± 0.02 |
| Cα angle valid (%) ↑ | 95.87 ± 0.25 | 95.92 ± 0.25 |
| MolProbity ↓ | 3.031 ± 0.052 | 3.024 ± 0.054 |
Therefore, CrypticFlow in its current form does not appear to generate structural diversity or alternative holo states. It is not an ensemble generator but a nearly deterministic predictor that maps one apo structure to one holo-like structure. We will state this explicitly in the limitations section of the revision.
I appreciate the authors' efforts to address these concerns. I will maintain the score of 3.
Response to Reviewer LaNz — Q1. Exposure-bias training
Yes, we tried this. As you point out, in standard training the midpoint flow sees only ground-truth parent contexts. We therefore applied the two methods of PAR that you suggested to CrypticFlow and retrained it with seed 1, keeping every other setting unchanged.
Test results (150 pairs, seed 1):
This appears to be because, in CrypticFlow, the train–inference mismatch does not amplify errors. At inference, the midpoint flow receives as parents the top-level anchors generated by the model, which differ from the ground truth. If the mismatch were a problem, the error of the final structure, after the midpoints are filled in, would exceed the error of the anchors. However, on the 150 test pairs (five seeds), the RMSD of the final structure was almost identical to the RMSD measured on the anchor residues alone (median difference +0.013 ± 0.004 Å). The midpoint stage thus adds no error even when given incorrect anchors, and the final error is set by the error of the top-level anchors. Because SS and NCL change only the training of the midpoint stage and leave the training of the EGNN that produces the anchors unchanged, they cannot reduce this error. We will add these results to the appendix of the revision.