- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces Geometry-Consistent Distillation (GCD), which adapts SiD to accelerate sequence-conditioned protein conformational ensemble generation for AlphaFlow and ESMFlow. The authors motivate the paper by the fact that typical Euclidean SiD as used in images, for example, results in structural collapse. Therefore, the paper proposes a method similar to SiD in which harmonic prior and Kabsch alignment is used for noising. They also derive the corresponding harmonic score-velocity identity and compute both the auxiliary projector loss and the generator score-distillation loss in residue-local coordinate frames. The paper evaluates 1 and few-step distilled students on the 100-target PDB ensemble benchmark and the 82-target ATLAS MD benchmark.
Critical strengths
The general issue identified with a simple application of SiD to a model like AlphaFlow is correct and the general problem of accelerating these models is definitely interesting and relevant.
Critical weaknesses
1.Paper's Motivation:
- One of the main issues of the paper in my opinion is that it seems to be solving an AlphaFlow-specific issue. AlphaFlow is especially slow at inference as it requires injecting the noisy structure into AlphaFold2's template stack before the Evoformer trunk, which results in many Evoformer passes to generate a small ensemble. However, more recent architectures such as AlphaFold 3 [1], Boltz-1 [2], which run the expensive trunk once and roll out only a lightweight diffusion/structure head. Even the more recent AlphaFlow alternative, AlphaFlow-Lit [3] achieves a speedup over AlphaFlow without distillation by simply caching the Evoformer trunk.
- Additionally, method loses relevance when considering all-atom and sequence–structure co-generation models. While the introduction section motivates the method via de novo protein design and inverse folding, all experiments are restricted to sequence-conditioned generation. The specific distillation approach in GCD also does not transfer to modern all-atom diffusion models (such as AlphaFold 3), or joint sequence–structure co-generation models (such as DISCO [4]). Have the authors considered how a 3-network bilevel distillation framework remain relevant for more recent approaches that rely on cached-trunk architectures or all-atom diffusion?
- Issues with the methodology and ablations:
- Training GCD-SiD requires maintaining 3 full AlphaFold/ESMFold models simultaneously in GPU memory and backpropagating through the teacher and projector at every generator update. This cost is significant and prohibitive in many cases. More importantly, the paper fails to include any training compute metrics in the results.
- Additionally, I found the ablations to be confounding multiple factors. Moving from SiD (0.49% Precision reported in Table 1a) to GCD-SiD (81.10% Precision) is essentially changing three things at one: (1) using the harmonic prior (2) applying Kabsch RMSDAlign and (3) computing losses in residue-local frames. The AlphaFlow teacher was originally trained with the harmonic prior and RMSDAlign so feeding unaligned isotropic Gaussian noise as is done in the SiD baselines is simply OOD for the teacher. There is no ablation isolating the local-frame score loss from simply matching the teacher's native corruption process. It would be helpful to include the performance of Euclidean SiD when using the teacher's native harmonic prior and RMSDAlign ) without the local-frame objective.
- Baselines are incomplete and experimental results are overstated:
- AlphaFlow's native 1-step distilled baseline (AlphaFlow-Distilled) and few-step teacher truncation are not included in the Tables 1, 2, and 3.
- There also seems to be a mode collapse in GCD-DMD and GCD-SC which contradicts transferability claims made in the paper. In the abstract and in Section 4, the paper claim that GCD transfers effectively across distillation paradigms (GCD-DMD and GCD-SC): “The table also includes GCD-DMD and GCD-SC as additional controls, demonstrating that the geometry-consistent formulation can be incorporated into multiple distillation paradigms.” However, Table 1a shows that both GCD-DMD and GCD-SC seem to experience significant mode collapse as the diversity drops to 2.35% for GCD-DMD and 2.59% for GCD-SC from the 18.47% Alphaflow teacher model.
- Finally, the parametrization, training and sampling of the few step GCD models are not explained. Tables 1, and 2 show results for 2-step and 4-step distillation, however, the paper never specifies how these few-step models are parameterized or trained. In Section 3 and Algorithm 1 the student generator is written as a 1-step map without noisy structure or timestep arguments. Therefore, it isn’t clear to me how this model can be used for few-step distillation as well.
[1] Abramson, J., Adler, J., Dunger, J. et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature 630, 493–500 (2024). https://doi.org/10.1038/s41586-024-07487-w
[2] Boltz-1 Democratizing Biomolecular Interaction Modeling Jeremy Wohlwend, Gabriele Corso, Saro Passaro, Mateo Reveiz, Ken Leidal, Wojtek Swiderski, Tally Portnoi, Itamar Chinn, Jacob Silterra, Tommi Jaakkola, Regina Barzilay bioRxiv 2024.11.19.624167; doi: https://doi.org/10.1101/2024.11.19.624167
[3] Li, S., Li, M., Wang, Y., He, X., Zheng, N., Zhang, J., & Heng, P.-A. (2024). Improving AlphaFlow for Efficient Protein Ensembles Generation. https://arxiv.org/abs/2407.12053
[4] Rector-Brooks, J., Lambert, T., Skreta, M., Roth, D., Long, Y., Li, Z.-Q., Zhang, X., Cretu, M., Li, F.-Z., Ganapathy, T., Jin, E., Bose, A. J., Yang, J., Neklyudov, K., Bengio, Y., Tong, A., Arnold, F. H., & Liu, C.-H. (2026). General Multimodal Protein Design Enables DNA-Encoding of Chemistry. https://arxiv.org/abs/2604.05181
Questions for authors
See questions above.
3 comments
3. Teacher-matched corruption and the requested Cartesian control
The collapsed Euclidean SiD baseline and GCD use the same harmonic prior, RMSD-aligned interpolation, frozen AlphaFlow teacher, and corrupted teacher inputs . Here, 'Euclidean' refers to the distillation discrepancy, not the noising process. In particular, the Cartesian baseline does not use isotropic Gaussian noise or omit RMSD alignment. The two formulations differ in how their teacher–projector–generator residuals are constructed: the baseline compares predictions directly in unframed global 3D coordinates, whereas GCD uses residue-local frame alignment together with its geometric loss construction.
Thus, the teacher-native harmonic-prior and RMSD-aligned Cartesian control requested in the review is already the reported Euclidean SiD baseline, which gives 0.49/0.44/0.00 Precision/Recall/Diversity. These comparisons support the importance of geometry-consistent distillation discrepancies rather than an out-of-distribution corruption mismatch.
4. Native distillation and reduced-step teacher baselines
We additionally evaluated the original AlphaFlow teacher at reduced sampling steps and the publicly released native supervised AlphaFlow-Distilled model. All numbers below are median PDB-ensemble Precision/Recall/Diversity (%) under the same evaluation protocol.
| Method | Steps | Precision | Recall | Diversity |
|---|---|---|---|---|
| AlphaFlow teacher | 1 | 84.02 | 79.64 | 2.74 |
| AlphaFlow teacher | 2 | 83.32 | 81.27 | 9.11 |
| AlphaFlow teacher | 4 | 82.24 | 80.63 | 14.50 |
| AlphaFlow teacher | 10 | 80.19 | 79.78 | 18.47 |
| AlphaFlow-Distilled | 1 | 82.75 | 80.11 | 9.60 |
| GCD-SiD | 1 | 81.10 | 80.47 | 18.31 |
| GCD-SiD | 2 | 82.16 | 81.66 | 17.39 |
| GCD-SiD | 4 | 79.53 | 80.00 | 21.92 |
These results distinguish one-step ensemble preservation from one-step structure prediction alone. Direct one-step teacher sampling gives high precision (84.02) but only 2.74 diversity. AlphaFlow-Distilled, which uses supervised targets obtained from complete teacher rollouts, achieves 82.75/80.11/9.60. In comparison, one-step GCD-SiD increases diversity by 8.71 percentage points and recall by 0.36 points, with a 1.65-point precision trade-off. Its diversity remains close to the ten-step teacher. At two steps, GCD-SiD also produces a substantially broader ensemble than the truncated two-step teacher (17.39 versus 9.11 diversity), while retaining comparable precision and recall.
5. Generality across distillation objectives
The GCD-DMD and GCD-SC experiments are not ablations of SiD; they use different training objectives and update rules. SiD uses an auxiliary projector to approximate the student-side score and constructs a score-identity-based distribution-matching objective; DMD adopts KL-based distribution matching; and consistency distillation enforces consistency across a teacher trajectory. The resulting fidelity–diversity trade-offs can differ. Indeed, GCD-DMD and GCD-SC retain high Precision/Recall (83.21/79.59 and 83.49/79.57) but produce narrower ensembles (2.35 and 2.59 diversity). This differs from the Cartesian SiD baseline, whose Precision/Recall fall to 0.49/0.44, indicating severe structural degeneration. Thus, the geometric construction supports accurate structural generation across these objectives, with GCD-SiD providing the strongest one-step ensemble preservation among the evaluated variants.
6. Parameterization and few-step training
Our few-step sampling follows the original AlphaFlow sampling procedure, iteratively applying the same time-conditioned student network . Starting from harmonic noise , each step predicts a clean structure and updates the intermediate state:
\qquad \hat x_k=[\widehat S_k]_\beta,$$ $$x^{(k)}= \frac{\tau_k}{\tau_{k-1}} \operatorname{RMSDAlign}(x^{(k-1)},\hat x_k) +\left(1-\frac{\tau_k}{\tau_{k-1}}\right)\hat x_k,$$ where $d(\cdot)$ denotes pairwise structural distances and ${\tau_k}_{k=0}^{K}$ is the sampling schedule. For training, we uniformly sample $J\in{1,\ldots,K}$ and apply the GCD objective to the prediction at step $J$, with the preceding $J-1$ steps executed without gradient tracking. The student prediction is detached during projector training, while generator training backpropagates through the selected step with fixed teacher and projector parameters. At inference, all $K$ steps are performed to obtain the final structure $\widehat S_K$.
We thank the reviewer for emphasizing the computational challenges of iterative protein generation and for asking how the proposed formulation relates to more recent architectures. We clarify the ensemble-generation objective, the actual training costs, and the controlled baseline comparisons.
1. Motivation and applicability beyond AlphaFlow
Our work addresses a different question from accelerating single-structure prediction: preserving a conditional protein conformational ensemble distribution during one-step distillation. The novelty of our work is one-step ensemble-preserving distillation, rather than single-step generation alone.
The computational bottleneck addressed by GCD is not limited to repeated Evoformer computation. Diffusion and flow-matching models typically generate samples by progressively denoising an initial noise sample toward a plausible structure, a process that often requires tens to hundreds of sampling steps. AAlphaFlow-Lit accelerates this process by caching conditioning features and avoiding redundant computation across sampling steps. In contrast, GCD reduces the number of sampling steps required for generation through distillation. These approaches target complementary aspects of inference efficiency: AlphaFlow-Lit reduces the computational cost per step, while GCD reduces the number of steps. Consequently, the two approaches are compatible in principle and could be combined to achieve further acceleration.
Beyond inference efficiency, the reviewer also raises the question of whether GCD can be extended to newer all-atom and joint sequence–structure generative models. Importantly, AlphaFlow/ESMFlow and our distilled students already generate all-heavy-atom protein structures, including both backbone and side-chain atoms, through their AlphaFold/ESMFold structure modules. The pseudo-β coordinates in our formulation define the stochastic flow state rather than limit the resolution of the generated structures. For AF3-style models, which operate on atom-coordinate diffusion states, extending GCD would require adapting the distillation objective to the corresponding denoising/score parameterization and atom-level geometric representation. Joint sequence–structure generators additionally require handling discrete sequence variables. Thus, while the specific formulation depends on the teacher's generative representation, the underlying principle of geometry-consistent distributional distillation is not inherently restricted to AlphaFlow-like architectures.
2. Training cost and the projector–generator scheme
Compared with standard teacher–student distillation, our bilevel setup introduces an additional projector network, which increases training memory requirements. However, AlphaFlow's native supervised distillation requires complete teacher rollouts to construct training targets, whereas GCD-SiD queries the teacher at student-induced corrupted states without requiring full teacher trajectories.
We additionally measured peak GPU memory and training throughput under the same single-H200, batch-size-1 configuration:
GCD-SiD requires higher peak memory during generator updates because gradients propagate through the frozen teacher and projector. Nevertheless, its measured training throughput corresponds to approximately 35% less training time than online supervised distillation under the matched configuration. The 24k-sample setting corresponds to our actual GCD-SiD training budget, while 160k samples is the training budget used by the officially released AlphaFlow-Distilled model.