- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
Molecular Dynamics simulation are slow, and standard diffusion models require many forward steps to generate protein conformation ensembles. This work proposes ConformFlow, a normalizing flow (RealNVP) that generates protein conformations in a single forward pass. The flow operates on a latent space obtained from the ESM3 structures encoded from and it is trained on AlphaFold structures and MD simulated data. The authors compare against BioEmu model in both accuracy and speed showing it is competitive in accuracy and significantly improved in speed.
Critical strengths
Strengths
- The work is 1~2 orders of magnitude faster than BioEmu.
- The reported accuracy is competitive with BioEmu.
- Scales NVP Normalizing Flows model to protein ensembles.
- The method section is clear and easy to follow.
Critical weaknesses
Baselines
- BioEmu is the only external baseline. BioEmu is a standard diffusion model that requires many denoising steps, so being faster than it is expected. The literature has since made significant progress in sampling speed (e.g., distillation, consistency models), but these approaches are neither discussed nor compared against.
- A comparison with alternative ensemble generators such as AlphaFlow/ESMFlow would strenghten the paper. Specially AlphaFlow-Lit (Li et al., 2024) which is a distillation of AlphaFlow.
- ESMDiff (Lu et al., 2024) is related but not cited. It also builds on the ESM3 structure tokenizer for sequence conditioned conformation generation, and its main selling point is also efficiency (a 20-100× speedup).
- Other related work, such as Kim et al. (2024) (Scalable Normalizing Flows Enable Boltzmann Generators for Macromolecules) on normalizing flows for proteins, is also not discussed.
Evaluation metrics:
- Only the JSD over four low dimensional variables is reported. Different ensembles can produce similar histograms. A structural validity checks and qualitative visualizations of generated samples would improve the evaluation.
- Details of the metric are missing, such as the histogram binning and the log base which significantly affect the reported values.
Scalability claim
- Scalability is one of the main claims of the paper, but the test protein sizes are not reported, and accuracy as a function of protein length is not shown.
Reliability of Results
- Table 3 (K=4) and Table 4 (four mask ConformFlow) describe different architectures, however 7 entries are identical and all 12 uncertainties match exactly. Is there a reporting error? This makes the reported numbers hard to trust.
- The timing comparison uses a fixed batch size of 64 for ConformFlow, while BioEmu uses its default length dependent batch size (which can vary from 1,500 batch size for octapeptides, to batch size 2 for large protein). Please report the timings with consistent settings.
- The uncertainties in parentheses are not explained, and many of them are exactly 0.000.
Questions for authors
- Tables 3 (K=4) and 4 (four-mask) report identical values for 7 entries, could the authors clarify or correct these results?
- What are the sizes of the test proteins? How does accuracy vary with protein length?
- Which JSD settings were used (binning, log base)?
- Did the authors consider more recent efficient sampling methods, such as consistency models or MeanFlow? It would be interesting to see how such models behave for protein ensemble generation
2 comments
Baselines We now cite and discuss ESMDiff, AlphaFlow-Lit, AlphaFlow/ESMFlow, and Kim et al. 2024 (NF Boltzmann generators for macromolecules). We agree that “faster than a many-step diffusion model is expected”; our point is different and we have rewritten it accordingly: ConformFlow reaches competitive accuracy in a single network evaluation (1 NFE) at protein scale, which is the efficiency regime that distillation/consistency methods approximate for diffusion models — here it is obtained by construction, not by a post-hoc distillation step, and with exact likelihood (on latent space). A strictly controlled empirical comparison to ESMDiff/AlphaFlow-Lit is confounded by different training corpora and conditioning signals, so we use BioEmu (trained on comparable large-scale MD) as the primary reference and discuss the others qualitatively.
Evaluation metrics (validity + qualitative viz; binning/log base) We add backbone structural-validity metrics on generated structures — per-residue (,) JSD against MD and a steric-clash rate — which directly test the “different ensembles, similar histograms” concern, together with FES visualizations (2D TICA and 1D Rg/EED/RMSD) comparing MD, ConformFlow, and BioEmu. We now fully specify the JSD protocol: natural logarithm; 60 equal-width bins for 1D observables; 40x40 for 2D TICA; 20 bins (18x18) for Ramachandran.
Scalability — test sizes and accuracy vs length We now report per-dataset sequence-length ranges (min/median/max): CATH1 50/65/75, octapeptides 8, ATLAS test 38/164/511, and the training corpora up to 512 residues.
Reliability — Table 3 vs Table 4 duplication Thank you for catching this! It was a reporting error. In the submitted version the four-mask column of Table 4 was reported incorrectly. The corrected tables report the true results. The corrected numbers do not change any conclusion (the full odd–even ConformFlow remains best). We have triple checked the results for a week to make sure it reflects the true performance.
Reliability — timing batch size We clarify the setting: each model is run at the largest batch that fits on one L40S GPU — BioEmu’s default length-dependent batch and ConformFlow’s batch of 64 — because BioEmu’s small batch for large proteins is imposed by its much larger memory footprint, not by our choice; this reports realistic end-to-end throughput.
Reliability — many 0.000 The parenthesized values are the standard error of the mean of the per-protein JSD over the N test proteins; entries shown as 0.000 are SEMs below 10^{-3}, which occur on the length-matched octapeptide benchmark where per-protein JSDs are tightly concentrated.
Q — consistency models / MeanFlow These are promising directions. We note that ConformFlow is already single-step (1 NFE), so it occupies the same low-latency regime that consistency/MeanFlow models target for diffusion, but via an exact invertible map rather than a learned shortcut. This is definitely an interesting future direction to explore.
We thank the reviewer for the detailed review. The revision (all additions in blue) makes the following changes that address concerns shared across reviews:
Citations and positioning. We add and discuss Jet, idpSAM, ESMDiff, AlphaFlow-Lit, Kim et al. (NF Boltzmann generators), NF-ULA, and Wu & De la Torre. As noted, most of the individual components of ConformFlow more or less have existed before. However we do not believe this reduces the novelty. ConformFlow provides a generative framework that demonstrates scalability of normalizing flow towards larger proteins’ conformational ensemble generation.
Beyond low-dimensional JSD. We add backbone structural-validity metrics computed on generated structures — per-residue (,) JSD against MD and a steric-clash rate — plus FES visualizations, and we fully specify the JSD protocol.
Sequence separation and test-protein sizes ATLAS is test-only; we quantify train/test redundancy with MMseqs2 and construct a 40-chain test set with 0% detectable overlap at every identity threshold. We report per-dataset sequence-length ranges.
Corrected ablation tables The submitted Table 3/Table 4 contained an error (the four-mask column duplicated the K=4 column). We sincerely apologize for the mistake and have used a week to triple check all results and make sure they demonstrate the true performance of the model and benchmark.
Training/compute specification We report parameter counts for all variants and the autoencoder, and note that the four flow variants are matched in parameters, layers, conditioner, batch, epochs, and hardware, so the ablation isolates architecture from budget.