- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Exploration-NFT post-trains protein dynamics generators to explore a broader range of conformations. Trajectories starting from the same structure define a shared, non-whitened TICA space. Each trajectory segment receives a reward for visiting new grid cells, with a separate penalty for structural invalidity. DiffusionNFT uses these rewards to update the denoiser while keeping each generator’s original sampling procedure. Experiments use BioKinema and DyneTrion on ATLAS, CATH1, and fast-folding proteins, measuring structural diversity, reference-MD coverage, ensemble agreement, and kinetics. The paper also tests transfer across protein lengths and temporal intervals.
Critical strengths
-
Exploration-NFT can post-train different protein dynamics generators without replacing their sampling procedures. The reward uses generated coordinates and topology without accessing reference MD trajectories. For the update, DiffusionNFT reuses each backbone’s forward corruption process and denoising prediction, avoiding the need to evaluate sampling likelihoods (Sections 3.1–3.4). The experiments demonstrate this with BioKinema and DyneTrion, which use different samplers and generate trajectories in different-sized blocks. Coverage improves for both models on CATH1 and fast-folding proteins (Tables 2–3).
-
The BioKinema results show substantial coverage gains and better agreement with reference MD on fast-folding proteins. At step 800, ATLAS [email protected] increases from 0.150 to 0.336, CATH1 coverage from 0.525 to 0.772, and fast-folding coverage from 0.576 to 0.791 (Tables 1–3). On fast-folding proteins, RMWD also decreases from 6.041 to 4.087 Å and occupancy JSD from 0.496 to 0.429. For MD-referenced coverage evaluation, each protein has a fixed TICA basis fitted only to reference MD, independently of the coordinates used for the training reward (Appendix C).
Critical weaknesses
-
The coverage term rewards new grid cells, which can also be reached by taking larger conformational steps. The experiments do not show whether segment-wise credit helps the model keep finding new regions beyond this effect. On fast-folding proteins, the 10-ns generated-to-MD mean TICA-displacement ratio increases from 0.874 for pretrained BioKinema to 2.221 at step 800 (Table 6). On CATH1, this checkpoint increases coverage while worsening RMWD from 1.654 to 2.562 Å and transition JSD from 0.434 to 0.611 (Table 2). The authors discuss this tradeoff and focus their contribution on expanding coverage. The comparisons use the complete procedure against pretrained models, without a simpler coverage reward or a control producing comparable displacement. Figures 1 and 3 compare training checkpoints rather than discovery over rollout time.
-
The physical-validity checks cover only a narrow set of structural errors. The bond criteria check whether consecutive Cα distances are below 4.5 Å and C–N distances below 2.0 Å, without checking lower distance bounds, bond angles, or torsions (Appendix B.2). Clash detection considers only N, Cα, C, O, and Cβ atoms, using a 1 Å cutoff. Evaluation first calculates the fraction of eligible nonlocal atom pairs that do not clash in each frame, then averages across frames (Appendix C.1). A frame can contain a few severe clashes and still receive a score close to 100%. Training uses a residue-level clash gate, which differs from this reported percentage. Section 5.1 uses the near-perfect validity scores to argue that the additional diversity comes from physically admissible frames. That conclusion needs broader structural validation.
Questions for authors
-
Can the authors compare segment-wise rewards with a single reward for total coverage at the end of the rollout, using the same training and sampling budgets? A separate comparison with the pretrained sampler adjusted to produce similar mean TICA displacement would help test how much larger motions explain the gains. For these comparisons, can the authors report cumulative coverage and new-cell discovery over rollout time in the same fixed evaluation space, especially during later segments? These comparisons could use one existing benchmark.
-
For the existing generated trajectories, can the authors report how many frames and residues contain clashes, along with bond-length, bond-angle, and torsion statistics? Please also report the fraction of frames failing the residue-level clash criterion used during training. Do the coverage and RMWD improvements persist when the same stricter geometry checks are applied to each model?
3 comments
Q2: For the existing generated trajectories, can the authors report how many frames and residues contain clashes, along with bond-length, bond-angle, and torsion statistics? Please also report the fraction of frames failing the residue-level clash criterion used during training. Do the coverage and RMWD improvements persist when the same stricter geometry checks are applied to each model?
R2: We thank the reviewer for this suggestion. We evaluated the pretrained BioKinema and BioKinema–NFT at 800 training steps on the same 12 fast-folding proteins. Each model generated a 100 μs trajectory at 10 ns/frame for each protein. All reported values are unweighted means across proteins.
We first quantified clashes using the same residue-level criterion as training: clashes are detected among N, Cα, C, O, and Cβ atoms with a 1.0 Å cutoff, excluding residue pairs separated by at most two positions along the same chain. A clashing-residue occurrence denotes one residue involved in a clash in one frame.
| Model | Mean clash-containing frames | Mean clashing-residue occurrences | Clash–fail@98% |
|---|---|---|---|
| Pretrained BioKinema | 11.83 / 10,000 (0.118%) | 28.75 / 435,000 (0.0066%) | 0.118% |
| BioKinema–NFT | 97.67 / 10,000 (0.977%) | 252.33 / 435,000 (0.0580%) | 0.977% |
The average protein contains 43.5 residues; therefore, each 10,000-frame trajectory contains an average of 435,000 residue-frame observations. Clash–fail@98% is the fraction of frames in which fewer than 98% of residues are clash-free.
We further measured backbone bond lengths, bond angles, and Ramachandran statistics:
| Model | N–Cα (Å) | Cα–C (Å) | C–N (Å) | C–O (Å) | C–N–Cα (°) | N–Cα–C (°) | Cα–C–N (°) | Ramachandran outlier |
|---|---|---|---|---|---|---|---|---|
| Pretrained BioKinema | 1.454 | 1.520 | 1.321 | 1.222 | 124.087 | 112.913 | 117.874 | 1.325% |
| BioKinema–NFT | 1.452 | 1.517 | 1.304 | 1.220 | 123.770 | 113.272 | 117.536 | 4.014% |
Ramachandran outliers are defined using the general allowed region of the MDAnalysis reference map. The mean bond lengths and angles remain close to those of the pretrained model, with only a modest increase in the Ramachandran-outlier rate.
Finally, we recomputed Coverage and RMWD after retaining only Valid@98% frames. A frame is Valid@98% when its valid Cα–Cα bond fraction, valid C–N bond fraction, and residue-level clash-free fraction are all at least 98%.
| Model | Coverage–All ↑ | Coverage–Valid98 ↑ | RMWD–All (Å) ↓ | RMWD–Valid98 (Å) ↓ |
|---|---|---|---|---|
| Pretrained BioKinema | 0.273 | 0.270 | 6.455 | 6.468 |
| BioKinema–NFT | 0.509 | 0.501 | 4.738 | 4.775 |
After applying the same validity filter to both models, BioKinema–NFT still improves Coverage by 85.7% and reduces RMWD by 26.2% relative to the pretrained model. Thus, the ensemble improvements persist after excluding frames that fail the training-aligned structural-validity criterion.
Dear Reviewer HHaH,
We hope our response has addressed your questions and concerns. We would greatly appreciate any further feedback and would be happy to clarify or address any remaining issues.
Thank you once again for the time and effort you have dedicated to reviewing our paper.
Best regards,
Submission59851 Authors
Q1: Can the authors compare segment-wise rewards with a single reward for total coverage at the end of the rollout, using the same training and sampling budgets? A separate comparison with the pretrained sampler adjusted to produce similar mean TICA displacement would help test how much larger motions explain the gains. For these comparisons, can the authors report cumulative coverage and new-cell discovery over rollout time in the same fixed evaluation space, especially during later segments? These comparisons could use one existing benchmark.
R1: We thank the reviewer for this suggestion. We added two controlled comparisons on the Fast-Folding benchmark.
First, we trained a terminal-reward baseline using the same coverage definition, validity regularization, training setup, and sampling budget as Exploration–NFT. The final cumulative coverage after the complete 1 μs rollout is used as a single trajectory-level reward.
Second, we constructed a motion-amplitude control by increasing the pretrained sampler’s noise scale from 1.75 to 1.98. This value was selected on 16 stratified ATLAS training proteins and matched the mean 100 ns TICA displacement of Exploration–NFT within 1.75% (0.06750 versus 0.06634).
For evaluation, each 1 μs trajectory was divided into ten 100 ns segments. All methods were projected into the same frozen, protein-specific TICA space and evaluated using the same fixed 50×50 grid. Cumulative coverage is the fraction of MD-supported cells visited up to a given segment, while new coverage counts cells first visited in that segment. We only count frames for which the valid Cα–Cα, valid C–N, and residue-level clash-free fractions are each at least 98%. Valid@98% denotes the percentage of frames satisfying this criterion.
The temporal evolution is shown as cumulative coverage / newly discovered coverage, both in percentage points:
At the end of the rollout, segment-wise credit assignment improves cumulative coverage from 10.224% to 10.869% relative to the terminal-reward baseline. It also increases discovery during segments 6–10 from 3.604 to 4.113 percentage points, a 14.1% relative improvement, supporting the benefit of segment-wise credit assignment during later rollout stages. The terminal-reward baseline also exhibits a more irregular late-stage discovery profile, whereas segment-wise credit assignment sustains new coverage more smoothly across later segments.
Although the noise=1.98 sampler’s mean displacement on Fast Folding was 18.3% larger than that of Exploration–NFT, its lower valid coverage indicates that increasing conformational motion alone provides limited coverage gains. Overall, segment-wise training achieves the highest valid cumulative coverage and late-segment discovery under the controlled training and sampling budget.