acceptodds
Withdrawn from ICLR 2027

Exploration-NFT: Reinforcing Protein Dynamics Generators for Persistent Conformational Exploration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The scarcity of long-timescale molecular dynamics (MD) trajectories restricts protein dynamics generators to short-horizon supervision, biasing autoregressive rollouts toward local free-energy basins. We present Exploration-NFT, a backbone-agnostic forward-process reinforcement framework for persistent conformational exploration. Same-condition rollout groups define a shared, non-whitened time-lagged independent component analysis space; segment-wise new-cell increments provide temporal exploration rewards, while structural validity is regularized independently. DiffusionNFT maps these rewards to denoising updates without modifying native samplers or evaluating their likelihoods. We evaluate Exploration-NFT with BioKinema and DyneTrion on 100-ns ATLAS and 10-µs CATH1 and fast-folding benchmarks. Across both backbones and all three benchmarks, post-training improves valid diversity or reference-MD coverage while maintaining all reported local-validity scores above 99.96%. For BioKinema, ATLAS diversity increases 2.24×, and CATH1 and fast-folding coverage increases by 47% and 37%, respectively. At the maximum-coverage fast-folding checkpoint, all reported thermodynamic and kinetic discrepancies also decrease. Improvements beyond the training sequence-length range and at unseen temporal intervals further demonstrate transfer across conditions. These results establish forward-process reinforcement as an effective mechanism for expanding the conformational support of protein dynamics generators.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
protein dynamics, conformational exploration, reinforcement learning, generative modeling, diffusion models, molecular dynamics, enhanced sampling, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HHaHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Exploration-NFT post-trains protein dynamics generators to explore a broader range of conformations. Trajectories starting from the same structure define a shared, non-whitened TICA space. Each trajectory segment receives a reward for visiting new grid cells, with a separate penalty for structural invalidity. DiffusionNFT uses these rewards to update the denoiser while keeping each generator’s original sampling procedure. Experiments use BioKinema and DyneTrion on ATLAS, CATH1, and fast-folding proteins, measuring structural diversity, reference-MD coverage, ensemble agreement, and kinetics. The paper also tests transfer across protein lengths and temporal intervals.

Critical strengths
  1. Exploration-NFT can post-train different protein dynamics generators without replacing their sampling procedures. The reward uses generated coordinates and topology without accessing reference MD trajectories. For the update, DiffusionNFT reuses each backbone’s forward corruption process and denoising prediction, avoiding the need to evaluate sampling likelihoods (Sections 3.1–3.4). The experiments demonstrate this with BioKinema and DyneTrion, which use different samplers and generate trajectories in different-sized blocks. Coverage improves for both models on CATH1 and fast-folding proteins (Tables 2–3).

  2. The BioKinema results show substantial coverage gains and better agreement with reference MD on fast-folding proteins. At step 800, ATLAS [email protected] increases from 0.150 to 0.336, CATH1 coverage from 0.525 to 0.772, and fast-folding coverage from 0.576 to 0.791 (Tables 1–3). On fast-folding proteins, RMWD also decreases from 6.041 to 4.087 Å and occupancy JSD from 0.496 to 0.429. For MD-referenced coverage evaluation, each protein has a fixed TICA basis fitted only to reference MD, independently of the coordinates used for the training reward (Appendix C).

Critical weaknesses
  1. The coverage term rewards new grid cells, which can also be reached by taking larger conformational steps. The experiments do not show whether segment-wise credit helps the model keep finding new regions beyond this effect. On fast-folding proteins, the 10-ns generated-to-MD mean TICA-displacement ratio increases from 0.874 for pretrained BioKinema to 2.221 at step 800 (Table 6). On CATH1, this checkpoint increases coverage while worsening RMWD from 1.654 to 2.562 Å and transition JSD from 0.434 to 0.611 (Table 2). The authors discuss this tradeoff and focus their contribution on expanding coverage. The comparisons use the complete procedure against pretrained models, without a simpler coverage reward or a control producing comparable displacement. Figures 1 and 3 compare training checkpoints rather than discovery over rollout time.

  2. The physical-validity checks cover only a narrow set of structural errors. The bond criteria check whether consecutive Cα distances are below 4.5 Å and C–N distances below 2.0 Å, without checking lower distance bounds, bond angles, or torsions (Appendix B.2). Clash detection considers only N, Cα, C, O, and Cβ atoms, using a 1 Å cutoff. Evaluation first calculates the fraction of eligible nonlocal atom pairs that do not clash in each frame, then averages across frames (Appendix C.1). A frame can contain a few severe clashes and still receive a score close to 100%. Training uses a residue-level clash gate, which differs from this reported percentage. Section 5.1 uses the near-perfect validity scores to argue that the additional diversity comes from physically admissible frames. That conclusion needs broader structural validation.

Questions for authors
  1. Can the authors compare segment-wise rewards with a single reward for total coverage at the end of the rollout, using the same training and sampling budgets? A separate comparison with the pretrained sampler adjusted to produce similar mean TICA displacement would help test how much larger motions explain the gains. For these comparisons, can the authors report cumulative coverage and new-cell discovery over rollout time in the same fixed evaluation space, especially during later segments? These comparisons could use one existing benchmark.

  2. For the existing generated trajectories, can the authors report how many frames and residues contain clashes, along with bond-length, bond-angle, and torsion statistics? Please also report the fraction of frames failing the residue-level clash criterion used during training. Do the coverage and RMWD improvements persist when the same stricter geometry checks are applied to each model?

3 comments

Q1: Can the authors compare segment-wise rewards with a single reward for total coverage at the end of the rollout, using the same training and sampling budgets? A separate comparison with the pretrained sampler adjusted to produce similar mean TICA displacement would help test how much larger motions explain the gains. For these comparisons, can the authors report cumulative coverage and new-cell discovery over rollout time in the same fixed evaluation space, especially during later segments? These comparisons could use one existing benchmark.

R1: We thank the reviewer for this suggestion. We added two controlled comparisons on the Fast-Folding benchmark.

First, we trained a terminal-reward baseline using the same coverage definition, validity regularization, training setup, and sampling budget as Exploration–NFT. The final cumulative coverage after the complete 1 μs rollout is used as a single trajectory-level reward.

Second, we constructed a motion-amplitude control by increasing the pretrained sampler’s noise scale from 1.75 to 1.98. This value was selected on 16 stratified ATLAS training proteins and matched the mean 100 ns TICA displacement of Exploration–NFT within 1.75% (0.06750 versus 0.06634).

For evaluation, each 1 μs trajectory was divided into ten 100 ns segments. All methods were projected into the same frozen, protein-specific TICA space and evaluated using the same fixed 50×50 grid. Cumulative coverage is the fraction of MD-supported cells visited up to a given segment, while new coverage counts cells first visited in that segment. We only count frames for which the valid Cα–Cα, valid C–N, and residue-level clash-free fractions are each at least 98%. Valid@98% denotes the percentage of frames satisfying this criterion.

MethodValid@98% ↑[email protected]μs ↑Cov@1μs ↑New coverage in seg. 6–10 ↑New cells in seg. 6–10 ↑
Pretrained98.675.8428.1212.27916.00
Pretrained, noise=1.9886.335.8499.9234.07424.83
Terminal reward96.176.62010.2243.60424.17
Segment-wise reward (Ours)96.676.75510.8694.11325.25

The temporal evolution is shown as cumulative coverage / newly discovered coverage, both in percentage points:

SegmentPretrainedPretrained, noise=1.98Terminal rewardSegment-wise reward (Ours)
0.5 μs5.842 / 0.9485.849 / 1.1076.620 / 1.1336.755 / 1.090
0.6 μs6.576 / 0.7346.929 / 1.0807.647 / 1.0287.897 / 1.142
0.7 μs7.175 / 0.6007.805 / 0.8768.238 / 0.5918.805 / 0.908
0.8 μs7.472 / 0.2978.623 / 0.8188.848 / 0.6109.571 / 0.766
0.9 μs7.794 / 0.3229.271 / 0.6489.482 / 0.63410.252 / 0.681
1.0 μs8.121 / 0.3269.923 / 0.65210.224 / 0.74110.869 / 0.617

At the end of the rollout, segment-wise credit assignment improves cumulative coverage from 10.224% to 10.869% relative to the terminal-reward baseline. It also increases discovery during segments 6–10 from 3.604 to 4.113 percentage points, a 14.1% relative improvement, supporting the benefit of segment-wise credit assignment during later rollout stages. The terminal-reward baseline also exhibits a more irregular late-stage discovery profile, whereas segment-wise credit assignment sustains new coverage more smoothly across later segments.

Although the noise=1.98 sampler’s mean displacement on Fast Folding was 18.3% larger than that of Exploration–NFT, its lower valid coverage indicates that increasing conformational motion alone provides limited coverage gains. Overall, segment-wise training achieves the highest valid cumulative coverage and late-segment discovery under the controlled training and sampling budget.

Q2: For the existing generated trajectories, can the authors report how many frames and residues contain clashes, along with bond-length, bond-angle, and torsion statistics? Please also report the fraction of frames failing the residue-level clash criterion used during training. Do the coverage and RMWD improvements persist when the same stricter geometry checks are applied to each model?

R2: We thank the reviewer for this suggestion. We evaluated the pretrained BioKinema and BioKinema–NFT at 800 training steps on the same 12 fast-folding proteins. Each model generated a 100 μs trajectory at 10 ns/frame for each protein. All reported values are unweighted means across proteins.

We first quantified clashes using the same residue-level criterion as training: clashes are detected among N, Cα, C, O, and Cβ atoms with a 1.0 Å cutoff, excluding residue pairs separated by at most two positions along the same chain. A clashing-residue occurrence denotes one residue involved in a clash in one frame.

ModelMean clash-containing framesMean clashing-residue occurrencesClash–fail@98%
Pretrained BioKinema11.83 / 10,000 (0.118%)28.75 / 435,000 (0.0066%)0.118%
BioKinema–NFT97.67 / 10,000 (0.977%)252.33 / 435,000 (0.0580%)0.977%

The average protein contains 43.5 residues; therefore, each 10,000-frame trajectory contains an average of 435,000 residue-frame observations. Clash–fail@98% is the fraction of frames in which fewer than 98% of residues are clash-free.

We further measured backbone bond lengths, bond angles, and Ramachandran statistics:

ModelN–Cα (Å)Cα–C (Å)C–N (Å)C–O (Å)C–N–Cα (°)N–Cα–C (°)Cα–C–N (°)Ramachandran outlier
Pretrained BioKinema1.4541.5201.3211.222124.087112.913117.8741.325%
BioKinema–NFT1.4521.5171.3041.220123.770113.272117.5364.014%

Ramachandran outliers are defined using the general allowed region of the MDAnalysis reference map. The mean bond lengths and angles remain close to those of the pretrained model, with only a modest increase in the Ramachandran-outlier rate.

Finally, we recomputed Coverage and RMWD after retaining only Valid@98% frames. A frame is Valid@98% when its valid Cα–Cα bond fraction, valid C–N bond fraction, and residue-level clash-free fraction are all at least 98%.

ModelCoverage–All ↑Coverage–Valid98 ↑RMWD–All (Å) ↓RMWD–Valid98 (Å) ↓
Pretrained BioKinema0.2730.2706.4556.468
BioKinema–NFT0.5090.5014.7384.775

After applying the same validity filter to both models, BioKinema–NFT still improves Coverage by 85.7% and reduces RMWD by 26.2% relative to the pretrained model. Thus, the ensemble improvements persist after excluding frames that fail the training-aligned structural-validity criterion.

Dear Reviewer HHaH,

We hope our response has addressed your questions and concerns. We would greatly appreciate any further feedback and would be happy to clarify or address any remaining issues.

Thank you once again for the time and effort you have dedicated to reviewing our paper.

Best regards,

Submission59851 Authors

Reviewer_RXkHOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a backbone-agnostic forward-process reinforcement framework for conformational exploration. It needs a well-trained foundational conformation generative model and can enhance the basic model's exploration ability. The experiments suggest the effectiveness of the approach.

Critical strengths
  1. This method sounds technical and interesting. Exploring the conformation is an important and challenging task.
  2. Backbone-agnostic seems to be promising and can be widely used in foundation MD generative models.
Critical weaknesses
  1. The writing is confusing. What is the relationship between the proposed method and DiffusionNFT? And Method Section 3 is not well motivated: why use the segment as the unit instead of an atom or residue, and why use Eq(8) and Eq(10)? More explanation is needed to better understand the approach.

  2. The experiments on ATLAS didn’t report the fidelity of the ensemble, such as RMSF, RMSD, RMWD, TICA, etc. Those metrics are important to evaluate if the model can well-balance coverage and fidelity. Only diversity cannot promise the sampling conformation are similar to the real trajectory or ensemble. From my understanding, the proposed method should prioritize maintaining fidelity first and then improve diversity.

  3. ATLAS and Fast-folding suggest the model achieves the best performance at 800 steps ( BioKinema-NFT), while CATH1 achieves the best performance on BioKinema-NFT (300). The inconsistency between these datasets with different numbers of steps should be explained further.

[1] DiffusionNFT: Online diffusion reinforcement with forward process

Questions for authors

For 800 steps BioKinema-NFT, how long does it take to train a model?

5 comments

Q1-1: What is the relationship between the proposed method and DiffusionNFT?

R1-1: We thank the reviewer for pointing out that the relationship to DiffusionNFT was not sufficiently explicit. Our method is an application of DiffusionNFT to long-horizon protein dynamics generation. DiffusionNFT was originally formulated to optimize image diffusion models against terminal, image-level rewards through forward-process reinforcement. In protein dynamics, the scarcity of long-timescale MD data biases pretrained generators toward short-lag dynamics, often confining long rollouts to local free-energy basins. We address this domain-specific limitation by defining segment-wise rewards for newly visited cells in a group-fitted TICA space and by compensating for the increasing difficulty of conformational discovery at later rollout stages. Thus, we retain the forward-process optimization of DiffusionNFT while introducing a temporally resolved exploration objective tailored to long-horizon protein dynamics.

Q1-2: Why use the segment as the unit instead of an atom or residue?

R1-2: Segments define the temporal resolution of credit assignment, whereas atoms and residues are spatial components of a conformation. In BioKinema–NFT, each segment corresponds to 100 ns of generated dynamics, and its reward measures whether that interval discovers previously unvisited regions in a collective TICA space. Protein conformational transitions arise from coordinated motions and generally cannot be uniquely attributed to individual atoms or residues. Directly rewarding spatial displacement may instead increase diversity through large but physically incoherent coordinate changes.

To test this hypothesis, we trained a controlled RMSD-diversity baseline using the same 100-frame rollout, sampling budget, validity regularization, and optimization setting. The ablation was evaluated on 100 μs trajectories generated for the 12 fast-folding proteins at a 10 ns frame interval. After backbone Kabsch alignment, this baseline rewards the mean pairwise all-heavy-atom RMSD over the generated trajectory. Valid@98% denotes the percentage of generated frames for which the valid Cα–Cα bond fraction, valid C–N bond fraction, and residue-level clash-free fraction are all at least 98%. Although the RMSD reward rapidly increased coordinate-space diversity, structural validity deteriorated correspondingly:

ModelStepPairwise Cα RMSD (Å) ↑Valid@98% ↑
Pretrained02.50699.71
RMSD–diversity2003.81998.86
4005.15896.95
6007.42192.94
8008.08089.89

Thus, an unconstrained spatial-diversity objective increases apparent exploration primarily by amplifying coordinate motion at the expense of structural validity.

We therefore compare our segment-wise model with RMSD–diversity–200, the RMSD checkpoint that still retains at least 98% valid frames. Under this structural-quality constraint, segment-wise credit assignment produces consistently better conformational and dynamical agreement:

MethodValid@98% ↑Coverage ↑RMWD ↓Occ–JSD ↓ACF–MAE ↓Trans–JSD ↓
Pretrained99.71.2706.484.539.621.657
RMSD–diversity98.86.4325.201.495.629.615
Segment–wise98.33.5014.766.491.599.583

Both trained models retain more than 98% valid frames, but the segment-wise model achieves higher coverage and lower errors in ensemble geometry, state occupancy, autocorrelation, and transition dynamics. These results support temporal segments as the appropriate unit of exploration credit: they reward the discovery of collective conformational regions over time rather than coordinate displacement that can increase without preserving physically coherent dynamics.

Q1-3: Why use Eq(8) and Eq(10)?

R1-3: Equations (8) and (10) provide the mechanism that converts segment-level conformational exploration into a forward-process denoising update. We adopt this mechanism from DiffusionNFT, but define its optimality signal specifically for long-horizon protein dynamics.

Consider one 100 ns segment from a group of eight trajectories. If this segment discovers more previously unvisited TICA cells than the other candidates while maintaining structural validity, it receives an above-average reward and hence an optimality score og,k>0.5o_{g,k}>0.5. Conversely, a segment that mainly revisits previously explored cells or contains more invalid frames receives og,k<0.5o_{g,k}<0.5.

Equation (8) constructs two symmetric predictions from a single trainable denoiser:

D+=(1−β)sg⁡(Dθ~)+βDθ,D^{+} = (1-\beta)\operatorname{sg}(D_{\tilde{\theta}}) + \beta D_{\theta},

D−=(1+β)sg⁡(Dθ~)−βDθ.D^{-} = (1+\beta)\operatorname{sg}(D_{\tilde{\theta}}) - \beta D_{\theta}.

For conformations belonging to a productive segment, minimizing the reconstruction loss based on D+D^{+} moves DθD_{\theta} in the ordinary denoising direction, increasing the model’s ability to reproduce the corresponding explored conformational region. For an unproductive segment, D−D^{-} reverses the dependence on DθD_{\theta}; minimizing its reconstruction loss therefore suppresses the denoising behavior associated with that segment. The two predictions allow positive and negative feedback to be implemented with one denoiser, without differentiating through the rollout or estimating reverse-process likelihoods.

Equation (10) selects between these directions according to segment optimality:

Lg,kNFT=cβ[og,kLg,k++(1−og,k)Lg,k−].\mathcal{L}^{\mathrm{NFT}}_{g,k} = \frac{c}{\beta} \left[ o_{g,k}\mathcal{L}^{+}_{g,k} + (1-o_{g,k})\mathcal{L}^{-}_{g,k} \right].

A high-reward segment places greater weight on Lg,k+\mathcal{L}^{+}_{g,k}, reinforcing denoising behavior that generated new conformational regions. A low-reward segment places greater weight on Lg,k−\mathcal{L}^{-}_{g,k}, suppressing behavior associated with redundant exploration or invalid structures. Segments with intermediate optimality receive a correspondingly weaker directional update. The factor c/βc/\beta normalizes the update scale so that it is governed primarily by the standardized segment advantage rather than by the auxiliary mixing coefficient β\beta.

Thus, Equation (8) defines how a protein-dynamics segment can be reinforced or suppressed in the forward denoising objective, while Equation (10) determines which direction and strength are assigned from its measured conformational exploration. This is also why segment-level rewards are useful: different temporal portions of the same rollout can receive different denoising updates depending on whether they discover new conformational regions.

Q2: The experiments on ATLAS didn’t report the fidelity of the ensemble, such as RMSF, RMSD, RMWD, TICA, etc. Those metrics are important to evaluate if the model can well-balance coverage and fidelity. Only diversity cannot promise the sampling conformation are similar to the real trajectory or ensemble. From my understanding, the proposed method should prioritize maintaining fidelity first and then improve diversity.

R2: We thank the reviewer for highlighting the importance of evaluating ensemble fidelity alongside conformational diversity. We therefore added an ATLAS ensemble-fidelity evaluation using three independently generated 100 ns trajectories per protein at 1 ns/frame. We report: (i) the cross-protein Pearson correlation between generated and MD mean pairwise Cα RMSD, (ii) the median per-protein correlation of heavy-atom RMSF, and (iii) the AlphaFlow-style RMWD between generated and MD ensembles. Higher correlations and lower RMWD indicate better fidelity.

To quantify the intrinsic uncertainty associated with the short ATLAS trajectories, we additionally evaluated the three pairs of independent MD replicas: R1–R2, R1–R3, and R2–R3. Their mean and standard deviation provide an empirical reference for finite-trajectory variability.

ModelPairwise RMSD corr. ↑RMSF corr. ↑RMWD ↓ (Å)
Independent MD replicas.753 ± .037.901 ± .0102.177 ± .091
BioKinema.752.8992.217
BioKinema–NFT.715.8772.628
DyneTrion.867.7883.681
DyneTrion–NFT.862.7973.791

BioKinema–NFT largely preserves the ensemble-level correlations of its pretrained backbone. Pairwise RMSD correlation changes from 0.752 to 0.715, a difference of 0.036, which is comparable to the standard deviation of 0.037 observed across independent MD-replica pairs. RMSF correlation changes from 0.899 to 0.877; for reference, independent MD-replica pairs themselves yield an RMSF correlation of 0.901 ± 0.010.

For DyneTrion, Pairwise RMSD correlation remains nearly unchanged, from 0.867 to 0.862, and RMSF correlation slightly improves from 0.788 to 0.797. These results indicate that NFT largely preserves the protein-wise ordering of conformational amplitudes and the spatial patterns of atomic fluctuations.

RMWD increases moderately, from 2.217 to 2.628 Å for BioKinema and from 3.681 to 3.791 Å for DyneTrion. However, the ATLAS trajectories contain only 100 ns per replica and therefore provide a noisy estimate of the underlying equilibrium ensemble. Even two independent ground-truth MD replicas exhibit an average RMWD of 2.177 Å. The additional RMWD introduced by NFT is 0.411 Å for BioKinema and 0.111 Å for DyneTrion, corresponding to 18.9% and 5.1% of this intrinsic MD-to-MD discrepancy, respectively. We therefore interpret these changes as a modest finite-horizon fidelity cost within the practical uncertainty of estimating an ensemble from 100 ns trajectories, rather than evidence of ensemble collapse.

Together with the reported improvements in conformational coverage and diversity, these results show that Exploration–NFT expands the sampled conformational space while largely preserving short-horizon ensemble characteristics. We will include these fidelity metrics and clarify the finite-horizon limitation of the ATLAS evaluation in the revised manuscript.

Reviewer_hHfwOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The manuscript introduces Exploration-NFT, a reinforcement post-training framework for improving sustained conformational exploration in protein dynamics generators. It rewards segment-wise discovery of new grid cells in a shared, group-fitted, non-whitened TICA space, while controlling structural validity through a separate penalty. DiffusionNFT converts these rewards into forward-process denoising updates without modifying the backbone’s native sampler or requiring sampling likelihoods. Validation uses BioKinema and DyneTrion on ATLAS, CATH1, and fast-folding benchmarks, assessing structural diversity, reference-MD coverage, local validity, ensemble agreement, and kinetics. Additional experiments examine transfer across protein lengths and temporal intervals. The reported results show increased diversity or coverage across both backbones while maintaining high local-validity scores, with dataset- and checkpoint-dependent effects on thermodynamic and kinetic agreement.

Critical strengths

The framework offers a practical way to post-train heterogeneous protein dynamics generators. Exploration-NFT connects temporally resolved exploration rewards with forward-process denoising updates while preserving each backbone’s native sampler. Reward computation requires neither reference MD trajectories nor sampling-likelihood evaluation. Its implementation with two architecturally distinct backbones demonstrates useful adaptability and addresses an important obstacle to reinforcement post-training in this domain.

Critical weaknesses
  1. The paper does not isolate the effects of segment-wise novelty rewards, non-whitened TICA coordinates, or the separate validity penalty. Consequently, the results demonstrate that post-training increases coverage, but do not establish which proposed components enable persistent exploration. This distinction is consequential because Table 6 also shows substantially increased motion.

  2. Tables 1-3 primarily compare pretrained backbones with their NFT variants, without a matched comparison against the cited Learning Implicit Bias approach or simpler exploration controls. Although updating the generator differs from inference-time steering, the experiments do not establish when this approach offers a practical advantage under comparable sampling and computational budgets.

Questions for authors
  1. Could you add a controlled baseline that assigns final cumulative coverage as a trajectory-level reward, using the same coverage definition, validity regularization, and sampling budget, to isolate the benefit of segment-wise credit assignment?

  2. Can you quantify the benefit of preserving TICA scale through an ablation comparing non-whitened and whitened coordinates, with grid resolution appropriately controlled?

  3. How do coverage and structural validity change when the separate invalid-frame penalty is removed or replaced with validity-gated exploration rewards?

  4. Could you compare Exploration-NFT with Learning Implicit Bias on a common compatible backbone, using identical test proteins and rollout horizons and comparable inference compute?

6 comments

Q1: Could you add a controlled baseline that assigns final cumulative coverage as a trajectory-level reward, using the same coverage definition, validity regularization, and sampling budget, to isolate the benefit of segment-wise credit assignment?

R1: We thank the reviewer for this suggestion. We trained a terminal-reward baseline using the same coverage definition, validity regularization, training setup, and sampling budget as Exploration–NFT. The final cumulative coverage after the complete 1 μs rollout is used as a single trajectory-level reward.

For evaluation, each 1 μs Fast-Folding trajectory was divided into ten 100 ns segments. All methods were evaluated in the same frozen, protein-specific TICA space and fixed 50×50 grid. Cumulative coverage is the fraction of MD-supported cells visited up to a given segment, while new coverage counts cells first visited in that segment. We only count frames whose valid Cα–Cα, valid C–N, and residue-level clash-free fractions are each at least 98%.

MethodValid@98% ↑[email protected] μs ↑Cov@1 μs ↑New coverage in seg. 6–10 ↑New cells in seg. 6–10 ↑
Pretrained98.675.8428.1212.27916.00
Terminal reward96.176.62010.2243.60424.17
Segment-wise reward (Ours)96.676.75510.8694.11325.25

The temporal evolution is reported as cumulative coverage / newly discovered coverage, both in percentage points:

SegmentPretrainedTerminal rewardSegment-wise reward (Ours)
0.5 μs5.842 / 0.9486.620 / 1.1336.755 / 1.090
0.6 μs6.576 / 0.7347.647 / 1.0287.897 / 1.142
0.7 μs7.175 / 0.6008.238 / 0.5918.805 / 0.908
0.8 μs7.472 / 0.2978.848 / 0.6109.571 / 0.766
0.9 μs7.794 / 0.3229.482 / 0.63410.252 / 0.681
1.0 μs8.121 / 0.32610.224 / 0.74110.869 / 0.617

Segment-wise credit assignment improves cumulative coverage from 10.224% to 10.869% and increases new coverage during segments 6–10 from 3.604 to 4.113 percentage points, a 14.1% relative improvement. The terminal-reward baseline exhibits a more irregular late-stage discovery profile, whereas segment-wise credit assignment sustains new coverage more smoothly.

We additionally compared the models over complete 100 μs trajectories. The Valid@98% of the terminal-reward model decreases from 99.713% at initialization to 99.357%, 99.013%, 98.389%, and 98.004% at steps 200, 400, 600, and 800, respectively. We therefore selected Terminal–600 as the checkpoint most closely matched to Segment-wise–800 in structural validity: 98.389% versus 98.327%. All metrics below use only 98%-compliant frames.

MethodValid@98% ↑Coverage ↑RMWD ↓Occ–JSD ↓ACF–MAE ↓Trans–JSD ↓
Pretrained99.7130.2706.4840.5390.6210.657
Terminal reward98.3890.4804.9260.4900.6170.598
Segment-wise reward (Ours)98.3270.5014.7660.4910.5990.583

At matched structural validity, segment-wise training achieves higher coverage, lower RMWD, and better autocorrelation and transition statistics than the terminal-reward baseline, while occupancy error remains comparable. Both trained models substantially improve over the pretrained sampler.

Q2: Can you quantify the benefit of preserving TICA scale through an ablation comparing non-whitened and whitened coordinates, with grid resolution appropriately controlled?

R2: We thank the reviewer for this suggestion. We added a controlled ablation comparing our physical-scale TICA coordinates with covariance-whitened TICA coordinates. All settings—including the training proteins, rollout budget, reward definition, optimizer, and random seed—were held fixed. The ablation was evaluated on 100 μs trajectories generated for the 12 fast-folding proteins at a 10 ns frame interval.

Because the two coordinate systems have different numerical scales, using the same grid spacing would not constitute a controlled comparison. We therefore calibrated the grid spacing on the same 16 stratified training proteins using pretrained step-0 rollouts. A spacing of 0.03 for physical-scale coordinates and 1.0 for whitened coordinates produced closely matched mean terminal cumulative coverage before training: 0.170 and 0.176, respectively. Thus, the comparison does not arise from a trivially finer grid in either coordinate system.

ComparisonTICAStepValid@98% ↑Coverage ↑RMWD ↓Occ–JSD ↓ACF–MAE ↓Trans–JSD ↓Disp–Log–MAE ↓
Validity-matchedOurs80098.33.5014.766.491.599.583.408
Whitened40098.47.4874.751.495.613.593.397
Equal budgetOurs80098.33.5014.766.491.599.583.408
Whitened80096.27.6333.579.470.604.544.622

At matched structural validity, physical-scale TICA yields higher coverage and lower occupancy, autocorrelation, and transition errors, while RMWD and displacement error remain similar. At the same 800-step training budget, whitening increases raw coverage more aggressively, but Valid@98% decreases by 2.06 percentage points and Disp–Log–MAE increases from 0.408 to 0.622, indicating excessive acceleration of the generated dynamics.

This behavior is consistent with the role of whitening. Whitened coordinates normalize the covariance scale of each TICA direction, so excursions along intrinsically low-amplitude directions receive the same geometric weight as motion along physically larger collective modes. This produces denser exploration rewards but can over-amplify motion. Our non-whitened construction instead uses a Euclidean-orthonormal projection in the original Cα-distance feature space, preserving the relative physical scale of the learned slow subspace. The ablation therefore supports preserving TICA scale as a means of achieving a better coverage–validity–dynamical-fidelity balance, rather than maximizing coverage alone.

Q3: How do coverage and structural validity change when the separate invalid-frame penalty is removed or replaced with validity-gated exploration rewards?

R3: We evaluated the three reward formulations on the 12 fast-folding proteins. For each protein, we generated a 100 μs trajectory at a 10 ns frame interval and applied the same evaluation protocol. Evaluation metrics are computed using only 98%-compliant frames. Valid@98% is the percentage of generated frames for which the valid Cα–Cα bond fraction, valid C–N bond fraction, and residue-level clash-free fraction are all at least 98%.

The three formulations are:

  • Separate invalid-frame penalty (ours):

    rg,k=ΔCg,k−0.5Pg,kinvalid.r_{g,k} = \Delta C_{g,k} - 0.5 P^{\mathrm{invalid}}_{g,k}.

  • Penalty removed:

    rg,k=ΔCg,k.r_{g,k} = \Delta C_{g,k}.

  • Validity-gated exploration:

    rg,k=ΔCg,kvalid,r_{g,k} = \Delta C^{\mathrm{valid}}_{g,k},

    where only 98%-compliant frames contribute to newly discovered TICA cells, with no separate invalid-frame penalty.

Reward formulationMetricPretrainedStep 200Step 400Step 600Step 800Step 1000
Penalty removedCoverage ↑.270.334.413.483.559.655
Valid@98% ↑99.7199.4999.0098.4096.8594.29
Validity-gated explorationCoverage ↑.270.436.603.683.713.714
Valid@98% ↑99.7198.8796.4493.3888.1485.98
OursCoverage ↑.270.304.368.454.501.502
Valid@98% ↑99.7199.6499.3698.8598.3398.35

At a fixed training budget, removing the penalty or replacing it with validity gating increases Coverage more rapidly, but this increase is accompanied by substantially faster degradation in structural validity. At step 800, Coverage increases from 0.501 with the separate penalty to 0.559 without the penalty and 0.713 with validity gating, while Valid@98% decreases from 98.33% to 96.85% and 88.14%, respectively. The divergence becomes larger by step 1000.

To separate genuine conformational improvement from gains associated with reduced structural validity, we additionally compare checkpoints with approximately matched and practically acceptable Valid@98%. We use the reported 800-step model for our method and the closest structurally valid checkpoints for the two ablations.

MethodStepValid@98% ↑Coverage ↑RMWD ↓Occ–JSD ↓ACF–MAE ↓Trans–JSD ↓
Pretrained099.71.2706.484.539.621.657
Penalty removed60098.40.4834.806.491.604.588
Validity-gated exploration20098.87.4365.256.495.628.610
Ours80098.33.5014.766.491.599.583

When structural validity is controlled, the separate invalid-frame penalty achieves the highest Coverage and the lowest RMWD, ACF–MAE, and Trans–JSD, while matching the best occupancy agreement. Removing the penalty produces slightly lower Coverage and slightly weaker ensemble and kinetic agreement. Validity gating produces the lowest Coverage and the weakest RMWD, occupancy, autocorrelation, and transition results among the trained variants at comparable validity.

These results show that validity gating and an explicit invalid-frame penalty are not equivalent. Gating restricts which frames can earn exploration credit, but it does not provide a negative learning signal proportional to the amount of invalid structure generated. Consequently, it favors increasingly aggressive exploration as training proceeds. The separate invalid-frame penalty provides the best overall coverage–validity–fidelity trade-off and is therefore retained in the final method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.