acceptodds
Under review as a conference paper at ICLR 2027

Restrained Diffusion Bridges: Targeted Denoiser Training and Refinement

Abstract

Diffusion models are often inaccurate in critical, data-sparse regions of the target distribution. Sampling data from restrained distributions, biased distributions used in classical enhanced sampling to concentrate sampling near chosen configurations, allows control over the location of training data, but alters the target distribution. We establish a remarkable cancellation that occurs when the forward noising process from restrained data is conditioned on its value at an appropriately chosen intermediate time. The result allows a pretrained model fine-tuned on restrained data to achieve high accuracy in any chosen region of the target distribution, regardless of how probable that region is, using standard denoising or flow matching objectives and without importance weights. When no unrestrained target data are available, Gibbs sampling allows inference from a model trained exclusively on restrained data.

Primary area:
generative models
Keywords:
Diffusion models, Diffusion bridges, Fine-tuning, Rare events, Enhanced sampling, Molecular sampling, Gibbs sampling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MfvYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes targeted denoiser training using samples from Gaussian-restrained target distributions. Matching the restraint width to a diffusion time and constructing posterior-initialized bridges preserves the ordinary unconditional score or velocity regression target while changing training coverage. The method supports one-shot refinement by combining a pretrained high-noise model with a refined low-noise model, and Gibbs sampling when only the low-noise model is available. Experiments cover the Muller-Brown potential, explicit-solvent alanine dipeptide, and latent-space CLN025.

Critical strengths
  1. The posterior-matched bridge is a useful construction for concentrating training on underrepresented regions while preserving the population denoising target. The conditional-regression argument in Theorem 1 and Appendix A is convincing under exact posterior sampling. The linear-CV construction and application to implicit molecular marginals are particularly useful when an explicit marginal energy is unavailable. The contribution lies in turning these probability identities into a practical training procedure.

  2. The one-shot refinement experiments provide encouraging evidence at matched training-update budgets and without additional MD. Chignolin shares the representation and branch checkpoint, preserves the optimizer schedule, and uses the same sampling grid. Across four data budgets and both bridge times, the reported rare-state population error decreases. At the largest budget, t*=0.35 reduces relative error from 14.2% to 4.1%, while JS distance increases from 0.10875 to 0.11016 (Table 4). The consistent local improvement, together with the clear construction, makes me lean toward acceptance despite the attribution concerns below.

Critical weaknesses
  1. The refinement comparison does not isolate the contribution of targeted bridge training. Refinement changes both the sampling distribution and the allocation of updates across noise levels. A continuation of ordinary denoising training restricted to [0,t*], from the same checkpoint and with the same stitched sampler, is missing. Averaging the matched bridge over its natural endpoint marginal recovers ordinary denoising training, making this an especially direct control. Without it, the advantage over simply prioritizing low-noise training remains unclear. Both rare-state and global metrics matter here, since the refined Chignolin models have higher global JS distances.

  2. The from-scratch comparisons also change inference. In Muller-Brown, the baseline uses 400,000 independent 100-step reverse trajectories, while the proposed method uses 1,000 chains, 5,000 sweeps and 100 reverse steps per sweep to retain 401,000 correlated configurations. Counting each configuration separately, this is 12.5 times the total denoiser evaluations. The matched MD budget controls acquisition cost, but the contribution of the sampling protocol and its effective inference cost remain unresolved. The existing baseline could also use the same Gibbs sampler. Moreover, Algorithm 2's exactness is an invariance statement; convergence after the chosen burn-in needs empirical support.

Questions for authors
  1. For one representative refinement setting, could you compare ordinary denoising continuation over [0,t*] from the same checkpoint, matching optimizer handling, coordinate preprocessing, update budget and stitched sampling? Please report the existing rare-state and global metrics.

  2. On one representative Gibbs benchmark, how does the existing baseline checkpoint perform under the same Gibbs protocol? Please provide occupancy versus sweeps, burn-in sensitivity, and effective sample size together with sampling cost. This comparison can use the existing trained models.

  3. For Chignolin, what are the rare-state frequencies in each training prefix, and how stable is the reference frequency across trajectory segments? The reported model-seed intervals do not capture uncertainty in the shared MD reference. Statistics from the existing trajectories would help interpret the reported probability accuracy.

2 comments
AuthorsOpenReview ↗

Response to Reviewer MfvY (1/2)

We thank the reviewer for the suggestions to disentangle targeted bridge training, low-time update allocation, and Gibbs inference, and to report reference uncertainty. All newly completed matched experiments are detailed in Reply to All Reviewers, A1–A2. We summarize their implications here and provide the requested Chignolin prefix statistics.

Q1. Matched ordinary low-time continuation

The refinement comparison does not isolate the contribution of targeted bridge training. Refinement changes both the sampling distribution and the allocation of updates across noise levels. A continuation of ordinary denoising training restricted to [0,t*], from the same checkpoint and with the same stitched sampler, is missing. Averaging the matched bridge over its natural endpoint marginal recovers ordinary denoising training, making this an especially direct control. Without it, the advantage over simply prioritizing low-noise training remains unclear. Both rare-state and global metrics matter here, since the refined Chignolin models have higher global JS distances.

For one representative refinement setting, could you compare ordinary denoising continuation over [0,t*] from the same checkpoint, matching optimizer handling, coordinate preprocessing, update budget and stitched sampling? Please report the existing rare-state and global metrics.

We agree that ordinary denoising continuation over [0,t⋆][0,t_\star] is the most direct control. We implemented this control in Setting I for ALDP and Chignolin, matching checkpoints, additional updates, optimizer initialization, preprocessing, time interval, and sampling (A2.1; Appendices F.6 and G.7, Tables 4 and 9). We also include low-time continuation in the new image experiment, Notch (A3; Appendix H, Table 10). Across all three systems, the results demonstrate that RDB provides benefits beyond simply reallocating training to low diffusion times.

AuthorsOpenReview ↗

Response to Reviewer MfvY (2/2)

Q2. Matched inference, Gibbs behavior, and sampling cost

The from-scratch comparisons also change inference. In Muller-Brown, the baseline uses 400,000 independent 100-step reverse trajectories, while the proposed method uses 1,000 chains, 5,000 sweeps and 100 reverse steps per sweep to retain 401,000 correlated configurations. Counting each configuration separately, this is 12.5 times the total denoiser evaluations. The matched MD budget controls acquisition cost, but the contribution of the sampling protocol and its effective inference cost remain unresolved. The existing baseline could also use the same Gibbs sampler. Moreover, Algorithm 2's exactness is an invariance statement; convergence after the chosen burn-in needs empirical support.

On one representative Gibbs benchmark, how does the existing baseline checkpoint perform under the same Gibbs protocol? Please provide occupancy versus sweeps, burn-in sensitivity, and effective sample size together with sampling cost. This comparison can use the existing trained models.

A2.2 now includes matched inference controls for both Müller–Brown and ALDP, with the population comparisons added to Appendix E.1 (Table 1) and Appendix F.6 (Table 5), respectively. In Müller–Brown, the baseline and RDB use the same Gibbs protocol, and direct baseline sampling is also matched at 5×1095\times10^9 denoiser evaluations per run. For ALDP, Appendix F.6 (Figure 7) reports occupancy versus sweeps under the matched Gibbs sampler, including the standard inference initialization control; the burn-in table in A2.2 compares sweeps 21–50, 31–50 and 41–50 across 20 training seeds.

To measure sampling efficiency, we additionally record 10,000 chains × 50 sweeps for each frozen seed-42 ALDP checkpoint. For the R5 occupancy averaged over sweeps 21–50, the variance-equivalent ESS is 48,385 for Restrained-MD-pair RDB and 55,048 for ordinary low-time continuation, from 300,000 retained states each. Both runs cost 50.49 million denoiser evaluations, including initialization and burn-in, and approximately 75.7 s per model on one RTX PRO 6000 Blackwell GPU. The ESS definition, Monte Carlo standard errors, and ESS per million evaluations are reported in A2.2.

Q3. Chignolin training prefixes and trajectory-segment variability

For Chignolin, what are the rare-state frequencies in each training prefix, and how stable is the reference frequency across trajectory segments? The reported model-seed intervals do not capture uncertainty in the shared MD reference. Statistics from the existing trajectories would help interpret the reported probability accuracy.

We computed misfold populations from the original Chignolin trajectory using the same state definition and excluding the initial 2 µs:

Total trajectory budgetTraining windowObserved misfold population
3 µs(2, 3] µs0.4720%
4 µs(2, 4] µs0.3930%
5 µs(2, 5] µs0.3473%
6 µs(2, 6] µs0.3060%
7 µs(2, 7] µs0.2938%
12 µs(2, 12] µs0.3607%
17 µs(2, 17] µs0.3707%
22 µs(2, 22] µs0.3747%

In addition, A1 reports an entirely independent Chignolin evaluation reference. In the November 16 snapshot, three independently seeded trajectories provide 42.58 µs after burn-in (all-frame population 0.35968%). Using 20 complete 2 µs blocks (40 µs) yields 0.35975%, with approximate 95% block-tt interval [0.33611%, 0.38339%].

Figure 4(c) now marks the training population separately for each MD budget, alongside the independent reference and its 95% CI band. The independent-reference protocol is documented in Appendix G.2.

Reviewer_5k2zOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper shows that a harmonically restrained (umbrella-sampling) density equals the diffusion posterior p(x0∣xt∗)p(x_0 \mid x_{t^*}) when the restraint width matches the noise level at t∗t^*. As a result, forward-noising restrained samples along a bridge pinned at xt∗x_{t^*} cancels the restraint bias exactly. A standard denoising or flow-matching loss then recovers the unconditional score on [0,t∗][0, t^*] without importance weights. The authors use this to fine-tune a pretrained model near chosen rare regions, or, when no pretrained model exists, to sample via a learned Gibbs chain. At matched MD budgets, experiments on Müller–Brown, explicit-solvent alanine dipeptide, and latent-space Chignolin show rare-state populations moving closer to the reference with little change in global accuracy.

Critical strengths
  1. An exact, minimal-change mechanism for targeted training without loss weights.
  • The cancellation is exact and simple to verify. It needs no change to the objective, architecture, or sampler, and it holds for any distribution of probe centers. This lets practitioners put training effort wherever they choose without the high-variance importance weights that usually defeat reweighting in rare regions. It also connects diffusion training directly to existing umbrella-sampling infrastructure. This is the paper's main contribution, and it is what makes the work likely to be reused: the idea is easy to adopt and the guarantee does not depend on tuning.
  1. Consistent rare-state gains under matched cost.
  • In all three systems the method moves rare-state populations toward the reference, at matched MD budget and matched training updates. In ALDP Setting II and Chignolin, the gains also grow with MD budget while the baseline appears to plateau. This consistency is what lifts the theoretical observation into a plausible practical method.
Critical weaknesses
  1. Rare-state accuracy is evaluated against the training data, with no reference uncertainty. The ALDP reference is built from the same trajectories used for base training and Setting I (App. F.1). The Chignolin reference at 22 µs is exactly the training window (App. G.2). At the full budget, simply counting training frames would therefore match the reference, so results such as 14.2% → 2.2% measure how well the model fits its training data, not the equilibrium population. The headline empirical claim currently lacks an independent test.

  2. The no-bias guarantee depends on unverified restrained-sampling convergence, and results are highly sensitive to t∗t^*. In Fig. 5 (Müller–Brown), raising t∗t^* from 0.02 to 0.1 inflates the central-basin occupancy roughly 30× at β = 2, which the authors attribute to unequilibrated restrained trajectories. The claimed insensitivity to t∗t^* rests on two nearby values in Chignolin, where no restrained sampling is involved. In real rare-region applications without ground truth, this failure would be large and undetectable.

Questions for authors
  1. Preference fine-tuning (DPO / Diffusion-DPO) targets a reward-tilted density πrefexp⁡(r/β)\pi_\text{ref}\exp(r/\beta), and the restrained density is such a tilt with a quadratic reward. DPO moves the model toward the tilt, whereas your bridge cancels it. How do you view this relationship? By Proposition 1, for which rewards could tilted data refine a base model without shifting its distribution?
  2. Autoguidance [Karras et al., 2024] (https://arxiv.org/abs/2406.02507) also targets errors where a model is under-trained, by extrapolating away from a weaker version of itself, but it intentionally departs from the data distribution. How do you position your method relative to it? Could the gap between the base and refined denoisers serve as a guidance signal or as an error indicator for adaptive probe placement?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 5k2z (1/2)

We thank the reviewer for emphasizing independent evaluation, the assumptions behind conditional sampling, and the connections to preference optimization and autoguidance. The new shared evidence is collected in Reply to All Reviewers, A1–A5; we address the conceptual and methodological points below.

Q1. Independent evaluation and reference uncertainty

Rare-state accuracy is evaluated against the training data, with no reference uncertainty. The ALDP reference is built from the same trajectories used for base training and Setting I (App. F.1). The Chignolin reference at 22 µs is exactly the training window (App. G.2). At the full budget, simply counting training frames would therefore match the reference, so results such as 14.2% → 2.2% measure how well the model fits its training data, not the equilibrium population. The headline empirical claim currently lacks an independent test.

To evaluate against independent data, we generated additional trajectories entirely excluded from model training and synthetic-posterior construction. A1 reports 50 µs ALDP and 40 µs Chignolin production references, analyzed using nonoverlapping 2 µs blocks and approximate Student-tt intervals. The resulting population estimates are 0.18024% for ALDP R5 (95% interval [0.15897%, 0.20151%]) and 0.35975% for Chignolin misfold (95% interval [0.33611%, 0.38339%]). Chignolin results are also consistent with 3 µs and 5 µs block analyses.

These references are now used for rare-state evaluation in Sections 6.2–6.3 and Figures 3–4, with their construction and uncertainty estimates documented in Appendices F.1 and G.2. The revised Chignolin errors are also reported in Appendix G.6 (Table 8).

Q2. Restrained-sampling convergence and t⋆t_\star sensitivity

The no-bias guarantee depends on unverified restrained-sampling convergence, and results are highly sensitive to t star. In Fig. 5 (Müller–Brown), raising tstar from 0.02 to 0.1 inflates the central-basin occupancy roughly 30× at β = 2, which the authors attribute to unequilibrated restrained trajectories. The claimed insensitivity to tstar rests on two nearby values in Chignolin, where no restrained sampling is involved. In real rare-region applications without ground truth, this failure would be large and undetectable.

The reviewer is correct that matched bridges preserve the intended target only when the conditional samples are drawn from the required restrained distribution. We do not infer convergence of finite-length restrained MD merely from the form of the bridge. In particular, the strong Müller–Brown occupancy shift between t⋆=0.02t_\star=0.02 and 0.10.1 is a warning that an inappropriate bridge width or inadequate conditional equilibration can lead to large errors; the invariance result does not rule out this practical failure. A4 now reports matched downstream comparisons alongside richness and retention.

We evaluate the 2 ns ALDP warmup in A5 by comparing real and synthetic conditional statistics in 0.5 ns windows and averaging the per-probe discrepancies over all 5,000 probes. After 2 ns, circular-moment MSE and restraint-statistic error remain stable, supporting the warmup used for training-pair construction. The protocol and results have been added to Appendix F.7 (Table 6).

AuthorsOpenReview ↗

Response to Reviewer 5k2z (2/2)

Q3. Preference optimization and admissible rewards

Preference fine-tuning (DPO / Diffusion-DPO) targets a reward-tilted density, and the restrained density is such a tilt with a quadratic reward. DPO moves the model toward the tilt, whereas your bridge cancels it. How do you view this relationship? By Proposition 1, for which rewards could tilted data refine a base model without shifting its distribution?

Both DPO-style preference optimization and RDB involve a tilted density, but they optimize different objectives. Preference optimization aims to move the learned distribution toward the reward-tilted target. RDB uses a tilt to reallocate training effort while retaining the original population denoising target.

The distinction is the matched bridge: when the endpoint tilt and bridge are paired as required, the training conditional obeys q(x0∣xt)=π(x0∣xt)q(x_0\mid x_t)=\pi(x_0\mid x_t). Thus, training samples may emphasize selected regions without changing the optimal original-population denoiser.

Proposition 1 gives a sufficient condition: the tilt must be representable by a nonnegative endpoint likelihood under the forward process, together with its corresponding matched bridge. Our quadratic restraints and positive mixtures of Gaussian endpoint likelihoods fall in this class. We do not claim that every possible reward admits this construction or that every such bridge can be sampled efficiently.

Q4. Autoguidance and adaptive probe placement

Autoguidance [Karras et al., 2024] also targets errors where a model is under-trained, by extrapolating away from a weaker version of itself, but it intentionally departs from the data distribution. How do you position your method relative to it? Could the gap between the base and refined denoisers serve as a guidance signal or as an error indicator for adaptive probe placement?

Autoguidance and RDB both address nonuniform model accuracy, but Autoguidance acts at sampling time, extrapolating the denoising field away from a weaker model, whereas RDB changes the trained denoiser using matched conditional data while preserving the original population denoising target. RDB requires no inference-time extrapolation for the reported improvements.

The pretrained–refined score difference could in principle be used for guidance, e.g., sref+γ(sref−spretrained)s_{\mathrm{ref}}+\gamma(s_{\mathrm{ref}}-s_{\mathrm{pretrained}}), but that extrapolation does not inherit our target-preservation guarantee. Alternatively, score disagreement could help prioritize future probes. Large disagreement may signal regions requiring attention, but it is not a calibrated error estimate and may also reflect benign changes. An adaptive probe procedure would still require appropriately matched conditional samples and bridges. We have not evaluated either extension and identify them as prospective directions rather than established results.

Reviewer_3XAcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper targets for improving diffusion models in low-probability regions. The key observation is that a Gaussian-restrained target distribution coincides exactly with the diffusion posterior at a suitably matched noise level. The authors use this identity to establish that standard denoising score matching or flow-matching objectives recover the unconditional target score at the population optimum, despite sampling training data from restrained distributions. The method is evaluated on the Müller–Brown potential, explicit-solvent alanine dipeptide, and implicit-solvent Chignolin. The reported results show improvements in the estimated populations of selected rare states.

Critical strengths
  1. The identification between the restrained density and a matched diffusion posterior provides an intuitive explanation for why targeted training can preserve the correct population-level denoising objective.
  2. The method does not require additional importance weights or a fundamentally new loss function, making it relatively easy to integrate into existing diffusion or flow-matching pipelines.
  3. The paper considers both fine-tuning a pretrained model and training a restricted-time model for subsequent Gibbs sampling, broadening the potential applicability of the framework.
Critical weaknesses
  1. The principal empirical comparisons are against standard diffusion baselines and variants of the proposed method. These comparisons demonstrate improvements over the chosen baselines, but do not establish the method's competitiveness against established enhanced-sampling methods or closely related diffusion-assisted samplers. For example, comparisons with a suitable umbrella-sampling baseline would be valuable.
  2. The theoretical result permits targeting regions with arbitrarily low probability, but practical performance still depends on choosing suitable probe points and obtaining representative restrained or synthetic posterior samples.
  3. The choice of bridge time remains largely heuristic. Although sensitivity results are reported, more actionable guidance would improve the method's usability.
  4. The experiments are primarily conducted on low-dimensional toy distributions and molecular systems, with no evaluation on image-generation benchmarks. It is therefore unclear whether the proposed approach can scale effectively to high-dimensional image data or improve targeted sampling in more complex generative modeling tasks. Evaluating the method on standard image-generation benchmarks would strengthen the empirical evidence for its scalability and broader applicability.
Questions for authors

Please check the issues in critical weaknesses section.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 3XAc (1/2)

We thank the reviewer for the constructive comments. Our Reply to All Reviewers presents the shared new evidence: independent MD references (A1), low-time and Gibbs controls (A2), the Notch image experiment (A3), bridge-time selection diagnostics (A4), and aggregate ALDP warmup diagnostics (A5). Below, we address the reviewer-specific implications.

Q1. Comparisons with enhanced-sampling and diffusion-assisted samplers

The principal empirical comparisons are against standard diffusion baselines and variants of the proposed method. These comparisons demonstrate improvements over the chosen baselines, but do not establish the method's competitiveness against established enhanced-sampling methods or closely related diffusion-assisted samplers. For example, comparisons with a suitable umbrella-sampling baseline would be valuable.

Our central contribution is the restrained diffusion bridge (RDB) loss, used in two settings with different objectives. In Setting I, the available dataset defines the population denoising target. The baseline model underrepresents rare configurations, and RDB aims to improve the accuracy. Ordinary diffusion training and matched continuation are therefore the most direct controls for testing whether RDB improves underlearned regions while preserving global model quality. In Setting II, RDB is part of an energy-based data-acquisition and sampling pipeline. We agree that umbrella sampling, replica exchange, and related diffusion-assisted methods are relevant baselines for evaluating the complete pipeline.

To investigate the connection to classical reweighting, we performed an exploratory MBAR analysis on the same ALDP restrained-MD data used to train Setting II RDB. The restrained windows exhibited very weak overlap, leading to severely concentrated estimated MBAR reweighting weight: the weight-based ESS was approximately 4.7 out of 150,000 configurations, with a single configuration contributing approximately 41% of the total weight. The MBAR fit did not fully converge, and the rare-state estimate was sensitive to normalization across weakly connected groups. Consequently, MBAR did not provide a reliable rare-state population estimate for these data. In contrast, RDB learns from local conditional samples through the matched bridge without explicitly estimating relative normalization constants across restraint windows. Using the same restrained data, the Restrained-MD-pair RDB pipeline yielded a median ALDP R5 population of 0.13450%.

We have added to Section 7 (Conclusions) that systematic comparisons with established enhanced-sampling and diffusion-assisted samplers, including the cost of collecting restrained data, remain future work.

However, Our long-term goal is to obtain accurate generative models of equilibrium distributions across molecular systems. Microsoft Research’s BioEmu2 illustrates this direction: it is trained on diverse biomolecular data and generates conformational ensembles conditioned on the molecular system. Classical sampling typically seeks to improve sampling for a specified system.

We have begun exploring this direction by applying RDB to BioEmu2 on Chignolin. In these preliminary experiments, the pretrained model’s sampled distribution differs substantially from our equilibrium reference, and early results suggest that RDB fine-tuning can substantially reduce these discrepancies. These observations motivate further investigation of RDB for refining large pretrained molecular generative models. We plan to pursue this direction in future work.

AuthorsOpenReview ↗

Response to Reviewer 3XAc (2/2)

Q2. Probe selection and representativeness of conditional samples

The theoretical result permits targeting regions with arbitrarily low probability, but practical performance still depends on choosing suitable probe points and obtaining representative restrained or synthetic posterior samples.

We agree that the guarantee is conditional on appropriate probe coverage and representative posterior samples.

In Setting I, probe anchors are chosen from configurations already represented in the dataset. Refinement cannot recover configurations entirely absent from it. Probe selection may use known rare attributes (as in Notch, A3) or data-driven allocation. For molecular systems, we sample anchors in proportion to the square root of the occupancy of their collective-variable bin, increasing the relative allocation to underrepresented bins. Given a dataset, the empirical posterior weights are directly computable, so resampling from these weights requires no new MD trajectories or energy evaluations. Dataset coverage and posterior concentration remain practical limitations.

In Setting II, we obtain probe candidates from high-temperature trajectories and collect conditional samples through restrained target-temperature MD. A5 compares real and synthetic conditional statistics over all 5,000 ALDP probes in equal-length time windows. The stable aggregate discrepancies after 2 ns support the warmup discard used to construct the training pairs. These diagnostics are now included in Appendix F.7 (Table 6).

We have added an explicit acknowledgment of these limitations to Section 7 (Conclusions), covering probe selection, dataset coverage in Setting I, and adequate sampling in restrained simulations in Setting II.

Q3. Actionable guidance for selecting t⋆t_\star

The choice of bridge time remains largely heuristic. Although sensitivity results are reported, more actionable guidance would improve the method's usability.

We propose a t∗t^* selection method based on restrained posterior richness and retension. details see A4.

Q4. Image evaluation and broader applicability

The experiments are primarily conducted on low-dimensional toy distributions and molecular systems, with no evaluation on image-generation benchmarks. It is therefore unclear whether the proposed approach can scale effectively to high-dimensional image data or improve targeted sampling in more complex generative modeling tasks. Evaluating the method on standard image-generation benchmarks would strengthen the empirical evidence for its scalability and broader applicability.

We have completed the Notch controlled image experiment (A3). It is a continuous Gaussian mixture over 972 shape templates, with a rare gap occurring in 2% of samples. A 1.62M-parameter unconditional U-Net is trained with matched continuation/refinement budgets. The known analytical target score permits both local and global score-error evaluation.

At t⋆=0.18t_\star=0.18, RDB produces 1.71%/1.72% clear valid-gap samples across two seeds, close to the target 2%, versus 0.19%/0.02% for the baseline model and 0.58%/0.15% for ordinary low-time continuation. Naive oversampling overproduces gaps at 7.21%/7.74%; the full comparison, including score errors and generated examples, appears in A3 and the new Appendix H (Table 10 and Figure 10), with a brief description in Section 6. The bridge-time scan appears separately in A4 as part of the shared selection procedure.

Comments on the paper

6 comments
AuthorsOpenReview ↗

General Response (1/6)

We thank all reviewers for their careful reading and constructive feedback. In response to the shared concerns, we have completed five additional sets of experiments and analyses: (A1) independent MD references with uncertainty estimates; (A2) Comparisons of Low-Time Training and Gibbs Sampling; (A3) an image experiment (Notch); (A4) a practical bridge-time selection procedure; and (A5) diagnostics supporting the 2 ns ALDP restrained MD convergence.

Manuscript revisions are highlighted in blue. We identify the corresponding sections, figures, and tables below.

A1. Independent MD references and revised population errors

The original rare-state population reference was computed from the same dataset used for training. We therefore generated new MD trajectories excluded from model training and synthetic-posterior construction. After discarding warmup, we analyze 50 µs for ALDP and 40 µs for Chignolin. For the confidence interval (CI) estimates, we divide these trajectories into nonoverlapping 2 µs blocks. The mean of the block populations and an approximate Student-tt 95% interval based on between-block variability give:

SystemIndependent productionReference rare-state populationApproximate 95% block-tt interval
ALDP (R5)50 µs0.18024%[0.15897%, 0.20151%]
Chignolin (misfold)40 µs0.35975%[0.33611%, 0.38339%]

These intervals rely on approximate independence of the blocks. As a sensitivity check, we also test 3 µs and 5 µs blocks and they are consistent with the 2 µs analysis.

The new molecular results is as following. Absolute relative error is 100∣pmedian/pref−1∣100\lvert p_{\mathrm{median}}/p_{\mathrm{ref}}-1\rvert, using the independent reference estimates above. The improvements of RDB persist when evaluated against trajectories not used to construct the training data

System and modelMedian generated populationAbsolute relative error
ALDP — Baseline model0.12600%30.09%
ALDP — Synthetic-pair RDB0.17050%5.40%
ALDP — Restrained-MD-pair RDB0.13450%25.38%
Chignolin — Baseline model0.31000%13.83%
Chignolin — Synthetic-pair RDB0.35850%0.35%

Manuscript changes. We updated the rare-state references in Sections 6.2–6.3 and Figures 3–4. The figures now show the independent reference with its 95% CI band and red dashed training references, using each budget's own training population in panel (c). The independent-reference protocols are documented in Appendices F.1 and G.2, and rare-state evaluation and population errors are updated in Appendices F.3, F.5, and G.6 (Table 8).

AuthorsOpenReview ↗

General Response (2/6)

A2. Comparisons of Low-Time Training and Gibbs Sampling

We conducted complementary controls to distinguish gains due to targeted bridge training, training-time allocation, and generation.

A2.1. Setting I: Comparison with Low-Time Continuation

Since RDB fine-tunes only the low-time diffusion, we compare it against low-time continuation. Both methods start from the same pretrained model checkpoint and keep training for same additional updates. Low-time continuation uses the standard denoising loss on uniformly sampled equilibrium configurations, with diffusion times restricted to [0,t∗][0,t^*]. Both use the same inference strategy: the frozen pretrained network above t∗t^* and the respective network below. RDB achieves lower rare-population error, demonstrating benefits beyond simply reallocating training to low diffusion times.

ALDP method (50k total updates)t⋆t_\starMedian rare fractionAbsolute relative errorMedian global JS
Baseline model—0.12600%30.09%0.237320
Low-time continuation0.400.14650%18.72%0.238434
Synthetic-pair RDB0.400.17050%5.40%0.237895
Chignolin method (160k total updates)t⋆t_\starMedian rare fractionAbsolute relative errorMedian global JS
Baseline model—0.31000%13.83%0.109266
Low-time continuation0.300.29300%18.55%0.109071
Synthetic-pair RDB0.300.35850%0.35%0.116221
Low-time continuation0.350.29400%18.28%0.108277
Synthetic-pair RDB0.350.36200%0.63%0.110654

Manuscript changes. We added these controls to Appendix F.6 (ALDP, Table 4) and Appendix G.7 (Chignolin, Table 9). Both appendices note that the baseline was rerun, accounting for the small numerical differences from Figures 3–4.

AuthorsOpenReview ↗

General Response (3/6)

A2.2. Comparison under the Same Gibbs Sampling Strategy

In the inference or generation stage, Gibbs sampling strategy cost more. So it is fair to use compare the Restrained-MD-pair RDB gibbs sampling results with low-time continuation gibbs sampling.

ALDP: identical Gibbs sampling for ordinary low-time and Restrained-MD-pair RDB. We apply the same proximal Gibbs sampler to the low-time continuation networks trained on equilibrium data and Restrained-MD-pair RDB networks trained from scratch on restrained data, using the same 25.25 µs MD budget. Both use 20 training seeds, 100,000 chains per seed, t⋆=0.40t_\star=0.40, 50 Gibbs sweeps and start from gaussian noise.

Restrained-MD-pair RDB achieves a lower relative error of 25.38%, compared with 40.63% for low-time continuation under the same Gibbs sampler. The latter is even worse than its standard inference result (18.72%, A2.1), which uses the frozen pretrained model at higher diffusion times. To investigate this discrepancy, we also initialize low-time Gibbs sampling from samples generated by standard inference, but obtain nearly identical final results. These findings demonstrate that Restrained-MD-pair RDB learns a better low-time score than ordinary low-time continuation.

Gibbs modelInitializationMedian R5 population at sweep 50 [seed IQR]Absolute relative error
Low-time continuationGaussian0.10700% [0.10025%, 0.11775%]40.63%
Restrained-MD-pair RDBGaussian0.13450% [0.12625%, 0.14825%]25.38%
Low-time continuationStandard inference0.10750% [0.09825%, 0.11925%]40.36%

The population versus Gibbs sweeps is now shown in Appendix F.6 (Figure 7), including both initialization choices for low-time continuation.

Burn-in sensitivity. For the runs with Gaussian initialization in Appendix F.6 (Figure 7), we average the R5 population over each retained window within each training seed, then report the median across the 20 seeds:

Gibbs modelSweeps 21–50Sweeps 31–50Sweeps 41–50
Low-time continuation0.10863%0.10708%0.10495%
Restrained-MD-pair RDB0.13927%0.13913%0.13340%

R5 effective sample size and sampling cost. We additionally record chain-level histories using the frozen seed-42 checkpoints, with M=10,000M=10{,}000 independent chains per model, the same Gaussian initialization, t⋆=0.40t_\star=0.40, and 50 Gibbs sweeps. Retaining sweeps 21–50 gives 300,000 states per model. Let Ic,jI_{c,j} equal 1 if chain cc is in R5 at sweep jj, and 0 otherwise. The retained-window R5 population for each chain and its mean across chains are

\qquad \hat p=\frac{1}{M}\sum_{c=1}^{M}a_c.$$ Let $s_a$ be the sample standard deviation of the $M$ chain means. We estimate the Monte Carlo standard error and the corresponding **variance-equivalent ESS** as $$\begin{aligned} s_a^2&=\frac{1}{M-1}\sum_{c=1}^{M}(a_c-\hat p)^2,\\ \mathrm{MCSE}&=\frac{s_a}{\sqrt{M}} =\sqrt{\frac{1}{M(M-1)}\sum_{c=1}^{M}(a_c-\hat p)^2},\\ N_{\mathrm{eff}}&=\frac{\hat p(1-\hat p)}{\mathrm{MCSE}^2}. \end{aligned}$$ | Gibbs model | R5 population | MCSE (percentage points) | R5 ESS | ESS / retained states | ESS / million denoiser evaluations | |:---|---:|---:|---:|---:|---:| | Low-time continuation | 0.1380% | 0.0158 | 55,048 | 18.3% | 1,090 | | Restrained-MD-pair RDB | 0.1730% | 0.0189 | 48,385 | 16.1% | 958 | Each run uses 50.49 million sample-level denoiser evaluations, including the initial reverse pass and all 50 sweeps: $10{,}000\times51\times99$. Measured sampling time is 75.7 s per model on one NVIDIA RTX PRO 6000 Blackwell GPU.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.