acceptodds
Under review as a conference paper at ICLR 2027

Steering Diffusion Models to Rare Events with Sequential Monte Carlo

Abstract

Diffusion models are increasingly used as surrogates for expensive simulators in weather prediction, molecular dynamics, and materials design. In these models, computing the probability of an event is difficult, especially when the event of interest is rare. A stable estimate using Monte Carlo becomes computationally intractable, requiring a growing sample size to compensate for an increasing rarity. In this paper, we present Diffusion Importance Sampling of Rare Events or , a sequential Monte Carlo scheme that guides a population of weighted samples towards the rare event, giving access not only to samples but also to a calibrated estimate of its probability. We set up our guidance using an analytical relaxation of the event set, allowing the method to easily extend to a wide range of user-defined rare events. We validate our method on a toy problem with analytical solutions and on a score-based climate emulator, where we obtain accurate rare-event probabilities on a range of rarities from to , achieving net speed-ups of to over Monte Carlo.

Primary area:
generative models
Keywords:
diffusion models, guidance, rare events, sequential monte carlo

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_5DYVOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper estimates rare-event probabilities p[E] under a pretrained diffusion model and samples from p( x |E) by running SMC along the reverse chain. Intermediate targets are reward-tilted, γ_k = p_k e^{r_k}, with a guided proposal and importance weights in closed form for one-step Gaussian integrators. The normalising constant of the SMC gives \hat{p}[E]. It is unbiased for the discretised law when rejuvenation is exactly invariant. A chi squared bound for the no-resampling case identifies the Doob h-transform as the optimal proposal . The practicall reward is a sigmoid function threshold indicator evaluated at a Tweedie Gaussian look-ahead, with schedules \lambda_k and \tau_k . Experiments use a 2D Gaussian mixture and a climate emulator (Pacific Northwest hot and dry extremes). They report \hat{p}/p near 1 down to p ≈ 1e-5, lower CV than Manshausen et al. (2026) on the GMM, and net speed-ups of 6×–651× over crude Monte Carlo.

Critical strengths

The paper is very well written and nicely structured and outlines the sequential Monte Carlo approach very nicely. The theory is well developed. The ablations are done rigorously and this is generally a very nice paper.

Critical weaknesses
  1. I can't shake the feeling that the approach is essentially standard sequential Monte Carlo applied to diffusion models as first pioneered for steering in Boltz (https://www.biorxiv.org/content/10.1101/2024.11.19.624167v3.full.pdf) in Section 4: Boltz Steering.
  2. The authors draw the comparison to Xie et al and there the similarities become even stronger as Xie et al did use MBAR, but also used Delta G diffusion where the potential is a a slope as well. So the comparison comes down to potential design whether one uses the authors sigmoid or the Delta G potential. In the case of Xie et al, one can recover the normalization constant from the weights as well as the expectation over unnormalized weights E_proposal[ unnormalized weights ]. Similarly, MBAR is just method to combine multiple tilts together and trivially reduces to reweighting when only one potential is used. So in my understanding MBAR then is a more general case.
Questions for authors

Regarding 2: Can the authors juxtaposition their work more detailed on a theoretical basis to Xie et al (https://arxiv.org/pdf/2602.16634) and Boltz 1 Steering (https://www.biorxiv.org/content/10.1101/2024.11.19.624167v3.full.pdf) and Twisted SMC (https://arxiv.org/pdf/2404.17546)? On the experimental side all three approaches yield algorithms to which the authors approach should be compared to. A comparison between DireSMC, FKC with a suitable potential and Twisted SMC should be done.

7 comments

We thank the reviewer for highlighting where our contributions and novelty can be made clearer. Firstly, we see DireSMC as a specific instantiation of Sequential Monte Carlo within a diffusion model, in particular of Twisted Diffusion Sampling (TDS; Wu et al., 2023), which is a special case of the Feynman–Kac steering of Singhal et al. (2025) used by Boltz-1. These frameworks provide the general machinery but do not prescribe how to build the reward path (Section 3.2, Eqs. 16 and 18), the guidance (Eq. 17, App. E) or the weight computation (Lemma 1, Theorem 1), or how these interact with the numerical integrator (App. D.1). Our contribution is to elucidate these choices for the rare-event probability estimation problem, where the normalisation constant, not only the samples, is required. Secondly, we see our work as complementary to Xie et al. (2026). They focus on adaptively constructing bias potentials and combining them with MBAR, with Feynman–Kac correctors (FKC; Skreta et al., 2025) generating the particles under each bias; we instead focus on that guidance step itself. These design choices measurably matter: DireSMC's weights stay on target where FKC weights underestimate by about 20% under a learned score (Table R3.1), and the difference potential, as in TDS, remains stable where the max and sum potentials of Singhal et al. (2025) collapse to a single lineage (Q1). Additionally, Lemma 1 extends TDS and DireSMC beyond Euler–Maruyama to any first-order stochastic integrator, where an exponential integrator reduces the discretisation bias (Table 6, App. G.2.2). Hence, DireSMC could serve as the steering component within the framework of Xie et al. (2026). We make this positioning precise below and report the requested comparisons, which we will add to the revised manuscript.

W.1 I can't shake the feeling that the approach is essentially standard sequential Monte Carlo applied to diffusion models as first pioneered for steering in Boltz (link) in Section 4: Boltz Steering.

We agree that DireSMC is SMC applied to a diffusion model. It builds upon TDS (Wu et al., 2023), which the Feynman–Kac steering used in Boltz-1 (Singhal et al., 2025; Wohlwend et al., 2025) generalises (see Q1 below). Our contribution is not the framework but how to instantiate it when the goal is a rare-event probability, and this goal is the key difference from Boltz-1: like Boltz-1, we use SMC to improve the quality of guided samples, but we also use it to estimate the normalisation constant itself, which places stricter demands on each component. Specifically, we elucidate:

  • Weights and integrator. Lemma 1 gives the exact incremental weight for any first-order stochastic integrator through a single scalar ζk\zeta_k (App. D.1), with no extra network calls, so the estimate is unbiased for the discretised sampler for any KK, NN and reward (Theorem 1). We also study how the integrator affects the discretisation bias, and find an exponential integrator (SEEDS-1; Gonzalez et al., 2023) preferable to Euler–Maruyama (App. G.2.2, Table 6).
  • Reward path. Theorem 2 separates the variance into the contribution of the endpoint reward and that of the transition kernels (Section 3.2). This motivates the Gaussian-CDF endpoint (Eq. 16), which gives a usable gradient for an indicator event while keeping the endpoint factor bounded, and the look-ahead reward path (Eq. 18).
  • Guidance. For linear observables the look-ahead is closed-form under a Gaussian posterior with a Π\PiGDM covariance proxy, rather than a point estimate, and extends to multiple constraints (App. E, E.1). The schedules λk\lambda_k and τk\tau_k are ablated in App. G.5 and H.5.
  • Resampling and rejuvenation. We study when each helps, and when approximate rejuvenation introduces bias (App. G.3.3, H.2.1, App. I).

References

Gonzalez, M., et al. (2023). Seeds: Exponential sde solvers for fast high-quality sampling from diffusion models. Advances in Neural Information Processing Systems, 36, 68061-68120.

Singhal, R., et al. (2025). A general framework for inference-time scaling and steering of diffusion models. In Forty-second International Conference on Machine Learning.

Skreta, M., et al. (2025). Feynman-kac correctors in diffusion: Annealing, guidance, and product of experts. arXiv:2503.02819.

Wohlwend, J., et al. (2025). Boltz-1 Democratizing Biomolecular Interaction Modeling. bioRxiv 2024.11.19.624167.

Wu, L., et al. (2023). Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems, 36, 31372-31403.

Xie, Y., et al. (2026). Enhanced diffusion sampling: Efficient rare event sampling and free energy calculation with diffusion models. The Journal of Chemical Physics, 165.16. arXiv:2602.16634.

W.2/Q.1 The authors draw the comparison to Xie et al. and there the similarities become even stronger as Xie et al did use MBAR, but also used Delta G diffusion where the potential is a a slope as well. So [..]

As noted above, we view the two works as complementary. Xie et al. (2026) target free energies of metastable states, focusing on how to adaptively construct bias potentials to estimate them. To do this, they use Feynman–Kac correctors (FKC; Skreta et al., 2025) to generate particles under each bias potential and combine these using MBAR. Our focus is the guidance step itself: for a single, fixed event (i.e. a single tilt), how to design the reward, schedule and weights so that the probability estimate is unbiased and has low variance (Theorems 1 and 2, Lemma 1). Nothing prevents the two from being combined: DireSMC could serve as the steering component within the framework of Xie et al., and their adaptive biasing and MBAR could be layered on top of our estimator. We agree that single-tilt MBAR reduces to importance reweighting and that MBAR is the more general tool for pooling several tilts; for a single rare event, its analogue is the splitting estimator (App. C.4).

DireSMC and FKC. As requested, we compare DireSMC against FKC as used in Xie et al. (2026), under the same look-ahead potential to keep the comparison fair. Under a learned score, FKC weights systematically underestimate the probability by about 20% (Table R3.1), an error that does not vanish as K→∞K \to \infty (Table R3.2). In contrast, DireSMC's weights remain on target. We will add the detailed discussion and additional experiments to the camera-ready version.

FKC vs. exact weights. Both methods simulate a guided reverse SDE, dX=(b+c,a∇Rt),dtˉ+a,dWdX = (b + c\\,a\nabla R_t)\\,d\bar t + \sqrt{a}\\,dW, with b=−f+a,sθb = -f + a\\,s_\theta, a=g(t)2a = g(t)^2, and c=1c = 1 (ours) or c=12c = \tfrac12 (FKC default). Let qtq_t be the marginal of the learned sampler and ht=∇log⁡qth_t = \nabla\log q_t its true score. The Feynman–Kac weight rate that makes the weighted process target γt=qteRt\gamma_t = q_t e^{R_t} is

gc=−∂tR+b⋅∇R−a(1−c),ht⋅∇R+a(c−12)(ΔR+∣∇R∣2).g_c = -\partial_t R + b\cdot\nabla R - a(1-c)\\,h_t\cdot\nabla R + a\big(c - \tfrac12\big)\big(\Delta R + \\|\nabla R\\|^2\big).

At c=12c = \tfrac12 (FKC) the Laplacian drops out, but hth_t remains and is unknown, so FKC substitutes sθs_\theta. This omits the term

a2,(sθ−ht)⋅∇Rt,\tfrac{a}{2}\\,\big(s_\theta - h_t\big)\cdot\nabla R_t,

which we denote as the self-consistency error or the network's score versus the score of the discretised sampler's marginals, projected onto the reward gradient. It vanishes only for an exact score and is present for a learned network. With a larger reward gradient, i.e. for increasing rarity, this error grows with ∣∇Rt∣\\|\nabla R_t\\|.

At c=1c = 1 the hth_t term vanishes, but the rate needs ΔR\Delta R (dd extra second-derivative passes per step).

In both cases, integrating the rate along the chain with Euler steps adds an O(1/K)O(1/K) quadrature error.

Our weights are instead the exact likelihood ratio of the Gaussian transitions we simulate (base Pk=N(mk,vkI)P_k = \mathcal N(m_k, v_k I), guided Qk=N(mk+c,εk∇Rk,vkI)Q_k = \mathcal N(m_k + c\\,\varepsilon_k\nabla R_k, v_k I), zkz_k the step's noise, ζk=c,εk/vk\zeta_k = c\\,\varepsilon_k/\sqrt{v_k}):

Δlog⁡wk=Rk+1(xk+1)−Rk(xk)−ζk22∣∇Rk∣2−ζk,∇Rk⋅zk.\Delta\log w_k = R_{k+1}(x_{k+1}) - R_k(x_k) - \tfrac{\zeta_k^2}{2}\\|\nabla R_k\\|^2 - \zeta_k\\,\nabla R_k\cdot z_k.

Since sθs_\theta enters PkP_k and QkQ_k identically, it cancels: the weight is exact for any network and any KK, and needs neither hth_t nor ΔR\Delta R. At c=1c = 1 it recovers g1g_1 path by path as K→∞K \to \infty.

References

Skreta, M., et al. (2025). Feynman-kac correctors in diffusion: Annealing, guidance, and product of experts. arXiv:2503.02819.

Xie, Y., et al. (2026). Enhanced diffusion sampling: Efficient rare event sampling and free energy calculation with diffusion models. The Journal of Chemical Physics, 165.16. arXiv:2602.16634.

Experiment. We ran additional experiments to compare to the FKC weights on the 2D GMM model (with τ=10\tau = 10, K=1000K = 1000, adaptive resampling, no rejuvenation, 96 runs per cell; same SEEDS-1 integrator and look-ahead twist for every arm). FKC's original reward, evaluated on the noisy state, diverges at this rarity (∣∇r∣∼105\\|\nabla r\\| \sim 10^5), so we give FKC our look-ahead twist. To isolate the weight rule, we also accumulate the FK weight on the exact-weight sampler's own particles (N=104N = 10^4) and report FK/exact.

Overall, we find:

  • DireSMC weights are on target in this setting, while FK weights are under-estimates (Table R3.1). We also found in additional experiments that with the learned score, the gap grows with rarity (0.94 at τ=7\tau = 7 to 0.77 at τ=10\tau = 10, same paths, K=1000K = 1000).
  • With the analytic score, FKC's error is dominated by the discretisation error alone, while with the learned score, an error appears to remain as K→∞K \to \infty (Table R3.2). For the analytic score, ht=sθh_t = s_\theta and the omitted term is zero, whereas for the learned score the omitted term a2(sθ−ht)⋅∇Rt\tfrac{a}{2}(s_\theta - h_t)\cdot\nabla R_t does not vanish with KK. It also agrees with our MADM (Lam et al., 2026) analysis (Fig. 12, App. G.3.3): the learned score has a non-zero antisymmetric Jacobian, increasingly so as t→0t \to 0 (Vuong et al., 2025), so it is not the gradient of any potential and cannot equal ht=∇log⁡qth_t = \nabla\log q_t. The limit here is an extrapolation from K≤4000K \le 4000, and since hth_t is unknown for a network, we identify the size of this term by elimination rather than measuring it directly.
  • Cost. For c = 1, FK weight requires the Laplacian ΔR\Delta R, whereas DireSMC weights need no evaluations beyond the sampler's own.

Table R3.1. 2D GMM, τ=10\tau = 10, K=1000K = 1000, 96 runs per cell. Own runs: each method runs with N=1000N = 1000 and resamples on its own weights; p^/p∗±\hat p/p^* \pm s.e. (CV), where p∗p^* is the exact probability for the analytic score and, for the learned score, a crude Monte Carlo estimate of the event probability under its discretised sampler at the same KK, with a relative Monte Carlo error of ±1.6\pm 1.6\\%. Same paths: we run DireSMC (exact weights, Eq. 12) with the same push cc as the FK arm, i.e. DireSMC itself for c=1c = 1 and DireSMC with a half push for FKC (N=104N = 10^4), and, along each particle's path, also accumulate the FK log-weight. Both estimates are computed from the same particles, so their ratio isolates the weight rule. Note that for c=1c = 1, the FK weights are an Euler discretisation of the continuous-time weight and are exact only as K→∞K \to \infty; ours are exact for the simulated chain at every KK.

\text{Score} & \text{Arm} & \text{Own runs: } \hat p/p^*\\ \text{(CV)} & \text{Same paths: FK} \div \text{DireSMC weights} \\\\ \hline \text{analytic} & \textbf{DireSMC } (c=1) & 0.998 \pm 0.032\\ (0.31) & - \\\\ \text{analytic} & \text{FKC } (c=\tfrac12,\\ \text{FK}) & 0.881 \pm 0.069\\ (0.76) & 0.897 \pm 0.020 \\\\ \text{analytic} & c=1,\\ \text{FK (needs } \Delta R) & 0.848 \pm 0.025\\ (0.28) & 0.859 \pm 0.001 \\\\ \hline \text{learned} & \textbf{DireSMC } (c=1) & 1.024 \pm 0.027\\ (0.26) & - \\\\ \text{learned} & \text{FKC } (c=\tfrac12,\\ \text{FK}) & 0.802 \pm 0.062\\ (0.75) & 0.765 \pm 0.015 \\\\ \text{learned} & c=1,\\ \text{FK (needs } \Delta R) & 0.856 \pm 0.025\\ (0.28) & 0.843 \pm 0.001 \end{array}$$ **References** Lam, K. H., et al. (2026). Metropolis-adjusted diffusion models. arXiv:2605.09654. Vuong, A. B., et al. (2025). Are we really learning the score function? Reinterpreting diffusion models through Wasserstein gradient flow matching. arXiv:2509.00336. ```
Reviewer_W2QAOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an SMC based diffusion algorithm to sample from rare events. It builds on [1], but fixes one of the major problems when trying to "tilt" the distribution towards a potential of the form 1E(x)1_{E}(x) which is the non smoothness and flat gradients of the potential. This is done by approximating the reward by a smooth version. Furthermore, the authors introduce several tools to make the SMC functions better, such as using a Gaussian CDF for the approximation, using PiGDM and not simply the DPS approximation and also using MADM.

The theoretical part recaps and contextualize well known results of SMC to the problem in hand and is well written.

The numerical evaluation is sound with a controlled setting example and a relevant climate application which is nice.

[1] Wu, L., Trippe, B., Naesseth, C., Blei, D., & Cunningham, J. (2023). Practical and asymptotically exact conditional sampling in diffusion models. Advances in neural information processing systems, 36, 31372-31403.

Critical strengths

The paper is well written, the context of guided sampling, sequential Monte Carlo and their challenges are clearly explained. The proposed algorithm is sound and combine several of the available tools in the literature in an elegant way. The numerical section is well designed, presenting both a controlled scenario and a realistic application of such models.

Critical weaknesses

Theorem 2 is done only without resampling, and as I understand, it indicates only that Doob's h transform is the ideal guidance, something which is already well known in the literature, at least for standard guiding functions. A much more relevant bound would be including the resampling strategy and ideally indicate how one could tune the proposal kernel (namely the PiGDM approximation and the choice of λ\lambda) and / or the value of δ\delta.

As I understand, the introduction of MADM, imply that a bias is introduced in the algorithm which can not be removed (due to the score approximation). While this seem to enhance the performance, it breaks the asymptotic correctness of the SMC procedure.

The authors could have made a more clear state of the art concerning diffusion guidance and guidance within SMC. In particular, the current algorithm is closely related to [1], namely the proposals are both based on DPS (Chung et al. 2023) and a reweighting mechanism based on the "look ahead" potentials. Instead, the authors refer to Uehara et al. 2025, which is an arxiv paper. This is also true for the introduction of the guidance in the context of diffusion, namely around equation (9) and I think that a citation to Chung et al. 2023 is better suited than a citation to Uehara et al. 2025.

[1] Wu, L., Trippe, B., Naesseth, C., Blei, D., & Cunningham, J. (2023). Practical and asymptotically exact conditional sampling in diffusion models. Advances in neural information processing systems, 36, 31372-31403.

Questions for authors
  1. Maybe I'm missing something, but it seems that theorem 2 simply states the optimality of the Doob transform. Wouldn't a more relevant bound include the variance of the SMC estimator rather than the "resampling off" estimator ?
  2. When does it break ? I think a nice addition would be to evaluate, for a given number of particles, the precision of the estimates while increasing the value of τ\tau. Do the sampler actually manage to go "further then the prior" ?
  3. Would it be relevant to add an arm to table 8 consisting of "no resampling + rejuvenating" ? It seems so, given that rejuvenating seems to be what is really driving most of the gains for the case τ=10\tau = 10.
7 comments

We thank the reviewer for their detailed feedback; we will update the manuscript to clarify the comments raised accordingly. In response, (i) we will add the no-resampling + rejuvenation arm to Table 8 and test whether rejuvenation alone is sufficient (Q3), (ii) clarify how resampling affects Theorem 2 (Q1).

W.1/Q.1 Maybe I'm missing something, but it seems that theorem 2 simply states the optimality of the Doob transform. Wouldn't a more relevant bound include the variance of the SMC estimator rather than the "resampling off" estimator?

While the optimality of the Doob transform is well known, the contribution of Theorem 2 is the quantitative decomposition of the estimator's variance into two design choices: the endpoint reward, through the factor γ0[1]/(c,p0[E])\gamma_0[1]/(c\\,p_0[E]), and the transition kernels, through χ2(P0:K⋆,∣,Q0:K)\chi^2(\mathbb P^\star_{0:K}\\,\\|\\,\mathbb Q_{0:K}), whose minimum under Doob's transitions is the initialisation error χ2(πK⋆,∣,pref)\chi^2(\pi^\star_K\\,\\|\\,p_{\mathrm{ref}}). This decomposition guides the design choices we use throughout the paper. In particular, the variance need not grow with rarity if the path quality is controlled; in Fig. 2 (middle), the CV of DireSMC + MADM (Lam et al., 2026) stays roughly constant across the rarities tested at equal compute.

We will include an extended variant of the theorem in the Appendix. Without resampling and with the endpoint reward er0=c,1Ee^{r_0} = c\\,\mathbf 1_E, Theorem 2 holds with equality and gives the relative variance in terms of the path divergence from the Doob-transformed process:

CV2[p^0[E]]=1N,χ2(P0:K⋆,∣,Q0:K).\mathrm{CV}^2\big[\hat p_0[E]\big] = \frac{1}{N}\\,\chi^2\big(\mathbb P^\star_{0:K} \\,\\|\\, \mathbb Q_{0:K}\big).

With resampling at JJ times followed by rejuvenation, and under the idealised assumption of perfect mixing between consecutive resampling times, the path divergence splits into the incremental divergences χj2\chi^2_j, one per segment between resampling times (Syed et al., 2026, Theorem 1(a)):

CV2[p^0[E]]≈∏j=0J(1+χj2N)−1≈1N∑j=0Jχj2.\mathrm{CV}^2\big[\hat p_0[E]\big] \approx \prod_{j=0}^{J}\left(1 + \frac{\chi^2_j}{N}\right) - 1 \approx \frac{1}{N}\sum_{j=0}^{J} \chi^2_j.

Setting J=0J = 0 (no resampling), the single segment is the whole path, χ02=χ2(P0:K⋆,∣,Q0:K)\chi^2_0 = \chi^2(\mathbb P^\star_{0:K} \\,\\|\\, \mathbb Q_{0:K}), and the product reduces exactly to Theorem 2. This shows what resampling buys: without it, the incremental divergences compound multiplicatively along the path, 1+χ2(P0:K⋆,∣,Q0:K)=∏j(1+χj2)1 + \chi^2(\mathbb P^\star_{0:K} \\,\\|\\, \mathbb Q_{0:K}) = \prod_j (1 + \chi^2_j) under the same assumption, whereas with resampling they add.

We caveat this analysis as we only have access to an approximate MCMC move with MADM (Lam et al., 2026). Different resampling schemes do not change this approximation but add different amounts of resampling noise (Chopin et al., 2022); we use systematic resampling, a standard low-variance choice.

References

Chopin, N., et al. (2022). On resampling schemes for particle filters with weakly informative observations. The Annals of Statistics, 50.6, 3197–3222.

Lam, K. H., et al. (2026). Metropolis-adjusted diffusion models. arXiv:2605.09654.

Syed, S., et al. (2026). Optimized annealed sequential Monte Carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, qkag082.

Tuning: The tuning parameters affect only the variance of the estimator, not what it estimates (Theorem 1). We set τ(t)=τ\tau(t) = \tau for simplicity. Following Theorem 2, δ(0)\delta(0) is kept small so that the endpoint factor γ0[1]/(c,p0[E])\gamma_0[1]/(c\\,p_0[E]) is close to its minimum. λ\lambda is pinned at λ(1)=0\lambda(1) = 0 and λ(0)=1\lambda(0) = 1. In between, λ(t)=0\lambda(t) = 0 for t≥0.9t \ge 0.9 and then rises, acting as a tempering of the look-ahead likelihood. Holding it at zero early was selected heuristically, as Tweedie's estimate may be poor at high noise. This tempering mirrors guidance schedules used in the literature (e.g. Kynkäänniemi et al., 2024; Wang et al., 2024). The transport-time width δ(t)\delta(t) and the Π\PiGDM proxy Σk\Sigma_k (Song et al., 2023) together set the look-ahead width δ2(t)+a⊤Σka\sqrt{\delta^2(t) + a^\top \Sigma_k a}, which grows with the uncertainty of Tweedie's estimate, similar to the likelihood inflation of Rozet & Louppe (2023). We start δ(t)\delta(t) wide and narrow it towards δ(0)\delta(0), which we find empirically improves performance (Table 22). We chose all schedules by the CV of short pilot runs on one problem per setting and reused them, which suggests the choices are robust. We do note that the selected schedules may be suboptimal for individual problems. Additionally, we leave τ(t)\tau(t) as well as a more principled method for selecting the schedules as planned future work.

W.2 As I understand, the introduction of MADM, imply that a bias is introduced in the algorithm which can not be removed (due to the score approximation). While this seem to enhance the performance, it breaks the asymptotic correctness of the SMC procedure.

We agree with the reviewer that MADM is not an exact MCMC kernel for the discrete sampler's law and injects bias through a reducible discretisation error as well as the score approximation error; we study both of these effects in Figs. 12 and 13 and Table 9 (App. G.3.3). We mitigate this by gating rejuvenation to times where the "score curl" (the antisymmetric part of the score's Jacobian) is low, applying moves only for t≥0.3t \ge 0.3 (Fig. 12, Table 9), but we acknowledge that this breaks the theoretical assumptions of SMC. An area for further research could be (i) defining an exact MCMC kernel for a diffusion model and (ii) developing a practical tolerance threshold for an approximate MCMC kernel that trades off diversification against exactness. Hence, we present MADM as an optional variance reduction strategy that can be employed by the practitioner, which we turn off in the headline climate results quoted throughout the paper.

W.3 The authors could have made a clearer state of the art concerning diffusion guidance and guidance within SMC. In particular, the current algorithm is closely related to [1], namely, the proposals are both based on DPS (Chung et al. 2023) and a reweighting mechanism based on the "look ahead" potentials. Instead, the authors refer to Uehara et al. 2025, which is an arxiv paper. This is also true for the introduction of the guidance in the context of diffusion, namely around equation (9) and I think that a citation to Chung et al. 2023 is better suited than a citation to Uehara et al. 2025.

We agree and thank the reviewer for pointing this out. DireSMC directly builds on the Twisted Diffusion Sampler (TDS; Wu et al., 2023) and uses proposals based on DPS/Π\PiGDM. We will cite Chung et al. (2023) in place of Uehara et al. (2025) around Eq. (9) and expand the related work on diffusion guidance and on guidance within SMC in the revision.

References

Chung, H., et al. (2023). Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations.

Kynkäänniemi, T., et al. (2024). Applying guidance in a limited interval improves sample and distribution quality in diffusion models. Advances in Neural Information Processing Systems, 37, 122458-122483.

Rozet, F., & Louppe, G. (2023). Score-based data assimilation. Advances in Neural Information Processing Systems, 36, 40521-40541.

Song, J., et al. (2023). Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations.

Uehara, M., et al. (2025). Inference-time alignment in diffusion models with reward-guided generation: Tutorial and review. arXiv:2501.09685.

Wang, X., et al. (2024). Analysis of classifier-free guidance weight schedulers. arXiv:2404.13040.

Wu, L., et al. (2023). Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems, 36, 31372-31403.

Q.2 When does it break? I think a nice addition would be to evaluate, for a given number of particles, the precision of the estimates while increasing the value of τ\tau. Do the sampler actually manage to go "further then the prior"?

We agree that it is a useful addition and will add a figure showing the CV of p^\hat p at a fixed number of particles against τ\tau for both settings, bringing together results that are currently spread across the paper. On the GMM at N=1000N = 1000, the CV of the deployed configuration stays between 0.06 and 0.09 from τ=7\tau = 7 (p⋆≈7×10−3p^\star \approx 7 \times 10^{-3}) to τ=10\tau = 10 (p⋆≈1.4×10−5p^\star \approx 1.4 \times 10^{-5}), for both the analytic and the learned score (Table 7), while the CV of crude Monte Carlo at equal compute grows with rarity (Fig. 2, middle). On the climate emulator at N=1024N = 1024, the CV with adaptive resampling stays between 0.07 and 0.13 from 10−310^{-3} to 10−510^{-5} for the hot and dry events.

Theorem 2 indicates where the breaking point lies, depending on how far the guided kernels are from Doob's h-transform: for the indicator endpoint, CV2=χ2(P0:K⋆,∣,Q0:K)/N\mathrm{CV}^2 = \chi^2(\mathbb P^\star_{0:K}\\,\\|\\,\mathbb Q_{0:K})/N, so rarity enters only through this divergence. The estimator therefore breaks down when this divergence becomes comparable to NN, i.e. when the ESS falls to a handful of particles, rather than when p⋆p^\star approaches 1/N1/N, as for crude Monte Carlo with CV2=(1−p⋆)/(Np⋆)\mathrm{CV}^2 = (1-p^\star)/(Np^\star). We expect the divergence to grow with rarity, since the guidance must steer further into the tail, where the Gaussian look-ahead is less accurate. Within the ranges tested, the sampler has not broken down, but the growth is visible. Without resampling, ESS/N on the climate emulator falls from 0.06 at 10−310^{-3} to 0.02 at 10−510^{-5}, and the Pareto k^\hat k exceeds 0.7 at 10−510^{-5}, indicating unreliable importance weights (Table 16); with resampling, one of the 16 dry runs at 10−410^{-4} collapsed. Since the ESS tracks this divergence during sampling, a run approaching the breaking point can be detected without a reference; we temper this diagnostic with the fact that we did not explicitly tune the parameters for each problem setting.

Further than the prior: DireSMC samples from the diffusion model's distribution conditioned on the event, so it cannot go beyond the model's support, but within it reaches events that crude Monte Carlo does not produce at matched compute (Figs. 3 and 4). Where the learned score is least reliable, at high rarity, the samples are only as informative as the model; the rarest samples we generated appear physically plausible (Figs. 16–18), but a systematic physical evaluation is beyond the scope of this work.

Q.3 Would it be relevant to add an arm to table 8 consisting of "no resampling + rejuvenating"? It seems so, given that rejuvenating seems to be what is really driving most of the gains for the case τ=10\tau=10.

We agree that this is an interesting addition and show an updated Table 8 below. Resampling and rejuvenation address different sources of degeneracy: resampling prunes accumulated weight imbalance by replicating high-weight particles, at the cost of additional Monte Carlo variance (Chopin et. al., 2022), while rejuvenation restores particle diversity. Their combinations give sequential importance sampling, sequential importance resampling, annealed importance sampling and sequential Monte Carlo. The efficiency of each of the regimes depends on the guidance schedule and on how well the MCMC kernels mix (Syed et al., 2026), but a full treatment of these within the steered diffusion setting would be interesting future work.

References

Chopin, N., et al. (2022). On resampling schemes for particle filters with weakly informative observations. The Annals of Statistics, 50.6, 3197-3222.

Syed, S., et al. (2026). Optimized annealed sequential Monte Carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, qkag082.

Reviewer_upagOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces DireSMC, a method for sampling rare events and estimating their probabilities under a pretrained diffusion model. It combines guided reverse diffusion with sequential importance weighting and adaptive resampling. A smooth Gaussian-CDF relaxation of the event indicator provides tractable approximate look-ahead guidance for linear observables, with extensions to multiple constraints. Under the stated assumptions, the probability estimator is unbiased for the endpoint distribution of the discretised base sampler. The paper also analyses its variance through the discrepancy from an ideal Doob-transformed process.

Experiments use a two-dimensional Gaussian mixture with analytical and learned scores, followed by a pretrained climate emulator generating monthly surface fields. The climate experiments consider temperature extremes, compound hot-and-dry events, and changes in event probability with global warming. The authors report substantial reductions in computational cost relative to crude Monte Carlo, including estimated speed-ups of approximately 6–651 times for compound events at matched relative precision.

Critical strengths

A central strength is the integration of rare-event generation and probability estimation within a single denoising procedure. The importance weights correct for the guided proposal, while the smooth event relaxation provides a practical mechanism for directing particles towards the event. The distinction between the probability under the discretised model and the probability under the underlying data distribution is particularly valuable. This makes the contribution relevant to scientific applications where obtaining extreme samples alone is insufficient.

The empirical investigation is substantial. The Gaussian-mixture experiments examine analytical and learned scores, discretisation effects, resampling, rejuvenation, and conditional sample quality. The climate experiments demonstrate applicability beyond a low-dimensional example and include correlated compound events. Selecting the climate guidance schedule on one event and reusing it across the other settings also provides useful evidence of robustness. The reported gains account for the additional per-sample cost of guidance, which strengthens their practical relevance.

Critical weaknesses

The high-dimensional experiments establish an advantage over crude Monte Carlo, but provide less evidence about the advantage over specialised alternatives. The comparison with the probability-flow ODE estimator of Manshausen et al. is restricted to the two-dimensional Gaussian mixture, where the compute adjustment uses an evaluation-count model. A comparison on the climate emulator, or a representative reduced setting, using measured runtime and matched estimation accuracy would better establish the practical benefit relative to an existing rare-event probability estimator.

Questions for authors

Table 13 indicates that guided runs use thresholds from an earlier reference pool, while the reported reference probabilities use slightly different thresholds. Could you report comparisons evaluated at exactly the same event thresholds and include uncertainty from both the guided estimates and the Monte Carlo reference? This is particularly relevant at probabilities around 10^-5, where the reference itself has appreciable relative uncertainty. It would help distinguish sampling variability from an actual discrepancy in probability estimation.

Could you provide a measured-runtime comparison with the Manshausen et al. estimator on the climate emulator, or on a representative reduced configuration? Accounting for numerical bias as well as variability would help assess whether the advantage observed in the Gaussian-mixture experiment transfers to the application setting.

5 comments

We thank the reviewer for their detailed review. We have conducted additional experiments which significantly expand the comparison to existing work - see our response to Q2 below as well as our response to Reviewer "5DYV", where we theoretically and empirically compare against the weight computation used in Xie et al. (2026) based on Feynman–Kac correctors (FKC; Skreta et al., 2025), which we hope will alleviate the reviewer's remaining concerns.

Q.1 Could you report comparisons evaluated at exactly the same event thresholds and include uncertainty from both the guided estimates and the Monte Carlo reference?

We have updated all guided runs and now use exactly the reference pool's quantile threshold (Table 13 is now revised). We will propagate the update throughout the manuscript. Table R1.1 reports each estimate against its reference with both uncertainties combined. Eight of the nine ratios lie within one combined standard deviation of 1; the most common joint event lies 2.1σ2.1\sigma below (0.965±0.0170.965 \pm 0.017), in line with what is expected across nine comparisons. At the rarest level we test (10−510^{-5}), the reference itself contributes over 90% of the combined variance, so agreement there is limited by the 9-million-draw reference rather than by DireSMC.

Table R1.1. DireSMC with adaptive resampling (N=1024N = 1024, K=200K = 200, 16 seeds) at the reference thresholds. p⋆p^\star: crude Monte Carlo reference from 9.0×1069.0 \times 10^6 unguided draws; ref. s.e.: its binomial relative standard error. Guided s.e.m.: standard error of the mean of p^/p⋆\hat p/p^\star over the 16 seeds. The last column combines the two, which are independent, in quadrature. Joint events use per-variable thresholds at the 10−210^{-2}, 10−310^{-3} and 10−410^{-4} marginal quantiles.

\text{event} & p^\star & \text{ref. s.e.} & \text{guided s.e.m.} & \hat p / p^\star \pm \text{comb. sd} \\\\ \hline \text{hot} & 10^{-3} & 1.1\\% & 0.019 & 0.982 \pm 0.022 \\\\ \text{hot} & 10^{-4} & 3.3\\% & 0.021 & 1.010 \pm 0.040 \\\\ \text{hot} & 10^{-5} & 10.5\\% & 0.034 & 1.048 \pm 0.115 \\\\ \hline \text{dry} & 10^{-3} & 1.1\\% & 0.018 & 0.996 \pm 0.021 \\\\ \text{dry} & 10^{-4} & 3.3\\% & 0.055 & 1.015 \pm 0.065 \\\\ \text{dry} & 10^{-5} & 10.5\\% & 0.029 & 1.055 \pm 0.114 \\\\ \hline \text{hot } \\& \text{ dry} & 3.1 \times 10^{-3} & 0.6\\% & 0.016 & 0.965 \pm 0.017 \\\\ \text{hot } \\& \text{ dry} & 2.1 \times 10^{-4} & 2.3\\% & 0.028 & 0.990 \pm 0.036 \\\\ \text{hot } \\& \text{ dry} & 1.4 \times 10^{-5} & 8.9\\% & 0.021 & 1.044 \pm 0.095 \end{array}$$ **References** Skreta, M., et al. (2025). Feynman-kac correctors in diffusion: Annealing, guidance, and product of experts. arXiv preprint arXiv:2503.02819. Xie, Y., et al. (2026). Enhanced diffusion sampling: Efficient rare event sampling and free energy calculation with diffusion models. *The Journal of Chemical Physics*, 165.16. arXiv:2602.16634.

Q.2 Could you provide a measured-runtime comparison with the Manshausen et al. estimator on the climate emulator, or on a representative reduced configuration? Accounting for numerical bias as well as variability would help assess whether the advantage observed in the Gaussian-mixture experiment transfers to the application setting.

We have conducted additional extensive comparisons with the estimator of Manshausen et al. (2026). Before we present the results, note that under a learned score, the ODE and SDE samplers define different marginal/terminal laws (Deveney et al., 2025), so we score each estimator against the law of its own discretised sampler at matched KK. Each sampler's reference is a crude Monte Carlo run of that sampler at the same KK. On the GMM where p0[E]p_0[E] is known exactly, the ODE sampler has the larger discretisation bias (Fig. 7, App. G.2.2; Fig. 2, right), consistent with Xu et al. (2023, Thm. 3) and Gonzalez et al. (2023). We report the hot event at 10−310^{-3} and 10−410^{-4} with K=50K = 50 and 200200; the ODE references currently hold 0.6–0.8 million draws, and we will extend them for the camera-ready version. Both methods use identical guidance with gain λ=1\lambda = 1, which a sweep over λ∈[0.25,3]\lambda \in [0.25, 3] showed has the lowest variance (as well as discretisation bias, but this is not readily discernible without access to the MC estimate). We report with rejuvenation turned off for both methods.

Table R1.2. DireSMC vs. PF-ODE importance sampling on the Northwest July heat event (N=1024N = 1024 particles, 16 runs per cell, H100 GPU). Each estimator is scored against its own sampler's law at the same KK. p^\hat p/own law: mean estimate over runs divided by that law, ±\pm the run and reference standard errors combined in quadrature. σ\sigma: distance of the ratio from 1 in total standard errors; estimates more than 2 away are in bold. CV: standard deviation over runs divided by the mean. Time: GPU-seconds per run. Fraction of particles ending in the event: DireSMC with resampling 83% (10−310^{-3}) and 92–96% (10−410^{-4}), without resampling 22–31%; PF-ODE 22% (10−310^{-3}) and 15% (10−410^{-4}).

\text{Setting} & \text{Method} & \hat p\\,/\\,\text{own law} & \sigma & \text{CV} & \text{Time (s)} \\\\ \hline p=10^{-3},\\ K=50 & \text{DireSMC, no resampling} & 1.033 \pm 0.048 & +0.7 & 0.145 & 159 \\\\ & \text{DireSMC, resampling} & 1.006 \pm 0.037 & +0.2 & 0.087 & 158 \\\\ & \text{PF-ODE} & \mathbf{1.559 \pm 0.068} & \mathbf{+8.2} & 0.086 & 1592 \\\\ \hline p=10^{-3},\\ K=200 & \text{DireSMC, no resampling} & 1.024 \pm 0.052 & +0.5 & 0.199 & 590 \\\\ & \text{DireSMC, resampling} & 0.983 \pm 0.022 & -0.8 & 0.078 & 592 \\\\ & \text{PF-ODE} & \mathbf{1.102 \pm 0.041} & \mathbf{+2.5} & 0.093 & 6362 \\\\ \hline p=10^{-4},\\ K=50 & \text{DireSMC, no resampling} & 0.984 \pm 0.105 & -0.2 & 0.251 & 152 \\\\ & \text{DireSMC, resampling} & 0.928 \pm 0.084 & -0.9 & 0.101 & 153 \\\\ & \text{PF-ODE} & \mathbf{1.758 \pm 0.203} & \mathbf{+3.7} & 0.156 & 1588 \\\\ \hline p=10^{-4},\\ K=200 & \text{DireSMC, no resampling} & 1.021 \pm 0.058 & +0.4 & 0.191 & 596 \\\\ & \text{DireSMC, resampling} & 1.011 \pm 0.037 & +0.3 & 0.082 & 590 \\\\ & \text{PF-ODE} & 1.004 \pm 0.093 & +0.0 & 0.177 & 6353 \end{array}$$ Note that DireSMC agrees with its own sampler's law in all eight cells, each within 1 standard deviation. PF-ODE is within 10% of its own law at $K = 200$ ($+10 \pm 4\\%$ at $10^{-3}$, $1.00 \pm 0.09$ at $10^{-4}$), but overshoots at $K = 50$ (56% and 76%); we believe that this is a systematic bias arising from the reversibility assumptions of an explicit Euler integrator, as discussed in App. J. With resampling, DireSMC and PF-ODE estimators have the same CV at $10^{-3}$; at $10^{-4}$ and $K = 200$, PF-ODE's CV is $2.2\times$ that of DireSMC. At the same $K$, a PF-ODE run takes 10–11× more GPU time than a DireSMC run due to the additional likelihood solves and Hutchinson trace estimator required (3 probes, as prescribed in Manshausen et al.). **References** Deveney, T., et al. (2025). Closing the ODE–SDE gap in score-based diffusion models through the Fokker–Planck equation. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, 383(2298), 20240503. Gonzalez, M., et al. (2023). Seeds: Exponential sde solvers for fast high-quality sampling from diffusion models. *Advances in Neural Information Processing Systems*, 36, 68061-68120. Manshausen, P., et al. (2026). Towards accurate extreme event likelihoods from diffusion model climate emulators. arXiv:2605.03802. Xu, Y., et al. (2023). Restart sampling for improving generative processes. *Advances in Neural Information Processing Systems*, 36, 76806-76838

Additionally, we found that the PF-ODE estimator requires computation in true float32 to avoid error accumulation in its log-weight computation (Table R1.3), while our method can leverage the TensorFloat32 format, leading to additional wall-clock gains. At matched variance, i.e. with the number of particles adjusted, PF-ODE needs 10–15× DireSMC's GPU time at 10−310^{-3} and 25–50× at 10−410^{-4}, before accounting for its bias at K=50K = 50. In summary, PF-ODE matches DireSMC's variability at 10−310^{-3} and is within 10% of its own law at K=200K = 200, but at matched accuracy DireSMC is an order of magnitude cheaper on the emulator, and the gap grows with rarity. Here, the time is measured for one H100 GPU.

Table R1.3. PF-ODE under different arithmetic precisions (hot event, p=10−3p = 10^{-3}, seeds 0–7, same initial noise in every row). Ratio: p^\hat p divided by pODE(K)p_{\mathrm{ODE}}(K), with run and reference errors combined in quadrature. Time: GPU-seconds per run. TF32 is the JAX default.

\text{Arithmetic} & K & \hat p\\,/\\,\text{own law} & \text{CV} & \text{Time (s)} \\\\ \hline \text{TF32 (JAX default)} & 50 & \mathbf{37 \pm 7} & 0.56 & 821 \\\\ \text{True float32} & 50 & 1.55 \pm 0.07 & 0.05 & 1591 \\\\ \text{Float64} & 50 & 1.56 \pm 0.06 & 0.04 & 5264 \\\\ \hline \text{TF32 (JAX default)} & 200 & \mathbf{3.1 \pm 0.3} & 0.31 & 3287 \\\\ \text{True float32} & 200 & 1.10 \pm 0.04 & 0.07 & 6372 \end{array}$$

Comments on the paper

2 comments

We thank the reviewers for their thoughtful and measured reviews. We are pleased that the reviewers thought our method "combine(s) several of the available tools in the literature in an elegant way" (W2QA), makes a "contribution relevant to scientific applications" (UPAG), and found the theory "well developed" (5DYV).

In our individual responses below, we further clarify the contributions of our work compared to existing work and provide additional theoretical and empirical results which we think will strengthen the paper further. If the reviewers think there are any outstanding points to discuss, we are happy to address them until the end of the discussion period.

To summarise, we (i) updated the climate experiments to use the same thresholds as the reference pool; (ii) provided experiments that show the PF-ODE variant of Manshausen et al. (2026) on the climate model costs 10× more GPU wall-clock time for the same event, and at a 10−410^{-4} event requires 25–50× more compute to match the variance; (iii) showed that our choice of weight computation based on Twisted Diffusion Sampling outperforms, theoretically and experimentally, the Feynman–Kac corrector variant of Skreta et al. (2025) that is utilised in Xie et al. (2026); and (iv) extended the variance analysis of Theorem 2 to the estimator with resampling.

References

Manshausen, P., et al. (2026). Towards accurate extreme event likelihoods from diffusion model climate emulators. arXiv:2605.03802.

Skreta, M., et al. (2025). Feynman–Kac correctors in diffusion: Annealing, guidance, and product of experts. arXiv:2503.02819.

Wu, L., et al. (2023). Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems, 36, 31372-31403.

Xie, Y., et al. (2026). Enhanced diffusion sampling: Efficient rare event sampling and free energy calculation with diffusion models. The Journal of Chemical Physics, 165.16. arXiv:2602.16634.

We thank all reviewers for their detailed feedback and for raising their scores to recommend acceptance. In light of the discussions, we will incorporate the reviewers' feedback to improve the paper’s clarity, main contributions, and its theoretical and empirical robustness. A revised manuscript will address the prose, incorporate all new comparisons against existing methods, as well as the discussed quantitative and qualitative analyses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.