acceptodds
Under review as a conference paper at ICLR 2027

Robust Ensemble Guidance for Scientific Inverse Problems

Abstract

Ensemble guidance combines pretrained diffusion priors with black-box forward models to solve inverse problems without differentiating through the physical simulator. However, observation coordinates with large predictive spread or extreme residuals can dominate the ensemble correction, degrading reconstruction accuracy. We show that two simple modifications, weighting and clipping, substantially improve this correction. Our method, Robust Ensemble Guidance (REG), uses ensemble predictive spread to balance observation scales and adaptively clips standardized residuals to limit the influence of extreme discrepancies. Both operations reuse existing particles and forward predictions, requiring no additional denoiser or forward-model evaluations. Under a local linear Gaussian model, we derive conditions for reduced one-step estimation risk, bound the influence of individual observation coordinates, and characterize when these benefits persist with finite ensembles. Experiments on Navier–Stokes inversion, black-hole imaging, and acoustic full-waveform inversion demonstrate improved reconstruction over the underlying ensemble solver. In particular, REG **increases black-hole reconstruction PSNR by 6.2–8.2 dB** across three observation regimes and **reduces Navier–Stokes reconstruction error by 26.4%** in a matched-budget comparison. These findings highlight the importance of observation heterogeneity and residual influence in designing reliable generative solvers for scientific inverse problems.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Inverse problem, Diffusion models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_pjbDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Robust Ensemble Guidance (REG) for scientific inverse problems with pretrained diffusion-based priors. The method modifies ensemble Kalman guidance using predictive-spread weighting and residual clipping, with the goal of improving robustness to heterogeneous observation scales and large residuals. Both local theoretical analysis and numerical experiments on Navier--Stokes, black-hole imaging, and full-waveform inversion are provided to justify the effectiveness of proposed methodology.

Critical strengths

The method is simple, computationally inexpensive, and easy to incorporate into existing ensemble-guidance algorithms. The experiments also quite comprehensive by covering several challenging scientific inverse problems.

Critical weaknesses

The reviewer's main concern is that the methodological novelty relative to standard normalization/preconditioning and robust ensemble-Kalman methods is somewhat limited. In addition, the theoretical results are mainly local, while the implemented algorithm operates in a nonlinear iterative regime, so the connection between the theory and the observed empirical gains could be explained more clearly.

The empirical improvements are also somewhat uneven across settings. For example, the reported Navier--Stokes gain varies substantially with ensemble size, and some of the measurement-consistency claims in the black-hole experiment do not appear uniformly stronger than competing methods.

Finally, the manuscript quality could be further enhanced by expanding the discussion on related work. Ensemble-based sampling and reweighting ideas have also recently been studied from the perspective of inference/test-time scaling in diffusion models [1-7]. It would also be useful to clarify how REG relates to this broader family of ensemble-based inference algorithms and compare their performance in practice if possible.

Questions for authors
  1. Would it be possible for the authors to clarify the main conceptual novelty of predictive-spread weighting and residual clipping relative to existing robust/regularized ensemble Kalman methods?

  2. How sensitive is REG to the clipping quantile and ensemble size?

References:

[1] Singhal, R., Horvitz, Z., Teehan, R., Ren, M., Yu, Z., McKeown, K., & Ranganath, R. (2025). A general framework for inference-time scaling and steering of diffusion models. arXiv preprint arXiv:2501.06848.

[2] Skreta, M., Akhound-Sadegh, T., Ohanesian, V., Bondesan, R., Aspuru-Guzik, A., Doucet, A., ... & Neklyudov, K. (2025). Feynman-kac correctors in diffusion: Annealing, guidance, and product of experts. arXiv preprint arXiv:2503.02819.

[3] Uehara, M., Zhao, Y., Wang, C., Li, X., Regev, A., Levine, S., & Biancalani, T. (2025). Inference-time alignment in diffusion models with reward-guided generation: Tutorial and review. arXiv preprint arXiv:2501.09685.

[4] He, J., Hernández-Lobato, J. M., Du, Y., & Vargas, F. (2025). Rne: a plug-and-play framework for diffusion density estimation and inference-time control. arXiv e-prints, arXiv-2506.

[5] Wang, C., Wang, W., Ren, Y., Blanchet, J., & Lu, Y. (2026, May). Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures. In Forty-third International Conference on Machine Learning.

[6] Ren, Y., Gao, W., Ying, L., Rotskoff, G., & Han, J. (2026, April). Driftlite: Lightweight drift control for inference-time scaling of diffusion models. In International Conference on Learning Representations (Vol. 2026, pp. 76664-76702).

[7] Chen, H., Ren, Y., Min, M. R., Ying, L., & Izzo, Z. (2025). Solving inverse problems via diffusion-based priors: An approximation-free ensemble sampling approach. arXiv preprint arXiv:2506.03979.

4 comments
AuthorsOpenReview ↗

Response to Reviewer pjbD (Part 1)

We sincerely thank the reviewer for the constructive comments and helpful references. We address your concerns and questions below:

Q1. Conceptual novelty relative to existing robust/regularized ensemble Kalman methods.

REG starts from an analysis of why the diffusion counterpart of ensemble Kalman inversion fails with finite ensembles, and each component is derived from that analysis. Relative to existing ensemble Kalman methods, the conceptual differences are as follows:

  • Regularized EKI (Iglesias, 2016; Iglesias & Yang, 2021) controls the global step size via regularization parameters or early stopping, leaving the relative weighting across observation coordinates unchanged. REG specifically targets this coordinate-wise imbalance. Because EnKG replaces the inverse innovation covariance in the Kalman gain with a scalar, coordinates with large predictive spread dominate the update (Eq. 8). Since this inverse cannot be computed when J≪mJ \ll m, REG restores its diagonal, which can be estimated reliably from the finite ensemble.
  • Robust EnKF (Roh et al., 2013) clips innovations at fixed thresholds calibrated from a known observation-error covariance, and adaptive quality control (Sakov & Sandery, 2017) inflates observation errors using a fixed scalar. In diffusion guidance, however, the dominant error originates from the denoiser, lacks a known covariance, and varies in scale across reverse steps. REG therefore derives its clipping threshold at every step as a quantile of the standardized residuals, ensuring scale equivariance (App. A.2). Crucially, REG clips the residuals of each particle individually, yielding a robust ensemble mean correction (Eqs. 19–20) rather than a simple affine transformation of the ensemble.
  • Guarantees for the diffusion setting. Our analysis directly addresses the central question: when does a spread-informed geometry provably outperform the identity geometry of EnKG with finite ensembles? We prove when weighting lowers single-step risk and when this advantage persists at rate O(log⁡m/J)O(\sqrt{\log m / J}) (Eq. 14; App. A.4), while bounding individual residual influence (Eqs. 18, 20).

In summary, REG is a Huber-robust, Jacobi-preconditioned ensemble Kalman update whose weights and threshold are designed for, and analyzed in, the finite-ensemble diffusion setting. We have added this comparison to Section 2 (Robust estimation, in red).

Q2. Sensitivity to the clipping quantile and the ensemble size.

The gain of REG over EnKG is consistent across both hyperparameters. We ran new experiments while keeping all other settings as in the paper:

  • Quantile qq (added as Section 4.5 and Table 5, in red). For every q∈{0.5,0.75,0.9}q\in\{0.5,0.75,0.9\}, REG improves the mean over EnKG on all three tasks—reducing relative error by 33–40% in Navier–Stokes and 8–11% in FWI, and increasing PSNR by 5.8–9.4 dB in black-hole imaging.
  • Ensemble size JJ. In black-hole imaging (JJ from 64 to 512), REG wins in 36 of 40 cases with mean PSNR gains of 2.7–8.5 dB. In FWI (JJ from 32 to 128), it wins in 14 of 15 cases with 9–11% lower relative error. REG's own accuracy is stable: over an 8×8\times range of JJ, its mean PSNR varies by at most 1.7 dB, whereas that of EnKG varies by more than 6 dB.
JJBH 3%: EnKG / REGBH 10%: EnKG / REGJJFWI: EnKG / REG
6419.7 / 26.416.5 / 24.2320.184 / 0.164
12819.1 / 26.722.7 / 25.4640.180 / 0.162
25621.8 / 27.018.9 / 25.71280.175 / 0.159
51221.6 / 27.417.4 / 25.9

We have added the qq sweep to Section 4.5 and Table 5 of the revised manuscript (in red).

AuthorsOpenReview ↗

Response to Reviewer pjbD (Part 2)

W1. Methodological novelty and connection between the local theory and empirical gains.

We address the methodological novelty relative to normalization/preconditioning and robust EnKF in Q1 above. Regarding how the local theory connects to the empirical gains of the nonlinear iterative algorithm, two pieces of evidence bridge this gap:

  • Fig. 2(c) evaluates the one-step criterion (Eq. 14) on 120 problems: the analytic and Monte Carlo risk gains correlate at r=0.993r=0.993 and agree in sign in 97.5% of cases.
  • The experiments match the mechanisms the theory predicts. In the ablation (Table 2), weighting and clipping each improve over EnKG. In the new contamination test (Section 4.6 and Table 6, in red), clipping caps the effect of corrupted observations.

We also extend the analysis beyond a single step:

  • Under the same local model, with the weights and threshold held fixed, repeated REG updates are preconditioned gradient descent on a convex Huber loss. The loss therefore decreases monotonically, and the ensemble converges to the Huber estimate. The deviation caused by clipping stays within the data-informed subspace and is bounded uniformly in the number of steps.
  • When the linearization varies along the trajectory, the per-step errors add up linearly rather than multiplying, provided each step is non-expansive up to a summable factor.

A global guarantee for the nonlinear sampler is not available for any polynomial-time diffusion posterior sampler (Gupta et al., 2024). Moreover, Kalman-type updates are not posterior-consistent for nonlinear models (Ernst et al., 2015).

We have added the contamination test to Section 4.6 and Table 6 of the revised manuscript (in red).

W2. Uneven gains across settings.

(a) Dependence on JJ in Navier–Stokes. REG improves over EnKG at every tested JJ in Table 2. The size of the gain varies because the two methods are limited by different factors:

  • At small JJ, REG's weights are themselves estimated from few samples, and Eq. (14) predicts a smaller advantage.
  • At large JJ, particle-based methods generally become more accurate.

Our method reduces the standard deviation of the results, as shown in Table 1. Also, Q2 shows that the gain is consistent across JJ in black-hole imaging and FWI.

(b) Measurement consistency in black-hole imaging. We agree that REG does not attain the lowest χ~2\tilde{\chi}^2 among all methods. Two observations put the χ~2\tilde{\chi}^2 results in context:

  • χ~2\tilde{\chi}^2 is not aligned with image fidelity in this problem. It measures agreement with the closure data, with an ideal value near 1. Because closure quantities lose the absolute position and total flux, many images fit the data equally well. For example, at 3% observation time, SMILI attains the closure-phase χ~2\tilde{\chi}^2 closest to 1 (1.48) but the lowest PSNR (18.51 dB). REG attains the highest PSNR (26.97 dB) with a closure-phase χ~2\tilde{\chi}^2 of 4.28.
  • Relative to EnKG, REG improves both χ~2\tilde{\chi}^2 values by factors of 3–70. It also reduces their case-to-case spread by up to two orders of magnitude, which removes the divergent cases of EnKG.

W3. Relation to ensemble-based inference-time scaling.

Thank you for these references. We have added a paragraph to Section 2 (Particle-based inference-time scaling, in red). The key difference lies in the correction mechanism:

  • These methods correct the ensemble by reweighting and resampling. Their accuracy therefore relies on importance weights, which degenerate when the likelihood is sharp and high-dimensional (Snyder et al., 2008). This is the typical regime of scientific inverse problems with many precise measurements.
  • REG instead transports every particle along an ensemble-estimated direction, so the ensemble cannot collapse onto a few particles. It also addresses observation-scale imbalance and residual dominance, two failure modes specific to transport-based corrections.

We have added this discussion to Section 2 (Particle-based inference-time scaling) of the revised manuscript (in red).

We hope these clarifications and the newly added experiments address your concerns, and we welcome any further discussion.

Reviewer_pjbDOpenReview ↗

Response

The reviewer appreciates the authors' detailed response, which has satisfactorily addressed most of the concerns raised in the original review. Consequently, the reviewer would be glad to raise the rating from 2 to 3.

Thank you very much for your approval. Thanks again for your insightful comments and suggestions.

Reviewer_fNc7OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a simple but useful method to inprove scientific inverse problems including black-hole reconstruction, acoustic full-waveform inversion and Navier–Stokes reconstruction. This paper focuses on the problem that observation coordinates with large predictive spread or extreme residuals can dominate the ensemble correction, degrading reconstruction accuracy. The authors propose REG (Robust Ensemble Guidance), uses ensemble predictive spread to balance observation scales and adaptively clips standardized residuals to limit the influence of extreme discrepancies. The proposed method shows significant improvement over baseline methods. The authors claim that these findings highlight the importance of observation heterogeneity and residual influence in designing reliable generative solvers for scientific inverse problems.

Critical strengths

1.The proposed method is simple but effective. It brings valuable improvement on scientific reconstruction tasks. In particular, REG increases black-hole reconstruction PSNR by 6.2–8.2 dB across three observation regimes and reduces Navier–Stokes reconstruction error by 26.4% in a matched-budget comparison. This is a good achivement.

  1. The method does not neet to retrain the diffusion model, and does not need additional forward evaluations.
Critical weaknesses
  1. The proposed method mainly combines predictive-spread weighting and residual clipping, both are closely related to established techniques in numerical optimization and robust statistics. Although the integration method is useful, the paper does not sufficiently demonstrate what distinguishes REG from the existing techniques.
Questions for authors
  1. What is the main novelty of REG beyond combining existing weighting and clipping techniques?
  2. How sensitive is REG to the clipping quantile? Are different values used across tasks? Why?
  3. Can the authors provide more evidence that REG improves robustness, rather than only reconstruction accuracy?

If your response solve my concerns, I would like to reconsider my rating.

4 comments
AuthorsOpenReview ↗

Response to Reviewer fNc7 (Part 1)

We sincerely thank the reviewer for the insightful questions. We address your concerns and questions below:

Q1&W1. Novelty & Distinction beyond Combining Weighting and Clipping.

Rather than a combination of existing techniques, REG is the direct outcome of analyzing why derivative-free ensemble guidance fails. Our analysis identifies two fundamental failure modes, derives the minimal targeted correction required to eliminate each, and proves when these corrections provably reduce estimation risk. The simplicity of the final algorithm is a consequence of this theoretical diagnosis rather than its starting point.

  1. Analysis: Where ensemble correction breaks down. Examining the EnKG update reveals two core failure modes:

    • Coordinates with large predictive spread dominate the update: with a scale ratio LL, the stable step range shrinks to α<2/(1+L2)\alpha < 2 / (1 + L^2), while perturbations are amplified by a factor of LL (Eq. 8).
    • Because the update is strictly linear in residuals, extreme residual values can exert unbounded leverage and dominate particle coupling.
  2. Derivation: what removes each failure mode. Comparing the EnKG update with the standard Kalman update reveals the source of the first failure: EnKG drops the inverse innovation covariance in the Kalman gain. While the full inverse cannot be formed when J≪mJ \ll m, its diagonal can be reliably estimated from the ensemble. Restoring this diagonal yields our predictive-spread weighting. The second failure is eliminated by bounding the influence of each residual, corresponding to one reweighting step of a Huber-robust Kalman update. As now discussed in Section 2 (Robust estimation, in red), classical robust ensemble Kalman filters (Roh et al., 2013) calibrate fixed thresholds from a known observation-error covariance. In diffusion guidance, however, the dominant error comes from the denoiser and has an unknown, step-varying covariance, so REG estimates both the weights and the threshold from the ensemble at every step.

In short, our contribution lies in diagnosing the failure modes of derivative-free guidance and proving that these targeted remedies suffice. The resulting simplicity is precisely what enables REG to add zero extra model evaluations and be seamlessly adopted in practice.

We have added this comparison with robust ensemble Kalman filters to Section 2 (Robust estimation) of the revised manuscript (in red).

Q2. Sensitivity to qq and Parameter Choices Across Tasks.

We ran a new sweep over q∈{0.5,0.75,0.9}q\in\{0.5,0.75,0.9\} across all three tasks and added the full results to the revised manuscript as Section 4.5 and Table 5 (in red). For every qq, REG improves the mean over EnKG on all tasks—reducing relative error by 33–40% in Navier–Stokes and 8–11% in FWI, and increasing PSNR by 5.8–9.4 dB in black-hole imaging. The variation across qq is small compared with the gain over EnKG: at most 0.017 versus at least 0.075 in Navier–Stokes relative error, 0.005 versus 0.014 in FWI, and 1.6 dB versus 5.8 dB in black-hole PSNR. Notably, even when the default q=0.75q=0.75 is not the task-specific optimum, a single default works reliably across all tasks without per-task tuning.

We have added the full results to Section 4.5 and Table 5 of the revised manuscript (in red).

AuthorsOpenReview ↗

Response to Reviewer fNc7 (Part 2)

Q3. Robustness of REG.

We provide two complementary pieces of evidence showing that REG improves robustness—both against external observation corruption and by eliminating catastrophic reconstruction failures across individual cases.

(1) Robustness to corrupted observations (added as Section 4.6 and Table 6, in red). To test clipping directly under outliers, we follow the settings of Section 4.5 with the default q=0.75q=0.75 (evaluating black-hole imaging at the 10% observation time across 5 cases). In each observation block, we corrupt a random fraction p∈p \in {5%, 20%} of the entries with additive Gaussian noise matching the clean block's standard deviation. Both methods retain the original noise model and are evaluated against the clean ground truth. As shown in Table 6, 5% contamination increases the Navier–Stokes error of EnKG by 44% (from 0.229 to 0.329), whereas REG increases by only 4% (from 0.146 to 0.152). In black-hole imaging, REG changes by less than 0.1 dB across all contamination levels (25.67, 25.58, and 25.64 dB at p=0,5%,20%p=0, 5\%, 20\%), whereas EnKG degrades from 18.94 dB to 15.89 dB at p=20%p=20\% and fails on one or two test cases at every level. This directly reflects the bounded influence of clipped residuals (Eqs. 18, 20).

(2) Eliminating catastrophic failures at the individual-case level. Even without external outliers, REG improves not only mean accuracy but also worst-case reliability by removing the divergent trajectories of EnKG. The table below summarizes black-hole imaging across all three observation times (following the settings of Section 4.5 / Table 5):

MethodCases improved over EnKG ↑PSNR < 15 dB (Failures) ↓PSNR < 20 dB ↓Worst-case PSNR (dB) ↑
EnKG–10/30 (33%)17/30 (57%)8.3
REG, q=0.5q=0.527/30 (90%)0/30 (0%)4/30 (13%)16.5
REG, q=0.75q=0.75 (default)27/30 (90%)0/30 (0%)1/30 (3%)19.7
REG, q=0.9q=0.927/30 (90%)0/30 (0%)0/30 (0%)21.7

For every q∈{0.5,0.75,0.9}q\in\{0.5, 0.75, 0.9\}, REG attains a higher PSNR than EnKG in 27 of 30 cases (90%). More importantly, EnKG suffers severe failures (PSNR <15< 15 dB, dropping as low as 8.3 dB) in 10 of 30 cases (33%). REG eliminates all such failures across every qq. At the default q=0.75q=0.75, its worst-case PSNR is raised from 8.3 dB to 19.7 dB. Consistent with the 1–2 orders of magnitude reduction in χ~2\tilde{\chi}^2 standard deviation in Table 3, this confirms that REG substantially lowers the risk of failed reconstructions rather than merely shifting the mean.

We have added the contamination experiment in (1) to Section 4.6 and Table 6 of the revised manuscript (in red).

We hope these clarifications and the newly added experiments address your concerns, and we welcome any further discussion.

Thank you for your responses. Your clarification, revisions, and additional experiments have addressed my main concerns. I would like to raise my rating to 3.

Good luck!

Thank you for raising your score and supporting our work! We are delighted that our rebuttal have successfully resolved your previous concerns.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to Reviewer NVoN (Part 1)

We sincerely thank the reviewer for the positive review. Below we address each of the concerns in detail.

W1. Novelty clarification.

Our contribution lies not in these classical operations themselves, but in diagnosing where EnKG fails, deriving the targeted corrections from the Kalman framework, and establishing finite-ensemble guarantees:

  • Kalman derivation. EnKG updates each particle via xj+=xj+γ Cxyrjx_j^+=x_j+\gamma\,C^{xy}r_j, whereas the standard ensemble Kalman update is xj+=xj+Cxy(Cyy+Γ)−1rjx_j^+=x_j+C^{xy}(C^{yy}+\Gamma)^{-1}r_j. Dropping the innovation precision (Cyy+Γ)−1(C^{yy}+\Gamma)^{-1} discards the observation geometry, directly causing the scale instability in Eq. (8). Because rank(Cyy)≤J−1≪m\mathrm{rank}(C^{yy})\le J-1\ll m, the full inverse cannot be formed; REG restores its diagonal, which coincides with the normalized diagonal Kalman precision when predictive spread dominates observation noise. Furthermore, writing ψδ(u)=ω(u) u\psi_\delta(u)=\omega(u)\,u with ω(u)=min⁡(1,δ/∣u∣)\omega(u)=\min(1,\delta/\vert{}u\vert{}) casts the REG update into the exact Kalman form with gain γ CxyWΩjW\gamma\,C^{xy}W\Omega_jW (Ωj=diag(ω(ujk))\Omega_j=\mathrm{diag}(\omega(u_{jk}))). REG is thus a Huber-robust, Jacobi-preconditioned ensemble Kalman update.
  • Distinction from robust EnKF (added to Section 2, Robust estimation, in red). Classical robust EnKF (Roh et al., 2013) clips the mean innovation at fixed per-coordinate thresholds calibrated from a known noise covariance and applies a shared gain to all particles. In diffusion guidance, however, the dominant error originates from the denoiser, lacks a known covariance, and varies in scale across reverse steps. REG therefore (i) adapts its threshold at every step as a quantile of the standardized residuals, ensuring scale equivariance (App. A.2), and (ii) clips each particle's residuals individually, yielding a robust ensemble mean correction (Eqs. 19–20) rather than a simple affine transformation of the ensemble.
  • Theoretical guarantees. We prove when the spread-informed geometry lowers single-step risk relative to EnKG's identity geometry and when this advantage persists at rate O(log⁡m/J)O(\sqrt{\log m/J}) with finite ensembles (Eq. 14; App. A.4), while bounding individual residual influence (Eqs. 18, 20). Multi-step convergence is addressed in W3 below.

We have added this distinction from robust EnKF to Section 2 (Robust estimation) of the revised manuscript (in red).

W2. Computational cost.

We agree that ensemble methods require more forward evaluations than gradient-based (DPS-type) guidance. While DPS-type methods (DPS, LGD, DiffPIR, RED-diff, DAPS, PnP-DM) use one forward–adjoint pair per step, an ensemble method evaluates the forward model across JJ particles per step. However, this cost is inherent to derivative-free particle methods—shared by EnKF/EKI (Evensen, 2003; Iglesias et al., 2013) and SMC diffusion samplers (Wu et al., 2023; Dou & Song, 2024; Section 2, Particle-based inference-time scaling, in red), rather than specific to REG.

This forward-evaluation computational cost brings critical capabilities that gradient-based methods lack:

  • Derivative-free access. Only forward calls are required, enabling direct use of black-box, non-differentiable, or legacy simulators without deriving or maintaining adjoint code (Iglesias et al., 2013; Kovachki & Stuart, 2019).
  • Stable sensitivities in chaotic dynamics. For turbulent or long-horizon flows, adjoint gradients diverge exponentially with trajectory length (Lea et al., 2000; Wang et al., 2014), whereas ensemble differences provide finite-amplitude, statistically averaged sensitivities.
  • Batch parallelism. All JJ evaluations within a reverse step are completely independent and execute in parallel batches on GPU.
  • Uncertainty statistics. The ensemble directly supplies predictive spreads and cross-covariances at no extra cost, forming the basis of REG's preconditioning.
  • Compute-scaling accuracy. Reconstruction quality scales smoothly with particle count (Table 2), matching recent inference-time scaling findings (Wu et al., 2023; Singhal et al., 2025).

We have added the discussion of particle-based samplers to Section 2 (Particle-based inference-time scaling) of the revised manuscript (in red).

AuthorsOpenReview ↗

Response to Reviewer NVoN (Part 2)

W3. Single-step theory and multi-step sampling.

The accumulation of clipping bias over steps does not push samples away from the posterior.

(1) Clipping introduces no new error directions. At every step tt, both the clipped REG correction γtCtxyWt ψδt(Wtrj)\gamma_t C^{xy}_t W_t\,\psi_{\delta_t}(W_t r_j) and its unclipped counterpart γtCtxyWt2rj\gamma_t C^{xy}_t W_t^2 r_j lie strictly within range(Ctxy)\mathrm{range}(C^{xy}_t). Their difference—the clipping bias—therefore remains confined to the exact same data-informed subspace on which the ensemble correction already acts, never leaking into unconstrained prior directions.

(2) The bias remains uniformly bounded rather than accumulating. Under the local linear model with fixed weights and threshold, KK repeated REG updates correspond to preconditioned gradient descent on the convex Huber loss Lδ(x)=∑kρδ(wk(yk−hk⊤x))L_\delta(x)=\sum_k \rho_\delta\big(w_k(y_k-h_k^\top x)\big). Consequently, LδL_\delta decreases monotonically and the ensemble converges to the Huber estimate. The convergence means the iterative dynamics never amplify the clipping effect.

(3) Linear error addition under time-varying linearizations. When Ct,Ht,Wt,δtC_t, H_t, W_t, \delta_t and the prior step vary across reverse steps tt, the per-step errors (including the clipping bias) accumulate linearly rather than multiplicatively, provided each step is non-expansive up to a summable factor.

Why global nonlinear posterior guarantees are unattainable. A global guarantee of exact posterior recovery for nonlinear samplers is out of reach for any polynomial-time diffusion posterior sampler: posterior sampling is computationally intractable in general even with exact scores (Gupta et al., ICML 2024); guidance methods do not sample their nominal targets even in simple settings (Chidambaram et al., NeurIPS 2024). Kalman-type updates are not posterior-consistent for nonlinear models even as J→∞J\to\infty (Ernst et al., SIAM/ASA JUQ 2015). And multi-step EKI convergence holds only in linear or near-linear regimes (Schillings & Stuart, 2017; Weissmann et al., 2022). Our local analysis aligns with this established literature.

AuthorsOpenReview ↗

Response to Reviewer NVoN (Part 3)

W4. Sensitivity to qq and no need for manual grid searches.

  • Clipping reduces extreme residuals rather than discarding them. Setting q=0.75q=0.75 does not remove 25% of the residual components. Instead, standardized residuals exceeding the qq-quantile are capped at δt\delta_t while preserving their signs (Eqs. 15, 17). All coordinates remain active and contribute to the update with magnitude δt\delta_t, simply preventing the largest 25% from dominating the particle coupling.
  • Robust improvements across a wide range of qq eliminate per-task grid searches (added as Section 4.5 and Table 5, in red). To test sensitivity directly, we ran a new sweep over a wide range q∈{0.5,0.75,0.9}q\in\{0.5, 0.75, 0.9\} across all three benchmarks while keeping all other settings fixed (Section 4.5, Table 5). For every qq, REG consistently improves over EnKG on all tasks—reducing relative error by 33–40% in Navier–Stokes and 8–11% in FWI, and increasing PSNR by 5.8–9.4 dB in black-hole imaging. Crucially, the variation across qq is minor compared with the gain over EnKG: at most 0.017 vs. at least 0.075 in Navier–Stokes relative error, 0.005 vs. 0.014 in FWI, and 1.6 dB vs. 5.8 dB in black-hole PSNR. Because REG yields consistent gains across such a wide range of qq, it is convenient to use in practice without per-task tuning.

We have added the full qq sweep to Section 4.5 and Table 5 of the revised manuscript (in red).

W5. Performance, data misfit, and applicability in FWI.

  • Superior velocity recovery over all baselines. In FWI, the primary objective is to reconstruct the subsurface velocity map. In Table 4, REG achieves the lowest relative ℓ2\ell_2 error (0.160) and highest SSIM (0.574) among all compared methods. Notably, it outperforms every adjoint-based baseline, including DiffPIR (0.204), DAPS (0.201), LGD (0.244), and DPS (0.250). Relative to EnKG, REG also improves accuracy consistently: it wins in all 5 cases at the default setting, 14 of 15 cases across J∈{32,64,128}J\in\{32,64,128\}, and all 15 cases across q∈{0.5,0.75,0.9}q\in\{0.5,0.75,0.9\} (Table 5).
  • Why REG and EnKG have similar data misfit (~268) yet outperform DiffPIR (88.53). At each step, an ensemble method updates particles within the (J−1)(J-1)-dimensional subspace spanned by the J=64J=64 observation anomalies. Because EnKG and REG share this low-dimensional subspace, their raw seismic misfits are naturally close (269.6 vs. 268.9). Within this subspace, however, REG's weighting and clipping prevent extreme trace discrepancies from distorting the update, yielding a much better velocity model. By contrast, DiffPIR uses full-space adjoint gradients to drive the waveform misfit down to 88.53. In this ill-posed inverse problem, aggressively fitting the seismic traces overfits local waveform mismatches and degrades the recovered velocity map (0.204 error vs. REG's 0.160).
  • Strong applicability to complex, cycle-skipping wave problems. Severe multiple scattering and cycle skipping create sharp local minima in the waveform misfit, which often trap point-wise adjoint gradients. Ensemble differences instead capture non-local, statistically averaged sensitivities across particles. Combined with residual clipping to cap cycle-skipped phase errors, REG provides a highly effective and widely applicable solution for complex nonlinear wave inversion.

The qq sweep referenced above is reported in Section 4.5 and Table 5 of the revised manuscript (in red).

We hope these clarifications, multi-step analysis, and new sensitivity and robustness experiments address your concerns, and we welcome any further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.