acceptodds
Under review as a conference paper at ICLR 2027

Schrödinger Bridge Data Assimilation with A Consistent Probabilistic View

Abstract

Data assimilation (DA), which estimates the current atmospheric state by combining prior forecasts with sparse observations, is critical to the accuracy of modern weather prediction systems. The recently developed diffusion DA methods exhibit several superior aspects in comparison to the classical ones. Although DA targets a state distribution conditioned on both the background and observations, they learn a background-conditioned distribution during training and incorporate observations only during inference, separating observation conditioning from generative learning. To address this limitation, we propose Schrödinger Bridge Data Assimilation (SBDA), which integrates classical 3D-Var with diffusion Schrödinger bridges through an observation-enhanced reparameterization of DA conditioning, providing a consistent probabilistic treatment of observations across bridge learning and sampling. The resulting bridge generates analyses directly from observation-enhanced background states rather than Gaussian noise. We evaluate SBDA using background forecasts from a deep learning weather model, ERA5 reference fields, and simulated observations at 1% of the ERA5 grid points. SBDA achieves lower single-step assimilation errors than the evaluated baselines on most variables, with a mean relative reduction of 12.9% across all variables. More importantly, SBDA consistently improves downstream forecast across all variables, yielding a forecast time gain of up to 13.3 h relative to the strongest evaluated baseline. Furthermore, SBDA retains substantial assimilation gains at unseen observation locations without retraining.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Data assimilation, Diffusion Schrödinger bridge, Machine learning weather prediction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JfN9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Schrödinger Bridge Data Assimilation (SBDA), a diffusion-bridge-based method for single-step weather data assimilation. First, authors propose bridge learning which trains the model to learn a generative bridge between OI enhanced background x^b\hat x_b and the reference x∗x^*. Then, the bridge sampling is proposed to refine the final output using the given observation. Evaluated on the ERA5 dataset using 48-hour forecasts as background and observations at 1% grid points, the proposed method demonstrates improvements in both assimilation accuracy and downstream forecast performance. The authors also provide ablation studies on observation and evaluate zero-shot performance at unseen observation locations.

Critical strengths
  • Simple and Flexible Framework: The proposed method combines OI-based observation updates with diffusion-bridge reconstruction in a straightforward framework. The same trained model supports both deterministic inference and stochastic ensemble generation.

  • Broad Empirical Evaluation: The paper evaluates on various setting, including single-step assimilation accuracy, downstream forecasts and zero-shot generalization.

  • Informative Ablations and Controlled Comparisons: The ablation studies support the contributions of each component of the proposed method. Furthermore, the comparison with DiffDA-OI conditioned on the same OI-enhanced background provides empirical evidence for the benefits of the proposed bridge-based approach.

Critical weaknesses
  • Limited Methodological Novelty: The proposed method seems to simply combine existing components for new domain and frozen model, using the paired bridge learning of I2SB, classical OI, and a per-step data-consistency correction commonly used in existing works. Also, the motivation of using bridge model rather than the other paired transport model is not clarified.

  • Limited Comparison with Related Works: The experiments compare SBDA only with 3D-Var and a re-implemented DiffDA, although the paper discusses several closely related approaches, such as score-based filters (SF/EnSF), related bridge models (PalSB, PICSB) and comparison with the other types of transport models. It is recommended to add more comparisons to clarify the contribution.

Questions for authors
  1. Why is the Schrödinger bridge formulation preferred over other paired transport models, such as conditional flow matching or stochastic interpolants with the same OI-enhanced endpoints?

  2. For experimental setting, could the authors clarify the reason for noiseless observation setting? Also, it is recommended to provide the result for noisy observation setting.

  3. The results in Table 2 shows that central 80% ensemble intervals are under-covered across all variables. This suggests that the generated samples do not faithfully represent the target conditional distribution. Could authors provide any discussion or analysis on this result?

5 comments

Q1: What is SBDA’s methodological contribution, and why choose a bridge model?

A1: Our contribution starts from identifying two design limitations in the diffusion-based DA methods considered here: observations enter only during inference, and analysis generation starts from Gaussian noise despite the availability of a structured background. To address these limitations, we propose a flexible framework with a consistent probabilistic view, based on a DA-target-preserving reparameterization. This framework incorporates observations into both bridge learning and sampling and generates analyses directly from observation-enhanced backgrounds. Additionally, the reparameterization is not restricted to OI: any transformation invertible in xb\mathbf{x}^b for fixed y\mathbf y preserves the DA conditioning.

Our choice of a bridge model is motivated by two requirements: initialization from a structured background and ensemble generation from a fixed input. A deterministic flow ODE from a fixed OI endpoint, without extra randomness, yields one trajectory; our stochastic bridge supports ensembles.

The paired I2SB construction [1] we adopt can also be viewed within the stochastic-interpolant framework [2]: it defines a stochastic interpolation between paired endpoints, with interpolation weights and noise scale specified by a Schrödinger bridge with Brownian reference dynamics. This offers one approach to improving upon Gaussian-noise initialization in diffusion DA; comparing alternative paired transports remains future work.


Q2: Could you compare SBDA with more baselines to clarify its contribution?

A2: Thank you for suggesting broader baseline comparisons. We clarify below how the methods you mention differ from our experimental setting.

SF [3]/EnSF [4] require propagated forecast ensembles, whereas our benchmark supplies a single deterministic Aurora background. Their comparison requires an additional protocol for initializing and propagating ensembles, with its associated computational budget. The original EnSF also requires adaptation to sparse observations, as addressed by its subsequent inpainting extension [5]. PalSB [6] and PICSB [7] focus on PDE-constrained field reconstruction, incorporating explicit physics constraints into training; PICSB additionally addresses learning without full-field supervision. Our benchmark assimilates sparse observations into Aurora [8] forecast backgrounds using paired ERA5 supervision, without prescribed PDE-residual operators. Comparing these methods would therefore require additional atmospheric physics formulations and adaptations to their training protocols.

Following Reviewer aM4w’s suggestion, we broadened the comparison by adding GEN_BE 3D-Var [9], L3DVar [10], and VAE-Var [11]. These baselines now appear throughout the main-text analysis and forecast comparisons, including single-step assimilation, case studies, cycling assimilation, and unseen observation layouts.

Following Reviewer qjDM’s suggestion, we also extended the original 48-h downstream forecast evaluation to 240-h cycling assimilation. Each method assimilates every 48 h into its own forecast, then forecasts from its updated analysis.

SBDA vs baselineSingle-step LW-RMSE reductionCycling LW-RMSE reduction
GEN_BE 3D-Var4.03%11.75%
L3DVar9.83%21.74%
VAE-Var4.93%6.54%

Reductions average equally across 69 variables; cycling first averages each variable’s errors over 40 forecast times. Gains persist against the strengthened classical and non-diffusion baselines.

Q3: Why use noise-free observations, and how does SBDA perform with noisy observations?

A3: Noise-free ERA5 observations follow the column-mask protocol of DiffDA [12] and VAE-Var [11]. DiffDA uses simulated observations because it treats observations as ground truth without an observation quality-control procedure, while VAE-Var adopts this setting for comparison with DiffDA. We follow this protocol to compare assimilation methods without introducing additional differences in observation preprocessing and quality control.

To further test SBDA’s generalization to noisy observations, we add independent Gaussian noise with standard deviation 0.1Bii0.1\sqrt{B_{ii}} and variance 0.01Bii0.01B_{ii}. We estimate B\mathbf B using GEN_BE [9], a framework for estimating background-error covariance, from Aurora forecast–ERA5 training pairs. This models relatively accurate but noisy observations, scaled to each variable’s background uncertainty. The SBDA checkpoint and OI parameters remain unchanged, without retraining.

ObservationsMean LW-RMSE reduction vs backgroundImproved variables
Noise-free16.6324%68/69
Noisy16.6277%68/69

Mean relative LW-RMSE changes by only +0.005%, showing good generalization to the tested noisy observations.


Q4: Could you discuss or analyze the undercoverage of the central 80% ensemble intervals across all variables?

A4: The nominal 80% intervals of SBDA’s ensemble predictions are undercovered, with mean coverage of 53.83%. To investigate this undercoverage beyond empirical coverage measurements, we added a linear–Gaussian benchmark with an analytical posterior (200 test cases, 1,000 members each).

MetricAnalytical posteriorw/o ISBDA
Posterior mean error00.1131050.112561
Variance ratio10.63350.6677
80% coverage80%66.94%67.63%
Relative covariance error00.9778910.979311

Undercoverage persists in the controlled linear–Gaussian benchmark, where SBDA’s nominal 80% intervals achieve 67.63% coverage. Removing stepwise OI refinement lowers coverage to 66.94%. These results show that undercoverage persists without weather dynamics or stepwise OI refinement. One possible contributor is the basic DDPM-style reverse sampler: each transition substitutes a single predicted endpoint into a Gaussian bridge kernel with prescribed covariance, potentially underrepresenting the remaining endpoint uncertainty.

To assess relative probabilistic skill, we also compare SBDA with DiffDA using 20-member ensembles at the same 304 analysis times:

MetricDiffDASBDA
Mean relative CRPS reduction vs. DiffDA—13.23%
Variables with lower CRPS than DiffDA—66/69
Mean coverage of nominal 80% intervals43.67%53.83%

SBDA improves both probabilistic accuracy and interval coverage relative to DiffDA, demonstrating the relative benefits of the proposed approach for ensemble prediction.


References

[1] Liu et al. (2023). I²SB: Image-to-Image Schrödinger Bridge.

[2] Albergo et al. (2025). Stochastic Interpolants: A Unifying Framework for Flows and Diffusions.

[3] Bao et al. (2023). A Score-based Nonlinear Filter for Data Assimilation.

[4] Bao et al. (2024). An Ensemble Score Filter for Tracking High-Dimensional Nonlinear Dynamical Systems.

[5] Liang et al. (2025). Ensemble score filter with image inpainting for data assimilation in tracking surface quasi-geostrophic dynamics with partial observations.

[6] Li et al. (2025). Physics-aligned field reconstruction with diffusion bridge.

[7] Bu et al. (2026). Sparsely-Supervised Data Assimilation via Physics-Informed Schrödinger Bridge.

[8] Bodnar et al. (2025). A foundation model for the Earth system.

[9] Descombes et al. (2015). Generalized background error covariance matrix model (GEN_BE v2.0).

[10] Fan et al. (2026). Physically consistent global atmospheric data assimilation with machine learning in latent space.

[11] Xiao et al. (2025). VAE-Var: Variational Autoencoder-Enhanced Variational Methods for Data Assimilation in Meteorology.

[12] Huang et al. (2024). DiffDA: a Diffusion model for weather-scale Data Assimilation.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer JfN9,

Thank you again for your time and thoughtful review. As the discussion period is nearing its end, we would appreciate it if you could take a look at our response to your review and let us know whether any concerns remain. We would be happy to provide further clarification.

Best regards, The Authors

Reviewer_aM4wOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Schrödinger Bridge Data Assimilation (SBDA), which combines an observation-enhanced background endpoint with a paired-data diffusion bridge and observation-based refinement during sampling. An optimal-interpolation (OI) update constructs the enhanced background, and the same update refines the predicted clean state at each reverse step. Experiments on 1.5° ERA5 fields with simulated sparse observations evaluate analysis accuracy, downstream forecasting, generalization to unseen observation locations, and ensemble uncertainty. The reported results are encouraging, but the motivation for the specific bridge construction, the scope of its probabilistic consistency, and the competitiveness of the baseline comparisons require further clarification.

Critical strengths

This paper provides useful validation beyond analysis accuracy. The evaluation includes downstream weather forecasts and previously unseen observation locations, providing evidence that the improvements extend beyond reconstruction under a single observation layout. The ensemble evaluation also provides useful information about uncertainty quality and its limitations. Most previous papers on AI for data assimilation only focused on analysis accuracy but analysis field should be served for forecasting and uncertainty quantification; and I think in this respect this paper has moved a step forward.

Critical weaknesses
  1. The motivation for the bridge formulation and the justification of the complete sampling procedure remain insufficiently established. Generating an analysis from a structured background field is not unique to Schrödinger bridges; other conditional transport or flow-based constructions can share this property. The paper does not sufficiently explain what the particular bridge formulation contributes beyond this capability. Moreover, the exact conditional reparameterization in Eq. (7) does not establish that the subsequent OI-modified sampler targets the intended posterior. The local Gaussian surrogate in Eq. (11), including the reuse of fixed covariance matrices across sampling steps, introduces additional approximations whose effects are not fully characterized. Using observations in both stages is not inherently incorrect, but the connection between the two stages and the claimed probabilistic consistency needs a clearer justification. The existing ablations demonstrate empirical utility within this implementation; they do not establish posterior correctness or rule out independently optimized alternatives.

  2. The baseline comparison is too narrow to support broad conclusions about the method’s advantages. The 3D-Var baseline is a valid OI implementation, but its fixed, isotropic covariance is applied independently to each variable-level channel, omitting cross-variable and vertical error correlations. Its competitiveness and the sensitivity to its covariance parameters are not sufficiently established. Replacing DiffDA’s original GraphCast backbone with the shared U-Net is a useful architectural control, but does not establish performance against the original implementation. More importantly, the learned baselines are limited to diffusion-based approaches. Relevant non-diffusion methods, such as latent-space variational assimilation and VAE-based DA approaches, are missing. Including at least one suitable non-diffusion AI baseline would provide a broader and more convincing assessment.

Questions for authors
  1. Can you justify the complete observation-conditioned sampling procedure and explain the specific role of the bridge formulation? Please specify the target distribution and explain how observation information entering both the enhanced endpoint and the per-step OI refinement is accounted for. In particular, which steps are exact, which rely on Gaussian or other approximations, and what consistency guarantees, if any, apply to the complete algorithm? If exact posterior sampling is not guaranteed, please state the approximation explicitly and provide evidence that the combined updates do not introduce problematic bias or overconfidence.

  2. Do the reported advantages persist against stronger and more diverse baselines? Please include at least one competitive non-diffusion learning-based assimilation method, such as a latent-space or VAE-based approach, and strengthen the 3DVar comparison by incorporating vertical and cross-variable background-error correlations in addition to horizontal correlations. Please explain how the covariance configuration and relevant hyperparameters are estimated or tuned, and use comparable data and a clearly documented computational budget for learned methods. Results showing that the gains persist under these comparisons would substantially strengthen the empirical case and could materially change my assessment.

5 comments

Thank you for the review. Below are our responses to your questions.

Q1: Why use a Schrödinger bridge rather than another conditional transport model?

A1: We require structured-background initialization and ensemble generation from a fixed input. A deterministic flow ODE from a fixed OI endpoint, without extra randomness, yields one trajectory; our stochastic bridge supports ensembles.

The paired I2SB construction [1] we adopt can also be viewed within the stochastic-interpolant framework [2]: it defines a stochastic interpolation between paired endpoints, with interpolation weights and noise scale specified by a Schrödinger bridge with Brownian reference dynamics. This offers one approach to improving upon Gaussian-noise initialization in diffusion DA; comparing alternative paired transports remains future work.


Q2: How are observations accounted for in both the enhanced endpoint and per-step OI refinement? Which parts of the complete algorithm are exact or approximate, what consistency guarantees apply, and what evidence shows that the combined updates do not introduce problematic bias or overconfidence?

A2: Thank you for suggesting that we clarify the sampling approximations. The target is p(x∗∣xb,y)p(\mathbf{x}^*\mid\mathbf{x}^b,\mathbf{y}). The invertible OI reparameterization preserves this conditioning for fixed parameters and R≻0\mathbf R\succ0. The reparameterized target is proportional to p(x∗∣x^b),p(y∣x∗,x^b)p(\mathbf{x}^*\mid\hat{\mathbf{x}}^b)\\,p(\mathbf y\mid\mathbf{x}^*,\hat{\mathbf{x}}^b). Bridge learning models the first factor; the second factor p(y∣x∗,x^b)p(\mathbf y\mid\mathbf{x}^*,\hat{\mathbf{x}}^b) encodes the observation constraint.

We incorporate this constraint during sampling through observation-conditioned reverse transitions:

p(xn−1∣xn,y)≈q(xn−1∣OI⁡(x0ϵ,y),xn).p(\mathbf{x}_{n-1}\mid\mathbf{x}_n,\mathbf{y})\approx q(\mathbf{x}_{n-1}\mid\operatorname{OI}(\mathbf{x}_0^{\epsilon},\mathbf{y}),\mathbf{x}_n).

OI refines the predicted reference using a Gaussian surrogate. Its MAP update is exact; the Gaussian surrogate and the assumption that observations only shift the reverse-transition mean are approximations.

Observations thus enter both training and inference. The invertible reparameterization preserves target conditioning, while Bayesian factorization links the stages. This formulation-level consistency is our “consistent probabilistic view”. SBDA does not guarantee exact target-posterior sampling.

To empirically test whether stepwise OI corrections introduce additional bias or overconfidence, we added a linear–Gaussian benchmark with an analytical posterior (200 test cases, 1,000 ensemble members per case). FULL SBDA and w/o I (FULL without stepwise OI corrections) share the checkpoint, enhanced endpoint, and sampling noise:

Unobserved-dimension metricw/o IFULL
Posterior mean error0.1131050.112561
Variance ratio (target: 1)0.63350.6677
80% coverage66.94%67.63%
Relative covariance error0.9778910.979311

No added posterior-mean bias or overconfidence is observed with stepwise OI corrections: mean error falls, while variance ratio and coverage rise.

We repeat this ablation for weather ensembles:

Metricw/o IFULL
Mean CRPS reduction vs w/o I—3.80%
Mean 80% coverage53.70%53.83%
Mean interval-width reduction vs w/o I—4.75%

CRPS improves on all 69 variables, with narrower intervals and no worsening of mean coverage. These results further demonstrate the benefits of the combined observation updates in the implemented sampler.

Q3: Do SBDA’s gains persist against non-diffusion baselines and a stronger 3D-Var configuration?

A3: Thank you for suggesting more diverse baselines, which help us better assess SBDA’s contribution. GEN_BE 3D-Var [3], L3DVar [4], and VAE-Var [5] now appear throughout the main-text analysis and forecast comparisons, including single-step assimilation, case studies, cycling assimilation, and unseen observation layouts. L3DVar and VAE-Var provide the requested non-diffusion baselines.

Following Reviewer qjDM’s suggestion, we extended the original 48-h downstream forecast evaluation to 240-h cycling assimilation. Each method assimilates every 48 h into its own forecast, then forecasts from its updated analysis.

SBDA vs baselineSingle-step LW-RMSE reductionCycling LW-RMSE reduction
GEN_BE 3D-Var4.03%11.75%
L3DVar9.83%21.74%
VAE-Var4.93%6.54%

Reductions average equally across 69 variables; cycling first averages each variable’s errors over 40 forecast times. Gains persist against the strengthened classical and non-diffusion baselines.


Q4: How are the 3D-Var covariance configuration and relevant hyperparameters estimated or tuned? Do the learned methods use comparable data, and are their computational budgets clearly documented?

A4: As detailed in the appendix, GEN_BE [3] estimates covariance statistics from 1,000 training-period Aurora [6] 48-h forecast–ERA5 pairs. From these errors, we estimate cross-variable balance relationships, vertical EOFs, and horizontal spherical-harmonic power spectra. These define a multivariate B\mathbf B with cross-variable, vertical, and horizontal correlations. We minimize the variational cost using L-BFGS in preconditioned control space. For corresponding observed components, diagonal R\mathbf R has Rii=0.01BiiR_{ii}=0.01B_{ii} (standard-deviation ratio: 0.1).

L3DVar and VAE-Var retain their original default method-specific hyperparameters. All learning-based methods use the same data and U-Net backbone configuration with necessary input/encoder/decoder adaptations (e.g. VAE-Var uses two U-Nets: one as the encoder and the other as the decoder). Moreover, all learning-based baselines have at least as many parameters as SBDA. All learning-based methods use the same training budget: 20,000 updates, global batch size 8, and eight RTX 4090 GPUs.

MethodParametersTraining time (h:mm:ss)
SBDA37.01M3:42:06
DiffDA37.45M4:19:13
L3DVar66.85M3:31:11.50
VAE-Var66.04M3:29:40.08

References

[1] Liu et al. (2023). I²SB: Image-to-Image Schrödinger Bridge.

[2] Albergo et al. (2025). Stochastic Interpolants: A Unifying Framework for Flows and Diffusions.

[3] Descombes et al. (2015). Generalized background error covariance matrix model (GEN_BE v2.0).

[4] Fan et al. (2026). Physically consistent global atmospheric data assimilation with machine learning in latent space.

[5] Xiao et al. (2025). VAE-Var: Variational Autoencoder-Enhanced Variational Methods for Data Assimilation in Meteorology.

[6] Bodnar et al. (2025). A foundation model for the Earth system.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer aM4w,

Thank you again for your time and thoughtful review. As the discussion period is nearing its end, we would appreciate it if you could take a look at our response to your review and let us know whether any concerns remain. We would be happy to provide further clarification.

Best regards, The Authors

Reviewer_qjDMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces SBDA, a weather data assimilation method based on Schrödinger bridges. The method first incorporates sparse observations into a background forecast through optimal interpolation (OI), then trains a bridge model to generate analysis fields from the corrected background, with further observation-based corrections during generation. The experiments evaluate single-step assimilation, subsequent 48-hour forecasts, generalization to unseen observation locations, and uncertainty in the generated ensembles

Critical strengths
  1. The approach is intuitive and has merit. Background forecasts already contain substantial information about atmospheric structure, so generating analysis fields from an observation-corrected background and incorporating observation information during training are reasonable design choices. Through ablations and a DiffDA-OI comparison, the paper also attempts to distinguish the contributions of observation preprocessing, the bridge model, and further corrections during generation.
  2. The evaluation goes beyond analysis errors to examine subsequent forecasts and changes in observation locations. In particular, the finding that single-step assimilation errors and subsequent forecast performance yield different rankings for some variables deserves attention. It highlights the need to assess whether an analysis provides a suitable initial state for forecasting, rather than evaluating only its fit at the assimilation time.
Critical weaknesses
  1. There remains a gap between the paper’s claimed “consistent probabilistic view” and the actual sampling procedure. While the reformulation of the conditional distribution has a mathematical basis, the stepwise observation corrections rely on a local Gaussian approximation, and their fidelity to the target posterior is not established. The probabilistic results also show substantial undercoverage: nominal 80% intervals achieve only about 54% coverage on average. The evidence, therefore, more clearly supports improvements in analysis and forecast errors than reliable uncertainty quantification.
  2. Although the ablations provide some support for the bridge model, the value of the generative approach over simpler alternatives could be tested more directly. The main comparisons are with 3D-Var and an adapted version of DiffDA; there is no deterministic model using the same inputs and network architecture to directly predict the analysis field or error correction. The probabilistic evaluation also primarily compares the generated ensemble against the method’s own deterministic output. It therefore remains unclear how much of the bridge model’s value lies in better state correction and how much lies in more useful probabilistic predictions.
  3. The experiments do not yet establish performance under cycling assimilation. Each background is a 48-hour forecast initialized from ERA5, rather than a forecast initialized from the method’s previous analysis and repeatedly passed through subsequent assimilation cycles. The observations are also simulated, noise-free, and include all variables at each observed location. Although unseen observation locations are tested, it remains unclear whether the method retains its advantage as background errors evolve across cycles or when observations are noisy or missing some variables.
Questions for authors
  1. How do the stepwise OI corrections relate to the target posterior, and which parts of this relationship are approximate? Could a small-scale experiment with a tractable posterior assess sampling bias and help explain the interval undercoverage observed in the main experiments?
  2. How would a model trained to directly predict the analysis field perform using the same background forecasts, observations, and network architecture? In the probabilistic evaluation, does SBDA retain a clear advantage when compared with ensembles generated by DiffDA?
4 comments

Thank you for the review. Below are our responses to your questions.

Q1: How does stepwise OI relate to the target posterior, and can a tractable benchmark assess sampling bias?

A1: In the reparameterized target, bridge learning models the first factor. The second factor p(y∣x∗,x^b)p(\mathbf y\mid\mathbf{x}^*,\hat{\mathbf{x}}^b) encodes the observation constraint. We incorporate this constraint during sampling through observation-conditioned reverse transitions:

\approx q(\mathbf{x}_{n-1}\mid\operatorname{OI}(\mathbf{x}_0^\epsilon,\mathbf{y}),\mathbf{x}_n).$$ OI refines the predicted reference using a Gaussian surrogate. Its MAP update is exact; the Gaussian surrogate and the assumption that observations only shift the reverse-transition mean are approximations. Observations thus enter both training and inference. The invertible reparameterization preserves target conditioning, while Bayesian factorization links the stages. **This formulation-level consistency is our “consistent probabilistic view”.** SBDA does not guarantee exact target-posterior sampling. To test whether stepwise OI adds additional bias or overconfidence, we added a linear–Gaussian benchmark with an analytical posterior (200 test cases, 1,000 members each). FULL SBDA and w/o I (FULL without stepwise OI corrections) share the checkpoint, enhanced endpoint, and sampling noise: | Unobserved-dimension metric | w/o I | FULL | |---|---:|---:| | Posterior mean error | 0.113105 | 0.112561 | | Variance ratio (target: 1) | 0.6335 | 0.6677 | | 80% coverage | 66.94% | 67.63% | | Relative covariance error | 0.977891 | 0.979311 | **No added posterior-mean bias or overconfidence is observed with stepwise OI corrections**: mean error falls, while variance ratio and coverage rise. Undercoverage persists in the controlled linear–Gaussian benchmark, where SBDA’s nominal 80% intervals achieve 67.63% coverage. Removing stepwise OI refinement lowers coverage to 66.94%. These results show that undercoverage persists without weather dynamics or stepwise OI refinement. **One possible contributor is the basic DDPM-style reverse sampler**: each transition substitutes a single predicted endpoint into a Gaussian bridge kernel with prescribed covariance, potentially underrepresenting the remaining endpoint uncertainty. We repeat this ablation for weather ensembles: | Metric | w/o I | FULL | |---|---:|---:| | Mean CRPS reduction vs w/o I | — | **3.80%** | | Mean 80% coverage | 53.70% | 53.83% | | Mean interval-width reduction vs w/o I | — | **4.75%** | **CRPS improves on all 69 variables, with narrower intervals and no worsening of mean coverage.** Undercoverage is already present without stepwise OI. --- **Q2: How does SBDA compare with matched direct regression and DiffDA ensembles?** **A2:** We evaluate state correction against matched regression and probabilistic prediction against DiffDA ensembles [1]. Regression uses the same U-Net, OI-enhanced inputs, data, and training budget. Ensembles have 20 members at the same 304 analysis times. | SBDA comparison | Mean reduction | Improved variables | |---|---:|---:| | Direct regression (LW-RMSE) | **9.86%** | **60/69** | | DiffDA ensembles (CRPS) | **13.23%** | **66/69** | **Both comparisons favor SBDA.** LW-RMSE reductions vs background are 16.63%/7.22% for SBDA/regression; mean 80% coverage is 53.83%/43.67% for SBDA/DiffDA.

Q3: Does SBDA retain its advantage under cycling assimilation with evolving background errors?

A3: Thank you for suggesting a cycling assimilation evaluation.

We expanded the main-text 48-h forecast evaluation into a 240-h cycling assimilation experiment with Aurora [2]. At 0 h, each method updates its initial 48-h background with sparse observations. Thereafter, Aurora advances each method’s own analysis by 48 h; new observations update that forecast, and the resulting analysis initializes the next cycle. There is no ERA5 reinitialization between cycles, so backgrounds evolve from previous analyses.

Baselines include GEN_BE 3D-Var [3], DiffDA [1], and the newly added L3DVar [4] and VAE-Var [5]. Forecast LW-RMSE is evaluated every 6 h; errors are averaged within each period before computing reductions against VAE-Var, the strongest baseline:

Forecast period (h)Mean LW-RMSE reductionImproved variables
6–483.78%61/69
54–965.97%61/69
102–1446.97%61/69
150–1927.40%61/69
198–2407.28%60/69
Overall, 6–2406.54%61/69

The advantage persists across all five cycles as background errors evolve. SBDA also yields the lowest t2mt2m forecast error at every evaluation time from 18 to 240 h, despite weaker single-step analysis accuracy.


Q4: Why use noise-free observations, and how does SBDA perform with noisy observations?

A4: Noise-free ERA5 observations follow the column-mask protocol of DiffDA [1] and VAE-Var [5]. DiffDA uses simulated observations because it treats observations as ground truth without an observation quality-control procedure, while VAE-Var adopts this setting for comparison with DiffDA. We follow this protocol to compare assimilation methods without introducing additional differences in observation preprocessing and quality control.

To further test SBDA’s generalization to noisy observations, we add independent Gaussian noise with standard deviation 0.1Bii0.1\sqrt{B_{ii}} and variance 0.01Bii0.01B_{ii}. We estimate B\mathbf B using GEN_BE [3], a framework for estimating background-error covariance, from Aurora forecast–ERA5 training pairs. This models relatively accurate but noisy observations, scaled to each variable’s background uncertainty. The SBDA checkpoint and OI parameters remain unchanged, without retraining.

ObservationsMean LW-RMSE reduction vs backgroundImproved variables
Noise-free16.6324%68/69
Noisy16.6277%68/69

Mean relative LW-RMSE changes by only +0.005%, showing good generalization to the tested noisy observations.


References

[1] Huang et al. (2024). DiffDA: a Diffusion model for weather-scale Data Assimilation.

[2] Bodnar et al. (2025). A foundation model for the Earth system.

[3] Descombes et al. (2015). Generalized background error covariance matrix model (GEN_BE v2.0).

[4] Fan et al. (2026). Physically consistent global atmospheric data assimilation with machine learning in latent space.

[5] Xiao et al. (2025). VAE-Var: Variational Autoencoder-Enhanced Variational Methods for Data Assimilation in Meteorology.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer qjDM,

Thank you again for your time and thoughtful review. As the discussion period is nearing its end, we would appreciate it if you could take a look at our response to your review and let us know whether any concerns remain. We would be happy to provide further clarification.

Best regards, The Authors

Dear Reviewer qjDM,

We are writing to briefly follow up before the discussion period closes. We would be grateful for any feedback on our response, particularly regarding any remaining concerns that we could help clarify.

Thank you again for your time and consideration.

Best regards, The Authors

Comments on the paper

1 comment

We thank the reviewers for their constructive feedback and their responses to our rebuttal. Based on their comments, we have revised the manuscript as follows:

  1. Section 3.4: Clarified that per-step observation refinement provides a heuristic approximation to the observation-conditioned reverse transition p(xn−1∣xn,y)p(\mathbf{x}_{n-1}\mid\mathbf{x}_n,\mathbf{y}).

  2. Section 4: Added strong non-diffusion baselines, L3DVar and VAE-Var, throughout the experimental comparisons and strengthened the 3D-Var baseline with a GEN_BE-based implementation. Full results across all 69 variables are provided in the Appendix.

  3. Section 4.2, Probabilistic Analysis: Added comparisons between 20-member SBDA and DiffDA ensembles, including CRPS and prediction interval coverage.

  4. Section 4.3: Extended the downstream forecasting evaluation from a single assimilation update to 240-h cycling assimilation, with observations assimilated every 48 h.

  5. Appendix: Added an analytically tractable posterior benchmark to assess whether per-step observation refinement introduces additional bias or overconfidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.