acceptodds
Under review as a conference paper at ICLR 2027

EnJoi: Ensemble Joint Score Filter for Generative Data Assimilation

Abstract

Data Assimilation (DA) aims to recover the full state of a dynamical system that is only partially observed. Score-based models have shown superior performance in reconstructing state trajectories that agree with the measurements. They are used in autoregressive fashion by conditioning on the previous state, however they do not take into account the uncertainty of their past predictions. We propose a new diffusion-based assimilation algorithm that dynamically balances the confidence in the current state and the new observations. Crucially, we choose to learn the distribution of the joint state containing both the past and future. This allows us to use a modified version of En4DVar, a classical DA algorithm that relies on the covariance of an ensemble of particles. Experiments on fluid and traffic flow simulations show improved reconstruction performance, especially in situations where observations are sparse and non-homogeneous.

Primary area:
learning on time series and dynamical systems
Keywords:
Diffusion, Data Assimilation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_5DzsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EnJoi is a diffusion-based filter for data assimilation. A score model is trained on joint (past, future) windows. At each assimilation step, every particle is sampled with two guidance terms: a Gaussian background term centred on that particle, using the localized ensemble covariance PkP_k, and the observation likelihood. The combined guidance is rescaled to a fixed norm and followed by a Langevin correction. The new ensemble then gives the next covariance, as in En4DVar. On Kuramoto–Sivashinsky, 2D Kolmogorov flow and an ARZ traffic model observed by probe vehicles, EnJoi gives the best CRPS in most settings, with the largest gains when observations are sparse or moving.

Critical strengths
  1. Section 4 identifies a real failure mode. Conditional and joint AR filters treat the past state as exact, so they cannot recover from a bad background, while "volatile" variants overfit noisy observations. Using the ensemble covariance to set a spatially varying trust in the background, and correcting particles locally instead of resampling them, is a sensible fix.
  2. The gains appear where the method should help: a moving sensor on KS (CRPS 0.162 vs 0.293 for the best baseline), 4×4 observations on Kolmogorov (0.12 vs 0.18), and traffic probes, where EnJoi with 10 particles beats a 128-particle filter (0.11 vs 0.18 at 5% penetration).
Critical weaknesses
  1. The covariance contribution is not isolated. EnJoi differs from the closest baseline, Joint AR (v)(g=scaled), in both P (empirical and localized vs constant) and R (true vs shrunk). There is no comparison of a tuned constant P, a diagonal empirical P and the localized full P under the same scaled guidance, R and corrector. With dense observations EnJoi is not better (KS σobs=2\sigma_{\mathrm{obs}} = 2: 0.229 vs 0.200; Kolmogorov 8×8: 0.047 vs 0.043), so it is unclear how much of the gain comes from the covariance.
  2. The sampler is loosely tied to the stated objective, and the paper does not match the code. Treating the background as a likelihood biases samples toward climatology even in the linear Gaussian case (App. C.1), and normalizing the guidance changes the stationary points of the MAP problem in Eq. (10). Eq. (11) uses x←x+τ,s/∣s∣+τ,ϵx \leftarrow x + \tau\\, s/\\|s\\| + \sqrt{\tau}\\,\epsilon, while the released DPM sampler uses δ=τ/mean(s2)\delta = \tau/\mathrm{mean}(s^2) and x←x+δs+2δ,ϵx \leftarrow x + \delta s + \sqrt{2\delta}\\,\epsilon on the guided score. The paper says the Langevin correction is necessary, so it matters which update produced the tables.
Questions for authors
  1. Which Langevin update produced Tables 1–3, Eq. (11) or the one in the released code?
  2. With guidance, R and corrector fixed, how do a tuned constant P, a diagonal empirical P and the localized full P compare on the KS Sine/Exponential and Kolmogorov 4×4 settings? How sensitive are results to the localization radius?
  3. Each particle is pushed toward an approximate MAP, and in Figs. 5 and 6 the EnJoi spread is close to zero over most of the domain while errors are still visible. Is the ensemble calibrated? Please report spread/skill ratios over the rollouts.
  4. Please proofread the paper: "non-homogneous" (Fig. 2 caption), "conjonction" and "netwok" (App. B.3), "fundemental", "inacurrate" and "choise" (App. C.1), and "Enjoi" vs "EnJoi" (Sec. 5.3).
3 comments

We thank the reviewer for the helpful feedback.

Question 1: The paper uses a different Langevin update rule than the DPM code

We thank the reviewer for spotting this mistake in the Langevin formula. The Langevin update we use is the one from the code, where the step size is proportional to the inverse mean squared score. We have corrected the paper and the appendix accordingly.

Question 2

2.a) No comparison to a tuned constant P

(We provide a shared answer to both reviewers AZpY and 5Dzs)

The reconstruction guidance strength parameter is indeed an important component of Joint AR and we should have explored parameter tuning more carefully. We run new experiments to conduct a hyperparameter sweep that shows the effect of modifying σpast\sigma_{\text{past}} on the best performing Joint AR (v) baseline (Appendix F, Figure 7). The CRPS is reported in the tables below, the best algorithm is bolded and the second is underlined.

Bernoulli observations (σobs=0.1\sigma_{\text{obs}}=0.1) (average 10 trajectories)

σpast\sigma_{\mathrm{past}}NA0.0050.010.050.1
EnJoi0.010±0.000\mathbf{0.010} {\scriptstyle \pm 0.000}
Joint AR (v) (g=scaled)0.135±0.0480.135 {\scriptstyle \pm 0.048}0.011‾±0.001{\underline{0.011} {\scriptstyle \pm 0.001}}0.019±0.0000.019 {\scriptstyle \pm 0.000}0.025±0.0000.025 {\scriptstyle \pm 0.000}
Joint AR (v) (g=SDA)0.014±0.0020.014 {\scriptstyle \pm 0.002}0.012±0.001{0.012} {\scriptstyle \pm 0.001}0.017±0.0000.017 {\scriptstyle \pm 0.000}0.020±0.0000.020 {\scriptstyle \pm 0.000}

Sine observations (σobs=0.5\sigma_{\text{obs}}=0.5) (one trajectory)

σpast\sigma_{\mathrm{past}}NA0.0050.010.050.1
EnJoi0.155
Joint AR (v) (g=scaled)0.5000.2860.1370.173
Joint AR (v) (g=SDA)0.1870.2250.1510.145‾\underline{0.145}

Exponential observations (σobs=0.5\sigma_{\text{obs}}=0.5) (one trajectory)

σpast\sigma_{\mathrm{past}}NA0.0050.010.050.1
EnJoi0.040
Joint AR (v) (g=SDA)0.1340.1290.1320.138
Joint AR (v) (g=scaled)0.3340.2230.105‾\underline{0.105}0.112

In the Bernoulli setting, the best baseline is obtained for the default value σpast=0.01\sigma_{\text{past}}=0.01. With Sine observations, the tuned Joint AR baselines reaches the performance of EnJoi by weakening the past guidance. However, in the exponential observation setting, EnJoi obtains better results by a large margin. In this last setting, the quality of the background varies depending on the time since the last observation, thus a constant σpast\sigma_{\text{past}} level is sub-optimal.

We will update the CRPS in Table 1 once we run the full 10 test trajectories for the Joint AR baseline tuned on one trajectory.

2.b) No comparison of diagonal empirical P and localized full P

We have added a sweep with respect to the covariance bandwidth (Appendix F, Figure 8). The diagonal empirical covariance (band$=0\Delta z$) obtains weaker results. The best performance is obtained with a band at 10Δz10\Delta z and degrades as the bandwidth increases. The optimal bandwidth is expected to vary between experiments since it influences the covariance strength.

CRPS on the Bernoulli experiment with σpast=0.5\sigma_{\text{past}}=0.5

Covariance bandwidth0135103060
EnJoi CRPS0.059±0.0130.059 {\scriptstyle \pm 0.013}0.055±0.0110.055 {\scriptstyle \pm 0.011}0.050±0.0040.050 {\scriptstyle \pm 0.004}0.045±0.0120.045 {\scriptstyle \pm 0.012}0.037±0.007\mathbf{0.037} {\scriptstyle \pm 0.007}0.037±0.0070.037 {\scriptstyle \pm 0.007}0.041±0.0050.041 {\scriptstyle \pm 0.005}

2.c) There are two experiments where EnJoi is not better (but second best)

In the case of KS with σobs\sigma_{\text{obs}}=2.0, this is a difficult setting and all models perform poorly. The resulting trajectories are mostly discontinuous.

In the case of Kolmogorov with grid 8, Joint AR obtains slightly better CRPS than EnJoi. This means that both the constant covariance and empirical covariance models are well calibrated. Observations are spatially dense and homogeneous so there is little gain of using the empirical spatial covariance in this case.

Question 3. Is the ensemble calibrated ? Please provide skill / spread ratios

We have added a spread and skill plot for the Kuramoto-Sivashinsky and Kolmogorov experiments (Appendix G). Figures 9 and 10 show that EnJoi is well calibrated. The results for two experiments settings are reported in the following tables. See the updated paper for a more complete picture.

The tables report the average metric over 10 trajectories and 3 standard errors.

KS Bernoulli experiment with σobs=0.5\sigma_\text{obs}=0.5

methodskillspreadspread skill ratio
Cond. P.F. (s) (g=MMPS)1.47±0.141.47 {\scriptstyle \pm 0.14}0.03±0.010.03 {\scriptstyle \pm 0.01}0.02±0.010.02 {\scriptstyle \pm 0.01}
Cond. AR (s) (g=SDA)1.10±0.111.10 {\scriptstyle \pm 0.11}0.94±0.050.94 {\scriptstyle \pm 0.05}0.86±0.090.86 {\scriptstyle \pm 0.09}
Joint AR (s) (g=SDA)1.13±0.071.13 {\scriptstyle \pm 0.07}1.07±0.031.07 {\scriptstyle \pm 0.03}0.95±0.040.95 {\scriptstyle \pm 0.04}
Joint AR (v) (g=SDA)0.18±0.000.18 {\scriptstyle \pm 0.00}0.07±0.000.07 {\scriptstyle \pm 0.00}0.42±0.010.42 {\scriptstyle \pm 0.01}
Joint P.F. (v) (g=scaled)0.10±0.010.10 {\scriptstyle \pm 0.01}0.02±0.000.02 {\scriptstyle \pm 0.00}0.16±0.020.16 {\scriptstyle \pm 0.02}
EnJoi (s) (ours)0.10±0.010.10 {\scriptstyle \pm 0.01}0.07±0.000.07 {\scriptstyle \pm 0.00}0.72±0.070.72 {\scriptstyle \pm 0.07}

Kolmogorov 4x4 experiment :

methodskillspreadspread skill ratio
Cond. AR (s) (g=SDA)0.52±0.050.52 {\scriptstyle \pm 0.05}0.64±0.090.64 {\scriptstyle \pm 0.09}1.23±0.141.23 {\scriptstyle \pm 0.14}
Cond. P.F. (s) (g=MMPS)0.63±0.130.63 {\scriptstyle \pm 0.13}0.21±0.100.21 {\scriptstyle \pm 0.10}0.33±0.140.33 {\scriptstyle \pm 0.14}
Joint AR (v) (g=SDA)0.41±0.060.41 {\scriptstyle \pm 0.06}0.36±0.060.36 {\scriptstyle \pm 0.06}0.90±0.120.90 {\scriptstyle \pm 0.12}
EnJoi (s) (ours)0.27±0.070.27 {\scriptstyle \pm 0.07}0.24±0.030.24 {\scriptstyle \pm 0.03}0.92±0.150.92 {\scriptstyle \pm 0.15}

Question 4. Please proofread the paper

We thank the reviewer for identifying mistakes in the wording, we have corrected them.

Thank you for correcting the Langevin update and adding the covariance and calibration analyses. These additions are helpful, but tuning narrows the advantage over Joint AR, and the covariance contribution is still not fully isolated under matched settings. I remain positive about the sparse-observation results and will maintain my weak-accept rating.

Reviewer_AZpYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work targets the problem of data assimilation using generative models. In particular, it introduces a new guidance method for sequential filtering that takes historical uncertainties into account. Previous approaches have focused on sampling from the optimal proposal p(xt∣xt−1,yt)p(x_{t}|x_{t-1}, y_{t}), which, unless combined with weights, ignores the uncertainty of the previous state. Enjoi instead introduces uses a joint prior p(xt,xt−1)p(x_{t}, x_{t-1}) and introduces a pseudo-observation on xt−1x_{t-1} using the ensemble covariance, which provides an uncertainty-informed soft conditioning.

Critical strengths

The motivation is good, methods targeting p(xt∣xt−1,yt)p(x_{t}|x_{t-1}, y_{t}) often diverge in sparse and complex settings due to their inability to revise past states. Fully-adapted particle filters try to fix this, but do it it using weights that will tend to degenerate in high-dimensions.

The space-time inhomogeneous observations are interesting, resembling how real-life observations such as radar move.

Critical weaknesses

My first main concern with the paper is that it doesn't demonstrate the usefulness of the method. While, the motivation is good, the underlying problem is that the usual target for these methods is the wrong one; either because they ignore weight updates or because they use a biased guidance method. The derived algorithm has a reasonable heuristic, but doesn't get closer to targeting the true filtering distribution.

The design choises are centered around linear gaussian observations, making it less surprising that the heuristics work in these settings. This is the setting where classical methods already work well, and not where generative models are needed. The paper doesn't compare to either (En)4DVar or LETKF, which would likely perform well for these problems. It also doesn't look at any nonlinear or multimodal observations where the classical approaches are known to fail.

The most related model is Joint AR which differs from the proposed method only in the choice of reconstruction term. However, as far as I understood, the \sigma_\min in the reconstruction term is chosen as a fixed value and not treated as a hyperparameter which you tune for each experiment. This gives the proposed method an advantage since it tunes the localization of the ensemble covariance P for each experiment, and rebalances the guidance by dividing by the norm of the gradient.

The only figure for the Kolmogorov flow conveniently displays the Cond PF method which performs much worse than enjoi, even though enjoi isn't even the best method in this setting. The paper cites DAISI, but doesn't compare to it, even though it shares the same motivation.

My second main concern with the paper is its presentation. The writing feels rushed, some of the wording and notation is non-standard, and the structure needs revisiting. For example, the main method isn't introduced until page five, after one of the experiments.

Questions for authors
  • Can you provide comparisons to LETKF and/or (En)4DVar?
  • Can you provide experiments with nonlinear and/or multimodal observations?
  • Can you clarify how Joint AR chooses the \sigma_\min parameter and guidance strength?
  • Can you provide a comparison to DAISI?
7 comments

We thank the reviewer for the helpful feedback. We present our answers to the raised concerns.

Questions 1,2 and 4: Can you provide a comparison to DAISI, LETKF and/or (En)4DVar ? Can you provide experiments with non-linear or multimodal observations ?

Following the setup described by Andrae et al [1], we implement EnJoi on the surface-quasi-geostrophic (SQG) dataset. Our method is compared to the LETKF and to DAISI.

The state consists in a 64x64 potential vorticity field which evolves across 100 timesteps. In the default observation setting, 25% of the grid cells are observed at random at each step, with Gaussian noise σobs=1\sigma_{\text{obs}}=1. We study four observation scenarios: Noisy (the noise is increased to σobs=5\sigma_{\text{obs}}=5), Sparse (only 5% of the cells are observed), Multimodal (the observation function is (x/7)2(x/7)^2) and Saturating (the observation function is arctan⁡(x)\arctan(x) and σobs=0.01\sigma_{\text{obs}}=0.01).

The following table presents the CRPS for the last 20 frames (the baseline metrics are the one reported in the DAISI paper).

The results were added in a new subsection, the changes are highlighted in red in the revised paper.

ExperimentEnJoiDAISILETKF
Noisy1.73±0.161.73 {\scriptstyle \pm 0.16}1.32±0.13\mathbf{1.32}{\scriptstyle \pm0.13}1.34‾±0.14\underline{1.34}{\scriptstyle \pm0.14}
Sparse1.61±0.28\mathbf{1.61} {\scriptstyle \pm 0.28}1.73‾±0.18\underline{1.73}{\scriptstyle \pm0.18}2.35±0.282.35{\scriptstyle \pm0.28}
Multimodal1.04±0.21\mathbf{1.04} {\scriptstyle \pm 0.21}1.81‾±0.44\underline{1.81}{\scriptstyle \pm0.44}1.97±0.69{1.97}{\scriptstyle \pm0.69}
Saturating2.18‾±0.31\underline{2.18} {\scriptstyle \pm 0.31}1.54±0.09\mathbf{1.54}{\scriptstyle \pm0.09}5.24±0.355.24{\scriptstyle \pm0.35}

We note that DAISI requires access to two models at inference time (one for prediction, the other for analysis), while the other baselines in our paper only use a single model.

Our EnJoi model has comparable performance to DAISI on the sparse setting and is robust to non-linear and multimodal observations. We note that we use MMPS guidance with the EnJoi covariance, since the other guidance methods underperform. The scaled and SDA guidances seem to struggle with nonlinear observations and the sharper features of the SQG dataset. The guidance scaling was introduced to handle long gaps in observations. Since SQG observations are dense in time, it is not needed here.

Question 3: Can you clarify how Joint AR chooses the parameter and guidance strength ? EnJoi has an unfair advantage over the Joint AR baseline because the parameter σpast\sigma_{\text{past}} was not tuned for each experiment.

(We provide a shared answer to both reviewers AZpY and 5Dzs)

The reconstruction guidance strength parameter is indeed an important component of Joint AR and we should have explored parameter tuning more carefully. We run new experiments to conduct a hyperparameter sweep that shows the effect of modifying σpast\sigma_{\text{past}} on the best performing Joint AR (v) baseline (Appendix F, Figure 7). The CRPS is reported in the tables below, the best algorithm is bolded and the second is underlined.

Bernoulli observations (σobs=0.1\sigma_{\text{obs}}=0.1) (average 10 trajectories)

σpast\sigma_{\mathrm{past}}NA0.0050.010.050.1
EnJoi0.010±0.000\mathbf{0.010} {\scriptstyle \pm 0.000}
Joint AR (v) (g=scaled)0.135±0.0480.135 {\scriptstyle \pm 0.048}0.011‾±0.001{\underline{0.011} {\scriptstyle \pm 0.001}}0.019±0.0000.019 {\scriptstyle \pm 0.000}0.025±0.0000.025 {\scriptstyle \pm 0.000}
Joint AR (v) (g=SDA)0.014±0.0020.014 {\scriptstyle \pm 0.002}0.012±0.001{0.012} {\scriptstyle \pm 0.001}0.017±0.0000.017 {\scriptstyle \pm 0.000}0.020±0.0000.020 {\scriptstyle \pm 0.000}

Sine observations (σobs=0.5\sigma_{\text{obs}}=0.5) (one trajectory)

σpast\sigma_{\mathrm{past}}NA0.0050.010.050.1
EnJoi0.155
Joint AR (v) (g=scaled)0.5000.2860.1370.173
Joint AR (v) (g=SDA)0.1870.2250.1510.145‾\underline{0.145}

Exponential observations (σobs=0.5\sigma_{\text{obs}}=0.5) (one trajectory)

σpast\sigma_{\mathrm{past}}NA0.0050.010.050.1
EnJoi0.040
Joint AR (v) (g=SDA)0.1340.1290.1320.138
Joint AR (v) (g=scaled)0.3340.2230.105‾\underline{0.105}0.112

In the Bernoulli setting, the best baseline is obtained for the default value σpast=0.01\sigma_{\text{past}}=0.01. With Sine observations, the tuned Joint AR baselines reaches the performance of EnJoi by weakening the past guidance. However, in the exponential observation setting, EnJoi obtains better results by a large margin. In this last setting, the quality of the background varies depending on the time since the last observation, thus a constant σpast\sigma_{\text{past}} level is sub-optimal.

We will update the CRPS in Table 1 once we run the full 10 test trajectories for the Joint AR baseline tuned on one trajectory.

1. Answer to the first main concern:

1.a) The usual target for these methods is the wrong one; either because they ignore weight updates or because they use a biased guidance method. The method doesn't get closer to targeting the true filtering distribution

We agree with the reviewer and we acknowledge that the proposed EnJoi method is an empirical heuristic and is biased. Our paper focuses on the practical implementation of assimilation methods rather than on the theoretical side. We argue that the true filtering distribution is generally intractable in high dimensions, thus we have to make practical approximations. We note that the best performing methods in the field are also well-motivated heuristics (e.g. DAISI [1], FA-APF [2]) and outperform classical baselines.

Perhaps we have not understood the exact issue the reviewer raises about heuristic diffusion-based filtering methods. In that case, could the reviewer be more precise about which approximations are problematic ?

A classical unbiased baseline, Bootstrap PF, requires a great number of particles in practice and perform worse in high dimensions, as shown by Savary et al. on 2D Navier-Stokes Flow ([2], Table 2, sparse (32x32) setting, the BPF has a skill of 2.88, FA-APF obtains 0.13).

1.b) Observations in experiments are all Linear-Gaussian. This is the setting where classical methods already work well, and not where generative models are needed.

We disagree with the reviewer that LETKF already works well on linear Gaussian observations. On the surface-quasi-geostrophic dataset in the Sparse observation regime (5% pixels observed), the generative methods are more effective than the linear Kalman update.

The CRPS for DAISI is 1.73, EnJoi obtains 1.61 while the LETKF has 2.35.

1.c) The Kolmogorov figure conveniently shows a worse performing method.

The reviewer is right and we have modified the figure to show all three methods (EnJoi, FA-APF and Joint AR) on the 4x4 grid where EnJoi performs better.

2. Answer to the second main concern: There is an issue with the presentation, the method is introduced after some experiments, some notation is non standard

We thank the reviewer for this feedback on the writing and flow of the paper. To improve the coherence of the paper, we have made changes to the presentation of the paper. In particular, we have swapped the order of sections 4 and 5. The description of our method now appears before the toy experiments. The edits are highlighted in red in the revision.

We are open to feedback on the notation and wording if you think some of it is misleading.

  • [1] DAISI: Data Assimilation with Inverse Sampling using Stochastic Interpolants. M. Andrae, E. Larsson, S. Takao, T. Landelius, F. Lindsten. International Conference on Machine Learning 2026 .
  • [2] Training-Free Bayesian Filtering with Generative Emulators. T. Savary, F. Rozet, G. Louppe. International Conference on Machine Learning 2026.
Reviewer_gF4vOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a method for joint score filter. Compared with the previous generative AI-based filtering algorithms, EnJoi maintains the uncertainty of the latent variable xkx^k throughout the filtering process. It proposes a framework to jointly sample xkx^k and xk+1x^{k+1} for each filtering step. The joint sampling explicitly considers the uncertainty in the previous state estimate. In the posterior sampling process, EnJoi proposes to normalize the score to balance the strength of the guidance.

Critical strengths

The idea is interesting and the presentation is clear. The experiments on heavy-traffic PDE create meaningful reconstructions of the traffic state.

Critical weaknesses

The empirical evaluation would benefit from additional baselines. Beyond Joint AR, Cond AR, and Cond PF, relevant methods include FA-APF and SURGE. Although these methods are discussed in the appendix, quantitative comparisons would help establish the proposed method’s advantages over existing approaches.

Questions for authors

Existing diffusion training strategies explicitly accommodate noisy conditioning inputs. Examples include conditioning augmentation in Cascaded Diffusion Models (Ho et al., 2022) and training with independent per-token noise levels in Diffusion Forcing (Chen et al., 2024). Have the authors considered adapting these strategies to learn p(xk+1∣x^k)p(x^{k+1}\mid \hat{x}^{k}), where x^k\hat{x}^{k} is an imprecise estimate or samples in a posterior ensemble of xkx^{k}, and comparing the resulting performance with joint sampling from p(xk+1,xk)p(x^{k+1},x^{k})? Such a comparison would help assess whether joint sampling offers advantages over conditional models explicitly trained to handle imperfect conditioning inputs.

References

  • Ho, Jonathan, Chitwan Saharia, William Chan, David J. Fleet, Mohammad Norouzi, and Tim Salimans. "Cascaded diffusion models for high fidelity image generation." Journal of Machine Learning Research 23, no. 47 (2022): 1-33.
  • Chen, Boyuan, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. "Diffusion forcing: Next-token prediction meets full-sequence diffusion." Advances in Neural Information Processing Systems 37 (2024): 24081-24125.
1 comment

We thank the reviewer for the helpful feedback. We present our answers to the raised concerns.

Weakness 1: The paper lacks a comparison to FA-APF and SURGE

We apologize for the confusion but we do in fact compare to the diffusion-based FA-APF from Savary et al. [1]. We used a different name in our paper and refer to it as the Cond. PF. We agree that this is confusing and we have edited the paper to use the full name FA-APF.

As a side note, the Joint PF baseline is our extension of the FA-APF algorithm but using a joint score.

Regarding the lack of comparison to SURGE, we argue that the two methods are similar in spirit and were presented concurrently at ICML 2026. Both are particle filters that use weights to resample the ensemble members. We couldn't find the implementation for SURGE online and thus it was difficult to make a fair comparison.

Question 1: Comparison with Diffusion Forcing

The idea of combining DA and diffusion forcing is being concurrently explored by another research team. Their method ForcingDAS was presented in a workshop earlier this year and is being submitted as a paper to ICLR 2027 [2].

We agree that this is an interesting research direction and that it has similarities with our work. We have cited it in our paper (section 5.3) but we think that the method is still not as established as the other baselines. One issue is that the authors don't use the same metrics as the literature for ensemble forecasting and assimilation (CRPS, spread, skill) and only report RMSE.

As we have stated in section 5.3 and appendix B.1, the idea of noising previous states is quite natural. However, one has to choose the appropriate noise level to effectively forget some of the past conditioning. To our knowledge, there is no principled way of choosing the level, so a fixed level is chosen (e.g. for each physical time step). Our algorithm can be seen as an attempt to choose the correct confidence for each spatial and temporal cell. Although we have to use guidance at inference time instead of conditioning during training.

The closely related DAISI [3] applies a reverse flow on the forecasted state until a chosen noise level. The level is a constant that is not derived from the ensemble and controls the influence of the past compared to observations. The flow model is then applied forward with observation guidance to obtain the analysis. We compare to DAISI in a new section and obtain comparable results on the SQG dataset.

Question 2: Have you explored conditional training on an ensemble of perturbed past states ?

Training on incorrect past states directly would indeed be very powerful. If the model can be conditioned directly on the ensemble, we could drop the Gaussian background assumption. In the ideal case it would require realistic assimilated trajectories, but this would turn into RNN full-trajectory training. If we understood correctly, the reviewer instead proposes to use augmentation and learn one step at a time. The model would learn p(xk+1∣x^k)p(x^{k+1}|\hat{x}^k) where x^k={xk,(i)}i=1N\hat{x}^k = \{x^{k,(i)}\}_{i=1}^N is an approximate ensemble of perturbed past states. The ensemble would be generated by posterior sampling xk,(i)∼p(xk∣z)x^{k, (i)} \sim p(x^k|z) but it is unclear what it should be conditioned on. For example, it could be conditioned on a previous state z=xk−Tz=x^{k-T} from a trajectory that leads to xk+1x^{k+1}. Another possibility would be to apply a perturbation (e.g. blurring, masking) z=H(xk)z=H(x^k). However, there is no guarantee that the distribution of the ensembles seen in training matches the ensembles from inference, especially if the observation model is not known beforehand.

Overall, this is an interesting direction that we didn't thought of and we thank the reviewer for pointing it out. We feel like this is a difficult open problem that would likely deserve its own paper.

  • [1] Training-Free Bayesian Filtering with Generative Emulators. T. Savary, F. Rozet, G. Louppe. International Conference on Machine Learning 2026.
  • [2] ForcingDAS: Unified and Robust Data Assimilation via Diffusion Forcing. anonymous authors, Submitted to The Fifteenth International Conference on Learning Representations, 2026 https://openreview.net/forum?id=uO19by6fII
  • [3] DAISI: Data Assimilation with Inverse Sampling using Stochastic Interpolants. M. Andrae, E. Larsson, S. Takao, T. Landelius, F. Lindsten. International Conference on Machine Learning 2026 .

Comments on the paper

3 comments

We thank the reviewers for their helpful feedback. We have submitted a new revision of the paper with changes highlighted in red. The paper was updated with a new comparison to DAISI and LETKF on the surface quasi geostrophic dataset (Section 6.4). We also swapped sections 4 and 5 to improve the flow and coherence of the paper. Two hyperparameter sweeps were added in Appendix F : we explore different values for σpast\sigma_{\text{past}} and the covariance bandwidth. We also provided average skill and spread for the KS and Kolmogorov experiments in Appendix G.

Dear authors and reviewers,

The author-reviewer discussion window will close tomorrow on November 18.

@authors: If not done already, please respond to the reviewers' comments before the discussion window closes. I would advise short and concise responses over lengthy explanations.

@reviewers: Please check the authors' responses and provide any additional comments before the discussion window closes. Consider updating your initial review if necessary, and in particular your score if your opinion has changed. You may increase your score if you feel the authors' responses warrant it, maintain your score if your opinion remains the same, or even decrease your score if you feel the other reviews or the authors' responses reveal new shortcomings. At the very least, please acknowledge the authors' responses.

Thank you for your attention and cooperation.

The AC

We have uploaded a new revision to adress reviewer feedback.

Following the comment of reviewer 5Dzs, we performed a hyperparameter sweep on the covariance bandwidth and obtain improved results for EnJoi by increasing the band parameter from 3 to 10 grid cells on KS and Kolmogorov. We also updated the Joint AR benchmark: the new experiments show that tuning the past deviation σpast\sigma_{\text{past}} parameter greatly improves the performance of the baseline. Although we argue that EnJoi tunes the past confidence automatically and obtains better results on most scenarios. Finally, we include details on the SQG experiment setting in the appendix, along with other minor changes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.