- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
EnJoi is a diffusion-based filter for data assimilation. A score model is trained on joint (past, future) windows. At each assimilation step, every particle is sampled with two guidance terms: a Gaussian background term centred on that particle, using the localized ensemble covariance , and the observation likelihood. The combined guidance is rescaled to a fixed norm and followed by a Langevin correction. The new ensemble then gives the next covariance, as in En4DVar. On Kuramoto–Sivashinsky, 2D Kolmogorov flow and an ARZ traffic model observed by probe vehicles, EnJoi gives the best CRPS in most settings, with the largest gains when observations are sparse or moving.
Critical strengths
- Section 4 identifies a real failure mode. Conditional and joint AR filters treat the past state as exact, so they cannot recover from a bad background, while "volatile" variants overfit noisy observations. Using the ensemble covariance to set a spatially varying trust in the background, and correcting particles locally instead of resampling them, is a sensible fix.
- The gains appear where the method should help: a moving sensor on KS (CRPS 0.162 vs 0.293 for the best baseline), 4×4 observations on Kolmogorov (0.12 vs 0.18), and traffic probes, where EnJoi with 10 particles beats a 128-particle filter (0.11 vs 0.18 at 5% penetration).
Critical weaknesses
- The covariance contribution is not isolated. EnJoi differs from the closest baseline, Joint AR (v)(g=scaled), in both P (empirical and localized vs constant) and R (true vs shrunk). There is no comparison of a tuned constant P, a diagonal empirical P and the localized full P under the same scaled guidance, R and corrector. With dense observations EnJoi is not better (KS : 0.229 vs 0.200; Kolmogorov 8×8: 0.047 vs 0.043), so it is unclear how much of the gain comes from the covariance.
- The sampler is loosely tied to the stated objective, and the paper does not match the code. Treating the background as a likelihood biases samples toward climatology even in the linear Gaussian case (App. C.1), and normalizing the guidance changes the stationary points of the MAP problem in Eq. (10). Eq. (11) uses , while the released DPM sampler uses and on the guided score. The paper says the Langevin correction is necessary, so it matters which update produced the tables.
Questions for authors
- Which Langevin update produced Tables 1–3, Eq. (11) or the one in the released code?
- With guidance, R and corrector fixed, how do a tuned constant P, a diagonal empirical P and the localized full P compare on the KS Sine/Exponential and Kolmogorov 4×4 settings? How sensitive are results to the localization radius?
- Each particle is pushed toward an approximate MAP, and in Figs. 5 and 6 the EnJoi spread is close to zero over most of the domain while errors are still visible. Is the ensemble calibrated? Please report spread/skill ratios over the rollouts.
- Please proofread the paper: "non-homogneous" (Fig. 2 caption), "conjonction" and "netwok" (App. B.3), "fundemental", "inacurrate" and "choise" (App. C.1), and "Enjoi" vs "EnJoi" (Sec. 5.3).
3 comments
2.c) There are two experiments where EnJoi is not better (but second best)
In the case of KS with =2.0, this is a difficult setting and all models perform poorly. The resulting trajectories are mostly discontinuous.
In the case of Kolmogorov with grid 8, Joint AR obtains slightly better CRPS than EnJoi. This means that both the constant covariance and empirical covariance models are well calibrated. Observations are spatially dense and homogeneous so there is little gain of using the empirical spatial covariance in this case.
Question 3. Is the ensemble calibrated ? Please provide skill / spread ratios
We have added a spread and skill plot for the Kuramoto-Sivashinsky and Kolmogorov experiments (Appendix G). Figures 9 and 10 show that EnJoi is well calibrated. The results for two experiments settings are reported in the following tables. See the updated paper for a more complete picture.
The tables report the average metric over 10 trajectories and 3 standard errors.
KS Bernoulli experiment with
| method | skill | spread | spread skill ratio |
|---|---|---|---|
| Cond. P.F. (s) (g=MMPS) | |||
| Cond. AR (s) (g=SDA) | |||
| Joint AR (s) (g=SDA) | |||
| Joint AR (v) (g=SDA) | |||
| Joint P.F. (v) (g=scaled) | |||
| EnJoi (s) (ours) |
Kolmogorov 4x4 experiment :
| method | skill | spread | spread skill ratio |
|---|---|---|---|
| Cond. AR (s) (g=SDA) | |||
| Cond. P.F. (s) (g=MMPS) | |||
| Joint AR (v) (g=SDA) | |||
| EnJoi (s) (ours) |
Question 4. Please proofread the paper
We thank the reviewer for identifying mistakes in the wording, we have corrected them.
Thank you for correcting the Langevin update and adding the covariance and calibration analyses. These additions are helpful, but tuning narrows the advantage over Joint AR, and the covariance contribution is still not fully isolated under matched settings. I remain positive about the sparse-observation results and will maintain my weak-accept rating.
We thank the reviewer for the helpful feedback.
Question 1: The paper uses a different Langevin update rule than the DPM code
We thank the reviewer for spotting this mistake in the Langevin formula. The Langevin update we use is the one from the code, where the step size is proportional to the inverse mean squared score. We have corrected the paper and the appendix accordingly.
Question 2
2.a) No comparison to a tuned constant P
(We provide a shared answer to both reviewers AZpY and 5Dzs)
The reconstruction guidance strength parameter is indeed an important component of Joint AR and we should have explored parameter tuning more carefully. We run new experiments to conduct a hyperparameter sweep that shows the effect of modifying σpast on the best performing Joint AR (v) baseline (Appendix F, Figure 7). The CRPS is reported in the tables below, the best algorithm is bolded and the second is underlined.
Bernoulli observations (σobs=0.1) (average 10 trajectories)
Sine observations (σobs=0.5) (one trajectory)
Exponential observations (σobs=0.5) (one trajectory)
In the Bernoulli setting, the best baseline is obtained for the default value σpast=0.01. With Sine observations, the tuned Joint AR baselines reaches the performance of EnJoi by weakening the past guidance. However, in the exponential observation setting, EnJoi obtains better results by a large margin. In this last setting, the quality of the background varies depending on the time since the last observation, thus a constant σpast level is sub-optimal.
We will update the CRPS in Table 1 once we run the full 10 test trajectories for the Joint AR baseline tuned on one trajectory.
2.b) No comparison of diagonal empirical P and localized full P
We have added a sweep with respect to the covariance bandwidth (Appendix F, Figure 8). The diagonal empirical covariance (band$=0\Delta z$) obtains weaker results. The best performance is obtained with a band at 10Δz and degrades as the bandwidth increases. The optimal bandwidth is expected to vary between experiments since it influences the covariance strength.
CRPS on the Bernoulli experiment with σpast=0.5