- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
LD-EnFF performs sequential data assimilation entirely in a learned latent space. An LDNet gives parameter-conditioned latent dynamics. A VAE encodes sparse observations, and an MLP " operator" maps the augmented latent state (latent state plus uncertain parameter) to noiseless observations. Together they define a local Gaussian latent likelihood whose covariance combines observation noise pushed through the encoder Jacobian and the encoder variance. Ensemble flow filtering (OT or perturbed F2P paths, Monte Carlo guidance) then updates latent states and parameters jointly. On Kolmogorov flow, tsunami propagation and PlanetSWE, with 0.1–0.44% of grid points observed, LD-EnFF has the lowest final RelRMSE among latent-space methods (e.g. 5.74% vs 13.09% for the matched LSTM-encoder EnFF on Kolmogorov).
Critical strengths
- Learning a forward map from latent state to observations, instead of an inverse map from observation histories to states, is a sensible design for sparse data, and it avoids full-state reconstruction during the analysis.
- The matched baselines (same LDNet with an LSTM encoder; same likelihood with IEnKF) help locate where the gains come from, and the ranking is consistent across the three benchmarks.
Critical weaknesses
- The main new component, the VAE-based latent likelihood, is not tested against the obvious simpler choice. Relative to LD-EnSF, the changes are EnSF → EnFF and the LSTM inverse encoder → VAE plus forward operator, so the likelihood is the key new piece. Since already predicts observations, one can use directly, with no encoder, no Jacobian and no reconstruction. This control is missing, and so is any calibration check of the added covariance; only the RelRMSE of the ensemble mean is reported. The comparison with the LSTM encoder changes two things at once: a forward vs an inverse model, and the form of the likelihood.
- The benchmarks are easier than the high-dimensional framing suggests. In all three, the initial physical state is shared or known and only one or two parameters are uncertain, and LDNet starts every trajectory from , so the task is mostly low-dimensional parameter identification. Figs. 2–3 fit this reading: on Kolmogorov, once the parameter error is down to 1–2%, the LD-EnFF-OT state error keeps growing (about 1% → 5.7%), following the surrogate. The F2P results are also hard to reconcile with the method. With Monte Carlo guidance the guided velocity is a convex combination of conditional velocities, so Eq. 46 with limits the parameter change to a few thousandths (normalized) per cycle; yet Fig. 3 shows F2P moving Re by several tenths within about 15 cycles. There is no test with uncertain initial fields or model error.
Questions for authors
- How does the same LDNet + EnFF perform with the direct likelihood ? Please report calibration (CRPS, spread/skill or coverage) for both.
- How does F2P achieve the parameter updates in Fig. 3, given Eq. 46 and (coordinates, integration interval, any inflation or jitter)? Does the implementation differ from Appendix B.2? Under OT, can parameter values leave the set spanned by the initial 20 samples?
- How does the method perform when the initial state is perturbed and not fully determined by ? What is the runtime compared with LETKF, given that efficiency is part of the motivation?
8 comments
Response to Reviewer MLYB (2/6)
Response to W1 and Q1. We thank the reviewer for pointing out the missing comparison. We agree that the comparison with the LSTM encoder changes more than one thing at once, including the direction of the observation map (inverse vs. forward) and the form of the likelihood.
Ablation (new Appendix E.3). As suggested, we ran the same LDNet + EnFF-OT with the direct likelihood , without the encoder and without the Jacobian, keeping everything else unchanged. This is a controlled comparison of the two likelihoods with the same forward operator. As calibration measure we report the spread–skill ratio (SSR) of the reconstructed full state.
| Benchmark | Likelihood | Final RelRMSE (%) | SSR |
|---|---|---|---|
| Kolmogorov Flow | Direct | 6.70 ± 0.56 | 0.205 ± 0.081 |
| Latent (ours) | 5.74 ± 0.26 | 0.536 ± 0.083 | |
| Tsunami Modeling | Direct | 5.99 ± 0.55 | 0.398 ± 0.124 |
| Latent (ours) | 6.53 ± 0.25 | 0.701 ± 0.122 | |
| Atmospheric Modeling | Direct | 4.80 ± 0.43 | 0.113 ± 0.017 |
| Latent (ours) | 4.42 ± 0.21 | 0.232 ± 0.025 |
The table reports mean standard deviation over 20 test trajectories (per trajectory, the RelRMSE is averaged and the SSR pooled over three random seeds). The spread–skill ratio (SSR) measures the ensemble spread relative to the estimation error, with values closer to one indicating better calibration and values below one indicating an overconfident ensemble.
Comparison with the LSTM-based baseline. LDNet+LSTM EnFF-OT shares the LDNet and the EnFF-OT analysis with both variants, but its observation model differs entirely: an inverse LSTM encoder, a likelihood defined in the latent space around the LSTM estimate, and its own covariance. Both forward variants attain lower final errors on all three benchmarks (6.70 / 5.99 / 4.80% and 5.74 / 6.53 / 4.42%, versus 13.09 / 9.87 / 6.18%). Since several components differ, this comparison shows that the observation model built on the forward operator outperforms the LSTM-based one, but it does not attribute the gap to a single component, such as the direction of the map or the space in which the likelihood is evaluated. The reviewer’s second factor, the form of the likelihood, is isolated by the controlled comparison above.
Effect of the latent likelihood (controlled comparison). Replacing the direct by the latent likelihood increases the SSR by a factor of 1.8–2.6 on all three benchmarks and roughly halves the variability of the final error across test trajectories. It also lowers the final error on Kolmogorov flow and atmospheric modeling, whereas the direct likelihood is slightly more accurate on tsunami modeling; there it even falls slightly below the oracle surrogate run with the true parameter (Appendix E.2). A likely reason is that selecting the members that best fit each observation partly compensates for surrogate error; this comes at the cost of an overconfident ensemble (SSR 0.40). The revised paper accordingly presents the empirical benefit of the latent likelihood mainly as improved calibration (Section 4.2 and the contributions) and robustness across trajectories (Appendix E.3). The similar accuracy is consistent with our setting: the parameters are time-invariant, the surrogate dynamics are deterministic, and the initial state carries no uncertainty beyond the parameters, so each member’s latent trajectory is determined by its parameter, up to the -scale jitter added at each analysis, and concentrating the ensemble on the best-fitting members mainly degrades calibration rather than accuracy (Appendix E.3). When the forecast is subject to unpredictable state errors, for example from uncertain initial conditions or stochastic model error, we expect this concentration to degrade accuracy as well, which is where the better-conditioned latent likelihood should matter most.
Response to Reviewer MLYB (3/6)
Why the latent likelihood helps (new Appendix B.4). Building on the likelihood-weighted resampling characterization of EnFF by Transue et al. (2025), we analyze how observation encoding affects the likelihood weights. Under an affine Gaussian approximation with fixed covariance, we prove that the latent likelihood reduces the observation-averaged variance of the log-likelihood weights relative to the direct physical likelihood (Proposition 1). We further establish a corresponding bound on the expected dispersion of normalized log-weights for any finite ensemble size (Corollary 2). These results provide a theoretical mechanism by which observation encoding can mitigate weight concentration in small ensembles, without guaranteeing larger effective sample size or improved filtering accuracy.
Calibration. Both ensembles remain underdispersed (SSR below one). For the latent likelihood, we identify several reasons, now discussed in Appendix E.3: late in the window, the full-state error is dominated by surrogate error that is largely common to all members (Appendix E.2) and is therefore only partly reflected in the ensemble spread; the selection-based analysis gradually reduces ensemble diversity (Corollary 5); and the approximation error of is not modeled (Appendix D.2). Covariance inflation or an explicit model-error term would further improve calibration, which we leave to future work.
Response to Reviewer MLYB (1/6)
We thank the reviewer for the careful reading of both the main text and the appendices. Several of the reviewer’s observations were correct and led to substantive revisions, summarized below and in the General Response.
W1. The main new component, the VAE-based latent likelihood, is not tested against the obvious simpler choice. Relative to LD-EnSF, the changes are EnSF → EnFF and the LSTM inverse encoder → VAE plus forward operator, so the likelihood is the key new piece. Since Hα already predicts observations, one can use N(y;Hα(κ),Γy) directly, with no encoder, no Jacobian and no reconstruction. This control is missing, and so is any calibration check of the added covariance; only the RelRMSE of the ensemble mean is reported. The comparison with the LSTM encoder changes two things at once: a forward vs an inverse model, and the form of the likelihood.
Q1. How does the same LDNet + EnFF perform with the direct likelihood N(y;Hα(κ),Γy)? Please report calibration (CRPS, spread/skill or coverage) for both.