- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a training-free data assimilation method using sparse observations to correct model predictions. It constructs a Gaussian mixture distribution from model predictions and measures the covariance between observed and hidden state variables to update states that are not need to recover. This method first uses a linear observation model to sample candidate states, then uses the true nonlinear likelihood to correct their weights. These weighted samples are used to estimate a velocity of the flow matching from Gaussian distribution to posterior distribution.
The experiments on Double-Well, Lorenz–96, and Kolmogorov flow prove the method's ability in capturing multiple posterior modes and improves the estimation of unobserved states and a significant improve from baselines.
Critical strengths
-
The method combines covariance-aware transport with nonlinear likelihood correction. By utilizing this, the method allows sparse observations to update unobserved states while preserving complex posterior distributions, while previous approaches like Ensemble Kalman Filter, rely mainly on mean and covariance information and may fail to capture multiple posterior modes.
-
The paper provides detailed proofs and derivations to support the proposed method.
Critical weaknesses
-
The paper are not clearly presented. The main figures provide limited information and seems have some errors, this need to double check before submission.
-
The dynamical model is also important for data assimilation, but the paper does not clearly show its prediction accuracy before the assimilation update. The EnKF results of Kolmogorov flow in Figure 2 appear noisy, and this may indicates implementation issues whether the update improves the model predictions or instead degrades them. Several baselines perform unexpectedly poorly. The large performance gaps need further explanation and validation to establish a fair comparison.
Questions for authors
How large is the prediction error when running the dynamical model without data assimilation?
If both the ground-truth data and the dynamical model predictions are generated using the same numerical solver, how are differences between their trajectories are introduced in experiments?
Can FREESIA also work when the governing equations are unknown and predictions come from learned / black-box dynamical models?
3 comments
Thanks for the responses and additional experiments. While the problem in figure 2 and results of Kolmogorov flow is still not solved. To best of my understanding, the prediction errors are introduced by using different setting in numerical solver, and the EnKF results in Kolmogorov flow indicates that data assimilation may even disrupt the underlying dynamics rather than improve the predictions.
We thank the reviewer for the follow-up comments.
Comment 1.
To best of my understanding, the prediction errors are introduced by using different setting in numerical solver.
Response:
In our experiments, the truth and forecast ensembles are initialized from different states in the post-spin-up dynamical regime. In chaotic systems, these initial-state differences naturally lead to diverging trajectories, even when the same dynamical model and numerical solver are used.
While Lorenz-96 also uses different numerical time steps for the truth and forecast, initial-state uncertainty is itself sufficient to generate prediction errors and motivate data assimilation.
Comment 2.
The EnKF results in Kolmogorov flow indicates that data assimilation may even disrupt the underlying dynamics rather than improve the predictions.
Response:
We have added comprehensive final-time comparisons of all seven methods under both observation regimes in Figures 18 and 19 of the revised manuscript.
Under 1% monotone observations, EnKF-MDA and LETKF successfully recover the dominant vorticity structures, achieving RMSEs of 0.148 and 0.218, respectively. EnKF also recovers parts of the flow, with an RMSE of 0.377.
The 2% non-injective observation experiment is considerably more challenging because the absolute-arctangent operator introduces sign ambiguity in the observed velocity components. This ambiguity poses particular difficulties for covariance-based linear updates.
Moreover, vorticity involves spatial derivatives of the velocity field, which amplify local estimation errors. The displayed vorticity fields characterize the accuracy of the final analysis rather than the accuracy of subsequent forecasts. The latter is a separate quantity from the reconstruction accuracy presented in these figures.
We have also revised the method illustrations and added a complete algorithm overview in Figures 1 and 2 to improve the clarity of the presentation.
Response to Reviewer 6IoV
We thank the reviewer for the constructive comments and suggestions regarding the numerical experiments and the applicability of FREESIA. We provide point-by-point responses below.
Comment 1.
How large is the prediction error when running the dynamical model without data assimilation?
Response:
We additionally evaluated the prediction error without data assimilation in the 1,000-dimensional Lorenz-96 experiment under stride-4 observations.
Without assimilation, the RMSE is 3.7248. Both the classical ensemble filters and FREESIA substantially reduce this prediction error by incorporating observational information. For example, EnKF-MDA achieves RMSE 2.056, while FREESIA further reduces it to 1.575, demonstrating the effectiveness of data assimilation in correcting model predictions.
Comment 2.
If both the ground-truth data and the dynamical model predictions are generated using the same numerical solver, how are differences between their trajectories are introduced in experiments?
Response:
The ground-truth trajectory and forecast ensemble are initialized from different states. Even when propagated using the same governing equations and numerical solver, different initial conditions lead to distinct trajectories, particularly in chaotic systems such as Lorenz-96 and Kolmogorov flow.
In Lorenz-96, both the truth and initial forecast ensembles are sampled from the post-spin-up chaotic regime. The truth uses a time step of 0.01, while the forecast uses 0.02.
For Kolmogorov flow, the truth and forecast ensembles are similarly initialized from the developed-flow regime after spin-up. These different initial states naturally lead to diverging trajectories during model propagation.
Comment 3.
Can FREESIA also work when the governing equations are unknown and predictions come from learned / black-box dynamical models?
Response:
Yes. FREESIA only requires a forecast ensemble generated by the prediction step and an evaluable observation operator. The forecast ensemble is used to construct the Gaussian-mixture prior surrogate, from which the posterior transport is derived.
Since the analysis step does not require derivatives or an explicit formulation of the dynamical model, the forecast ensemble can also be generated by learned or black-box prediction models. Therefore, FREESIA can be directly integrated with data-driven forecasting frameworks.