- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this submission, the authors propose a neural network to predict a preconditioner for the 4D-Var data assimilation algorithm. This algorithm relies on Gauss-Newton steps, each requiring a linear solve that is typically ill-conditioned. Existing methods rely on using preconditioning vectors from previous time steps, which is suboptimal because eigenspaces rotate across assimilation windows. The authors propose learning a mapping between the preconditioning vectors of an assimilation window and those of subsequent windows using supervised learning. A spectrally weighted training loss is designed to learn an optimal preconditioning map. Applications to the Lorenz 96 system and a quasigeostrophic model are presented, along with ablation studies. The method considerably improves the condition number of the 4D-Var linear system, thereby enhancing data assimilation accuracy at a fixed computational budget. Results suggest that the method exhibits robustness to changes in underlying dynamics.
Critical strengths
Four-dimensional data assimilation is a key method in operational weather forecasting. Improving its computational efficiency is an important and well-motivated problem.
The proposed method is evaluated on challenging, turbulent physical systems.
The numerical experiments demonstrate significant improvements, bringing performance close to the oracle baseline.
The ablation studies clearly confirm the utility of the proposed spectral loss weighting.
Critical weaknesses
1. Clarity in method exposition
The exposition of the method could be clearer. The paper introduces various matrix notations throughout with sporadic references, making the main objective difficult to follow.
Mathematical quantities are introduced without proper context. For example, Lanczos vectors are never formally defined, and the "invariant subspace matrix" should be precisely defined in terms of its eigendecomposition. Additionally, the exact output dimensions of the network prediction are not clearly specified.
Intermediate linear algebra derivations lack sufficient justification. For instance, the claim that the matrix is tridiagonal is inadequately explained, and its physicalinterpretation remains vague.
The Limited-Memory Preconditioner (LMP) framework is not properly defined in the text.
2. Computational cost and scalability
A major concern regarding the proposed approach is its computational cost. The method relies on extensive linear algebra operations on high-dimensional objects. In particular, the neural network outputs a matrix whose size scales with the state dimension, raising concerns about practical deployment in large-scale numerical weather forecasting.
The computational complexity of the method should be thoroughly analyzed with respect to state dimension and subspace dimension , and its applicability to operational weather-scale systems should be critically assessed.
The feasibility and challenges of obtaining ground-truth preconditioning vectors in large-scale systems need to be explicitly discussed.
Questions for authors
- Could the authors elaborate on what primarily causes the system matrix to be ill-conditioned in this setup?
- Could the authors provide a more thorough theoretical and empirical comparison of their method with LMP?
- As noted by the authors, the method's computational requirements for large-scale systems depend heavily on implementation choices. Recent works operate variational data assimilation in latent spaces to reduce the scale of optimization variables [1]. Could the authors discuss whether their neural preconditioner can be adapted to or benefit from such latent-space frameworks?
References
[1] Fan, H., Bai, L., Fei, B., Xiao, Y., Chen, K., Liu, Y., ... & Gentine, P. (2026). Physically consistent global atmospheric data assimilation with machine learning in latent space. Science Advances.
2 comments
Thank you for the detailed response. It clarifies the method development and I encourage the authors to include these clarifications in the revised versions of the submission.
The point on computational complexity confirms my concern about larger scale applications. The number of model evaluations for obtaining the ground truth vectors is , with the number of samples, which is at least in the thousands. In operational data assimilation, the dimension is at least of order , so I imagine must be at least a few hundreds. As mentioned in the discussion with reviewer mPew, the system matrices are not directly available, requiring further approximations. Furthermore, linear algebra operations involving the neural network scale with , and are thus expensive. These costs look prohibitive for the method in an operationnal setting, and it is not clear if it would scale.
For these reasons, and because of the general lack of clarity in the submission, I maintain my score.
Response to Reviewer BTXh
Thank you for highlighting the spectrally weighted loss and the experiments on the two dynamical systems, and for identifying where the notation, computational cost, and connections to prior work need clarification.
1. Sources of ill-conditioning and notation
The original system matrix A of 4DVar contains the background precision matrix and a curvature contribution from each observation time. After the control variable transform, the system matrix takes the form S=I+G⊤G, where G propagates background-scaled initial perturbations to observation space and weights them by observation uncertainty. Directions that the observations constrain strongly or that the dynamics amplify can have large curvature. In weakly observed directions, the curvature stays close to the background level. The resulting spread in eigenvalues makes the 4DVar system harder to solve.
Section 3.1 now distinguishes an exact invariant subspace from a truncated Lanczos subspace. For a set of eigenvectors Uj of S, we have SUj=UjΛj. Starting from q1, k Lanczos steps produce an orthonormal basis V of span{q1,Sq1,…,Sk−1q1} before breakdown. The three-term recurrence makes TGT=V⊤SV tridiagonal in exact arithmetic. For any coefficient vector y, the quadratic form y⊤TGTy gives the curvature along Vy. Weighting the training loss by TGT thus gives greater weight to high-curvature directions. Section 3.2.1 also specifies that the network output is reshaped into a d×k matrix and that QR factorization yields the predicted basis Q.
2. Theoretical and empirical comparison with LMP
Thank you for asking us to clarify the relationship to LMP. Our LMP baseline reuses the subspace from the previous window. Our method uses the historical basis and its associated analysis increment to predict a subspace for the current window. The purpose of prediction is to align the preconditioning subspace more closely with the current operator. We then solve the linear system with PCG.
Appendix A.1 motivates this distinction by examining how the target subspace evolves between assimilation windows. The principal-angle measurements show that neighboring windows can have similar subspaces overall while the historical subspace is still imperfectly aligned with current spectral directions. This persistence makes the historical basis informative, and the window-to-window variation motivates updating it through prediction. We will make this connection explicit in the main text and refer readers to Appendix A.1.
In the additional comparison, we form T=Q⊤SQ using the current operator for each basis and apply the same rule α=tr(T)k. With the historical and predicted bases, respectively, the mean iteration counts are 9.53 and 7.87 on L96, and 274.70 and 268.48 on QG. Under matched construction rules, prediction therefore reduces mean iterations by 17.4% on L96 and 2.3% on QG. This quantifies the additional benefit of prediction in each system.
3. Computational complexity, training labels, and operational feasibility
For state dimension d and subspace dimension k, the network output and the stored basis each require O(dk) space. In addition to network inference, setup requires k applications of the current operator. Thin QR factorization and assembly of Q⊤(SQ) require O(dk2) arithmetic operations, and factoring the projected matrix costs O(k3). Once these factors are available, each preconditioner application costs O(dk+k2).
We generate training labels offline by running Lanczos on the operator for each training window. Obtaining the k-step basis and its projection requires approximately k operator applications per sample and storage for a d×k basis. During operational use, the network reads historical information to predict the current basis. We have also conducted preliminary tests in an operational assimilation system. See Response 5 to Reviewer mPew.
4. Connection to latent-space assimilation
Thank you for pointing us to Fan et al. (2026), who perform variational assimilation in a learned, low-dimensional representation of the atmospheric state. If the latent inner loop yields a symmetric positive definite system, a subspace predictor could be trained for that operator. For latent dimension m and fixed k, the basis would contain mk entries instead of dk. Section 4.5 cites this work and identifies the combination as a direction for future research.