acceptodds
Under review as a conference paper at ICLR 2027

FERA: Forecast-Conditioned Error Representations for Data Assimilation

Abstract

Data assimilation is fundamental to weather prediction because forecasts must be continually corrected using observations that constrain only a small fraction of the atmospheric state. The central challenge is not merely fitting these observations, but propagating their information to unobserved locations and variables through a plausible representation of forecast uncertainty. Classical climatological covariances are stable but cannot adapt their correction geometry to the current flow, while small ensembles provide flow dependence only through a rank-limited sample. Neural methods offer more expressive representations, but often perform assimilation directly in learned latent spaces or retain nonlinear neural decoders inside the online optimization, making the connection between observations, physical corrections, and inference less explicit. We introduce FERA (Forecast-conditioned Error Representation for Assimilation), which separates learned error representation from online Bayesian inference. A forecast-conditioned neural transform constructs multivariate physical-space correction patterns, while held-out-variable and sparse-observation objectives train these patterns to recover errors from partial information. During assimilation, the neural transform is frozen: observations determine the correction coefficients through a linear-Gaussian Bayesian update, while the ensemble refines their prior uncertainty and a full-rank historical component preserves directions beyond the member span. For linear observations, this yields a unique, initialization-free analysis with an explicit observation-to-correction map, while retaining forecast-adaptive error structure across cycles. In month-long global cycling experiments with a 69-channel AI weather model, FERA outperforms the strongest matched baseline in 46 of 48 ensemble-size-diagnostic comparisons. In a fixed-input replay, FERA shows lower numerical sensitivity to equivalent observation reorderings than the tested nonlinear neural variational baseline, without optimizer-initialization dependence. Ablation studies further demonstrate the effectiveness of both forecast conditioning and partial-information training for sustained cycling. Together, these results link forecast-adaptive error learning to an explicit Bayesian analysis, providing a basis for reliable and interpretable assimilation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
data assimilation, representation learning, Bayesian inference, generative modeling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_P3CFOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FERA, a data assimilation method that learns a forecast-conditioned error representation to recover forecast error from partial information. It uses held-out-variable and sparse observation objectives for training, then combines the learned representation with historical and ensemble statistics in a linear-Gaussian Bayesian update. FERA outperforms the strongest matched baseline in 46 of 48 ensemble-size diagnostic comparisons in a month-long global cycling experiment. The benefits of forecast conditioning and partial information training are supported by ablation studies.

Critical strengths

The main strength is that FERA connects representation learning to recovery from partial information, which the representation must support during assimilation. The retraining ablations show that the held-out-variable and sparse-observation objectives improve cycling DA.

Critical weaknesses

The global cycling experiments give useful evidence for FERA’s error representation, but all use noise-free observations. Handling observation noise is a basic requirement in DA, and the baselines (VAE-Var and LoRA-EnVar) have already been tested with real observations. A controlled cycling comparison with noisy observations is the main missing experiment.

Questions for authors
  1. How did you choose the checkpoints and observation-error scalings for FERA and the baselines in the main comparison? In particular, what validation data, selection criteria, and tuning budgets did you use? These details would help readers interpret the reported results and guide users in tuning FERA.

  2. I would be inclined to change my recommendation if there is a controlled cycling comparison with noisy observations.

5 comments
AuthorsOpenReview ↗

Rebuttal from Authors (1/3)

Thank you for recognizing the value of learning forecast-conditioned error representations that recover errors from partial information. We address your questions and the concern about noise-free observations below using completed observation-error calibration, controlled noisy-observation cycling, and additional seasonal and variable-withholding experiments. FERA retains its advantage after baseline R tuning, with larger gains as observation noise increases over the reported range.

Q1. Checkpoint and observation-error scaling selection

The checkpoints used in the original comparison were selected empirically during method development, rather than through a matched checkpoint-search protocol. In particular, the VAE-Var baseline was retrained following the architecture, hyperparameters, and training settings of the original VAE-Var paper (Xiao et al., 2025).

To make error-scaling selection more systematic and comparable, we applied a common calibration protocol for all three methods. We set R=sRpaperR=sR_{\mathrm{paper}}, preserving each method's channel ratios, and test the same six variance multipliers on 20–24 December 2023. Each candidate runs for 16 six-hour cycles, with cycles 5–16 used for selection. The score is Qm(s)=112∑cWRMSE‾m,s,cWRMSE‾m,1,c,Q_m(s)=\frac{1}{12}\sum_c \frac{\overline{\mathrm{WRMSE}}_{m,s,c}}{\overline{\mathrm{WRMSE}}_{m,1,c}},

where the twelve diagnostics are z/q/u/v/t at 500/850 hPa, mean sea-level pressure, and 2-m temperature. Lower scores are better; scores are normalized within each method, so they measure scaling sensitivity rather than cross-method accuracy.

Variance multiplier s3DVar scoreVAE-Var scoreFERA score
0.06250.9963731.0045101.137979
0.250.9986540.9947661.022161
0.51.0005360.9981020.999379
1 (original R)1.0000001.0000001.000000
21.0097721.0023321.022699
41.0264701.0213951.066725

The selected scalings are 0.0625, 0.25, and 0.5 for 3DVar, VAE-Var, and FERA, respectively. Relative to using the original R in this calibration, the normalized score improves by 0.363%, 0.523%, and 0.062%. Thus, all three methods benefit modestly, and FERA's original R already performs very close to its best tested clean-observation setting. This is consistent with learned correction structure and direction being more important for stable assimilation than fine control of the overall observation-error magnitude within the tested range. The noisy experiments below separately test whether its advantage survives a broader baseline R search.

AuthorsOpenReview ↗

Rebuttal from Authors (2/3)

Q2 and the weakness. Controlled cycling with noisy and incomplete observations

We have completed 29-day cycling experiments (116 six-hour cycles, 3 January–1 February 2024) comparing FERA, 3DVar, and VAE-Var. Each cycle samples 1,000 horizontal columns on the 721×1440 grid and observes all 69 channels there, following the synthetic sparse-column observation convention used in prior weather-DA work (Huang et al., 2024; Xiao et al., 2025). The methods share the forecast model, initial background, observation locations, and noise realizations, then follow their own cycling trajectories.

At each observed location, noise is generated as yc=(Hxref)c+κScξc,Sc=σFengWu,cdc,ξc∼N(0,1).y_c=(Hx_{\mathrm{ref}})_c+\kappa S_c\xi_c,\qquad S_c=\sigma_{\mathrm{FengWu},c}d_c, \qquad \xi_c\sim\mathcal N(0,1).

Here σFengWu,c\sigma_{\mathrm{FengWu},c} is the forecast-model normalization standard deviation and dcd_c is the fixed channel error coefficient. Gaussian draws are independent across observed locations and channels and refreshed each cycle; methods receive identical draws for each condition. We report κ={0, 0.1, 0.25, 0.5}. For scale, κ=0.25 corresponds to noise standard deviations of approximately 25.15 m²/s² for z500, 0.129 g/kg for q850, 0.500 m/s for u500, and 0.164 K for t500.

The injected noise and the solver's R are separate quantities: κ scales noise standard deviation, while R=αRcleanR=\alpha R_{\mathrm{clean}} scales the solver variance. We test α={1, 10, 30, 100, 300, 1000} for each baseline, retaining 3DVar's 10 outer iterations/LBFGS maxiter=5 and VAE-Var's 10/maxiter=10. FERA retains its direct posterior solve and calibrated R. Thus, α=1 refers to the newly calibrated R described under Q1.

The table reports each baseline's post-hoc best joint scaling on the monthly grid, with one α shared across all diagnostics. Improvements are 100 meanc(1−EˉFERA,c/Eˉbaseline,c)100\,\mathrm{mean}_c(1-\bar E_{\mathrm{FERA},c}/\bar E_{\mathrm{baseline},c}), using all twelve diagnostics and all 116 cycles.

Observation noise κSelected 3DVar αSelected VAE-Var αFERA improvement over 3DVarFERA improvement over VAE-Var
0113.77%3.06%
0.110105.44%4.40%
0.251003026.13%20.12%
0.530010044.68%40.39%

FERA's advantage becomes larger as observation noise increases within the reported range, even after selecting the best tested scaling for each baseline. Across the three reported nonzero noise levels and all six baseline scalings, FERA achieves lower time-mean WRMSE on every diagnostic (432/432 noise × scaling × baseline × diagnostic comparisons).

The variable-level results show that the gains extend across geopotential, humidity, wind, and temperature. Below are absolute analysis WRMSEs averaged over the same 116 cycles; lower is better. The selected α is shared by all variables, as specified above.

Variable (unit)3DVar κ=0VAE-Var κ=0FERA κ=03DVar κ=0.25VAE-Var κ=0.25FERA κ=0.25
z500 (m²/s²)48.8747.0044.2188.5280.9145.09
z850 (m²/s²)39.3338.0638.7968.0057.8639.61
q500 (g/kg)0.47300.46870.46680.52650.49120.4697
q850 (g/kg)1.19431.17771.13221.38401.25681.1380
u500 (m/s)2.1192.0802.0182.6022.3812.043
u850 (m/s)1.6911.6721.6552.0081.8471.672
v500 (m/s)2.0822.0342.0092.5212.3352.037
v850 (m/s)1.6781.6591.6372.0041.8511.653
t500 (K)0.5720.5550.5540.7710.7020.563
t850 (K)0.8510.8530.8451.0480.9880.854

For example, FERA reduces z500 WRMSE by 49.07%/44.28% relative to 3DVar/VAE-Var at κ=0.25. At this noise level, FERA achieves lower WRMSE on all ten pressure-level diagnostics, including humidity, wind, and temperature.

AuthorsOpenReview ↗

Rebuttal from Authors (3/3)

Cross-season and observation-seed validation. We additionally evaluated 72 seven-day trajectories across four 2023 seasonal windows (January, April, July, and October), three observation seeds, κ=0/0.25, and all three methods. These windows lie outside the 2024 NMC training year. Checkpoints and R remain fixed across seasons and seeds, and the methods share paired initial backgrounds and observations. The table averages paired relative improvements across the ten pressure-level diagnostics and all twelve season–seed conditions:

Observation noise κFERA improvement over 3DVarFERA improvement over VAE-Var
07.06%4.75%
0.2515.87%11.37%

FERA retains a positive mean advantage in each of the four seasonal windows at both noise levels. At κ=0.25, all 120/120 season × seed × variable comparisons favor FERA against each baseline.

To also represent incomplete variable coverage, we ran seven-day experiments (28 cycles) at κ=0 and 0.25, withholding all 13 humidity levels or all wind observations, including 10-m winds. We retain the full-observation R settings and iteration budgets, without retuning for the masks. The following table reports FERA's mean relative WRMSE reduction on the withheld variables at 500/850 hPa:

Withheld observationsκImprovement over 3DVarImprovement over VAE-Var
All humidity08.41%28.46%
All humidity0.259.68%10.10%
All wind022.70%23.65%
All wind0.2528.74%27.48%

The individual withheld-variable results are shown below. Humidity rows use the all-humidity mask; wind rows use the all-wind mask. Values are analysis WRMSE averaged over the complete 28 cycles; lower is better.

Withheld variable (unit)κ3DVar analysisVAE-Var analysisFERA analysis
q500 (g/kg)00.56220.71460.5364
q500 (g/kg)0.250.58960.60030.5403
q850 (g/kg)01.41651.82781.2434
q850 (g/kg)0.251.40501.39241.2503
u500 (m/s)04.05764.19543.0240
u500 (m/s)0.254.43644.44773.0441
u850 (m/s)02.90982.91052.5068
u850 (m/s)0.253.19703.10472.5316
v500 (m/s)03.97364.07432.8478
v500 (m/s)0.254.28234.29042.8635
v850 (m/s)02.90922.89332.2361
v850 (m/s)0.253.20503.07482.2548

FERA achieves lower mean analysis WRMSE than both baselines for all six withheld diagnostics at both noise levels. Its advantage therefore extends to variables without direct observations, supporting its effectiveness under incomplete variable coverage.

These results support FERA's effectiveness under both noisy observations and missing-variable coverage.

VAE-Var (Xiao et al., 2025) has also been evaluated with real observations. However, we do not have a matched set of processed observations and preprocessing configurations for that benchmark, which prevents a directly controlled comparison with its reported results. We will investigate real-observation assimilation in future work.

References

Reviewer_h2yqOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes FERA, a data assimilation method that performs state correction in a learned space of forecast-conditioned correction patterns. The correction coefficients are obtained through a linear-Gaussian solve. The paper reports improvements over several baselines in a synthetic-observation setup.

Critical strengths
  1. Learned correction patterns provide flexibility within a Bayesian framework.
  2. The direct solve avoids iterative optimization during assimilation.
  3. Improvements over the baselines.
Critical weaknesses
  1. The uncertainty analysis is very limited, so the robustness of the reported improvements across training runs and evaluation conditions remains unclear.
  2. The realism of the proposed setup is limited. The representativeness of the NMC training proxies for actual background errors needs further validation. Also, the setup is highly idealized, as acknowledged by the authors: it observes all channels without noise at the selected grid points.
Questions for authors

Q1. Do the gains persist under comparable tuning effort and additional cycling seeds? Please provide per-cycle scores and a manifest listing the training files and their data timestamps. Q2. How does the architecture of the neural network that learns the correction patterns influence the results? Q3. Could you extend the uncertainty analysis to assess variability across training seeds, observation locations and noise levels, initial conditions, and evaluation periods (weather conditions)? Please also examine sensitivity to longer bootstrap blocks and to the construction of the NMC training targets, and report whether the main performance gains remain consistent under these variations.

5 comments
AuthorsOpenReview ↗

Rebuttal from Authors (1/5)

Thank you for recognizing the flexibility of the learned correction patterns and the benefit of a direct Bayesian update. We address your concerns below using completed experiments on observation noise, missing variables, different weather periods and observation realizations, network depth, longer bootstrap blocks, and an alternative NMC construction. The main advantages remain across the tested observation and NMC variations.

Q1. Comparable tuning effort, additional cycling seeds, and reproducibility

In the original comparison, the model checkpoints were selected empirically during development, and the methods used their respective observation-error settings; we did not conduct a matched checkpoint search across methods. The VAE-Var baseline was retrained following the architecture, hyperparameters, and training settings of the original study (Xiao et al., 2025). To test observation-error tuning sensitivity with a matched budget, we applied a common calibration protocol: each method tests the same six variance multipliers, R=sRpaperR=sR_{\mathrm{paper}}, with s∈0.0625,0.25,0.5,1,2,4s\in\\{0.0625,0.25,0.5,1,2,4\\}. Each candidate runs for 16 six-hour cycles on 20–24 December 2023; cycles 5–16 determine selection. The score averages diagnostic-wise WRMSE ratios relative to that method's s=1s=1 result, giving equal weight to twelve diagnostics: z/q/u/v/t at 500/850 hPa, mean sea-level pressure, and 2-m temperature. Checkpoints remain fixed throughout calibration and subsequent evaluation. The scores below are normalized within each method and measure tuning sensitivity; lower is better.

Variance multiplier ss3DVarVAE-VarFERA
0.06250.9963731.0045101.137979
0.250.9986540.9947661.022161
0.51.0005360.9981020.999379
11.0000001.0000001.000000
21.0097721.0023321.022699
41.0264701.0213951.066725

The selected scalings are 0.0625, 0.25, and 0.5 for 3DVar, VAE-Var, and FERA, respectively. Relative to using the original R in this calibration, the normalized score improves by 0.363%, 0.523%, and 0.062%. Thus, all three methods benefit modestly, and FERA's original R already performs very close to its best tested clean-observation setting. This suggests that FERA's performance is not primarily driven by fine-tuning the observation-error magnitude, but rather by the effectiveness of its learned correction geometry.

Per-cycle scores (code/rebuttal/per_cycle_scores_10vars.csv), covering all ten main variables in the original 72 seasonal runs, 24 alternative-NMC runs, and six depth-3 runs, and a training-data manifest (code/rebuttal/training_data_manifest.csv) will be provided in the updated supplementary material. The manifest also lists January 2024 files for completeness, but these were excluded from model training.

AuthorsOpenReview ↗

Rebuttal from Authors (2/5)

Q2. How does the correction-pattern network architecture affect the results?

We agree that the learned correction representation is sensitive to network architecture; we do not assume that changing the depth leaves assimilation performance unchanged. The original six-layer design was selected through empirical tuning for the available NMC training data and optimization budget. As in neural representation learning more generally, network capacity must be balanced against the diversity and size of the training set, while ensuring that the reconstruction, held-out-variable, and sparse-observation objectives are all learned to reasonably low losses. A low reconstruction loss alone is insufficient if the resulting patterns cannot support inference from partial observations.

To examine this sensitivity, we trained 3-, 6-, and 9-layer models using the same configured training data, objectives, optimizer settings, and 20-epoch budget. The six-layer model achieves the lowest final loss on all four reported training metrics:

DepthTotal training lossReconstructionHeld-out projectionSparse posterior
30.312750.154180.759980.47423
60.295070.140210.742150.46318
90.358640.178380.861560.54052

Under this fixed training recipe, reducing the depth to three layers degrades the fit, while increasing it to nine does not improve the final training losses. The latter result does not establish that a deeper model is intrinsically worse: it may require a different optimization or regularization schedule. We also completed six paired seven-day evaluations of the three-layer model: January, April, and July 2023, each at κ=0\kappa=0 and 0.25 with observation seed 0. The three-layer model has higher time-mean analysis WRMSE in all 60 variable–condition comparisons, with an average relative increase of 33.46% at κ=0\kappa=0 and 33.16% at κ=0.25\kappa=0.25 across the ten diagnostics and three windows.

We therefore regard six layers as an empirically supported choice for the present dataset and training budget, not a universally optimal or architecture-invariant configuration. The depth comparison makes clear that sufficient representational capacity and effective optimization of all training objectives are important for obtaining correction patterns that work in cycling assimilation.

AuthorsOpenReview ↗

Rebuttal from Authors (3/5)

Q3 and the weaknesses: uncertainty, observation realism, and NMC-target sensitivity

Observation noise and missing variables. The completed 29-day comparisons of FERA, 3DVar, and VAE-Var contain 116 six-hour cycles and 1,000 random horizontal observation columns on the 721×1440 grid. Each column observes all 69 channels. At each observed location, yc=(Hxref)c+κScξc,Sc=σFengWu,cdc,ξc∼N(0,1).y_c=(Hx_{\mathrm{ref}})_c+\kappa S_c\xi_c, \qquad S_c=\sigma_{\mathrm{FengWu},c}d_c, \qquad \xi_c\sim\mathcal N(0,1). Here σFengWu,c\sigma_{\mathrm{FengWu},c} is the normalization standard deviation and dcd_c is the fixed channel error coefficient. Noise is independently drawn across locations and channels, refreshed each cycle, and paired across methods. Here we report κ∈0,0.1,0.25,0.5\kappa\in\\{0,0.1,0.25,0.5\\}. For scale, κ=0.25\kappa=0.25 gives noise standard deviations of approximately 25.15 m²/s² for z500, 0.129 g/kg for q850, 0.500 m/s for u500, and 0.164 K for t500.

The injected noise amplitude is distinct from the assumed solver variance. We tested all six baseline scalings, R=αRcleanR=\alpha R_{\mathrm{clean}}, with α∈1,10,30,100,300,1000\alpha\in\\{1,10,30,100,300,1000\\}, while FERA retains its calibrated R. The table below gives each baseline its post-hoc best joint scaling on the complete monthly grid, using one scaling across all diagnostics.

Noise κ\kappaBest tested 3DVar α\alphaBest tested VAE-Var α\alphaFERA WRMSE reduction vs 3DVarvs VAE-Var
0113.77%3.06%
0.110105.44%4.40%
0.251003026.13%20.12%
0.530010044.68%40.39%

These improvements average 100(1−EˉFERA,c/Eˉbaseline,c)100(1-\bar E_{\mathrm{FERA},c}/\bar E_{\mathrm{baseline},c}) over the twelve original diagnostics; Eˉ\bar E is each variable's time-mean analysis WRMSE. FERA's advantage increases with noise over this tested range, even after the additional baseline tuning. At each reported nonzero noise level, FERA has lower WRMSE on all twelve diagnostics across all six tested baseline scalings. At κ=0.25\kappa=0.25, for example, z500 WRMSE is 45.09 for FERA versus 88.52/80.91 m²/s² for 3DVar/VAE-Var, while q850 is 1.138 versus 1.384/1.257 g/kg.

The seasonal and variable-withholding experiments use fixed α=(1,1,1)\alpha=(1,1,1) for 3DVar/VAE-Var/FERA at κ=0\kappa=0, and (100,30,1)(100,30,1) at κ=0.25\kappa=0.25. The completed 28-cycle variable-withholding experiments remove either all 13 humidity levels or all wind channels, including 10-m winds, without mask-specific R tuning. FERA has lower analysis WRMSE than both baselines on all six withheld 500/850-hPa diagnostics at both noise levels. Representative results are:

Withheld diagnosticκ\kappa3DVarVAE-VarFERA
q850, all humidity withheld (g/kg)01.41651.82781.2434
q850, all humidity withheld (g/kg)0.251.40501.39241.2503
u500, all wind withheld (m/s)04.05764.19543.0240
u500, all wind withheld (m/s)0.254.43644.44773.0441

We also tested whether the partial-information training objectives remain useful with noisy observations. In the completed noise test with two five-epoch models, identical R, 500 observation columns, and eight cycles at κ=0.25\kappa=0.25, retaining the held-out and sparse-posterior objectives lowers mean diagnostic-wise analysis WRMSE by 21.78% relative to jointly removing them. All ten time-mean diagnostics improve, including 33.64% for z500 and 29.86% for q850. This supports the usefulness of partial-information training under noisy observations, without treating the joint ablation as separate estimates of each loss's contribution.

We completed the corresponding test with all humidity observations withheld, under the same five-epoch, 500-column, eight-cycle protocol. The complete-loss model reduces mean diagnostic-wise WRMSE by 41.88%. For the unobserved humidity variables, q500 WRMSE falls from 1.246 to 0.533 g/kg and q850 from 2.960 to 1.303 g/kg, reductions of 57.21% and 55.96%, respectively. The loss benefits therefore persist when noise and missing variable coverage occur together.

Reviewer_rWZMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies data assimilation: correcting weather forecasts using sparse observations. The authors propose FERA, which first generates a set of candidate error correction patterns conditioned on the current forecast, then uses a linear Bayesian solve to determine how to combine them. During training, the model learns both to reconstruct the full forecast error and to infer errors in unobserved regions or variables from partial observations. Experiments evaluate global cycling assimilation with FengWu across different ensemble sizes and observation densities, with retrained ablations examining individual design choices.

Critical strengths
  1. The core idea is clear and valuable. A neural network learns state-dependent correction patterns, while the online update remains directly solvable. This combines learned error structure with an efficient assimilation procedure that avoids nonlinear optimization through the neural network at each assimilation step, constituting a meaningful methodological contribution.
  2. The training objectives are well aligned with the practical problem. Reconstructing a full error field does not necessarily imply the ability to infer unobserved errors from sparse observations, and the proposed training tasks explicitly address this distinction. The experiments also go beyond single-step corrections to evaluate cycling assimilation and subsequent forecasts, with retrained ablations providing relatively direct support for the design choices.
Critical weaknesses
  1. My main concern is that the performance gains in the main experiments are affected by the comparison setup. The methods use different observation-error weights, and their hyperparameter tuning budgets are not standardized. These choices influence how strongly each method trusts the observations, making it difficult to separate the contribution of the proposed representation from that of weighting and tuning. In particular, some variable-wise improvements are small, and counting the number of wins alone does not establish whether the advantage is robust.
  2. The experiments provide all 69 variables at randomly selected grid locations without adding observation noise. Although spatially sparse, the observations are therefore highly informative at each sampled location. The current results do not sufficiently establish whether the learned correction patterns remain effective when variables are missing, observations are unevenly distributed, or measurement noise is present. This also limits the evidence supporting the cross-variable inference design.
  3. The numerical stability experiment uses only one weather state, and the appendix indicates that none of the iterative baselines passed the reported convergence checks. The results therefore primarily characterize performance under the tested computational budget and do not establish how the nonlinear methods would perform with sufficient optimization. The timing comparison also covers only the solve stage, excluding the cost of generating correction patterns and other components of the full assimilation procedure.
Questions for authors
  1. How much of FERA’s advantage remains under comparable tuning budgets and a systematic evaluation of each method’s sensitivity to observation weighting? Could the improvements be confirmed over additional weather periods or random seeds for observation sampling?
  2. Does FERA still outperform the baselines when observation noise is introduced or an entire variable type is unobserved? How does removing cross-variable training or sparse-observation training affect performance in these settings? These experiments could more directly establish the contribution of each training objective.
4 comments
AuthorsOpenReview ↗

Rebuttal from Authors (1/4)

Thank you for recognizing the value of combining forecast-conditioned correction patterns with a direct Bayesian update. We address your questions and weaknesses below using completed experiments on observation weighting, noisy and missing observations, additional seasons and observation seeds, training-loss ablations, and numerical stability across multiple weather states. FERA retains its advantage across the tested observation settings, and the matched loss ablations support the contribution of partial-information training. We also clarify the scope of the convergence and timing comparisons.

Q1 and Weakness 1. Comparable observation-weight tuning and additional evaluation periods

In the original comparison, model checkpoints were selected empirically during development, while each method used its own observation-error settings; the original selection was not a matched checkpoint search. The VAE-Var baseline was retrained following the architecture, hyperparameters, and training settings of the original VAE-Var paper (Xiao et al., 2025). For the additional experiments, we used a common observation-error calibration protocol: R=sRpaperR=sR_{\mathrm{paper}}, with six variance multipliers s∈0.0625,0.25,0.5,1,2,4s\in\\{0.0625,0.25,0.5,1,2,4\\}, preserving each method's channel ratios. Each candidate is evaluated on 20–24 December 2023 for 16 six-hour cycles, and cycles 5–16 determine selection. The checkpoints remain fixed throughout calibration and evaluation; FERA uses its final epoch-20 model.

The selection score averages each diagnostic's time-mean WRMSE divided by its value at s=1s=1, over z/q/u/v/t at 500/850 hPa, mean sea-level pressure, and 2-m temperature. This normalization gives the twelve diagnostics equal weight. Scores below are normalized within each method and measure tuning sensitivity.

Variance multiplier ss3DVarVAE-VarFERA
0.06250.9963731.0045101.137979
0.250.9986540.9947661.022161
0.51.0005360.9981020.999379
11.0000001.0000001.000000
21.0097721.0023321.022699
41.0264701.0213951.066725

All three methods improve after selection, by 0.363%, 0.523%, and 0.062%, respectively. Thus, FERA's original scaling was already close to its best tested clean-observation setting. The broader noisy-observation search below further tests whether its advantage can be explained by baseline observation weighting.

We completed 72 seven-day cycling experiments: four seasonal windows in January, April, July, and October 2023, three observation seeds, two noise levels (κ=0,0.25)(\kappa=0,0.25), and three methods. Each run contains 28 six-hour cycles. Checkpoints and R settings are fixed across seasons and seeds; each matched comparison shares its initial background, observation locations, and noise realizations. These windows do not overlap the NMC training initialization or valid times.

The table gives FERA's mean relative analysis-WRMSE reduction, averaged over the ten pressure-level diagnostics and three paired observation seeds. Positive values favor FERA.

Seasonal windowκ=0\kappa=0, vs 3DVarκ=0\kappa=0, vs VAE-Varκ=0.25\kappa=0.25, vs 3DVarκ=0.25\kappa=0.25, vs VAE-Var
January4.36%2.27%20.20%14.39%
April5.56%3.18%16.82%10.81%
July5.89%4.04%13.45%10.39%
October12.43%9.51%13.01%9.89%
Overall7.06%4.75%15.87%11.37%

FERA retains a positive mean advantage over both baselines in every seasonal window at both noise levels. These effect sizes provide evidence beyond counting variable-wise wins in a single monthly trajectory.

AuthorsOpenReview ↗

Rebuttal from Authors (2/4)

Q2 and Weakness 2. Noisy observations, missing variables, and observation coverage

We completed 29-day experiments with 116 six-hour cycles, including the baseline R scans reported below. The period is 3 January to 1 February 2024. Each cycle samples 1,000 columns on the 721×1440 grid, observing all 69 channels at each column. Methods share the initial background and observation locations, then follow their own cycling trajectories. Observations are generated by yc=(Hxref)c+κScξc,Sc=σFengWu,cdc,ξc∼N(0,1),y_c=(Hx_{\mathrm{ref}})_c+\kappa S_c\xi_c,\qquad S_c=\sigma_{\mathrm{FengWu},c}d_c,\qquad \xi_c\sim\mathcal N(0,1), where σFengWu,c\sigma_{\mathrm{FengWu},c} is the forecast-model normalization standard deviation and dcd_c is the fixed channel error coefficient. Noise is independent across observed locations and channels, refreshed each cycle, and shared across methods. For scale, κ=0.25\kappa=0.25 gives noise standard deviations of approximately 25.15 m²/s² for z500, 0.129 g/kg for q850, 0.500 m/s for u500, and 0.164 K for t500.

The noise amplitude κ\kappa is separate from the solver's assumed error variance. For each baseline, we evaluate all six scalings R=αRcleanR=\alpha R_{\mathrm{clean}}, α∈1,10,30,100,300,1000\alpha\in\\{1,10,30,100,300,1000\\}, where RcleanR_{\mathrm{clean}} is selected by the calibration above. FERA keeps its calibrated R. We retain the original iteration settings: 10 outer iterations with L-BFGS maxiter=5 for 3DVar, and 10/maxiter=10 for VAE-Var.

The table gives each baseline its best joint scaling on the complete monthly grid, selected post hoc using the equally weighted twelve-diagnostic normalized score. One scaling is shared across all diagnostics. These are sensitivity results, not an independent validation-based selection.

Noise κ\kappaBest tested 3DVar α\alphaBest tested VAE-Var α\alphaFERA improvement over 3DVarFERA improvement over VAE-Var
0113.77%3.06%
0.110105.44%4.40%
0.251003026.13%20.12%
0.530010044.68%40.39%

The seasonal and missing-variable experiments use fixed α=(1,1,1)\alpha=(1,1,1) for 3DVar/VAE-Var/FERA at κ=0\kappa=0, and (100,30,1)(100,30,1) at κ=0.25\kappa=0.25.

Here improvements average 100(1−EˉFERA,c/Eˉbaseline,c)100(1-\bar E_{\mathrm{FERA},c}/\bar E_{\mathrm{baseline},c}) over the twelve diagnostics, with Eˉ\bar E denoting time-mean analysis WRMSE. FERA's advantage increases with noise over the reported range, even after this additional baseline tuning. At each nonzero level κ∈{0.1,0.25,0.5}\kappa\in\{0.1,0.25,0.5\}, FERA has lower time-mean WRMSE than both baselines on all twelve diagnostics across all six tested baseline scalings. For example, at κ=0.25\kappa=0.25, z500 WRMSE is 45.09 for FERA, versus 88.52/80.91 m²/s² for 3DVar/VAE-Var; q850 is 1.138, versus 1.384/1.257 g/kg.

We also completed seven-day, 28-cycle variable-withholding experiments. They test κ=0\kappa=0 and 0.25, withholding either all 13 humidity levels or all wind observations, including 10-m winds. R and iteration settings are retained from the corresponding full-observation cases, without mask-specific tuning. Below are time-mean analysis WRMSEs for the unobserved variables; humidity rows use the humidity mask and wind rows use the wind mask.

Withheld variable (unit)3DVar, κ=0\kappa=0VAE-Var, κ=0\kappa=0FERA, κ=0\kappa=03DVar, κ=0.25\kappa=0.25VAE-Var, κ=0.25\kappa=0.25FERA, κ=0.25\kappa=0.25
q500 (g/kg)0.56220.71460.53640.58960.60030.5403
q850 (g/kg)1.41651.82781.24341.40501.39241.2503
u500 (m/s)4.05764.19543.02404.43644.44773.0441
u850 (m/s)2.90982.91052.50683.19703.10472.5316
v500 (m/s)3.97364.07432.84784.28234.29042.8635
v850 (m/s)2.90922.89332.23613.20503.07482.2548

FERA has lower analysis error than both baselines on all six reported withheld diagnostics at both noise levels. This extends the evidence to entire variable types without direct observations.

A representative GDAS-like observing network is a further step. Its heterogeneous observation types have different, time-varying spatial coverage and require corresponding observation operators and preprocessing; a single location pattern would not capture this variability. Our random-column observation setup follows the synthetic sparse-observation convention used in DiffDA (Huang et al., 2024) and VAE-Var (Xiao et al., 2025), but does not reproduce realistic spatial clustering. We therefore treat a comprehensive GDAS-based evaluation as a dedicated extension. The present additions address measurement noise, missing variable types, and variation across weather periods and sampling seeds; realistic spatial clustering remains for future work.

AuthorsOpenReview ↗

Rebuttal from Authors (3/4)

Training-objective ablation (Q2).

The original experiments include a matched training ablation that jointly removes the held-out/cross-variable and sparse-posterior losses while retaining forecast conditioning and full-field reconstruction. In the original five-epoch, 500-column setting, retaining these losses reduces mean diagnostic-wise analysis WRMSE by 55.80% over 124 cycles and next-six-hour forecast WRMSE by 58.01% over 123 forecasts.

To address the request under the new observation settings directly, we completed four eight-cycle experiments. These compare the matched five-epoch complete-loss model A and joint-loss-removal model E at κ=0.25\kappa=0.25, with either all variables observed or all 13 humidity levels withheld. Both settings use 500 random observation columns and eight six-hour cycles over 1–3 December 2024. Within each A/E pair, the initial background, observation locations, noise realizations, R, architecture, and online solver settings are identical. Each model retains its own fitted prior covariance. R is not retuned for noise or masking.

Observation settingComplete model's mean analysis-WRMSE reductionImproved time-mean diagnostics
Noise, all 69 channels observed at sampled columns21.78%10/10
Noise, all humidity channels withheld41.88%10/10

The percentage first compares each diagnostic's eight-cycle mean WRMSE, then averages the relative reductions over z/q/u/v/t at 500/850 hPa. The humidity results give a direct test of learning corrections from other observed variables:

Observation settingDiagnostic (g/kg)Complete-loss ARemoved-loss EError reduction
Noise, all variables observedq5000.4980.66825.48%
Noise, all variables observedq8501.2631.80029.86%
Noise, humidity withheldq5000.5331.24657.21%
Noise, humidity withheldq8501.3032.96055.96%

The benefit of partial-information training therefore persists under both noisy observations and noisy observations with an entirely missing variable type. In the humidity-withholding case, the complete model approximately halves the errors of the two reported unobserved humidity diagnostics relative to the control. This ablation jointly removes the two objectives; separately isolating the held-out-variable and sparse-posterior losses requires additional global model retraining and cycling evaluations. Those experiments are computationally expensive and are currently underway; we will update the results when available.

AuthorsOpenReview ↗

Rebuttal from Authors (4/4)

Weakness 3. Numerical stability, convergence, and computational cost

Multiple weather states. We expanded the stability experiment to five states covering four seasons, with 340 completed runs. Within each state, methods use the same saved background and observations. The tests use observation-order permutations and, for iterative solvers, five random initializations with matched decoded amplitude. The metric is the channel-standardized, latitude-weighted RMSE between perturbed and reference analysis fields, averaged over all 69 channels. The table reports the largest difference over all tested states and perturbations.

Method / iteration budgetMaximum observation-order sensitivityMaximum initialization sensitivity
FERA direct1.55×10−71.55\times10^{-7}No initialization required
FERA, same-objective L-BFGS, 202.70×10−52.70\times10^{-5}3.38×10−43.38\times10^{-4}
FERA, same-objective L-BFGS, 2002.70×10−52.70\times10^{-5}3.99×10−53.99\times10^{-5}
VAE-Var, 3 outer steps1.70×10−21.70\times10^{-2}5.30×10−25.30\times10^{-2}
VAE-Var, 30 outer steps1.19×10−11.19\times10^{-1}1.65×10−11.65\times10^{-1}

FERA's direct update remains insensitive to observation ordering across all five states at approximately float32 rounding scale. Identical-repeat differences are zero in all 45 repeat groups.

Convergence and practical iteration budgets. We agree that these experiments compare solutions obtained under prescribed iteration budgets, rather than fully converged nonlinear optima. Unlike standard optimization benchmarks, where convergence to a tight numerical tolerance is often the primary criterion, operational data assimilation must prioritize analysis quality and subsequent forecast skill within strict computational constraints. For high-dimensional nonlinear assimilation problems, full convergence can be difficult to achieve or may offer limited practical benefit relative to its cost. For example, ECMWF's incremental 4D-Var historically used only two outer-loop iterations, and additional iterations could even destabilize minimization because of differences between the linearized and nonlinear model dynamics (Trémolet, 2005). Our extended iterative runs likewise do not meet the strict convergence criterion; we therefore assess their analysis and cycling performance at the stated computational budgets, without claiming a comparison of fully converged solutions.

A separate high-accuracy, same-objective check shows that, with a 500-iteration cap, iterative FERA agrees with the direct solution across all 1,159 local systems, with a maximum relative quadratic-energy gap of 9.92×10−159.92\times10^{-15} and a standardized analysis-field difference of 1.53×10−71.53\times10^{-7}. Thus, the direct update obtains the same quadratic solution without iterative convergence tuning. This check isolates the solver effect from differences in learned representations.

Timing scope. In the current implementation, correction patterns are generated sequentially tile by tile, without parallel optimization. In the A100 profiling case, this reconstruction takes 157.06 s of a 167.04 s mean analysis update, whereas the direct coefficient solve takes only 0.91 s. Tile-wise pattern generation can be parallelized across GPUs or cluster nodes, which we plan to explore in future work. The direct solve is therefore already fast; the current end-to-end time is dominated by pattern generation, with clear potential for parallel acceleration.

References

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Manuscript Revisions

Dear Area Chair and Reviewers,

We sincerely thank you for your constructive feedback and the valuable discussions. We have uploaded a revised version of our manuscript that incorporates additional experiments, clarifications, and improvements motivated by the reviews.

The main updates include:

  1. Robustness to observation conditions: We added experiments evaluating FERA across different observation-noise levels, seasonal conditions, and variable-withholding settings. The results further demonstrate the robustness of FERA, with increasingly pronounced advantages under higher observation uncertainty.

  2. Additional methodological validation: We expanded the ablation studies to further examine the contributions of forecast conditioning and partial-information training, including evaluations under noisy observations.

  3. Numerical reliability and experimental clarification: We provided additional evidence regarding the numerical stability of the direct Bayesian analysis and clarified the experimental settings and comparisons.

We have also revised the presentation and reorganized the supplementary material to improve the clarity and coherence of the manuscript.

We hope these additions help address the main concerns raised during the review process. We appreciate your time and consideration, and we welcome any further questions or feedback.

Thank you again for your thoughtful reviews.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.