acceptodds
Under review as a conference paper at ICLR 2027

Lagrangian Particles for Localized Extremes in Neural Weather ODEs

Abstract

Neural weather forecasts can lose skill on localized extremes even when global error remains low. We study LE-NODE, a continuous-time hybrid model that combines an Eulerian grid state with sparse, adaptively selected Lagrangian particles for localized anomalies. A synthetic advection probe characterizes high-wavenumber coefficient changes in Eulerian neural ODEs, motivating a particle state that follows localized structures. We evaluate the resulting model through forecast comparisons. On ERA5 at a five-day lead, LE-NODE obtains 221 m upper-tail Z500 RMSE versus 284 m for the strongest reported external baseline and 312 m for its Eulerian-only ablation under the same auxiliary objective. Recomputed lower-tail MSLP RMSE is 5.1 hPa versus 6.7 hPa for PanguLite. These results concern area-weighted pointwise tail metrics and the tested backbone. The results support a sparse particle branch as a useful complement to the tested Eulerian backbone.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural ODEs, Weather Forecasting, Lagrangian Methods

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_UnFhOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes LE-NODE, a hybrid weather forecasting framework that combines Eulerian grid-based Neural Ordinary Differential Equations (Neural ODEs) with sparse, adaptive Lagrangian particles. The proposed method leverages an Eulerian grid to represent the global atmospheric state, tracks local anomalies through continuous-time evolving Lagrangian particles, and dynamically updates the particle set at forecast segment boundaries to improve the prediction accuracy of localized extreme weather. The authors evaluate the approach on the ERA5 dataset, and experimental results demonstrate that LE-NODE achieves performance improvements in pointwise upper-tail metrics for Z500 and temperature, as well as lower-tail metrics for MSLP. Compared to a purely Eulerian model trained with the same objective, incorporating the particle branch yields a substantially greater reduction in tail prediction error than in full-field prediction error. Furthermore, through component ablation studies, evaluations across different forecast lead times, and typhoon case analyses, the authors further validate the effectiveness of the proposed method.

Critical strengths

None

Critical weaknesses

None

Questions for authors
  1. The paper demonstrates that introducing the Lagrangian particle branch significantly improves Tail-RMSE. However, adding the particle branch simultaneously increases both the number of model parameters and the computational cost. Consequently, it remains unclear whether the performance gains stem from the Lagrangian representation itself or from the additional model capacity. Could the authors compare their method against a stronger Eulerian-only model under a comparable parameter scale and computational budget?

  2. The authors note that under a spatially varying velocity field, energy transfer across Fourier modes can occur even during an exact advection process. Could the authors further compare the predictions with the reference solution at the same time step—for example, by analyzing the full spectral error or global norm variations—to more directly validate the limitations of fixed-grid representations in transporting localized structures?

3 comments
AuthorsOpenReview ↗

Response to UnFh: local-capacity control and reference-relative errors

Thank you for your assessment. We answer the two questions separately.

Critical Weaknesses:

No separate weaknesses were listed. The two questions below identify important controls for interpreting the improvement.

Questions For Authors:

Q1. Does the improvement exceed that from extra parameters and computation?

The original same-loss comparison reduces Z500 upper-tail RMSE from 312 to 221 m, but also adds capacity. We added a stationary local-residual alternative retaining the local-state budget, latent width, feature access, decoder, boundary refresh, and objective while fixing positions within each integration segment.

Five-day Z500Grid onlyStationary controlMoving LE-NODE
Upper-tail RMSE (m)312284221
Full-field RMSE (m)178172168
Parameters39.8M44.9M45.6M
Training wall time (h)8496108
Inference time per six-hour step (s)1.51.82.1

The stationary/moving parameter counts differ by approximately 1.6%, but LE-NODE has higher measured training and inference costs. This is an approximate capacity control, not a strict equal-compute comparison. The moving branch retains a 63 m tail-error advantage, with a paired 95% time-block interval [41,85] m for the reduction. The stationary-versus-grid difference is descriptive; it does not establish that a fixed percentage of the original gain is causally due to capacity.

Q2. Can predictions be compared with the same-time reference through spectra or global norms?

Yes. Physical spectral transfer means that the initial Fourier coefficient need not remain constant. We therefore compare to the evolved reference:

Diagnostic at t=10GridStationaryLE-NODE
Relative L2 error0.2410.2260.171
Relative L2-norm drift4.2%3.4%1.9%
High-band error0.280.250.16

These use 64 held-out initial fields and a pseudo-spectral reference at 512 squared resolution; the high band is kappa in [0.5,1.0]. They show lower prediction discrepancy for the moving implementation, without identifying the entire error as solver dissipation. We will state the characteristic argument as design motivation and avoid claiming that fixed grids necessarily fail or that particles remove all high-frequency error.

Thank you for the detailed response. The additional results have addressed my concerns. I encourage the authors to incorporate these experiments and clarifications into the revised manuscript. I will maintain my original rating.

AuthorsOpenReview ↗

Follow-up on the revised manuscript

Thank you for following up. We appreciate your suggestions. The stationary local-branch comparison and the same-time reference errors helped make the results clearer.

Both experiments are now included in the revised manuscript, together with the parameter counts and measured computational costs. We have also clarified the scale parameterization and the interpretation of the spectral diagnostic. Thank you again for the careful reading.

Reviewer_2RF9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LE-NODE, a hybrid neural ODE that combines an Eulerian grid with adaptive Lagrangian particles for localized extreme-weather forecasting. ERA5 experiments show improved five-day tail RMSE, with Z500 upper-tail RMSE decreasing from 312 m for the Eulerian-only ablation to 221 m.

Critical strengths
  1. The hybrid architecture provides a plausible way to represent localized atmospheric anomalies while preserving global spatial context.

  2. The Eulerian only ablation uses the same auxiliary objective and shows a substantial reduction in Z500 upper tail RMSE, from 312 m to 221 m.

  3. The paper clearly discusses limitations of its evaluation, including displacement sensitivity of pointwise metrics, differences in computational budgets, and the limited evidence provided by a single typhoon case.

Critical weaknesses
  1. The synthetic advection experiment measures changes in initial Fourier mode coefficients but does not separate spectral energy transfer, phase error, optimization error, and numerical dissipation. It therefore does not establish the proposed explanation for the forecasting problem.

  2. The evaluation lacks comparisons with strong Eulerian alternatives using higher order or semi Lagrangian transport. It remains unclear whether the gains arise from the particle representation itself or from additional model capacity.

  3. The extreme event weighted loss directly emphasizes tail errors, but the experiments do not fully isolate its contribution from the architectural improvement across all comparisons.

  4. The main evaluation uses observation conditioned pointwise tail RMSE, which can penalize spatial displacement. Comprehensive neighborhood based or object based verification is missing.

  5. The external comparison uses PanguLite, a reduced model trained from scratch, rather than the released Pangu Weather model. This limits the strength of claims about competitiveness with established weather forecasting systems.

Questions for authors
  1. Can the authors compare the proposed model with a capacity matched Eulerian model using higher order or semi Lagrangian transport?

  2. Can the synthetic diagnostic be validated against exact solutions using global norms and spectral energy distributions to distinguish spectral transfer from numerical dissipation?

  3. How much of the improvement comes from the particle branch compared with the extreme event weighted loss? Can the authors provide controlled comparisons that isolate these effects?

  4. Can the authors evaluate multiple tropical cyclones and other extreme weather events using object based metrics for track, intensity, and displacement?

  5. Can the authors quantify uncertainty across independent forecast cases and years, rather than reporting variability across training seeds alone?

  6. How does LE-NODE compare with stronger publicly available weather forecasting models under consistent data splits, forecast horizons, and evaluation protocols?

2 comments
AuthorsOpenReview ↗

Response to 2RF9 (1/2): critical weaknesses

Thank you for distinguishing implementation-level improvements from an established explanation of extreme-weather forecast errors.

Critical Weaknesses:

W1. Spectral transfer and error sources are not separated.

Initial-mode attenuation is not a dissipation estimate in spatially varying flow. We added same-time reference comparisons over 64 held-out initial conditions. At t=10, grid/stationary/moving relative L2 errors are 0.241/0.226/0.171, norm drifts 4.2%/3.4%/1.9%, and high-band errors 0.28/0.25/0.16. These measure discrepancy beyond physical spectral transfer but do not uniquely decompose learning, interpolation, optimization, and integration error. We will frame transport as a design motivation rather than an identified ERA5 error mechanism.

W2. Missing strong Eulerian alternatives and capacity control.

A stationary local-residual control retains the local-state budget, feature access, decoder, boundary refresh, and objective, while fixing positions within each segment. Its Z500 upper-tail RMSE is 284 m versus 221 m for LE-NODE; full-field RMSE is 172 versus 168 m. Parameters are 44.9M versus 45.6M, training times 96 versus 108 h, and inference times 1.8 versus 2.1 s per six-hour step. Thus, capacity is approximately matched, computation is not identical. We do not claim superiority over an untested high-order or semi-Lagrangian solver.

W3. Architecture and extreme weighting are not isolated.

With proxy penalties fixed, the architecture-by-extreme-weight results are:

Z500 upper-tail RMSE (m)alpha=0alpha=5
Eulerian backbone301312
LE-NODE267221

The hybrid reduces error by 34 m at alpha=0 and 91 m at alpha=5. Increasing alpha changes grid error by +11 m and hybrid error by -46 m. Thus, extreme weighting does not help both architectures uniformly; the results show an architecture-objective interaction. They do not justify additive percentages of improvement or attribution to representation alone.

W4. Pointwise tails are displacement-sensitive.

We added Z500 neighborhood FSS on saved forecasts using training-derived upper-tail thresholds. At 100/250/500 km, LE-NODE scores 0.34/0.46/0.58, versus 0.27/0.38/0.51 for the grid and 0.29/0.40/0.53 for the stationary branch. This supports improvement under spatial tolerance. It does not establish a comprehensive cyclone-track, intensity, or object-detection benchmark; the single Hagibis case remains illustrative.

W5. PanguLite is not released Pangu-Weather.

Correct. It is our approximately 50M-parameter reduction trained from scratch. Objectives and training costs also differ across external baselines. We will make this prominent and restrict the primary evidence to controlled comparisons within the tested backbone. Improvement over PanguLite will not be described as demonstrated competitiveness with released or operational forecasting systems.

AuthorsOpenReview ↗

Response to 2RF9 (2/2): questions for authors

This continues our response to 2RF9. W labels refer to Part 1.

Questions For Authors:

Q1. Capacity-matched Eulerian comparison with higher-order or semi-Lagrangian transport?

The stationary local-residual control in W2 directly tests a comparable allocation of additional local capacity. LE-NODE retains a 63 m Z500 tail-error advantage, with paired 95% time-block interval [41,85] m for the reduction. Its measured computation is higher, which we disclose. We have not tested an additional higher-order/semi-Lagrangian solver and therefore restrict conclusions to this backbone and control, without a necessity claim for particles.

Q2. Reference solutions, global norms, and spectra?

We use a pseudo-spectral reference at 512 squared resolution and compare at identical forecast times. At t=10, relative L2 errors are 0.241/0.226/0.171, norm drifts 4.2%/3.4%/1.9%, and high-band errors 0.28/0.25/0.16 for grid/stationary/moving variants. The high band is kappa in [0.5,1.0]. This avoids confusing physical mode transfer with prediction discrepancy; it does not uniquely identify numerical dissipation.

Q3. How much comes from particles versus extreme weighting?

The two-by-two table in W3 gives a 34 m hybrid-versus-grid difference without extreme weighting and 91 m with it. Turning on weighting increases grid error by 11 m but reduces hybrid error by 46 m. The difference-in-differences is -57 m in RMSE change. This indicates interaction rather than an additive decomposition. Likewise, the stationary control cannot be interpreted as assigning a causal fraction of the gain to capacity.

Q4. Multiple events with track, intensity, and displacement metrics?

Our added verification uses neighborhood scores across the available test forecasts, rather than generalizing from one storm. It establishes spatially tolerant field skill. A complete object-level evaluation requires a reproducible storm-selection and matching protocol, including misses and false alarms. We do not claim that benchmark here. In particular, the outputs lack 850-hPa vorticity, so pressure minima alone will not be presented as vorticity-qualified cyclone detection or validated operational tracks.

Q5. Uncertainty across forecast cases and years?

We use paired 14-day blocks of initialization dates, preserving complete fields and horizons within blocks. The test has 1,461 initializations; these are not independent grid-point or block replicates. For moving versus stationary tail RMSE, the reduction is 63 m with interval [41,85] m. Year-specific reductions against the original grid for 2018-2021 are 86, 96, 89, and 93 m. These describe temporal heterogeneity; four years are not a large independent sample. Training-seed variation is a separate source of uncertainty.

Q6. Comparison with stronger released weather models under a common protocol?

The submitted evidence does not establish this comparison. Released checkpoints differ in inputs, training provenance, and rollout setup; PanguLite cannot stand in for them. We prioritize the controlled capacity/objective comparisons and spatial verification, and will limit external competitiveness claims accordingly rather than add an unmatched released-model number.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to ZWya (1/2): critical weaknesses

Thank you for recognizing the same-objective evidence and the distinction between pointwise tails and event verification.

Critical Weaknesses:

W1. Contribution beyond DeepLag and additional local capacity.

DeepLag is the closest hybrid-representation precedent. Our contribution concerns the coupled continuous-time implementation, segment-boundary updates, and its tested utility for localized tail errors, rather than the first Eulerian-Lagrangian hybrid.

We added your suggested stationary local-residual control. It retains the local-state budget, feature access, latent width, decoder, boundary refresh, and objective, while fixing positions within each integration segment.

Five-day Z500Grid onlyStationary branchLE-NODE
Upper-tail RMSE (m)312284221
Full-field RMSE (m)178172168
Parameters39.8M44.9M45.6M
Training wall time (h)8496108
Inference time per six-hour step (s)1.51.82.1

The stationary and moving variants have similar parameter counts but different measured costs. This is an approximate capacity control, not an equal-compute experiment. The moving variant retains a 63 m tail-error advantage over the stationary variant. The 28 m stationary-versus-grid difference is a descriptive comparison; it is not a causal percentage of the original improvement. The external DeepLag comparison uses a different objective and does not isolate our additions.

W2. Initial-mode attenuation does not identify excess high-frequency error.

We now compare prediction and reference at the same time. At t=10, over 64 held-out initial fields, relative L2 errors are 0.241/0.226/0.171, L2-norm drifts 4.2%/3.4%/1.9%, and high-band errors 0.28/0.25/0.16 for grid/stationary/moving variants. The reference is pseudo-spectral at 512 squared resolution. These quantify prediction discrepancy while accounting for physical mode transfer; they do not uniquely separate learning, interpolation, optimization, and integration error. We will limit the mechanism interpretation accordingly.

W3. Implementation details and scale parameterization.

W3(a), proxy penalties: These penalize normalized squared changes in field sums, squared-field sums, and horizontal-vorticity-squared sums. They are two-dimensional regularizers, not atmospheric conservation laws. We will provide the complete variable-wise weighting, normalization, and stabilizing terms in the revision rather than imply physical conservation from their names.

W3(b), multivariable decoding: Particle intensity is scalar. Its splatted contribution to variable v is multiplied by a learned channel gain c_v; outputs use variable-wise standardized units. The latent state affects particle dynamics rather than entering the field directly.

W3(c), initialization and set updates: New particles use high-extremity positions, local-anomaly intensity, population-specific scale, and zero latent state. Removal uses the intensity/gate threshold or domain exit. The grid is not compensated when the set changes, so total-field continuity across these discrete updates is not guaranteed. Top-k indices are stop-gradient; selected particle states and gates remain differentiable within segments.

W3(d), sigma: The implementation evolves an unconstrained z_sigma and uses sigma=softplus(z_sigma). The head predicts dz_sigma/dt, not a nonnegative d(sigma)/dt. Equation 3 should use z_sigma in the evolved state. This corrects the notation and does not impose monotonically increasing scale.

W4. Limited event-level evidence.

Saved-forecast Z500 neighborhood FSS values at 100/250/500 km are 0.34/0.46/0.58 for LE-NODE, 0.29/0.40/0.53 for the stationary branch, and 0.27/0.38/0.51 for the grid. These use training-climatology upper-tail thresholds. The advantage persists with spatial tolerance, but does not separately establish cyclone-track or object-intensity skill. Hagibis remains illustrative.

AuthorsOpenReview ↗

Response to ZWya (2/2): questions for authors

Questions For Authors:

Q1. Same-objective DeepLag or a similarly sized nonmoving branch?

We use the second control. The stationary branch has 44.9M parameters versus 45.6M for LE-NODE; upper-tail Z500 RMSE is 284 versus 221 m and full-field RMSE is 172 versus 168 m. The paired 95% time-block interval for LE-NODE minus stationary tail RMSE is [-85,-41] m. Training and inference are more expensive for LE-NODE, as disclosed in W1, so this is not a strict equal-compute comparison. We do not claim a same-objective DeepLag comparison.

Q2. Proxy definitions, initialization, decoding, and sigma parameterization?

W3 addresses these separately. The key implementation facts are scalar intensity with learned output-channel gains, local-anomaly initialization, uncompensated grid state at set changes, and integration of z_sigma with sigma=softplus(z_sigma). The last point resolves the derivative/positivity inconsistency. Complete proxy formulas and update rules will be made explicit in the revision; they will not be described as atmospheric conservation constraints.

Q3. Same-time errors against the reference solution?

At t=10, grid/stationary/moving relative L2 errors are 0.241/0.226/0.171; norm drifts are 4.2%/3.4%/1.9%; high-frequency-band errors are 0.28/0.25/0.16. The high band is kappa in [0.5,1.0], over 64 held-out initial fields. This compares against the evolved reference rather than interpreting the initial coefficient as constant. The result supports lower error in this synthetic implementation without identifying numerical dissipation as its sole cause.

Q4. Does the ERA5 advantage persist with spatial tolerance?

Yes, for the tested Z500 verification. At 250 km, FSS is 0.46 versus 0.38 for the grid and 0.40 for the stationary branch. The paired 95% 14-day time-block interval for the grid comparison is [+0.05,+0.11]. The test contains 1,461 forecast initializations; these are cases, not 1,461 independent blocks. Positive differences also occur at 100 and 500 km, as listed in W4. This supports neighborhood-scale field skill, with no claim of a complete cyclone-object evaluation.

AuthorsOpenReview ↗

Summary of the discussion and manuscript revisions

Thank you to all reviewers for the feedback. We have incorporated the additional experiments into the revised manuscript. For the final assessment, we summarize the main concerns raised in the reviews and the evidence added during the discussion.

  • Does the gain come mainly from additional capacity?

We added the suggested stationary local-residual control, retaining the local-state budget, feature access, decoder, boundary refresh, and objective. With similar parameter counts (44.9M versus 45.6M), five-day Z500 upper-tail RMSE is 284 m for the stationary branch and 221 m for LE-NODE. The 63 m reduction has a paired 95% time-block interval of [41, 85] m. The revision reports the higher computational cost of the moving branch alongside these results.

  • Does the synthetic diagnostic account for physical spectral transfer?

We added comparisons against the evolved reference at the same forecast time, using 64 held-out initial fields. Relative L2 error decreases from 0.241 for the grid and 0.226 for the stationary branch to 0.171 for LE-NODE; high-band errors decrease from 0.28 and 0.25 to 0.16. The revised interpretation is based on these reference-relative errors.

  • Do the gains depend entirely on extreme weighting or pointwise verification?

The hybrid improves Z500 tail RMSE over the grid both without extreme weighting (301 to 267 m) and with it (312 to 221 m). This comparison shows an interaction between architecture and objective.

The advantage also persists under neighborhood verification: at 250 km, FSS is 0.46 for LE-NODE, 0.40 for the stationary control, and 0.38 for the grid. The LE-NODE–grid difference has a paired 95% interval of [+0.05, +0.11].

  • What has changed in the manuscript?

The revision includes these controls and uncertainty estimates, specifies particle initialization and decoding, and corrects the scale-state notation. It also places the contribution relative to DeepLag more precisely: coupled continuous-time evolution and its evaluation for localized weather extremes. PanguLite is explicitly identified as a reduced model trained from scratch, with the controlled internal comparisons providing the main evidence.

Reviewer UnFh has confirmed that the additional results addressed their concerns and maintained their positive rating. We appreciate the reviewers’ suggestions, which helped make both the experimental evidence and the presentation clearer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.