acceptodds
Under review as a conference paper at ICLR 2027

Rotary Position Embeddings on the Sphere for Global Weather Forecasting

Abstract

Modern transformers rely on positional encodings in the attention mechanism to account for the positions of input tokens. Rotary Position Embeddings (RoPE) are among the most popular approaches to date for this. For transformers applied to sequences defined on the sphere, for example in machine learning-based weather forecasting, extending RoPE to is non-trivial. We therefore develop S2RoPE, a spherical rotary position encoding for queries and keys defined on the sphere. We propose two variants: channel-wise modulation by normalized complex spherical harmonics (S2RoPE-SH) and rotations within complete harmonic degree spaces (S2RoPE-Wigner). The first approach modulates both amplitude and phase, whereas the second one preserves feature norms and encodes the relative position of two tokens through a rotation between their local frames in one spherical harmonics band. We apply both forms to two global medium-range forecasting models, the WeatherGenerator and Stormer. We find improvements over baseline methods in standard metrics, particularly at short and medium lead times. In WeatherGenerator, we further find improvements in the preservation of fine-scale detail and less drift in the balance between wind and geopotential during extended rollouts.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
rotary position embeddings, positional encoding, spherical harmonics, global weather forecasting, transformers

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_MS5xOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose a new position embedding approach, S2RoPE, for transformers operating on the sphere by adapting Rotary Position Embeddings (RoPE) to this non-Euclidean space, leveraging spherical harmonics and therefore better preserving spherical distances. Two variants of S2RoPE corresponding to the two interpretations of RoPE are proposed: S2RoPE-SH, which multiplies query and key features by spherical harmonic values, and S2RoPE-Wigner, which rotates groups of features according to each token’s position. The authors then evaluate these two variants against three baselines (No RoPE, 2D RoPE, and GeoRoPE) in two setups using ERA5 data with different resolutions, geometries, and architectures, focusing on global weather forecasting.

Critical strengths
  • The problem addressed is particularly important, as many Earth systems applications require good representations that preserve distances on the sphere.
  • The approaches look sound and are well motivated by the background. Having two extensions following the two readings of RoPE is meaningful.
  • Good performance on the two proposed setups.
Critical weaknesses

I think the scope of the paper and the evaluation of the approach are a bit limited:

  • Many Earth systems applications beyond global weather forecasting, such as ocean circulation, atmospheric transport, or tropical cyclone tracking, could benefit from approaches like S2RoPE. I understand that, as an application-driven paper, the authors wanted to focus on a single task, but I also believe the paper would be stronger if it demonstrated S2RoPE's potential to generalize to one or two additional applications.
  • In addition, even in the realm of weather forecasting, I think the experimental setups and baselines tested are slightly insufficient. Only two setups are considered, both using ERA5 data, and only three baselines are evaluated. In particular, a few mentioned related works (e.g., SpheRoPE, StereoRoPE) are not considered at all, despite being alternatives to S2RoPE.
  • I also feel that the paper could be improved by providing additional experiments on where S2RoPE is stronger than alternatives. For example, explicitly considering Earth's curvature is known to be particularly beneficial at high absolute latitude. I would therefore be interested to see how the baselines and S2RoPE performed across different latitude ranges. An in-depth analysis of the input resolution and its impact on the different position embedding approaches could also be relevant.
Questions for authors
  • Could you provide intuition and a corresponding experiment explaining why one variant could be preferred over the other?
  • Is there a specific reason why you only considered the task of weather forecasting?
  • Why are some baselines not considered for comparison?

I am willing to increase my rating if you address these questions and weaknesses well.

4 comments

Thank you for your review and for the suggestions on broadening the evaluation. We have expanded the evaluation with a third forecasting backbone, a StereoRoPE baseline, tropical cyclone tracking, and a latitude-resolved analysis. We also clarify the different strengths of the two S2RoPE variants.

1. Application scope and tropical cyclone tracking

We focused on global weather forecasting because it provides a demanding setting for evaluating positional representations on the sphere: forecasts must capture interactions across spatial scales and remain accurate during extended rollouts. The positional operators themselves use spherical token coordinates and do not depend on weather-specific variables, making them applicable in principle to other attention-based models on the sphere.

Following your suggestion, we have added tropical cyclone tracking using LaDCast ensemble forecasts. This evaluates a downstream application of the forecasts rather than a separately trained cyclone model. We use 20-member ensembles over the 2018 evaluation set, comprising 102 storms and 402 storm–initialization cases before lead-dependent eligibility filtering.

We evaluate two track estimators: tracking the ensemble-mean field, following the original LaDCast protocol, and tracking individual members before averaging their positions on the sphere. All methods use the same tracker configuration and eligible cases at each lead, with equal weighting of individual storms.

Both S2RoPE variants reduce track error relative to GeoRoPE at every reported lead under both estimators. At day 5, S2RoPE-Wigner reduces error by 20.66% under the original estimator and 15.96% under member-wise tracking and position averaging. The latter estimator also improves GeoRoPE itself, so the S2RoPE gains persist against this improved reference. StereoRoPE performs best at day 1 under both estimators, while an S2RoPE variant performs best at every reported lead from day 2 onward.

These results are included in the new appendix subsection “Tropical cyclone tracking” and summarized in the main text.

2. Additional forecasting architecture

We have added LaDCast, a probabilistic latent diffusion model, alongside WeatherGenerator and Stormer. Unlike the regression-based forecasting settings of the original experiments, LaDCast generates ensemble forecasts through iterative denoising in an autoencoder’s latent space. Training and evaluation settings are held fixed across positional encoding variants. We evaluate ensemble-mean RMSE and ACC, as well as CRPS.

The new LaDCast results include the requested StereoRoPE comparison. Each entry below gives the percentage changes in RMSE / CRPS relative to native GeoRoPE; negative values indicate lower error.

Method1 d3 d5 d7 d10 d
StereoRoPE−0.40 / −0.22−0.43 / −0.42−0.58 / −0.63−0.36 / −0.34−0.31 / −0.34
S2RoPE-SH−1.94 / −1.55−1.89 / −1.90−1.40 / −1.66−0.45 / −0.67−0.25 / −0.36
S2RoPE-Wigner−3.49 / −3.12−3.35 / −3.41−2.29 / −2.81−1.50 / −1.90−1.04 / −1.31

Both S2RoPE variants improve ensemble-mean RMSE, ACC, and CRPS relative to native GeoRoPE at all reported lead times. S2RoPE-Wigner achieves the best results among the evaluated methods across all three metrics and all reported leads (new Table 3). This broadens the architectural and forecasting settings of our evaluation.

3. Additional baselines

We have added a StereoRoPE adaptation in both LaDCast and WeatherGenerator. The LaDCast results are reported above; the WeatherGenerator comparison below uses the same zero-noise training settings as the other RoPE variants. Implementation details are provided in the appendix subsection “StereoRoPE implementation.” RMSE changes relative to No RoPE (DN, default noise), in %, are shown below.

Method6 h1 d3 d5 d7 d10 d14 d
2D RoPE−10.00−9.79−10.43−9.62−7.34−4.11−1.00
GeoRoPE−10.11−9.92−10.82−9.70−7.41−4.59−1.57
StereoRoPE−9.05−8.94−9.82−8.87−6.88−4.66−2.37
S2RoPE-SH−11.22−10.83−11.55−10.32−8.01−4.75−2.14
S2RoPE-Wigner−11.11−10.88−11.30−10.14−8.14−5.50−2.56

Both S2RoPE variants achieve lower RMSE than StereoRoPE at all reported leads through day 10. At day 14, StereoRoPE lies between S2RoPE-Wigner and S2RoPE-SH. An S2RoPE variant has the highest ACC in every column (revised Table 1).

SpheRoPE was introduced as a training-free, inference-time adaptation of pretrained diffusion transformers for panoramic generation. Our experiments instead train forecasting variants with the selected positional encoding. We prioritized StereoRoPE because it was introduced for atmospheric forecasting; SpheRoPE remains discussed in the related work but is not included in the current experimental comparison.

4. Performance across latitude ranges and spatial token counts

We have added a latitude-resolved comparison against axial 2D RoPE in WeatherGenerator. We use this reference to examine how the gains over a latitude–longitude rotary encoding vary with latitude under matched training settings. All compared models retain global positional embeddings and use zero latent training noise.

We evaluate nine variables in 10° latitude bands using RMSE and a zonal spectral diagnostic over physical wavelengths of 500–5,000 km. The figure in the appendix subsection “Forecast performance by latitude” reports both metrics across all latitude bands and forecast leads.

Both S2RoPE variants improve high-latitude RMSE and zonal spectral error at longer leads. At day 14, over regions with absolute latitude at least 60°, S2RoPE-SH and S2RoPE-Wigner reduce RMSE relative to 2D RoPE by 3.13% and 2.66%, respectively, and zonal spectral error by 11.16% and 7.86%. Improvements also occur at lower latitudes, with patterns varying by variant and forecast lead. These results support benefits at high latitudes without implying that gains increase monotonically with latitude.

Our backbones use different spatial token counts—12,288 in WeatherGenerator, 2,048 in Stormer, and 450 in LaDCast per forecast state. These experiments do not isolate input-resolution effects, since architecture and training settings also differ.

5. When to prefer S2RoPE-SH or S2RoPE-Wigner

The two variants retain different aspects of classical RoPE. S2RoPE-SH applies harmonic multipliers independently to complex feature channels, changing both amplitude and phase. Its normalization controls average squared gain across channels but does not preserve arbitrary feature norms. S2RoPE-Wigner jointly transforms channels within complete degree blocks using orthogonal matrices, preserving query and key norms.

The experiments show different strengths rather than a universal ranking. In WeatherGenerator, SH achieves lower log-spectral error in the forecast windows beyond day 3, while Wigner achieves lower RMSE than SH at the reported leads from day 7 onward. In Stormer, Wigner achieves the lowest RMSE at every reported lead. At day 14, it retains an RMSE improvement over No RoPE, whereas SH performs worse than No RoPE in both RMSE and ACC. In LaDCast, Wigner outperforms SH in ensemble-mean RMSE, ACC, and CRPS at every reported lead. Cyclone tracking likewise shows that their relative ranking depends on lead time and the track estimator.

Between the two S2RoPE variants, Wigner is a reasonable starting choice when prioritizing RMSE, and it also performs best on CRPS in LaDCast. SH offers stronger spectral fidelity in WeatherGenerator, so the preferred variant depends on the backbone and evaluation objective. These are empirical preferences in the evaluated settings; norm preservation alone does not establish that Wigner will always perform better.

I would like to thank the authors for addressing my concerns by adding a new baseline, a task, and an additional experiment on performance by latitude. I will therefore increase my rating from 2 to 3.

Thank you for reconsidering the paper. Your suggestions on the additional baseline, the cyclone-tracking evaluation, and the latitude-resolved analysis have made the paper stronger. We appreciate your time and careful review.

Reviewer_TuVjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes S2RoPE, a spherical rotary positional encoding framework with two variants, S2RoPE-SH and S2RoPE-Wigner, for Transformer-based global weather forecasting. Evaluated on WeatherGenerator and Stormer using ERA5 data, the method shows promising improvements in forecast accuracy and spectral fidelity, with reduced physical-consistency drift. However, concerns remain regarding experimental design and the generalizability of the results.

Critical strengths
  1. The paper provides a principled mathematical formulation for extending rotary positional encoding from conventional sequence or planar settings to spherical geometry. The authors leverage spherical harmonics and Wigner-D representations to construct position-dependent transformations.

2.The paper proposes two complementary variants, S2RoPE-SH and S2RoPE-Wigner, based on two interpretations of rotary positional encoding.

3.The paper reports performance at multiple forecast lead times. This reveals that the effectiveness of the proposed methods varies with the forecasting horizon and model architecture.

Critical weaknesses

1.The proposed method is evaluated on only two forecasting architectures, WeatherGenerator and Stormer. Evaluating S2RoPE on only two backbones is insufficient to establish its general effectiveness and generalizability.

2.The main comparison between No RoPE and S2RoPE does not control for latent noise. According to the experimental setup, the No RoPE baseline retains the default latent noise during both training stages, whereas the RoPE variants, including S2RoPE-SH and S2RoPE-Wigner, use zero latent noise. Consequently, positional encoding and latent noise are changed simultaneously.

Questions for authors

1.Can you extend the evaluation to additional, substantially different weather forecasting models to demonstrate that the observed improvements are not specific to WeatherGenerator and Stormer?

2.Can you ensure that the reported improvements are attributable to the proposed positional encoding rather than to differences in latent noise?

3.Can you provide controlled comparisons in which No RoPE, 2D RoPE, GeoRoPE, S2RoPE-SH, and S2RoPE-Wigner are evaluated under identical latent noise settings, with all other training and evaluation configurations held fixed?

5 comments

Thank you for your review and for these questions. We have extended the evaluation to LaDCast, a probabilistic latent diffusion forecasting model, and included the existing zero-noise No RoPE control in the main results, reporting RMSE, ACC, spectral error, and physical-consistency diagnostics.

Why we used No RoPE (DN, default noise) as the reference. Latent noise is part of WeatherGenerator's default training recipe, where it serves to counteract over-smoothing in long rollouts. We therefore retained it for the original No RoPE reference. Our noise ablations support this rationale: removing it leads to increasing spectral error and declining temporal activity at longer leads, and the updated Figure 3 also shows greater drift from the ERA5 ageostrophic-to-geostrophic wind-speed ratio. In contrast, the RoPE variants retain more temporal activity without latent noise, with S2RoPE additionally achieving lower spectral error. However, using different noise settings did not isolate the effect of positional encoding. We therefore now report both the default-noise reference and a matched zero-noise No RoPE baseline.

Question 1: Evaluation on a substantially different forecasting model

We agree that evaluating a broader range of architectures strengthens the evidence for transferability. WeatherGenerator autoregressively advances a latent state, whereas Stormer predicts physical-field increments; both use regression objectives in our experiments. LaDCast instead learns a conditional diffusion model in an autoencoder’s latent space and generates ensemble forecasts through iterative denoising.

We use the 1.6B-parameter LaDCast diffusion transformer and keep the authors’ released pretrained autoencoder frozen across all variants. S2RoPE replaces only the spatial components of the native GeoRoPE; the temporal component is retained. Training and evaluation settings are held fixed across positional encoding variants. We evaluate 20-member ensemble forecasts up to 10 days using ensemble-mean RMSE and ACC, together with CRPS for probabilistic forecast quality. We also include a StereoRoPE adaptation.

The new results are summarized below. Each entry gives the percentage changes in RMSE / CRPS relative to native GeoRoPE; negative values indicate lower error.

Method1 d3 d5 d7 d10 d
StereoRoPE−0.40 / −0.22−0.43 / −0.42−0.58 / −0.63−0.36 / −0.34−0.31 / −0.34
S2RoPE-SH−1.94 / −1.55−1.89 / −1.90−1.40 / −1.66−0.45 / −0.67−0.25 / −0.36
S2RoPE-Wigner−3.49 / −3.12−3.35 / −3.41−2.29 / −2.81−1.50 / −1.90−1.04 / −1.31

Both S2RoPE variants improve ensemble-mean RMSE, ACC, and CRPS relative to native GeoRoPE at all reported lead times. S2RoPE-Wigner achieves the best results among the evaluated methods across all three metrics and all reported leads (new Table 3). These results extend the evidence beyond the two original backbones to probabilistic forecasting.

We have added the LaDCast setup and results to the main paper, with architecture details and per-variable CRPS results in the appendix.

Questions 2–3: Controlled comparison under identical latent-noise settings

We agree that the original WeatherGenerator comparison changed both positional encoding and latent training noise. The original appendix already compared No RoPE with and without latent noise using spectral error and temporal activity. We now include the existing zero-noise No RoPE configuration in the main results, reporting RMSE, ACC, spectral error, and physical-consistency diagnostics (Table 1 and Figures 2–3). Together with the zero-noise RoPE variants, this provides the requested comparison of No RoPE, 2D RoPE, GeoRoPE, S2RoPE-SH, and S2RoPE-Wigner with zero latent noise in both training stages. All other training and evaluation settings are held fixed, including retaining the original global positional embeddings.

The revised Table 1 reports the following RMSE changes (%). All five rows use zero latent noise; the common denominator remains the original default-noise No RoPE reference, denoted No RoPE (DN).

Method6 h1 d3 d5 d7 d10 d14 d
No RoPE−8.44−8.87−10.05−8.71−6.58−4.95−4.61
2D RoPE−10.00−9.79−10.43−9.62−7.34−4.11−1.00
GeoRoPE−10.11−9.92−10.82−9.70−7.41−4.59−1.57
S2RoPE-SH−11.22−10.83−11.55−10.32−8.01−4.75−2.14
S2RoPE-Wigner−11.11−10.88−11.30−10.14−8.14−5.50−2.56

Removing latent noise substantially improves No RoPE’s RMSE, so the original reductions relative to No RoPE (DN) should not be attributed entirely to positional encoding. Nevertheless, both S2RoPE variants achieve lower RMSE than zero-noise No RoPE at all reported leads through day 7 and higher ACC at every reported lead. At day 3, the RMSE reductions for S2RoPE-SH and S2RoPE-Wigner exceed that of zero-noise No RoPE by 1.50 and 1.25 percentage points, respectively (11.55% and 11.30% versus 10.05%, all relative to No RoPE (DN)). Both variants also outperform 2D RoPE and GeoRoPE in RMSE at every reported lead under the shared zero-noise setting.

At day 14, zero-noise No RoPE achieves lower RMSE than either S2RoPE variant, but this advantage is accompanied by lower ACC and poorer spectral fidelity. Its log-spectral error is approximately 40% higher than No RoPE (DN) over days 10–14, whereas both S2RoPE variants have lower spectral error than both No RoPE configurations across all four forecast windows. The updated Figure 3 provides complementary evidence under the same zero-noise setting: both S2RoPE variants show less drift from the ERA5 ageostrophic-to-geostrophic wind-speed ratio than No RoPE during extended rollouts.

We have updated Table 1 and Figures 2–3 and added an explicit discussion in “Effect of latent training noise,” with further diagnostics in the appendix. In Stormer and LaDCast, training settings are held fixed across positional encoding variants, so those comparisons are not confounded by differences in training-noise settings.

As a brief update since our initial response, we have added results across three training runs for the zero-noise WeatherGenerator comparison in the appendix subsection “Variation across training runs.”

Both S2RoPE variants achieve lower mean RMSE and log-spectral error than 2D RoPE and GeoRoPE at every reported lead. Wigner’s RMSE advantage and both variants’ spectral advantages also hold across all individual runs against these two rotary baselines.

Relative to zero-noise No RoPE, both variants achieve lower mean RMSE at all reported leads through day 7 and higher mean ACC at every reported lead. At day 14, zero-noise No RoPE achieves lower mean RMSE, but lower mean ACC and substantially higher log-spectral error than both S2RoPE variants. These results reinforce our earlier finding that removing latent noise alone does not reproduce their combination of forecast accuracy and spectral fidelity.

Together with the LaDCast evaluation, these results provide additional evidence addressing the concerns about generality and latent-noise control. We would be happy to clarify any remaining questions.

Reviewer_fdg3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This is a really cool and interesting paper. It tests a genuinely relevant idea: How to best encode positions for transformers with spherical data? Essentially the authors extend rotary position embeddings to the sphere in two ways. S2RoPE-SH multiplies query/key channels by spherical harmonics. S2RoPE-Wigner rotates harmonic-degree blocks with Wigner-D matrices, which keeps norms and depends only on relative position. Both are tested in WeatherGenerator (HEALPix) and Stormer (1.4° lat-lon) on ERA5, against no RoPE, 2D RoPE and GeoRoPE.

Critical strengths

Strengths:

  • clean construction grounded in spherical-harmonic theory and SO(3) representations. Attention is exactly invariant to longitude shifts.
  • gains hold on two different backbones (Perceiver/HEALPix and ViT/lat-lon).
  • evaluations go beyond RMSE: power spectra, and geostrophic-balance drift out to 20 days.
Critical weaknesses

Weaknesses;

  • in WeatherGenerator, no RoPE keeps latent noise while the RoPE variants use zero noise.
  • there is no variance across training seeds
Questions for authors

Questions:

  • how do results vary across at least 3 training seeds? Are the gains over 2D RoPE significant?
  • what happens with no RoPE retrained at matched (zero) latent noise?
2 comments

Thank you for your review and for raising these two points. We now include the existing zero-noise No RoPE control in the main results, reporting RMSE, ACC, spectral error, and physical-consistency diagnostics. The three-seed experiments are close to completion.

Why we used No RoPE (DN, default noise) as the reference. Latent noise is part of WeatherGenerator's default training recipe, where it serves to counteract over-smoothing in long rollouts. We therefore retained it for the original No RoPE reference. Our noise ablations support this rationale: removing it leads to increasing spectral error and declining temporal activity at longer leads, and the updated Figure 3 also shows greater drift from the ERA5 ageostrophic-to-geostrophic wind-speed ratio. In contrast, the RoPE variants retain more temporal activity without latent noise, with S2RoPE additionally achieving lower spectral error. However, using different noise settings did not isolate the effect of positional encoding. We therefore now report both the default-noise reference and a matched zero-noise No RoPE baseline.

1. Matched latent-noise comparison

The original appendix (“Effect of latent training noise”) compared No RoPE with and without latent noise using spectral error and temporal activity. We now include the existing zero-noise configuration in the main comparison: Table 1 reports RMSE and ACC, Figure 2 compares accuracy and spectral error, and Figure 3 assesses physical consistency. This configuration uses zero latent noise in both training stages, matching the RoPE variants; the default-noise reference is denoted No RoPE (DN). RMSE changes relative to No RoPE (DN), in %, are shown below:

Method (zero latent noise)6 h1 d3 d5 d7 d10 d14 d
No RoPE−8.44−8.87−10.05−8.71−6.58−4.95−4.61
S2RoPE-SH−11.22−10.83−11.55−10.32−8.01−4.75−2.14
S2RoPE-Wigner−11.11−10.88−11.30−10.14−8.14−5.50−2.56

Removing latent noise substantially improves No RoPE’s RMSE, so the original reductions should not be attributed entirely to positional encoding. At day 3, the RMSE reductions for S2RoPE-SH and S2RoPE-Wigner exceed that of zero-noise No RoPE by 1.50 and 1.25 percentage points, respectively (11.55% and 11.30% versus 10.05%, all relative to No RoPE (DN)). Both variants achieve lower RMSE than the matched baseline at all reported leads through day 7 and higher ACC at every reported lead.

At day 14, zero-noise No RoPE achieves lower RMSE than either S2RoPE variant, but this advantage is accompanied by lower ACC and poorer spectral fidelity. Its log-spectral error is approximately 40% higher than No RoPE (DN) over days 10–14, whereas both S2RoPE variants have lower spectral error than both No RoPE configurations across all four forecast windows. The updated Figure 3 provides complementary evidence under the same zero-noise setting: both S2RoPE variants show less drift from the ERA5 ageostrophic-to-geostrophic wind-speed ratio than No RoPE during extended rollouts. We now discuss this explicitly in the main-text paragraph “Effect of latent training noise,” with additional diagnostics in the appendix subsection of the same name.

In Stormer and LaDCast, training settings are held fixed across positional encoding variants, so their comparisons are not confounded by differences in training-noise settings.

2. Variation across training seeds

We agree that evaluation over many forecast initializations does not establish robustness to training randomness. Our additional WeatherGenerator training runs and inference are now complete; the remaining evaluations are running or queued. We expect to report the individual-seed results, mean and sample standard deviation across three training seeds, and a comparison against 2D RoPE under matched zero-noise settings by November 15. These results will allow us to assess the consistency and magnitude of the gains across training seeds. We defer claims of statistical significance until the results have been collected and checked.

Thank you for your patience. The additional WeatherGenerator evaluations are now complete. We report three training runs for each of No RoPE, 2D RoPE, GeoRoPE, S2RoPE-SH, and S2RoPE-Wigner, comprising the original run and two additional runs with different random seeds. All five configurations use zero latent noise in both training stages and retain global positional embeddings.

Results across training runs

The table below summarizes the comparisons against 2D RoPE and GeoRoPE at days 3, 7, and 14. Entries are mean ± sample standard deviation across three training runs. RMSE changes are percentages relative to the same fixed No RoPE (DN, default noise) reference; more negative values indicate lower error.

Method3 days7 days14 days
2D RoPE−10.48 ± 0.11−7.46 ± 0.21−1.43 ± 0.42
GeoRoPE−10.57 ± 0.28−7.20 ± 0.25−1.76 ± 0.36
S2RoPE-SH−11.33 ± 0.20−7.56 ± 0.41−1.81 ± 0.34
S2RoPE-Wigner−11.43 ± 0.12−8.22 ± 0.20−2.65 ± 0.42

Both S2RoPE variants achieve lower mean RMSE than 2D RoPE and GeoRoPE at every reported lead from 6 h to 14 days. At day 3, their mean RMSE reductions exceed that of 2D RoPE by 0.85 percentage points for SH and 0.95 percentage points for Wigner, using No RoPE (DN) as the common reference.

Wigner’s RMSE advantage is consistent across the observed runs: at every reported lead, its highest RMSE remains below the lowest RMSE obtained by any of the three runs of either 2D RoPE or GeoRoPE. SH also improves mean RMSE at every reported lead, although its later-lead gains are smaller and individual results overlap with those of the baselines.

Individual-run results

The corresponding individual-run RMSE changes are shown below. Run 0 is the original experiment; runs 1 and 2 are the additional training runs.

MethodRun3 days7 days14 days
2D RoPE0−10.43−7.34−1.00
2D RoPE1−10.61−7.70−1.44
2D RoPE2−10.40−7.33−1.84
GeoRoPE0−10.82−7.41−1.57
GeoRoPE1−10.63−7.26−2.18
GeoRoPE2−10.27−6.92−1.54
S2RoPE-SH0−11.55−8.01−2.14
S2RoPE-SH1−11.16−7.21−1.47
S2RoPE-SH2−11.29−7.45−1.82
S2RoPE-Wigner0−11.30−8.14−2.56
S2RoPE-Wigner1−11.49−8.44−3.11
S2RoPE-Wigner2−11.52−8.07−2.28

The additional metrics support these findings. Wigner achieves higher mean ACC than 2D RoPE and GeoRoPE at every reported lead. For log-spectral error, all three runs of both S2RoPE variants outperform all runs of both baselines at every reported lead.

We have added the full mean and sample-standard-deviation results for all five methods, seven lead times, and three metrics to the appendix subsection “Variation across training runs.” With three runs per method, we do not interpret the reported standard deviations or the separation of the observed runs as a formal test of statistical significance.

We hope these results address your concern, and we are happy to clarify anything further.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions and discussion

We thank the reviewers and the area chair for their time and constructive feedback. Changes are marked in blue in the revised PDF. For ease of reference, we summarize the additional results below:

  1. Matched latent-noise comparison. Table 1 and Figure 2 include zero-noise No RoPE alongside the default-noise reference. The main-text paragraph “Effect of latent training noise” discusses the accuracy and spectral comparisons, with complementary physical-consistency diagnostics in Figure 3.

  2. Variation across training runs. The appendix subsection “Variation across training runs” reports means and sample standard deviations across three training runs for five WeatherGenerator configurations, covering RMSE, ACC, and log-spectral error. Individual-run RMSE results are provided in our follow-up response to Reviewer fdg3.

  3. Additional forecasting backbone. LaDCast extends the evaluation to probabilistic latent diffusion forecasting, with ensemble-mean RMSE and ACC and ensemble CRPS reported in Table 3.

  4. Additional baseline. We evaluate StereoRoPE as an additional baseline in both WeatherGenerator and LaDCast.

  5. Additional analyses. New appendix subsections report tropical cyclone tracking and forecast performance across latitude bands.

Detailed responses are provided in the individual review threads. We would be happy to provide any further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.