acceptodds
Under review as a conference paper at ICLR 2027

Learn from the Core, Fit to the Data: Deep Koopman Learning of the Atmosphere

19 views

Abstract

Deep Koopman learning typically consists of dictionary learning followed by operator fitting, but we find that this pipeline can severely overfit, especially when the rank of the fitted operator is overspecified, because the same stochastic transitions select the features and fit the operator. We study two ways to mitigate this issue: separating the data used for dictionary learning and operator fitting, and using the forecasts of an approximate prediction model, which we call a core model, to train the dictionary. Our main finding is that core-trained dictionaries can substantially improve the held-out VAMP-E score over data-trained ones. On ERA5 and a controlled stochastic shallow-water benchmark, the advantage becomes more pronounced at long lags, and in the benchmark also under stronger transition noise.

Primary area:
learning on time series and dynamical systems
Keywords:
Koopman operator, operator learning, physics-informed machine learning, predictability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_W6UpOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates overfitting in deep Koopman learning, particularly when dictionary learning and operator fitting reuse the same stochastic transitions. The authors propose a model-assisted approach that learns the dictionary from forecasts of an approximate dynamical core and fits the Koopman operator using observed transitions. Experiments on ERA5 and stochastic shallow-water equations show improved held-out VAMP-E scores, especially at longer lags and under stronger stochastic forcing.

Critical strengths
  1. The paper identifies a meaningful generalization issue in deep Koopman learning and provides a clear analysis of how reusing stochastic transitions can lead to overfitting.

  2. The proposed separation of dictionary learning and operator fitting is intuitive and effective. The experiments are thorough, especially the controlled stochastic forcing experiments and independent operator refitting analysis, which help explain where the improvements come from.

Critical weaknesses

The unique benefit of using a dynamical core is not fully established. Independent operator refitting already recovers most of the performance gap between data-trained and core-trained dictionaries. It remains unclear whether the additional improvement comes from physically meaningful dynamics or simply from using a deterministic, noise-free training target. More comparisons with alternative approaches would help clarify the necessity of the dynamical core.

Questions for authors
  1. How does the proposed method compare with standard data splitting or cross-fitting, where dictionary learning and operator fitting use independent subsets of the observed transitions under the same total data budget?

  2. Could a deterministic learned forecasting model replace the dynamical core for dictionary learning? Such a comparison would help clarify whether the improvement specifically comes from physical dynamics.

  3. How sensitive is the advantage to the choice of operator rank and shrinkage? In Appendix D.3.2, data-trained dictionaries outperform core-trained ones at 5- and 7-day lags when both hyperparameters are independently selected. Could the authors further discuss this result?

4 comments
AuthorsOpenReview ↗

Response to Reviewer W6Up [1/2]

We thank the reviewer for recognizing the diagnosis and finding the method intuitive and effective. Each alternative the reviewer raised trains a dictionary that scores below the core-trained one: a noise-free target without dynamics scores at the data-trained dictionary's level, a learned forecaster scores 12.7–14.6 lower in ERA5, and an equal-budget split closes 9–15% of the gap. The conventions are at the end of the general response.

W1 and Q2. A noise-free target, or a learned forecaster, in place of the core. A learned forecaster trains a dictionary within 1.4–3.2 of the core-trained one only where separate trajectories can be simulated. On ERA5's single record it must be fitted either on the training years, which reuses their transitions, or on four held-back years. Either way its dictionary scores below the core-trained one.

A noise-free target without dynamics fails (W1). It applies only the core's diffusion and step filter, which multiply each spherical-harmonic coefficient by a fixed factor of at most one, so it only damps the state. In the shallow-water benchmark it scores 0.6–2.7 at the fixed setting, about the data-trained dictionary's level, against 19.8–21.3 for the core.

A more accurate linear forecaster is not a better target (Q2). We tested the linear inverse model (L72), a least-squares linear map of the state's leading empirical orthogonal function (EOF) coefficients to their values one lag later. Unlike the dissipation, it mixes the coefficients: it is a linear approximation of the dynamics, fitted to data.

In ERA5, fitted on 2008–2011, years no dictionary trains on, the forecaster is more accurate than the core, yet trains a dictionary 12.7–14.6 below the core-trained one in every seed, at either setting. Its held-out error is 7% above persistence's, against 48% for the core. With 5,728 states for 1,540 EOF coefficients, it nearly memorizes those four years (0.18 times persistence's error there).

With rank and shrinkage tuned on the training years, the dictionaries whose targets contain the observed futures collapse, because that tuning rewards features fitted to their noise.

dictionaries trained on 1979–2007held out, fixed settingheld out, tuned on the training years
observed futures (data-trained)5.8–6.7about −31
linear forecaster, fitted on 2008–20117.1–7.49.6–9.8
dynamical core19.8–20.823.8–24.4

We also fitted the forecaster on the training years 1979–2011, the transitions the operator also uses, so its target contains their observed futures. Its dictionary then scores 13.2–14.5 at the fixed setting, against 20.3–21.9 for the core, and −11.3 to 7.9 with rank and shrinkage tuned on those years.

In the shallow-water benchmark, fitted on separate trajectories, the forecaster meets both conditions of our argument and trains a good dictionary, 1.4–3.2 below the core. Fitted on the operator's own transitions, it scores below the forecaster fitted on separate trajectories in every seed, since its target then contains their noise. Seeds 0 / 1 / 2:

target of the dictionaryfixed settingtuned on training trajectories
the observed futures (data-trained)3.21 / 0.63 / 2.24−10.68 / −10.97 / −11.42
linear forecaster, same transitions14.37 / 12.91 / 10.745.46 / 3.72 / 12.03
linear forecaster, separate trajectories16.96 / 18.19 / 18.0417.24 / 17.98 / 18.16
truth's equations (the accurate core)19.77 / 20.35 / 21.2718.63 / 19.83 / 20.22

The dynamical core needs no data beyond the present states it starts from, so a single record such as ERA5 does not limit it.

AuthorsOpenReview ↗

Response to Reviewer W6Up [2/2]

Q1. Splitting the data at the same budget. The refit closes three quarters of the gap (L438–443) only with twice the data. At the same budget, splitting halves the data and closes 9–15% of the gap (shallow-water benchmark, fixed setting).

In the split, the data-trained dictionary trains on one half of the data and its operator is fitted on the other, then the reverse, and the two scores are averaged. In ERA5, with alternate years of 1979–2010 (the split on half of the pairs), its dictionary scores 2.0–2.3 at the fixed setting, below the data-trained dictionary (3.8–4.7), while ours scores 16.8–20.2. Tuned on the training years, it scores 8.6–11.9 below ours.

The refit shows that the reuse causes the gap. It keeps the data-trained dictionary and fits its operator again on a regenerated set of trajectories, which a simulation can supply and the atmosphere's single record cannot. The core target needs no such set, and its operator is fitted on every observed transition. The revision will call the refit a diagnostic, not sample splitting, and will add the comparison above.

Q3. Sensitivity to rank and shrinkage, and the 5- and 7-day reversal. At 10 and 14 days ours scores above the data-trained dictionary however rank and shrinkage are chosen. At 5 and 7 days it depends on the rank: the reversal in App. D.3.2 comes from choosing the rank on halves of the held-out years (L616–617).

There each dictionary picks rank and shrinkage on one half of the held-out years and is scored on the other. The core-trained dictionary scores 4.1–9.7 higher on one half than on the other, so a pick made on one half can miss on the other. At 5 and 7 days the data-trained dictionary keeps all 200 functions at the strongest shrinkage in App. D.3.2's search, rank 200 at shrinkage 1, on both halves (L652). At short lags it overfits less, so its later functions still add held-out score once shrunk. This raises its score by 7.0–8.0 at 5 days and 3.4–4.1 at 7 days, enough to put it above ours. With the rank held at 60, ours scores 2.5–2.8 above it at 7 days in every seed, and −0.5 to +0.7 at 5 days.

Ours scores above the data-trained dictionary at every lag in every seed when rank and shrinkage are chosen on years before the held-out years, with every dictionary retrained without those years. It still scores above with the shrinkage searched up to 100, a range that contains the picks of App. D.3.2. Ours minus the data-trained dictionary:

tuned on5 days7 days10 days14 days
half of the held-out years 2012–2020 (App. D.3.2)−6.8 to −3.1−3.9 to −0.8+4.2 to +5.2+4.4 to +6.4
2008–2011, every dictionary retrained without them, shrinkage up to 0.1+5.4 to +7.4+4.9 to +5.5+5.8 to +8.6+8.1 to +10.3
the same, shrinkage up to 100+3.9 to +4.1+6.1 to +9.4+9.4 to +11.4+9.3 to +10.8

If these answers address your questions, we would be grateful if you would consider raising your rating.

Thank you for the addtional experiments and the detailed and thoughtful responses. I appreciate the experiments on the noise-free target, learned forecaster, and equal-budget data splitting. These comparisons help clarify the specific benefit of using a dynamical core, and provide more convincing evidence that the improvement cannot be simply explained by denoising or data splitting alone. Also, the additional analysis of rank and shrinkage gives a reasonable explanation for the results at shorter lags.

Generally speaking, I think these new experiments have addressed most of my concerns, and I encourage the authors to include these results and discussions in the revised paper. Overall, I remain positive about this work and will keep my original positive score.

Thank you for evaluating our responses and for your positive assessment. We will definitely add the new controlled experiments and their discussion to the paper.

Reviewer_WMYNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates overfitting failure within two-stage Koopman-style models, due to using the same stochastic transitions to both learn the dictionary and fit the operator. The main contribution proposes an approximate deterministic "core" to train the dictionary, while still fitting the final operator to observed data. Experiments on ERA5 and a stochastic shallow-water benchmark show improved held-out VAMP-E, especially at longer horizons and stronger stochastic forcing.

Critical strengths
  1. Relevant overfitting issue, particularly for long horizons and high-dimensional operators.
  2. Experiment on the independent operator-refitting clarifies reusing the same transitions contributes to generalization gap.
  3. Shallow-water experiment allows the effect of stochastic forcing to be isolated.
  4. The core is only used to learn the representation, not as the final predictor. The identity-target experiment also suggests that the benefit comes from useful dynamical information, not just noise removal.
Critical weaknesses
  1. The general overfitting problem is not new. The potentially novel part is using an approximate dynamical model as a training target for the dictionary. However, the paper does not clearly establish why this is preferable to existing approaches of controlling Koopman model complexity.
  2. Most experiments compare data-trained and core-trained variants within the authors' framework. There is little comparison with alternative approaches such as stronger regularization, feature/mode selection, or other deep Koopman methods. This makes it difficult to tell whether the proposed method is preferable to simpler alternatives. The sample-splitting experiment is useful but does not resolve this broader question.
  3. The paper shows that rank and shrinkage strongly affect performance, suggesting that operator complexity itself is an important part of the problem. It would be useful to see whether a carefully tuned data-trained dictionary with stronger operator regularization can close the gap. Also, the rank sweep truncates the fitted operator but does not retrain the dictionary at different dimensions.
  4. It is unclear how broadly the effect holds across different operator estimators and
Questions for authors
  1. How does the method compare with carefully tuned data-only regularization and feature/mode selection?
  2. How does it compare with existing deep Koopman/VAMP methods?
  3. How inaccurate can the core become before it becomes detrimental?
4 comments
AuthorsOpenReview ↗

Response to Reviewer WMYN [1/2]

We thank the reviewer for recognizing the overfitting problem and the controlled experiments. The simpler alternatives the reviewer asks about do not close the gap: with tuned weight decay, the data-trained dictionary scores 7.3–10.3 below ours, and VAMPnet with our 200 learned functions fails to train in any seed of the table below. The conventions are at the end of the general response.

W1, W2 and Q1. Tuned complexity control and mode selection. Tuned complexity control does not close the gap. Tuned on 2008–2011, weight decay lifts the data-trained dictionary by at most 0.8, which leaves it 7.3–10.3 below ours. Trained on 1979–2011 and tuned instead by cross-validation on those years, the data-trained dictionary scores about −24 held out, worse than no operator: selection on the training years rewards features fitted to their noise.

Choosing the rank, which the paper already does on validation rows (App. D.3.3), is the mode selection the reviewer asks about. New here are weight decay and one tuning for every method: each dictionary retrained on 1979–2007, tuned on 2008–2011 and scored once on 2012–2020. Seeds 0 / 1 / 2:

200 learned functionstuned on 2008–2011fixed setting
ours (core-trained), held-out score23.9 / 22.3 / 24.420.8 / 19.8 / 20.8
ours minus the data-trained dictionary+9.7 / +8.1 / +10.3+14.6 / +13.1 / +15.0
ours minus the same with weight decay+9.0 / +7.3 / +10.3+11.9 / +11.7 / +10.3
VAMPnet, published settings (Mardt et al., 2018)fails to train in every seedfails to train in every seed

Complexity control does not close the gap because the overfitting comes from the reuse: when the same noisy transitions train the dictionary and fit the operator, the dictionary selects features correlated with their noise. Complexity control lowers the variance of the fit but keeps the reuse. The core target removes the reuse and discards no data. What is new is this cause in two-stage learning and the error terms it inflates (reply to ayL7, W2).

Q2. Existing deep Koopman and VAMP methods. VAMPnet does not train in any seed of this protocol (table above), and fails in 10 of the 21 runs we made with 200 learned functions and in 8 of 8 with 400. Trained on 1979–2011, it trains in 5 of 9 seeds. In those five, it scores 10.6–13.0 below ours at the fixed setting, and tuning it on the training years collapses it on the held-out years in three.

Its score inverts the minibatch second moments, which approach singularity as the number of learned functions approaches the batch size, while our low-rank objective holds no inverse (L312). Trained on the core's forecasts, VAMPnet also fails in 3 of 3 seeds, so the failure comes from its objective.

AuthorsOpenReview ↗

Response to Reviewer WMYN [2/2]

W3. Stronger operator regularization, and retraining with other numbers of functions. Neither closes the gap. Ours scores 13.1–15.0 above the data-trained dictionary at the fixed setting (table above), and 9.3–10.8 above with rank and shrinkage tuned on 2008–2011 and the shrinkage searched up to 100 instead of 0.1.

We also retrained every method with 20–400 learned functions, which is the feature selection the reviewer asks about. Ours is best at 200. With its number of functions chosen by the held-out score, which favors it, the data-trained dictionary still scores 6.7–8.6 below ours at 14 days and 4.7–8.4 below at 7 days.

Q3. How inaccurate can the core become? Much less accurate than a useful forecast, because the dictionary needs the core's dynamics, not its accuracy. In ERA5 at 14 days the core's error is 48% larger than persistence's, and it still trains the best dictionary.

In the shallow-water benchmark the targets we tried fall into two groups (fixed setting, seeds 0–2). The data-trained dictionary scores 0.6–3.2.

above the data-trained dictionaryheld-out score
truth's equations integrated 10 days, the lag (the accurate core)20–21
the same integrated 1, 2, 14 or 20 days12–24
a hundredth of the viscosity20.3–20.9
without the Coriolis force11.2–12.3
the accurate core with noise at 1.6 times persistence's error11.9–12.1
near the data-trained levelheld-out score
the accurate core with noise at 5.1 times persistence's error3.6–4.4
the core's dissipation alone, no dynamics0.6–2.7

A core's score falls as its dynamics depart from the truth's, as without the Coriolis force, and comes near the data-trained level only with noise at five times persistence's error or with no dynamics. Forecast error itself does not order the cores: the core integrated 14 days forecasts the 10-day state better than the core integrated 2 days, yet trains a worse dictionary (14–17 against 23–24). Proposition 1 allows this: the operator fit corrects the core's error inside the dictionary's span.

W4. Other operator estimators. Under three estimators (canonical correlation, reduced-rank regression, principal components of the present-state features) the core-trained dictionary scores above the data-trained one at the fixed setting in all 27 cells (three estimators, 7, 10 and 14 days, three seeds). The review's W4 breaks off after "operator estimators and", and we would be glad to address the rest.

If these answers resolve your concerns, we would be grateful if you would consider raising your rating.

Reviewer_WMYNOpenReview ↗

response

Thank you for your comments and clarifications. I will retain the score as marked.

AuthorsOpenReview ↗

Response to Reviewer WMYN

Thank you for reading our responses. Could you tell us which concern remains, so that we can answer it before the discussion closes?

The review’s W4 also breaks off after “operator estimators and”, and we would be glad to address the rest of it.

Reviewer_ayL7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the overfitting problem in deep Koopman learning: when the same set of stochastic state-transition samples is used simultaneously for both feature dictionary learning and operator fitting, the learned operator may perform well on the training set while exhibiting significant degradation on held-out data. The authors propose a two-stage approach: first, features for current and future states are trained using deterministic predictions generated by an approximate physical dynamical kernel; subsequently, the Koopman operator is fitted using actual observed transitions. The paper further analyzes—via projection relationships within the feature space—which types of errors in the dynamical kernel can be corrected during the subsequent operator-fitting stage. Experiments conducted on the ERA5 dataset and a spherical shallow-water model with stochastic forcing demonstrate that this method achieves higher VAMP scores, particularly under conditions involving longer time lags or stronger stochastic forcing. Additionally, the study includes experiments designed to quantify the errors introduced by data reuse.

Critical strengths

(1)The paper focuses on the overfitting issue that arises when dictionary learning and operator fitting are applied to reused stochastic transition data. It identifies the sources of this overfitting through VAMP-E error decomposition, comparisons between training and testing performance, and experiments involving the refitting of independent trajectories. The experimental results provide compelling support for the proposed mechanism. The problem is clearly defined, and the mechanistic analysis is robust. (2)The paper proposes an approach that leverages a deterministic dynamical core to learn features, followed by fitting an operator using actual data—a strategy that is both conceptually clear and straightforward to implement. Experiments using ERA5 data and a controlled shallow-water model demonstrate the method's effectiveness under varying conditions of stochastic forcing and dynamical core errors. Furthermore, the paper provides a theoretical analysis explaining why a dynamical core—even when its predictions lack high accuracy—can still yield useful features. This physics-informed learning approach is highly practical, and the experimental validation is systematic.

Critical weaknesses

(1)Although the paper establishes a POD baseline based on a dynamical kernel and reports the state prediction error of that kernel, it lacks a direct comparison—using the same observable functions—between the learned Koopman operator and the original (or simply calibrated) dynamical kernel. Notably, under weak stochastic forcing, the POD baseline even performs better. Furthermore, the evaluation relies primarily on the held-out VAMP-E score; it lacks verification regarding physical interpretability, practical predictive gains, and computational costs, failing to sufficiently justify the necessity of learning a Koopman operator. The independent value of the learned Koopman operator, relative to existing dynamical kernels, remains unclear. (2)The dual-encoder architecture (Mardt et al., 2018), the low-rank dictionary learning objective (Ryu et al., 2024; Jeong et al., 2025), and the approach of using physical models to assist operator learning (Baddoo et al., 2023; Valva & Giannakis, 2025) are all grounded in existing literature, and the issue of overfitting in variational operator estimation has also been previously discussed (McGibbon & Pande, 2015). The primary innovation of this paper lies in combining feature learning for dynamical kernels with operator fitting on real-world data. However, the fact that re-fitting on individual trajectories can close approximately 70%–80% of the performance gap (albeit requiring additional data) suggests that the unique advantage of this approach—compared to general data-decoupling strategies—is not sufficiently compelling. Moreover, Proposition 1 characterizes the approximation error on the input side only under the "forecast-containment" condition; it neither controls the error on the output side nor guarantees that the learned feature subspace contains the principal Koopman singular functions of the true system—a concern underscored by the non-uniqueness of the objective function for deterministic dynamical kernels, as discussed in Section 3.3. Consequently, the incremental innovation over recent Koopman learning methods is limited.

Questions for authors

(1)For a given set of observable functions gg defined over future states, how do the predictions derived from a fitted Koopman operator compare to the results obtained by evaluating gg directly on the states predicted by the dynamical core? Does this discrepancy persist if a simple calibration step—using observational data—is applied to the dynamical core? Furthermore, how do the two approaches differ in terms of training and inference costs? Overall, what is the specific added value of this approach compared to existing dynamical cores? (2)While Figure 5 helps clarify the underlying mechanism, the re-fitting process utilizes additional trajectories. If the total budget of available observational data is held constant, how does this method perform in comparison to alternative strategies such as temporal data partitioning or cross-fitting? To what extent are the performance gains truly attributable to the physical dynamical information embedded within the dynamical core, as opposed to the decoupling between the training objective and the samples used for fitting? How can one disentangle the respective contributions of the physical dynamical core and the data-decoupling mechanism? (3)Given the non-uniqueness of the objective function for the deterministic dynamical core (as discussed in Section 3.3), what further evidence—beyond the analyses of feature persistence and subspace consistency provided in Appendices C.8 and E.3—can be offered to substantiate the physical or dynamical significance of the learned principal singular functions? The physical meaning of the learned modes remains unclear.

4 comments
AuthorsOpenReview ↗

Response to Reviewer ayL7 [1/2]

We thank the reviewer for recognizing the diagnosis and its mechanism, and the method's practicality. The fitted operator predicts as well as a calibrated core without running the core at inference, and decoupling alone, at the same data budget, closes 9–15% of the gap. The conventions are at the end of the general response.

Q1 and W1. The operator against the core itself. On the same functions, the fitted operator predicts far better than the core's own forecast and as well as the core calibrated with data. Also, it needs no core run at inference: 0.11 ms per state against 109 ms, about a thousand times faster. It also gives what a core forecast does not: its singular functions pair patterns of the present state with the future functions they predict best, ranked by how well (L130–133).

We compare the three on the same 60 functions, the singular functions the operator keeps at the paper's rank, one lag ahead. The core's own forecast gives a held-out R² near or below zero. Calibrated as the reviewer suggests, it comes level with the operator or just below. The calibration is a linear least-squares map fitted on the training years. Its input is the dictionary's 200 future-state functions evaluated at the core's forecast, and its output is the observed future of each of the 60 compared functions. ERA5, held-out R² (the share of each function's held-out variance predicted), seed means:

7 days10 days14 days
fitted operator0.6540.6090.551
the core (the functions at its forecast)0.032−0.175−0.710
calibrated core0.6500.5950.531

Cost: the inference times above are on one V100. Training needs the core's forecast of each training state once (about 87 minutes), and every dictionary trained at that lag uses it. Each dictionary takes about 85 minutes.

W1. POD under weak forcing. POD's advantage under weak forcing comes from an oracle core, and a POD basis does not scale to a weather model's resolution. The benchmark's accurate core integrates the truth's own equations (L396). In ERA5, where the core is far from an oracle, both learned dictionaries score above the POD basis read at the core's forecast from five to fourteen days. A POD basis also needs the full state in coordinates, about a hundred million at a weather model's resolution.

Q2 and W2. Physical information against decoupling. Fitting the operator on data the dictionary never saw removes a large part of the overfitting, but it needs those data. The refit closed 70–80% of the gap with twice the data. At the reviewer's fixed budget, where the data are split in half, fitting the operator on unseen data leaves most of the gap. The core needs no such data, since its forecasts are independent of the observed futures and still have dynamics.

The refit shows that reuse causes the gap (W2). It keeps the data-trained dictionary and fits its operator again on a regenerated set of trajectories (L438–443), which a simulation can supply and the atmosphere's single record cannot. The revision will call it a diagnostic, not sample splitting.

At the same budget, splitting closes 9–15% of the gap in the shallow-water benchmark. We ran the temporal partitioning with cross-fitting that the reviewer names: the data-trained dictionary trains on one half (in ERA5, alternate years) and its operator is fitted on the other, then the reverse. This removes the reuse from the operator fit but halves the data of each step. In ERA5 it scores 8.6–11.9 below ours (reply to W6Up, Q1).

Of the alternatives, only a learned forecaster as the target comes within 1.4–3.2 of the core-trained dictionary, and only on separately simulated trajectories. The split's dictionary still trains on its half's observed futures. The core's dissipation alone is free of them but has no dynamics, and scores at the data-trained dictionary's level. On ERA5's single record the learned forecaster's dictionary scores 12.7–14.6 below the core-trained one (reply to W6Up).

AuthorsOpenReview ↗

Response to Reviewer ayL7 [2/2]

W2. Novelty and Proposition 1. What is new is the diagnosis of which error terms reuse inflates, and a target that removes the reuse from them. With UU and VV the spans of the present- and future-state features and PUP_U, PVP_V the projections onto them, the fitted operator's error splits into three terms in Hilbert–Schmidt norm (App., the fit's error terms):

∥K−K^∥2=∥K(I−PV)∥2+∥(I−PU)KPV∥2+∥PUKPV−K^∥2.\lVert \mathcal K-\hat{\mathcal K} \rVert^2 = \lVert \mathcal K(I-P_V) \rVert^2 + \lVert (I-P_U)\mathcal K P_V \rVert^2 + \lVert P_U\mathcal K P_V-\hat{\mathcal K} \rVert^2.

The three terms are (i) the output side, (ii) the input side and (iii) estimation. Reuse inflates (iii): the coefficients fit the noise of the futures the features were selected on. Reuse also makes (i) and (ii) look smaller on the training years than held out (Eq. 5). Sample splitting removes the reuse from (iii) only, since its dictionary is still trained on its half's observed futures. It also halves the data, so at the same budget it leaves most of the gap. Our target removes it from all three, because the observed futures enter only the operator fit (Sec. 3).

Under forecast containment, Proposition 1 says what remains in (ii): the core's error outside UU, ∥(I−PU)(K−Kcore)PV∥2\lVert (I-P_U)(\mathcal K-\mathcal K_{\mathrm{core}}) P_V \rVert^2 (L276). As the reviewer notes, no result bounds (i). That is why every claim is measured by held-out VAMP-E, which subtracts all three terms from ∥K∥2\lVert \mathcal K \rVert^2.

Prior work detects this overfitting or uses physics on the operator. McGibbon and Pande (2015) detect this overfitting by cross-validation over independent trajectories (App. I). We remove its cause, so the dictionary is trained and tuned on all the data. Earlier uses of a physical model estimate the operator from the equations without data pairs (Valva & Giannakis, 2025) or constrain the operator by known physics (Baddoo et al., 2023). We use the core only to train the dictionary, and fit the operator on the data.

Q3. The physical meaning of the singular functions. At 14 days two of the leading functions track the atmosphere's slow patterns, the annual cycle and the quasi-biennial oscillation. Each present-state singular function is a pattern of the present state, ranked by how well its paired future function can be predicted (L130–133). Beyond App. C.8 and E.3, new here are their correlations with climate indices, on held-out years and in all three seeds:

  • The second function tracks the annual cycle. It correlates with the cosine of the day of the year at 0.94–0.96, against at most 0.46 for random combinations of the same features. 99% of its variance is its calendar-day mean, and its spectrum peaks at one year.
  • The fifth function tracks the quasi-biennial oscillation, the reversal of the equatorial stratospheric winds every two years or so. With the calendar-day mean removed from both the function and the index, it correlates at 0.97–0.98 with a quasi-biennial oscillation index, against at most 0.33 for random combinations. The index is the area-weighted mean over 10°S–10°N of the zonal-mean zonal wind at the model's top level (about 60 hPa). Measured on its own held-out series, the function decays with an e-folding time of 103–107 days against the index's 112, and its spectrum peaks at the index's period, about 650 days.

Sec. 3.3's non-uniqueness concerns which span the encoder reaches. Inside that span, operator fitting on the observed transitions ranks the functions by the data (L285–287), and these two are the second and the fifth in all three seeds.

If these answers resolve your concerns, we would be grateful if you would consider raising your rating.

The authors have: (1) Addressed the direct comparison question in Q1; (2) Clarified the splitting strategy under a fixed data budget in Q2; and (3) Provided two plausible examples of physical modalities for Q3. I have one final question: Can the predictive advantage extend to pre-specified meteorological variables, and what is the specific contribution of the physics-based dynamical core relative to other sources of dynamical information?

We thank the reviewer for the follow-up. In short:

  1. At rank 200, the advantage extends to both pre-specified variables we measured, at all three lags.
  2. The core supplies physical dynamics without training a forecaster on observed transitions, and most of its error falls inside the learned span, where the operator fit can correct it.

1. Predictive advantage on pre-specified variables. At rank 200 ours is above the data-trained dictionary on both variables we measure, both with the calendar-day mean removed:

  • Zonal-mean zonal wind at the model's 8 levels in three latitude bands from 20∘20^\circ to 80∘80^\circ in each hemisphere, which places the jets.
  • Temperature at every grid point and level.

The operator forecasts each through its projection onto the singular functions it keeps, and R2R^2 is evaluated against the variable's original held-out anomalies. Across the ranks we tested, ours rises or stays level, while the data-trained dictionary falls from rank 60 to 200 on both variables at every lag. The wind results are mixed at lower ranks.

Held-out R2R^2 at 7 / 10 / 14 days, mean of 3 seeds:

rankwind, ourswind, data-trainedtemperature, ourstemperature, data-trained
200.049 / 0.022 / 0.0080.063 / 0.057 / 0.049not measurednot measured
600.100 / 0.077 / 0.0650.110 / 0.074 / 0.0420.046 / 0.022 / 0.0140.042 / 0.019 / 0.001
2000.168 / 0.108 / 0.0660.102 / −0.028 / −0.0960.069 / 0.036 / 0.0160.021 / −0.043 / −0.081

2. What the core contributes. The dynamical core is a physical model and only approximate, but the operator fit can correct its transition error inside the span. Under forecast containment (Sec. 3.3), for each learned future-state function h∈Vh\in V and target Ψ\Psi, Proposition 1 gives

(I−PU)Kh=(I−PU)(K−KΨ)h,(\mathcal I-\mathcal P_U)\mathcal K h = (\mathcal I-\mathcal P_U)(\mathcal K-\mathcal K_\Psi)h ,

so the best fit within the span misses only the part of the target's error outside the span.

The core's error falls mostly inside the core-trained span. On ERA5 we trained a correction dictionary on the core's error, and a projection onto the core-trained span explains 82% to 91% of the held-out second moment of its present-state features at one, three and seven days (App. H.1).

Our ablations support the contribution of physical dynamics to dictionary quality: in the shallow-water benchmark, removing the Coriolis force from the core lowers its dictionary's held-out VAMP-E score from 19.8–21.3 to 11.2–12.3, and keeping only its dissipation lowers it to 0.6–2.7, the data-trained level (replies to WMYN and W6Up).

The core avoids fitting an additional forecaster to observed transitions. In the benchmark, a forecaster trained on separately simulated trajectories comes within 1.4–3.2 of the core. On ERA5, a forecaster trained on held-back years still yields a weaker dictionary, 7.1–7.4 against 19.8–20.8 at 14 days (reply to W6Up).

The revision will add both pre-specified variables with the rank comparison and this account of the core’s contribution to the paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their constructive feedback. We first note that our contribution is twofold:

  • (1) diagnosis. Deep Koopman learning overfits because it reuses the same transitions to train the dictionary and to fit the operator. Splitting the data between the two stages only partly eases this.
  • (2) solution. Training the dictionary on the dynamical core's forecasts, which are generated once, shared across trainings, and need no held-out data.

All three reviewers ask whether something simpler than the (2) solution would remove this overfitting: separating the data, a noise-free target, regularization, or a different Koopman learning method.

Main argument on the rebuttal. We aim to make the following argument of the paper more solid, by testing it against each of the above alternatives:

The gain over the data-trained dictionary needs a training target for the dictionary that has dynamics and that is independent of the observed futures in the training data. The target need not forecast accurately.

The core's forecasts have both properties, and each is necessary. Without dynamics, a noise-free target fails (first highlight below). With the observed futures in the target, the dictionary fits the transitions' noise, which does not recur on held-out years (the paper's Section 2).

The core need not forecast accurately. In ERA5 at 14 days the core's error is 48% larger than that of persistence, the present state used as the forecast. It still trains the best dictionary.

Highlights. Scores and settings are defined in the Conventions at the end.

Avoiding the reuse in other ways trains dictionaries that score below ours.

  • Without dynamics, a target fails even when noise-free and independent of the observed futures. In the shallow-water benchmark, the core's dissipation alone, which only damps the state, scores at most 2.7 against about 20 for the core (replies to W6Up and WMYN).
  • At the same data budget, splitting the data closes 9–15% of the gap, the held-out score by which ours exceeds the data-trained dictionary. The paper's refit closes three quarters only with twice the data (shallow-water benchmark, replies to W6Up and ayL7).
  • On ERA5's single record, a dictionary trained on a learned linear forecaster scores below ours. The forecaster, fitted on years no dictionary trains on, forecasts more accurately than the core, yet its dictionary scores 9.6–9.8 against 23.8–24.4 for ours. In the shallow-water benchmark, fitted on separate trajectories, its dictionary comes within 1.4–3.2 of the core-trained one (reply to W6Up).

Regularization, tuning and VAMPnet do not close the gap.

  • With tuned weight decay, the data-trained dictionary scores 7.3–10.3 below ours (Table 1, reply to WMYN).
  • VAMPnet (Mardt et al., 2018), an existing deep Koopman method, fails to train in 10 of 21 runs with 200 learned functions, every seed of Table 1 among them. Where it trains, it scores 10.6–13.0 below ours at the fixed setting (reply to WMYN).
  • With rank and shrinkage tuned on years before the held-out years, ours scores above the data-trained dictionary at every lag of 5–14 days. The 5- and 7-day reversal in App. D.3.2 comes from choosing the rank on halves of the held-out years (reply to W6Up, Q3).

The operator does not need the core at inference.

  • It predicts as well as the core calibrated with data, about a thousand times faster (reply to ayL7, Q1).

Table 1. Held-out VAMP-E. ERA5 at 14 days, every dictionary trained on 1979–2007 with 200 learned functions, rank and shrinkage tuned on 2008–2011, scored once on 2012–2020. Seeds 0 / 1 / 2.

dictionaryheld-out score
ours (core-trained)23.9 / 22.3 / 24.4
data-trained14.2 / 14.3 / 14.2
data-trained with weight decay15.0 / 15.0 / 14.2
VAMPnet (Mardt et al., 2018), published settingsfails to train in every seed

The revision will add these results to the appendix.

Conventions. Scores are held-out VAMP-E: each function the operator keeps adds at most 1, 0 predicts nothing, and a negative score predicts worse than nothing. ERA5 at 14 days with dictionaries trained on 1979–2011 unless stated. The shallow-water benchmark is the paper's stochastic model at forcing r = 0.3 and 10 days. Seeds 0–2 are separate trainings, and fails to train means the loss becomes NaN. L72 and the like are line numbers of the submission.

The operator fit keeps a rank and adds a shrinkage λtr⁡(C)/d\lambda \operatorname{tr}(C)/d to the diagonal of each d×dd\times d feature second-moment matrix CC. The fixed setting is the paper's rank 60 and shrinkage 0.01. Tuned settings choose both without the held-out years 2012–2020, in one of two ways: on 2008–2011, with every dictionary retrained on 1979–2007, or on the training years by 4-fold cross-validation. Either way the shrinkage is searched over 0–0.1 (up to 100 where stated).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.