- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes to apply the "Low Curvature" Jacobian variation loss term (taken from prior work) to autoregressive PDE forecasting. Prior Jacobian regularization approaches are argued to be too conservative; on the other hand, some form of stabilisation is known to work well for PDEs.
The text markets the method as a Koopman-based method, since the low curvature penalty pulls the latent dynamics towards linear, which essentially makes it locally/approximately Koopman (with slowly changing nonlinearities, i.e. curvature).
The paper presents theoretic error bounds for their approach, and empirically compares this regularization to a Dynamic Mode Decomposition-based regularization method, alongside experiments showing gains in the LCKoop regularization vs no-regularization scenario, on several popular PDE datasets, across several architectures.
Critical strengths
Your idea is novel, and is an interesting direction to pursue in the landscape of PDE rollout forecasting methods with regularization. As regularization often helps, and traditional regularization may be viewed as too conservative, this weaker stability penalty could bring certain benefits in the cases when a system itself is inherently unstable (and therefore, intuitively, cannot be truthfully represented by a stable latent trajectory).
The Koopman connection is interesting, conceptually. And, I find it is a good idea for PDE data, to try to bound the Hessian instead of the Jacobian.
Critical weaknesses
I did not find the Koopman framing to be used in the theory or proofs. Conceptually -- I agree with you, the low curvature regularization penalty Koopman-izes a model, making its latent dynamics "more linear". But, why does one need specifically Koopman theory here? Not simply a low curvature bound (which your proofs are written around), but Koopman theory? Sure, low curvature may be Koopman, but I don't see this as anything more than an interesting observation, in the current version of your paper.
As for the baselines, you already cite (Li et al., NeurIPS 2022, McCabe et al., TMLR 2023), but do not compare to them. On the other hand, I did not find any paper which does what your DMD regularization does: specifically, add a forecasting loss penalty from a window-fitted DMD as a PDE regularizer. A comparison, where the only baseline is to your own baseline, without prior art, is useful only as an ablation. Without a fair comparison to the already-established baselines for forecasting regularization, I do not see how your motivation is supported. Why, and in which scenario would one prefer LCKoop to other methods from the PDE field? And the choice here is explicitly not among solely Koopman-style methods, it's among PDE forecasting ones.
Koopman theory also has a wider variety of baselines than the ones you presented: the Koopman models (Lusch, Xiong, Cheng), which you already cite -- these would also be a nice addition, if the paper is marketed as Koopman.
Questions for authors
I have mostly one question, I phrased in several different ways for clarity:
How do you actually use the Koopman framing? What would change in your paper, if instead of the complicated Koopman theory you would talk only about the low curvature loss penalty? "Low Curvature is Koopman" -- what does this imply?
Suggestion, which would make me change the mark. Determine a specific scenario, where your LCKoop approach improves over stricter & conservative popular PDE forecasting regularization methods. Preferably -- both in theory and in practice. That would make your finding portable, and useful for the reader.
I would be interested in a paper which would show some unstable-system failure mode for other conservative regularization methods, and demonstrate that the curvature regularization approach does not fail here.
Your Koopman framing could also be useful here, if you were to show, say, that Koopman locally-regularized methods fare better at forecasting this instability.
4 comments
Thanks for your answer, I'd like to reiterate on question 1 before addressing everything else. As I said, I agree that LC may Koopman-ize the model, which is what your answer was about, I did not argue with that. But, I don't see why that matters in terms of actual takeaways, which is my main concern. If you were to strip all Koopman mentions from the paper -- what would change, for an outside reader? For someone who's reading your paper, someone who's in PDEs but doesn't know anything about Koopman theory: why should they care that "low curvature is/gives Koopman", as opposed to "low curvature makes the model more linear"?
Overall, I believe your paper could be written without any mention of Koopman: it's fundamentally about low-curvature being a good regularizer, and that does not need Koopman.
Response to Question1
We thank the reviewer for clarifying this concern. We agree that a PDE practitioner can use LCKoop simply as a low-curvature plug-in regularizer, without learning Koopman theory. This is our contribution to PDE forecasting: a practical alternative to more conservative stability controls that preserves nonlinear expressivity. The Koopman framing also highlights a separate contribution to the Koopman community.
For the Koopman community, our contribution is to identify a useful property of an affine Koopman transition that helps explain its forecasting benefits. Our analysis identifies its vanishing higher-order remainder as one such property and makes it transferable to a nonlinear model. This converts a property of a restricted affine transition into a soft constraint that can also be applied after a Koopman representation has already been learned.
We demonstrate this connection through an application of the plug-in to learned Koopman representations. Rather than choosing between a frozen linear Koopman transition and an unconstrained nonlinear refinement, one can add the plug-in penalty to interpolate between them. Section 6.5 demonstrates this setting directly and illustrates further applicability to the Koopman community: LCKoop can build on recent developments in Koopman autoencoders, retaining their learned representations while regularizing an added nonlinear refinement. Thus, for PDE readers, the mechanism is usable as a standalone regularizer; for Koopman readers, it explains and extends a specific benefit of the affine transition.
The current introduction places greater emphasis on the Koopman contribution. To make the practical takeaways clearer for PDE practitioners, we plan to add a short guide explaining when and how to apply LCKoop, without requiring familiarity with Koopman theory.
I am still unconvinced that this paper constitutes a contribution to the Koopman community. Since the method is effectively a low-curvature plug-in and does not directly utilize Koopman theory, framing this approach as anything other than "Koopman-inspired" is misleading.
Overall, my main concerns remain unresolved, I maintain my original score.
Response to reviewer q96u
We appreciate the reviewer’s constructive assessment, which helped us clarify the role of Koopman framing and strengthen practical evaluation. The changes include Section 4.1 error-direction revision, KNO results in Table 2, Section 6.4, comparing LCKoop with a dissipativity plug-in regularizer, and Section 6.5, directly testing LCKoop on a nonlinear residual transition built on a frozen learned Koopman representation.
Question 1. The paper is viewed primarily as a low-curvature penalty. How is the Koopman framing used, what would change without it, and what does “Low Curvature is Koopman” imply?
Response. We appreciate the reviewer’s question. We clarify the framing as “Low Curvature Gives Koopman.” Section 6.5 tests the applicability of the plug-in within a Koopman autoencoder framework: we freeze a pretrained encoder, decoder, and linear latent transition K, then refine only a nonlinear residual transition. Increasing FC reduces the directional Jacobian difference from K from 0.02254 (FC0) to 0.00023 (FC1; see Figure 7), showing that LCKoop makes the nonlinear transition locally more Koopman-like. Intermediate FC retains nonlinear forecasting accuracy: FC0.1 reduces 18-step MSE by 47.3% relative to frozen K and 50-step MSE by 60.4% relative to the unrestricted residual transition. Thus, LCKoop provides a plug-in interpolation between a learned linear Koopman transition and an expressive nonlinear refinement, balancing short- and long-horizon forecasting---a trade-off that also arises implicitly in neural operators. We also clarify that our curvature-based error-propagation proof reveals an implicit advantage of linear Koopman models, without relying on very specific, complex setting of full Koopman theory.
Question 2. The DMD comparison is considered an ablation rather than a fair comparison with established PDE forecasting regularizers. Why, and in which scenario, should one prefer LCKoop to other PDE forecasting regularization methods?
Response. We treat window-fitted DMD as an ablation and compare LCKoop with Li et al.'s dissipativity plug-in regularizer [1], a conservative plug-in penalty. The table reports ms9 MSE, including their combination:
Li et al.'s regularizer [1] uses additional synthetic states from a shell outside the attractor to encourage dissipative return and thereby stabilize long rollouts that leave the attractor-supported ergodic regime. Accordingly, dissipativity is effective for outer-sphere forecasts, but worsens average near-attractor Lorenz accuracy. In contrast, LCKoop improves the ordinary regime, where the relevant difficulty is error-direction curvature inside the bounded state radius. These results suggest that curvature reduction through LCKoop complements dissipativity on both KS and Lorenz, with LCKoop alone giving the largest improvement for near-attractor Lorenz forecasts.
Question 3. The paper does not identify a setting where other conservative regularization fails while local curvature regularization remains effective. Can the authors show such an unstable-system setting and demonstrate the effectiveness of LCKoop in that context?
Response. We appreciate the reviewer’s helpful suggestion. Figure 4 identifies a weakness of the ordinary forecaster within the near-attractor regime: error-direction curvature varies substantially across bounded-radius initial conditions, although dissipativity does not improve average accuracy there. Grouping these initial conditions by baseline error-direction curvature, LCKoop improves ms9 MSE by 15.3%, 37.5%, and 25.2% in the three upper-curvature quartiles; the lowest-curvature quartile instead degrades by 15.6%, with the high-curvature groups in (b) corresponding to regions that benefit from LCKoop in (a) (see Figure 4).
This trade-off indicates that LCKoop is useful where higher-order local error is pronounced. However, dissipativity regularization by Li et al. [1] substantially harms ordinary near-attractor Lorenz forecasting in this setting, and does not explicitly penalize error-direction curvature. On outer-sphere states, dissipativity is still appropriate because the problem is an out-of-distribution regime, where contraction toward the attractor is beneficial; we also stress that adding LCKoop further lowers forecasting MSE. Together with the KS result, this supports the interpretation that conservative radius control and local curvature control address distinct failure modes, and suggests complementary use of the two plug-in penalties.
[1] Z. Li et al. Learning Chaotic Dynamics in Dissipative Systems. NeurIPS, 2022.