acceptodds
Under review as a conference paper at ICLR 2027

OPINE: Orthogonal Projection in Invertible Nonlinear Embeddings for Turbulent Flows

Abstract

Proper orthogonal decomposition (POD) represents physical fields with orthogonal modes ordered by variance, so that one decomposition gives reduced coordinates in every dimension, but it needs many modes when the data lie near a curved manifold. Nonlinear autoencoders are more compact, but their information loss happens inside a nonlinear encoder, and a model trained with one latent dimension gives no model with a smaller one. We introduce Orthogonal Projection in Invertible Nonlinear Embeddings (OPINE), which learns an invertible nonlinear change of coordinates and performs POD in the transformed coordinates. Because the map is invertible, all information loss occurs in the POD projection, and ordering its basis after training gives reduced coordinates of every smaller dimension from a single model. Training alternates gradient updates of the map with a damped update that tracks the evolving POD subspace, and we prove that this update converges locally to the exact POD subspace. We evaluate OPINE on synthetic manifolds, chaotic Kuramoto–Sivashinsky dynamics, forced Burgers turbulence, two-dimensional Kolmogorov flow, and three-dimensional minimal channel flow. OPINE is more accurate than POD at every tested retained dimension above one, and on the Kuramoto–Sivashinsky and Burgers dynamics, it is more accurate than convolutional and fully connected autoencoders by more than two orders of magnitude at the largest retained dimension.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
scientific machine learning, autoencoders, dimensionality reduction, optimization, invertible neural networks

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_CTnkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes OPINE, an architecture involving POD projection as a bottleneck layer between invertible nonlinear networks, and provides a methodological and theoretical perspective, coupled with demonstrations on synthetic data and turbulent data examples. The paper begins with problem framing and related work in the first two sections, introduces the method in section 3, with numerical results in section 4 and concluding remarks in section 5.

Critical strengths
  • This paper is very well written, the text and figures are complementary and present the material in a clarifying way. I really enjoyed reading it.
  • I think the structure the paper follows is excellent, and that the objectives and ablations are clearly stated (I have not read the appendix in detail however.) The use of POD as the only bottleneck mechanism pairs with the use of exactly-invertible networks, to produce a `good' hierarchical compression model for high-dimensional data.
  • The step for training `through' the eigendecomposition is methodologically elegant, and that presentation is clear.
  • The examples lead from toy synthetic settings to more complicated turbulent settings, showcasing the methodology; the authors note that the point is not to outperform an unconstrained autoencoder.
Critical weaknesses
  • Overall, it is clear to me that, if one allows for a sufficient number of POD modes in the bottleneck layer, the invertible networks are near lossless, and would recover a good reconstruction, even if the latent space is not particularly meaningful; this is not only true for turbulent flows, its true for any dataset, it would also be true for image data sets of other sorts. Thus, as the bottleneck widens, the method is reasonably made for reconstruction. Is it possible that this comes at a price, and does not give good reduced coordinates for subsequent inference (of, e.g., a simple dynamical system)?
  • Training invertible networks is often hard (due to stiffness) and costly; additionally, stacking a larger number of blocks does not automatically yield a more expressive model that is easy to optimize. I would be interested in understanding how expensive training OPINE is, compared to a comparable CNN or autoencoder.
Questions for authors
  • Following the first bullet from above, a subsequent simulation demonstrating, as a proof of concept, that the reduced coordinates (especially in a small-coordinate-limit) are good latent coordinates for a reduced-order-model to evolve would strengthen the motivation for using the method. If the authors can address that firs bullet in another way, I will consider that evidence as well.
  • I would like to see a comparison of the training cost between the proposed methods.
  • Can the authors clarify the remark regarding training for a fixed DD? I do not fully understand how performance degrades when one trains OPINE on a fixed D′D' and then evaluates on D<D′D<D', vs. training and evaluating on DD. Is there a loss that can simultaneously ask for good performance for several values of D?
4 comments
AuthorsOpenReview ↗

Response to Reviewer CTnk (Part 1/2)

We thank the reviewer for the careful review and for the kind words about the writing and about training through the eigendecomposition. The main question, whether accurate reconstruction comes at the price of poor reduced coordinates, is the right one to ask of this method, and we spent much of the revision on it. New results are in Section 4.7 and Appendix D of the revised PDF, marked in red.

Weakness 1 and Q1. Does the reconstruction come at the price of poor reduced coordinates? Our short answer is that we found no such price, but also no dynamical advantage. At small DD the OPINE coordinates are meaningful and support a reduced-order model as well as POD and autoencoder coordinates do; they do not make the dynamics more predictable. What they add is a transformed field of the size of the data that keeps all of its information and can be inspected directly. In more detail:

Small DD. The reviewer is right that a wide bottleneck makes reconstruction easy. This is why we compare all methods at small, equal DD; in OPINE the rank-DD projection is always the only lossy step. The gain comes from that projection, not from the map alone: with the same map, keeping the first DD entries of zz is usually no more accurate than POD, whereas the POD projection of zz is much more accurate (Tables 1 and 3). On Kuramoto–Sivashinsky (KS), the error drops sharply between D=6D=6 and D=8D=8 (Table 2), the inertial-manifold dimension estimated by Linot & Graham (2020).

Meaning of the coordinates. They are ordinary POD coordinates of the transformed variable, ranked by variance, and the reconstructions at all DD are exactly nested. Decoded to physical space, the leading directions look like POD modes, with the same shape, ordering and pairing (Figure D4 for KS and Burgers, Figure 5 for Kolmogorov flow); the nonlinear map makes the responses to positive and negative amplitudes differ.

A reduced-order model, as you suggested. We trained the same model ηt+1=ηt+gϕ(ηt)\eta_{t+1}=\eta_t+g_\phi(\eta_t) in POD, OPINE and MLP-autoencoder coordinates on KS, with the same architecture search, training and seeds (Section 4.7, Appendix D.9). All long rollouts are stable. At D=8D=8 the model in OPINE coordinates reproduces the joint distribution of (ux,uxx)(u_x,u_{xx}) better than the models in POD and autoencoder coordinates, and as well as models with D=16D=16. We want to be clear about the limits of this result: its short-time error is lower only because the forecast starts from a more accurate reconstruction, and its forecasts do not stay accurate longer than in POD coordinates.

What OPINE offers instead. We think this is another highlight of the method, which could be useful. Because the map is invertible, the transformed variable z=fθ(x)z=f_\theta(x) is a field of the same size as the data that keeps all of its information (similar to using vorticitiy or streamfunction to express the same field), and the reduction is an ordinary POD of that field. One can therefore look at exactly what the reduced model works with. The new Figures D5 and D6 show zz, aligned with the data, next to the physical field. On KS at D=8D=8 the two are almost indistinguishable. On Kolmogorov flow, zz keeps the large vortices but moves variance from the energetic scales to smaller ones, and less so as DD grows (Appendix D.8).

Weakness 2 and Q2. Training cost. On the turbulent flows OPINE costs more to train than the CNN autoencoder: on one V100 GPU a minimal-channel run takes about 14 h against 1.2 h, with several times the memory, and on Kolmogorov flow training takes about 2.5 times as long (Table D1). On KS and Burgers a run takes about a minute, somewhat longer than a tuned MLP autoencoder and much less than a tuned CNN autoencoder (Figure D1).

On stiffness and depth: with the coupling log-scales bounded and the damped update, we observed no divergence in any OPINE run, and a stack with seven times fewer weights is nearly as accurate as the larger one (Table 5). One OPINE model also gives every smaller DD (Q3), which matters for the total cost when several dimensions are needed.

AuthorsOpenReview ↗

Response to Reviewer CTnk (Part 2/2)

Q3. Training at D′D' and evaluating at D<D′D<D', and a loss over several DD. We agree that our remark in the paper was too brief. A model trained at D′D' is evaluated at D<D′D<D' by keeping the first DD of its eigenvalue-ordered POD modes:

x^D=fθ−1(μz+QDQD⊤(fθ(x)−μz)).\hat x_D=f_\theta^{-1}\big(\mu_z+Q_DQ_D^\top(f_\theta(x)-\mu_z)\big).

These reconstructions are exact nested projections, but the map is trained only through the projection at D′D', so they are less accurate than a model trained at DD. How much depends on the system (Tables 6 and 7): on the minimal channel the prefixes are within a few percent of separately trained models, on Kolmogorov flow they are about twice as inaccurate, and on the synthetic families the gap can be much larger at small DD. The new Table D3 shows the general pattern: the further D′D' lies above DD, the closer the prefix comes to POD. Our sentence "one training run thus gives accurate models of every size" overstated this, and we have corrected it.

To the reviewer's second question: yes, a loss can ask for several DD at once, and we added one as an option (Appendix D.5):

L(θ)=∑r=1D′wr E∥x−x^r∥2/∑rwr,\mathcal L(\theta)=\sum_{r=1}^{D'}w_r\,\mathbb E\|x-\hat x_r\|^2\Big/\sum_r w_r ,

where each wrw_r is the inverse of the POD training NMSE at rr, so that every rr counts on the same scale (D′D' is written KK in the paper). All prefixes come from one encoding, at the cost of one inverse pass per rr. It involves a trade-off: it makes the intermediate prefixes several times more accurate, at some cost in accuracy at D′D' and in training time (Tables D3 and D4). On KS, one model trained this way is more accurate, averaged over all DD, than ordered autoencoders trained with the same loss, although autoencoders trained separately at each small DD remain more accurate.

We thank the reviewer again for a review that changed how we present the method.

AuthorsOpenReview ↗

Follow-up on Revised Experiments and Remaining Concerns

Dear Reviewer CTnk,

Thank you for your constructive suggestions. We wanted to briefly follow up during the discussion period. In the revision, we have added the reduced-order dynamics experiment you suggested, quantified the training-cost trade-off, and introduced and evaluated a multi-rank training objective.

We would appreciate your feedback on whether these additions address your main concerns and whether they warrant an updated assessment.

Thank you for your time and consideration.

Reviewer_CTnkOpenReview ↗

Response to Author Rebuttal

I thank the authors for their detailed responses and their willingness to engage with my comments. The new material and answers largely resolve my concerns, and I will be raising my score to a 3.

Reviewer_ZC8zOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes OPINE, a nonlinear dimensionality-reduction method for physical fields. The field is mapped by an invertible, dimension-preserving neural network fθf_\theta (Glow-style affine coupling layers with invertible channel mixing, plus a space-to-depth rearrangement for scalar fields). A rank-r POD projection is then applied in the transformed coordinates, and the result is mapped back with fθ−1f_\theta^{-1}. Because f_θ is bijective, the POD projection is the only lossy step. After training, diagonalizing the projected covariance orders the basis by variance, so a single model provides nested reduced coordinates for every r≤Kr \leq K. Training alternates gradient steps on θ\theta (basis fixed) with a damped, orthogonal-iteration step that tracks the POD subspace of the transformed data, avoiding backpropagation through an eigendecomposition. A local convergence result for this update is claimed (Appendix A). Experiments cover synthetic 1-D manifolds (with a projection ablation), Kuramoto–Sivashinsky, forced Burgers, 2-D Kolmogorov flow and a 3-D minimal channel, compared with POD and with CNN/MLP autoencoders at an equal number of retained coefficients.

Critical strengths
  1. A clean and useful design. Keeping POD as the only bottleneck, inside an invertible change of coordinates, gives ordered, nested and approximately interpretable modes from one trained model. It also gives an a-priori error estimate from the eigenvalues in transformed space. These are properties that practitioners in fluid mechanics value and that standard autoencoders lack. The bottleneck ablation shows that the POD projection, not the invertible map alone, drives the gains: keeping the first D entries of z, as nested-dropout flows do, performs no better than plain POD.

  2. Convincing results on 1-D chaotic and shock-dominated dynamics, consistent gains over POD. On KS and Burgers, OPINE is more than two orders of magnitude more accurate than the autoencoders at the largest D, and it shows a sharp error drop near the known inertial-manifold dimension of KS. It also improves on POD at every D on Kolmogorov flow and the minimal channel. The evaluation spans a meaningful range of complexity, up to 3-D wall-bounded turbulence.

Critical weaknesses
  1. The baselines and the parameter-count comparison are not adequate to support the claims.
  • The paper repeatedly states that OPINE has "far fewer parameters". This count excludes the POD basis QK∈RN×KQ_K \in R^{N\times K} and the mean μz\mu_z, which are learned from data and needed for decoding. This is potentially orders of magnitude more than the CNN. The comparison should count these.
  • The autoencoders "level off" at large D (KS, Burgers, minimal channel), which is a well-known symptom of under-tuned optimization rather than of an intrinsic limitation. Stronger and more relevant baselines are missing: POD-precompressed autoencoders (POD-DL-ROM, Fresca & Manzoni 2022), autoencoders with a POD/linear skip path, quadratic manifolds (Geelen et al., 2023), kernel PCA, and above all the closest methods, Neural-PCA (Li & Hooi, 2022) and Principal Component Flows (Cunningham et al., 2022), plus a nested-dropout or Matryoshka autoencoder for the nested-coordinate claim. These are discussed in the related work but only compared indirectly through the "first D entries" ablation on synthetic data.
  1. The theoretical contribution and the interpretation of "lossless" are limited.
  • The convergence result concerns the damped orthogonal-iteration update for a fixed map fθf_\theta, and is local. The paper acknowledges that it does not cover joint training. Convergence of damped, aligned orthogonal iteration to the dominant eigenspace for a fixed matrix is close to standard numerical linear algebra.
  • The decoder evaluates fθ−1f_\theta^{-1} at points z^r\hat z_r that generally lie off the image of the training data. Affine couplings with exponential scales can be poorly conditioned there. No analysis of the inverse's stability, or of out-of-distribution or longer-horizon test data, is provided.
  • Evaluation is reconstruction NMSE only. No downstream use (e.g. a reduced-order dynamical model in OPINE coordinates) and no physically motivated metrics (energy spectra, Reynolds-stress profiles for the channel) are reported, so the claimed suitability for model reduction remains untested.
Questions for authors
  1. What is the total storage of OPINE versus each baseline for every dataset and D? Does the "far fewer parameters" statement still hold?
  2. How do Neural-PCA (Li & Hooi, 2022) and Principal Component Flows (Cunningham et al., 2022) perform on KS, Burgers and Kolmogorov flow under your protocol?
  3. How were the autoencoders tuned (architecture, learning-rate schedule, training length)? Is the plateau at large D reduced with longer training or a residual/POD-skip design?
  4. How well conditioned is fθ−1f_\theta^{-1} at the projected points? Have you observed instabilities or artifacts when decoding at small rr, or for test states outside the training distribution (e.g. a later time window for Kolmogorov flow)?
  5. What is the wall-clock training and inference cost of OPINE (including the subspace steps on N×KN\times K bases) compared with the CNN autoencoder on the minimal channel?

Suggestions: 6. Physically meaningful error metrics (spectra, statistics) for the turbulent cases would complement NMSE.

2 comments
AuthorsOpenReview ↗

Response to Reviewer ZC8z (Part 1/2)

Thank you for the careful review and for engaging so closely with the method. Your main concern, that the baselines and the parameter count did not support our claims, was justified, and answering it changed several statements in the paper. All new results are in Section 4.7 and Appendix D of the revised PDF, marked in red.

Q1. Total storage. The reviewer is right that "far fewer parameters" left out the basis QKQ_K and the mean μz\mu_z. They are computed rather than trained, but they must be stored, so we now write "trainable parameters" and report the full stored model (Table D1). With them included, OPINE stores slightly more than POD on the turbulent flows. Against the CNN autoencoder the answer depends on the flow: OPINE stores less than half as much on Kolmogorov flow, but several times more on the minimal channel, where the basis is large. On KS and Burgers the weights of the map dominate.

Q2. Neural-PCA and Principal Component Flows. These are the closest methods, and we now compare with them directly (Appendix D.4, Figure D2). We trained both by likelihood with a coupling stack of the same kind as OPINE's and truncated them afterward. Neither is competitive in reconstruction: on KS and Burgers, Neural-PCA is about as accurate as POD and PCF is accurate only at the one dimension where its retained coordinates form a whole channel of the squeeze; on Kolmogorov flow, with OPINE's map, data and schedule, both are less accurate than POD at every DD. We think the reason is that both are trained by likelihood, whereas OPINE trains its map on the reconstruction error itself.

Q3. Tuning of the autoencoders. On KS and Burgers your suspicion was correct: the plateau was largely a matter of training length. In the submission, both autoencoders were trained exactly like OPINE (3,000 AdamW steps with a cosine schedule, keeping the best validation checkpoint; the CNN has four stride-2 circularly padded convolutions and a mirrored decoder, the MLP two hidden layers of width 256). Trained ten times longer, with the learning rate selected on validation data, they become up to an order of magnitude more accurate (Figure D1). OPINE remains more accurate at comparable training time, but by less than we claimed, so we have removed "more than two orders of magnitude" from the abstract. On the minimal channel the answer is different: training the CNN autoencoder three times longer, or with a larger learning rate, barely changes its test error while its training error keeps falling. There the plateau is a limit of generalization, not of optimization (Table D7).

The POD-skip design, as the reviewer suggested, x^=μ+VDξ+g(ξ)\hat x=\mu+V_D\xi+g(\xi), turned out to be one of the strongest baselines (Figures 3, 4 and D2). OPINE is more accurate on Burgers at every DD and on KS at the smaller DD. POD-skip is more accurate on KS at the larger DD (except at D=16D=16 with the faster basis update α=0.3\alpha=0.3) and on Kolmogorov flow up to D=128D=128, where it uses 30 to 80 times as many trainable weights as OPINE.

AuthorsOpenReview ↗

Response to Reviewer ZC8z (Part 2/2)

Q4. Conditioning of fθ−1f_\theta^{-1}. On KS the condition number of the Jacobian of fθ−1f_\theta^{-1} is of order 100, and it is no larger at projected points than at data points, except for a moderate increase at the smallest rr of one model (Appendix D.7). The round trip is exact to rounding error, and we observed no instabilities or artifacts.

Q5. Cost on the minimal channel. OPINE is more expensive on this flow, and the revision says so (Table D1). On one V100 GPU a training run takes about 14 h against 1.2 h for the CNN autoencoder, and encoding or decoding a snapshot takes about 11 ms against at most about 1 ms.

Suggestion 6. We agree that physical statistics say more than NMSE alone. For the minimal channel we compared the Reynolds-stress profiles and spectra of the reconstructions at D=256D=256 with those of the simulation (Appendix D.10, Table D6). OPINE is closer to the simulation than POD and the CNN autoencoder in almost every statistic, most of all near the wall and at small scales, where POD keeps almost none of the fine-scale energy and OPINE about a fifth of it.

Weakness 1. The baselines are covered in Q2 and Q3. In short, on KS and Burgers all methods use identical data and validation-selected hyperparameters. OPINE is the most accurate method on Burgers at every DD and on KS at small DD; at intermediate DD on KS the autoencoders are more accurate, and kernel PCA with a tuned kernel width at the largest DD. For nested coordinates, an ordered autoencoder has more accurate intermediate prefixes but is about two orders of magnitude less accurate at the full DD (Figure D3).

Weakness 2. We agree with this assessment. Theorem 1 is a local result for a fixed map, while Proposition 2 bounds the lag of the basis only while the map changes slowly, and Appendix D.6 shows that our KS and Burgers runs do not meet this condition; we make no claim about convergence of the joint training. In addition, the paper now reports how close the trained basis is to the exact POD of the final map (Table D5).

Weakness 3. Section 4.7 now trains the same reduced-order model in POD, OPINE and autoencoder coordinates on KS. In OPINE coordinates at D=8D=8 it reproduces the statistics of the flow as well as models with D=16D=16, which POD does not, but its forecasts do not stay accurate longer (Appendix D.9). We think the more distinctive use of OPINE is the one Appendix D.8 shows: because the map is invertible, the transformed field has the size of the data and can be compared with the physical field directly.

Thank you again. Your questions made the paper more accurate in several places.

Reviewer_HTPAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a novel method for capturing the major degrees of freedom for physical fields. Based on the classical Proper Orthogonal Decomposition (POD) method, the paper instead conducts POD on a transformed basis, where the basis transformation is a learned invertible map. Comprehensive experiments across 1D, 2D, and 3D with different PDEs are tested against classical baselines such as the vanilla OPD, CNN, and MLP.

Critical strengths

The paper is well-written and the idea is clever. It combines the benefits that classical POD has linear span optimality, while the neural networks are good at capturing and learning nonlinear structures. Under the current frame, classical methods such as spectral POD may also be put into a similar framework of training via FFT.

Critical weaknesses

See the questions below.

Questions for authors

The paper is sound in general. Here is the list of questions I have, where the first point is my major concern:

  1. In most of the experiments, the comparisons are conducted between OPINE trained at retained dimension DD and other baselines at rank DD. The only comparison of the setting "training at a retained dimension DD, evaluate and compare at rank r<Dr<D with other baselines at rank rr" is done in the appendix, specifically in Table 6 and 7. However, it seems like there's a general trend that as D−rD-r becomes larger and larger, the gap between the POD and OPINE at rank rr becomes smaller and smaller, while the gap between the OPINE at rank DD and rank rr becomes larger and larger. Therefore,

    (a)\textbf{(a)} The choice of DD in practice is very important, since either being too large or too small will cause trouble: being too small cannot needs an re-training if the dimension is not enough, while being too large means a more succinct representation can only be as effective as POD without much gain. My question is then: Is there any standard for choosing a good DD is practice, or at least a suitable range of DD? And how to choose it across different problems?

    (b)\textbf{(b)} If time permits, can you do another experiment that we fix a prefix rr for evaluation, and train at different DD, to see for a fix rr if larger training DD would be able to buy us anything?

    I will raise my score to 3 or 4 if the problems above are addressed. Edit: I've raised my score to 3.

  2. Can you conduct more comparisons to modern baselines, such as [1]?

References\textbf{References}:

[1] Mou, Changhong, Binghang Lu, and Guang Lin. "Neural-POD: A plug-and-play neural operator framework for infinite-dimensional functional nonlinear proper orthogonal decomposition." arXiv preprint arXiv:2602.15632 (2026).

3 comments
AuthorsOpenReview ↗

Response to Reviewer HTPA (Part 1/2)

We thank the reviewer for the review, and for stating clearly what would change the assessment. We ran the experiment the reviewer asked for in (b). Regarding (a), we tested training for several dimensions at once on Kolmogorov flow. Both are in Appendix D.5 of the revised PDF (Tables D3 and D8), with guidance on choosing DD. Section 4.6 and the Conclusion now state that the nested coordinates are most accurate near the dimension of training. All new and revised text is marked in red; changes made for all reviewers are summarized in the general response.

In short: with the paper's loss, a larger training dimension does not help at a fixed rr, as the reviewer suspected. But the difficulty described in (a) can be avoided. One model trained at a large DD with a loss over several dimensions comes close to models trained separately at each dimension, so rr can be chosen after training.

(b) Fixed rr, varying training dimension DD. Test NMSE at a fixed rr of models trained with the paper's loss at different DD (Table D3 has more rows):

trained at D=rD=rD=2rD=2rD=4rD=4rPOD
KS, r=4r=42.2×10−22.2\times10^{-2}1.6×10−11.6\times10^{-1}2.5×10−12.5\times10^{-1}2.5×10−12.5\times10^{-1}
KS, r=8r=84.9×10−54.9\times10^{-5}2.6×10−22.6\times10^{-2}–3.0×10−23.0\times10^{-2}
Burgers, r=8r=89.1×10−39.1\times10^{-3}2.0×10−22.0\times10^{-2}4.4×10−24.4\times10^{-2}5.4×10−25.4\times10^{-2}
Burgers, r=16r=166.7×10−56.7\times10^{-5}3.0×10−33.0\times10^{-3}–1.2×10−21.2\times10^{-2}
Kolmogorov, r=32r=320.2180.2180.2450.2450.3050.3050.3580.358
Kolmogorov, r=64r=640.0930.0930.1230.1230.1700.1700.1950.195

On all three systems the model trained at D=rD=r is the most accurate, and the error at rr moves toward that of POD as DD grows, which confirms the trend the reviewer saw in Tables 6 and 7. The reason is that training starts from POD and the paper's loss acts only at DD. When POD is already accurate at DD, the map changes the field only a little, and its leading modes stay close to those of POD (Appendix D.8).

One model for several dimensions. The trend in (b) comes from the loss, not from the method: the loss can ask for accuracy at several rr at once (Appendix D.5). On Kolmogorov flow we trained one model at D=256D=256 and drew one rr from {32,64,128,256}\{32,64,128,256\} for each batch, which takes about as long as an ordinary run (Table D8):

Kolmogorov, test NMSEr=32r=32r=64r=64r=128r=128r=256r=256
POD0.3580.3580.1950.1950.0800.0800.0260.026
trained at 256 only (paper's loss)0.3440.3440.1670.1670.0520.0520.0090.009
one model, loss over the four rr0.2760.2760.1170.1170.0370.0370.0120.012
trained separately at each rr0.2180.2180.0930.0930.0340.0340.0090.009

At all four dimensions the error of this one model is 10 to 27% above that of separately trained models, whereas the prefixes of the model trained at 256 only are 50 to 80% above them. How often each rr is drawn moves accuracy between small and large rr; Table D8 gives three choices.

On KS and Burgers, a loss over all r≤Dr\le D brings one model within a factor of about three of models trained separately at each dimension, except at the smallest rr on KS (Table D3). At r=12r=12 on KS it is more accurate than them. This loss about doubles the error at r=Dr=D and costs as much as 11 to 13 ordinary runs, which is still less than the 16 or 32 runs of a sweep over every dimension.

AuthorsOpenReview ↗

Response to Reviewer HTPA (Part 2/2)

(a) How to choose DD. We agree that the choice of DD matters, and the revision includes guidance in Appendix D.5 based on test experiments:

  1. When the dimension is known, train at it. This gives the most accurate model at that dimension.
  2. When it is not known, or several dimensions are needed, train once at an upper limit DD with the loss over several dimensions, and choose rr afterwards on validation data, without retraining. The upper limit is the dimension at which POD reaches the required accuracy, which the POD spectrum gives without training; OPINE was more accurate than POD at every D>1D>1 in our tests. The dimension DD can be chosen generously, because with this loss the model's errors at small rr no longer fall back to those of POD. A DD that is too small does require retraining: adding modes above DD does not reliably help, since the error does not fall on KS for D<8D<8, falls slowly on Burgers, and grows on Kolmogorov flow for D≤128D\le128. For example, for a target error of 5% on Kolmogorov flow, POD needs between 128 and 256 modes (errors 0.080 and 0.026), so D=256D=256 is an upper limit. The one model trained at D=256D=256 has an error of 0.037 at r=128r=128, so r=128r=128 is chosen without a second training.
  3. A sharp drop of the error against rr suggests a choice. On KS the error drops sharply between r=6r=6 and 8, the inertial-manifold dimension estimated by Linot & Graham (2020). On Burgers and the turbulent flows the error decreases steadily, and the dimension is set by the accuracy required.
  4. A sweep over DD is also affordable on small systems. On KS and Burgers a run takes about a minute, and in a small test, runs with a quarter of the training had errors within a few times of full runs and the same dependence on DD.

Modern baselines. The revision adds the quadratic manifold, kernel PCA, POD-DL-ROM, an ordered (Matryoshka) autoencoder, an autoencoder with a POD skip path, Neural-PCA and PCF, all on identical data (Figures 3, 4 and D2).

Neural-POD (Mou, Lu & Lin, 2026). We thank the reviewer for the reference, which we now cite. Neural-POD represents a field as u(x,t)≈Φ0(x)+∑p=1rΦp(x)Ψp(t)u(x,t)\approx\Phi_0(x)+\sum_{p=1}^{r}\Phi_p(x)\Psi_p(t), with neural networks fitted one mode at a time to the remaining residual. Its aims are a basis that can be evaluated at any resolution, used inside operator networks, and applied in reduced-order models at parameters without training snapshots. On a fixed grid its reconstruction lies in an affine subspace of dimension rr, so on the snapshots used to fit it, it cannot be more accurate than POD at the same rr. A first test on KS and Burgers, with our implementation of the settings in its authors' code, is consistent with this: its test NMSE was within 0.02% of that of POD at the dimensions of our benchmark, at r=16r=16 on KS after longer training (Appendix D.4). OPINE is 6 to several hundred times more accurate than POD there. We ran this test during the rebuttal, and it does not cover the uses Neural-POD is designed for. Because OPINE needs only an orthogonal projection in the transformed coordinates, other bases could also be used there, such as Neural-POD modes or, as the reviewer suggests, spectral POD; we mention this in the Limitations paragraph.

We hope these results answer the reviewer's questions, and we would be glad to discuss further.

Thanks the author for the comprehensive improvements of the original paper. The loss consisting of multiple rank rr up to dimension DD is indeed helpful based on the new resutls. I've raised my score to 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

Thank you all for careful and constructive reviews. Reviewer ZC8z asked whether our comparisons were fair: whether the autoencoders were tuned, whether the closest methods were included, and what OPINE really stores and costs. Reviewer CTnk asked whether the reduced coordinates are worth having beyond reconstruction, especially at small dimension. Reviewer HTPA asked how the retained dimension should be chosen when one model is also used at smaller dimensions; the revised PDF already contains the results for this question, and our reply to HTPA will follow shortly. We ran new experiments for all three questions, and some of the answers changed how we describe the method. All new and revised text in the revised PDF is marked in red. The main ones are:

  • A fairer comparison (ZC8z Q2, Q3, W1). Following ZC8z's suggestion, we trained the autoencoders ten times longer, which makes them up to an order of magnitude more accurate, and we added the quadratic manifold, kernel PCA, POD-DL-ROM, an ordered (Matryoshka) autoencoder, an autoencoder with a POD skip path, Neural-PCA and PCF, all on identical data (Figures 3, 4 and D2). Against these stronger baselines OPINE remains the most accurate on Burgers at every dimension and on KS at small dimensions; at intermediate dimensions on KS, and on Kolmogorov flow up to moderate dimensions, autoencoders are more accurate, the latter with many more trainable parameters. The abstract and the paper now state the comparison in these terms.
  • What OPINE stores and costs (ZC8z Q1, Q5; CTnk Q2). "Far fewer parameters" now reads "far fewer trainable parameters". We report the total stored model, including the basis, and the training time, memory and inference time against the CNN autoencoder. On the turbulent flows OPINE is the more expensive method.
  • What the reduced coordinates are good for (ZC8z W3; CTnk W1, Q1). A new Section 4.7 trains the same reduced-order model in POD, OPINE and autoencoder coordinates on KS. OPINE coordinates support it as well as the others, but they do not make the dynamics more predictable. What they offer instead is that the transformed field has the size of the data and can be compared with the physical field directly (Appendix D.8). On the minimal channel, OPINE's reconstructions are also closer to the simulation in most turbulence statistics (Appendix D.10).
  • Theory stated more carefully (ZC8z W2). Theorem 1 is described as a local result for a fixed map, Proposition 2 is restated in the update order used in training, and we measure how close the trained basis is to the exact POD subspace.
  • Nested coordinates and the choice of dimension (CTnk Q3; HTPA). The prefixes of one model are less accurate than models trained at each dimension, and they approach POD as the training dimension grows above the dimension used (Table D3). The paper now says so, gives guidance on choosing the dimension, and adds a loss over several dimensions as an option.

The individual replies answer each reviewer's points in order.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.