acceptodds
Under review as a conference paper at ICLR 2027

Structure-Preserving Frozen Neural Operators

Abstract

Coupled PDE solvers must enforce physical constraints while controlling optimization error and saddle-point conditioning. We introduce Structure-Preserving Frozen Neural Operators (SP-FNO), a frozen Fourier representation with coefficient-space constraints and gradient-free solves. A rank-revealing SVD yields an orthogonal projector for linear constraints. For mixed systems, we solve the velocity problem on the constraint kernel and recover pressure through a Schur complement governed by the retained inf-sup spectrum, independent of the constraint-block coupling scale. On boxes, analytic Gram assembly yields quadrature-free functional error majorants, valid with proven constants in exact arithmetic; our float64 evaluations do not enclose roundoff. The gradient-free core attains relative errors of , , and on manufactured Stokes, Maxwell, and Navier–Stokes problems without frequency anchoring. For advection whose solution lies in the frozen span, it reaches relative error in seconds. A separate learned field-space hybrid achieves the best mean VRMSE on all five The Well datasets tested under the protocol, reaching one-step VRMSE on active matter with a M-parameter backbone and an eight-pass symmetry ensemble. Code: https://anonymous.4open.science/r/SP-FNO-C44A

Primary area:
learning on time series and dynamical systems
Keywords:
Neural Operators, Physics-Informed Machine Learning, Constrained Optimization, Coupled PDEs, Error Estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_zh67OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Paper proposes SPFNO, which combines frozen Fourier features with constrained coefficient solves.

Critical strengths
  1. Enforcing physical constraints directly through the representation is useful and avoids need to tune penalty weights.

  2. Experiments cover several PDEs.

Critical weaknesses
  1. Novelty is not clear enough.

  2. Well dataset results use a separate trained model based on the Flowers architecture. It is unclear how these results validate the frozen solver or its theoretical guarantees.

  3. Proposed learned model uses eight backbone passes, whereas the baselines use one. It makes it difficult to tell how much improvement comes from the structure-preserving components and how much comes from additional compute.

  4. Pressure filtering improves conditioning but makes one velocity component substantially worse in Table 4C. I am not convinced about its practical benefit.

  5. Well dataset experiments use only one training seed, so the gains are unclear. Table 7 also shows that WaveLiT-9.5M-PT obtains lower median errors with fewer parameters and only one backbone pass.

Questions for authors
  1. Could you explain the main methodological novelty??

  2. Which parts of frozen solver are actually used in the learned hybrid? Can you separate the gains from physical projection, refinement, and symmetry ensembling while keeping the backbone and training setup fixed?

  3. How would baselines perform with the same eight-pass symmetry ensemble?? Consider reporting inference time along with accuracy under a similar experimental setup.

  4. Can the pressure-filtering threshold improve stability without substantially hurting velocity accuracy?

  5. Can you show tighter error bounds on more realistic problems? How would these bounds help decide when to refine the solution or stop?

  6. Could you report results with multiple training seeds?

  7. I found the paper difficult to follow in several places, and some important details needs more explanation. I would lean towards rejecting the paper in its current form.

2 comments
AuthorsOpenReview ↗

Novelty, gains at equal compute, filtering, bounds, seeds, and presentation

Thank you for the review.

Q1. Null-space solves, Schur complements, divergence-free spaces, and functional majorants are established, and we cite them. We make them work, with stated guarantees, for frozen random Fourier features, which satisfy no boundary or divergence condition and pair velocity with pressure at random. An enveloped stream function makes divergence and trace exact in the continuum while keeping integrals analytic. A whitened filter gives the retained pressure space βh=σr\beta_h = \sigma_r and κ(S)≤τ−2\kappa(S) \le \tau^{-2}, independent of the coupling scale. Analytic assembly reduces the majorant to one linear solve, and only the conforming representation satisfies its assumptions. In Table R5, with identical features and five seeds, the conforming representation reaches L2(u)=1.45×10−8L^2(u) = 1.45 \times 10^{-8} at F=256F = 256, against 4.82×10−84.82 \times 10^{-8} for SVD null-space elimination and 1.92×10−51.92 \times 10^{-5} for the best penalty weight.

Q2. The learned model shares only the projection onto a constraint kernel, applied to the predicted field, so The Well results say nothing about Propositions 1 to 4; the revision gives it its own name and section. Table R1 adds one component at a time to a fixed backbone and recipe, with three seeds. The plain backbone gives one-step VRMSE 0.03182±0.000640.03182 \pm 0.00064. Training with projection gives 0.02974±0.000380.02974 \pm 0.00038 and lowers divergence RMS from 0.28410.2841 to 1.5×10−71.5 \times 10^{-7} for 0.260.26 ms per sample. Refinement gives 0.027450.02745 and eight-pass D4D_4 averaging 0.01679±0.000210.01679 \pm 0.00021 (0.018460.01846 without projection). Averaging is the largest gain and the most expensive, 31.8531.85 ms per sample against 4.384.38 ms.

Q3. Agreed, since any backbone can be averaged over D4D_4. Table R2 gives FNO, CNextU-Net, scOT, Flower, and WaveLiT-9.5M-PT the same eight-pass average plus Πphys\Pi_{\text{phys}} and times every row. Their one-step errors fall by 12%12\% to 30%30\% at about 7.87.8 times the latency. At equal cost we still lead Flower (0.016680.01668 against 0.01740.0174), but at one pass Flower is better (0.02490.0249 against 0.029600.02960). WaveLiT-9.5M-PT beats our ensemble already at one pass (0.01560.0156, 3.153.15 ms). Table 2 will separate single-pass and ensemble rows, and we withdraw the claim of best mean VRMSE on all five datasets.

Q4. The velocity loss in Table 4C comes from the test basis. With 88 or 1616 features and no boundary conformity, the velocity errors are 11.5011.50 and 14.4514.45 before filtering (Table 9), and releasing one divergence condition can move such a velocity far. The threshold is not delicate: every τ\tau from 10−410^{-4} to 10−210^{-2} gives the same system (Table 18A), and a cut at the largest spectral gap needs no τ\tau. Table R4 uses a boundary-conforming discretization with five seeds. At F=256F = 256, filtering at τ=10−4\tau = 10^{-4} or at the gap lowers the pressure error from 1.651.65 (940940 with the weak mode) to about 5.5×10−65.5 \times 10^{-6} and the velocity error from 3.8×10−63.8 \times 10^{-6} to 2.1×10−62.1 \times 10^{-6}. Only over-filtering at τ=10−2\tau = 10^{-2} hurts the velocity.

Q5. The Poisson effectivity of 4.158×1034.158 \times 10^3 comes from a deliberately poor split parameter; chosen analytically it gives 4.8894.889. Table R6 adds multiscale and localized Poisson, the Stokes functional majorant for a non-polynomial solution, and Maxwell at κ=10\kappa = 10. All 4040 checks satisfy M≥EM \ge E, with effectivity 1.671.67 to 3.483.48. These are still manufactured problems on boxes, and general domains need new constants and quadrature. The bound needs no reference solution, and M≤tolM \le \text{tol} implies E≤tolE \le \text{tol}, which a residual cannot give (Table 23B: residual 8.1×10−138.1 \times 10^{-13}, error 0.4470.447). In Table R7, doubling FF until M≤tolM \le \text{tol} meets the tolerance in 4040 of 4040 runs, while stopping on the residual halts one doubling earlier and meets it in 88 of 4040.

Q6. Table 2 uses one training seed, and the eight passes share weights. Tables R1 and R3 use three seeds. The one-step standard deviation is at most 0.000640.00064 on active matter and 0.00280.0028 elsewhere. At equal passes we are below Flower on all five datasets, by more than two standard deviations on four; on TRL 2D the gap is 1.61.6 standard deviations at one pass and 1.41.4 at eight. Your reading of Table 7 is correct: WaveLiT-9.5M-PT is better in every window with fewer parameters and one pass, and we claim no per-parameter or per-pass advantage.

Q7. We agree. Much of the difficulty comes from two methods sharing one name with interleaved results. The revision separates them, collects every guarantee and its assumptions in one table, states the question and setting at the start of each experiment, marks single-seed results, and adds a notation table. If particular passages were hard to read, please tell us which ones.

I thank the authors for their rebuttal. However, I found both the paper and the rebuttal difficult to follow in several places, and some important details still require clearer explanation. My concerns remain unresolved, so I will retain my original score and continue to lean toward rejection in its current form.

Note for the authors: Please make the rebuttal a little more detailed, update the manuscript accordingly, and address the reviewers’ concerns clearly.

Reviewer_qztNOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes SP-FNO, combining fixed Fourier features with coefficient-space constraint elimination, mixed-system solves, and analytically assembled error majorants. It also evaluates a separate learned hybrid that combines a trained backbone with physical projection and symmetry ensembling on The Well.

Critical strengths
  1. A useful integration of constrained numerical methods and fixed-feature PDE solvers. The paper connects constraint-preserving representations, null-space solves, pressure-space filtering, and functional error estimation. It also explicitly distinguishes discrete constraint satisfaction from continuum admissibility and acknowledges the limitations of floating-point error certificates.

  2. The experiments provide evidence of improved constraint satisfaction. The Stokes representation ablation demonstrates near-machine-precision divergence control and exact boundary enforcement, with improved solution accuracy when both are incorporated. The filtering experiments also expose the tradeoff between stability and approximation quality rather than presenting filtering as uniformly beneficial.

Critical weaknesses
  1. Limited evidence for the incremental contribution. The frozen core combines established fixed-feature and constrained numerical techniques. The structure ablations are useful, but demonstrate improved constraint satisfaction more clearly than consistent accuracy gains: Table 4A considers one seed and feature count, while Table 4C shows stability–accuracy tradeoffs. A matched comparison against residual-based constraint enforcement would better isolate the proposed formulation’s benefit. The separate learned-hybrid results do not directly validate the frozen core.

  2. Incomplete reproducibility of the learned benchmarks. Table 2 reports five The Well datasets, but I could locate SP-FNO-specific pipelines only for Active Matter. Its documented evaluation also requires an unavailable checkpoint and supporting JSON, without a complete recipe for reconstructing the checkpoint. Additionally, the wrapper scans symmetry groups on validation data but hard-codes D4 for testing. This may match the actual validation choice, but the released workflow does not verify that correspondence.

  3. The methodological positioning could be clearer. Although Section 2 cites relevant predecessors, the paper could more explicitly distinguish benefits inherited from ELM/random-feature coefficient fitting from those introduced by its structural treatment. It would also help to define which structures are preserved—sampled constraints, continuum divergence and boundary conditions, or field-space constraints—and clarify the distinction between the per-instance frozen solver and the separately trained operator.

Questions for authors
  1. What is the principal advance over existing fixed-feature/ELM solvers with classical constraint elimination? Could a matched, multi-seed comparison isolate its accuracy and computational benefits?

  2. Could you release the SP-FNO-specific configurations and evaluation scripts for all five The Well datasets, along with the headline Active Matter checkpoint or a complete training recipe and the corresponding result files?

2 comments
AuthorsOpenReview ↗

Advance over ELM with classical elimination, release of The Well pipelines, and positioning

Thank you for checking the code as well as the paper. The problems you found in the release are real.

Q1. We do not claim a new elimination algorithm. For the same sampled constraints, an ELM solver with a KKT system or classical null-space elimination produces the same coefficients as our collocation projection, assuming exact arithmetic and no truncation, and the paper should say so. Our contribution is the set of conditions under which the classical guarantees still hold for frozen Fourier features, which are global, non-conforming, and randomly paired. First, equation 4 bounds the cost of truncation at the sampled points. The enveloped stream function makes divergence and trace exact in the continuum and keeps every integral closed-form, and only it satisfies (D0), so only it admits the majorant. Second, the whitened filter gives the retained pressure space βh=σr\beta_h = \sigma_r and κ(S)≤τ−2\kappa(S) \le \tau^{-2}, independent of γ\gamma; finite elements obtain this from a designed stable pair. Third, analytic assembly on boxes turns the majorant into a quadratic in the dual coefficients that one solve minimizes.

You are right that Table 4A shows constraint satisfaction more clearly than accuracy. It uses one seed, and the factor of 2.92.9 between its collocation and conforming rows lies within the seed spread that Table 14A reports at F=128F = 128. Table R5 is the matched comparison you suggest. It fixes the features, points, Stokes problem (ν=0.01\nu = 0.01), and solver, and varies only the constraint treatment: penalty least squares with λ\lambda from 11 to 10001000 (how fixed-feature solvers handle boundary rows), KKT, our SVD null space, and the stream function with a penalized or enveloped boundary, over five seeds. At F=256F = 256, the best penalty weight (λ=1000\lambda = 1000) reaches only L2(u)=1.92×10−5L^2(u) = 1.92 \times 10^{-5}, with condition number 7.15×10117.15 \times 10^{11}. KKT and SVD null-space solves of the same sampled constraints reach 9.45×10−89.45 \times 10^{-8} and 4.82×10−84.82 \times 10^{-8}. They coincide in exact arithmetic, and the gap reflects conditioning (2.18×10112.18 \times 10^{11} against 1.24×1071.24 \times 10^{7}). The enveloped stream function reaches 1.45×10−81.45 \times 10^{-8} in uu and 1.82×10−81.82 \times 10^{-8} in pp, with divergence RMS 3.92×10−163.92 \times 10^{-16}, zero boundary residual, the lowest conditioning (1.68×1061.68 \times 10^{6}) and online time (0.0360.036 s), and majorant effectivity 2.152.15.

Q2. Yes. The release has a pipeline only for active matter, its evaluation depends on a checkpoint and JSON files that are missing, and the wrapper hard-codes D4D_4. We will post here when the fixed release is live. For each of the five datasets we are adding the configuration, command, seed, and an evaluation script for all three windows. We are also adding the per-sample JSON files behind our entries in Table 2 and Table 4D, and a script that recomputes every mean and median from them. The Table 2 baselines are marked as copied from Muser et al. (2026). For active matter we will document the full recipe (split, normalization statistics, optimizer, schedule, effective batch size, selection rule, refiner stage), so the checkpoint can be rebuilt without any of our files. We cannot share model weights during anonymous review; all checkpoints will be released with the de-anonymized code. For the reported active-matter run, the validation scan over {Id,C4,D4}\{\text{Id}, C_4, D_4\} selected D4D_4, and the test wrapper was then fixed to that choice. The wrapper will now read the group from the scan's output file and stop if the file is missing, and Table R8 lists the validation error of each group on each dataset; D4D_4 is lowest on all five.

W3. The revision will name three levels of structure and keep them apart. Sampled constraints hold only at collocation points; the first row of Table 4A has divergence RMS 1.371×10−41.371 \times 10^{-4} over the domain. Continuum constraints come from the enveloped stream function (divergence 2.033×10−162.033 \times 10^{-16}, boundary residual 00), and only these meet the assumptions of Proposition 3. Grid constraints come from Πphys\Pi_{\text{phys}} in the learned model: spectral divergence on the periodic grid and the symmetry and trace conditions on DD and EE. Propositions 2 to 4 say nothing about the grid constraints. Section 3 will gain a table of these levels with their guarantees and assumptions. From ELM-style fitting we inherit gradient-free solves, analytic derivatives, and very small errors when the solution lies in the span. The structural treatment adds penalty-free constraints, admissibility for the bound, γ\gamma-independent conditioning, and retained-space inf-sup stability. The frozen solver trains nothing: each new forcing gets its own coefficient solve, which reuses the cached factorization. The learned model is a separately trained Flowers backbone that borrows only the projection idea and none of the bounds; it will get its own name and section.

Thanks for the response. The added results did not improve the limited scope of the work and I'm retaining my score.

Reviewer_y8o3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SP-FNO, which is a PDE framework using fixed Fourier features with explicit coefficient-space constraints. It combines constraint elimination with pressure-mode filtering. Under the stated assumptions, it also derives error bounds that can be assembled analytically. Experiments evaluate accuracy, constraint preservation, and stability on advection and manufactured PDE problems. The paper also evaluates a separate trained predictor on datasets from The Well. This predictor uses physical projection, optionally combined with refinement or symmetry ensembling, but does not inherit the frozen solver’s error guarantees.

Critical strengths

The framework brings together constraint enforcement and pressure stabilization in a clear way. Pressure filtering uses normalized coordinates, while Table 4A shows why enforcing constraints at sampled points differs from choosing representations that are divergence-free throughout the domain and satisfy the boundary conditions exactly.

The experiments offer useful insight into the method’s limitations. Better conditioning does not always lead to more accurate solutions, and valid error bounds can still be loose. The authors also clearly separate the frozen solver’s theoretical guarantees from the trained predictor’s experimental results.

Critical weaknesses

The role of physical projection in the strongest prediction results needs further clarification. The current comparisons make it difficult to separate its contribution from changes in the model, training setup, and computational budget.

For the frozen solver, the practical benefits and added value of the combined framework need clearer support. The current results leave open how much pressure filtering helps at useful solution accuracies and what the integration offers beyond the established strengths of its individual components.

Questions for authors

Q1

How much of the gain in the final Flowers configuration can be attributed to physical projection and symmetry ensembling, rather than differences in the backbone or training setup? What evidence shows that these gains are consistent across training seeds and remain meaningful when inference costs are taken into account?

Q2

Table 9 shows a substantial improvement in conditioning after pressure filtering, but the solution errors remain large. What do these results tell us about the filter’s practical value? Is there evidence that its stability benefits extend to boundary-conforming Stokes solutions with reasonably good accuracy?

Q3

The paper brings several established techniques together in a unified framework. Could you clarify what additional capabilities or insights emerge from their combination in SP-FNO? Which results best demonstrate the value of this integration beyond the benefits already expected from the individual techniques?

1 comment
AuthorsOpenReview ↗

Where the active-matter gain comes from, filtering at useful accuracy, and what the combination adds

Thank you for the review. Your summary keeps the frozen solver and the trained predictor apart more cleanly than our paper does, and the revision will follow it.

Q1. Until now our only evidence was Table 4D, which is single-seed: one pass (0.029600.02960) and refinement (0.027330.02733) are both worse than Flower's 0.02490.0249, and only the eight-pass D4D_4 ensemble (0.016680.01668) beats it. Table R1 adds one component at a time to a fixed 17.34M backbone and recipe, over three seeds. The plain backbone gives 0.03182±0.000640.03182 \pm 0.00064. Training and evaluating with Πphys\Pi_{\text{phys}} gives 0.02974±0.000380.02974 \pm 0.00038 and lowers divergence RMS from 0.28410.2841 to 1.5×10−71.5 \times 10^{-7}, refinement gives 0.027450.02745, and D4D_4 averaging 0.01679±0.000210.01679 \pm 0.00021, or 0.018460.01846 without Πphys\Pi_{\text{phys}}. Averaging is the largest factor, at 31.85 ms31.85\text{ ms} per sample against 4.38 ms4.38\text{ ms} for one projected pass.

Table R2 gives every baseline the same eight passes, with ρ(g)=N∘gphys∘N−1\rho(g) = N \circ g_{\text{phys}} \circ N^{-1}, plus Πphys\Pi_{\text{phys}}. Their one-step errors fall by 12%12\% to 30%30\% at about 7.87.8 times the cost. At equal passes we still lead Flower trained with the reference recipe in all three windows (0.016680.01668 against 0.01740.0174 one-step, 0.751430.75143 against 0.78950.7895 and 1.397391.39739 against 1.41801.4180 in rollouts), while at one pass Flower is better at one step (0.02490.0249 against 0.029600.02960). WaveLiT-9.5M-PT beats us in every window, already at one pass (0.01560.0156, 3.15 ms3.15\text{ ms}). In Table R3, with three seeds on all five datasets, we are below Flower at equal passes everywhere, and the one-step gap exceeds two seed standard deviations except on TRL 2D. Its active-matter Flower uses our ablation recipe and lands above the published value (0.031820.03182 against 0.02490.0249), while matching it within 0.3%0.3\% on the other four datasets. Our recipe omits Muser et al.'s rotational augmentation.

We withdraw the claim of best mean VRMSE on all five datasets. Table 2 will separate single-pass and ensemble rows with pass counts and timings and include WaveLiT, and the text will attribute the one-step gain mainly to group averaging.

Q2. Table 9 is a stability test only, and we should have said so. It uses 88 or 1616 features on a basis that violates the boundary condition, and we added a pressure mode at ω≈(500,550)\omega \approx (500, 550) that barely couples to the divergence. At F=8F = 8 the velocity errors are 11.5011.50 and 14.4514.45 and ∥∇⋅uh∥\Vert\nabla \cdot u_h\Vert is 79.5779.57 before filtering, so neither velocity approximates the solution. A nearly uncoupled pressure direction gives κ(S)=8.553×109\kappa(S) = 8.553 \times 10^9 and pressure error 2.698×1042.698 \times 10^4. Removing it gives βh=1.688\beta_h = 1.688, κ(S)=51.34\kappa(S) = 51.34, and pressure error 1.3371.337. The cut is insensitive to τ\tau: Table 18A keeps the same six directions for every τ\tau from 10−410^{-4} to 10−210^{-2}. To test accuracy, Table R4 applies the filter to a boundary-conforming mixed discretization at F=64F = 64 and 256256, with five seeds, with and without the weak mode, for several τ\tau and for a cut at the largest spectral gap. At F=256F = 256, τ=10−4\tau = 10^{-4} and the gap cut lower L2(p)L^2(p) from 1.651.65 (940940 with the weak mode) to about 5.5×10−65.5 \times 10^{-6}, and L2(u)L^2(u) falls with it, from 3.8×10−63.8 \times 10^{-6} to 2.1×10−62.1 \times 10^{-6}. Only τ=10−2\tau = 10^{-2} makes uu worse.

Q3. Each component exists, and we cite it. What the combination adds is that, on a box, one offline factorization with closed-form Gram matrices gives every new right-hand side a solution that satisfies the constraints in the continuum, together with a bound on its energy error. Each costs one linear solve, with no penalty weight and no quadrature. Frozen-PINN, FastLSQ, and the random feature method impose constraints as weighted residual rows, so their solutions violate assumption (D0) of Proposition 3 and the majorant does not apply to them. Mixed elements get inf-sup stability from a designed pair of spaces, and with random features the whitened filter takes over that role. The enveloped stream function makes the velocity divergence-free and zero on the boundary, which the bound needs, and it keeps every integral analytic, so the same matrices serve the solve and the bound. Table 4A shows the progression on one seed: divergence RMS falls from 1.371×10−41.371 \times 10^{-4} to 7.597×10−157.597 \times 10^{-15}, the boundary residual vanishes, and the velocity error drops to 5.573×10−65.573 \times 10^{-6}. With identical features and five seeds (Table R5), at F=256F = 256 the conforming representation reaches L2(u)=1.45×10−8L^2(u) = 1.45 \times 10^{-8}, against 4.82×10−84.82 \times 10^{-8} for null-space elimination and 1.92×10−51.92 \times 10^{-5} for the best penalty weight. Across the harder problems of Table R6 the majorant's effectivity stays between 1.671.67 and 3.483.48. Used to decide when to enlarge the basis (Table R7), it meets the tolerance in 4040 of 4040 runs, while stopping on the residual meets it in 88 of 4040.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.