acceptodds
Under review as a conference paper at ICLR 2027

Interpolating Neural Operator (INO): A Data-Free and Efficient Approach for Learning PDE Solution Operators

Abstract

Neural operators have become a popular approach to approximate the solution operators of parametric partial differential equations (PDEs). However, existing neural operators either require a large amount of simulation data or a long physics-informed training on GPUs, and they cannot tell how accurate an individual prediction is. In this paper, we propose the Interpolating Neural Operator (INO), a data-free interpolating neural network that is trained directly on the weak form of the PDE. In INO, the Karhunen-Loève coordinates of the input field are treated as additional inputs together with the spatial coordinates, and each input is approximated by a C-HiDeNN sub-network whose trainable parameters are nodal values. Since the network is multilinear in its parameters, training reduces to a sequence of one-dimensional linear solves by greedy alternating least squares. As a result, INO trains on one CPU core and predicts a new solution in microseconds. For coercive problems, the total error of every prediction is bounded by a computable residual bound that requires no reference solution, and the same bound applies to the predictions of other methods that satisfy the boundary conditions exactly. Before each prediction, INO checks whether the leading coordinates of the input lie within the range on which it is trained, and inputs outside this range can be passed to a conventional solver or to an INO trained on a wider range. INO is compared with physics-informed FNO and DeepONet on different benchmarks. INO is the most accurate model on most of these problems, by 15 on two-dimensional Helmholtz at and 53 on the diffusion-reaction benchmark, and on the one- and two-dimensional problems its training on one CPU core takes 3-80 less time than the physics-informed baselines on one GPU.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Scientific Machine Learning, Neural operators, Tensor decomposition, Parametric PDEs, Operator learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_aeJoOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents INO, a data-free PDE operator trained through alternating one-dimensional linear solves, with residual error bounds for coercive problems. Experiments on 1D–3D PDEs demonstrate efficient training and accurate predictions compared with physics-informed neural operators.

Critical strengths

I believe that constructing neural operators in a reduced solution space is a valuable direction, and this paper provides a practical realization of that idea. The separated representation expresses the solution through compact spatial factors whose coefficients depend on the input parameters. This structure makes the parametric PDE more tractable: weak-form training proceeds through alternating one-dimensional solves, and a new prediction is evaluated by contracting the learned factors. I particularly appreciate that the representation directly informs how the problem is solved. Reducing the complexity of operator learning in this way has substantial practical value for applications that require many solutions within a PDE family.

The data-free construction further strengthens this contribution. The spatial and parametric factors are learned from the governing equations without first generating reference solution trajectories. This avoids an expensive preliminary step that can limit the usefulness of a learned surrogate. The paper also examines solving a small projected system for the modal coefficients while retaining the learned spatial space. The resulting accuracy improvements help demonstrate that the reduced representation captures useful solution structure and can support different choices in the trade-off between evaluation cost and accuracy.

Overall, I find the practical value more consequential than the degree of methodological novelty. A useful reduced-space operator that is inexpensive to construct and evaluate can be valuable to the community even when it builds on established numerical ideas. I believe this paper demonstrates enough of that value to support acceptance.

Critical weaknesses

The methodological novelty appears incremental when viewed alongside established reduced-order approaches such as POD. Representing solutions in a compact space and determining input-dependent coefficients is a familiar strategy. The separated factors and alternating training procedure are more directly related to PGD and low-rank tensor methods. I therefore would not regard the reduced-space principle itself as a fundamentally new numerical idea. The contribution lies in making this structure work effectively as a data-free solution operator, with a practical training and evaluation procedure. For further improvement, I would suggest making this practical positioning particularly clear. Explaining how the construction of the reduced space and the evaluation of its coefficients relate to traditional reduced-order methods would help readers understand the contribution.

Questions for authors

None that would change my current acceptance recommendation.

1 comment

Thank you for the positive assessment and helpful suggestion. We agree that we should explain the practical value and the relation to PGD, POD and reduced-basis methods more clearly.

Following PGD/INN-TD, INO builds a separated space from the weak form without any training data/solution snapshots. We can evaluate the coefficients directly from the trained factors, or solve a small Galerkin system on the spatial modes (the reduced solve). We use the standard residual bound for certification. Since the residual is computed with the original input, the bound also includes the input-approximation error.

We added a comparison with POD using the same discretization, reduced solve and certificate. For the largest problems in the paper (3D at 129³, S=164, about 2 million unknowns), INO reaches similar accuracy at a lower cost, without solved inputs. The table gives within-domain error and total offline time (estimated for POD), with every step on one CPU core:

Helmholtzdiffusion
INO, no solved inputs5.8e-5 / 12 min1.4e-3 / 35 min
POD, 40 solved inputs6.0e-4 / 15 min1.45e-2 / 1.0 h
POD, 160 solved inputs6.4e-5 / 43 min1.6e-3 / 4.5 h

For these INO results, we removed a step that scaled with S² from the 3D code. Training is faster with unchanged accuracy, and we will update Tables 1 and 4. POD time includes the full solves, POD and projection of the operator terms onto its modes. We estimate it from a few timed solves and a sample of timed terms. POD becomes more accurate with more data (1.7e-5 and 3.6e-4 with 320 solved inputs). In 2D, where a full solve takes milliseconds, POD matches INO with about 80 solved inputs at a lower cost.

The reduced solve also improves INO without retraining. At 65², using 100 KL modes for the input field, the mean relative L2 error on all inputs drops from 1.07e-2 to 1.7e-4. At 129³, it drops from 1.42e-2 to 1.6e-3 for diffusion and from 1.76e-3 to 5.8e-5 for Helmholtz. Checking the bound with the original field certifies 99% of the 2D inputs at a 3% relative energy-error tolerance.

This comparison helps explain the practical value: in 3D, INO builds a space of similar accuracy to POD at a lower cost. We can then use the same space for fast direct predictions or more accurate reduced solves. We will add the comparison and clarify this point in the revision.

Reviewer_jTg5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper constructs a reusable map from input fields to PDE solutions without first generating solved training examples. It represents each input field using Karhunen–Loève (KL) coefficients and writes the solution as a sum of products of one-dimensional functions. The model is trained directly from the weak form of the PDE by alternating small linear solves, with an additional linearization for the nonlinear problems. The paper also provides an error bound for individual predictions and evaluates the method on several linear and nonlinear PDEs.

Critical strengths
  1. The paper gives an interesting way to compute a reusable solution function directly from the equations. This avoids paying for a separate numerical solution for every training example, while the product representation keeps the stored model small. The CPU training results and the accuracy on Helmholtz and nonlinear diffusion make this a useful approach to study, even though the comparisons have limitations discussed below.

  2. The error checks help a user judge an individual prediction without knowing its true solution. It checks the prediction against the discrete equation using the original input field, so the error bound also accounts for information lost when that field was compressed. Figure 4 checks the bound against actual errors and shows that the same check can also be applied to FNO predictions. The authors also disclose which inputs their range check keeps, report results on all inputs, and account for the cost of checking the bound.

Critical weaknesses
  1. My main concern is how the method differs from INN-TD. From my reading, Appendix E.2.7 of that paper already represents a conductivity field through covariance eigen coordinates and predicts its solution using products of C-HiDeNN functions within a data-free solving framework. This seems close to the construction here. I understand that this paper adds the domain check, the error bound evaluated with the original field, and a broader evaluation. These are useful, but I am still unclear whether the operator construction or solving procedure also changes. A direct comparison of the formulations would help me assess the contribution. If there is a change to the method, a comparison on a shared problem would help show its benefit.

  2. The baseline results make the size of the reported improvements difficult to judge. The authors explain that FNO training stalls on finer grids, while training on the coarse grid and transferring to finer grids gives better results and outperforms INO on 2D diffusion. These results are already reported in the appendix, but I think they should be included in the main accuracy and training-time comparison. DeepONet also performs worse than the mean-input solution on several problems. The authors attribute this to boundary enforcement and residual sampling, but it remains unclear how much these issues account for its poor performance. Including transferred FNO and testing a targeted adjustment to the failed DeepONet cases would help establish which gains remain with more effective baseline settings.

Questions for authors
  1. Compared with the operator example in INN-TD, Appendix E.2.7, what changes in the formulation or solving procedure? If these are unchanged, which additional result do the authors consider the main advance?

  2. How would the main accuracy and training-time comparison change if it included coarse-trained FNO transferred to finer grids, evaluated on the same inputs? For DeepONet, could the authors test whether addressing the reported boundary or sampling issue improves a representative failed case?

  3. For the diffusion settings in Table 2, what lower bound or enforcement procedure ensures that the coefficient stays positive for every permitted combination of training parameters?

2 comments

Thank you for the helpful suggestions. We address your 1st question as follows.

Q1 and Weakness 1. Relation to INN-TD

The formulation and the inference change, and the training update differs from the one used for E.2.7. The table compares the two formulations:

INN-TD, App. E.2.7INO
operatorconductivity k(x)k(x) to u(x,t)u(x,t) only in 1Dcoefficient or source fields to uu in 1D–3D (diffusion, Helmholtz, nonlinear diffusion, moving source)
input expansioneigenvectors of the nodal covariance, bounded domainproduct-covariance KL, each mode a product of 1D functions, d(S+1)d(S+1) terms (App. B); Hermite factors without a bounded domain for source inputs
nonlinear operatorsnot treatedfrozen coefficient (a polynomial in ∇u\nabla u) kept separated and compressed to a fixed rank; term count independent of SS (App. C.2)
trainingboosting (App. D); all modes updated jointly in the E.2.7 codeboosting as in INN-TD, then an amplitude update of all modes (Sec. 3.2); block updates for the moving source (App. C.2)
inferenceproduct formproduct form, or reduced solve (App. D.6)
error check per prediction–residual bound with the true input (Eq. (3))

We also tested both methods on a shared problem, transient diffusion on the INN-TD space-time mesh (71×15171\times 151 nodes). Both methods use the same 71 KL coordinates (the number in E.2.7, with unscaled eigenvectors as in the INN-TD code) and the same 200 test fields. We ran the INN-TD solver with its original random initialization and a QZ-based Sylvester solve. The table gives relative L2 errors (geometric means over three initializations). The reduced solve keeps the trained space-time modes and solves for their coefficients using the input field's r×rr\times r Galerkin system.

methodproduct formreduced solve
INN-TD, rank 708.6e-33.3e-3
INO, M=70M=701.55e-24.8e-3
INO, M=140M=1401.05e-21.5e-3

At the same rank, INN-TD is more accurate. This is because joint updates scheme satisfies Galerkin orthogonality and give a better space here. We use progressive updates because each step solves only one-dimensional systems, whereas joint updates couple all modes (size M⋅nM\cdot n per direction). The reduced solve improves both methods, and the bound applies to both with no violations. The main advance is therefore the construction of 2D/3D and nonlinear operators, together with the inference and certification with the true input. We will add this comparison to Section 2 and correct its sentence that INO makes predictions for arbitrary input fields.

Q2 and Weakness 2. Stronger baselines

We ran both suggested tests. For PI-DeepONet, exact boundary conditions multiply the output by sin⁡(πx)sin⁡(πy)\sin(\pi x)\sin(\pi y) (one sine factor per direction in 3D, as in PI-FNO), and by sin⁡(πx)⋅t\sin(\pi x)\cdot t for the moving source, which also enforces its initial condition. The training budget is unchanged. For PI-FNO, we add the network trained at 65265^2 and applied on the finer 2D meshes. The table is Table 1 with these baselines added: relative L2 errors within the parametric domain (moving source and diffusion–reaction: all samples), geometric means over three initializations. The 3D PI-FNO values are those of Table 1, and the 3D INO values use the faster code (general response, item 8).

problemPI-DeepONet, published → exact BCPI-FNO, Table 1 (in parentheses: trained at 65265^2)INO, product form
1D moving source3.89e-1 → 1.12e-13.98e-12.92e-3
1D diffusion–reaction5.21e-32.56e-34.84e-5
2D diffusion 65265^25.42e-2 → 1.58e-25.53e-34.34e-3
2D diffusion 1292129^25.85e-2 → 1.80e-24.81e-2 (2.58e-3)4.23e-3
2D diffusion 5132513^25.25e-2 → 2.14e-21.63e-1 (2.93e-3)4.58e-3
2D Helmholtz 65265^21.04e-1 → 6.50e-36.17e-34.14e-4
2D Helmholtz 1292129^21.17e-1 → 6.79e-36.70e-2 (4.06e-3)4.17e-4
2D Helmholtz 5132513^29.93e-2 → 8.45e-37.99e-2 (2.47e-3)3.74e-4
3D diffusion 65365^32.08e-1 → 3.33e-22.08e-37.78e-3
3D diffusion 1293129^32.00e-1 → 3.92e-26.57e-39.24e-3
3D Helmholtz 65365^34.07e-1 → 1.38e-21.20e-35.93e-4
3D Helmholtz 1293129^34.16e-1 → 1.22e-29.35e-46.51e-4
2D nonlinear diffusion 65265^22.21e-19.11e-32.79e-3
2D nonlinear diffusion 1292129^22.21e-15.28e-2 (6.83e-3)2.58e-3

We applied exact boundary conditions where PI-DeepONet was less accurate than the mean-input solution, and to 2D diffusion. They improve it by 2.5–34×, so the soft boundary penalty explains most of its error. Ten times more residual points help little (8.89e-2 against 1.04e-1 on Helmholtz 65265^2), and drawing half of the residual points at the moving spot does not help (1.86e-1 with exact conditions). We will remove the residual-sampling explanation of App. C.3. INO remains more accurate than the improved PI-DeepONet on every problem: 3.6–23× within the domain, 1.8–7.9× on all samples, and 38× for the moving source.

PI-FNO trained at 65265^2 and applied on the finer meshes is more accurate than the product form of INO for 2D diffusion, while INO stays more accurate for Helmholtz and nonlinear diffusion. Trained on the finer meshes with the published settings, PI-FNO does not converge. INO trained at 65265^2 can also be applied on the finer meshes (4.36e-3 at 1292129^2, 4.38e-3 at 5132513^2), so one 42 s training serves all three meshes. For training on each mesh, INO takes 79 s and 530 s (diffusion 1292129^2, 5132513^2), 35 s and 373 s (Helmholtz) and 348 s (nonlinear diffusion 1292129^2) on one CPU core, against 1,718 s, 1,768 s and 813 s for PI-FNO trained at 65265^2 on one GPU.

Without retraining, the reduced solve of INO gives 8e-5 to 9e-5 for 2D diffusion at 65265^2–5132513^2. In 3D at 1293129^3, it gives 1.4e-3 (diffusion) and 5.8e-5 (Helmholtz), compared with 6.57e-3 and 9.35e-4 for PI-FNO, at an inference cost of 0.6–2 ms in 2D and about 1.4 s in 3D per input. We will report the reduced solve separately from the product form. With the faster code, the training time of INO for 3D diffusion at 1293129^3 drops from 5.7 h in Table 1 to 35 min on one CPU core. We will update Table 1 with the new baselines and times.

Q3. Positivity

We did not enforce positivity for the ±2\pm 2 diffusion domains. The minimum of kk over the parametric domain at each spatial point is kˉ−L2σ∑j∣φj(x)∣\bar{k} - \frac{L}{2}\sigma\sum_j \lvert\varphi_j(x)\rvert. The computed minimal values are −0.71 (2D, S=25S=25) and −1.54 / −2.76 (3D, S=56S=56 / 164).

We ran 45 trainings on positive and non-positive domains. The error grows with domain width, but we did not see a sudden change at the tested positivity thresholds. This does not prove stability or convergence for indefinite training.

For certification, we use the original input's coercivity constant, which is positive for every test input. We will correct the training-domain assumption and give positive settings, e.g. ±1.15\pm 1.15 at σ=0.2\sigma=0.2 for 2D, S=25S=25. With the reduced solve, an operator trained on ±1.15\pm 1.15 still evaluates all test inputs, with errors of 1.5e-4 on the Table 1 inputs and 2.8e-4 on all inputs (9.1e-5 and 1.7e-4 for ±2\pm 2). The tested Helmholtz domains are coercive.

Reviewer_5A2ZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

INO represents an input field using truncated Karhunen–Loève coordinates and approximates the solution jointly over spatial and parametric coordinates by a sum of separated products. Its one-dimensional C-HiDeNN factors are trained on the PDE weak form through greedy enrichment and alternating linear solves, avoiding simulation-generated training pairs.

For coercive discrete problems, the paper evaluates a standard residual error bound with the true input field, thereby including input-truncation effects in the error relative to the discrete reference solution. A leading-coordinate domain check controls which inputs are predicted. Experiments compare INO with physics-informed FNO and DeepONet on several linear and nonlinear problems, with additional studies of rank, error bounds, full-population performance, and reduced coefficient solves.

Critical strengths
  1. The motivation identifies an opportunity to exploit mathematical structure instead of treating all operator learning as generic nonlinear optimization. Combining spatial and KL coordinates in a separated representation makes alternating linear solves possible. This is a sensible and practically valuable approach for the class of problems whose weak forms permit efficient separation.

  2. The paper treats prediction reliability as part of the task. Evaluating the residual with the true input field, rather than only its truncated KL representation, prevents a small error on the surrogate input from being confused with a small error on the actual input. The distinction is useful even though the underlying coercive residual estimator is standard.

  3. The modal-coefficient analysis provides a more informative diagnosis than error tables alone. The learned spatial span can approximate the solution much more accurately than the product-form parametric weights achieve, and a projected solve substantially closes that gap. This suggests that representing the solution space and predicting its coordinates are different bottlenecks, and helps explain where the current method loses accuracy.

Critical weaknesses
  1. The central motivation establishes a case against unnecessarily generic optimization, but not yet a distinct contribution over the closest structured methods. Section 2 acknowledges parameter-as-coordinate PGD, greedy low-rank enrichment, C-HiDeNN/INN-TD factors, and residual-based estimation. Recasting these ingredients as an operator with an input projection and domain check does not by itself establish a new learning principle. The missing contribution could be a new capability, approximation mechanism, or materially better integration, but the paper needs to identify it explicitly and substantiate it against those predecessors.

  2. The efficiency insight is conditional on favorable structure, and that condition should define the claim. Efficient separated assembly in Appendix B uses product covariance modes and separable PDE terms; smooth inputs and admissible coefficient ranges keep the representation tractable. Linear parameter-count growth at a fixed rank does not establish that the rank or assembly cost remains favorable as input complexity grows. The informative question is therefore when this solution family has a compact separated representation and how INO improves access to it, rather than whether a generic neural baseline is slower on such a family.

  3. The evidence does not yet establish the proposed combination as a better reliable-solver workflow. The domain check excludes a substantial fraction of inputs, the error bound certifies a discrete coercive problem rather than the continuous PDE, and full-field certification is much slower than the quoted pure-prediction time. These limitations are disclosed, but they affect the motivating promise of fast reliable predictions. Comparisons with structured reduced models, including the paper's own projected-coefficient variant, would clarify the value of the parametric predictor and certification policy beyond their separate established ingredients.

Questions for authors
  1. What is the principal methodological insight beyond PGD/INN-TD with input projection and a standard residual estimator? Please identify a capability or design consequence that the closest prior construction does not already provide, and distinguish the integration contribution from its known ingredients.

  2. Which properties of the covariance, coefficient field, and solution family keep the separated rank and assembly cost manageable? How should the efficiency claims be interpreted when those properties weaken, rather than only when the spatial grid is refined?

  3. What role should the learned parametric weights play given the strong projected-coefficient results? Can the existing evidence compare the intended reliable-prediction workflow, including projection, coverage, certification, and fallback, with an appropriate structured numerical alternative?

3 comments
AuthorsOpenReview ↗

Answers to questions and concerns

Thank you for the careful review. We address your three questions and major concerns as follows. New results are marked New. The supporting tables are attached next.

Q1 and Weakness 1. Principal insight

The key new capability is a data-free operator whose error is bounded for the true input field. The table separates the known ingredients from what INO adds.

ingredientknown fromwhat INO adds
separated space trained on the weak form, C-HiDeNN factorsPGD, INN-TD2D/3D random input fields through a product covariance (d(S+1)d(S+1) terms, SS up to 164); nonlinear operators with a term count independent of SS (App. C.2)
Galerkin solve on a reduced spacereduced-basis (RB) methodsthe space is built without solved inputs; the input enters through a 100-mode KL expansion of the true field instead of the training truncation
residual error boundRB error estimationevaluated with the true input and a closed-form coercivity constant, so it covers input truncation and inputs outside the parametric domain without EIM or SCM

Together, INO builds a reduced space without solved inputs, and its prediction for any coercive input, inside or outside the domain, is either certified with the true input or replaced by a full solve (Q3).

New (Table 1 diffusion, 65265^2): without retraining, the reduced solve lowers the mean relative L2 error from 4.3e-3 (product form) to 9.1e-5 within the domain, and from 1.07e-2 to 1.7e-4 on all inputs. In 3D, POD needs about 160 solved inputs and an estimated 3.6–7.7× INO's training time for similar accuracy (Part 2, B). On INN-TD's own operator example, its solver gives a better product form at equal rank, the reduced solve improves both networks, and the bound holds for both (Part 2, A). We will clarify this in Section 2 and credit prior work for the training algorithm, factors and bound.

Q2 and Weakness 2. When the rank and assembly cost stay manageable

We agree that linear growth in parameter count does not guarantee a small rank. We will state three conditions. (1) The input enters affinely, so the term count TT grows at most linearly with SS; a degree-PP expansion of k=exp⁡(g)k=\exp(g) would need d(P+SS)d\binom{P+S}{S} terms. (2) The covariance is a smooth product kernel, so each KL mode is a product of 1D functions. (3) The solution has a compact separated form. This depends on where the input enters (in 2D, Helmholtz needs fewer modes than diffusion, Table 3) and on the correlation length ℓ\ell.

New (Part 2, C): an isotropic Matérn-5/2 kernel needs 69 modes instead of 22 for the same field error and raises TT from 46 to about 970. Reducing ℓ\ell from 0.3 to 0.15 raises SS from 24 to 75, training from 190 s to 852 s, and the reduced-solve error at M=192M=192 from 2.5e-5 to 1.4e-4. Our efficiency claim does not cover discontinuous inputs or transport-dominated problems. For the class considered here, the bound rejects predictions when the rank is insufficient. Within this class, INO reaches the compact form without solved inputs; in 3D it matches a 160-input POD space at an estimated 3.6–7.7× lower cost (Part 2, B).

Q3 and Weakness 3. Parametric weights and the certified workflow

For linear operators, the parametric weights mainly serve to build the spatial modes without solution data, and the reduced solve is the more accurate predictor when about 1 ms per input is acceptable. The weights remain the predictor when 0.03 ms is needed (about 75% of these predictions are certified at 3%), and for nonlinear operators, where the reduced solve takes 0.5 s (Table 7). For the energy bound Δ\Delta (Eq. (3)), we accept a prediction only if ∥uM∥>Δ\lVert u_M\rVert > \Delta and Δ/(∥uM∥−Δ)≤3\Delta/(\lVert u_M\rVert - \Delta) \le 3%; otherwise, we solve the same discrete system. New, all 200 test inputs of 2D diffusion at 65265^2, one core:

policyaccepted without fallbackguaranteedestimated ms/input
domain check, product form71.0%no4.6
bound, reduced solve99.0%yes1.3

The last policy needs no domain check. Computing the bound now takes 0.52 ms instead of 0.2–0.5 s, with no violation in 1,000 bounds. The certified workflow is about 12× (65265^2) and 16× (1292129^2) cheaper than solving every input. In 3D at 1293129^3 (New), it certifies 100% and 97.5% of inputs (Helmholtz, diffusion), with no violation in 480 bounds, at an estimated 2.6 s and 5.3 s per input against 8.0 s and 43.5 s per solve. The bound is for the discrete solution. In 2D this solution changes by 4.6e-6 from 65265^2 to 5132513^2 (New), 20× below INO's 9.1e-5.

We hope these results address your concerns, and we would be grateful if you could consider raising your score.

AuthorsOpenReview ↗

Tables Supporting the New Results

A. INN-TD on a shared problem (Q1)

We trained both networks on the operator example of INN-TD (App. E.2.7: 1D transient diffusion, conductivity field as input, 71×15171\times 151 space-time nodes). Relative L2 errors on 200 fields, geometric means over three initializations:

networkproduct formreduced solve
INN-TD solver, rank 708.6e-33.3e-3
INO, rank 701.55e-24.8e-3
INO, rank 1401.05e-21.5e-3

At equal rank, the INN-TD solver, which updates all modes jointly in this example, is more accurate since this solution scheme satisfies the Galerkin orthogonality. The reduced solve improves both networks, and the bound holds for both, with no violation. The training algorithm follows PGD and INN-TD, and the advance is in the 2D/3D and nonlinear operator constructions, the inference with the true input and the certification. Our reply to Reviewer jTg5 compares the two formulations in detail.

B. POD reduced basis in 3D (Q1, Q3)

Both spaces use the same 1293129^3 discretization (S=164S=164), the same reduced solve with the exact 533-mode input, and the same bound. The table gives the error on the 28 of 40 test inputs within the domain and the total offline time on one CPU core. For POD, the time is estimated from timed solves and projections, and includes the solves, the POD and the projection of the operator terms. POD's solves can run in parallel, while INO's training mostly cannot.

spaceHelmholtzdiffusion
INO, no solved inputs (training data)5.8e-5 / 12 min1.4e-3 / 35 min
POD, 40 solved inputs6.0e-4 / 15 min1.45e-2 / 1.0 h
POD, 80 solved inputs1.2e-4 / 32 min5.3e-3 / 2.0 h
POD, 160 solved inputs6.4e-5 / 43 min1.6e-3 / 4.5 h

POD becomes more accurate with more solved inputs. INO reaches the accuracy of 160 solved inputs at 3.6× (Helmholtz) and 7.7× (diffusion) lower cost, without solving any input. The INO times use the faster 3D code (general response, item 8). In 2D, a full solve takes milliseconds, and POD matches INO with about 80 solved inputs at a lower cost.

C. Rank and cost for rougher inputs (Q2)

2D diffusion at 65265^2, one core, M=192M=192, errors within the domain, with the same eigenvalue tolerance for every correlation length ℓ\ell:

ℓ\ellSS / TTtrainingproduct formreduced solve
0.324 / 50190 s3.1e-32.5e-5
0.247 / 96431 s5.4e-35.5e-5
0.1575 / 152852 s7.4e-31.4e-4

In these runs, the training time grows about as S1.3S^{1.3}. For a 4% field error at ℓ=0.3\ell=0.3, our squared-exponential kernel needs 22 modes, Matérn-5/2 needs 69 and Matérn-3/2 needs 217. An isotropic Matérn mode is not a product of 1D functions, so it has to be separated, which raises TT (Part 1, Q2). We will add these results and the three conditions of Q2 to the paper.

We hope these results address your concerns, and we would be grateful if you could consider raising your score.

Thank you again for your thoughtful review. As the discussion period ends tomorrow, we wanted to check whether our responses have addressed your concerns. We would be glad to clarify anything further before the deadline.

Comments on the paper

1 comment

We thank the reviewers for their careful reading and helpful suggestions. The main concerns were the contribution relative to PGD, INN-TD and other reduced-order methods (all reviewers). The advance of INO is the construction of 2D/3D and nonlinear operators and the prediction and certification with the true input field.

We ran the following new experiments to address reviewers's concerns.

  1. Reduced solve with the true input (5A2Z, aeJo). We keep the trained spatial modes and solve the r×rr×r Galerkin system for the queried input, expanded with 100 KL modes instead of the training truncation. Without retraining, the error for 2D diffusion at 65² drops from 4.3e-3 with the trained parametric factors (product form) to 9.1e-5 within the parametric domain, and from 1.07e-2 to 1.7e-4 on all inputs. At 129³, it drops from 1.42e-2 to 1.6e-3 (diffusion) and from 1.76e-3 to 5.8e-5 (Helmholtz) on all inputs.

  2. Certified workflow (5A2Z). We accept a reduced-solve prediction when the bound, evaluated with the true input, gives a relative energy error of at most 3%, and otherwise solve the discrete system. In 2D at 65², 99.0% of the inputs are certified, at an estimated 1.3 ms per input against 15.9 ms for solving every input. In 3D at 129³, 100% (Helmholtz) and 97.5% (diffusion) are certified, at 2.6 s and 5.3 s per input against 8.0 s and 43.5 s per solve. No bound was violated.

  3. POD reduced-basis baseline (5A2Z, aeJo). With the same discretization, reduced solve and bound, POD needs about 160 solved inputs to match INO at 129³, and then costs an estimated 3.6× (Helmholtz) and 7.7× (diffusion) the training time of INO on one CPU core.

  4. INO and INN-TD on a shared problem (jTg5, 5A2Z). On the operator example of INN-TD method (App. E.2.7), the INN-TD solver gives a more accurate product form than INO at equal rank (8.6e-3 vs 1.55e-2) because it updates all the ranks simultaneously (Galerkin Orthogonality). The reduced solve improves both networks (3.3e-3 for INN-TD and 4.8e-3 for INO at rank 70), and the bound holds for both.

  5. Resolutional independence of PI-FNO (jTg5). PI-FNO trained at 65² and applied on the finer meshes is more accurate than the product form of INO for 2D diffusion, and we will add it to the main comparison. But this doesn't hold for all cases. Exact boundary conditions improve PI-DeepONet by 2.5–34×, and INO remains more accurate than the improved PI-DeepONet on every problem.

  6. Positivity (jTg5). The diffusion coefficient is not positive at the corners of the ±2 parametric domains. In 45 trainings across the positivity threshold, the error grows smoothly with the domain width. An operator trained on a positive domain (±1.15) evaluates all test inputs with the reduced solve (1.5e-4 on the Table 1 inputs, 2.8e-4 on all inputs).

  7. Random fields with smaller correlation length (5A2Z). Reducing the correlation length from 0.3 to 0.15 raises S from 24 to 75, the training time from 190 s to 852 s, and the reduced-solve error at M=192 from 2.5e-5 to 1.4e-4.

  8. Faster 3D training in new implementation. We removed a step that scaled with S² from the 3D code. The training time for 3D diffusion at 129³ drops from 5.7 h (Table 1) to 35 min on one CPU core, with similar accuracy.

In the revision, we will address the following: 1. state this contribution in Section 2 and credit prior work; 2. add transferred PI-FNO, PI-DeepONet with exact boundary conditions, the reduced solve and the new 3D training times to Table 1; 3. add the certified workflow and the POD and INN-TD comparisons; 4. correct the positivity assumption and give positive settings; 5. state the conditions under which the separated representation stays compact.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.