acceptodds
Under review as a conference paper at ICLR 2027

Neural Fixed-Point Solvers: Learning to Iteratively Correct for Parameter-Varying Transient PDE Solving

Abstract

Neural PDE solvers are typically direct predictors: they map problem data to a solution with a learned neural operator. As a result, they are agnostic to variations in PDE parameters and provide no mechanism for enforcing the governing physical constraints while solving. Classical iterative solvers address both issues by repeatedly evaluating the residual and using a preconditioner to map this feedback to a correction. Inspired by this residual-to-correction principle, we introduce the Fixed-point Neural Operator (FiNO), which uses a neural operator as a learned preconditioner. At each iteration, the PDE residual is evaluated on the current solution estimate and fed to the network, which predicts a correction that is applied repeatedly to refine the solution. This design incorporates equation feedback directly into inference and enables a shared solver to handle parameter-varying transient PDEs. Experiments across multiple problems show that the iterative solver substantially improves solution accuracy and physical consistency over direct neural operators, while providing stronger generalization across equation parameters.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operator, Fixed Point

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Tbg5OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes FiNO, an iterative neural PDE solver that maps the residual of the current solution to corrections. The authors state conditions under which this update is locally contractive and design a loss that encourages accurate corrections and contractive behavior, among other properties. Finally, the paper validates their method on three different PDEs.

Critical strengths
  1. Each term of the loss function served an important purpose, and the loss ablations justified each term.
  2. The method appears to have strong generalization across varying parameters and forecast horizons
  3. The presentation of the paper is very clear.
Critical weaknesses
  1. The experimental baselines do not test FiNO against other learned iterative solvers. Although the authors include IRNO, they do not empirically compare with residual-driven methods such as HINTS or the Deep Conjugate Direction Method (DCDM).
  2. While Theorem 1 provides a useful result, the authors do not establish whether the trained FiNO models satisfy its assumptions in practice. Figure 3 shows that the residual of the current iterate can increase with further iterations, making it especially important to report how often the learned update is contractive across test cases.
Questions for authors
  1. What is the intuition behind FiNO's robustness to Long horizons (Figure 5)?
3 comments

Q1 (comparison to HINTS and DCDM). HINTS and DCDM share the general residual-driven principle with FiNO. However, both are designed for solving discrete linear systems, particularly sparse linear equations, whereas FiNO targets nonlinear transient PDEs. Despite the shared design philosophy, the methods differ fundamentally in the classes of equations they address, their state-update mechanisms, and their underlying numerical solution procedures. Therefore, a direct experimental comparison would not be methodologically meaningful.

Applying HINTS or DCDM to the problems considered in this work would require substantial methodological adaptation rather than straightforward deployment, including the linearization of nonlinear PDEs, spatial and temporal discretization, and integration with an appropriate numerical solution framework. Such adaptations would introduce additional design choices and potentially alter the original methods, making a direct comparison difficult to interpret fairly. We have accordingly discussed in the Related Work section the design principles shared between FiNO and HINTS, while also clarifying their differences in target problem settings and methodological formulations.

Q2 (contraction assumption satisfaction). Please refer to Response to common questions Q1.

Q3 (intuition behind long horizon robustness). The robustness of FiNO over long prediction horizons can be attributed to two complementary factors. First, unlike autoregressive neural operators that directly learn state-to-state mappings, FiNO adopts an iterative, residual-driven correction mechanism. By explicitly incorporating PDE-based residuals into each update, the evolution is continuously guided by the governing equations rather than relying solely on the predicted state. This physics-informed closed-loop correction helps mitigate the accumulation of prediction errors over successive iterations.

Second, a sufficiently large overlap between consecutive temporal windows provides additional stability for long-horizon rollout. The overlap helps maintain continuity when transitioning from one window to the next. In our experiments, we found that overly aggressive window advancement could lead to substantial deviations in the first frame of the subsequent window, making the subsequent long-horizon rollout considerably more sensitive and fragile. Thus, an appropriate overlap is important for maintaining stable temporal propagation.

Thank you to the authors for the detailed rebuttal and additional experiments. I appreciate the effort to address my concerns.

I find the authors' explanation of why a direct comparison with HINTS/DCDM may not be appropriate convincing, given the differences in the problem settings, and consider this concern addressed. I also appreciate the explanation of FiNO's long-horizon robustness in terms of residual-driven corrections and overlapping temporal windows. Both explanations seem reasonable.

I appreciate the additional empirical analysis of contraction behavior. The results provide reassuring evidence that FiNO exhibits approximately contractive behavior in practice, which largely addresses my original concern. One aspect I would appreciate further clarification on is the observation that removing the contraction-related losses actually increases the percentage of contracting steps, despite worsening the final RMSE. The authors suggest that these losses help maintain stable contraction over longer iteration sequences rather than increasing the contraction percentage itself. Could the authors provide additional evidence supporting this explanation?

Apologies for the late response. Our experiments indicate that stable contraction typically occurs during the initial phase, a stage that can be captured through early stopping. And contraction and solution loss help maintain contraction over a longer sequence of time steps. To investigate this further, we extended the number of iterations for the ablation methods to match that of the full version; the results show a significant reduction in the contraction rate for the ablation methods. When the number of iterations for all methods is increased to 128, the contraction ratio decreases across the board.

Iteration steps aligned with the Full:

MethodContraction percentagenon-contracting steps meanRMSE
Full91.74%0.057%0.172
w/o solution contraction85.92%0.738%0.195
w/o residual contraction83.91%0.181%0.214
w/o both75.73%0.499%0.237

Iteration steps = 128:

MethodContraction percentagenon-contracting steps meanRMSE
Full31.62%0.602%0.175
w/o solution contraction31.09%1.089%0.213
w/o residual contraction30.52%0.728%0.189
w/o both23.31%0.804%0.338

It can be observed that at 128 iterations, the contraction ratio drops (91.74% to 31.62%). However, since the increase in the number of non-contraction steps is minimal (0.602%), the final RMSE shows only a marginal increase (0.172 to 0.175) compared to the early-stopping approach. This is consistent with the curve presented in Figure 3. When ablating solution or residual contraction loss, the convergence becomes unstable.

Reviewer_qK7QOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes FiNO, an iterative neural method for solving parameter-varying PDEs. Instead of predicting the full solution once, the method repeatedly checks how much the current prediction violates the PDE and uses this information to correct the solution. The paper tests this idea on Navier-Stokes, Burgers, and Reaction-Diffusion equations. The results show gains over neural operator methods and gives lower PDE residual errors. The paper also studies parameter generalization and long-term prediction. The idea is interesting, but the comparison in terms of computational cost and the evidence for actual convergence need to be stronger.

Critical strengths
  • Using the PDE residual during inference is a possible choice to improve physical consistency.
  • The improvements on Navier-Stokes and Burgers are large compared with the tested neural operator baselines.
  • The paper also studies parameter generalization and long-term rollout, which are important for this type of solver.
Critical weaknesses
  • The novelty is somewhat limited in my opinion. Iterative neural correction, learned preconditioning, and residual-based refinement already exist, although this paper combines them differently for transient nonlinear PDEs.
  • The experiments are only on three PDEs with periodic and regular grids. It is not clear how well this works for more difficult boundary conditions, irregular geometries, or different discretizations.
  • The main results do not report variation over different training seeds. This makes it difficult to judge the stability of the reported improvements.
  • The inference cost seems to be large because the model may require many correction iterations. The paper does not give a clear accuracy versus runtime comparison with the baselines or classical solvers.
  • The convergence result is conditional on strong assumptions. The paper does not show that the trained model actually satisfies these assumptions, so the theory does not guarantee convergence of the learned solver in practice.
Questions for authors
  • Can the authors report results over multiple random seeds for the main comparisons?
  • Can the authors provide a direct accuracy versus inference-time comparison with FNO, IRNO, and classical numerical solvers?
  • How does the method perform on PDEs with non-periodic boundary conditions, irregular geometries, or different spatial discretizations?
  • Can the authors empirically verify whether the learned update satisfies the contraction assumptions used in the theoretical analysis?
  • How sensitive is the method to the damping factor, stopping rule, and maximum number of iterations? Can these choices be fixed across different PDEs without tuning?
5 comments

Q1 (random seeds). We trained the model for 20k steps on the NS dataset using 5 different random seeds to evaluate the impact of random seeds on model accuracy. The experiments indicate that the variation in accuracy attributable to the random seeds is within 6%. Due to time constraints, among the comparative methods, we only evaluated the accuracy of IRNO across multiple random seeds.

Methodseed 1seed 2seed 3seed 4seed 5meanstd
FiNO0.17210.20040.18090.18410.18330.18420.01024
IRNO0.41880.42880.43640.44010.44520.43390.00924

Q2 (inference-time vs accuracy). Please refer to Response to common questions Q2.

Q3 (transfer to more geometries). FiNO is a residual-driven iterative solution framework. One implementation of this framework utilizes an FNO backbone designed for periodic boundary conditions and Cartesian grids. In fact, it can be naturally extended to accommodate more general boundary conditions and grid structures. As a case study involving unstructured triangular mesh and non-periodic boundary conditions, we employ a cylinder flow problem, using Transolver as the backbone. In this case, the mesh contains 7k+ triangle nodes, and Re varies from 80 to 120. The residual is evaluated with finite-element method. We train the Transolver model and our FiNO with Transolver backbone for fair comparison. The results are as follows:

MethodVelocity rel. L2Pressure rel. L2Momentum ResidualContinuity ResidualNormalized RMSE
Direct Transolver4.08%8.69%4.121.180.0396
Transolver-backbone FiNO3.53%6.78%2.820.650.0268

Q4 (contraction assumption satisfaction). Please refer to Response to common questions Q1.

Q5 (hyper-parameter sensitivity). We evaluate the sensitivity of damping factor, stopping patience, stopping relative improvement threshold and max iterations as following.

Damping factor. We evaluated three representative damping factors and observed that a smaller damping factor leads to a higher number of iteration steps, without a continuous improvement in accuracy. Considering results across multiple datasets, 0.75 is a reasonable choice.

Datasetdamping factorRMSEdifferenceiteration steps
NS0.50.166958-2.31%54.29
 0.75 (default)0.170912037.77
 10.179700+5.14%34.74
Burgers0.50.216203+13.04%23.85
 0.75 (default)0.191257014.98
 10.179544-6.12%14.99
Gray-Scott0.50.156183+2.25%29.15
 0.75 (default)0.152739020.47
 10.167753+9.83%17.61

Stopping patience.

Datasetpatience (default 8)RMSE differenceiteration steps
NS4 / 8 / 16+0.35% / - / -0.63%36.69 / 37.77 / 38.89
Burgers4 / 8 / 16-0.02% / - / -0.13%13.70 / 14.98 / 16.22
Gray-Scott4 / 8 / 16+0.00% / - / 0.00%20.40 / 20.47 / 20.47

Stopping relative improvement threshold.

Dataset1e-41e-3 (default)1e-2
 differnceiter stepsdiffernceiter stepsdiffernceiter steps
NS0.00%45.38-44.10-0.51%39.50
Burgers0.00%22.63-21.26-0.01%17.00
Gray-Scott0.00%28.22-27.26+0.02%24.56

Max iterations.

DatasetK=32K=64K=128 (default)
 differencereaching%differencereaching%differencereaching%
NS+22.96%67%+0.58%21%-0%
Burgers-0.05%12%0.00%1%-0%
Gray-Scott+0.44%32%0.00%0%-0%

Experiments show that accuracy is not sensitive to the hyperparameter settings of the stopping rules. The max iterations parameter can be set based on the number of steps actually required for convergence, ensuring that the residual reaches its minimum value. For the three datasets used in our study, these hyperparameters can be kept fixed.

Thanks to the authors for answering my concerns. The responses have mostly answered my doubts. However, I missed the discussion of the accuracy and compute cost of the classical numerical solver. The authors only report the runtime but do not report the accuracy. This makes it tough to understand the merits of the proposed method. A fair comparison with numerical methods is important to understand the potential of the proposed method. If the authors can answer this satisfactorily, I will reconsider increasing my score.

Thank you for pointing this out. The classical solver is used to generate the reference trajectories (GT), so its RMSE is zero by construction. FiNO and other learned methods are evaluated by their error relative to this numerical reference, while the classical solver’s runtime is reported separately. We have clarified this convention in Appendix D.

Reviewer_XyPqOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a Fixed-point Neural Operator (FiNO) for solving parameter-dependent nonlinear transient PDEs. At each iteration, the method recomputes the PDE residual along the current trajectory and feeds the current state and residual into an FNO-style neural preconditioner to predict the next correction: u^((k+1))=u^((k))+ αG_θ (u^((k)),r^((k))) At inference time, FiNO uses the PDE residual to decide whether to stop early and selects the state with the smallest residual among the iterates as the output. The paper provides theoretical analyses of local contraction mappings, uniform convergence across parameter families, and the local equivalence between residuals and solution errors. Experiments cover two-dimensional Navier-Stokes, two-dimensional reaction-diffusion, and one-dimensional Burgers equations, and compare FNO, DeepONet, PINO, IRNO, and residual-extension variants. The results show that FiNO achieves lower residual RMS across all three PDEs, with a more pronounced RMSE advantage on Navier-Stokes and Burgers. The paper also examines parameter extrapolation, 512-frame long-horizon rollouts, loss ablations, and residual refresh frequencies.

Critical strengths
  1. The method is conceptually clear: the PDE residual is computed explicitly at inference time and directly used to drive a learned correction. This differs from one-shot prediction models and from methods that impose physics losses only during training.
  2. The experimental scope is fairly complete, covering three PDEs, varied parameter settings, long-horizon rollouts, and several ablations. The residual-refresh-frequency study is especially useful because it shows that FiNO's improvements are not simply a consequence of making more network calls.
Critical weaknesses
  1. The convergence claims rely on strong assumptions, including T(B) ⊆ B, a Jacobian norm of the iteration map below 1, Gθ(u,0)=0, a correction-error ratio ρ<1, uniform contraction over the parameter family, and local invertibility of the discrete Jacobian. The paper does not report the trained model's actual contraction factor, correction-error ratio, or zero-residual output, so it is currently unclear whether these conditions hold for the experimental model. Lcont and Lzero used in training are soft constraints on finite samples and cannot replace the conditions required by the theorem over an entire neighborhood. One further point needs clarification: the experimental residual uses the Crank-Nicolson discretization on a fixed grid, whereas the reference trajectories are generated by an integrating-factor Runge-Kutta method. The paper mentions possible discretization error but does not quantify its effect on residual-based stopping or the theoretical interpretation.

  2. Accuracy and computational cost are not compared under a common budget. FiNO averages approximately 38.7 iterations and 333 ms per solve on Navier-Stokes. Direct FNO, DeepONet, and PINO mainly perform one forward pass, whereas IRNO uses six refinements. Because the paper does not compare methods at matched wall-clock time, FLOPs, or numbers of network calls, it is not yet possible to determine how much of FiNO's accuracy advantage comes from the method itself or whether it offers a favorable accuracy-cost trade-off in practice.

Questions for authors
  1. To validate the theoretical convergence conditions, could the authors report the following three quantities for a trained FiNO: the iteration-map Jacobian norm or empirical contraction factor q from Eq. (10), the correction-error ratio ρ from Eq. (12), and the output norm ||Gθ(u,0)|| under zero-residual input? Please report both mean and worst-case values over test trajectories and across different PDE parameters.

  2. Appendix C.3 uses a residual computed with a fourth-order Crank-Nicolson discretization, whereas the reference trajectories in Appendices D.1-D.3 are generated by an integrating-factor Runge-Kutta method. Please report the residual floor of the reference trajectories under the Crank-Nicolson residual and clarify whether u* in the paper is treated as an exact zero of this discrete residual. If possible, test the relationship between the residual-based stopping rule and actual solution error by matching the time-discretization method, varying the time step, or changing the spatial grid.

  3. Could you compare FiNO, IRNO, FNO, PINO, DeepONet, and classical iterative solvers at matched wall-clock time, FLOPs, or numbers of network calls? Please provide accuracy-versus-computation curves rather than only RMSE at a fixed iteration count or on the final output.

  4. FiNO averages approximately 38.7 iterations, IRNO uses six refinements, and direct-prediction baselines typically make only one forward pass. Please report parameter counts, per-call network cost, residual-computation cost, total inference time, and GPU memory usage for each method, so that readers can assess whether FiNO's accuracy gains persist under a common computational budget.

1 comment
AuthorsOpenReview ↗

Response to Reviewer XyPq

Q1 (theoretical convergence condition satisfaction). Please refer to Response to common questions Q1.

Q2 (residual divergence of CN4 and ifRK4). The divergence of Crank-Nicolson residual (CN4) and integrating-factor Runge-Kutta (ifRK) may lead to conflict in the supervision signals. To address this, we evaluate the CN4 residual values on the GT dataset generated via ifRK to determine whether their zero points are consistent.

residual meanresidual medianresidual P95max residual
2.43e-42.91e-66.80e-41.13e-2

On the 100 in-distribution test windows, the mean normalized CN4 defect of the IF-RK4 reference trajectories is 2.43e-4, well below the mean residual of 1.59e-2 at the returned iterates. Thus, the reference residual floor does not dominate practical stopping behavior in these tests.

We compared a residual-based stopping rule with an oracle stopping rule (based on solution error), which yielded an RMSE of 0.163207 and 0.161895, respectively—a relative difference of only 0.81%. These results support the practical effectiveness of residual-based stopping and the utility of the CN4 residual.

Q3 (performance at matched budget). We added an FLOP-matched comparison between FiNO and FNO on Navier–Stokes. Specifically, we retrained a compact FiNO (width 16, projection width 32) and an enlarged direct FNO (width 152, projection width 284), using the same training data. There is an approximately 30 times difference in FLOPs between the FiNO and FNO models, a design choice intended to align the inference overhead of the iterative approach with that of the direct approach. The results are as follows:

MethodGFLOPs / callMean net callsMean GFLOPsRMSE
Large FNO32.094132.0940.51887
Small FiNO1.06930.9832.8080.37654

The results show that FiNO achieves better performance over FNO at matched FLOPs budget.

Q4 (budget vs performance). Please refer to Response to common questions Q2.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to common questions.

Q1 (theoretical convergence condition satisfaction). All three reviewers concerns about how the theoretical convergence conditions are satisfied for learned models.

We first evaluate whether the trained model is contracting. We track the number of contracting steps and non-contracting steps during the iteration process, the contraction ratio, the magnitude of the RMSE increase during non-contracting steps, and the final converged RMSE. The results are summarized in the table below (models are from loss ablation study).

contracting stepsnon-contracting stepscontraction percentagenon-contracting steps meanRMSE
full34.23.191.70%0.1%0.172
w/o solution contraction30.61.695.10%0.3%0.195
w/o residual contraction12.60.397.60%0.4%0.381
w/o both9.70.693.70%0.1%0.396

Experiments show that the model satisfies the contraction condition for over 90% of the time steps. Incorporating the contraction of the solution and the residual does not increase the contraction rate itself, but it helps the model maintain a stable contraction rate over a longer sequence of time steps. Without this, the model is more likely to trigger the early-stopping condition—due to an increase in the residual—which leads to insufficient convergence.

Next, we evaluate the empirical qq (Eq. 10), the correction-error ratio ρ\rho (Eq. 12), and the output norm ∥Gθ(u,0)∥\|G_\theta(u, 0)\| under zero-residual input.

qq meanqq maxρ\rho meanρ\rho maxmean zero-residual output norm
test set0.925831.00860.907251.0330.0069776
re=10000.96471.00170.956431.00320.020327
re=100000.970621.00340.965621.01520.025609
re=500000.972141.00360.968061.04110.025892
forcing=0.50.963911.00150.955641.00490.014257
forcing=1.00.964981.00180.956481.11580.014988
forcing=2.00.966951.0010.958351.11370.017134

Q2 (inference-time vs accuracy). Reviewer XyPq and qK7Q request for computing budget versus performance for each method. Due to its iterative structure, FiNO entails higher computational overhead, but it offers higher accuracy and better satisfies the governing equations. We further offer a FLOPs-matched performance comparison between FiNO and FNO (response to reviewer XyPq Q3).

methodsParams (M)net. cost per call (ms)res. cost per call (ms)time (ms)peak GPU Mem (MB)RMSE
DeepONet0.040.701.822.3656.020.6192
FNO50.378.511.8210.37497.060.573
PINO50.376.561.828.41497.060.6807
IRNO50.4110.78 (6 refine.s)1.8212.64497.210.5418
FiNO50.396.821.82333.24592.750.1808
Classical spectral IF-RK4------1514.56----

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.