acceptodds
Under review as a conference paper at ICLR 2027

Error-Conditioned Neural Solvers

Abstract

Neural surrogate models offer fast approximate mappings from PDE parameters to solutions, but they typically treat solving as a purely statistical task: once trained, they struggle to correct their own constraint violations and extrapolate beyond the training distribution. Recent hybrid methods promote physical correctness by targeting the PDE residual via gradient descent or Gauss–Newton steps, but inherit the compute cost and instability of the underlying classical optimizers. We show, theoretically and empirically, that numerically minimizing the PDE residual can be an unreliable proxy for reconstruction accuracy in ill-conditioned systems, explaining why these methods often do not make accurate predictions despite achieving low residuals. We propose error-conditioned Neural Solvers (ENS), built on a different principle: rather than an optimization target, the PDE residual field is passed as a direct input to the network at each iteration, enabling it to read the spatial structure of its own errors and learn an update policy to iteratively correct its predictions. Across four PDE families, ENS attains the highest prediction accuracy in the large majority of settings, with gains reaching on turbulent Kolmogorov flow, while avoiding the expensive compute cost of hybrid methods. ENS's learned correction policy generalizes under distribution shift, including zero-shot parameter changes and cross-equation transfer, where its relative advantage is largest in the ill-conditioned regimes where residual minimization is least reliable.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
partial differential equations, neural operators, scientific machine learning, hybrid PDE solvers, residual correction, iterative refinement, diffusion models, distribution shift, operator learning, physics-informed machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_k1ZQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes error-conditioned neural solvers, which pair a predictor network that gives an initial PDE solution with a recurrent corrector that refines it. At each step the corrector inputs the current prediction together with its PDE residual field and outputs an update. Both networks are trained jointly with a reconstruction loss on few unrolled iterates without a residual loss. At inference the corrector runs until the residual plateaus. Ablations show that the gain comes from conditioning on the residual rather than from extra iterations, and that conditioning on the residual works better than conditioning on the physics-loss gradient.

Critical strengths
  • The idea is very simple yet general, despite being already exploited.
  • The evaluation covers a large series of PDE, which is rich and demanding.
  • The ablation is informative.
Critical weaknesses
  • Prior work and novelty.
    • The paper leaves out its closest prior work: Le Boudec et al. (2025). That work already proposes a learned "neural solver" for parametric PDEs. A network takes a physics-error signal (the gradient of the PDE residual loss) together with the PDE parameters and forcing/boundary terms, and outputs an iterative correction. The update is applied with a fixed step size, the iterations are unrolled for a few steps during training, the model is supervised only on reconstruction error against ground-truth solutions, and an FNO is used as the backbone. The work is motivated by the ill-conditioning of the physics loss, and it is evaluated on Helmholtz, Poisson and Darcy, among other problems. ENS follows the same template. The main differences are that it conditions on the residual field rr rather than ∇LPDE=Jr⊤r\nabla L_{PDE} = J_r^\top r, works directly in field space rather than on basis coefficients, and adds a learned predictor for initialization.
    • Three claims in the paper therefore need to be revised. The first is that ENS is "the first Neural Solver to receive the residual as a direct input" (Tab. 1). The second is that Neural Solvers are "a new class of methods" (Sec. 4). The third is that existing hybrid methods all use the residual as an optimization target. Le Boudec et al. train on reconstruction loss alone, so the residual-reconstruction gap of Prop. 1 does not explain any weakness in their method, and the taxonomy in Tab. 1 has no category that fits it. The residual-versus-gradient conditioning ablation is in effect a comparison against this prior design. It should be presented as such with credit, rather than as an internal design choice. I ask the authors to cite and discuss this work, to state their contribution relative to it and ideally to include it as a baseline on the shared Helmholtz, Poisson and Darcy settings.
  • No comparison to classical solvers or PINNs. ENS is presented as a solver, yet the paper never compares it with classical solvers or PINNs at matched accuracy. Le Boudec et al. (2025) compare their learned solver against SGD, Adam and L-BFGS on the same discretization, and against several physics-informed baselines (PINNs, PPINNs, P2INNs, PI-DeepONet, PINO). ENS has nothing equivalent. Newton's method is shown only diverging from a noisy initialization (Fig. 4), even though the authors use it to generate their nonlinear Helmholtz data. PINNs appear only as a residual curve in Fig. 4. On NS and Kolmogorov flow, the speed-up is measured only against the high-fidelity data solver, with no accuracy-matched classical baseline.
  • For each setting, including the shifted ones, the number of steps KK and the step size β\beta are tuned on held-out samples from the target distribution. It is unclear whether the baselines got comparable tuning.

References:

  • Le Boudec et al. (2025), Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methods, ICLR 2025
Questions for authors

See Weaknesses.

Reviewer_vfwAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ENS uses the PDE residual as an input to the network, instead of minimizing it as a loss. A predictor gives an initial solution, and a corrector updates it using the current solution and residual. Training only uses reconstruction loss (K=5).

Proposition 1 shows that a small residual does not always mean a small solution error, especially for ill-conditioned systems. Experiments cover five PDEs and different OOD settings.

Critical strengths

Simple and fast. ENS is 100–1000 times faster than other listed method;

Good ablations. Removing the residual input or using the residual gradient makes the performance worse;

The authors also report cases where ENS is not the best, very honest.

Critical weaknesses

Weak theory. Proposition 1 is a classical worst-case result. It does not prove that baselines actually have this problem. Also, the constant is inconsistent between the main paper and appendix (ε/σmin vs. ε/(2σmin)). ENS has the same problem. Figure A1 shows that the residual keeps decreasing while the L2 error increases. So ENS still needs early stopping, and its accuracy depends on the initial solution.

Questions for authors

Without ground truth, how can we decide when to stop the correction steps?

Would ENS still outperform the baselines if they were properly tuned?

Reviewer_9XqzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce Error-Conditioned Neural Solvers (ENS), which refine PDE solution estimates by conditioning on the current prediction and its PDE residual field. ENS uses two networks: a predictor network and a corrector network. The predictor produces an initial estimate, and the corrector is applied repeatedly to refine the predicted solution field or trajectory. The authors evaluate ENS on multiple PDEs across a range of settings, including parameter shifts, super-resolution, and transfer between equations. ENS improves prediction accuracy in most evaluated settings and has substantially lower inference cost than the evaluated test-time optimization and generative correction methods.

Critical strengths
  • ENS demonstrates strong empirical results. Across the reported systems, it improves reconstruction accuracy, including substantial gains on turbulent flows and under distribution shifts.
  • I appreciate the simplicity and potential generality of the approach. The formulation does not prescribe a single architecture and uses a straightforward reconstruction loss. The reported training procedure detaches inputs between correction steps, avoiding backpropagation through the residual computation across iterations.
  • The evaluations cover a range of settings and are well designed. The authors report reconstruction error, PDE residual, and inference cost, and investigate correction beyond the number of training iterations and sensitivity to initialization.
Critical weaknesses
  1. Robustness to ill-conditioning and stiffness remains unclear. The paper motivates ENS by pointing to a mismatch between residual and reconstruction error. However, training with reconstruction loss does not itself establish robustness to poorly conditioned or stiff problems. Remark 2 and Figure A1 show that ENS’s residual can converge while reconstruction error increases. This limits the claim that ENS avoids the residual–reconstruction gap. Performance in strongly stiff regimes is also not explicitly characterized.
  2. The evaluation is limited to 2D systems. Moving to 3D can increase computational and memory requirements and introduce more complex multi-scale dynamics. It remains unclear how effectively ENS transfers to these settings.
  3. The consistency of the benefits across architectures remains unclear. ENS uses both FNO-based and transformer-based backbones, and Table A6 evaluates several architectures. However, the architecture sweep is restricted to in-distribution linear Helmholtz and does not compare ENS variants with its corresponding non-ENS baselines. Further analysis would help establish when residual conditioning improves a given backbone.
Questions for authors
  1. Remark 2 and Figure A1 show that the residual can remain low while reconstruction error increases. What happens to the corrector’s predicted updates after reconstruction error reaches its minimum? Section 4 describes selecting KK using residual convergence on held-out validation samples: are reconstruction labels also consulted, and how does this criterion avoid the deterioration shown in Figure A1?
  2. Can the authors discuss the challenges of applying ENS to a 3D system and, if feasible, provide an initial experiment? I recognize that a full 3D evaluation may be difficult within the short discussion period, but a limited demonstration would greatly strengthen the evidence for scalability.
  3. What architectural properties are necessary for successful residual conditioning, and how should readers interpret the pure-FNO failure in Table A6? Have the authors considered using different architectures for the corrector and predictor?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.