- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This manuscript proposes a preconditioned residual loss function and preconditioned PDE solver training routine, realized through geometric and algebraic multigrid. The proposed routine applies to linear and nonlinear equations, steady or time-dependent, on structured and unstructured meshes, is agnostic to the neural operator architecture, and adds no cost at inference.
Critical strengths
- Strong motivation and a clear mathematical interpretation of the proposed components.
- Seamless integration with classical and neural PDE-solving methods, with many favorable properties (such as not requiring changes to the NO architecture, applying to multiple PDE formulations across multiple discretization routines, etc.)
- Theoretical analysis of how preconditioning changes the conditioning of the residual loss.
- Performance gain across multiple baselines and PDEs.
Critical weaknesses
- Improving the conditioning of the middle PDE-dependent factor does not necessarily guarantee good conditioning of the full parameter-space matrix. This is also an author-recognized theoretical limitation.
- Performance gains over "supervised learning" routines on Poisson and Allen-Cahn are less significant than on Stokes, which limits the practical scope of this solver, especially in cases that do not need to be physics-informed.
- Baselines only cover PINO and physics-informed DeepONet, which are old baselines. No newer baseline is discussed or compared with.
- The proposed nonlinear preconditioning strategy is problem-specific and still needs manual design/tuning. The overall process is not automated, and adapting the solver to new problems requires manual effort and a solid mathematical background. The actual generalization cost is higher than claimed in the paper.
Questions for authors
- Please see "weaknesses" above.
2 comments
Thank you for your detailed explanation. I am keeping my original score of weak accept.
We thank you for taking the time to review our manuscript. We comment on your mentioned weaknesses:
The hypothesis of the manuscript is that supervised neural operator learning works ``reasonably well'', especially in comparison to the physics-informed alternatives before this work. In that sense our goal was to mitigate the spectral ill-conditioning introduced through physics-informed loss functions. To which extend ill-conditioning of neural networks in general is a problem is a broader question that we do not attempt to answer. Historically, the introduction of initialization schemes and momentum-based first order methods were landmark results in this direction. We see our contribution as orthogonal to that development.
We don't expect to improve upon data-driven learning, even though physics-informed learning improved upon Stokes in our specific test example. Rather, the purpose of the method is to eliminate the data-generation process and allow training at the infinite-data limit where no sample is revisited.
We now consider more baselines, including a preconditioned variant of DeepONet and Transolver, please compare to the global answer for details.
We agree with this statement. Preconditioning as a mathematical discipline however is mature, so for many equations a suitable preconditioner is textbook material. We therefore see the reviewer's observation as a strength as well: physics-informed operator learning can draw directly on decades of preconditioning research for classical PDE solvers, and every advance there carries over without changes to the network architecture.