- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes using Fourier neural operators trained with the standard loss to derive efficient preconditioners for solving a PDE. The paper theoretically analyzes the effectiveness of the method on neural operators with and without nonlinearities. The method shows empirically that the proposed post-processing technique yields significant improvements over using the Jacobian of the neural operator directly.
Critical strengths
- The theoretical results are strong. The results are quite robust to the nonlinearities that are often present in FNOs. Furthermore, the theoretical results also hold up in experiments.
- The experiments are very rigorous. The method has been tested on linear/non-linear PDEs, varying spatial dimensions, and varying boundary conditions.
Critical weaknesses
- The results of the paper would be most satisfying if purification outperformed the Fast Poisson Solver across all settings. Purification improves on the direct FNO Jacobian, which highlights the importance of the proposed method if we wanted to use the FNO Jacobian as a preconditioner. However, the motivation for using an FNO Jacobian at all seems lost in the experiments.
Questions for authors
- How did the Direct Jacobian perform in Darcy2D setup? The results have not been presented in Table 3.
- FPS seems to outperform the proposed method in most cases. What is the intuition behind this performance gap? Can this be supported theoretically?
3 comments
Thank you for your response. The inclusion of the missing Darcy2D results addresses my first question. I also appreciate the explanation of the performance gap between FPS and purification, particularly the discussion of frequency-dependent errors.
While I still have some reservations about the practical advantages of purification over classical preconditioners, I recognize that the primary contribution of this work is demonstrating that useful preconditioning structure can be extracted from neural operators trained solely for solution prediction.
Overall, I believe the authors have adequately addressed my questions, and I will maintain my original score.
Thank you for your thoughtful comments and constructive feedback. We sincerely appreciate your recognition of the central contribution of our work and are glad that our clarifications have addressed your questions.
Response to Reviewer pgWh: Darcy Direct results, applicability, and limitations
Thank you for your thoughtful and constructive feedback. Below, we clarify the omission of the Darcy2D Direct results and discuss the applicability and significance of our approach.
1. Direct Jacobian results in Darcy2D (Q1)
This was an oversight in our presentation. Although the caption of Table 3 stated that Direct fails in all tests, we did not include its results in the table itself. We have revised Table 3 to include explicit Direct rows, showing 0/50 successes under both initializations in every Darcy2D setting.
2. Applicability and limitations: motivation and comparison with FPS (W1 & Q2)
We agree that FPS achieves lower iteration counts in most tested settings. The motivation for using the FNO Jacobian is that a model trained only for solution prediction may already contain useful information about the inverse PDE action beyond its prediction as an initial guess. FPS uses multipliers derived from a prescribed Poisson reference operator. Purification instead extracts problem-dependent multipliers from the learned source-to-solution map. For nonlinear FNOs, these multipliers can also depend on the input source. Comparing Direct with Purify shows that structural extraction can make this learned information useful for preconditioning without retraining.
FPS is particularly effective in diffusion-dominated problems because its multipliers capture the dominant diffusion structure. Purification's flexibility does not ensure that every mode is accurately learned. Modes with little energy in the training solutions contribute little to the prediction loss, so solution supervision may constrain their relative multiplier errors weakly. This is reflected in the relative mode-wise errors in Figure 3 and quantified by the frequency-dependent error bound in Corollary 2.3. Thus, the gap with FPS can arise even when purification recovers useful preconditioning information.
In the revisions, we added a Helmholtz2D comparison with the Graph Neural Preconditioner (GNP), a learned preconditioner that constructs a nonlinear approximate inverse action from the sparse system matrix. Table 1 in Section 3.3 reports the results:
In this setting, Purify performs better than GNP in terms of iteration count, although FPS achieves the lowest iteration counts.
Strong nonlinearity or transport can make purification less effective when it discards important mode coupling or phase information. As discussed in Section 4, our work offers insight for more complex PDEs by showing that useful preconditioning structure can be recovered from neural operators trained only for solution prediction. This insight motivates the development of extraction methods that retain useful mode coupling while preserving a structure suitable for efficient preconditioning. Beyond FNOs, other prediction-trained neural operators may also encode useful preconditioning information. Understanding their learning biases could guide its extraction instead of imposing additional training constraints.