acceptodds
Under review as a conference paper at ICLR 2027

Spectral Preconditioning from Predicted Invariant Subspaces

Abstract

Krylov subspace methods are indispensable for solving large sparse linear systems, but their cost can be dominated by slowly converging spectral components. Preconditioning is widely used to accelerate Krylov subspace methods, and several preconditioners have been implemented in scientific computing packages. Neural Krylov Iteration (NeurKItt) showed that a neural operator can amortize the prediction of an invariant subspace and use it to augment a Krylov process for PDE-induced linear systems. The key idea behind NeurKItt is to address the slowly converging components separately. We argue that the invariant subspace can also define a problem-specific preconditioner. Using the invariant subspace predicted by NeurKItt, we define a symmetric positive definite (SPD), low-rank structured spectral preconditioner. The construction requires only matrix–vector products and dense algebra at the dimension of the predicted subspace. It clusters the spectrum of the preconditioned system and thereby reduces the number of Krylov iterations in methods such as GMRES. Theoretical analysis and experiments demonstrate the feasibility and effectiveness of the learned preconditioner.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Linear Systems, Krylov Subspace Method

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_FkLmOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes repurposing the predicted subspace from a neural operator method as a preconditioner for Krylov subspace methods, particularly GMRES. The paper presents a theoretical motivation for this approach and validates it on a set of parametric PDEs.

Critical strengths

S1: Important problem

Krylov methods are the mainstay of much of large-scale scientific computing, and good preconditioning is key to their efficiency.

S2: Potentially novel approach

I'm not aware of other learned approaches that leverage invariant subspaces for preconditioning.

Critical weaknesses

W1: Clarity

The related work section is terse to the point where it's practically uninformative. Besides, it is hard to understand before having learned about the proposed method.

Key steps and definitions are missing in the background section, which makes it impossible to follow as it stands. For instance:

  • pm(A)p_m(A) in eq. (3) not defined. Not clear how (4) follows from (3).
  • In (6), how can you map [a,b][a, b] to [−1,1][-1, 1]? How can you just choose to work with Chebyshev polynomials and why is the ratio still a polynomial?
  • In (7), there is no explanation whatsoever of why you can bound (6) from above as stated.
  • In (10), how do get the right-most bound?
  • It is not sufficiently explained why "These bounds show that moving eigenvalues away from zero and clustering the positive and negative spectra can accelerate Krylov methods".

Remark 7 would make much more sense directly after defining R at the start of section 4.1.1.

In theorem 4, I don't see how eq. (29) has any bearing on why P would be SPD.

In 5.1, state and motivate the assumption going into the problems under consideration. Since you're using FNO, I suppose you consider PDEs defined on gridded domains. From the supplementary material, I infer that you're only considering system matrices arising from finite difference methods - this type of information is important to state in the main paper.

W2: Unclear novelty and significance

It's not clear whether the theoretical results in section 4.1 are claimed to be novel - it looks like standard matrix/numerical analysis to me - please clarify. In 4.1.1, I don't follow why you're talking about how to choose α\alpha, despite this seemingly being an important consideration.

Since the analysis builds directly on that from NeurKItt, it is important to distinguish what is, in fact, additional contributions of this work.

W3: Experiments ignore preconditioning time

A critical flaw is that the comparisons in Tables 1 and 2 don't account for the computational cost of obtaining the preconditioner itself. Based on the reported metrics, you could get a perfect performance (convergence in a single iteration) if the "preconditioner" was A−1A^{-1}. Moreover, I suppose you'd expect the GMRES iteration count to gradually converge to 1 as you increase kk, so the interesting thing to consider is what value of kk strikes the best balance between preconditioning cost and GMRES iteration cost (not that increasing kk descreases the GMRES iteration count - that is just a sanity check).

I see that you comment on this at the end of the experiment section, but all details are deferred to the supplementary materials. There, you indeed acknowledge the important limitations with the sentences:

Increasing k reduces the iteration count only up to a point while increasing the costs of inference, orthogonalization, storage, and low-rank application. Our experiments primarily report iteration counts and hence do not yet establish end-to-end wall-clock or memory advantages in all regimes. Frankly, "hiding" this away in the supplementary materials is not scientifically honest.

W4: Missing strong baselines

The lack of strong baselines makes it hard to assess the value of the proposed method. Currently, only classical, general-purpose preconditioners are included, which are not tailored for the problem class. Neither is there any comparison with other learned preconditioners, of which a number has been proposed in the last two years, for instance:

  • Chen, "Graph neural preconditioners for iterative solutions of sparse linear systems." ICLR (2025).
  • Giraud et al. "Neural network preconditioning: a case study for the solution of the parametric Helmholtz equation." Numerical Linear Algebra with Applications (2026).
  • Häusner et al. "Learning incomplete factorization preconditioners for GMRES." Northern Lights Deep Learning Conference (2024).
  • Trifonov et al. "Learning from linear algebra: A graph neural network approach to preconditioner design for conjugate gradient solvers." Computational Methods in Applied Mathematics 26.2 (2026).

The recent survey paper by Meng et al. (https://arxiv.org/abs/2610.07211) includes additional pointers, and could potentially also improve the related work section.

Minor:

  • The theorem numbering is strange (the first theorem is numbered 2, which is followed by theorem 4).
  • There is no section 4.2, hence you could promote subsubsections to subsections.
  • Some references refer to arxiv preprints that are in fact published in venues such as ICLR. Please double-check all references thoroughly.
Questions for authors

Q1: Is your question from the introduction, "can the predicted subspace itself define a robust preconditioner?", really new and descriptive of your work? How is it different from how the established literature on numerical analysis approaches the problem, e.g., in the context of deflation and augmentation methods?

Q2: How long does it take to train your models? Is it memory, compute, or time that limits the problem sizes (which are too small for any gains to be practically significant)? Have you considered using a stochastic approximation of the loss in eq. (45) as in Häusner et al. (TMLR, 2024)?

Q3: In section 5, please elaborate on why "The preconditioner in Section 4 requires spectral information that would be too expensive to recompute for every member of a parametric PDE family". What information is too expensive to recompute, more precisely, and why?

Reviewer_WwXGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper uses the invariant subspace predicted by NeurKItt to build a spectral preconditioner for GMRES. The preconditioner corrects the predicted subspace through a small Rayleigh–Ritz matrix and leaves its orthogonal complement scaled by a parameter. The paper analyzes the spectrum for exact and approximate subspaces. On three families of PDE linear systems, the method often needs fewer GMRES iterations than the classical preconditioners tested.

Critical strengths
  1. The preconditioner reuses an existing subspace predictor and needs no additional learned model. Its construction is simple, and it is positive definite under the stated assumptions.
  2. The reduction in iteration count is substantial for the tested problems. In Table 2, the k=10k=10 or k=20k=20 variant has the lowest count in every reported setting.
Critical weaknesses
  1. The main new component is a deterministic spectral preconditioner built from a predicted subspace. The neural predictor and its training setup largely follow NeurKItt. The paper does not clearly show what new machine-learning insight this construction provides, or why a learned predictor is needed instead of obtaining or reusing a subspace through numerical methods.
  2. The experiments do not isolate the value of learning. In particular, they do not compare the proposed solver with NeurKItt using the same prediction, or with a numerical subspace method under comparable costs. Training also requires eigenspace targets, but the cost of producing those targets and the number of solves needed to recover that cost are not reported. Table 2 reports iterations rather than total solve time.
Questions for authors
  1. What does the learned predictor add beyond the spectral preconditioner itself? Could you compare predicted subspaces with subspaces obtained or reused by a numerical method, including the cost of obtaining each?

  2. How does your method compare with NeurKItt’s augmented GMRES when both use the same predicted subspace?

  3. How expensive is it to compute the eigenspace targets and train the FNO? Approximately how many new linear systems must be solved before these costs are recovered?

  4. Does the predictor still help when the PDE coefficients differ from the training distribution?

Reviewer_AYzMOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes NEURSP, a learned spectral preconditioner for PDE-induced sparse linear systems. It reuses invariant subspaces predicted by NeurKItt and converts them into an SPD low-rank spectral preconditioner for GMRES. The method is evaluated on three two-dimensional PDE families, Helmholtz, heat conduction, and Darcy flow, using iteration counts against unpreconditioned GMRES and several standard preconditioners.

Critical strengths
  1. The paper addresses a relevant numerical linear algebra problem. Accelerating Krylov methods for PDE-induced sparse systems is important, and connecting learned invariant subspaces to preconditioning is a reasonable direction.

  2. The proposed preconditioner has a simple spectral motivation. Reusing a predicted invariant subspace to build a low-rank correction gives the method a clear connection to classical deflation and spectral preconditioning.

Critical weaknesses
  1. The contribution appears highly incremental over NeurKItt and classical spectral preconditioning. The paper mainly takes the invariant subspace already predicted by NeurKItt and converts it into a low-rank preconditioner, but it is not clear that this represents a substantial new method.

  2. The evaluation relies too heavily on iteration counts rather than end-to-end solver cost. The paper itself notes that the proposed preconditioner may have higher per-iteration cost than alternatives such as ICC, so lower GMRES iteration counts alone do not establish practical efficiency.

  3. The experimental evidence is too limited for the strength of the claims. The main evaluation covers only three PDE families and a small number of tables, with no convergence curves, no spectrum visualizations, and no detailed robustness study across discretizations, distributions, or solver settings.

  4. The baseline comparison is not sufficiently convincing. The main tables omit direct comparisons with tuned multigrid/algebraic-multigrid solvers, stronger domain-decomposition baselines, and the NeurKItt/GCRODR-style augmentation approach that is closely related to the proposed method.

  5. The relationship between the theory and the empirical results is underdeveloped. The analysis discusses spectral clustering and perturbation bounds, but the experiments do not directly measure subspace quality, eigenvalue clustering, or failure cases where the predicted invariant subspace is inaccurate.

  6. The presentation of empirical results is quite sparse. The paper contains no figures and only a few main result tables, which makes it difficult to assess convergence behavior, stability, and practical trade-offs beyond the reported iteration counts.

Questions for authors

Please see Critical Weaknesses. I would be happy to reconsider my rating if the authors can provide convincing responses to these concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.