acceptodds
Withdrawn from ICLR 2027

Learning Compact Lanczos Continuations for Efficient PDE Response Evaluation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Accurate evaluation of scalar responses of partial differential equations (PDEs) can be computationally demanding, particularly in high-contrast heterogeneous media. Limiting the number of Lanczos iterations reduces this computational cost, but leaves unresolved contributions that compromise response accuracy. We address this trade-off by learning a compact continuation of the Lanczos model, improving its response approximation without additional Lanczos iterations. For a class of linear second-order elliptic PDEs, we establish an existence guarantee for finite-dimensional continuation: accurate energy response approximation over an entire prescribed high contrast interval is possible with a continuation dimension independent of the full discretization size. At the same reduced dimension, we derive an explicit improvement factor over the classical Lanczos response bound under stated high-contrast conditions. Experiments on heterogeneous conductivity problems demonstrate lower computational cost at comparable attained response accuracy on the evaluated benchmarks, alongside improved accuracy in the tested contrast-extrapolation regimes. The sharper guarantee comes from adapting the approximation space to the inverse-response kernel while retaining the information encoded by the computed Lanczos prefix.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Partial differential equations, Krylov subspace methods, Learning-based model reduction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ts3SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies scalar energy responses of boundary-driven, two-phase affine elliptic PDEs. It retains a short Lanczos prefix and its terminal coupling, then predicts a compact tridiagonal continuation from instance features. For a given instance, the continuation is independent of the contrast query and can serve multiple response evaluations. A recursive decoder enforces two-phase energy inequalities and applicable known zero-energy endpoint conditions when the required connectivity information is available.

Theorem 1 establishes the existence of such continuations with a uniform relative response error bound over a prescribed finite contrast interval. Proposition 1 and Corollary 1.2 compare analytic error bounds at the same reduced dimension, giving a strictly tighter continuation existence bound when q≥3,S≥4q\ge3,S\ge4. Experiments cover contrast interpolation and extrapolation on held-out IID geometries, encoder and constraint ablations, and query costs for a separately trained high-resolution model.

Critical strengths

Strength 1: The theoretical guarantee concerns the evaluated response

Theorem 1 in Section 3 and Appendix A construct compact continuations preserving the Lanczos prefix and terminal coupling, with a uniform relative energy-response error bound on a prescribed finite interval. The required continuation dimension is independent of the full discretization dimension. Appendix B also explicitly converts the ordinary Lanczos estimate to a response error, giving the bounds at equal total dimension a common target.

Strength 2: The constraints reflect the two-phase PDE's energy structure

Section 4 and Appendix D incorporate two-phase energy inequalities into the recursive decoder and use known connectivity to impose applicable zero-energy endpoint conditions. The constraints act directly on the predicted matrix. In Table 2, removing the endpoint equalities while retaining the energy inequalities increases the mean relative error at s=99999s=99999 from 1.811.81\\% to 115.49115.49\\%, demonstrating the practical role of endpoint information in this extrapolation task.

Strength 3: The experiments connect accuracy gains with query cost

At s=99999s=99999, Table 1 reports a mean error of 1.811.81\\% for the constrained MLP and 353353\\% for ordinary 28-step Lanczos. Across twenty 409624096^2 geometries, Table 3 reports an MLP query time of 11.72±1.9511.72\pm1.95 seconds and response error of (3.83±0.425)×10−4(3.83\pm0.425)\times10^{-4}. FFT-PCG takes 31.24±14.8231.24\pm14.82 seconds with an error of (8.21±1.91)×10−4(8.21\pm1.91)\times10^{-4}. The attained errors are comparable, although unequal. Appendix E's timings include instance assembly, numerical iterations, connectivity features, network evaluation, and response reconstruction. They measure initialized query work and exclude offline costs such as training, label generation, and reusable reference-operator setup.

Critical weaknesses

Weakness 1: The conditional network's additional benefit has not been isolated

Section 4 uses instance features to predict seven decoder logits, but Table 2 varies the constraints while training every variant. The decoder in Appendix D.1 still depends on the instance prefix and energy scalars even when the network outputs the same logits for every instance; Appendix D.2 also applies instance-dependent endpoint rules. A control fitting seven shared logits with the same labels and constraints would therefore retain an instance-dependent completed matrix. Neither this control nor the zero-logit continuation is evaluated. Comparing them with the conditional MLP would distinguish the effects of the default construction, fitting shared coordinates, and conditional prediction. The current results establish the complete method's advantage over ordinary Lanczos, while leaving the conditional network's additional contribution unclear.

Weakness 2: The high-resolution efficiency comparison lacks records needed for independent checking

Section 5 states that Table 3 uses a separately trained 409624096^2 model, and Appendix E specifies frozen seed-11 weights. The reported 512/128 training and validation split concerns the 1282128^2 data. The high-resolution model's dataset size, split, relationship to the twenty timed geometries, and checkpoint-selection procedure are incompletely specified. Appendix E.2 gives a general validation objective without clearly documenting its application to this model.

Solver budgets are calibrated offline and frozen before timing, and timed calls do not access reference answers. What remains needed is the actual iteration budgets, whether calibration is global or per instance, and reference-solver tolerances or accuracy-refinement checks. These records would allow readers to check Table 3's error scale around 4×10−44\times10^{-4} and its cost comparison.

Minor comments

  1. Table 1's "MLP free ext." and Table 2's variant with both constraints disabled have means consistent at Table 1's reporting precision, but different standard deviations. They report 18.08 versus 19.10 at s=9999s=9999, and 234 versus 238.77 at s=99999s=99999. Please clarify whether the fits and aggregation are the same and align the values if they are.
Questions for authors
  1. With the same prefixes, scalars, energy constraints, and endpoint rules, what are the test errors of the zero-logit continuation, seven shared decoder logits fitted using the same training labels, and the conditional MLP? How does conditional prediction change the test error relative to fitting shared coordinates?
  2. Could you specify the high-resolution dataset size, training/validation split, relationship to the timed geometries, and checkpoint-selection rule, together with Table 3's reference-accuracy checks, actual solver budgets, and calibration granularity?
  3. On the same test instances and queries, how do response error and complete query time change as ordinary Lanczos continues iterating? With the learned model kept frozen, what is the cost of matching its error level, and which tighter accuracy targets can each method meet?
Reviewer_mM2uOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper targets fast evaluation of a scalar energy response RX(s)R_X(s) of boundary-driven, two-phase affine elliptic PDEs over a range of contrasts ss. After mm Lanczos steps on the induced load, the unresolved part of the Jacobi matrix is replaced by a short q×qq \times q tridiagonal "continuation" that keeps the computed prefix JmJ_m and the terminal coupling βm\beta_m, and is used for every s∈[0,S]s \in [0, S].

  • Theory. Theorem 1 shows, through rational Gauss quadrature on a reweighted spectral measure, that such a qq-dimensional tail exists with a uniform relative error bound Bm,q(S)B_{m,q}(S) independent of the discretization size. Corollary 1.2 shows that this bound is tighter than the classical CG/Chebyshev bound for (m+q)(m+q)-step Lanczos when q≥3q \geq 3 and S≥4S \geq 4.
  • Method. A network (MLP, Transformer or sequence FNO) maps the Lanczos prefix, a few scalars and phase-connectivity indicators to tail coordinates. A differentiable decoder enforces the two energy inequalities and known zero-energy endpoint conditions. Training uses response-level supervision only.
  • Experiments. 2-D two-phase conductivity at 1282128^2 shows large improvements over 24- and 28-step Lanczos in contrast extrapolation. An ablation shows that the endpoint equalities are crucial. Timings on large-scale problems report lower end-to-end cost than Lanczos, FFT-PCG and AMG-PCG at comparable error
Critical strengths
  1. A principled, structure-preserving way to combine learning with a Krylov method. The method learns only the unresolved part of a Jacobi matrix while keeping the computed prefix exactly. Its decoder guarantees physically admissible outputs (energy inequalities, correct asymptotic growth). Theorem 1 gives a meaningful existence result, i.e., a discretization-independent continuation dimension suffices over a whole contrast interval. This motivates the representation and distinguishes the work from black-box surrogates. The contrast is clear in Table 1, where field FNO/Transformer models fail completely under extrapolation, and unconstrained residual heads degrade sharply.

  2. Strong extrapolation results (Table 2). Trained only up to s=999s = 999, the constrained model keeps a mean relative error of about 1.81.8% at s=99,999s = 99,999, where Lanczos with 2828 steps has a mean error of about 350350%. The three encoder families behave consistently, which suggests that the gain comes from the representation and decoder rather than from a specific architecture.

Critical weaknesses
  1. Non-learned numerical baselines that exploit the same information are missing, so it is unclear how much the learning contributes.
  • The ablation (Table 2) shows that almost all of the extrapolation gain comes from the endpoint equalities. Without them, the error at s=99,999s = 99,999 goes from 1.81.8% to about 115115%, roughly the Lanczos level.
  • In quadrature terms, imposing a zero-energy endpoint condition is closely related to Gauss–Radau / Gauss–Lobatto rules with a prescribed node. The theorem itself is built on rational Gauss quadrature.
  • Without such baselines, the paper does not show that a learned tail outperforms classical tail completions that use the same structural knowledge.
  1. The theory and the method are only loosely connected, and the evaluation is narrow.
  • Theorem 1 is an existence result whose construction requires the full spectral measure of the instance. The learned network has no guarantee. Corollary 1.2 compares two upper bounds: a tighter bound does not imply a smaller actual error than (m+q)-step Lanczos (App. C acknowledges that "a small ratio alone is not a small absolute-error guarantee"). The theory therefore motivates the representation but does not explain the empirical gains, which mainly come from the endpoint information (see above).
  • The timing study (Table 3) uses a separately trained model at a single fixed contrast and a fairly loose accuracy. The cost of generating 4096² training labels and of training is excluded.
Questions for authors

Clarifications:

  1. How does a training-free completion perform?
  2. With the endpoint equalities enforced, how much does the network output matter?
  3. What is the total offline cost (label generation, training)?
  4. How does the method behave at tighter tolerances?

Suggestions: 5. Discussing explicitly how the decoder's endpoint constraints relate to Gauss–Radau/Lobatto quadrature, and how Theorem 1 relates to existing rational-quadrature results, would clarify what is new in the theory. 6. An experiment on a second PDE family covered by the theory would support the generality claim.

1 comment
Reviewer_mM2uOpenReview ↗

Discussion update

The discussion period is closing and I want to note that I've not received a response from the authors. I want to remark that my concerns remain unanswered, but that I remain open to further discussion. Otherwise, my assessment is unchanged.

Reviewer_KPbCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a framework for predicting scalar energy responses of parametric elliptic PDEs by retaining a computed mm-step Lanczos prefix and learning a compact tridiagonal tail of dimension qq that respects the two phase-energy inequalities. It proves a finite-continuation existence theorem with a discretization-independent uniform bound (Theorem 1) and derives an explicit comparison with the ordinary Lanczos bound at the same reduced dimension (Corollary 1.2).

Critical strengths

This manuscript correctly identifies the gap it fills: no existing work combines a retained Lanczos prefix with a learned, energy-feasible tail for parametric PDE responses.

Critical weaknesses

First, this manuscript is a little bit hard to follow. The biggest concern is the "existence vs. learnability". While Theorem 1 guarantees the "existence" of such a tail block, the authors have yet to demonstrate how the network, which relies on only a 56-dimensional input, can effectively bridge this gap. The idea of learning a compact, energy-feasible continuation of a Lanczos prefix is not covered by prior work on Krylov methods, rational quadrature, or learned solvers. The main weaknesses are the gap between existence and learnability, the absence of numerical robustness diagnostics, and the omission of deflated Krylov baselines.

Questions for authors

See Critical Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.