acceptodds
Under review as a conference paper at ICLR 2027

Learning What Multigrid Leaves Behind: Objective Alignment and Its Limits

Abstract

A learned coarse space adds a low-rank correction to a multigrid preconditioner, but which space should the network predict? We separate three requirements. Target: the training target should complement the deployed base preconditioner, not only the operator's low-energy modes. Prediction: a network must predict that target on new operators. Cost: the correction must beat strong classical solvers in total time. For a standard two-level balancing correction, a known result says the condition-number-optimal space consists of the directions the base preconditioner reduces most slowly. We train an FNO on a finite solver target that approximates these directions by repeatedly applying the preconditioner's error propagator to random vectors. With architecture, rank, training operators, and deployment fixed, this target raises the mean iteration reduction on 24 held-out 2D operators from 17.6% (operator-only Rayleigh training) to 31.7%, beating even the Rayleigh objective's exact optimum (18.8%), and targets crossed between two preconditioners lose on every operator. The target requirement therefore holds in this 2D family, and a useful target also exists in a 3D obstacle family. Prediction holds only inside the training band, where it is slightly better than reusing a stored target; it fails on disjoint angles, on lognormal coefficients, and in 3D. Cost favors classical solvers: the corrected preconditioner needs about as many iterations as two V-cycles and more flops, and BoomerAMG is faster in total time.

Primary area:
optimization
Keywords:
learned numerical solvers, multigrid, coarse spaces, neural operators, preconditioning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TXDnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This manuscript does not propose a specific framework; instead, it investigates what a multigrid preconditioner should learn. The authors split this into three aspects, target, prediction, and cost, and studied all three. However, this paper does not yield anything meaningful that can directly build a better solver. The experimental results show that the proposed "aligned" solver underperforms the default AMG. Thus, it is hard to recommend acceptance.

Critical strengths
  • The split of "target/prediction/cost" is interesting.
Critical weaknesses
  • The manuscript is poorly organized, hard to follow, and its contributions are unclear. The abstract fails to convey what is proposed by the authors or what conclusion the authors want to draw. The same problem applies to the remainder as well.
  • The "target-prediction-cost" model looks insightful but does not yield anything meaningful that can be used to construct a new, better solver end-to-end.
  • The 2D rotated anisotropy family used for experiments is too narrow and insufficient to support the claim. A broader evaluation over multiple anisotropy regimes, grid sizes, coefficient contrasts, and base-preconditioner variants is a must to yield persuasive empirical results.
  • The learned method does not demonstrate a compelling end-to-end speedup. At line 430, on the Common-hardware suite, default AMG takes 6.82s wall-clock time, and the proposed variants take 7.39s-18.35s. If a neural method, trained with the proposed "aligned" routine, decelerates the MG convergence, why would we bother to use it?
Questions for authors
  • Please see "weaknesses" above.
2 comments

Thank you for the review. We agree that the submitted abstract and framing were unclear, and we have uploaded a revised PDF that rewrites the abstract and introduction and adds four studies in Appendix F.

The abstract fails to convey what is proposed by the authors or what conclusion the authors want to draw.

The paper is a controlled study of what a learned coarse space should be trained to predict, not a new solver. Its main finding is that training on the error the deployed preconditioner leaves behind works better than the usual Rayleigh objective: on 24 held-out 2D operators the mean iteration reduction rises from 17.6% to 31.7%, beating even the exact Rayleigh optimum (18.8%). The network predicts this target only inside its training range, and classical solvers remain faster. The revised abstract now states these three conclusions, including the negative ones, and ends with "This is a diagnostic study, not a faster solver."

At line 430, on the Common-hardware suite, default AMG takes 6.82s wall-clock time, and the proposed variants take 7.39s-18.35s.

To clarify the table: 6.82 s and 7.39 s are two BoomerAMG settings, 18.35 s is the base Evolution AMG without correction, and 17.85 s is the same base with our correction. You are right that the corrected solver is about 2.6 times slower than BoomerAMG, and we do not claim otherwise.

If a neural method, trained with the proposed "aligned" routine, decelerates the MG convergence, why would we bother to use it?

As a solver replacement, you should not. The correction does reduce iterations of its own base (28.4 to 19.5), but it is not faster than BoomerAMG and only marginally faster than its base at many right-hand sides. The value of the paper is the evaluation lesson: the training target must match the deployed preconditioner, and a learned correction must be compared against an extra cycle, simple lookup, flops and total time. Section 5 turns this into a checklist for future learned solvers.

The "target-prediction-cost" model looks insightful but does not yield anything meaningful that can be used to construct a new, better solver end-to-end.

We agree that it does not produce a faster solver, and we no longer frame it as one.

A broader evaluation over multiple anisotropy regimes, grid sizes, coefficient contrasts, and base-preconditioner variants is a must to yield persuasive empirical results.

We ran two new studies along these axes. Both narrow our claims.

Anisotropy regimes. We retrained both objectives over all angles (0 to 90 degrees) and anisotropy ratios from 10 to 10^4. Solver-aligned training still beats Rayleigh training, but by about one iteration rather than several, and a second AMG cycle is better than either network (8.33 against 10.65 iterations).

Wider range (24 operators)Mean iterationsMean reduction
One AMG cycle (no correction)12.53
Rayleigh network11.684.4%
Exact Rayleigh space11.355.8%
Solver-aligned network10.659.0%
Finite solver target (ideal)10.2612.7%
Two AMG cycles8.3333.9%

Coefficient contrast. We found that the paper's lognormal family, labeled "contrast 10^4", actually has a max/min coefficient ratio of 10^10 to 10^16, because 10^4 is the generator's parameter. The revised paper corrects this. We then ran a new study at moderate contrast (10^2.7 to 10^4). It is negative: solver-aligned training is about one iteration worse than Rayleigh training, and even the ideal target only ties two AMG cycles. Two of the three solver-aligned models ran out of their preset training time, so this study is incomplete, and we report it as such.

Moderate contrast (24 operators)Mean iterations
One AMG cycle (no correction)9.76
Default AMG8.92
Solver-aligned network (one pair)8.35
Rayleigh network7.28
Exact Rayleigh space7.14
Finite solver target (ideal)6.58
Two AMG cycles6.56

Grid size. We have not tested learned 2D predictors at other grid sizes. In 3D, the Rayleigh network's gain falls from 21.3% to 6.6% as the grid grows from 48 to 256, which the revised abstract now states.

Base preconditioners. The Rayleigh network is evaluated around four 2D base preconditioners (Table 1), with gains from 4% to 48%. Swapping the training targets between two preconditioners makes both worse on every operator, which supports the main finding that the target must match its base.

3D. With 200 training operators, solver-aligned training leads Rayleigh training by about one iteration on three fresh 3D suites, but the lead is not confirmed on the last, untouched suite.

Soundness: 1: I have significant concerns about the paper's soundness that do not appear easy to address.

We read the soundness concern as the narrow evaluation and the missing speedup. The revised paper now reports the wider evaluation above, including its negative results, and states plainly that the method is not faster than classical solvers. A scorecard table (Table 5) summarizes where each claim holds and where it does not.

Reviewer_CLUTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies learned coarse spaces that add a low-rank balancing correction to a multigrid preconditioner. It separates three requirements: the training target must suit the deployed preconditioner, a network must predict that target on new operators, and the result must beat strong classical solvers in total time. Instead of the usual Rayleigh objective, which targets the operator's low-energy modes, an FNO is trained on a ``finite solver target'' obtained by applying the preconditioner's error propagator 6464 times to random vectors. On 2D2\text{D} rotated-anisotropy operators, this raises the mean iteration reduction over Evolution AMG from 17.6%17.6\% to 31.7%31.7\%. However, prediction fails outside the training angle band, on lognormal coefficients and in 3D3\text{D}, and BoomerAMG remains about 2.6×2.6\times faster in total time.

Critical strengths

Clear framing: Separating target, prediction and cost is a useful way to evaluate learned solvers, and the checklist in Section 5 is practical for future work.

Controlled experiments: The objective swap, crossed targets between preconditioners and the FNO vs U-Net comparison change one factor at a time and support the solver-alignment claim well.

Honest reporting: The paper reports all failed families and stopped studies, compares against strong baselines like BoomerAMG, and does not overstate contributions.

Critical weaknesses

No practical speedup: In iterations, the corrected preconditioner only ties two Evolution AMG cycles (21.2621.26 vs 21.6321.63). It costs more flops per iteration than two cycles, and BoomerAMG is much faster in total time (6.82 s6.82\text{ s} vs. 17.85 s17.85\text{ s}). The method therefore has no clear use case.

Network adds little over lookup: Inside the training band, the FNO is only slightly better than reusing the nearest stored training target (20.1820.18 vs. 20.6220.62 iterations). nearest-target retrieval is even better than the learned space (26.4126.41 vs. 26.6826.68), and in timing the FNO is not separated from target reuse.

Narrow and selected training family: The angle band θ∈[16,25]∘\theta \in [16, 25]^\circ was chosen after seeing default AMG fail there. The positive prediction result does not hold outside this band, on lognormal coefficients, in 3D or on public data.

Limited novelty: Lemma 1 is a known result, as the authors state. Building targets by relaxing random vectors is established in adaptive and bootstrap AMG. The main new element is distilling this target into an FNO, which is a modest contribution.

Hard to follow: The paper uses many separate suites (12) whose numbers cannot be compared with each other, and much of the evidence is in a 34-page appendix. It is often hard to tell which result supports which claim.

Questions for authors
  1. In what realistic setting would the learned correction be preferred over simply using a second cycle or BoomerAMG?
  2. Since nearest-target retrieval performs almost as well as the FNO, what does the network contribute beyond interpolating a densely sampled two-parameter family?
  3. How does the method perform when trained on a wider range of angles and anisotropy ratios, rather than the narrow band where default AMG fails?
  4. Why does the solver-aligned objective not help in 3D (18.8118.81 vs 18.0418.04 for the Rayleigh FNO), even though the target itself is useful there?
2 comments

Thank you for the clear and fair review. We have uploaded a revised PDF that adds four studies in Appendix F and rewrites the abstract and introduction to state the scope plainly. We answer your questions in order.

In what realistic setting would the learned correction be preferred over simply using a second cycle or BoomerAMG?

We have not shown one, and we no longer suggest otherwise. In the training band the correction ties a second cycle in iterations, and BoomerAMG is faster in total time at every tested count (6.82 s against 17.85 s at 64 right-hand sides). A learned space could matter for new operators that a stored library does not cover. We tested this twice and both results are unfavorable: outside the training band the network loses to reuse, and over the wider range it loses to a second cycle. The revised abstract ends with "This is a diagnostic study, not a faster solver."

Since nearest-target retrieval performs almost as well as the FNO, what does the network contribute beyond interpolating a densely sampled two-parameter family?

Very little inside this family. On 24 fresh operators the network beats nearest reuse by 0.6 iterations, interpolation by 0.4, and a quadratic fit on (angle, anisotropy ratio) with no network by 0.5. These are real but small differences, and we now say so in the paper. What the paper contributes is the finding about the training target: with everything else fixed, training on the error the preconditioner leaves behind beats Rayleigh training and even the exact Rayleigh space (31.7% against 17.6% and 18.8% mean reduction).

How does the method perform when trained on a wider range of angles and anisotropy ratios, rather than the narrow band where default AMG fails?

We ran this. Over 0 to 90 degrees and ratios 10 to 10^4, solver-aligned training still beats Rayleigh training, by about one iteration, mostly on the strongly anisotropic operators. But a second AMG cycle is clearly better (8.33 against 10.65 iterations), and even the ideal target does not beat it. This is now in the abstract, Section 4 and Appendix F.2.

Wider range (24 operators)Mean iterationsMean reduction
One AMG cycle (no correction)12.53
Rayleigh network11.684.4%
Exact Rayleigh space11.355.8%
Solver-aligned network10.659.0%
Finite solver target (ideal)10.2612.7%
Two AMG cycles8.3333.9%

It is often hard to tell which result supports which claim.

The introduction now points to a single scorecard table (Table 5) that lists, for each setting, whether the target, prediction and cost requirements hold, including all new studies. New studies are in one appendix section, and earlier table numbers are unchanged.

Why does the solver-aligned objective not help in 3D [...]

With the 64 training operators used in the paper, we do not know. The target itself is useful, but the solver-aligned network did not beat the Rayleigh network (18.81 against 18.04). Our best guess was overfitting, so we trained both objectives on the same 200 operators (three seeds each, only the loss changed) and tested on three fresh suites.

3D, 200 training operatorsSuite 1Suite 2Suite 3 (untouched)
One GMG cycle (no correction)26.2324.5423.15
Nearest stored target21.9518.74
Rayleigh network18.6517.7216.16
Two GMG cycles18.4117.2016.19
Exact Rayleigh space15.69
Solver-aligned network16.8916.8515.42
Classical AMG8.138.057.84

Blank cells were not run on that suite. In words:

  • First fresh suite: solver-aligned training beats Rayleigh training by 1.76 iterations and also beats two GMG cycles.
  • Second fresh suite: the lead is 0.88 iterations; the win over two cycles does not repeat.
  • Third, untouched confirmation suite: the lead is 0.75 iterations, but the uncertainty interval includes zero, so the confirmation fails. Here it ties the exact Rayleigh space and two GMG cycles.

On all suites the network clearly beats reusing a stored target from the most similar training geometry (by about 3 to 5 iterations), so in 3D prediction beats lookup by far more than in 2D. Classical AMG still needs about half as many iterations as any GMG-based method here, so this does not change the cost conclusion. The revised abstract says solver-aligned training "leads Rayleigh training by about one iteration, but the lead is not confirmed on an untouched suite" (Appendix F.4).

The positive prediction result does not hold [...] on public data.

Correct. Only the Rayleigh network was tested on public geometries, and it does not improve its base on average. We make no positive public-data claim.

The main new element is distilling this target into an FNO, which is a modest contribution.

We agree the target construction and the optimal-space result are known. What we add is the controlled test of whether the target matters for a learned predictor, with architecture, rank, data and deployment fixed. It does in 2D, including against the exact optimum of the competing loss. Over the wider range and in 3D the effect is smaller, and the revised paper says so.

Reviewer_VYASOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies which coarse space to learn when adding a low-rank correction to an existing multigrid preconditioner. It compares Rayleigh training, which selects low-energy operator modes, with supervision from slow error modes of the deployed preconditioner. On a 2D rotated-anisotropy family, the latter increases mean per-operator iteration reduction from 17.6% to 31.7%, outperforming even the exact Rayleigh space. Crossed-preconditioner and U-Net experiments support the role of the training target. The paper also documents limits: prediction generalizes poorly beyond this family, does not recover the full target benefit in 3D, and does not beat BoomerAMG in total time.

Critical strengths
  • The comparison with the exact Rayleigh space is particularly convincing. The solver-aligned model beats the optimum of the competing objective, so the difference cannot be explained solely by poor optimization of the Rayleigh network. Crossing targets between preconditioners and repeating the objective comparison with a U-Net provide further support for the same conclusion.
  • The classical controls substantially improve the value of the study. Extra V-cycles, target reuse, BoomerAMG, flop counts, and total time expose limitations that iteration counts alone would miss. The unsuccessful lognormal and 3D experiments are also useful: they show that finding a good target and learning to predict it are separate difficulties.
Critical weaknesses
  • The contribution beyond established coarse-space ideas is fairly limited, and the strongest empirical result has a narrow scope. The optimal-space result is known, relaxed random vectors are standard in adaptive/bootstrap AMG, and predicting coarse spaces has close precedents in NeurKItt and NeuraLSP. The main advance is therefore the controlled objective comparison. However, its positive prediction result is confined to spatially constant tensors in a selected angle band, effectively a two-parameter family. It does not carry over to the lognormal family or to the Large 3D models, where Rayleigh training performs better. This leaves limited evidence for how broadly changing the objective improves a learned predictor, beyond demonstrating the known dependence of the target on the preconditioner.
  • The computational case for using a neural predictor is not established. In Table 7, FNO needs 20.18 iterations, compared with 20.46 for four-neighbor interpolation and 19.97 for two V-cycles. In Table 11, its total time at 64 right-hand sides is 14.255 s versus 14.357 s for nearest-target reuse, without a resolved timing advantage. The model is smaller than the target library, but the paper does not identify a regime where this storage benefit gives neural prediction a clear overall advantage. Avoiding 2,048 V-cycle applications for online target construction is appealing, yet that comparison uses plain block iteration; accelerated construction and Krylov recycling remain untested. Individual timings for inference, QR, and coarse-system construction would also help explain the deployment cost. These gaps matter because the correction costs more flops than two V-cycles and remains slower than BoomerAMG.
Questions for authors
  • How much time is spent on network inference, QR, and coarse-matrix construction/factorization separately? At what repeated-solve budget does the full learned method become faster than target reuse or interpolation, if at all? The total-time comparisons do not currently establish this advantage.
  • Have you compared prediction with a compact surrogate using angle and anisotropy ratio, or a rank-32 basis extracted from the stored solver-target spaces? These would complement the existing four-neighbor interpolation and solution-snapshot POD controls, and help assess whether this two-parameter family benefits from a neural predictor.
  • Do you have evidence that the objective swap helps on a second nontrivial SPD operator family? The lognormal results are negative, and Table 18 reports an elasticity target screen but no trained predictor. As additional validation, a comparison on spatially varying anisotropic diffusion or linear elasticity would help establish whether the benefit extends beyond the selected two-parameter family.
2 comments

Thank you for the careful review. We agree with your reading that the main contribution is the controlled objective comparison, and we have uploaded a revised PDF with four new studies in Appendix F. Below we quote each point and answer it directly.

How much time is spent on network inference, QR, and coarse-matrix construction/factorization separately?

Setup takes about 348 ms per operator. Of this, 271 ms is the AMG hierarchy, which every method builds. The correction adds 77 ms: 29 ms network inference, 3 ms QR, 36 ms building the coarse problem, and 9 ms preparing the input. Only inference and input preparation (38 ms) are specific to the network; the rest is also paid by target reuse.

At what repeated-solve budget does the full learned method become faster than target reuse or interpolation, if at all?

We did not find one. Nearest-target reuse is faster at 1 and 8 right-hand sides (by about 60 to 80 ms), and at 32 and 64 the difference is within noise. The submission called this "not separated"; the revised paper now states it more precisely and less favorably to us.

Have you compared prediction with a compact surrogate using angle and anisotropy ratio, or a rank-32 basis extracted from the stored solver-target spaces?

Yes, on 24 fresh operators. A quadratic ridge regression on (angle, log anisotropy ratio) comes within about half an iteration of the network (20.33 against 19.81), interpolation within 0.4, and nearest reuse within 0.6. A single stored rank-32 basis is clearly worse (21.89). So inside this two-parameter family the network adds little beyond interpolation, and we now say so in the paper. Our claim is about which target to train on, not that a network is needed for this family.

Method (24 fresh operators)Mean iterations
One AMG cycle (no correction)29.02
Single stored rank-32 basis21.89
Nearest stored target20.41
Quadratic map, no network20.33
Four-neighbor interpolation20.18
Two AMG cycles19.94
Solver-aligned network19.81

the paper does not identify a regime where this storage benefit gives neural prediction a clear overall advantage.

We agree. The network checkpoint (18 MiB) is about 88 times smaller than the stored target library, but the quadratic map is also compact (a 32 MiB basis), so storage alone does not make the case. The revised paper now says this explicitly.

its positive prediction result is confined to spatially constant tensors in a selected angle band, effectively a two-parameter family.

We retrained both objectives over the full range of angles (0 to 90 degrees) and anisotropy ratios (10 to 10^4). Solver-aligned training still beats Rayleigh training and has a lower mean than the exact Rayleigh space, but the margin shrinks: the mean per-operator iteration reduction is 9.0% against 4.4%, compared with 31.7% against 17.6% in the original band. Most of the gain comes from the strongly anisotropic operators. Over this wider range a second AMG cycle is better than any rank-32 correction we tried, learned or exact (8.33 iterations against 10.65 for the network). This is now in the abstract, Section 4 and Appendix F.2.

Wider range (24 operators)Mean iterationsMean reduction
One AMG cycle (no correction)12.53
Rayleigh network11.684.4%
Exact Rayleigh space11.355.8%
Solver-aligned network10.659.0%
Finite solver target (ideal)10.2612.7%
Two AMG cycles8.3333.9%

It does not carry over [...] to the Large 3D models, where Rayleigh training performs better.

With 64 training operators this is correct. We then trained both objectives on the same 200 operators and tested on three fresh suites. Solver-aligned training led on all three (by 1.76, 0.88 and 0.75 iterations), but on the last, untouched suite the difference is not statistically clear. There, the network also beats stored-target reuse by about 3 iterations, while it only ties the exact Rayleigh space and two GMG cycles. We therefore report a small, unconfirmed 3D effect (Appendix F.4).

Do you have evidence that the objective swap helps on a second nontrivial SPD operator family?

No. We also need to correct the paper here. The lognormal family was labeled "contrast 10^4", but that is the generator's parameter; the actual max/min coefficient ratio is 10^10 to 10^16, which is an extreme case. We therefore ran a new matched study at moderate contrast (10^2.7 to 10^4). It is negative: solver-aligned training is about one iteration worse than Rayleigh training (8.35 against 7.28), and even the ideal target only ties two AMG cycles. Two of the three solver-aligned models ran out of their preset training time, so the study is incomplete; we report it as is. The revised paper corrects the contrast label and adds this result (Appendix F.3).

Moderate contrast (24 operators)Mean iterations
One AMG cycle (no correction)9.76
Default AMG8.92
Solver-aligned network (one pair)8.35
Rayleigh network7.28
Exact Rayleigh space7.14
Finite solver target (ideal)6.58
Two AMG cycles6.56

In summary, the revised paper presents this work as a diagnostic study, not a faster solver: the training target matters clearly in one 2D family and less elsewhere, prediction works only inside the training range, and classical solvers remain faster.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.