acceptodds
Under review as a conference paper at ICLR 2027

Alioth: Neural Modulators for Scientific Computing

10 views

Abstract

Scientific computing underpins modern science and engineering, and solving partial differential equations (PDEs) lies at its core. For decades, the field has pursued PDE solvers that are both accurate and generalizable. However, existing approaches rarely achieve both. Classical numerical algorithms guarantee accuracy but solve each problem anew, whereas neural operators generalize but cannot guarantee accuracy. More recently, neural preconditioners embed learning into numerical solvers but typically train for each problem or problem family. In this work, we introduce the neural modulator, a single pretrained network that generates solver components across problems and solvers. We present Alioth, a neural modulator that generates coarse spaces, the solver components that capture slow-to-converge error modes. Coarse spaces are transferable, as these modes follow local operator structure shared across PDEs, and error-tolerant, as imperfect ones only slow convergence. Pretrained once on synthetic diffusion problems with a label-free spectral objective, Alioth serves domain decomposition and algebraic multigrid, the two workhorse solvers for large-scale PDEs. With domain decomposition on 13 scalar SPD systems spanning reservoir simulation, heat transfer, fluid dynamics, structural mechanics, and electromagnetics, it needs 48% fewer iterations than the classical Nicolaides coarse space and 20% fewer than unlearned subspace iteration with the same number of matrix products. Because accuracy is set by the solver, five iterations on Darcy flow reach errors over 100× below neural operators and PDE foundation models, in and out of their training distribution. These results point to a new direction for AI-enabled scientific computing: learn the solver, not the solution.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI for Science, Scientific Machine Learning, Partial Differential Equations, Neural-Numerical Hybrid Solvers, Learned Preconditioners

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_wxKUOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Alioth, based on the idea of a “neural modulator”: rather than learning the PDE solution directly, it learns reusable components of a conventional numerical solver. Specifically, Alioth predicts coarse spaces capturing slow-to-converge error modes for domain decomposition (DD) and algebraic multigrid (AMG). The key motivation is that imperfect learned coarse spaces may slow convergence but do not determine the final attainable accuracy; residual-controlled numerical iteration still provides the accuracy guarantee.

Critical strengths
  1. A simple and compelling conceptual contribution.
  2. Strong zero-shot transfer results.
Critical weaknesses
  1. The strongest practical claim is weakened considerably by the end-to-end runtime. The learned encoder itself is reasonably inexpensive, but the complete Alioth prototype remains substantially slower than a production solver. The paper reports 17.3–154.4 s for Alioth versus 1.0–4.7 s for hypre BoomerAMG, with graph partitioning accounting for 42–82% of the Alioth runtime. Even excluding partitioning, the prototype remains 6–14× slower. Thus, the iteration-count improvements do not currently translate into an end-to-end computational advantage. The authors acknowledge this limitation clearly, but I think it is the largest practical weakness of the work.
  2. The demonstrated generality is narrower than the high-level “across PDEs and solvers” framing suggests. All primary evaluation systems are scalar SPD systems, while vector-valued systems, nonsymmetric systems, and indefinite systems remain untested. More importantly, the second solver family, AMG, is not yet a convincing success case: Alioth is approximately on par with standard smoothed aggregation overall (geometric-mean iteration ratio 1.08), and it can degrade substantially—for apache2, the iteration count increases from 11 to 91. The paper itself attributes this to difficulties in assembling local modes into global AMG candidates. Thus, the strongest evidence currently supports transferable learned coarse spaces for domain decomposition rather than a generally effective neural modulator across numerical solvers.
Questions for authors

Could you please explain or address the weaknesses raised above?

Reviewer_VRSUOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces “neural modulators”: networks pretrained once to generate solver components instead of solutions. Alioth is one such network. It takes the local blocks of a sparse SPD matrix and produces approximate slow eigenvectors for each subdomain. Fixed assembly heads turn these into a coarse space for domain decomposition or an extra candidate vector for smoothed-aggregation AMG. The network is small, trained without labels on a handful of synthetic diffusion matrices, and applied zero-shot to 13 real SPD systems. The paper reports fewer PCG iterations than the Nicolaides coarse space and than unlearned subspace iteration at the same matvec budget, parity with smoothed aggregation for AMG, and much lower error than neural operators on Darcy flow after a few iterations.

Critical strengths

The experiments are controlled and honestly reported. All domain decomposition variants share the same partition, local solves, and assembly, so the coarse vectors are the only difference. The appendices openly list where the implementation departs from the theory, where AMG and runtime fall short, and a flaw in the Darcy data generator. The label-free training objective is sensible, and the reported numbers are internally consistent.

Critical weaknesses
  1. The learned part is not shown to matter. The only comparison that isolates learning is Alioth versus unlearned subspace iteration at the same budget, where Alioth wins by about 20%. But simply running the unlearned method twice as long ties Alioth. So the network is worth about one doubling of a cheap classical computation. That baseline is also a weak eigensolver. Standard alternatives such as Chebyshev-filtered subspace iteration or LOBPCG would likely close the gap, and none are tried. The budget comparison is not even equal, because the extra matrix products inside the MLP are not counted. There is no ablation of the network’s components. After reading the paper, I cannot tell whether learning contributes anything beyond what a good classical eigensolver gives at the same cost.

  2. The method is not shown to be useful. For AMG it is a net loss: slightly more iterations at 2.5 times the operator complexity, and a severe regression on one system. For domain decomposition, iterations drop, but no timings are given, and the coarse problem is nine times larger than the baseline’s. Exact local eigenspaces, a known classical method, remain far better on the hard high-contrast systems. End to end, the full pipeline is 16 to 33 times slower than hypre, and still 6 to 14 times slower with partitioning excluded. Nothing in the paper shows a solver that gets faster because of Alioth.

  3. The headline results come from classical numerics, not from Alioth. The “over 100x below neural operators” claim is matched by plain smoothed aggregation with no learning. It shows that iterative solvers beat one-shot surrogates, which is already known, and it ignores cost entirely. The 3.8x in Figure 1 is the best single case against the weakest baseline; the average is under 2x. The Darcy “out-of-distribution” families are in Alioth’s own pretraining data.

  4. Transfer and novelty are overstated. The “five application domains” are dataset labels. All 13 systems are scalar, diffusion-like SPD matrices, close to the pretraining data. The largest gains are on Poisson problems, which contradicts the text’s claim that gains are largest beyond diffusion. Learned coarse spaces and learned AMG components already exist; the closest work (https://arxiv.org/abs/2607.06261, Klawonn et al. 2026) is cited but not compared. “Neural modulator” is mostly a new name. The propositions are standard results and, as the appendix admits, do not apply to the implementation. Training uses seven matrices per seed, and variance across probe draws and right-hand sides is not reported.

Questions for authors
  1. Does Alioth beat Chebyshev-filtered subspace iteration or LOBPCG at a truly equal cost, counting all matrix products?

  2. Is there any end-to-end setting, measured in wall-clock time, where Alioth makes a solver faster than its classical counterpart?

  3. How does it compare with existing learned coarse spaces, in particular Klawonn et al. (2026)?

  4. What does the Darcy neural-operator comparison show about Alioth specifically, given that smoothed aggregation alone reaches the same accuracy?

  5. Have you tested any system that is not a scalar diffusion-like matrix?

Reviewer_3nf4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Alioth introduces a neural modulator that learns numerical-solver components rather than directly predicting PDE solutions. A pretrained encoder processes local matrix pencils using learned matrix-polynomial filters and pointwise channel mixing to generate low-energy local modes. These modes are orthogonalized, ranked by Rayleigh quotients, and deterministically assembled into coarse spaces for domain decomposition or algebraic multigrid. The model is pretrained without solution or eigenvector labels by minimizing a weighted Rayleigh-quotient objective, encouraging its outputs to approximate the lowest-energy local eigenspaces. The network parameters are fixed after pretraining, while each new linear system produces its own coarse space from its local matrices, allowing the same encoder to be reused across problems.

Critical strengths

A well-motivated solver-level learning paradigm. Alioth learns reusable coarse-space components instead of directly predicting PDE solutions, preserving the accuracy control of iterative numerical solvers while reducing problem-specific setup. This addresses a meaningful gap between generalizable neural models and accurate but problem-specific numerical methods.

Evidence of zero-shot transfer in a controlled setting. A small, label-free encoder pretrained only on synthetic scalar diffusion systems improves domain-decomposition convergence on 13 unseen scalar SPD systems, reducing iterations by 48% versus the Nicolaides coarse space and by 20% versus equal-budget unlearned subspace iteration. These results suggest that the learned solver component transfers across problem instances and provides benefits beyond simply increasing the computational budget.

Critical weaknesses

The paper does not establish an end-to-end speedup. The main improvement is reported as a reduction in PCG iteration counts, but the paper does not provide a complete timing comparison for the 13 domain-decomposition systems. Alioth also incurs network, coarse-space construction, larger coarse solves, and partitioning costs. In the available end-to-end comparison, the prototype is 16–33× slower than hypre BoomerAMG. This is consequential because the paper’s practical motivation is to accelerate PDE solving, while the current results mainly demonstrate iteration-count improvements rather than wall-clock improvements.

The evidence for broad zero-shot generalization is limited. All evaluation systems are scalar SPD problems, and pretraining uses only scalar diffusion matrices. Non-symmetric, indefinite, vector-valued, and more general PDE systems are not tested. Moreover, Alioth is only comparable to or slightly worse than smoothed aggregation in AMG, and stronger adaptive coarse-space baselines are not evaluated across the full benchmark suite. Consequently, the experiments support transfer within a relatively narrow problem class but do not yet justify broad claims about reusable solver components for general PDEs.

Questions for authors

Core question — end-to-end efficiency. What are the complete end-to-end setup and solve times for Alioth, Nicolaides, and SubIter on all 13 domain-decomposition systems, including partitioning, network inference, coarse-space construction, coarse solves, and PCG iterations? The paper’s main efficiency claims are based on iteration counts, while the available wall-clock comparison is mainly against hypre BoomerAMG on four systems.

Provenance and significance of the SPD assumption. All 13 main evaluation systems are treated as scalar SPD problems. For each benchmark source—including SPE10, the six SuiteSparse matrices, Poisson, and Darcy flow—can the authors identify the originating PDEs and the discretization or boundary-condition assumptions that lead to an SPD system, rather than providing only application labels? More broadly, what evidence supports treating scalar SPD systems as a central and representative class for the PDE workloads targeted by the paper, given that many important PDE discretizations produce nonsymmetric, indefinite, saddle-point, or vector-valued systems?

Training-data diversity and scale. The paper reports seven training matrices per seed and three pretraining seeds, but it is unclear whether these matrices are distinct across seeds and how much overlap exists in their coefficient distributions or mesh structures. How do the authors justify that this training set is sufficiently diverse to support the reported zero-shot claims? Do training-set-size or training-distribution ablations show how transfer performance changes with more diverse data?

Reviewer_qXeEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Proposes Alioth, a pretrained neural modulator that generates local coarse-space modes for iterative PDE solvers. Learns from synthetic diffusion via a label-free spectral objective, then transfers to domain decomposition and algebraic multigrid. On 13 scalar SPD systems, the domain-decomposition variant reports 48% fewer iterations than a classical coarse space, while retaining iterative-solver accuracy.

Critical strengths
  1. Learning a reusable solver component, rather than a one-shot solution map, is a compelling and practically relevant research direction.
  2. The spectral training objective and convergence argument connect the neural component to established numerical linear algebra (Propositions 1–3).
  3. Cross-domain SPD tests and matched subspace-iteration ablations support genuine gains from learning, rather than merely spending more matrix products (Table 1; Figure 4).
Critical weaknesses
  1. End-to-end speed is not yet competitive: the prototype takes 17.3–154.4 s versus 1.0–4.7 s for hypre BoomerAMG on four timed cases (Section 5.5).
  2. The algebraic-multigrid head is generally on par with or worse than smoothed aggregation and substantially degrades apache2 (Table 1).
  3. Evidence covers scalar SPD problems; vector-valued systems and nonsymmetric/indefinite operators are outside the current validatio
Questions for authors
  1. What is the expected end-to-end speedup after replacing the Python partitioning/setup stages with optimized implementations?
  2. Can the authors explain and mitigate the AMG regression on apache2 (91 versus 11 iterations) without retraining for that case?
  3. How does Alioth compare, at matched total setup-plus-solve cost, with adaptive AMG or algebraic GenEO/GDSW-style coarse spaces across the full suite?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.