acceptodds
Under review as a conference paper at ICLR 2027

Unrolled Optimized Restricted Additive Schwarz Methods

Abstract

Large linear systems from PDE discretizations are typically solved iteratively, and their runtime depends heavily on the quality of the preconditioner. Domain-decomposition preconditioners are attractive because they split the global problem into parallel local solves, but their convergence depends on how information is exchanged across subdomain interfaces. Classical optimized Schwarz methods prescribe these interface parameters analytically, using Fourier minimax arguments that are exact only under idealized assumptions. We introduce UORAS, an unrolled optimized Schwarz preconditioner that keeps the classical analytical parameter rule as a backbone and learns only a multiplicative correction. We also introduce a residual-conditioned non-stationary variant that learns an iteration-dependent schedule through a shared neural network. On the theory side, we show that minimax tuning can be strictly suboptimal for finite-depth distributional losses with positive overlap, and we connect this gap to stationary and residual-conditioned UORAS hypothesis classes. Empirically, UORAS improves on classical optimized Schwarz baselines and is designed to retain the benefits of learning-based interface models with far fewer parameters and improved scalability.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Iterative solvers, algorithm unrolling, optimized schwarz methods

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_sGiZOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper show that learning transmission parameters (p,q) in Optimized Restricted Additive Schwartz algorithm can improve the convergence. ORAS is an important linear solver where preconditioning is used in each subdomains, and the control parameters between the domains are critical for optimizing the main (joined) task. The authors show that learning those parameters outperforms the minimax-tuned approach.

Critical strengths

Algorithm unrolling is a strong tool to combine the strength of iterative algorithms with learnable units. The author showed that even a simple learnable unit replacing a sub-optimimal parameter is a worthy cause. In practice they show improved results on a known and important algorithms widely used.

Critical weaknesses

Usually there is a price to pay when w learn very few parameters (2 here) in poor generalization. A small number of experiments was done in a similar setup of modified Helmholtz Equation on a square. This approach might not generalize well to other equations on one hand, or might be irrelevant at all for a general linear equation seen for the first time.

Questions for authors

Did you consider changing p and q during runtime and not keeping them fixed (non-stationary) ? Do you have additional results to share? Did you try to learn p,q from one domain and transfer to another? (show generalization)

3 comments

"Did you consider changing p and q during runtime and not keeping them fixed (non-stationary)" --- Yes, this is our main variant. UORAS predicts new corrections to pp and qq at every iteration from the current residual. Fig.1b shows how they evolve.

"(show generalization)" Because UORAS only corrects the analytical OO2 parameters, the small number of learned parameters helps generalization. The same model transfers to grids that are more than 10 times larger than the training grids and also across different numbers of subdomains J.

“Other equations.” We added the indefinite Helmholtz equation (Section 6.2 in the revised uploaded manuscript), where UORAS reduces the iterations of OO2 by 40–60%, and diffusion with discontinuous coefficients (Appendix H.2). UORAS is not a black-box solver for linear systems: like OO2, it requires analytical optimized parameters, which are available for many PDEs of practical interest.

We noticed that the score was lowered from 3 to 2. We do not see a comment on this change and would appreciate knowing what concerns remain before the discussion period ends.

Usually there is a price to pay when w learn very few parameters (2 here) in poor generalization.

We also added an ablation on the MLP architecture (Appendix H.5). Note that UORAS does not learn two fixed parameters: an MLP predicts the two correction coefficients from the problem features and the current residual, so they adapt to each problem and iteration. The ablation includes a baseline that learns two fixed coefficients, and it shows that this baseline generalizes poorly to larger grids, whereas MLP with 2 hidden layers consistently improves upon OO2.

We wanted to follow up on our previous comment, as the discussion period ends soon. We would be grateful for feedback on our responses and the additional experiments, in particular regarding the concerns that led to the change in score.

Reviewer_zG4JOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The manuscript unrolled optimized restricted additive Schwarz methods proposes UORAS, a learning-based Schwarz method for linear system of equations arising from PDEs. The method is based on optimized restricted additive Schwarz where a learnt multiplicative correction is applied to the uniformly optimized choice (OO2). The authors proof that under certain assumption for the modified Helmholtz model UORAS could improve over OO2. Experiments show that a trained method gains better results.

Critical strengths
  1. The paper is well organized, and the presentation is clear.

  2. The algorithm is partially supported by theory.

Critical weaknesses
  1. The scope of the paper is too narrow. Focusing only on the modified Helmholtz model, and in the experiments the PDE parameter is fixed.

  2. Also the modified Helmholtz problem is too simple. The shift makes the problem more positive definite than Poisson problem and AMG is known to be near optimal for this problem. To make the algorithm more useful and convincing, at least some nonsymmetric/indefinite problems should be discussed or tested.

  3. The performance seems to depends on the distribution of the training initial guesses and testing initial guess. Discussion on this is somewhat missing.

Questions for authors
  1. Some guidance on selecting the size of the MLP would be helpful.

  2. More discussion/experiments with different training/testing distributions would be helpful.

  3. I don’t think time allows. But if possible, at least test varying eta, and some ill-conditioned/indefinite problems. Otherwise the paper is not convincing to me.

3 comments

We respectfully disagree that the modified Helmholtz equation is a too simple test case. It is the model problem from which the OO2 parameters were derived (Gander, 2006), and the benchmark of prior work on learned optimized Schwarz methods at NeurIPS (Taghibakhshi et al., 2022). The fact that AMG performs well on it does not make it less relevant for domain decomposition.

Nevertheless we agree that indefinite problems make the case more convincing and have added them. In both, the PDE parameter varies across problems. We also ran an ablation to see how performance varies as a function of distributions of initial training and testing initial guesses. We keep it short here as the details are in the uploaded revisited manuscript.

For indefinite Helmholtz, Section 6.2, we get complex symmetric, indefinite system matrix A and complex-valued u. Here UORAS reduces the iteration count of OO2 by around 40-60%. The model is trained on grids with 4⋅1024\cdot10^2–1.2⋅1041.2\cdot10^4 DoFs, validated on 1.41.4–2.6⋅1042.6\cdot10^4 DoFs, and tested on grids with up to 5⋅1055\cdot10^5 DoFs. We also plot the convergence on a single problem: UORAS needs 23 iterations, compared to 52 for OO2. Runs on grids with up to 10610^6 DoFs are in progress.

For diffusion with discontinuous coefficients, Appendix H2, the coefficient alternates between 11 and 1/μ1/\mu across 1010 subdomains, with μ∈{5,10,20}\mu\in\{5,10,20\}. Here UORAS outperforms OO2 or performs on par with it for all μ\mu, with the largest gains, on small and medium grids. As before, the model is trained on grids with 4⋅1024\cdot10^2–1.2⋅1041.2\cdot10^4 DoFs, validated on 1.41.4–2.5⋅1042.5\cdot10^4 DoFs, and tested on grids with up to 10610^6 DoFs.

We trained models on normal and on uniform initial guesses and tested each on both, Appendix H3. For a given model, the test distribution has a negligible effect on performance.

We also added an ablation study on the MLP architecture in Appendix H.5 to address question 1 and provide guidance on selecting the MLP size. As the discussion period is coming to an end, we would greatly appreciate your feedback on our responses and the additional experiments. If they address your concerns, we would be grateful if you would consider updating your score.

We wanted to follow up on our previous comment, as the discussion period ends soon. We would be grateful for feedback on our responses and the additional experiments.

Reviewer_Dk9tOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose UORAS, a learned optimized Schwarz preconditioner. It keeps the analytical OO2 transmission parameters as a backbone and uses an MLP (∼2300\sim 2300 parameters) to learn a multiplicative correction to them, trained without supervision by unrolling preconditioned iterations; a non-stationary variant conditions this correction on the residual norm. On the theory side, the authors show that minimax-tuned parameters can be suboptimal for finite-depth expected-error losses and that, under certain assumptions, UORAS can improve on classical OSM. Experiments on 2D modified Helmholtz show fewer iterations than OO2 for many subdomains (J=10,20J = 10, 20) and better scaling than MLORAS on small two-subdomain problems.

Critical strengths
  1. UORAS learns only a small correction to OO2 (∼2,300\sim 2,300 parameters vs. ∼1.3\sim 1.3M for MLORAS) and starts exactly at OO2. Since it predicts just two scalars, it extrapolates to much larger grids and more subdomains. This is the paper’s main contribution: an interpretable and scalable alternative to fully learned interface operators.
  2. The two-mode counterexample (Proposition 1) shows explicitly that minimax-tuned transmission parameters are not optimal for a finite-depth expected error objective, even for uniform mode weights once overlap is present. The bounded-subdomain analysis (Proposition 4) identifies a second source of mismatch in the OO2 derivation. This matters because it provides a reason, beyond the empirical results, why a learned correction can help, and it frames the method as a principled extension of OSM.
Critical weaknesses
  1. The gains over OO2 are modest and noisy. For example, with J=10J = 10 the non-stationary variant needs 31.38±8.2631.38 \pm 8.26 iterations versus 33.29±1.5533.29 \pm 1.55 for OO2. The stationary variant is worse than OO2 in Richardson for every JJ. More importantly, the non-stationary variant changes the transmission parameters at every iteration, which requires refactorizing the local subdomain matrices each time. The paper reports no wall-clock times, so it is unclear whether the method is faster at all. This is decisive, because the paper’s main claim is improved solver performance.
  2. The paper’s main motivation is that OO2 is used outside its regime of validity, yet all experiments use the setting closest to that regime: 2D modified Helmholtz with η=1\eta = 1 on the unit square. Although η\eta is a network input, it is never varied. The cases where a learned correction should matter most are not tested, such as heterogeneous coefficients, advection-diffusion, or indefinite Helmholtz. On the comparison side, MLORAS is evaluated only for J=2J = 2, small grids and Richardson, rather than in its original GMRES setting with more subdomains. There is also no comparison against per-problem numerically optimized (p,q)(p, q) or standard preconditioners such as AMG or two-level ORAS. As a result, it is unclear whether the learned correction adds value beyond simple tuning, or whether the approach generalizes beyond the easiest case.
Questions for authors

If the authors address the concerns raised in the Weaknesses and support their answers with experiments, I will raise my score.

5 comments

"Runtime comparison." We added a runtime comparison on the indefinite Helmholtz equation in Appendix H.4. With variable transmission parameters, one indeed cannot directly reuse the local subdomain matrices across iterations. However, they can be reused as long as the parameters do not change significantly, and the computed matrices can be stored and reused for subsequent error equations with the same system matrix A. This allows us to reduce computation cost and runtime. To keep it brief (details are in the updated manuscript), the UORAS needs about half as many iterations as OO2 and solves 5 error equations faster than OO2 on grids with up to 3.8⋅1053.8\cdot10^5 DoFs. The runtime gains in absolute scale grow with the grid size.

"Gains are grid-dependent, not noisy". The large standard deviations in Table 1 come from averaging over all test grid sizes. Fig. 1a shows the results per grid: UORAS(10) needs around 40% fewer iterations than OO2 on small and medium grids, and the gap closes only on the largest grids, far outside the training range. With larger grids in the training set, UORAS(10) remains better than OO2 up to 10610^6 DoFs. The stationary variant needs fewer iterations than OO2 on grids up to about 10510^5 DoFs, and requires more iterations on larger grids, which dominate the average in Table 1. As a GMRES preconditioner, stationary UORAS improves on OO2 for J=10 and J=20, Table 2.

"More complex problems." We added two of the cases the reviewer names:

Indefinite Helmholtz (Section 6.2): here η=−ω2\eta=-\omega^2 varies across problems, and a single model covers all of them. UORAS needs around half as many iterations as OO2 on grids up to 5⋅1055\cdot10^5 DoFs. We are currently evaluating this experiment for grids up to 10610^6 DoFs.

Diffusion with discontinuous coefficients (Appendix H.2): UORAS outperforms OO2 or performs on par with it on grids up to 10610^6 DoFs.

"Comparison against numerically optimized coefficients." Numerically optimizing (p,q)(p,q) for each problem requires repeated solves of that problem, which costs more than solving it once. For non-stationary parameters this is not feasible at all, since the search space grows with the number of iterations and depends on the current residual. UORAS learns this mapping once and applies it to unseen problems at the cost of a small MLP evaluation per iteration.

"Comparison against MLORAS and AMG." MLORAS predicts a number of parameters that grows with the problem size. In its original work, it was trained on grids with 90...850 nodes and tested on grids with up to 40⋅10340\cdot 10^3 nodes (Taghibakhshi et al., 2022), whereas our test grids reach 10610^6 DoFs. AMG and two-level methods improve the coarse space, while UORAS improves the transmission conditions. Combining UORAS with a coarse correction is future work.

We also added an ablation study on the MLP architecture in Appendix H.5. The “Constant” baseline, a single pair of correction coefficients tuned on the training grids, is worse than OO2 on the larger test grids, which relates to your question on simple tuning.

As the discussion period is coming to an end, we would be grateful to know whether our responses and additional experiments address your concerns. If so, we would be happy if you would consider raising your score as you indicated.

We wanted to follow up on our previous comment, as the discussion period ends soon. We would be grateful for feedback on our responses and the additional experiments.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

Dear Area Chair and Reviewers,

As the discussion period ends, we summarize the revisions made in response to the concerns raised.

Scope and problem difficulty (all reviewers). We added two problems beyond the modified Helmholtz equation, using non-overlapping optimized Schwarz methods:

Indefinite Helmholtz (Section 6.2): UORAS needs 40...60% fewer iterations than OO2 on grids up to 1⋅1061\cdot10^6 DoFs. A single model covers all wavenumbers, which relates to the variable eta concern brought up by reviewers Dk9t and zG4J.

Diffusion with discontinuous coefficients (Appendix H.2): UORAS outperforms OO2 or performs on par with it on grids up to 10610^6 DoFs.

We also note that the modified Helmholtz equation is the model problem of the analysis of optimized Schwarz methods (Gander, 2006) and of prior work on learned optimized Schwarz methods (Taghibakhshi et al., NeurIPS 2022).

Wall-clock time, reviewer Dk9t. Variable transmission parameters do not allow UORAS to precompute the modified subdomain matrices. However, UORAS is still faster than OO2 baseline when we (i) reuse the modified subdomain matrices when the parameter change is < threshold and (ii) reuse the modified subdomian matrices across five right-hand sides. On the indefinite Helmholtz UORAS is 15...30% faster than OO2 with up to 3.8⋅1053.8\cdot10^5 DoFs (Appendix H.4).

Generalization and MLP size, reviewers zG4J and sGiZ. Our ablation (Appendix H.5) shows that MLPs with two hidden layers of width 32 to 128 reduce the iterations of OO2 by about 20%. This also gives guidance on the choice of the MLP size. We also compare two interpretable baselines and show they generalize poorly, further supporting the MLP approach we undertook.

Distributions of initial guesses reviewer zG4J. Models trained on normal and on uniform initial guesses perform nearly identically on both distributions (Appendix H.3).

Noisy gains, reviewer Dk9t. The large standard deviations in Table 1 come from averaging over all grid sizes. Per grid, UORAS needs up to 40% fewer iterations than OO2 (Fig. 1a). The stationary variant likewise improves on OO2 on grids up to 10510^5 DoFs, as well as when used as a GMRES preconditioner for J≥10J\geq10. Regarding the comparison with MLORAS: in its original work, MLORAS was trained on grids with at most 850 nodes, whereas our tests reach 10610^6 DoFs, which explains why we compare against it on small problems.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.