acceptodds
Under review as a conference paper at ICLR 2027

Closed-Loop Preconditioning for Anisotropic Diffusion via Reinforcement Learning

Abstract

Preconditioners are commonly held fixed throughout a linear solve, but the evolving residual provides feedback on the effectiveness of earlier corrections. We investigate whether learning to exploit this feedback can improve convergence on strongly anisotropic diffusion problems. Our Closed-Loop Preconditioner (CLP) uses reinforcement learning to incrementally update an explicit preconditioner within first-order Richardson iteration. A permutation-equivariant graph policy updates a diagonal-plus-low-rank preconditioner that is symmetric positive definite by construction, with analytic Galerkin weighting of the learned subspace. Controlled comparisons with learned static and open-loop policies support the benefit of solver feedback. CLP reduces iteration counts relative to algebraic multigrid-preconditioned Richardson and achieves shorter measured solver times than the tested classical baselines. The policy transfers from short training trajectories on small systems to substantially larger systems and supports longer solves through defect-correction cycles, without retraining.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Learned preconditioning, Iterative linear solvers, Reinforcement learning, Anisotropic diffusion, Graph neural networks

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_dKMqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The manuscript closed-loop preconditioning for anisotropic diffusion via reinforcement learning proposes CLP, a reinforcement-learning-based preconditioning framework for Symmetric Positive Definite (SPD) linear system of equations. The preconditioner is designed to maintain SPD and could update to adapt solver status. Experiments show that a trained policy can be generalized to longer solves and larger systems.

Critical strengths
  1. The paper is well organized, and the presentation is clear.
  2. The motivation behind the manuscript is reasonable and the algorithm design is convincing.
  3. The experiment design is well aligned with the claims.
Critical weaknesses
  1. The scope of the paper is too narrow, and thus I think the potential of this algorithm is somewhat limited. The paper only discusses the strategy for linear SPD systems with single right-hand side using Richardson iteration. It is unclear how this strategy could be extended to practical applications for serval reasons.

    • Cost may be too high. For large distributed machines, k extra MatVecs per update plus message passing would require significant communication, and may not be considered as “computationally tractable”.
    • Training is hard. Practical solvers are much more complicated than Richardson iteration. I am convinced that this strategy works for Richardson iteration, but I am not convinced that this method would work for many realistic solvers with much more parameters and considerations.
    • It is unclear how this method could be generalized to multiple right-hand sides cases (block solves).
  2. The update policy is somewhat local. I understand that U is trying to capture some global information, but I am not convinced that local message passing could provide good global basis vectors to capture smooth components in the error, which may also be the limitation of many GCN-based preconditioners.

  3. The baselines are not convincing. ILU factorizations in scipy like ILUT (based on supernodes) is different from standard textbook implementations. Also, pyAMG is not the best baseline. Those options are convenient but unfair as baselines, especially when you are also report timing.

  4. Theorem support is limited.

Questions for authors
  1. It is not too hard to compare with SOTA ILU and AMG with packages including hypre and PETSc. Adding those tests are easy for small problems considered in the paper and does not require a lot of CPU/GPU hours.

  2. It would be great if the authors could add some discussion on the potential of this algorithm to real applications: large scale problems with practical solvers on distributed memory systems. Extra MatVecs and message passing could cause significant overhead. More discussion/experiments on the cost would be great.

  3. How this method handles smooth components in the error? Could this policy be applied as a low-rank correction in addition to standard AMG? How this method work within CG/GMRES and/or with multiple right-hand sides (block solve)?

3 comments

Thank you for the detailed feedback. We address the concerns below with additional experiments and clarifications.

W3 / Q1: Added hypre BoomerAMG and PETSc ILU baselines

Thank you for suggesting comparisons with additional numerical libraries. Our original SciPy ILU and PyAMG baselines follow implementations used in prior learned-preconditioner evaluations, including Graph Neural Preconditioners (ICLR 2025). To strengthen the evaluation and assess whether CLP's convergence advantage persists against alternative classical preconditioners, we additionally evaluated PETSc ILU and hypre BoomerAMG on the main anisotropic-diffusion problem (n=256n=256).

All methods use first-order Richardson iteration with separately tuned step sizes, isolating the effect of the preconditioning strategy. Hypre BoomerAMG uses Falgout coarsening and classical interpolation, while PETSc ILU is evaluated across fill levels k∈0,1,2,3,4k\in{0,1,2,3,4}, with k=1k=1 selected for the reported comparison. Additional implementation details are provided in Appendix B.1. Median iterations to a relative residual of 10−810^{-8} are:

PreconditionerMedian iterations to 10−810^{-8} (↓)
hypre BoomerAMG1325
PETSc ILU2566
CLP (ours)754

The additional library implementations improve convergence relative to the original ILU and AMG baselines. Nevertheless, CLP retains the lowest iteration count, requiring 1.8× fewer iterations than hypre BoomerAMG and 3.4× fewer than PETSc ILU. These results strengthen the evidence that CLP's convergence advantage within first-order Richardson iteration persists against stronger classical preconditioners.

We have incorporated these comparisons and their implementation details into Appendix B.1 of the revised manuscript.

W1/Q2: Cost analysis, multi-GPU execution, and multiple right-hand sides

Thank you for highlighting the computational and communication costs associated with large-scale deployment. For sparse PDE operators with bounded sparsity, CLP has linear per-iteration work and storage complexity in the number of unknowns at fixed network size and correction rank. Its main additional costs are GNN inference and Galerkin updates, which require k=4k=4 sparse matrix-vector products per iteration. Distributed execution involves communication between neighboring partitions and global reductions for the low-rank correction. Although these operations do not require dense global matrices, their communication and synchronization costs remain important considerations for scaling.

To investigate these costs, we implemented a distributed-memory version of CLP that partitions each linear system across two NVIDIA RTX A6000 GPUs. Measured execution times for 100 iterations, including computation and inter-GPU communication, were:

Grid1 GPU2 GPUsSpeedup
96296^20.192 s0.297 s0.65×
2562256^20.626 s0.418 s1.50×
5122512^22.587 s1.676 s1.54×

The two-GPU implementation is slower at the smallest resolution, consistent with communication and coordination overhead, but achieves approximately 1.5× speedup on the larger systems. We additionally evaluated convergence to a relative residual tolerance of 10−810^{-8}, obtaining median end-to-end solve times of 3.98 s at n=2562n=256^2 and 23.36 s at n=5122n=512^2 on two GPUs. For reference, the tested CPU-based AMG-preconditioned Richardson implementation required 20.72 s and 98.08 s, respectively. These timings demonstrate the performance of the tested implementations on their respective hardware configurations, rather than a hardware-matched comparison.

You also asked about multiple right-hand sides (W1 and Q3). Using the same trained policy without retraining, independent solves can be batched on a single GPU. For 200 right-hand sides on a specific magnetized diffusion problem (n=256n=256), the average time per right-hand side to reach a relative residual tolerance of 10−610^{-6} was:

ImplementationAverage Time per RHS (ms) (↓)
Sequential CLP317
Batched CLP (GPU)17

Batched execution achieves an 18.6× improvement in throughput by exploiting GPU parallelism across independent solves. This experiment demonstrates efficient multiple-right-hand-side execution, although it does not implement a coupled block Krylov method or shared block correction space.

Together, these experiments establish the feasibility of partitioned CLP execution across two GPUs and demonstrate substantial throughput gains from batching independent solves. Scaling to larger distributed-memory systems and developing block-solver extensions remain open directions. We have incorporated these experiments and implementation details into Appendix B.8 of the revised manuscript.

W1/W2/Q3: Solvers beyond Richardson (CG), smooth error components, and combining the low-rank correction with AMG

Compatibility with accelerated solvers. To evaluate CLP beyond first-order Richardson, we retrained the policy separately within second-order Richardson and flexible CG, retaining the original architecture, preconditioner parametrization, and observations. The policy does not observe momentum, search directions, or Krylov-subspace information. On magnetized diffusion (n=256n=256), median iterations to a relative residual of 10−810^{-8} were:

Solver (with CLP)Median iterations to 10−810^{-8} (↓)
First-order Richardson754
Second-order Richardson697
Flexible CG520

Both accelerated configurations improve convergence relative to CLP with first-order Richardson. These preliminary results show that the CLP architecture can be trained within accelerated solvers without incorporating their additional dynamical information into the policy. Explicitly conditioning preconditioner updates on momentum and Krylov-subspace information remains a promising direction for further development.

Local message passing and smooth error components. Although the GNN aggregates information locally, its outputs define a globally supported low-rank basis, with correction directions coupled through the Galerkin weighting. Thus, the learned preconditioner can produce global corrections despite the local receptive field of the graph network. Moreover, residual feedback provides information about the evolving error, allowing the policy to adapt its correction as the solve progresses.

To examine whether CLP effectively reduces error components that are difficult for local relaxation methods, we decomposed the error into eigenvalue bands over 200 magnetized-diffusion instances (n=256n=256). Figure 8 in Appendix B.5 shows that CLP substantially attenuates error in the 10% of eigenmodes associated with the smallest eigenvalues, outperforming AMG-preconditioned Richardson in this band. It also reduces error effectively across the middle and largest-eigenvalue bands. These results show that CLP's advantage is not restricted to high-eigenvalue error components, supporting the effectiveness of its globally supported low-rank correction despite local message passing.

Integrating the low-rank correction with AMG. The CLP parametrization can also accommodate an existing SPD preconditioner. Replacing the diagonal component in Equation 4 with an SPD AMG preconditioner M−1M^{-1} gives

Pt−1=M−1+UtCtUt⊤,P_t^{-1}=M^{-1}+U_tC_tU_t^\top,

which remains SPD because the learned low-rank correction is positive semidefinite. AMG could provide a classical coarse-space correction, while CLP adapts to the remaining error components. We consider it a promising direction for future work.

The question on multiple right-hand sides is addressed in our response to Q2 above.

W4: Theoretical support for the CLP construction

Thank you for raising the question of theoretical support. We agree that it is important to distinguish the structural properties guaranteed by the CLP construction from convergence properties established empirically. The manuscript establishes several mathematical properties that hold independently of the learned policy parameters:

  • Positive definiteness and descent (Section 4.1). Pt−1≻0P_t^{-1} \succ 0 at every iteration because its diagonal component is strictly positive and its regularized low-rank correction is positive semidefinite. Consequently, Pt−1rt−1P_t^{-1} r_{t-1} is a descent direction for the quadratic energy 12x⊤Ax−b⊤x\tfrac12 x^\top A x - b^\top x whenever rt−1≠0r_{t-1} \neq 0.
  • Galerkin optimality and basis invariance (Section 4.1, Equation 6). When Ut⊤AUtU_t^\top A U_t is nonsingular, the unregularized Galerkin correction UtCtUt⊤rt−1U_t C_t U_t^\top r_{t-1} minimizes the quadratic energy over corrections in span(Ut)\mathrm{span}(U_t). It depends only on this subspace rather than on the particular basis.
  • Symmetries of the linear system (Section 4.3, Appendix B.6). The policy is permutation equivariant, ensuring that consistently relabelling the unknowns produces correspondingly relabelled preconditioner updates. The canonicalized observations are also invariant, up to numerical stabilization, under positive diagonal changes of variables A↦ΛAΛA \mapsto \Lambda A \Lambda, b↦Λbb \mapsto \Lambda b, x↦Λ−1xx \mapsto \Lambda^{-1} x for any positive diagonal Λ\Lambda.
  • Consistency of training and deployment (Sections 4.2 and 4.4). The logarithmic residual-reduction rewards telescope over the training episode, with the implemented numerical offset retained in the identity. During deployment, each defect-correction cycle satisfies r−Aet=b−A(x+et)r-Ae_t=b-A(x+e_t) (Equation 10), so the correction residual corresponds to the residual of the original linear system.

These properties provide structural guarantees for the learned preconditioner and establish the consistency of the training and deployment formulations.

I really thank the authors for the response and for adding many new experiments and significantly modified the manuscript. However, I would still keep my score of 2 for the following reason:

  • The authors add more timing comparisons, for example, a limited distributed system with GPUs. However, the communication bottleneck is inherent to the algorithm. The speedup is only 1.5x with two GPUs, and as communication becomes dominant, the scalability potential of this algorithm is very limited.
  • In addition, the advantage over baselines does not generalize to more advanced solvers (mostly limited to Richardson).
  • B.5 is tested on a 16x16 grid, and thus message passing could already reach about 10% of the nodes. The concern is still for larger problems, where message passing becomes more and more local relative to the problem size.
  • Several of my comments including the multiple-RHS question were not fully addressed.

And lastly, this is a general comment, not a limitation of this paper. Graph Neural Preconditioners and several follow-up works indeed use SciPy ILU, and in fact those papers motivated me to run some very simple tests comparing different ILU options. I am not saying that the ILU in SciPy is wrong. Just that even for simple SPD matrices, the ILUT in SciPy can produce substantially different sparsity patterns in L and U compared with other commonly used ILU implementations. Therefore, results based on a single ILU implementation should be interpreted with some care.

Reviewer_7XABOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes to develop a RL policy to evolve preconditioners during the solve. Preconditioners are often treated as static and fixed before the solve begins and the paper argues for updating preconditioners as the solve proceeds via closed loop feedback. The authors mainly consider a strongly anisostropic diffusion problem. The paper proposess their method inside the simple iterative solver, namely Richardson's iteration. Authors make good design choices to ensure the preconditioner stays well-behaved and is positive-definite by constuction, thus ensuring the solver's guarantee that each step is a descent direction. CLP, writes the P−1P^{-1} as sum of a diagonal matrix D that is forced to be strictly positive (through an exponential) and a low-rank congruence transformation. Rather than letting the network pick the weights of the coupling matrix CC, thr authors compute C with Galerkin weighting. All in all, through their design by incorporating this prior knowledge the preconditioner stays SPD at every step, regardless of what the Network proposes for U and D. Coming to the RL, the authors use a PPO algorithm with each solve of T=25 richardson steps as one one episode and outputs an increment to add to the diagonal and to the basis U. The reward at each step is the log of how much the residual shrank and since it is a telescopic sum, maximizing total reward would be identifcal to minimizing final residual. They use separate message passing graph network as the policy and critic. The authors show that CLP improves convergence over classical preconditioners. At the training size, CLP reaches a residual of 10^{-8} in roughly one-third the iterations required by AMG-preconditioned Richardson, and the same policy generalizes to systems up to 36X larger. Importantly, the ablations are comprehensive and demonstrate that benefits come comes from closed-loop adaptation.

Critical strengths

The design choices are really well-thought out especially guaranteeing about the preconditioners positive definiteness. The ablation suite is also very comprehensive and shows the relative advantages of each design choices, for example between feedback and no feedback, b/w MLP and GraphNN, analytic vs learned subspace weighting etc. Atleast, in the problem settings considered they establisjed the generalization benefits of the approach.

Critical weaknesses
  1. A natural feedback is that the paper mainly considers Richarson's iteration and compares with AMG without Kyrlov acclerators like CG or GMRES (https://hypre.readthedocs.io/en/latest/solvers-boomeramg.html). This makes the comparison results with AMG not fully apples-to-apples because AMG in practical scenarios are almost apired with Kyrlov. I noticed that the authors in the last line of conclusion state that extending CLP to Kyrlov is future work but it would be important to compare the current CLP with AMG+ CG.

  2. The GNN represents off-diagonal matrix entries using their absolute values ((|A_{ij}|)), discarding sign information. This representation may fail to distinguish different SPD operators with identical entrywise magnitudes but different off-diagonal signs. It is unclear whether this information loss is harmless for the specific magnetized diffusion problems considered or whether it limits generalization to broader classes of linear systems. Could the authors clarify the effect of this design choice?

  3. The policy incrementally updates the preconditioner parameters ((d_t,U_t)), but these accumulated states are not directly included in its observations, introducing partial observability. It is unclear whether this affects the effectiveness or stability of closed-loop adaptation, particularly when the policy is repeatedly deployed through correction cycles. An ablation incorporating the current preconditioner state, or a compact summary of it, would help evaluate the design choice.

4.Although the authors show that the policy transfers from (n=256) to (n=9216) without retraining, these systems are still relatively small for practical sparse linear algebra applications. Moreover, Figure 1 suggests that the iteration counts of several classical baselines also remain stable as the system size increases. It would be helpful to clarify whether these experiments represent mesh refinement on a fixed physical domain or increasing domain size, since these settings have different implications for numerical scalability.

Questions for authors

Please see the weaknesses box above.

3 comments

Thank you for your constructive and positive feedback. We address each concern below and provide additional experimental evidence.

Weakness 1: Comparison with Krylov-accelerated solvers

We agree that AMG is commonly combined with Krylov methods in practical applications, and that comparisons restricted to Richardson iteration do not establish practical solver competitiveness. Our study uses first-order Richardson iteration to isolate the contribution of feedback-conditioned preconditioner adaptation from momentum and Krylov-subspace dynamics. Within this controlled setting, CLP requires 2.77× fewer iterations than AMG-preconditioned Richardson.

To contextualize these results, we evaluated AMG-preconditioned second-order Richardson and CG baselines, alongside preliminary applications of CLP to second-order Richardson and flexible CG. For the accelerated CLP experiments, we retrained the policy separately within each solver while retaining the original architecture, preconditioner parametrization, and observation space. On the main anisotropic diffusion problem (n=256n=256), median iteration counts to a relative residual of 10−810^{-8} are:

SolverAMGCLP
First-order Richardson2,092754
Second-order Richardson638697
CG / Flexible CG188520

AMG-preconditioned CG substantially outperforms the current CLP configurations, confirming the importance of Krylov acceleration for practical solver performance. Nevertheless, CLP also achieves improved convergence when retrained within accelerated solvers: second-order Richardson reduces its iteration count from 754 to 697, while flexible CG reduces it to 520. Importantly, although these solvers exploit momentum and Krylov-subspace information, the CLP policy retains the observations originally designed for first-order Richardson iteration. Its learned preconditioner updates therefore do not explicitly depend on momentum, search directions, or Krylov history. These preliminary results demonstrate that the CLP formulation can be trained within accelerated solvers without modifying its policy architecture or observation space, while suggesting that incorporating solver-specific dynamical information into the policy is a promising direction for further development.

We will incorporate these comparisons and clarify the solver-specific training procedures in Appendix B.1 of the revised manuscript.

Weakness 2: Sign information in the operator graph

We agree that using absolute edge weights discards off-diagonal sign information from the policy's graph representation, potentially limiting its ability to distinguish SPD operators with identical entrywise magnitudes. However, the signed operator remains part of the numerical computation: CLP evaluates the residual rt=b−Axtr_t=b-Ax_t and the Galerkin weighting Ct=(Ut⊤AUt)−1C_t=(U_t^\top A U_t)^{-1} using the original matrix AA. Thus, operator signs influence the evolving solver state and low-rank correction, even though they are not explicitly provided as graph-edge features.

To assess whether this representation limits performance on our target problem family, we retrained CLP using signed edge weights while keeping the architecture and training configuration unchanged. On magnetized diffusion (n=256n=256), median iterations to a relative residual of 10−810^{-8} were:

PolicyIterations to 10−810^{-8} (↓)
CLP (ours)754
CLP with signed edges830

Explicitly incorporating edge signs does not improve convergence under the tested conditions. This suggests that the absolute-value representation is adequate for the magnetized-diffusion family considered here, although it does not rule out representational limitations or potential benefits from signed features for more general SPD operators.

We have included this ablation in Appendix B.2 of the revised manuscript.

Weakness 3: Partial observability of the accumulated preconditioner

Thank you for raising the question of whether observing the accumulated preconditioner state could improve subsequent policy updates. We agree that omitting (dt,Ut)(d_t,U_t) introduces partial observability, since these quantities influence the solver dynamics. To assess its practical impact, we retrained CLP with (dt,Ut)(d_t,U_t) included in the observations, keeping the training budget and other settings unchanged. On magnetized diffusion (n=256n=256), median iterations to a relative residual of 10−810^{-8} were:

MethodMedian iterations to 10−810^{-8} (↓)
CLP (ours)754
CLP with accumulated state observations7,726

The degradation is also evident within the 25-step training horizon, where the best validation residual increases from 0.2214 to 0.3263. This suggests that the augmented policy is less effectively learned under the fixed training budget.

One possible explanation concerns the representation of the accumulated state. In the unregularized Galerkin formulation, the low-rank correction depends on span(Ut)\mathrm{span}(U_t) rather than its particular basis, so the raw coordinates of UtU_t introduce representation dependence that is not necessary to characterize the instantaneous correction. This may complicate learning, although the ablation does not isolate this mechanism.

We have included this ablation in Appendix B.2 of the revised manuscript.

Weakness 4: System-size transfer and mesh refinement

Thank you for highlighting the distinction between system-size transfer and generalization under mesh refinement. In our original experiment (Figure 1, right), we refine the mesh on a fixed periodic unit-square domain while scaling the time step as Δτ∝h2\Delta\tau\propto h^2 (Appendix A.1). This keeps the discrete conditioning approximately constant across resolutions, explaining the relatively stable iteration counts observed for both CLP and the classical baselines.

To evaluate generalization under increasingly ill-conditioned systems, we additionally performed mesh refinement with a fixed time step. We applied the same CLP policy trained at n=256n=256, without retraining, and compared against AMG-preconditioned Richardson, measuring iterations to a relative residual tolerance of 10−610^{-6} (Figure 7, Appendix B.4). Both methods require more iterations as resolution increases, but CLP retains a substantial advantage. Across resolutions where both methods converge, CLP requires at least 2.2× fewer iterations than AMG. At n=9216n=9216, CLP converges while AMG does not reach the target tolerance within the iteration budget.

These results demonstrate that the learned policy retains its convergence advantage over AMG-preconditioned Richardson under increasingly ill-conditioned discretizations, rather than only when conditioning remains approximately constant. Although neither method exhibits mesh-independent convergence, CLP's advantage persists under this more challenging refinement regime.

We additionally evaluated distributed-memory execution on systems with up to n=5122=262144n=512^2=262144 unknowns, partitioning each solve across two GPUs (see our response to Reviewer dKMq, Question 2). These experiments provide complementary evidence of implementation feasibility at substantially larger scales.

We have clarified the distinction between the two refinement regimes and included the new results in Appendix B.4 of the revised manuscript.

Reviewer_7XABOpenReview ↗

Thank you for additional experiments

Thank you for the detailed experiments. I have also read through the other reviews and responses. I maintain my positive score and lean towards acceptance of the paper.

Comments on the paper

3 comments

We thank all three reviewers for their thoughtful feedback and constructive suggestions. We believe the additional experiments and manuscript revisions substantially strengthen the paper and clarify its contributions.

We are pleased that the reviewers recognized the motivation and convincing design of CLP (dKMq), particularly its structure-preserving construction and analytic Galerkin weighting (7XAB, 7jAp). They also highlighted the comprehensive ablations demonstrating the benefit of closed-loop feedback (7XAB, 7jAp), the alignment between experiments and claims (dKMq), and generalization to larger systems and longer solves without retraining (7XAB, 7jAp).

Classical baselines and accelerated solvers. To address concerns about baseline comparisons (7XAB, dKMq, 7jAp), we added PETSc ILU, hypre BoomerAMG, and AMG-preconditioned CG and FGMRES baselines (Appendix B.1). CLP retains a convergence advantage over PETSc ILU and hypre BoomerAMG within the matched Richardson framework. Preliminary experiments further demonstrate that CLP benefits from momentum and Krylov acceleration, extending the evidence for learned adaptation beyond first-order Richardson iteration.

Computational efficiency and scalability. To address practical efficiency and scalability (7XAB, dKMq, 7jAp), we clarified the hardware configurations and repeated end-to-end timing measurements, confirming a 2.4× speedup over AMG-preconditioned Richardson at n=9216n=9216. New experiments demonstrate distributed execution on systems with up to 262,144 unknowns, multi-GPU speedups, and an 18.6× throughput improvement from batching multiple right-hand sides (Appendix B.8).

Generalization and numerical behavior. Finally, to investigate robustness and numerical behavior (7XAB, dKMq), we added ablations on signed graph edges and accumulated-state observations, experiments under increasingly ill-conditioned mesh refinement (Appendices B.2–B.4), and a spectral analysis demonstrating effective reduction of slowly converging error components despite local message passing (Appendix B.5).

Theoretical foundations. We also clarify the structural guarantees of CLP, including positive definiteness, Galerkin optimality in the unregularized formulation, permutation equivariance, and consistency between training and deployment, while distinguishing these properties from empirical convergence improvements.

We have incorporated the reviewers' feedback throughout the revised manuscript, with changes highlighted in blue, and provide detailed responses to each concern below.

Thank you for your constructive feedback. We address the concerns below with additional experiments and clarifications.

Q1: Baseline implementations and end-to-end timing

Thank you for identifying an inconsistency in our timing description. The SciPy SuperLU and PyAMG implementations used in our original experiments execute their preconditioner operations on the CPU, not the GPU. We evaluated both CPU-only execution and hybrid CPU/GPU execution, where preconditioner construction occurs on the CPU and solver iterations execute on the GPU. The CPU-only implementations were faster, as the hybrid approach introduced additional transfer and synchronization overhead. The baseline timings in Table 1 therefore use CPU-only execution, their faster configuration, while CLP runs on the GPU. We will state this explicitly in the revised manuscript.

To address the concern about end-to-end performance, we repeated the timing experiments using a single wall-clock measurement, including host execution, synchronization, data transfers, preconditioner setup, and correction-cycle resets. At the largest size (n=9216n=9216), as in Table 1, median times to a relative residual tolerance of 10−810^{-8} were:

MethodEnd-to-end time (s) (↓)
AMG-preconditioned Richardson3.93
CLP (ours)1.65

CLP retains a 2.4× end-to-end speedup over AMG-preconditioned Richardson, compared with the originally reported 2.6× advantage. The modest reduction reflects additional overhead included in the revised measurements, particularly graph-metadata preparation and host-side operations excluded from the original component-level timing protocol. Thus, the advantage persists under a more comprehensive wall-clock measurement.

We have updated the timing protocol and reported results in Table 1 and Appendix A.5 of the revised manuscript.

Q2: Added AMG-preconditioned CG and FGMRES baselines

Thank you for emphasizing the importance of accelerated solvers in assessing practical competitiveness. Our original experiments use first-order Richardson iteration to isolate the contribution of learned preconditioner adaptation from momentum and Krylov-subspace dynamics. We have added the requested AMG-preconditioned CG and FGMRES baselines.

On the main anisotropic-diffusion problem (n=256n=256), median iterations and median end-to-end times to a relative residual of 10−810^{-8}, measured with the wall-clock protocol of Q1, are:

SolverPreconditionerIterations (↓)Time (ms) (↓)
RichardsonCLP754377
RichardsonAMG2092414
CGAMG18823
FGMRES(256)AMG8829

At a larger problem size (n=2562=65536n=256^2=65536), the corresponding results are:

SolverPreconditionerIterations (↓)Time (ms) (↓)
RichardsonCLP13386633
RichardsonAMG237220718
CGAMG2111376
FGMRES(256)AMG1042906

As expected, classical acceleration substantially improves convergence, with AMG-preconditioned CG achieving the fastest convergence in wall-clock time among the evaluated methods. FGMRES is included as a flexible Krylov baseline.

Nevertheless, the wall-clock performance gap narrows substantially as system size increases, suggesting that CLP scales more favorably than the tested AMG implementation on this problem family. In particular, CLP's end-to-end advantage over AMG-preconditioned Richardson grows from 1.1× at n=256n=256 to 3.1× at n=2562n=256^2, while the gap to AMG-preconditioned CG narrows from 16× to 4.8×. These results suggest that the computational competitiveness of learned preconditioner adaptation improves with problem size, motivating further evaluation on substantially larger systems. Fully integrating CLP with Krylov acceleration also remains future work.

We incorporated these comparisons and clarified the distinction between demonstrating the benefit of learned adaptation and establishing practical solver competitiveness in the revised manuscript.

Thank you for your response. Most of my concerns are solved.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.