acceptodds
Under review as a conference paper at ICLR 2027

RINS: Residual-Image Neural Subspace Solvers for Large Sparse Linear Systems

Abstract

Neural subspace solvers offer a promising way to accelerate the large sparse linear systems arising from PDE discretizations, but their value depends on generating effective correction directions at low computational overheads. We study this problem from a residual-image viewpoint: a correction subspace is useful when its image under the system operator captures the current residual. Guided by this view, we propose Gate-RINS, which generates multiple correction directions from shared polynomial residual probes and modulates them with a lightweight residual- and coordinate-dependent pointwise gate. We also introduce a hybrid schedule that uses GRANS-style graph controllers for the first correction block and Gate-RINS thereafter, combining an expressive graph based initial step with a lower-cost corrector as the solve progress. Theoretical analysis establishes a unified residual-image and dynamic approximate-inverse perspective for Gate-RINS and G-RANS, showing that a single neural-generated direction can emulate the residual quality of fixed-step GMRES. Numerical experiments on six PDE benchmarks demonstrate that Gate-RINS reaches a relative residual of with 1.6-2.3 and 2.2-3.0 less wall-clock time than G-RANS and GMRES respectively. For Helmholtz systems with meshsize up to , Gate-RINS also achieves the lowest residual and outperforms a GPU-accelerated parallel GMRES implementation, while GRANS fails to reach relative residual of in the same time budget.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural solver, linear system, krylov iteration

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_hExCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The manuscript proposes RINS, which is a family of neural Subspace solvers. The paper is motivated by the high cost of graph-attention-based generation of correction directions. The proposed residual-image viewpoint argues that a correction subspace is useful when its image under the system operator captures the current residual. The proposed Gate-RNIS module constructs shared polynomial residual probes using Chebyshev recurrences and SpMVs. A hybrid inference setup adopts a graph-based G-RANS controller for the first correction block and Gate-RINS for the remainder. Evaluations on six linear PDEs report that Gate-RINS reaches a relative residual of 1e-5 with reported mean wall-clock speedups of 1.6-2.3x over G-RANS and 2.2-3.0x over GMRES.

Critical strengths
  • The residual-image viewpoint is insightful.
  • The authors conducted a theoretical interpretation.
  • The reported speedup is significant and includes details such as a convergence plot, residual, e2e wall clock time, etc.
Critical weaknesses
  • Somewhat limited benchmark. All six PDEs have approximately 1k unknowns.
  • Theoretical analysis terminates midway. The GMRES-emulation theorem does establish the existence of a continuous neural generator, but it does not guarantee that Gate-RINS can represent that neural generator or that the training procedure can converge to it. Although this is empirically verified by the experimental results, more theoretical reasoning would be appreciated.
  • The main benchmark compares Gate-RINS against GMRES and G-RANS without an external preconditioner. However, such legacy numerical routines are typically fine-tuned, especially when the type of PDE is known.
Questions for authors
  • Please see "weaknesses" above.
3 comments
AuthorsOpenReview ↗

response to reviewer hExC (part 1)

Thank you for your in-depth review and constructive feedback.

Thank you for recognizing the residual-image perspective, theoretical interpretation, and details of the convergence curves and end-to-end timings. Below, we respond separately to the comments on experimental scope, theoretical analysis, and comparisons with preconditioned baselines.

Q1: 【Weakness 1】All six PDE benchmarks have only approximately 1k unknowns, so the experimental scope is limited.

A1: We added low-wavenumber (k0=8k_0=8) and high-wavenumber (k0=24k_0=24) experiments on 10k two-dimensional indefinite Helmholtz, with 10 test systems per task. The table below reports the mean full solve time ± inter-system standard deviation over successful systems.

Method10k low-wavenumber (k0=8k_0=8): successful systems10k low-wavenumber (k0=8k_0=8): time (s)10k high-wavenumber (k0=24k_0=24): successful systems10k high-wavenumber (k0=24k_0=24): time (s)
GMRES10/10111.4760 ± 47.306610/10824.6460 ± 361.0820
FGMRES + Jacobi10/1094.6788 ± 39.334110/10741.7620 ± 337.7650
FGMRES + AMG10/101.1393 ± 0.556010/10122.0310 ± 153.8910
GRANS0/10Threshold not reached, no successful-system mean0/10Threshold not reached, no successful-system mean
Gate-RINS10/1016.1987 ± 8.16600/10Threshold not reached, no successful-system mean

The target is a true relative residual of 10−510^{-5}; Gate-RINS uses 500 rounds, 20 directions per round, and 4000 basis vectors, while the numerical baselines use a restart/maximum iteration count of 4000. At low-wavenumber (k0=8k_0=8), Gate-RINS achieves a 5.84× speedup over Jacobi-FGMRES and a higher success rate than GRANS, but remains slower than AMG; at high-wavenumber (k0=24k_0=24), neither neural method reaches the threshold. The standard deviations in the table reflect differences among the 10 test systems.

Q2: 【Weakness 2】The GMRES emulation theorem only proves the existence of a continuous generator, guaranteeing neither that Gate-RINS can represent this generator nor that the training process can converge to it; despite the empirical validation provided by the experiments, more theoretical reasoning is still desired.

A2: We evaluated the Gate-RINS architecture through independent training and held-out solve experiments: each of the original six approximately 1k tasks uses five training seeds and 10 fixed test systems, and the actual trained Gate-RINS reaches a true relative residual of 10−510^{-5} in all 50 records for each task, with mean solve times of 0.0717–0.1350 seconds. The five-seed experiments on three approximately 1k indefinite Helmholtz tasks also reach the threshold in 50/50 cases each, with mean full solve times of 0.4383, 0.6610, and 0.1813 seconds for two-dimensional low-wavenumber (k0=8k_0=8), two-dimensional high-wavenumber (k0=24k_0=24), and three-dimensional low-wavenumber (k0=8k_0=8), respectively.

Furthermore, we conducted five-seed experiments on Poisson under three scale and geometry transfer settings, freezing the models after source-domain training and solving directly in the target domain. The table below gives the wall-clock time to first reach a true relative residual of 10−510^{-5} on the prespecified targets, reporting the mean ± seed × sample joint standard deviation; grid transfer tests 10 systems per seed, geometry transfer tests 20 systems per seed, and three timings are first averaged in each cell.

Transfer settingGate-RINS wall-clock time to threshold(s)GRANS wall-clock time to threshold(s)
1k→3.5k0.6779±0.06742.0122±0.3311
5k→2.5k0.4670±0.04441.4281±0.2178
Rectangle→I-shape0.1004±0.00580.5772±0.1355
AuthorsOpenReview ↗

response to reviewer hExC (part 2)

Q3: 【Weakness 3】The main comparisons do not use external preconditioners, but practical numerical solvers are usually extensively tuned for known PDE types.

A3: Thank you for your suggestion. We added preconditioned baselines for approximately 1k two-dimensional low-wavenumber (k0=8k_0=8), two-dimensional high-wavenumber (k0=24k_0=24), and three-dimensional low-wavenumber (k0=8k_0=8) indefinite Helmholtz, with both numerical and neural methods running on GPUs. The following reports the mean full solve time ± inter-system standard deviation, with the number of successful systems in parentheses.

MethodTwo-dimensional low-wavenumber (k0=8k_0=8): time (s)Two-dimensional high-wavenumber (k0=24k_0=24): time (s)Three-dimensional low-wavenumber (k0=8k_0=8): time (s)
FGMRES + Jacobi8.4673 ± 3.3170 (10/10)45.9039 ± 17.7388 (10/10)4.8896 ± 2.9109 (10/10)
FGMRES + AMG0.6991 ± 0.4427 (10/10)25.9626 ± 11.3193 (9/10)2.3241 ± 1.1246 (8/10)
GRANS1.1303 ± 0.1260 (10/10)1.1518 ± 0.1067 (10/10)0.6770 ± 0.1369 (10/10)
Gate-RINS0.4127 ± 0.0732 (10/10)0.6549 ± 0.0708 (10/10)0.1836 ± 0.0093 (10/10)

The relative residual threshold is 10−510^{-5}; the numerical baselines use a restart/maximum iteration count of 4000, and the neural models use 20 directions per round, at most 200 rounds, and 4000 basis vectors. Jacobi uses inverse diagonal scaling, and AMG uses the PyAMG classical Ruge–Stuben hierarchy and a GPU V-cycle. At two-dimensional low-wavenumber (k0=8k_0=8), Gate-RINS achieves a 1.69× speedup over AMG-FGMRES and a 2.74× speedup over GRANS; AMG has failed systems in the high-wavenumber (k0=24k_0=24) and three-dimensional cases, and the table retains its success proportions. These same-GPU comparisons show lower mean solve times for Gate-RINS on the tested approximately 1k indefinite Helmholtz systems.

Q4: 【Question 1】Please see the weaknesses above.

A4: Please see the responses to Q1–Q3.

Thank you again for your time and careful consideration, and for the constructive suggestions that help us improve the presentation of the method and the experimental comparisons.

Thank you for the response. I am keeping my original score of weak acceptance.

Reviewer_eiGrOpenReview ↗
Rating
1
Confidence
3(+1)
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper speeds up Krylov-type solvers for sparse linear systems from PDEs. A neural network generates correction directions, and a small least-squares solve picks the best combination. The proposed Gate-RINS builds directions from shared Chebyshev polynomial probes of the residual (few sparse matrix-vector products), and adds a small pointwise gate. A hybrid schedule uses G-RANS for the first block and Gate-RINS afterwards. On six PDE systems with n ≈ 1000, it reaches residual 1e-5 about 1.6–2.3× faster than G-RANS and 2.2–3.0× faster than GMRES.

Critical strengths
  1. The idea is simple, and the cost argument (shared probes, fewer sparse matrix-vector products) is clear.
  2. The experiments cover several PDEs, a one-direction test, preconditioning, and size and geometry changes.
Critical weaknesses
  1. The problems are too small and too easy. Most tests have n ≈ 1000, solved in 0.06–0.3 s. Direct solvers or AMG-preconditioned CG solve such systems in milliseconds. The one large test (n ≈ 10⁵ Helmholtz) is a single figure with no table or variance.
  2. Weak baselines. The main comparison is against unpreconditioned GMRES and G-RANS. There is no AMG, ILU-preconditioned Krylov, or direct solver. GMRES runs on CPU and the neural methods on GPU, so the timing comparison may not be fair. GMRES implementation details are missing.
  3. Easy test cases. The paper admits its Helmholtz system has a positive reaction term and is not the hard indefinite high-frequency case. Strongly nonsymmetric, indefinite or ill-conditioned systems, where Krylov methods struggle, are not tested.
  4. No training cost or break-even analysis. Each PDE needs its own model. The paper does not say how long training takes or how many solves are needed to pay it back.
  5. Weak link between theory and method. Theorem 1 is an existence result from universal approximation and uses the full (A, r₀) as input, which is far from the actual method. The gate breaks the Krylov structure, and no convergence guarantee is given.
  6. Narrow generalization. Size and geometry tests are on Poisson only, with a single training seed.
  7. Some claims are too strong or unclear. In Table 3, Gate-RINS has a worse single-step residual than G-RANS, yet is said to support the hybrid result. GRANS fails with one direction, which may mean the baseline is not well tuned. Appendix D.2 calls the one-direction model both Static-RINS and gate-RINS.
Questions for authors
  1. How does it compare with AMG-CG, ILU-preconditioned GMRES, and direct solvers, on the same hardware?
  2. What are the training cost and the break-even number of solves?
  3. Does it work on large, indefinite or strongly nonsymmetric systems?
  4. Please give full large-scale results with variance over seeds.
8 comments
AuthorsOpenReview ↗

response to reviewer eiGr (part 1)

Thank you for your careful evaluation and for pointing out the issues that require clarification.

Thank you for recognizing the simplicity of the method and the clarity of the cost-saving idea of sharing probes and reducing sparse matrix-vector multiplications. We also thank you for acknowledging the coverage of experiments on multiple PDE types, single-direction methods, preconditioning, and changes in scale and geometry; below, we address your concerns about problem difficulty, comparison fairness, training costs, theoretical connections, and statistical reliability one by one.

Q1: 【Weakness 1】The problems are too small and simple: they are mainly approximately 1k systems, which classical direct methods or AMG-CG can solve in milliseconds; the only approximately 100k Helmholtz experiment has curves but no complete table or variability statistics.

A1: To examine performance as scale increases, we added low-wavenumber (k0=8k_0=8) and high-wavenumber (k0=24k_0=24) experiments on 10k two-dimensional indefinite Helmholtz, comparing against GRANS, unpreconditioned GMRES, and Jacobi/AMG-preconditioned FGMRES. All results below use an RTX 4090, 10 test systems per task, and training seed 2026; times are the mean full solve time ± inter-system standard deviation over successful systems.

Method10k low-wavenumber (k0=8k_0=8): successful systems10k low-wavenumber (k0=8k_0=8): time (s)10k high-wavenumber (k0=24k_0=24): successful systems10k high-wavenumber (k0=24k_0=24): time (s)
GMRES10/10111.4760 ± 47.306610/10824.6460 ± 361.0820
FGMRES + Jacobi10/1094.6788 ± 39.334110/10741.7620 ± 337.7650
FGMRES + AMG10/101.1393 ± 0.556010/10122.0310 ± 153.8910
GRANS0/10Threshold not reached, no successful-system mean0/10Threshold not reached, no successful-system mean
Gate-RINS10/1016.1987 ± 8.16600/10Threshold not reached, no successful-system mean

At 10k low-wavenumber (k0=8k_0=8), Gate-RINS achieves a 5.84× speedup over Jacobi-FGMRES and reaches the threshold on all 10 systems on which GRANS fails to do so, but AMG-FGMRES remains 14.22× faster. At high-wavenumber (k0=24k_0=24), neither neural method reaches the threshold. The common target is a true relative residual of 10−510^{-5}; Gate-RINS has a budget of 500 rounds, 20 directions per round, and 4000 basis vectors, while the numerical baselines use a restart/maximum iteration count of 4000.

In addition, we added strong-baseline comparisons for 1k two-dimensional low-wavenumber (k0=8k_0=8), high-wavenumber (k0=24k_0=24), and three-dimensional low-wavenumber (k0=8k_0=8) indefinite Helmholtz; see Q2 for details. The original approximately 100k curves correspond to the Helmholtz* task with a positive reaction term.

AuthorsOpenReview ↗

response to reviewer eiGr (part 2)

Q2: 【Weakness 2】The comparisons include only unpreconditioned GMRES and G-RANS, lacking AMG, ILU-Krylov, and direct methods; CPU/GPU differences may cause unfairness, and GMRES implementation details are missing.

A2: Thank you for your suggestion. We added comparisons on approximately 1k indefinite Helmholtz on the same GPU to avoid drawing conclusions solely from CPU/GPU timing differences. Both numerical and neural methods run on an RTX 4090; the following reports the mean full solve time ± inter-system standard deviation, with the number of successful systems in parentheses.

MethodTwo-dimensional low-wavenumber (k0=8k_0=8): time (s)Two-dimensional high-wavenumber (k0=24k_0=24): time (s)Three-dimensional low-wavenumber (k0=8k_0=8): time (s)
GMRES8.7489 ± 3.3776 (10/10)51.6184 ± 19.4162 (10/10)5.0123 ± 2.7995 (10/10)
FGMRES + Jacobi8.4673 ± 3.3170 (10/10)45.9039 ± 17.7388 (10/10)4.8896 ± 2.9109 (10/10)
FGMRES + AMG0.6991 ± 0.4427 (10/10)25.9626 ± 11.3193 (9/10)2.3241 ± 1.1246 (8/10)
GRANS1.1303 ± 0.1260 (10/10)1.1518 ± 0.1067 (10/10)0.6770 ± 0.1369 (10/10)
Gate-RINS0.4127 ± 0.0732 (10/10)0.6549 ± 0.0708 (10/10)0.1836 ± 0.0093 (10/10)

Gate-RINS succeeds on 10/10 systems in all three tasks, achieving speedups of 20.52×, 70.10×, and 26.63× over Jacobi-FGMRES and 2.74×, 1.76×, and 3.69× over GRANS. For two-dimensional low-wavenumber (k0=8k_0=8), it achieves a 1.69× speedup over AMG-FGMRES; AMG succeeds on only 9/10 and 8/10 systems for high-wavenumber (k0=24k_0=24) and three-dimensional cases, respectively.

The threshold is a relative residual of 10−510^{-5}, with a restart/maximum iteration count of 4000. The neural models use 20 directions per round, at most 200 rounds, and 4000 basis vectors. Jacobi uses inverse diagonal scaling, and AMG uses the PyAMG classical Ruge–Stuben hierarchy and a GPU V-cycle.

Q3: 【Weakness 3】The Helmholtz in the submission has a positive reaction term and is not a difficult indefinite high-wavenumber system (the reference wavenumber in the additional tests in this round is k0=24k_0=24); tests on strongly nonsymmetric, indefinite, or ill-conditioned systems are missing.

A3: Thank you for pointing out this issue. The Helmholtz in the original six-task benchmark indeed uses the positive-reaction-term form A=K+MαA=K+M_{\alpha}, which we label Helmholtz*, and we will correct the wording in the abstract that may lead readers to interpret the original large-scale task as indefinite Helmholtz.

The new experiments in this round use the negative-mass-term form A=K−k2McA=K-k^2M_c, including two-dimensional low-wavenumber (k0=8k_0=8), two-dimensional high-wavenumber (k0=24k_0=24), and three-dimensional low-wavenumber (k0=8k_0=8) indefinite Helmholtz. The reference wavenumber varies by ±30%, with the two settings corresponding to ranges of [5.6,10.4][5.6,10.4] and [16.8,31.2][16.8,31.2], respectively. See Q2 for the 1k results, where Gate-RINS succeeds on 10/10 systems in all three tasks; see Q1 for the 10k results, where all systems still reach the threshold at low-wavenumber (k0=8k_0=8), but the threshold is not reached at high-wavenumber (k0=24k_0=24). We therefore explicitly distinguish the new evidence on indefinite systems from the original positive-reaction-term task. A systematic evaluation of strong nonsymmetry and condition-number stratification remains for future research.

Q4: 【Weakness 4】Each PDE requires a dedicated model, but training costs and the number of solves required to break even are not reported.

A4: RINS targets repeated solves within the same task family where a model can be reused; when switching tasks and retraining, the offline training cost needs to be counted again. The following gives the cumulative training wall-clock time for three approximately 1k Helmholtz tasks on an RTX 4090, all trained for 350 epochs.

TaskGate-RINS training (min)GRANS training (min)Gate-RINS training time reduction
Two-dimensional low-wavenumber (k0=8k_0=8)37.4054.2331.03%
Two-dimensional high-wavenumber (k0=24k_0=24)37.6457.6934.75%
Three-dimensional low-wavenumber (k0=8k_0=8)40.6054.9926.17%

Gate-RINS reduces training wall-clock time by approximately 26%–35% compared with GRANS; see Q2 for the corresponding solve times. In the 10k low-wavenumber (k0=8k_0=8) results in Q1, Gate-RINS remains slower than AMG but is faster than Jacobi-preconditioned FGMRES, and increasing the number of solves for the same task may allow its training cost to be amortized.

AuthorsOpenReview ↗

response to reviewer eiGr (part 3)

Q5: 【Weakness 5】The theorem relies on universal approximation and the full (A,r0)(A,r_0) input, leaving a gap from the actual Gate-RINS; gating breaks the strict Krylov structure, and no convergence guarantee is given for the specific method.

A5: The actual Gate-RINS takes polynomial residual probes and node coordinates as inputs to a pointwise MLP and generates search directions that vary with iterations through residual-dependent gating. We directly examine the effectiveness of this specific architecture through repeated training and solve experiments on held-out systems.

We added experiments with five training seeds on three approximately 1k indefinite Helmholtz tasks: the 200/20/10 data split is fixed, and only the training seed varies over 2026–2030, using RTX 4090, 150/100/100 training epochs, 20 directions per round, at most 500 rounds, and 4000 basis vectors. Each model reaches a true relative residual of 10−510^{-5} on the five seeds × 10 test systems for each task; three timings are first averaged for each system, then the 10 systems for that seed are averaged, and the table reports the mean of the five seed means ± the inter-seed sample standard deviation.

Approximately 1k indefinite Helmholtz taskGate-RINS solve(s)GRANS solve(s)
Two-dimensional low-wavenumber (k0=8k_0=8)0.4383±0.02341.1502±0.0122
Two-dimensional high-wavenumber (k0=24k_0=24)0.6610±0.00321.1534±0.0064
Three-dimensional low-wavenumber (k0=8k_0=8)0.1813±0.00270.6558±0.0124

Q6: 【Weakness 6】The scale and geometry generalization experiments cover only Poisson and use only one training random seed, so the scope of generalization evidence is limited.

A6: Thank you for pointing this out. We separately quantify training randomness and test-sample differences: with a fixed data split, we use five independent training seeds, 2026–2030; three timings are first averaged for each “seed × test sample,” after which we report the overall mean and the joint standard deviation across seeds and samples, with the standard deviation of seed means and the standard deviation of sample means listed separately.

We added scale and geometry generalization experiments with three models and five independent training seeds, totaling 45 source-domain training runs; the same source-domain checkpoint is frozen for all target grids/geometries, without target-domain fine-tuning. The grid experiments fix 10 test systems for each target scale, and the geometry experiments use 20 I-shaped systems and 20 rectangular controls with matched parameters. The table below selects the prespecified targets: 1k→3.5k, 5k→2.5k, and rectangle→I-shape; see the separate generalization report for all target scales. Times are the mean ± seed × sample joint SD for the first true residual of 10−510^{-5}, with seed-mean SD and sample-mean SD listed separately; three timings are first averaged within each cell.

TransferModelSuccessful systemsTime to threshold(s)Seed-mean SD(s)Sample-mean SD(s)
1k→3.5kGate-RINS50/500.6779±0.06740.05490.0391
1k→3.5kGRANS50/502.0122±0.33110.31500.1485
1k→3.5kBlock-GRANS50/501.4728±0.19070.09600.1469
5k→2.5kGate-RINS50/500.4670±0.04440.03540.0276
5k→2.5kGRANS50/501.4281±0.21780.19430.1156
5k→2.5kBlock-GRANS50/501.2577±0.18980.07160.1200
Rectangle→I-shapeGate-RINS100/1000.1004±0.00580.00240.0032
Rectangle→I-shapeGRANS100/1000.5772±0.13550.14930.0138
Rectangle→I-shapeBlock-GRANS100/1000.4038±0.01800.01060.0092

All the above experiments use RTX4090 and test transfers within the Poisson family. The joint SD describes variation across training seeds and test samples. Threshold-attainment metrics use the same systems that succeed across all five seeds, while success proportions retain all samples.

The main experiments also switched to Gate-RINS and GRANS weights from five independent training seeds, 2026–2030; with 10 fixed test systems per task, we remeasured the cost of first reaching a true relative residual of 10−510^{-5}. Three timings are first averaged within each “seed × sample” cell, after which statistics are computed over 50 cells, avoiding treating timing repetitions as independent training runs. The following time means ± joint standard deviations include both seed and sample variation; parentheses give the seed-mean SD and sample-mean SD, respectively (seconds).

Reviewer_TL6aOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a hybrid neural-numerical approach to iteratively solve a large sparse linear system. At each outer iteration a neural network proposes multiple search directions from the current residual. The directions are orthogonalized against an accumulated basis, and an update comes from a projected minimum-residual solve. The paper builds on G-RANS (Liao et al., ICML 2026) and replaces the G-RANS graph-attention basis generator with "residual-image" generators: Static RINS build a Chebyshev dictionary and uses learned linear combination of it. Gate RINS additionally does point wise multiplication of each combination with an MLP output. Futhermore, the paper adds a hybrid schedule that uses G-RANS for the first block and Gate RINS afterwards, an "approximate-inverse" interpretation of the generated directions and a theorem stating that a single neural direction step can match m GMRES steps. Experiments show that Gate RINS reaches comparable results in 1.6-2.3x less wall-clock time than G-RANS and 2.2-3.0x less than GMRES.

The overall framework is taken from G-RANS. Thus the contribution of the paper is the new generator architecture and the hybrid schedule as well as the theoretical analysis. Learning search directions or subspaces for Krylov-type solvers has been done before and the paper also covers the related work properly. I do not consider myself a full expert in the domain. However, to my understanding, Gate RINS can be understood as a flexible block Krylov method with a learned, residual-dependent diagonal preconditioner. The paper does not frame it this way and does not compare against the corresponding non-learned methods: to classical polynomial preconditioning (e.g., Loe & Morgan, 2022), and flexible GMRES with variable preconditioners (Saad, 1993). With that line of thinking, several closely related learned-preconditioner works are not considered either:

  • Li et al., 2023 (learned preconditioners for CG PDE solvers);
  • Trifonov et al., 2024 (GNN preconditioners for CG);
  • Dimola et al., 2025, which trains neural preconditioners with a Krylov-subspace-angle loss through flexible GMRES and is conceptually close to the "residual-image" criterion if I am not mistaken.

J. A. Loe, R. B. Morgan. Toward Efficient Polynomial Preconditioning for GMRES. Numerical Linear Algebra with Applications, 2022. DOI 10.1002/nla.2427. Y. Saad. A Flexible Inner-Outer Preconditioned GMRES Algorithm. SIAM J. Sci. Comput. 14(2), 1993. Y. Li, P. Y. Chen, T. Du, W. Matusik. Learning Preconditioners for Conjugate Gradient PDE Solvers. ICML 2023. V. Trifonov, A. Rudikov, O. Iliev, Y. M. Laevsky, I. Oseledets, E. Muravleva. Learning from Linear Algebra: A Graph Neural Network Approach to Preconditioner Design for Conjugate Gradient Solvers. arXiv 2405.15557, 2024. N. Dimola, A. Coclite, P. Zunino. Neural Preconditioning via Krylov Subspace Geometry. arXiv 2507.15452, 2025. PyAMG: N. Bell, L. N. Olson, J. Schroder. PyAMG: Algebraic Multigrid Solvers in Python. Journal of Open Source Software, 2022.

Critical strengths
  • The residual-image viewpoint which says that useful directions are those whose image under A captures the current residual is intuitive and well motivated.
  • Sharing the polynomial probes across all directions keeps the SpMV cost low and is clearly cheaper than graph attention.
  • The method keeps a numerical minimum residual projection which allows the solver to certify a residual tolerance, unlike end-to-end neural surrogates.
  • The paper includes multiple useful diagnostics, including a single direction test, scale and geometry transfer, a Jacobi preconditioned version and spectral-overlap analysis.
  • The experimentation protocol is well documented, including CUDA synchronization and fixed CPU threading.
Critical weaknesses
  • In my opinion the most important issue comes from baselines: the comparison does not show practical value. The only classical baseline in the main benchmark is GMRES without preconditioner, run on the CPU with a single thread. The neural solvers run on an H100. A Jacobi-preconditioned variant only appears in the appendix. The to-my-knowledge standard solvers for these problems are missing: sparse direct solvers, AMG-preconditioned CG/GMRES and ILU/IC-preconditioned Krylov methods. For 2D FEM systems of this size, these typically solve in milliseconds (n about 1000) up to well below a second (n about 100,000). The reported times are around 0.1–0.3 s at n approx 1000. As it stands, the speedups only show progress within the neural-solvers but not over the state of actual practice.
  • The test problems are small, two-dimensional and mostly symmetric positive definite. For SPD systems, CG rather than GMRES would be the natural baseline in this case. The "Helmholtz" problem is not Helmholtz in the usual sense as far as I can tell. Appendix B states that it is a positive reaction-diffusion problem, "not the indefinite high-frequency Helmholtz regime". The abstract highlights a "Helmholtz system with mesh size up to 100,000" though. There are no strongly anisotropic, high-contrast, indefinite or 3D problems, and no real-world matrices (e.g., SuiteSparse).
  • Compared to G-RANS, the contribution of the paper essentially reduces to replacing graph attention with a Chebyshev dictionary and a pointwise gate. The hybrid schedule is only shown on Poisson. So I would not consider the paper as contribution rich. A further variant (Block GRANS) is introduced in the appendix, where it is shown to sometimes be better than Gate RINS.
  • Several costs are not reported in the paper:
    • The basis is not restarted, and 20 vectors are added per iteration, so the accumulated basis becomes a substantial fraction of n at convergence. Memory and orthogonalization costs are not reported.
    • No SpMV counts or basis dimensions are given next to the wall-clock times. At n approx 1000, wall-clock time on a GPU is likely dominated by launch overhead rather than arithmetic.
    • The models are trained per PDE family and scale, but training time and the number of solves needed to amortize it are not reported.
Questions for authors
  1. How does RINS compare in wall-clock time with AMG-preconditioned CG/GMRES and with a sparse direct solver at n 1000 and n 100,000? Is there any setup in which RINS outperforms the best classical solvers?
  2. Please report SpMV counts, accumulated basis dimension and memory for RINS, G-RANS and GMRES. training time per PDE family.
  3. How does the method perform on harder systems like indefinite high-frequency Helmholtz, anisotropic coefficients, 3D problems, or SuiteSparse matrices?
8 comments
AuthorsOpenReview ↗

response to reviewer TL6a (part 1)

Thank you for your detailed review and constructive suggestions.

Thank you for recognizing the intuitive nature of the residual-image criterion, the computational efficiency of shared polynomial probes, and the design that retains numerical minimum-residual projection. We also thank you for acknowledging the diagnostic experiments on single directions, scale and geometry transfer, preconditioning, and spectral analysis, as well as timing details such as CUDA synchronization and CPU thread settings; below, we focus on related-work positioning, comparisons with classical solvers, problem difficulty, and cost statistics.

Additional comment: Related work and method positioning (from the discussion following the Paper Summary, not using Weakness/Question numbering).

Additional response: Thank you for this helpful interpretation and the suggested references. We have revised the abstract, introduction, related work, methodology, and conclusion to frame Gate-RINS as lightweight, residual-dependent neural preconditioning within a flexible minimum-residual solver. The revised manuscript explicitly connects the method to polynomial preconditioning and variable-preconditioner GMRES.

For each generated direction, the action is bt,j=Mt,jrt=Dt,j(rt)pj(A~)rtb_{t,j}=M_{t,j}r_t=D_{t,j}(r_t)p_j(\widetilde A)r_t. A shared pointwise MLP produces the diagonal modulation from polynomial residual probes and coordinates. The complete action Mt,jM_{t,j} combines this diagonal factor with a polynomial operator. The network supplies candidate correction directions; orthogonalization and numerical least squares select the update over the accumulated basis. Our short-horizon bootstrap objective trains these directions through their post-projection residuals.

Related work and methodological distinctions:

  • Classical polynomial preconditioning. Loe and Morgan [1] provide a classical counterpart to the operator-polynomial component. Static-RINS learns scalar combinations of shared polynomial probes; Gate-RINS adds residual-dependent diagonal modulation, producing adaptive direction blocks at a probe cost determined by polynomial order. The modulation generally does not commute with AA, so the resulting actions vary with the residual.

  • Flexible GMRES. Saad [2] establishes the variable-preconditioning framework. Standard FGMRES applies the active preconditioner to an Arnoldi vector; our solver generates a block from the current residual and retains its directions in an accumulated minimum-residual solve. We therefore describe the standalone and hybrid methods as generalized neural FGMRES-style constructions. The hybrid changes the learned action from a GRANS prefix to Gate-RINS while preserving the numerical update.

  • CG-oriented learned preconditioners. Li et al. [3] learn approximate matrix factorizations for PCG, while Trifonov et al. [4] learn sparse corrections to incomplete-Cholesky factors. These methods exploit SPD structure. RINS targets general systems, including the indefinite Helmholtz problems added in this revision. We discuss these CG-oriented approaches as related work; the numerical comparisons for the indefinite tasks use GMRES and Jacobi/AMG-preconditioned FGMRES.

  • Geometry-aware neural preconditioning. Dimola et al. [5] train a nonlinear inverse-action preconditioner through FGMRES using a multi-step angle-sine objective. Their angle criterion and our projected-residual criterion share the same minimum-residual projection geometry: alignment of the residual with the operator-applied correction space determines residual reduction. The methodological distinction lies in the parameterization and training procedure. Their network supplies an inverse action within FGMRES; Gate-RINS directly emits several diagonally modulated polynomial directions and uses progressive bootstrap training with projected-residual losses.

  • G-RANS and implementation attribution. We retain G-RANS's [6] projected minimum-residual update and residual-adaptive block-generation framework. Our generator replaces recurrent graph-attention propagation with shared polynomial probes and a lightweight pointwise MLP. We also added the PyAMG citation [7] for the classical AMG hierarchy used by the numerical baselines.

AuthorsOpenReview ↗

response to reviewer TL6a (part 2)

Revision location: These changes are already present in the revised manuscript: the Introduction introduces the preconditioning perspective; Section 2.2, in the paragraph “Polynomial, flexible, and learned preconditioning,” discusses all five reviewer-suggested works and their relationship to RINS; Section 3.1 presents the minimum-residual geometry, Section 3.2 defines the diagonally modulated polynomial action, and Section 3.3 explains variable learned preconditioners and the generalized neural FGMRES-style hybrid. Appendix C develops the dynamic approximate-inverse interpretation, and Section 4.1 cites PyAMG for the AMG baseline. New and revised wording is marked in blue.

[1] Loe J A, Morgan R B. Toward efficient polynomial preconditioning for GMRES[J]. Numerical Linear Algebra with Applications, 2022, 29(4): e2427.

[2] Saad Y. A Flexible Inner-Outer Preconditioned GMRES Algorithm[J]. SIAM Journal on Scientific Computing, 1993, 14(2): 461-469.

[3] Li Y, Chen P Y, Du T, et al. Learning Preconditioners for Conjugate Gradient PDE Solvers[C]. ICML, 2023, PMLR 202: 19425-19439.

[4] Trifonov V, Rudikov A, Iliev O, et al. Learning from Linear Algebra: A Graph Neural Network Approach to Preconditioner Design for Conjugate Gradient Solvers[J]. arXiv preprint arXiv:2405.15557, 2024.

[5] Dimola N, Coclite A, Zunino P. Neural Preconditioning via Krylov Subspace Geometry[J]. arXiv preprint arXiv:2507.15452, 2025.

[6] Liao W, Feng M, Zhang Z, et al. G-RANS: Generalizable Residual-Aware Neural Solvers for Sparse Systems[C]. ICML, 2026, PMLR 306: 73763-73776.

[7] Bell N, Olson L N, Schroder J. PyAMG: Algebraic Multigrid Solvers in Python[J]. Journal of Open Source Software, 2022, 7(72): 4142.

Q1: 【Weakness 1】The main baseline is unpreconditioned GMRES on a single-threaded CPU, whereas the neural solvers use H100; sparse direct methods, AMG-CG/GMRES, and ILU/IC-Krylov are missing, so practical value beyond improvements among neural solvers has not yet been demonstrated.

A1: Thank you for your suggestion. We added same-GPU comparisons with strong preconditioned baselines to evaluate solver efficiency under matched hardware. The comparison focuses on preconditioned Krylov methods, consistent with the paper's study of search-space generation during iterative solution.

Specifically, we have added approximately 1k-scale indefinite Helmholtz experiments for two-dimensional low wavenumber (k0=8k_0=8), two-dimensional high wavenumber (k0=24k_0=24), and three-dimensional low wavenumber (k0=8k_0=8); both numerical and neural methods run on the same GPU. Below are the mean complete solve times ± inter-system standard deviations, with the numbers of successful systems in parentheses.

MethodTwo-dimensional low wavenumber (k0=8k_0=8): Time (s)Two-dimensional high wavenumber (k0=24k_0=24): Time (s)Three-dimensional low wavenumber (k0=8k_0=8): Time (s)
GMRES8.7489 ± 3.3776 (10/10)51.6184 ± 19.4162 (10/10)5.0123 ± 2.7995 (10/10)
FGMRES + Jacobi8.4673 ± 3.3170 (10/10)45.9039 ± 17.7388 (10/10)4.8896 ± 2.9109 (10/10)
FGMRES + AMG0.6991 ± 0.4427 (10/10)25.9626 ± 11.3193 (9/10)2.3241 ± 1.1246 (8/10)
GRANS1.1303 ± 0.1260 (10/10)1.1518 ± 0.1067 (10/10)0.6770 ± 0.1369 (10/10)
Gate-RINS0.4127 ± 0.0732 (10/10)0.6549 ± 0.0708 (10/10)0.1836 ± 0.0093 (10/10)

Gate-RINS achieves 10/10 success on all three tasks and the lowest mean solve time among the methods in the table that succeed on all systems. Its speedups over Jacobi-FGMRES are 20.52×, 70.10×, 26.63×, respectively; over the neural baseline GRANS, they are 2.74×, 1.76×, 3.69×, respectively. On the two-dimensional low-wavenumber (k0=8k_0=8) task, where all methods succeed, the speedup over AMG-FGMRES is 1.69×. For high wavenumber (k0=24k_0=24) and three dimensions, AMG succeeds on only 9/10 and 8/10 systems, respectively, and its times in the table are computed only over its successful subsets.

Comparison protocol. The relative residual threshold is 10−510^{-5}, with numerical-baseline restart/maximum iteration count 4000; neural models generate 20 directions per round, with at most 200 rounds and 4000 basis vectors. AMG uses the PyAMG classical Ruge–Stuben hierarchy and a GPU V-cycle.

AuthorsOpenReview ↗

response to reviewer TL6a (part 3)

Q2: 【Weakness 2】The tasks are small, two-dimensional, and mostly SPD, for which CG is more natural; Helmholtz in the submission is actually a positive-reaction-term system, and the large-scale Helmholtz wording in the abstract may be misleading, while strong anisotropy, high contrast, indefinite, three-dimensional, and SuiteSparse problems are missing.

A2: Please see the response to Q1 for the numerical comparisons on the newly added indefinite Helmholtz systems. CG/PCG is a natural specialized choice for SPD problems. Our minimum-residual framework targets general systems, including non-SPD problems, and the added indefinite experiments evaluate this application directly.

Thank you for pointing out the potentially misleading Helmholtz naming. The operator in the original six-task benchmark has the positive-reaction-term form A=K+MαA=K+M_{\alpha}, which is not the indefinite wave Helmholtz setting we are now adding. The new experiments use a negative-mass-term operator A=K−k2McA=K-k^2M_c and include three settings: two-dimensional low wavenumber (k0=8k_0=8), two-dimensional high wavenumber (k0=24k_0=24), and three-dimensional low wavenumber (k0=8k_0=8): the low-wavenumber reference wavenumber is k0=8k_0=8 and the high-wavenumber reference wavenumber is k0=24k_0=24, with sample variation ranges of [5.6,10.4][5.6,10.4] and [16.8,31.2][16.8,31.2], respectively. In all these approximately 1k experiments, Gate-RINS achieves 10/10 success and the lowest mean solve time in the Q1 table; for two-dimensional low wavenumber (k0=8k_0=8), the full-cohort speedup over the tested AMG configuration is 1.69×.

The Helmholtz* notation distinguishes the original approximately 100k positive-reaction-term system from the new indefinite Helmholtz experiments in the abstract, main text, and appendix. Strong anisotropy, high-contrast coefficients, and SuiteSparse matrices are directions for future evaluation.

Q3: 【Weakness 3】The contribution relative to G-RANS is mainly replacing the generator, hybrid scheduling is demonstrated only on Poisson, and Block-GRANS in the appendix sometimes outperforms Gate-RINS, so the richness of the contribution and the comparative conclusions still need explanation.

A3: Thank you for pointing out that the contribution positioning and comparative conclusions need to be clearer. Relative to GRANS, we retain the projected minimum-residual update and focus on the structural design of the search-subspace generator, solve efficiency, and its preconditioning interpretation: shared low-order polynomial residual probes replace the heavier graph-attention generation process, and lightweight pointwise gating then produces direction corrections that depend on the current residual and local features.

Specifically, the jj-th direction can be written as bt,j=Dθ,j(rt) pj(A~)rtb_{t,j}=D_{\theta,j}(r_t)\,p_j(\widetilde A)r_t, after which the minimum-residual procedure selects a linear combination of these directions. The network learns pointwise modulation of operator-dependent probes. This yields an adaptive search space generated by residual-dependent diagonally gated polynomial actions. The gate's dependence on rtr_t gives a variable, generally nonlinear preconditioning action. Q1 reports its complete solve costs on the tested indefinite Helmholtz systems.

The Poisson hybrid experiments combine GRANS's initial correction capability with the lower recurrent cost of RINS under the same projected minimum-residual update. Independent complete solve trajectories evaluate the benefit of this schedule on the tested Poisson tasks and configurations.

Block-GRANS outperforms Gate-RINS in some appendix experiments, showing a task-dependent tradeoff between generator capacity and computational cost. We compare these generators through complete solve times, training costs, and controlled ablations. The contribution is the lightweight, structured generator and its preconditioning interpretation, alongside the hybrid schedule evaluated on Poisson.

Reviewer_hEkfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RINS, a neural subspace solver for large sparse linear systems arising from PDE discretizations. The main method, Gate-RINS, generates correction directions from shared polynomial residual probes and modulates them with a residual- and coordinate-dependent pointwise gate. The paper also introduces a hybrid schedule that uses a G-RANS-style graph controller for the first correction block and Gate-RINS afterward. Experiments evaluate the method on six PDE-derived sparse linear systems, compare against G-RANS and GMRES, and include additional studies on large Helmholtz systems, scale/geometry transfer, spectral alignment, and approximate-inverse behavior.

Critical strengths
  1. The residual-image viewpoint gives the method a clear numerical interpretation. Framing a useful correction subspace through the image (AQ) and the current residual helps connect the neural basis generator to minimum-residual solvers.

  2. Gate-RINS is a reasonably concrete architectural simplification over graph-based neural basis generation. The use of polynomial residual probes and lightweight gating directly targets the overhead issue of repeatedly invoking a graph controller.

  3. The experiments cover several PDE-derived systems rather than a single toy problem. The paper reports results on six PDE benchmarks and includes additional large-scale and transfer diagnostics, which makes the empirical section more substantial than a minimal demonstration.

Critical weaknesses
  1. The practical value is not clearly established against strong classical solvers. The main benchmark explicitly uses no external preconditioner and mainly compares with unpreconditioned GMRES and G-RANS, while practical PDE solvers often rely on preconditioning, multigrid, or other problem-specific accelerators.

  2. The end-to-end cost of the learned solver remains unclear. The reported wall-clock results focus on solve time after training, but the cost of training, tuning, and amortizing the neural basis generator across PDE families is not sufficiently reflected in the main comparison.

  3. The strongest results are still limited in scale and scope. Most main timing results are at (n\approx 1000), with one separate large Helmholtz experiment and limited scale/geometry transfer tests, so the broader usefulness for realistic large sparse systems is not fully demonstrated.

  4. The methodological novelty appears moderate. Gate-RINS combines polynomial residual probes, pointwise gating, and the same projected minimum-residual update, but it is not clear that this changes the solver landscape beyond a more efficient variant of learned subspace generation.

  5. The theoretical result does not directly support the empirical performance claims. The emulation statement shows that a sufficiently expressive neural generator can match the residual quality of fixed-step GMRES under assumptions, but it does not show that the trained Gate-RINS model should outperform GMRES or strong preconditioned solvers in practice.

Questions for authors

Please see Critical Weaknesses.

4 comments
AuthorsOpenReview ↗

response to reviewer hEkf (part 1)

Thank you for your careful review and constructive feedback.

We appreciate your recognition of the numerical interpretation provided by the residual-image viewpoint, the simplification of the graph generator through polynomial probes and lightweight gating, and the experimental coverage of six PDE classes and scale and geometry transfer diagnostics. Below, we address your comments on classical baselines, full costs, scale applicability, methodological contributions, and the theoretical connection point by point.

<!-- One-to-one correspondence: Weakness 1—5 → Q1—Q5; Question 1 (please see the weaknesses) → Q6. -->

Q1: 【Weakness 1】The practical value of RINS remains unclear compared with strong classical solvers such as preconditioned and multigrid methods.

A1: Thank you for your suggestion. We added approximately 1k 2D low-wavenumber (k0=8k_0=8), 2D high-wavenumber (k0=24k_0=24), and 3D low-wavenumber (k0=8k_0=8) indefinite Helmholtz experiments, comparing GMRES, Jacobi-FGMRES, AMG-FGMRES, and GRANS, all on RTX 4090. The table below reports mean full solve time and between-system standard deviation, with the number of systems reaching the threshold in parentheses; each task contains 10 test systems.

Method2D low-wavenumber (k0=8k_0=8): time (s)2D high-wavenumber (k0=24k_0=24): time (s)3D low-wavenumber (k0=8k_0=8): time (s)
GMRES8.7489 ± 3.3776 (10/10)51.6184 ± 19.4162 (10/10)5.0123 ± 2.7995 (10/10)
FGMRES + Jacobi8.4673 ± 3.3170 (10/10)45.9039 ± 17.7388 (10/10)4.8896 ± 2.9109 (10/10)
FGMRES + AMG0.6991 ± 0.4427 (10/10)25.9626 ± 11.3193 (9/10)2.3241 ± 1.1246 (8/10)
GRANS1.1303 ± 0.1260 (10/10)1.1518 ± 0.1067 (10/10)0.6770 ± 0.1369 (10/10)
Gate-RINS0.4127 ± 0.0732 (10/10)0.6549 ± 0.0708 (10/10)0.1836 ± 0.0093 (10/10)

Using unrounded means, Gate-RINS achieves solve-time speedups of 20.52×, 70.10×, 26.63× over Jacobi-FGMRES and 2.74×, 1.76×, 3.69× over GRANS, respectively. On the 2D low-wavenumber (k0=8k_0=8) task, where all methods succeed, its speedup over AMG-FGMRES is 1.69× (0.6991 s → 0.4127 s, a reduction of 0.2864 s). The AMG means for the high-wavenumber (k0=24k_0=24) and 3D tasks summarize their successful subsets; the table reports the corresponding success rates. In all three tested settings, Gate-RINS has the lowest mean solve time among methods that succeed on all systems.

Protocol and scope. The threshold is a relative residual of 10−510^{-5}, with CGS1, numerical-baseline restart/iteration limits of 4000, and neural models generating 20 directions per iteration with at most 200 iterations and 4000 basis vectors. AMG uses the archived implementation's classical PyAMG Ruge–Stuben hierarchy, GPU V-cycle, and small coarse-level direct solve; Jacobi uses inverse-diagonal scaling.

Q2: 【Weakness 2】Reporting only inference-time solve cost does not adequately reflect the end-to-end costs of training, tuning, and amortizing the model across PDE families.

A2: RINS targets a family of systems over which the same model can be reused; if switching tasks requires retraining each time, the offline cost must be counted again. We report cumulative training wall-clock time separately from inference time; the following costs were measured on RTX 4090 over 350 training epochs.

Approximately 1k Helmholtz taskGate-RINS training (min)GRANS training (min)Gate-RINS training-time reductionGate-RINS solve (s)GRANS solve (s)
2D low-wavenumber (k0=8k_0=8)37.4054.2331.03%0.41271.1303
2D high-wavenumber (k0=24k_0=24)37.6457.6934.75%0.65491.1518
3D low-wavenumber (k0=8k_0=8)40.6054.9926.17%0.18360.6770

These records show that Gate-RINS not only solves faster than GRANS but also reduces training wall-clock time by approximately 26%–35%.

AuthorsOpenReview ↗

response to reviewer hEkf (part 2)

Q3: 【Weakness 3】The main results focus on approximately 1k systems, with one separate large-scale Helmholtz case and limited transfer tests, which are insufficient to establish broad practical usefulness for realistic large sparse systems.

A3: To examine performance as scale increases, we compare Gate-RINS, GRANS, and classical baselines on the 10k 2D indefinite Helmholtz low-wavenumber (k0=8k_0=8) and high-wavenumber (k0=24k_0=24) tasks. The records below use RTX 4090, 10 systems from the same task-specific test set for each task, and a single training seed of 2026; time is the mean full solve time ± between-system standard deviation over successful systems.

Method10k low-wavenumber (k0=8k_0=8): successful systems10k low-wavenumber (k0=8k_0=8): time (s)10k high-wavenumber (k0=24k_0=24): successful systems10k high-wavenumber (k0=24k_0=24): time (s)
GMRES10/10111.4760 ± 47.306610/10824.6460 ± 361.0820
FGMRES + Jacobi10/1094.6788 ± 39.334110/10741.7620 ± 337.7650
FGMRES + AMG10/101.1393 ± 0.556010/10122.0310 ± 153.8910
GRANS0/10Threshold not reached; no successful-run mean0/10Threshold not reached; no successful-run mean
Gate-RINS10/1016.1987 ± 8.16600/10Threshold not reached; no successful-run mean

On the 10k low-wavenumber (k0=8k_0=8) task, Gate-RINS reduces solve time by 78.4801 s relative to Jacobi-FGMRES, achieving a 5.84× speedup, and reaches the threshold on all 10 test systems on which GRANS fails to do so; AMG-FGMRES, however, remains 14.22× faster. The supported findings are therefore an improvement in convergence success rate on this low-wavenumber (k0=8k_0=8) task and a speed advantage over Jacobi.

These 10k records use the true relative residual ∥b−Ax∥2/∥b∥2≤10−5\|b-Ax\|_2/\|b\|_2\le10^{-5} at every iteration; Gate-RINS has a budget of 500 iterations, 20 directions/iteration, and 4000 basis vectors, while the numerical baselines have an iteration limit/restart of 4000.

These results show a speed advantage over Jacobi-FGMRES and improved convergence success over GRANS on the 10k low-wavenumber (k0=8k_0=8) task. Closing the performance gap to AMG while retaining low application cost is the next design target.

Q4: 【Weakness 4】It remains unclear whether the combination of polynomial probes, pointwise gating, and the existing minimum-residual update is merely an efficiency improvement to a learned subspace generator and how substantial its methodological novelty is.

A4: Thank you for pointing out the need to position the contribution more clearly. Relative to GRANS, we retain the projected minimum-residual closure and focus on the structural design of the search-subspace generator, solution efficiency, and its preconditioning interpretation: shared low-order polynomial residual probes replace the heavier graph-attention generation process, and lightweight pointwise gating then produces direction corrections that depend on the current residual and local features.

Specifically, the jjth direction can be written as bt,j=Dθ,j(rt) pj(A~)rtb_{t,j}=D_{\theta,j}(r_t)\,p_j(\widetilde A)r_t, and the existing minimum-residual procedure selects a linear combination of these directions. This structure retains the physical and algebraic information in the matrix action while the network learns pointwise adjustments to the probes. It generates an adaptive search space through residual-dependent diagonally gated polynomial actions. The gate's dependence on rtr_t makes the overall action variable and generally nonlinear.

Our contribution is a structured, lightweight learned subspace generator with a preconditioning interpretation. Q1–Q2 report its inference and training costs on the Helmholtz tasks, and Q3 reports its measured behavior at 10k.

AuthorsOpenReview ↗

response to reviewer hEkf (part 3)

Q5: 【Weakness 5】The GMRES emulation theorem is an expressive-capacity result under assumptions and cannot directly support empirical claims that trained Gate-RINS outperforms GMRES or strong preconditioned solvers.

A5: We provide empirical experimental results to demonstrate that the Gate-RINS method performs better than GMRES and the wall preconditioner. The theoretical proof will be reserved for future research. To evaluate the trained Gate-RINS correction directions, we use five independent training seeds, 2026–2030, on the original six approximately 1k tasks and recompute solve time to the first true relative residual of 10−510^{-5} on each task's 10 fixed held-out systems; Gate-RINS and GRANS both reach the threshold in all 50 seed×system records for each task.

The following are evaluations of the actual models on RTX 4090, with neural computation in FP32 and true residual checks using the original matrix values. Three timing repetitions are first averaged within each seed×system cell; the table reports mean solve time ± the combined seed×system sample standard deviation.

Approximately 1k taskGate-RINS solve (s)GRANS solve (s)
Poisson0.1148±0.01150.3826±0.0623
Helmholtz*0.1237±0.00800.3476±0.0538
Heat0.0717±0.00510.2564±0.0413
Advection–Diffusion0.1171±0.00690.4110±0.0363
Klein–Gordon0.1305±0.00700.3591±0.0379
Wave0.1350±0.00840.3769±0.0391

These results directly concern finite-width, actually trained Gate-RINS: on all six tasks, it forms correction spaces that enable the full solve to reach the threshold, and its mean solve time is lower than that of GRANS. Its relationship to strong numerical baselines is given by the same-GPU indefinite Helmholtz experiments in Q1, where Gate-RINS takes 0.4127 seconds and AMG-FGMRES takes 0.6991 seconds on the approximately 1k 2D low-wavenumber (k0=8k_0=8) task, with both succeeding on 10/10 systems; Q3 also records that Gate-RINS is slower than AMG on the 10k low-wavenumber (k0=8k_0=8) task. These experiments establish the solution efficiency of the actual architecture across different tasks and scales.

Q6: 【Question 1】Please see the Critical Weaknesses above.

A6: Please see our responses to Q1–Q5.

Thank you again for your time and careful consideration, and for the constructive suggestions that help us improve the methodological presentation and experimental comparisons.

Thanks for the detailed response. The new Helmholtz experiments against AMG/Jacobi-FGMRES, the training-time breakdown, and the 10k-scale results address most of my concerns. The clarified positioning vs. GRANS also helps. I'll raise my score accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.