- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes a hybrid neural-numerical subspace solver for sparse linear systems. The approach uses residual-conditioned graph directions and an explicit polynomial numerical prior to form search directions. Incremental block orthogonalization reduces the computational cost of retraining directions across iterations without restart. Empirical studies on a synthetic benchmark and a subset of SuiteSparse problems show that the hybrid solver can generalize across held-out equation families, addressing an important problem when ML approaches applied in practice. However, it is not clear which components are responsible for the generalization capability, and the computational benefits have not been demonstrated against sufficiently competitive practical baselines.
Critical strengths
- The research problem is important and well motivated.
- The method is generally described clearly, with detailed information on architectural choices and the training objective.
- The paper includes extensive evaluation and comparisons with several baselines.
Critical weaknesses
- The benefit of the hybrid construction is not sufficiently isolated. LMM is close to G-RANS, but adds a hybrid graph–polynomial basis and an accumulated search space. Although the paper includes some component ablations, removing the polynomial branch also reduces 60 candidate directions. What happens if all 360 directions are graph-generated, or if the solver uses a polynomial-only construction? The current evidence does not fully distinguish the benefit of the hybrid combination from the benefit of a larger search space. This is important because learning reusable solver priors is the paper’s central claim.
- The classical baselines are insufficient to establish a practical advantage. The comparisons primarily use unpreconditioned methods and Jacobi-GMRES. Stronger preconditioned baselines are needed to establish whether LMM justifies the coverage advantage over conventional solvers as well as the additional cost of neural-network inference. This limits the practical significance of the results.
Questions for authors
- What defines an “equation family”? How different are the held-out families from the training families?
- How does LMM compare with an ungated polynomial-only numerical solver using the same accumulation and projection procedure? Such a comparison would help establish how much of the observed generalization comes from learning.
- Can LMM work with conventional preconditioners to further improve performance?
- How do the results change under different residual tolerances and different wall-clock budgets?
Reply to Reviewer S7uz (Part 2 of 3)
Q5 — Whole-subfamily and whole-class exclusion.
We performed genuine matched-budget retraining, rather than inference masks, while preserving the original tests. Removing all six NM1 training families leaves 100/100 withheld successes versus 100/100 for the control; the five superclass exclusions give:
Each class-out arm removes 16 training families and retains 64, using the original 60-epoch/37500-step budget and seed 43. N-minus exhibits a clear negative transfer result; the other four show no success-count drop, which does not establish equivalence or arbitrary unseen-class robustness.
Q6 — Means and spread over all three main-table seeds.
The table below uses all three archived main checkpoints, with failed/nonfinite outcomes retained in success denominators. SD is the sample SD across seeds; the residual column averages the per-seed finite-outcome geometric means, using the same 1e-30 logarithm floor.
These are the recovered paper-source runs: seed 43 used one H200 host and seeds 44/45 another, so the observed spread is not a pure training-seed-only uncertainty estimate. In particular, Syn-OOD-S instability is retained rather than presenting only the best seed.