acceptodds
Under review as a conference paper at ICLR 2027

FlowSMO: Scalable End-to-End Sparse Matrix Operators via Latent Flow Matching

Abstract

Large-scale scientific computing often requires repeatedly applying matrix operators associated with sparse matrices , including inverse and inverse-square-root operators. Classical algebraic methods, including incomplete factorizations, Krylov subspace methods, and polynomial or rational approximations, remain effective but are difficult to map efficiently to modern GPUs due to problem-specific iterations, triangular solves, and global synchronization. Learning-based methods offer a promising alternative, yet existing approaches are largely confined to inverse operators for linear system solving and often require CPU-side refinement or non-GPU-resident post-processing, breaking end-to-end GPU execution. We propose FlowSMO (Flow-based Sparse Matrix Operator), a size-generalizable GPU-resident generative framework for approximating sparse matrix operators across varying matrix dimensions. For a given operator class, FlowSMO first encodes the input sparse matrix into a dimension-agnostic conditioning representation. Conditioned on this representation, a flow-matching model samples latent operator variables with a cost independent of the matrix size. To recover an operator at the original scale, a sparse decoder predicts a local sparse component over candidate edges induced by the input sparsity pattern, while a low-rank component captures global correlations and fill-in effects. The resulting operator is represented implicitly as , enabling matrix-free application without explicitly assembling dense matrices. All stages, including generation and refinement, are executed on GPU, with computational cost scaling as for rank . We validate FlowSMO across three complementary operator settings: Helmholtz inverse operators for preconditioner generation under moderate spectral difficulty, inverse-square-root operators on sparse SPD matrices, and Schur complements as block-structured operators beyond standard spectral matrix functions. Experiments show that FlowSMO improves approximation quality while enabling GPU-resident near-linear inference and cross-scale generalization.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
learned preconditioning, scientific machine learning, sparse matrix operator learning, numerical linear algebra

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_edUUOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a novel way to approximate matrix functions using a sparse plus low-rank output combining a GNN and a size-agnostic latent-space flow-matching approach. Their approach strictly generalizes previous approaches and leads to better performance than existing data-driven preconditioners in their experiments. The authors further show the benefit of the latent space flow design and size generalization results for the model.

Critical strengths

S1: New method with good performance

The low-rank plus sparse generated output proposed is a novel contribution that outperforms existing learning methods and is supported by theory.

Critical weaknesses

W1: Clarity

The paper overall lacks clarity. In particular figure 1 is hard to follow and several of the design choices are not properly motivated. For example:

  • what kind fo GNN layers are used to encode the input response embedding and how is this trained?
  • between which elements is cross-attention computed to compute z[end]z_[end]?
  • Why is a MoE decoder used and how exactly does this model look like?

An improved figure and more detailed description in section 3 could help to improve the clarity of the method.

W2: Experiments and their limitations

The experiments and their limitations should be emphasized more clearly. The experiment in section 4.1 should include comparisons to inexact baselines for all methods, which are currently omitted for the Schur complement part where only an exact baseline is used. At least a comparison with a Newton-Schulz method should be included.

Further, the authors should be more upfront about the experimental limitations. While the proposed method outperforms existing data-driven methods, specialized methods work better for the individual subproblems and the cost to amortize the training has not been included in these experiments. The method should be evaluated along the Pareto front of solving time and performance rather than on these metrics individually.

Questions for authors

Q1: Why is the response f(A)V_in kept fixed during trainnig? Will this not effect the actual latent space dynamics?

Q2: To what degree is Theorem 1 applicable to the trained model? Is it possible in theory for the GNN to learn this mapping or do the inherent known limitations of GNNs make this theorem not directly applicable to the learned method? Could this metric be evaluated numerically on some of the experimental data?

Q3: Could a single model be trained for all 3 of the problem families?

Reviewer_u5MFOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The goal of this work is to create an efficient approximation to a matrix operator f(A), where A is a sparse matrix. A key property that the authors accomplish is that the approximation can be entirely computed with a GPU at inference time. Examples of f(A) include A^{-1} and A^{-1/2}. They learn, through a flow-based formulation, an implicit representation of the operator that does not require forming the matrix f(A) directly (and consists of sparse and low-rank components). They validate their method comparing it to competitors across different types of operators.

Critical strengths

It's an interesting goal to make these matrix operators computable on GPU only. The fact that one can apply this across different matrix sizes is quite interesting - and I like the experiments where it is trained on smaller matrices and applied to larger matrices.

Critical weaknesses
  • Table 1 has dramatic results... the ability to approximate such large sparse matrices is great. I do, however, have several questions/concerns about whether this is a fair comparison:

a) Were the "exact GPU" and "exact CPU" solvers allowed to exploit the special structure (sparse, diagonally dominant SPD)? I feel like a natural competitor for these experiments would be a sparse eigensolver (e.g. SLEPc should have these).

b) You make the point that FlowSMO doesn't assemble the full dense M but rather represents it approximately through an operator that can be applied to vectors. It doesn't feel fair to time your method for getting that approximate operator but then make your competitor have to construct the full M. Given that preexisting methods can efficiently approximate A^{-1/2}x without constructing the full A^{-1/2}, it would be better to compare to these methods. One would want to compare both approximation algorithms to a similar accuracy and then compare their times. One way to do this would be to form a set of test vectors x where you compute A^{-1/2}x. Then, letting M() denote any particular method's approximation to the operator f(A), you would report the average error |M(x)-A^{-1/2}x| and the average amount of time to go from A to computing all of the M(x)'s. The point is that many researchers know not to first compute A^{-1/2}. This makes "exact GPU" and "exact CPU" seem like unrealistic competitors. I do like that you report rho(I-MAM). The point is to make sure that this is an apples-to-apples comparison though.

  • In two of the experiments the low-rank head is disabled, so we're not really seeing an evaluation of the full method as described in the paper. While I could see it could be interesting as an ablation to disable it, this would require being able to compare to the full model.
Questions for authors
  • How sensitive is FlowSMO to "distribution drift" in A? If I train FlowSMO on sparse matrices A that have a large range of values for their nonzeros but then apply it to binary matrices (where all nonzeros are equal to 1), how well will it work? While a theoretical answer to such questions would be fascinating, at least a numerical study of this seems important.

  • If we'd like higher accuracy, can that be achieved in some way? If yes, in Section 4.1 it'd be interesting to see a plot showing the empirical tradeoff between accuracy of approximation rho(I-MAM) and time for FlowSMO.

Reviewer_Rf4tOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Flow-based Sparse Matrix Operator (FlowSMO), which is a fully GPU-resident method for approximating matrix operators associated with sparse matrices across varying matrix dimensions. The proposed approach leverages generative modeling combined with GNN-based encoders and decoders to approximate different matrix operators. The authors test their method on operators such as matrix inverses, inverse square roots, and Schur complements, showing cross-size generalization and improvements in linear solver experiments by using their proposed operator as a preconditioner.

Critical strengths

S1. I like the overall motivation of learning a reusable matrix operator for sparse matrices that is GPU-resident and can hopefully work across different matrix sizes. In general, the method seems well motivated, with the authors providing explanations and intuitions for the different components of their framework. In particular, the sparse-plus-low-rank representation is a clean way to model their predictions.

S2. The proposed two-stage training procedure seems like an interesting way to learn these matrix operators. I like the idea of learning these operator representations through supervision from small matrices, followed by joint optimization of the flow-matching model and the decoder using the operator residual loss function. This second stage avoids explicitly constructing target matrix functions, which could be expensive for large matrices.

S3. The mathematical exposition and technical writing are clear and easy to follow. The authors present some improvements in GMRES iterations and total solve time over different learned baselines. In general, the empirical results seem promising to me.

Critical weaknesses

W1. One of my main concerns is that the paper does not sufficiently discuss or establish the novelty of different components presented in the paper. For example, the method uses conditional flow matching to approximate matrix operators conditioned on global summaries. In particular, Li et al. (2024) (cited in the paper as well) propose a similar graph-conditioned variational autoencoder framework for generating sparse approximate inverse preconditioners. It seems to me that the underlying motivation is quite similar. The authors do not adequately justify the choice of flow matching over existing generative approaches, which weakens the novelty of the core proposed method.

W2. Another key concern is that the paper does not clearly establish the contribution of the learned operator relative to the additional refinement procedure. In particular, the inverse/preconditioning experiments apply three Neumann–Schulz refinement steps after decoding the operator, which is a reasonable approach to improving inverse approximations. It is unclear to me how these refinement steps are implemented while maintaining the proposed sparse-plus-low-rank representation of the output. For example, the Neumann–Schulz updates can introduce additional fill-in and low-rank directions, so it is unclear whether the authors apply these updates implicitly or perform some additional sparsification or rank truncation. I think these details are important for understanding the computational cost of applying the final preconditioner and how much of the reported improvement comes from the learned operator itself rather than the subsequent refinement.

W3. While the empirical evaluations cover different matrix operators, I think the current experiments do not sufficiently establish some of the broader claims made in the paper, particularly regarding general applicability and computational efficiency. For example, in the inverse-square-root experiments, the authors primarily compare against dense eigendecomposition, which does not seem to be the most appropriate baseline for approximating sparse matrix functions. Other practical numerical approaches, such as polynomial or Krylov-based approximations, could be more relevant baselines. Similarly, for the Schur-complement experiments, the Neumann-5 and Neumann-10 baselines achieve both lower reported approximation errors and faster construction times across all the reported matrix sizes. I think stronger numerical comparisons would help establish these claims more convincingly.

Questions for authors

Q1. In the inverse-square-root experiments, the authors report the spectral radius of the residual operator, $ \rho(I-MAM) $. What is the reasoning behind using this metric? Does it necessarily establish that the learned operator approximates the principal inverse square root A−1/2A^{-1/2} correctly? Is positive definiteness of the predicted operator explicitly enforced?

Q2. For the Schur-complement experiments, the approximation error is only evaluated on the selected sparse pattern. Schur complements can introduce significant fill-in outside this pattern. Can the authors justify this evaluation or provide some additional evaluation of the full operator approximation?

Q3. How sensitive is the proposed method to the (k)-hop sparsity neighborhood, low-rank dimension (r), and the number of operator probes (P) used during supervised initialization? Is there a relationship between these parameters and the spectral properties or condition number of the input matrix?

Q4. The authors optimize both the operator residual loss and a power-iteration-based spectral objective during the second training stage. Can the authors provide an ablation on the relative contributions of these two objectives, particularly with respect to the final GMRES iteration counts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.