- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces a novel way to approximate matrix functions using a sparse plus low-rank output combining a GNN and a size-agnostic latent-space flow-matching approach. Their approach strictly generalizes previous approaches and leads to better performance than existing data-driven preconditioners in their experiments. The authors further show the benefit of the latent space flow design and size generalization results for the model.
Critical strengths
S1: New method with good performance
The low-rank plus sparse generated output proposed is a novel contribution that outperforms existing learning methods and is supported by theory.
Critical weaknesses
W1: Clarity
The paper overall lacks clarity. In particular figure 1 is hard to follow and several of the design choices are not properly motivated. For example:
- what kind fo GNN layers are used to encode the input response embedding and how is this trained?
- between which elements is cross-attention computed to compute ?
- Why is a MoE decoder used and how exactly does this model look like?
An improved figure and more detailed description in section 3 could help to improve the clarity of the method.
W2: Experiments and their limitations
The experiments and their limitations should be emphasized more clearly. The experiment in section 4.1 should include comparisons to inexact baselines for all methods, which are currently omitted for the Schur complement part where only an exact baseline is used. At least a comparison with a Newton-Schulz method should be included.
Further, the authors should be more upfront about the experimental limitations. While the proposed method outperforms existing data-driven methods, specialized methods work better for the individual subproblems and the cost to amortize the training has not been included in these experiments. The method should be evaluated along the Pareto front of solving time and performance rather than on these metrics individually.
Questions for authors
Q1: Why is the response f(A)V_in kept fixed during trainnig? Will this not effect the actual latent space dynamics?
Q2: To what degree is Theorem 1 applicable to the trained model? Is it possible in theory for the GNN to learn this mapping or do the inherent known limitations of GNNs make this theorem not directly applicable to the learned method? Could this metric be evaluated numerically on some of the experimental data?
Q3: Could a single model be trained for all 3 of the problem families?