Large Matrix Model: A Pretrained Sparse Solver Across Equation Families

Sparse linear solvers are traditionally specialized to the algebraic structure of their operators, and existing learned solvers largely inherit this specialization by training for individual matrices or within a single equation family. We ask whether sparse solving can instead admit a pretraining paradigm: can a single model learn reusable numerical priors that transfer across equation families and to previously unseen real-world operators? We introduce the Large Matrix Model (LMM), a pretrained neural–numerical solver that maps a sparse operator and its current residual to a correction subspace and is deployed with frozen weights. LMM combines residual-conditioned graph representations with node-wise gated Chebyshev directions, which overcome the rank ceiling of a shared polynomial dictionary, and couples the resulting subspace with restart-free incremental orthogonalization and an exact float64 minimal-residual projection. To evaluate cross-equation-family generalization, we construct a corpus of 240 equation families spanning five algebraic classes and 15 structured subfamilies, pretrain LMM on 80 families, and hold out the remaining 160 families by identity. A single frozen checkpoint converges on 1,590 of 1,600 systems from unseen equation families and, without adaptation, on 489 of 556 systems in a large SuiteSparse split disjoint from our smaller adaptation test. Under matched downstream data and compute, pretrained initialization improves convergence on the group-disjoint 200-system SuiteSparse test from 174 to 181 and reduces geometric-mean residual by $1.72\times$ relative to random initialization. These results provide evidence that reusable solver priors can be learned across heterogeneous equation families, moving learned sparse solvers from problem-specific training toward pretrained generalist numerical models.