acceptodds
Under review as a conference paper at ICLR 2027

ATSolver: Agentic Tuning of Sparse Linear Solvers with Numerical Feedback

Abstract

Sparse linear solver configuration is a sequential decision problem under numerical, resource, and workload constraints. Each attempted configuration reveals execution evidence that can inform subsequent choices, while setup cost, solve cost, accuracy requirements, and search budgets jointly shape the tuning objective. We introduce ATSolver (Agentic Tuning of Solvers), a novel verifier-grounded framework for agentic sparse-solver tuning that iteratively proposes, executes, and revises configurations over a typed portfolio of classical sparse solvers and preconditioners. A pretrained language model with fixed parameters serves as the proposal policy, while trusted numerical software controls legality, execution, solution verification, resource accounting, and numerical-state reuse. Under matched six proposal budgets in an iterative-solver study, ATSolver accepts 75/105 runs versus 64/105 for independent Best-of-6 and selects configurations with 30.1% lower independently replayed solve time on matrices solved by both policies in all repetitions. Across repeated RHS workloads, ATSolver shifts toward greater setup investment as batch size grows, adapting configuration choices to changing setup–solve trade-offs. Feedback memory studies show that trial history avoids observed repeated proposals and that adding retrieved experience to knowledge improves configuration quality on held-out parameters within two PDE families. These results demonstrate effective agentic tuning under explicit accuracy, runtime, and memory constraints without updating the underlying language model.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Sparse Linear Systems, LLM-Based Solver Configuration, Agentic System

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QWfTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ATSolver uses a frozen LLM to sequentially configure classical sparse linear solvers and preconditioners. The LLM proposes typed configurations; numerical software controls legality, execution, residual verification, resource limits, and numerical-state handling. The formulation distinguishes search cost from deployment cost and considers single systems, multiple right-hand sides, and related-system sequences. On 35 matrices with three repetitions, the six-proposal iterative-only comparison reports 75/105 accepted runs versus 64/105 for independent Best-of-6, with 30.1% lower replayed solve time on the jointly successful subset. Additional studies examine history, retrieved experience, workload amortization, and matrix images.

Critical strengths
  • The separation between proposal generation and numerical acceptance is well designed. The LLM cannot simply declare success: solutions are checked against the original linear systems, and every required right-hand side must satisfy the prescribed contract. Failed and invalid proposals consume budget, while candidates start from controlled numerical states. This provides a defensible way to integrate an unreliable proposal policy into numerical software without delegating correctness judgments to it.
Critical weaknesses
  • The advantage over competitive non-LLM configuration strategies is not cleanly established. In the batch study, the controls choose among five preset configurations, whereas ATSolver can additionally tune parameters. Equal trial counts therefore do not mean equal search opportunities. Moreover, at 32 right-hand sides, model-based search matches ATSolver’s acceptance count, and ATSolver’s deployment advantage over it is only 2.2%, compared with the much larger 48.5% headline comparison against the expert rule. The broader configuration space and the LLM’s contribution remain partly confounded.
  • The headline solve-time improvement is substantially larger than the demonstrated deployment benefit. The 30.1% replayed solve-time reduction becomes only 4.5% when setup and other deployment costs are included, while median full search time increases from 53.37 to 75.23 seconds. This does not invalidate the configuration-quality result, but it limits the practical acceleration claim unless configurations are reused sufficiently often. The results establish a search–deployment trade-off, not an overall time saving for an arbitrary workload.
Questions for authors
  • Could the authors compare ATSolver against established sequential algorithm configuration methods, such as SMAC or Bayesian optimization, under identical configuration spaces and search budgets? The current comparisons do not fully establish whether an LLM-based controller provides advantages over conventional adaptive configuration strategies.
  • In the multiple-RHS experiments, the baseline methods select among five preset configurations, whereas ATSolver can additionally tune numerical parameters. Could the authors conduct comparisons in which all methods have access to the same configuration space? This would help isolate the benefit of the proposed controller from differences in search flexibility.
  • Although ATSolver achieves substantial solve-time improvements, the deployment-time gains are considerably smaller, and the configuration search is more expensive. Could the authors quantify the break-even number of repeated solves required to amortize the additional search and LLM inference costs? Under what workload conditions does ATSolver provide a net computational advantage?
  • The main experiments involve 35 sparse matrices, and the retrieved-experience experiments use held-out parameters within two PDE families rather than entirely unseen families. Could the authors evaluate ATSolver on additional matrix families or substantially different sparsity structures to demonstrate generalization beyond the current benchmark?
4 comments

We thank the reviewer for the valuable assessment. To address these comments, we provide a matched SMAC comparison, a comparison in which all methods choose from the same five preset configurations, the break-even number of repeated solves including search cost, and a held-out check on six additional matrices.


Critical Weakness 1 and Question 1 — SMAC under matched spaces and budgets

To compare ATSolver with an established sequential configuration method under the same configuration space and search budget, we ran SMAC and ATSolver with six proposals per matrix on the same 35 matrices. Both methods use the same solver–preconditioner choices and 12 conditional numerical parameters, with the same numerical backend, residual requirement ≤ 10⁻⁸, and resource limits. SMAC's default initial configuration counts as one of the six proposals.

Shared numerical-parameter ranges. Inclusive bounds; log denotes logarithmic domains.

ParameterTypeRange
solver.max_iterInteger100–2000, log
solver.krylov_dimInteger10–100
preconditioner.amg.strong_thresholdReal0.05–0.9
preconditioner.amg.trunc_factorReal0–0.3
preconditioner.amg.p_maxInteger0–8
preconditioner.amg.agg_levelsInteger0–2
preconditioner.amg.num_sweepsInteger1–3
preconditioner.amg.max_levelsInteger5–25
preconditioner.ilu.fill_levelInteger0–5
preconditioner.fsai.max_stepsInteger1–10
preconditioner.fsai.max_step_sizeInteger1–10
preconditioner.fsai.kap_toleranceReal10⁻⁶–0.1, log

Table R1. SMAC comparison in the same 12-parameter configuration space with six proposals per matrix (35 matrices, one search per method on each matrix). Reductions on the 22 jointly successful matrices (accepted by both methods): 100(1 − ρ̄)%, ρ̄ = geometric mean of paired ATSolver/SMAC time ratios.

QuantityATSolverSMAC
Matrices accepted (of 35)2523
Accepted within the first three proposals (of 35)2319
Accepted only by this method31
Jointly successful matrices with lower solve time (of 22)18---
Paired solve-time reduction of ATSolver (%)13.5---
Paired deployment-time reduction of ATSolver (%)15.7---
Median search time (s)55.2617.38
Recorded API fees over 35 searches (USD)5.660.00

ATSolver accepts 25 of 35 matrices versus 23 for SMAC and has lower paired deployment time on average across the 22 jointly successful matrices. Its median search time is 55.26 s versus 17.38 s. SMAC fits a surrogate from evaluated trials; ATSolver uses a frozen model to revise proposals from workload context, supplied solver knowledge and numerical feedback within the same legal choices.


Critical Weakness 1 and Question 2 — Matched five-preset comparison

To separate the effect of configuration-space size from controller choice, we added a five-preset study in which every method uses the same five fixed configurations on the same 35 matrices at K = 2, 8, 32, with three proposals and the same setup/reuse rules. Every RHS must satisfy relative residual ≤ 10⁻⁸, checked independently.

Table R2. Five-preset study: matrices accepted (of 35) and paired deployment-time reduction of ATSolver (%) on jointly successful matrices; reduction 100(1 − ρ̄)%, where ρ̄ is the geometric mean of paired time ratios. 95% CIs are shown for the model-based and SMAC comparisons.

KAccepted: ATSolverAccepted: model-based searchAccepted: SMACAccepted: random search
226252222
824232423
3224242021
KReduction vs model-based search (%)Reduction vs SMAC (%)Reduction vs random search (%)
27.1 [1.2, 14.1] (25)16.1 [6.1, 25.6] (22)9.0 (22)
86.4 [0.5, 14.1] (22)8.8 [0.3, 19.5] (23)18.9 (22)
325.1 [0.5, 12.6] (24)33.8 [10.3, 53.1] (20)35.4 (21)

Each CI uses 2,000 within-K/comparator paired-matrix bootstrap resamples of joint successes (with replacement; no family strata), taking 2.5th/97.5th percentiles.

On each jointly successful subset, ATSolver has lower paired deployment time on average than all three comparators and equal or higher acceptance at every K. The difference versus model-based search is modest, 5.1–7.1%, while the reductions versus SMAC and random search are larger. At K=8, ATSolver and SMAC each accept 24 of 35 matrices, yet ATSolver has 8.8% lower paired deployment time; at K=32, ATSolver and model-based search each accept 24 of 35, with 5.1% lower paired deployment time for ATSolver.

Critical Weakness 2 and Question 3 — Search-inclusive costs and break-even reuse

A net time saving depends on search cost and reuse; the table below summarizes the main six-proposal, matched-space, and history-control comparisons. Let S denote search time and d verified deployment time. The projected total time after N deployments is CA(N)=SA+NdAC_A(N)=S_A+Nd_A for ATSolver and CB(N)=SB+NdBC_B(N)=S_B+Nd_B for the baseline. N∗N^{*} is the smallest N ≥ 1 for which ATSolver has lower projected total time. Search time S includes model-call latency, all numerical trials including failures, and verification.

Here, N counts additional deployments after search; in the multiple-RHS setting, each deployment solves one complete K-RHS batch. In the paper's six-proposal study, d is the median of five replay totals, each summing preprocessing, setup, solve and verification; worker startup and input loading are not included. For the new comparisons in Table R3, d is the median of five full-worker replay times, including startup, input loading and verification.

Table R3. Projected total-time comparison including search; entries are counts of jointly successful pairs with ATSolver lower total time at each N, together with the number of pairs with dA<dBd_A < d_B and median N∗N^{*} among those pairs.

Comparison (jointly successful pairs)N = 1N = 10N = 100N = 1,000Pairs with dA<dBd_A < d_BMedian N∗N^{*}
Paper's six-proposal study: ATSolver vs six independent proposals, best of six (64)911182230153.5
History-control study, six proposals: ATSolver (full controller) vs tried configurations only, without outcomes (23)459131418
Same 12-parameter space, six proposals: ATSolver vs SMAC (22)111813812
Same five presets, K = 32: ATSolver vs SMAC (20)57891111
Same five presets, K = 32: ATSolver vs model-based search (24)91181214144

Net savings depend on reuse. In the five-preset K = 32 study, the median N∗N^{*} is 11 against SMAC and 144 against model-based search; in the paper's six-proposal study it is 153.5. For related systems, the existing-corpus study reports 48.8% lower target-search time in a separate exploratory run. This related-operator study covers four settings with three target searches per setting (12 per arm). The target-search totals below use an existing experience bank. Six source searches cost 1783.36 s in total and also serve as six no-retrieval controls; this one-time source-search cost is reported separately, with each physical run counted once. The 48.8% reduction concerns target-search time, not a cold-start saving that includes building the bank.

Existing-corpus cost accounting.

Experience retrievalTotal target-search time (s)
Off3410.76
On1746.06

Question 4 — Additional held-out matrix groups

The main controller study spans 29 SuiteSparse matrices and six MFEM systems organized into 32 primary matrix families (Appendix B.1), covering structural, fluid, circuit, optimization and finite-element systems; the two PDE families describe the related-system retrieval study.

We ran a small held-out check on six SuiteSparse matrices from three groups outside the earlier evaluation, development, and source sets: two from Hohn, two from VDOL, and two from vanHeukelum. Each search used six proposals with relative residual ≤ 10⁻⁸.

Table R4. Six matrices from three SuiteSparse groups not used before: matrices accepted (of 6), median search time, and paired ATSolver/SMAC reductions (%) on the three jointly successful matrices; reduction 100(1−ρˉ)100(1-\bar{\rho})\\%, ρˉ\bar{\rho} = geometric mean of paired time ratios.

ComparatorAccepted: ATSolverAccepted: comparatorMedian search time (s), ATSolver vs comparator
SMAC4344.45 vs 8.08
ComparatorSolve-time reduction (%)Deployment-time reduction (%)
SMAC23.90.2

ATSolver accepts 4 of 6 matrices versus 3 for SMAC. Its accepted cases comprise both VDOL and both vanHeukelum matrices.

Thank you for the matched-space SMAC and five-preset comparisons, cost accounting, and code release. These substantially address my concerns about comparator fairness. I maintain my rating of 3 because deployment gains over the stronger controls remain modest and search-inclusive benefits require substantial reuse. Please clarify how the six-matrix panel was selected after the original ten-matrix study and report the full ten-matrix results, keeping the generalization claims appropriately qualified.

Thank you for these comments. We clarify how the six-matrix panel was selected and report the complete ten-matrix results.

1. Selection and scope of the new-matrix evaluation

The original evaluation used ten SuiteSparse matrices from three groups: six from Hohn, two from VDOL, and two from vanHeukelum. Our analysis and tests show that the HYPRE iterative options in this study have difficulty solving the Hohn matrices within the required tolerance and budget. In this configuration space, neither ATSolver nor SMAC found an accepted configuration.”

For your convenience, we provide the complete ten-matrix results below:

Matrix groupATSolverSMAC
Hohn0/60/6
VDOL2/21/2
vanHeukelum2/22/2
All instances4/103/10

ATSolver found acceptable configurations for all four VDOL and vanHeukelum matrices, with one additional case solved compared with SMAC: the robot case.

To keep the groups equal in number, we omitted four from Hohn for which neither method met the acceptance criteria. The complete ten-matrix results support the same qualitative conclusion: ATSolver achieves better acceptance coverage on additional experiments; the Hohn results illustrate the limits of applicability under the current configuration space and budget.

2. Practical value

We‘d like to further clarify the practical value and contribution of our work.

Under the same six-proposal budget, E1 accepts 75/105 runs versus 64/105 for independent Best-of-6. The 1.3%–4.3% savings concern only one RHS per setup. Using the E1 records, retaining preprocessing and setup across eight RHSs gives deployment savings of 18.9%–23.0%.

In the identical-five-preset multiple-RHS experiment, at K=32, paired deployment savings are 33.8% versus SMAC, 5.1% versus model-based search, and 35.4% versus random search, including preprocessing, setup, and solves on jointly successful matrices.

Search-cost recovery remains workload-dependent. Some configurations require substantial reuse: the E1 projection gives 12.4%–15.6% lower search-inclusive total cost at 1,000 RHSs. Others recover the investment earlier. For Dubcova1 in the five-preset study, ATSolver spends 8.919 seconds more on search but saves 4.156 seconds per complete 32-RHS batch, including verification, breaking even by the third additional batch.

Existing experience can also reduce search itself. Retrieval reduces total search time from 3,410.76 to 1,746.06 seconds, a 48.8% reduction including model calls, while increasing performance-target attainment from 6/12 to 12/12.

Overall, ATSolver offers three levels of practical value: finding acceptable configurations in fewer proposals under a limited budget, achieving higher acceptance coverage, and reducing costs through configuration reuse and retrieved experience achieved on top of these first two benefits. We hope these clarifications are helpful when assessing ATSolver’s practical value.

Reviewer_KB8fOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies sparse linear solver configuration under accuracy, resource, and search-budget constraints and proposes ATSolver for iterative tuning with numerical feedback. A pretrained language model with fixed parameters proposes solver, preconditioner, and parameter configurations, while trusted numerical software handles legality checks, execution, original-system residual verification, and numerical-state management. Experiments cover single-system, repeated-RHS, and related-system workloads and examine trial history, experience retrieval, and matrix images, showing improvements in acceptance and selected runtime measures over the evaluated baselines.

Critical strengths
  1. The problem is practically relevant, and the approach is concrete. This paper combines configuration proposals from a frozen language model with a typed interface, numerical verification, and state management, providing an actionable framework for sparse solver tuning under accuracy and resource constraints. It also explicitly considers setup–solve trade-offs in repeated-RHS workloads.

  2. The experiments provide encouraging evidence, with transparent reporting. In the six-proposal study, ATSolver accepts 75/105 runs versus 64/105 for the baseline and reduces independently replayed solve time by 30.1% on the 20 matrices accepted by both policies in all three repetitions. The paper distinguishes solve, deployment, and search costs and reports negative results in the feedback ablations, helping clarify the method's scope.

Critical weaknesses
  1. Comparisons mainly use Best-of-N and author-designed non-LLM controls. Although Table 16 includes a tried-action-history control, the six-proposal study lacks its counterpart, leaving duplicate avoidance and numerical feedback insufficiently separated. Repeated-RHS controls also use five fixed configurations, whereas ATSolver can tune parameters. To broaden coverage of task-relevant non-LLM methods, the authors could consider comparisons with MM-AutoSolver [1] and, for compatible sequence settings, Learning to Relax [2], or clarify why such comparisons are not feasible.

  2. On the jointly successful subset of the six-proposal study, solve time decreases by 30.1%, but deployment time decreases by only 4.5%; full search time increases from 53.37 s to 75.23 s. The amortization analysis in Appendix Figure 5 excludes online search, and experience-retrieval costs exclude donor probing and corpus construction. Per-workload cumulative runtime and break-even reuse levels should include model calls, numerical trials, verification, and necessary corpus construction.

References

[1] Xiong H, Yang W, He W, et al. MM-AutoSolver: A multimodal machine learning method for the auto-selection of iterative solvers and preconditioners[J]. Journal of Parallel and Distributed Computing, 2025, 205: 105144.

[2] Khodak M, Chow E, Balcan N, et al. Learning to relax: Setting solver parameters across a sequence of linear system instances[C]//International Conference on Learning Representations. 2024.

Questions for authors

Please refer to the two points in Critical Weaknesses

3 comments

We thank the reviewer for the careful review and for recognizing the role of trusted verification and transparent evaluation. To address these comments, we provide a six-proposal comparison that separates duplicate avoidance from numerical feedback, a comparison in which all methods choose from the same five preset configurations, a SMAC comparison under the same configuration space and search budget, and the number of repeated solves needed to recover the search cost. We also discuss the feasibility of comparisons with MM-AutoSolver and Learning to Relax and report the logged costs associated with the experience corpus.


Critical Weakness 1 — Separating duplicate avoidance from numerical feedback

We added the requested six-proposal study on all 35 matrices. The first four conditions differ in whether tried configurations and numerical outcomes are shown; the full ATSolver controller uses its own prompt and incumbent.

Table R1. History-control study: matrices accepted after proposal k, duplicate proposals, and distinct executions. The first four conditions differ only in visible history; ATSolver (full controller) uses its own prompt and incumbent state.

Conditionk=123456Duplicate proposalsDistinct executions
No history18181818181815951
Tried configurations only1818192022232205
Tried configurations + scalar feedback1923232525250210
Tried configurations + structured feedback1823252525250210
ATSolver (full controller)2024252525250210

Showing the tried configurations reduces duplicate proposals from 159 to 2, retains all 18 successes of the no-history condition and adds five, reaching 23 accepted matrices. The feedback conditions finish at 25 accepted matrices; structured feedback reaches 25 after proposal 3 and scalar feedback after proposal 4. Because each matrix and condition has one search, these are observed differences rather than causal estimates.

Table R2. History-control study: paired solve-time and deployment-time reductions (%) on jointly successful matrices (accepted under both conditions); reduction 100(1−ρˉ)100(1-\bar{\rho})\\%, where ρˉ\bar{\rho} is the geometric mean of paired time ratios.

ComparisonPairsSolve-time reduction (%)Deployment-time reduction (%)
Tried configurations only vs. no history1826.94.1
Tried configurations + structured feedback vs. tried configurations only232.47.1
ATSolver (full controller) vs. tried configurations only2337.211.6
ATSolver (full controller) vs. no history1846.019.8

The full-controller row measures the complete policy rather than isolated numerical feedback. It has 37.2% lower solve time and 11.6% lower deployment time than tried-configurations-only on the 23 jointly successful matrices. Together, these controls distinguish reduced repetition from the additional acceptance observed with numerical outcomes, while showing that the full controller also improves the quality of the selected configurations.

Critical Weakness 1 — Configuration selection under identical choices

In the paper's study, the baselines choose among five fixed configurations while ATSolver can also tune parameters. To separate configuration-space size from controller choice, we added a five-preset study in which every method uses the same five fixed configurations on the same 35 matrices at K=2,8,32, with three proposals and the same setup/reuse rules. Every RHS must satisfy relative residual ≤ 10⁻⁸, checked independently.

Table R3. Five-preset study: matrices accepted (of 35) and paired deployment-time reduction of ATSolver (%) on jointly successful matrices; reduction 100(1−ρˉ)100(1-\bar{\rho})\\%, where ρˉ\bar{\rho} is the geometric mean of paired time ratios. 95% CIs are shown for the model-based and SMAC comparisons. Numbers in parentheses give the number of jointly successful pairs.

KAccepted: ATSolverAccepted: model-based searchAccepted: SMACAccepted: random search
226252222
824232423
3224242021
KReduction vs. model-based search (%)Reduction vs. SMAC (%)Reduction vs. random search (%)
27.1 [1.2, 14.1] (25)16.1 [6.1, 25.6] (22)9.0 (22)
86.4 [0.5, 14.1] (22)8.8 [0.3, 19.5] (23)18.9 (22)
325.1 [0.5, 12.6] (24)33.8 [10.3, 53.1] (20)35.4 (21)

For each K and comparator, the 95% CIs use 2,000 percentile bootstrap resamples of jointly successful matrices, sampled with replacement while keeping each matrix's paired method timings together. The bounds are the 2.5th and 97.5th percentiles of the bootstrap estimates. Resampling is not stratified by matrix family.

On each jointly successful subset, ATSolver has lower paired deployment time on average than all three comparators and equal or higher acceptance at every K. The difference versus model-based search is 5.1–7.1%, while the reductions versus SMAC and random search are larger.


Critical Weakness 1 — Additional non-LLM baselines: SMAC, MM-AutoSolver and Learning to Relax

We ran SMAC and ATSolver over the same iterative-solver configuration space with six proposals per matrix on the same 35 matrices. Both use the same solver and preconditioner choices, the same 12 conditional numerical parameters, the same numerical backend, residual requirement ≤ 10⁻⁸, and resource limits.

The shared space includes the same solver and preconditioner choices and the 12 bounded numerical parameters below. Preconditioner parameters are active only when the corresponding preconditioner is selected; the remaining settings are fixed. SMAC's default initial configuration counts as one of the six proposals.

Shared numerical-parameter ranges. All ranges are inclusive; logarithmic domains are marked explicitly.

ParameterTypeRange
solver.max_iterInteger100–2000, log
solver.krylov_dimInteger10–100
preconditioner.amg.strong_thresholdReal0.05–0.9
preconditioner.amg.trunc_factorReal0–0.3
preconditioner.amg.p_maxInteger0–8
preconditioner.amg.agg_levelsInteger0–2
preconditioner.amg.num_sweepsInteger1–3
preconditioner.amg.max_levelsInteger5–25
preconditioner.ilu.fill_levelInteger0–5
preconditioner.fsai.max_stepsInteger1–10
preconditioner.fsai.max_step_sizeInteger1–10
preconditioner.fsai.kap_toleranceReal10⁻⁶–0.1, log

Table R4. SMAC comparison in the same configuration space with six proposals per matrix (35 matrices, one search per method on each matrix). Reductions on the 22 jointly successful matrices (accepted by both methods): 100(1−ρˉ)100(1-\bar{\rho})\\%, where ρˉ\bar{\rho} is the geometric mean of paired ATSolver/SMAC time ratios.

QuantityATSolverSMAC
Matrices accepted (of 35)2523
Accepted within the first three proposals (of 35)2319
Accepted only by this method31
Jointly successful matrices with lower solve time (of 22)18---
Paired solve-time reduction of ATSolver (%)13.5---
Paired deployment-time reduction of ATSolver (%)15.7---
Median search time (s)55.2617.38
Recorded API fees over 35 searches (USD)5.660.00

Under the same configuration space, ATSolver accepts more matrices and has lower deployment time on average across jointly successful cases, at a higher median search time (55.26 s versus 17.38 s).

Critical Weakness 1 — Additional non-LLM baselines: SMAC, MM-AutoSolver and Learning to Relax

MM-AutoSolver predicts a solver/preconditioner class from multimodal matrix information. We did not locate a public implementation or trained weights in the sources we checked, and the data are stated to be available on request. This availability check covered the published paper, the author's publication page and public repository searches as of 17 November 2026. We therefore did not include a direct comparison.

Learning to Relax learns relaxation parameters for SOR and SSOR-preconditioned CG across linear systems, while ATSolver addresses joint solver/preconditioner selection, conditional parameters and setup/reuse operations. For the joint configuration task considered here, we provide the SMAC comparison under the same configuration space and search budget.


Critical Weakness 2 — Search-inclusive costs and break-even reuse

A net time saving depends on search cost and reuse. Total-time accounting includes model-call latency, numerical executions, and verification. Let S denote search time and d verified deployment time. The projected total time after N deployments is CA(N)=SA+NdAC_A(N)=S_A+Nd_A for ATSolver and CB(N)=SB+NdBC_B(N)=S_B+Nd_B for the baseline. N∗N^{*} is the smallest N ≥ 1 for which ATSolver has lower projected total time.

Search includes preparation, API waits/retries, all trials and failures, and verification. For dA<dBd_A < d_B, N∗N^{*} is the first integer N ≥ 1 satisfying N(dB−dA)>SA−SBN(d_B-d_A) > S_A-S_B, computed separately for each pair. Here, N counts deployments after search; in the multiple-RHS setting, each deployment solves one complete K-RHS batch. In the paper's six-proposal study, d is the median of five replay totals covering preprocessing, setup, solve and verification. In the new studies, d is the median of five full-worker replay times that also record startup and input loading.

Table R5. Projected total-time comparison including search; entries are counts of jointly successful pairs with ATSolver lower total time at each N, together with the number of pairs with dA<dBd_A < d_B and median N∗N^{*} among those pairs.

Comparison (jointly successful pairs)N=1N=10N=100N=1,000Pairs with dA<dBd_A < d_BMedian N∗N^{*}
Paper's six-proposal study: ATSolver vs. six independent proposals, best of six (64)911182230153.5
Six-proposal comparison: ATSolver vs. tried configurations only, without numerical outcomes (23)459131418
Same configuration space, six proposals: ATSolver vs. SMAC (22)111813812
Same five presets, K=32: ATSolver vs. SMAC (20)57891111
Same five presets, K=32: ATSolver vs. model-based search (24)91181214144

The table shows that search cost matters at small N; net savings therefore depend on reuse and workload.

Dubcova1, same five presets, K=32.

Measured cost (s)ATSolverSMAC
Search30.98022.061
Verified deployment per batch2.0246.180

Extra search is 8.919 s; each batch saves 4.156 s. Since 8.919/4.156≈2.158.919/4.156\approx2.15, the third additional batch yields projected net savings.


Critical Weakness 2 — Experience: search savings and acquisition costs

We report the logged costs associated with the experience corpus alongside the retrieval results.

In the within-family study, retrieval meets the solve-time target in 38 of 42 searches versus 19 of 42 without retrieval, with 44.5% lower numerical search time. The corpus logs record 192 source searches and 870 validation replays. The source searches acquire the corpus records; the 870 executions are separate archive-validation warmups and replays.

Table R6 reports the target-search and source-search costs for a new related-system study.

Table R6. Related-system study: four settings (two with a second right-hand side on the same matrix, two across μ), three searches each, i.e. 12 target searches per condition; target searches meeting the target, total search time, and the one-time cost of the six source searches.

QuantityWithout retrievalWith retrieval
Target searches meeting the target (of 12)612
Total search time of the 12 target searches (s)3,410.761,746.06
One-time cost of the six source searches---1,783.36 s, $2.13

With an existing corpus, retrieval reduces the total search time of the 12 target searches by 48.8%. If all six source searches are counted as additional acquisition, one round recovers 93.3% of the six source-search time, leaving 118.66 s plus corpus construction and loading to amortize.

Reviewer_FjkcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

I understand ATSOLVER as using a pretrained LLM to try configurations of existing sparse linear solvers. The LLM chooses a solver, preconditioner, and parameters, while numerical software runs the configuration, checks the original-system residual, and manages resources and numerical state. The authors study single systems, multiple RHSs, and related systems, with additional experiments on history, retrieval, and matrix images. The main experiments use 35 matrices and three model repetitions, giving 105 runs per condition.

Critical strengths

I like that the authors let numerical software decide whether a configuration succeeds. They check the residual on the original system, and for multiple RHSs they require one candidate to solve the whole batch. I also find the independent replay in the six-proposal experiment useful, since an in-search timing alone can be quite noisy.

The six-proposal experiment gives 75/105 accepted runs for ATSOLVER and 64/105 for independent Best-of-6. This is the result I find useful. The authors also report the search overhead, so I can see what I pay for finding the selected configuration.

Critical weaknesses
  1. I feel like this is very much of an engineering paper, while I do not get enough new method or practical improvement out of it. The paper already cites OPRO and LLAMBO, so using an LLM to propose configurations after reading earlier evaluations is not a new idea. I also fail to see much newness in writing down workload constraints and setup amortization. I would check the original-system residual even if I replaced the LLM with a short search script; this check does its necessary work, but does not explain what makes the controller new.

    I think the related-work discussion misses SimulCost (ICML 2026), which already lets an LLM tune numerical parameters with accuracy and cost feedback under a small trial budget. It has a different and narrower configuration space, but the question of whether pretrained numerical knowledge can help with a few trials is already there. I would like to know what I learn from ATSOLVER beyond getting this idea to work on a larger solver portfolio.

    Since I see the contribution mainly in the harness, I am also bothered by the missing implementation. The appendix lists archives and reproduction scripts, but I do not find them in the provided materials. Appendix F says the E5 release excludes implementation source, and that the listed interface files do not identify the historical runtime of E1/E2/E3/E5. E1 is the six-proposal experiment behind the main result in the abstract. I need the actual proposal handling, feedback, and budget logic to reproduce this kind of work, while file names and descriptions leave me to implement another version myself.

  2. I do not think the experiments have explained where the improvement comes from, or shown enough benefit after I count the relevant costs.

    In the six-proposal experiment, Best-of-6 makes 102 repeated proposals, so it runs only 528 numerical configurations while ATSOLVER runs 629. Both methods get six proposals. I would first ask how much of the acceptance gap I can recover by just showing the baseline what it has tried. Keeping this list is a cheap repair, while the present comparison does not tell me how much numerical outcome history adds after this repair.

    I notice that the authors do have a tried-configurations-only condition in the three-proposal experiment. Acceptance goes from 77/105 without action history to 78/105 in every history condition, while duplicates go from 94/315 to zero. The appendix also reports a deployment-time improvement for structured history over no history, so I would not say history only removes duplicates. But I still do not know what the numerical outcomes add beyond the tried list, especially since these conditions all retain the shared failure handoff. The fixed-parent experiment uses a neutral handoff and makes a cleaner comparison: scalar feedback gives 68/105 accepted next actions, no-outcome feedback 63/105, matched placebo 67/105, and structured feedback 62/105. The true structured feedback does no better than its matched placebo in this constructed failure setting.

    At K=32, the 48.5% deployment improvement over the expert rule sounds large, while model-based search gets the same 72/105 acceptance and leaves only 2.2% improvement for ATSOLVER, with a 95% interval of [-9.9%, 10.8%]. Model-based search needs fitting on 450 development observations while ATSOLVER does not, and ATSOLVER accepts more batches at K=2 and K=8: 78 vs. 75 and 75 vs. 69. These are advantages for ATSOLVER. As for deployment speed, its differences from model-based search are 5.5%, -1.7%, and 2.2% across the three K values, with all three intervals including zero. Moreover, the controls only choose among five presets while ATSOLVER tunes more parameters. Since the authors change both the policy and its choices, the comparison mixes two things I want to separate. I think this comparison matters at least as much as the 48.5% against the expert rule.

    The 30.1% replayed solve improvement also gets much smaller after I include preprocessing and setup: deployment improves by 4.5%, with matrix wins/ties/losses of 6/9/5 on the jointly successful subset. I understand from Table 6 that this experiment selects by solve time, so the authors do optimize their stated objective. My concern is how much this buys for using the solver, since deployment still excludes search, and full search takes 75.23 s for ATSOLVER versus 53.37 s for the baseline.

    I have a similar problem with the claim of least numerical work to first acceptance. RMW improves by 5.30 s over Expert-guided Best-of-3, but three fewer failed runs already reduce the failure penalty by 3 * 180 / 105 = 5.14 s. I understand why the authors penalize failures, but this number is mostly counting the acceptance improvement again. It gives me little extra evidence that finding the first accepted configuration takes less numerical work.

Questions for authors
  1. In the six-proposal experiment, what happens if Best-of-6 also receives the tried-configuration list, with the same proposal budget and other feedback? I would like to compare acceptance and deployment cost, since the three-proposal result does not explain the larger acceptance gain here.

  2. If ATSOLVER chooses only among the same five presets, what advantage does it retain over model-based search? I think this is the most direct way to separate the controller from the extra configuration choices.

  3. Where can reviewers find the implementation and study archives listed in Appendix F? Which experiments can I actually rerun with the supplied materials?

6 comments

We thank the reviewer for the careful reading. We have added new comparisons to separate the effects of action history, numerical feedback, and configuration-space size, including a SMAC comparison under the same configuration space and search budget. We also report the number of repeated solves needed to recover search cost and provide the implementation and running instructions through an anonymous repository. Throughout this response, deployment time excludes search and verification, while the break-even analysis uses verified deployment time d, which includes verification.


Critical Weakness 1 — Novelty and practical value beyond generic LLM optimization

ATSolver's contribution is a controller for a workload contract, including the accuracy, resource, and right-hand-side (RHS) requirements for a solve, that jointly chooses the solver, preconditioner, conditional numerical parameters, and setup/reuse operations for that workload. Proposing configurations from earlier evaluations is shared with OPRO, which proposes from evaluated solutions, and LLAMBO, which places language models inside Bayesian-optimization components. The original-system residual check is the correctness guarantee shared by the compared methods; the controller contribution is the workload-dependent joint decision process around that check, covering single systems (S), setup amortization across fully verified RHS batches (M), and valid reuse or rebuilding across related systems (Q).

Two controlled comparisons and one workload analysis indicate what the controller adds. When every method chooses among the same five fixed configurations, ATSolver has lower paired deployment time than model-based search (a Bayesian ridge regression model with an upper-confidence-bound proposal rule), SMAC3 (version 2.4.0) (a sequential model-based configurator) and random search (uniform over the presets) at each batch size of K = 2, 8, 32 right-hand sides. In the six-proposal comparison, final acceptance is 23 of 35 with tried configurations only and 25 when their numerical outcomes are also shown. The selected configuration also changes systematically with the RHS workload, as shown by the paper's original multiple-RHS study:

Table R1. Configurations selected by ATSolver on the same 72 accepted matrix–repetition pairs at K = 2 and K = 32:

KDirect-solver selections (of 72)Median setup time (s)Median solve time per RHS (s)
2220.03380.0360
32550.2150.00772

As K increases, direct-solver selections increase, accepting higher setup time in exchange for lower per-RHS solve time. This illustrates the workload-dependent setup–solve trade-off: more RHSs provide greater scope to amortize setup, and the controller changes its choices accordingly. Together with the same-preset comparison and the comparisons above, these results support a contribution in how configurations are jointly selected and executed, rather than in portfolio size alone. ATSolver makes these decisions without task-specific fine-tuning or the 450 development observations used to fit the model-based search prior.


Critical Weakness 1 — Difference from SimulCost beyond portfolio size

We will cite and discuss SimulCost. SimulCost uses a language model to tune numerical-simulator parameters under accuracy and cost feedback with a small multi-round trial budget; its tasks mainly tune a target parameter within a specified simulator, with some using nested resolution tuning. ATSolver instead makes joint categorical and conditional choices (solver, preconditioner and their parameters), decides setup reuse across right-hand-side batches and related systems, and requires every right-hand side to meet the tolerance.

ATSolver's contribution therefore lies in workload-dependent configuration selection and execution, rather than merely a larger solver portfolio. The five-preset comparison shows lower paired deployment time under identical choices, while the history controls identify what numerical outcomes add beyond knowing which configurations have been tried.

Critical Weakness 1 and Question 3 — Code access, implementation and reproducible experiments

We have prepared a code release (core version 1.0.0) containing proposal handling, feedback, budget and resource enforcement, the numerical worker, and independent replay. The source, running instructions and historical records are available at:

https://anonymous.4open.science/r/ATSolver-51DC/README.md

The docs/REVIEWER_CODE_GUIDE.md maps these components and the appendix's projector.py, feedback.py, proposer.py, unified_runtime.py and rq3_m_controls.py to their executable implementations. Under shared/implementations/core/scripts/, run_main_table.py, run_baselines.py and run_memory.py run the three-proposal comparison, companion five-preset comparison and supplementary experience study, respectively. The experiments/ directory provides the additional history controls, comparisons under the same configuration space and evaluation on additional matrix groups, with preparation, search, replay and analysis instructions.

Historical archives under reproduction/ contain E1–E3 and E5 records, E5 donor/held-out probes, and the four-matrix M protocol and aggregate files named in Appendix F. Required MFEM inputs, the frozen control bundle and E5 corpus are also supplied. The reproduction guide gives checksum-verified extraction and offline commands: e123/reproduce.py reconstructs E1/E3 summaries and checks E1–E3 record coverage; e5/recompute.py reconstructs E5 statistics. E2 support covers record checks. These analyses reconstruct statistics from archived records offline. Numerical replay and new searches use the current implementation with the documented inputs and environment; model searches require provider credentials. We will also make the release publicly available after review.


Critical Weakness 2 and Question 1 — What action history and numerical outcomes contribute

The six-proposal control below measures the effect of showing tried configurations and numerical outcomes separately. We added the requested six-proposal study on all 35 matrices. The first four conditions differ in whether tried configurations and numerical outcomes are shown; ATSolver uses its standard prompt and search state.

Table R2. Six-proposal comparison: matrices accepted after proposal k, duplicate proposals, and distinct executions. The first four conditions differ only in visible history; ATSolver uses its standard prompt and search state.

Conditionk=123456Duplicate proposalsDistinct executions
No history18181818181815951
Tried configurations only1818192022232205
Tried configurations + scalar feedback1923232525250210
Tried configurations + structured feedback1823252525250210
ATSolver2024252525250210

For tried configurations only, the 210 proposal slots comprise 205 distinct executions, two duplicate proposals and three invalid outputs that were not executed.

Showing the tried configurations reduces duplicate proposals from 159 to 2 and is associated with 23 rather than 18 accepted matrices. The feedback conditions finish at 25 accepted matrices; structured feedback reaches 25 after proposal 3 and scalar feedback after proposal 4. The tried-configuration list helps avoid repeated trials; numerical outcomes additionally inform how configurations are revised. In this study, the list raises acceptance from 18/35 to 23/35, and adding outcome feedback raises it to 25/35. Because each matrix and condition has one search, these are observed differences rather than causal estimates.

Table R3. Six-proposal comparison: paired solve-time and deployment-time reductions (%) on jointly successful matrices (accepted under both conditions); reduction 100(1−ρˉ)100(1-\bar{\rho})\\%, where ρˉ\bar{\rho} is the geometric mean of paired time ratios.

ComparisonPairsSolve-time reduction (%)Deployment-time reduction (%)
Tried configurations only vs. no history1826.94.1
Tried configurations + structured feedback vs. tried configurations only232.47.1
ATSolver vs. tried configurations only2337.211.6
ATSolver vs. no history1846.019.8

This comparison reflects ATSolver as a whole rather than the isolated effect of numerical feedback. It has 37.2% lower solve time and 11.6% lower deployment time than tried-configurations-only on the 23 jointly successful matrices, solving faster on 21 of those 23 matrices. Among the 23 jointly successful pairs, ATSolver has lower verified deployment time in 14; N* = 1 for three pairs, and the median N* over all 14 pairs is 18 additional deployments.

Critical Weakness 2 and Question 1 — Three-proposal and fixed-parent ablations

In the three-proposal history study, showing tried configurations removes all duplicate proposals, while final acceptance changes from 77 to 78 of 105. On the 77 jointly successful runs, structured feedback gives 0.425 s lower deployment time and 4.52 s higher search time than no history.

Table R4. Three-proposal history study (35 matrices × 3 repetitions): runs accepted after proposal k (of 105) and duplicate proposals (of 315). All conditions keep a shared handoff that reports failures.

History shownk=123Duplicate proposals (of 315)
No history66717794
Tried configurations only6576780
Tried configurations + scalar feedback7277780
Tried configurations + structured feedback6978780

The fixed-parent study compares one recovery step from the same failed configuration under a neutral handoff. Structured feedback accepts 62 of 105 next proposals, compared with 68 for scalar feedback and 67 for the matched placebo.

Table R5. Fixed-parent study: next proposals accepted (of 105) after a deliberately failed configuration (GMRES without preconditioning, limited to two iterations), under four feedback conditions with the same neutral handoff.

Feedback shown with the failed configurationAccepted (of 105)
No-outcome feedback63
Scalar feedback68
Structured feedback62
Matched placebo67

Over six proposals, scalar and structured feedback both finish at 25 of 35 accepted matrices.


Critical Weakness 2 and Question 2 — Matched five-preset comparison

In a new study in which all methods choose among the same five fixed configurations, ATSolver has equal or higher acceptance than model-based search at each K = 2, 8, 32 and lower paired deployment time on average.

In the paper's study, the baselines choose among five fixed configurations while ATSolver can also tune parameters. To separate configuration-space size from controller choice, we added a five-preset study in which every method uses the same five fixed configurations on the same 35 matrices at K = 2, 8, 32, with three proposals and the same setup/reuse rules. Every RHS must satisfy relative residual ≤ 10⁻⁸, checked independently.

Table R6. Five-preset study: matrices accepted (of 35) and paired deployment-time reduction of ATSolver (%) on jointly successful matrices; reduction 100(1−ρˉ)100(1-\bar{\rho})\\%, where ρˉ\bar{\rho} is the geometric mean of paired time ratios. 95% CIs are shown for the model-based and SMAC comparisons. Parentheses give the number of jointly successful matrices.

Accepted matrices (of 35)

KATSolverModel-based searchSMACRandom search
226252222
824232423
3224242021

ATSolver paired deployment-time reduction (%)

Kvs. model-based searchvs. SMACvs. random search
27.1 [1.2, 14.1] (25)16.1 [6.1, 25.6] (22)9.0 (22)
86.4 [0.5, 14.1] (22)8.8 [0.3, 19.5] (23)18.9 (22)
325.1 [0.5, 12.6] (24)33.8 [10.3, 53.1] (20)35.4 (21)

For each K and comparator, the 95% CIs use 2,000 percentile bootstrap resamples of jointly successful matrices, keeping the paired method timings together. The bounds are the 2.5th and 97.5th percentiles of the bootstrap estimates; resampling is not stratified by matrix family.

On each jointly successful subset, ATSolver has lower paired deployment time on average than all three comparators and equal or higher acceptance at every K. Thus, with identical numerical choices, the comparison isolates how the configurations are selected rather than access to additional parameters.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.