acceptodds
Under review as a conference paper at ICLR 2027

Learning to Optimize Across Problem Scales: Size-Agnostic Solver Selection for Accelerated Constrained Optimization

Abstract

Standard numerical optimization typically fixes a single solver for the entire iterations, even though the local geometry of an optimization trajectory can shift across well-conditioned and ill-conditioned regimes with different computational tradeoffs. We formulate solver selection as a sequential decision process and learn a size-agnostic selection function that dynamically chooses between different solvers along a single trajectory. The selection function operates on a fixed-dimensional, problem-size-independent state representation, allowing one shared model to generalize across problem scales and structures without scale-specific retraining, in contrast to existing learning-to-optimize (L2O) methods that are usually scale-specific. We also propose an asynchronous selection mechanism to reduce the additional overhead of solver selection. Theoretically, we derive a critical condition number that governs local solver preference and show that heterogeneous condition-number regimes induce a positive solver-selection margin. We also characterize how selection errors under distribution shift translate into runtime regret and give a sufficient condition for improvement over static single-solver baselines. Experiments on nonconvex ACOPF and convex QP benchmarks show that our selector generalizes zero-shot across problem scales and unseen structures, while improving convergence reach and reducing wall-clock runtime by 15% on QP and 6% on ACOPF over existing L2O baselines.

Primary area:
optimization
Keywords:
Learning to Optimize, L2O, Algorithm Selection, Solver Selection, Condition Number

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_eWdZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a size-agnostic learning-based framework for dynamic solver selection in constrained optimization. The method learns to switch between a computationally cheap solver (ADMM) and a more expensive solver (PD-IPM) along an optimization trajectory. It employs an eight-dimensional state representation, long-horizon supervision for convergence reachability and remaining-runtime prediction, a ranking loss, and dataset aggregation. An asynchronous speculative execution mechanism is further introduced to reduce inference overhead. The paper also derives a critical condition number and provides conditional theoretical guarantees for runtime improvement. Experiments on QP and ACOPF demonstrate promising convergence performance and zero-shot transfer across problem scales and structures.

Critical strengths
  1. The paper addresses an important problem in learning-enhanced numerical optimization. Different solvers have distinct computational costs and convergence characteristics, making state-dependent solver selection a meaningful approach to improving end-to-end optimization efficiency. The combination of ADMM and PD-IPM is particularly interesting because these methods exploit different numerical mechanisms.

2.The fixed-dimensional state representation provides a lightweight way to deploy a shared selection model across different problem scales. The use of long-horizon reachability and runtime supervision is also well motivated, as immediate residual reduction does not necessarily reflect the overall computational cost of reaching convergence.

3.The paper reports encouraging experimental results on both convex QP and nonconvex ACOPF. Tables 1–2 show improvements in average runtime while maintaining high convergence reach. The zero-shot experiments in Tables 4–5 provide meaningful evidence of transfer to unseen topologies and matrix structures. In addition, Table 3 demonstrates that asynchronous inference effectively reduces runtime overhead.

Critical weaknesses
  1. The methodological novelty is not sufficiently established relative to existing learning-based dynamic algorithm selection methods.

Getzelman and Balaprakash (2021), Learning to Switch Optimizers for Quadratic Programming, already investigated reinforcement-learning-based switching among multiple optimizers during QP solving. Guo et al. (2024), Deep Reinforcement Learning for Dynamic Algorithm Selection, further explored state-dependent algorithm selection, algorithmic features, and context restoration for switching between algorithms. Although these works consider different solver families, the central idea of learning a state-dependent policy to dynamically switch algorithms is already established. Furthermore, dataset aggregation is closely related to DAgger, while pairwise ranking losses are standard learning techniques. The proposed framework combines these established components with a fixed-dimensional representation and asynchronous execution. I acknowledge that applying this combination to ADMM/PD-IPM switching and demonstrating cross-scale transfer are meaningful contributions. However, the paper does not sufficiently clarify which components constitute fundamentally new technical contributions beyond existing dynamic algorithm selection frameworks. I therefore view the principal novelty as the task-specific solver integration and size-independent state representation, rather than the overall learning and switching paradigm.

  1. The experimental comparisons and ablation studies are insufficient to establish the necessity of the learned selection strategy and its key components.

The paper mainly compares the proposed method against static Cheap/Heavy solvers and existing L2O approaches. However, it lacks comparisons with simple dynamic switching strategies, such as a tuned Cheap-to-Heavy schedule, residual-stagnation-based switching, or cost-aware heuristic selection.

This omission is important because the observed improvements may primarily arise from combining complementary solvers rather than from the proposed learning mechanism itself. Moreover, the paper does not provide sufficient ablations for several central components, including long-horizon supervision, ranking loss, reachability prediction, dataset aggregation, and historical solver-usage features. Consequently, it remains unclear how much each component contributes to the final performance. These experiments are particularly important for supporting the methodological contributions and differentiating the proposed approach from simpler alternatives.

  1. The paper does not establish a theoretical justification for the sufficiency of the proposed eight-dimensional representation across problem scales.

This is a particularly important concern because size-agnostic solver selection is one of the central contributions of the paper. Section 3.2 constructs an eight-dimensional state vector consisting of primal and dual residuals, convergence ratios, optimization progress, and solver-usage history. While its dimensionality is indeed independent of the problem size, this structural property alone does not guarantee that the representation contains sufficient information for reliable solver selection across different scales.

More importantly, Theorem 1 explicitly shows that the critical condition number governing solver preference depends on the problem dimension through the term dp−1d^{p-1}. However, the proposed state representation does not explicitly include problem dimensions, matrix sparsity, or solver computational costs. Consequently, two optimization problems with substantially different scales may produce nearly identical eight-dimensional features while having different optimal solver preferences. In such cases, a shared selection policy may be unable to distinguish the two states. The existing theoretical results do not resolve this issue. In particular, Theorem 3 establishes a conditional runtime improvement guarantee under assumptions concerning ranking loss, distribution shift, and the solver-selection margin. However, it does not establish that the proposed eight-dimensional representation preserves the information required for optimal or near-optimal solver selection across scales, nor does it provide a scale-uniform bound on the approximation error introduced by this representation.

I believe the paper would benefit from a more rigorous theoretical characterization of the proposed representation. Ideally, the authors should establish sufficient conditions under which these eight features preserve solver preference across problem scales, or derive a nontrivial performance bound, such as a lower bound on achievable selection quality or an upper bound on representation-induced runtime regret under clearly stated assumptions. If such a guarantee is not possible, characterizing the limitations or unavoidable regret caused by state aliasing would also be valuable. Without such analysis, the size-agnostic property is primarily supported at the architectural and empirical levels, rather than by a theoretical guarantee about the adequacy of the selected features.

Questions for authors

1.Could the authors clarify the fundamental differences between the proposed framework and existing learning-based dynamic algorithm selection approaches, particularly Getzelman and Balaprakash (2021) and Guo et al. (2024)? What technical challenges are specifically addressed by the proposed long-horizon supervision and size-agnostic representation that existing approaches cannot handle?

2.Could the authors provide comparisons against well-tuned heuristic dynamic switching strategies, such as fixed Cheap-to-Heavy schedules and residual-based switching? I would also like to see ablations of the ranking loss, reachability prediction, dataset aggregation, and long-horizon supervision. These comparisons would help establish whether the learned selection mechanism provides substantial benefits beyond simply switching between complementary solvers.

3.The paper establishes that these features have a fixed dimension, but does not appear to prove that they are sufficient for effective solver selection across problem scales. Could the authors establish a theoretical guarantee, or at least a nontrivial bound on the achievable selection quality or representation-induced regret, that holds across problem scales under clearly specified assumptions? Alternatively, could the authors characterize the conditions under which the eight-dimensional representation becomes insufficient? I believe such an analysis is important for substantiating the central claim of size-agnostic solver selection, beyond demonstrating that the network input dimension is fixed.

4.How well does the additive runtime assumption approximate the actual trajectories generated by ADMM/PD-IPM switching? Could the authors provide empirical evidence connecting local condition numbers, learned solver preferences, and actual runtime advantages? It would also be useful to investigate whether explicitly incorporating problem-scale or computational-cost features improves performance on these challenging cases.

7 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Question 1 – About related work and novelty

We thank the reviewer for highlighting the need to distinguish our contribution from existing dynamic algorithm selection methods. We agree that learning to switch between algorithms is not new, and we have revised the Related Work (Sec. 2.2 and Appendix E, Discussion about Related Works) to explicitly discuss these connections. We do not claim dynamic algorithm selection, long-horizon switching, DAgger, pairwise ranking, or asynchronous execution as individually novel contributions.

Our approach addresses a different numerical setting and its associated challenges. Specifically, Getzelman and Balaprakash (2021) use PPO to switch among Adam, gradient descent, and random search for QP, with rewards based on immediate objective improvement. Guo et al. (2024) similarly formulate dynamic algorithm selection through reinforcement learning for differential evolution. These works establish the effectiveness of learned algorithm switching, but their formulations do not directly address the numerical-state handoff and strict convergence requirements arising when switching between heterogeneous constrained solvers such as ADMM and PD-IPM. In particular, such transitions require maintaining consistent primal, dual, auxiliary, and interior-point states rather than simply switching between algorithm configurations.

Our first technical contribution is a prediction-based alternative to RL-style policy optimization for this setting. Instead of relying on locally defined rewards and policy exploration, we use paired rollouts from identical numerical states to supervise reachability and remaining wall-clock runtime for each candidate solver action. The queried action is evaluated through a common continuation policy, so its target captures both immediate computational cost and downstream effects on the numerical trajectory. This directly addresses long-horizon credit assignment while retaining the actual convergence criterion as part of the learning objective (Secs. 4 and 5.1).

Our second contribution is a shared fixed-dimensional, size-agnostic representation for routing between these heterogeneous numerical solvers. The eight features combine normalized primal/dual residuals, their recent ratios, optimization progress, and solver-usage history, without explicitly encoding problem dimensions. Unlike claiming that fixed dimensionality alone guarantees sufficiency, the revised theory explicitly characterizes when this representation loses solver-preference information. In particular, Proposition 1 in Sec. 5.3 decomposes selection loss into representation aliasing and excess learning loss, while Corollary 1 in Sec. 5.4 isolates the additional loss associated with omitting problem-scale information. This provides a theoretical characterization of both the capability and limitations of the proposed representation.

Finally, we emphasize that ranking loss and dataset aggregation are established techniques that we employ to learn relative solver preferences and collect supervision on the trajectories induced by the evolving selector. Their contributions are evaluated through the ablations reported in our response to Reviewer JD7b Weakness 2. Thus, our contribution is not a new general paradigm for dynamic algorithm selection, but a long-horizon, prediction-based formulation for heterogeneous constrained numerical solver routing, together with explicit numerical state handoff, a size-agnostic selection representation, and a theoretical characterization of its representation-induced limitations.

AuthorsOpenReview ↗

Response to Weakness 2 and Question 2 – About abalation experiment, additional baseline and generalization

We thank the reviewer for emphasizing the need to distinguish the benefit of learned solver selection from that of simply combining complementary solvers. We have substantially expanded our experimental evaluation with controlled switching baselines, training and feature ablations, and additional generalization experiments.

Switching baselines.

As reported in our responses to Reviewer JD7b (Weakness 2) and Reviewer CYLp (Question 2), we added two non-learning baselines: Fixed Prefix, which switches permanently from Cheap to Heavy after a validation-selected number of actions, and Residual Feedback, which makes switching decisions using a validation-tuned feasibility-residual threshold. Both use exactly the same numerical implementations, initialization, convergence criteria, and evaluation budgets as our method. As shown in Tables 1--2 of our response to Reviewer JD7b, on generated ACOPF, our method achieves 0.932 s compared with 1.668 s and 1.724 s for Fixed Prefix and Residual Feedback, respectively. On generated QP, the corresponding runtimes are 24.91, 30.72, and 29.23 ms. These comparisons demonstrate the additional benefit of learned, state-dependent routing beyond heuristic switching.

Training and feature ablations.

We added controlled ablations of ranking loss and dataset aggregation in Table 4. On generated ACOPF, removing ranking loss reduces reach from 99.94% to 61.56%, while removing both ranking loss and aggregation reduces it to 51.00%. On generated QP, removing both components reduces reach from 100% to 88%. These results demonstrate that the two components provide complementary contributions to reliable solver selection.

We further evaluate the eight-dimensional representation in Appendix G.4, Table 9. Removing historical features increases runtime by 12.16%--26.12%, while removing residual-state information increases runtime by 24.09%--56.41% across the four evaluation groups. In particular, residual-state removal increases ACOPF runtime by 47.77% and 56.41% on generated and original instances, respectively. We also clarify in Appendix G.4 the complementary roles of reachability and long-horizon supervision: the former predicts whether an action can reach the prescribed convergence criterion within the remaining budget, while the latter captures its downstream effect on the complete optimization trajectory and remaining runtime. The corresponding ablation results and discussion are also summarized in our response to Reviewer JD7b, Weakness 2.

We further report leave-one-case/topology-out generalization in Appendix G.6, Tables 11--12. Across 500 held-out QP inputs, our method maintains 100% reach and achieves a 1.123× average speedup. Across 400 held-out ACOPF inputs, it improves reach from 85% to 100%, with a 1.646× average speedup on mutually successful inputs. These results complement the frozen cross-scale evaluation by testing generalization to structures explicitly excluded from training and dataset aggregation.

Finally, Appendix G.3, Table 8, reports three-seed runtime results with 95% Student-t confidence intervals. Our method maintains 100% reach on both QP splits and 99.94%/100% reach on generated/original ACOPF, respectively, with relatively tight runtime confidence intervals.

Together, these additions provide controlled evidence that the improvement is not merely due to switching between complementary solvers. The heuristic comparisons isolate the additional benefit of learning, the ablations identify the contributions of the training objectives and state representation, and the cross-scale and leave-one-case-out experiments demonstrate transfer to additional problem sizes and unseen structures. The implementation, trained models, evaluated instances, and evaluation scripts are available at our anonymous repository: https://anonymous.4open.science/status/Solver_Selection-FC06

AuthorsOpenReview ↗

Response to Question 3 – About generalization ability of 8 feature

We added a single-model cross-scale evaluation in Appendix G.5 (Single-Model Cross-Scale Evaluation), Table 10. We evaluate frozen selectors on newly generated ACOPF operating points from nine additional network topologies, without retraining or modifying the normalization statistics or input representation. We additionally evaluate 26 QP benchmark cases spanning 2 to 2,597 primal variables.

Table 10. Single-model cross-scale evaluation on additional ACOPF topologies. The selector is frozen without retraining. Size denotes the number of buses.

Unseen caseSizeInstancesHeavy reach (%)Our reach (%)Heavy (ms)Our (ms)Speedup
case3_lmbd34100.00100.00373.05182.542.044×
case24_ieee_rts244100.00100.00248.52245.741.011×
case39_epri394100.00100.00462.27313.901.473×
case57_ieee574100.00100.00214.95201.601.066×
case73_ieee_rts734100.00100.00312.17310.351.006×
case89_pegase894100.00100.00750.72347.532.160×
case118_ieee11820100.00100.00373.69362.641.030×
case162_ieee_dtc1622085.00100.001140.25390.761.531×
case200_activ20020100.00100.00364.53348.291.047×
Pooled--8496.43100.00559.71338.581.260×

Table 10 (continued). Single-model cross-scale evaluation on QP benchmark cases. Size denotes the number of primal variables.

Unseen caseSizeInstancesHeavy reach (%)Our reach (%)Heavy (ms)Our (ms)Speedup
HS2121100.00100.003.873.781.025×
QPTEST21100.00100.004.693.621.296×
TAME21100.00100.003.723.641.021×
ZECEVIC221100.00100.007.255.701.272×
HS3531100.00100.004.193.921.068×
HS35MOD31100.00100.007.647.511.017×
HS7641100.00100.004.974.411.125×
HS5151100.00100.002.762.661.038×
HS5251100.00100.002.582.541.019×
HS5351100.00100.003.903.731.046×
GENHS28101100.00100.002.312.261.024×
KSIP201100.00100.0026.3322.641.163×
QAFIRO321100.00100.008.907.401.204×
DUAL1851100.00100.0010.6410.171.046×
DUAL2961100.00100.0011.349.371.210×
DPKLO11331100.00100.004.344.171.043×
VALUES2021100.00100.0013.3012.541.061×
QSC2052031100.00100.0011.6310.751.082×
PRIMALC22311100.00100.0014.4213.921.036×
GOULDQP26991100.00100.0038.1824.401.565×
GOULDQP36991100.00100.0068.8834.871.975×
QSCSD17601100.00100.0030.0229.841.006×
MOSARQP29001100.00100.0041.2837.331.106×
QSCSD613501100.00100.00114.2458.941.938×
MOSARQP125001100.00100.00399.52289.901.378×
CONT-05025971100.00100.001243.681100.091.131×
Pooled--26100.00100.0080.1865.771.219×

Our method achieves 100% reach across all 84 additional ACOPF inputs and 26 QP cases. On ACOPF, it improves Heavy reach from 96.43% to 100% and reduces average runtime from 559.71 ms to 338.58 ms, corresponding to a 1.260× speedup. On QP, both methods achieve 100% reach, while ours reduces average runtime from 80.18 ms to 65.77 ms, achieving a 1.219× speedup. Importantly, these results are obtained with a frozen selector, without retraining or modifying the input representation.

Reviewer_CYLpOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a learned selector that chooses at each iteration between a cheap solver (ADMM) and a heavy solver (primal-dual interior point) to minimize wall-clock time to convergence. The selector uses a problem-size-independent state, is trained on paired rollouts with dataset aggregation, and runs asynchronously to hide inference latency. The authors derive a critical condition number for solver preference and a sufficient condition for beating static single-solver baselines. Experiments on ACOPF and QP show full convergence reach and lower runtime than the heavy solver and learning-based baselines, including zero-shot transfer to unseen cases.

Critical strengths
  1. A single small selector with a size-independent state is applied across four ACOPF topologies and five QP families, and transfers zero-shot to unseen cases at 100% reach. This removes scale-specific retraining, which is the main practical limitation of existing L2O methods.

  2. On ACOPF, the selector reaches 100% convergence on all instances, versus 89.75% for both learning-based baselines on generated cases, with lower mean runtime (0.932 s versus 1.798 s for ML4OPF and 3.053 s for GridSFM).

Critical weaknesses
  1. On the same benchmark, the paper’s own heavy solver differs greatly between settings (ACOPF case300_ieee 47% reach and 6.024 s in Table 1 versus 86% and 1.497 s in Table 7, and QP mean 28.06 ms in Table 2 versus 17.75 ms in Table 6). This variation exceeds the reported gain over a matched heavy solver (1.077 to 1.143× on QP). The actions are custom single-iteration solvers, and Ipopt, OSQP and Clarabel are never timed. On QP the synchronous selector (32.92 ms) is slower than the heavy solver (28.06 ms), no variance is reported, and zero-shot transfer is inconsistent (0.615× on held-out case588_sdet, 0.969× on QSCSD1 and VALUES). There are also no ablations of the loss terms, dataset aggregation or state features, and no simple baseline such as a fixed cheap-then-heavy schedule or a residual-threshold rule, so the benefit of learning is not isolated.

  2. The additive runtime model assumes that selection does not substantially change the trajectory, which is in tension with the long-horizon motivation, and the runtime τ is not defined precisely. Theorem 2 essentially states that if each solver is favored at some state, neither static policy is optimal, and Theorem 3 depends on quantities (surrogate error, concentrability factor, generalization gap, target mismatch, margin) that are never estimated, so its condition cannot be checked. The selector never uses the condition number, no experiment relates decisions to the critical value, and linear ADMM convergence is assumed for nonconvex ACOPF with inexact subproblems.

Questions for authors
  1. Why does the heavy baseline differ so much on the same instances between Table 1 and Table 7 and between Table 2 and Table 6, and how do the end-to-end runtimes compare with Ipopt for ACOPF and with OSQP or Clarabel for QP at the same tolerance?

  2. Could the authors report the oracle runtime from the paired rollouts and the fraction of the margin captured, together with ablations of the loss terms, dataset aggregation and state features, and with a fixed cheap-then-heavy schedule and a residual-threshold rule?

  3. Does the asynchronous scheme use extra CPU threads relative to the baselines, and what are the CPU time and the variance over repeated runs? Table 3 reports 25.36 ms for generated QP while Table 2 reports 26.43 ms.

  4. Can the authors relate the selector’s decisions to the local condition number and its critical value on real trajectories, and estimate the quantities in Theorem 3?

4 comments
AuthorsOpenReview ↗

Response to Weakness 1, Question 1 and Question 2 – About experiment details, extra baseline and additional abalation

We thank the reviewer for raising these concerns regarding experimental consistency and baseline comparability. We have substantially expanded the experimental evaluation and clarified the numerical configurations.

First, the differences between Heavy results reflect intentionally different numerical configurations, rather than inconsistent repetitions of the same solver. Our purpose in introducing Configurations B and C is to test whether the proposed routing framework remains effective across different solver implementations and action granularities. For ACOPF, Configuration A uses Gauss--Newton branch updates and one step-controlled PIPS update per Heavy action, whereas B/C use different branch corrections and warm-started PIPS continuation. For QP, the configurations differ in ADMM settings, sparse linear algebra, and boundary fractions (0.9990.999 versus 0.9950.995); Configuration C additionally groups three ADMM sweeps per Cheap action. The exact differences are documented in Appendix H.4 (Numerical Action Definitions) and Appendix I (Solver Pair Mapping). Accordingly, we compare our method against its corresponding Heavy baseline. Importantly, the additional configurations demonstrate that the benefits of learned routing are not tied to one particular Cheap--Heavy implementation.

Second, we added end-to-end comparisons with mature numerical solvers. The complete results are shown in our response to Reviewer JD7b, Weakness 2, Additional baseline comparison, and in Appendix G.2, Table Auxiliary Native Solvers. Briefly, our method achieves 1.10×1.10\times and 1.11×1.11\times speedups over Ipopt/MUMPS on generated and original ACOPF, respectively, and 3.21×3.21\times and 1.08×1.08\times over OSQP on QP. Clarabel is faster than our implementation (11.3111.31 vs. 24.9124.91 ms and 11.6911.69 vs. 26.3726.37 ms), reflecting its optimized native Rust sparse kernels compared with our Python-based implementation. We further quantify this implementation-level difference through matched kernel benchmarks in Appendix G.2. All methods are evaluated using the same external endpoint criteria, with their respective solver tolerances and timing procedures documented in Appendix I.

Third, regarding synchronous inference and runtime variability, we added three-seed evaluations and report mean runtime with 95% Student-tt confidence intervals in Table 3.

Table 3. Multi-seed comparison of synchronous and asynchronous inference. Runtime is reported as mean [95% Student-tt CI] in ms.

ExecutionQP GeneratedQP OriginalACOPF GeneratedACOPF Original
Synchronous32.36 [31.43,33.28]32.36\,[31.43,33.28]32.58 [31.16,34.00]32.58\,[31.16,34.00]979.01 [978.27,979.74]979.01\,[978.27,979.74]956.71 [953.24,960.17]956.71\,[953.24,960.17]
Asynchronous25.26 [24.95,25.57]25.26\,[24.95,25.57]27.41 [26.61,28.20]27.41\,[26.61,28.20]931.44 [915.47,947.41]931.44\,[915.47,947.41]914.60 [901.27,927.94]914.60\,[901.27,927.94]
Speedup1.281×1.281\times1.189×1.189\times1.051×1.051\times1.046×1.046\times

Asynchronous execution consistently reduces wall-clock runtime across both problem classes. In particular, it reduces QP runtime by 21.6% on generated instances and 15.9% on original instances, demonstrating that speculative execution effectively hides inference latency. The confidence intervals are also relatively tight across the three seeds, supporting the robustness of the observed improvements.

Finally, we have substantially expanded the generalization experiments. As reported in our response to Reviewer JD7b, Weakness 2, About experimental results, reproducibility, and robustness, Appendix G.5 evaluates frozen selectors on additional scales and cases, achieving 1.260×1.260\times and 1.219×1.219\times average speedups on ACOPF and QP, respectively. Appendix G.6 further evaluates 500 held-out QP inputs and 400 held-out ACOPF inputs, achieving 100% reach in both domains and average speedups of 1.123×1.123\times and 1.646×1.646\times, respectively.

The Fixed Prefix and Residual Feedback comparisons, together with the ranking-loss, dataset-aggregation, and input-feature ablations, are provided in our response to Reviewer JD7b, Weakness 2. These experiments collectively establish the contribution of learned routing and its robustness across different numerical configurations.

AuthorsOpenReview ↗

Response to Weakness 2 and Question 4 – About the Theorem

We thank the reviewer for identifying these issues with the previous theoretical formulation. We have substantially rewritten the theoretical analysis to align it with the actual solver-selection procedure. As also explained in our response to Reviewer JD7b, Weakness 1 (Theory-method alignment), the previous condition-number-based analysis and Theorems 2--3 have been removed and replaced with an action-dependent long-horizon formulation.

First, we no longer assume that solver selection leaves the optimization trajectory unchanged. Sec. 5.1 explicitly models each solver action through an action-dependent transition kernel Kt(Xt+1,Ct∣Xt,at)\mathsf K_t(X_{t+1},C_t\mid X_t,a_t), which captures changes in numerical states, subsequent optimization progress, and switching costs. We also make the runtime objective precise: CtC_t denotes the realized numerical cost of each action, while H(π)\mathcal H(\pi) accounts for feature extraction, inference, and scheduling overhead. The total objective is Jwall(π)=J(π)+H(π)J_{\mathrm{wall}}(\pi)=J(\pi)+\mathcal H(\pi), with a terminal penalty for failure to reach the prescribed convergence criterion. Thus, the revised formulation directly captures the trajectory-dependent consequences of solver selection.

Second, the previous Theorem 2 is replaced by Theorem 1 (Long-horizon solver-selection improvement) in Sec. 5.2. Rather than relying on the observation that both solvers may be preferable at different states, the new result establishes an exact runtime identity and a ranking-based sufficient condition. It explicitly accounts for cumulative improvement opportunities, representation-induced loss, learning loss, the initial baseline gap, and selector overhead. In particular, the learned selector outperforms both static policies when the accumulated improvement exceeds these losses and overhead. Secs. 5.3--5.4 further characterize representation aliasing and the additional loss associated with omitting problem-scale information.

Third, regarding Question 4, we agree that the revised selector does not explicitly use or estimate the condition number, and we no longer claim that its decisions follow a critical condition-number threshold. We have removed the corresponding critical-value argument rather than attempting to establish an empirical relationship that is not used by our method. Instead, the eight-dimensional representation uses normalized residuals, residual ratios, solver history, and normalized optimization progress. These features describe the observed numerical behavior, while paired long-horizon rollouts directly supervise the downstream runtime consequences of alternative solver actions. The revised discussion in Sec. 4 explains how these signals characterize solver preference without requiring an explicit condition-number estimate.

Finally, the revised analysis makes no linear-convergence assumption for ADMM on nonconvex ACOPF. Both convex QP and nonconvex ACOPF are treated through their actual finite-horizon, action-dependent numerical transitions, including the inexact local updates used in ACOPF. The theoretical analysis requires bounded action costs and the stated coverage condition for the ranking-based bound, rather than a particular convergence rate of the underlying solver. We therefore no longer use local quadratic or linear convergence assumptions to justify the routing policy.

The complete derivations are provided in Appendix C. The revised theory is consequently based on the actual long-horizon solver-selection mechanism, rather than a condition-number threshold or a trajectory-independent runtime approximation.

AuthorsOpenReview ↗

Response to Question 2 and Question 3 – Oracle increase, Additional Baseline and Detail implementation about extra cpu use

We thank the reviewer for these suggestions. We have added an instance-level Oracle comparison, comprehensive ablations, and multi-seed runtime measurements to better quantify the benefits and computational overhead of our method.

Oracle comparison and margin captured.

We clarify that the instance-level Oracle is an evaluation reference rather than the paired-rollout training target. For each test instance, it enumerates Cheap-prefix/Heavy-tail schedules with k∈{0,…,6}k\in\{0,\ldots,6\} and selects the fastest successful schedule. Appendix H.2 and Tables 13--14 report the corresponding results. We further quantify the fraction of the available runtime improvement as:

Margin Captured (%) = (T_Heavy - T_Ours) / (T_Heavy - T_Oracle) × 100

Pooled runtime and fraction of the oracle margin captured.

SettingHeavyOracleOursMargin (%)
QP-B18.2115.8216.9254.0
QP-C18.1714.7816.0163.7
ACOPF-B1.1600.7070.72396.5
ACOPF-C1.1590.7060.706100.0

QP runtimes are in ms and ACOPF runtimes are in seconds. Our method captures approximately 54%--64% of the available QP improvement and over 96% on ACOPF, approaching the instance-specific Oracle without requiring an instance-specific schedule.

Baselines and ablations.

The Fixed Prefix and Residual Feedback comparisons are already reported in our response to Reviewer JD7b, Question 1, Tables 1--2. Both use identical Cheap/Heavy implementations and evaluation criteria. Our method achieves 0.932 s on generated ACOPF versus 1.668 s and 1.724 s for these two baselines, respectively; on generated QP, the corresponding runtimes are 24.91, 30.72, and 29.23 ms.

The requested ablations are reported in our response to Reviewer JD7b, Weakness 2, Experimental evaluation. Table 4 evaluates ranking loss and dataset aggregation individually and jointly. Removing ranking loss reduces generated ACOPF reach from 99.94% to 61.56%, while removing both components reduces it to 51%. Appendix G.4, Table 9, further evaluates state features: removing residual-state information increases runtime by 24.09%--36.84% on QP and 47.77%--56.41% on ACOPF. These results establish the complementary contributions of the training objectives, dataset aggregation, and numerical-state representation.

Asynchronous execution and CPU overhead.

Our asynchronous implementation uses CPU--CPU speculative execution. The numerical solver runs on one pinned CPU core, while feature construction and neural inference run in a separate process pinned to a second core. Thus, it does use one additional CPU core relative to a single-core synchronous execution. This is an explicit trade-off: numerical computation proceeds while the selector evaluates the current state, and speculative work is retained when it agrees with the predicted action.

Appendix F.4 reports the per-decision computational costs on a 649-variable QP instance: neural inference takes 6.55 μ6.55\,\mus and feature construction takes 1.67 μ1.67\,\mus, totaling 8.22 μ8.22\,\mus per decision. This is approximately 3.4% of one Cheap action (0.243 ms) and less than 0.5% of one Heavy action (1.739 ms). We report these measured per-decision CPU costs separately from wall-clock runtime; aggregate CPU-seconds over an entire solve are not separately reported.

The three-seed synchronous/asynchronous comparison is shown in Table 3 and was also presented in our response to Reviewer JD7b, Weakness 2. For generated and original QP, asynchronous execution reduces mean runtime from 32.3632.36 to 25.2625.26 ms and from 32.5832.58 to 27.4127.41 ms, respectively, corresponding to 1.281×1.281\times and 1.189×1.189\times speedups. The asynchronous 95% Student-tt confidence intervals are [24.95,25.57][24.95,25.57] ms and [26.61,28.20][26.61,28.20] ms. Similar improvements are observed on ACOPF, with speedups of 1.051×1.051\times and 1.046×1.046\times.

Finally, regarding the runtime discrepancy, the revised Table 2 reports the main experiment using the fixed global seed, whereas Table 3 reports the mean over three independent seeds with confidence intervals. They are therefore different statistical summaries, not repeated measurements under an identical seed protocol. We have made the multi-seed protocol and confidence intervals explicit in the revised manuscript.

Thank the authors for the detailed reply. The baselines and ablations address most of my concerns. A few points remain:

  1. The new IPOPT, OSQP and Clarabel timings show that the Heavy baseline in the main tables is weak on ACOPF (4.48 s vs 1.03 s for IPOPT). Against IPOPT the gain is about 1.1x, and on QP the selector is slower than Clarabel. The claims should be stated relative to these solvers.

  2. The asynchronous scheme uses a second CPU core and CPU time is not reported. Since the synchronous selector was slower than Heavy on the generated QP cases, please report CPU time or give the baselines the same core budget.

I will raise my score to weak accept.

Reviewer_JD7bOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a “learning to select the optimization solver” method. In particular, the method learns to select between interior point method (IPM) and ADMM based on the current status of the optimization process, trying to predict which solver would be more efficient (IPM iterations are more expensive but might be higher-quality, while ADMM ones are cheaper but might be lower-quality). The learning algorithm is quite simple as it casts solver selection as supervised prediction of each solver’s reachability and remaining runtime from the current state, with labels from paired rollouts. A significant advantage of the approach is that it allows generalizing L2O across problem scales, since the state is independent of the problem dimension. The algorithm is supported by some theoretical arguments on the critical condition number determining local solver preference and a sufficient condition for the learned selector to beat both static solvers. Finally, experiments on ACOPF and convex QPs show that the method converges on all instances and has the lowest average runtime across the compared methods, although not on every individual case.

Critical strengths
  1. The size-agnostic state representation of the method is simple and effective. Using only dimension-free quantities (residuals, their ratios, budget and usage counters), a single network is trained once and then deployed across different problem scales, which is a significant advantage over plenty of L2O methods.
  2. The method is framed with the minimization of wall-clock time, rather than iteration count, as the objective, and then the design remains consistent with this formulation. The runtime labels include a failure penalty, so the selector does not sacrifice convergence for speed, and the asynchronous speculative scheme ensures inference latency does not erode the gains (Table 3). The experiments show that the method converges on all instances while attaining the lowest average runtime across the compared methods on both benchmarks.
Critical weaknesses
  1. The first main weakness is around the methodology framing and supporting theory. Starting with the positioning of this work in the literature, the novelty/benefits of the paper are partially overstated on two fronts. First, learned per-run switching between optimizers already exists with long-horizon objectives (for example, an RL agent that switches between optimizers along a run on QPs is introduced in Getzelman & Balaprakash 2021, which is cited but not really discussed; another example, https://dl.acm.org/doi/abs/10.1007/978-3-031-30229-9_22, predicts the benefit of switching algorithms from trajectory features; and finally there is a very recent paper https://arxiv.org/abs/2609.01811 which learns an RL policy that picks the next optimizer and how long to run it.). Second, the assertion that existing L2O methods are scale-specific ignores unrolled-ADMM and GNN-based methods that explicitly train on small instances and deploy on much larger ones (for example https://openreview.net/forum?id=hzuumhfYSO introduces the idea of training small-deploying large using very similar learned policies conditioned on residuals that are scale agnostic or https://arxiv.org/abs/2510.06860 reports GNNs for ACOPF, that pretrain on small grids and generalize to systems of up to 2,000 buses and to unseen topologies). Moving to the methodology itself, there are a few critical parts that I have found unclear. In particular, it is not stated which policy is followed after action ata_t when computing the remaining-runtime label (committing to ata_t for the rest of the run or following the current selector), which would change what the network is actually estimating. Furthermore, the handoff between ADMM iterates and the IPM initialization (slacks, duals, barrier parameter) is not described, even though this is the part where where much of the practical difficulty comes from and partial primal-dual warm starts are known to hurt IPM convergence. Finally, the theory is quite disconnected from the method at some parts. In particular, the additive runtime model assumes solver choice does not alter the trajectory, which assumes away the core difficulty in Theorem 2, while Theorem 1 applies only in the quadratic convergence region where the heavy solver is favored anyway.
  2. The second main weakness concerns the experimental evaluation, where it is somewhat difficult to attribute the reported gains to the learned selector. First of all, some obvious baselines are missing, for example, the best fixed switch point (the “i cheap iterations then heavy” schedules already enumerated in T0) and a simple threshold rule (ADMM to a loose tolerance, then IPM) should definitely be reported. In addition, the heavy-solver baseline appears weak on ACOPF, as PD-IPM reaches only 47% on case300 and 89% on case500 validation points while the learned selector, using the same IPM as its heavy action, reaches 100% in a tenth of the time, which could be attributed to a poorly initialized IPM; maybe comparing against a mature solver such as IPOPT would also make sense here. The per-case results also do not fully support the headline claims, since on QP the method is best on only 2 of 5 generated and 3 of 5 original cases, on PRIMAL2 and PRIMAL3 all-ADMM converges 100% and is faster (so the selector fails to learn the all-cheap policy where it is optimal). Also, the comparison to GridSFM, ML4OPF, GNN-QP and GNN-DAQP is also not really apples-to-apples, as these predict solutions or active sets and their runtime depends entirely on the polishing procedure. Finally, no seeds, variance, or ablations (state features, ranking loss, dataset aggregation) are reported.
Questions for authors
  1. How does the learned selector compare against the best fixed switch point (cheap for i iterations, then heavy) and a simple residual-threshold rule? These are the natural baselines and would clarify how much is gained by learning versus just by switching once.
  2. Does the heavy solver include any globalization (merit or filter line search, inertia correction, restoration)? Appendix B.1 describes a single regularized Newton step with fraction-to-boundary step sizes, which would explain the 47% reach on case300 and why a few ADMM iterations followed by the same IPM reach 100%. Also, how are the IPM slacks, multipliers, and barrier parameter set when switching from the consensus-ADMM state, and why is IPOPT, already used in Appendix B.4 for the reference solutions, not included as a baseline?
  3. When computing the remaining-runtime label for action ata_t, which policy is followed afterwards (commit to ata_t until termination or the current selector)? This would determine what the network is estimating and how the ranking target relates to the oracle in Section 6.
7 comments
AuthorsOpenReview ↗

Response to Weakness 1 – Methodological positioning: novelty of long-horizon switching and scale-agnostic L2O

We thank the reviewer for identifying two issues in our original positioning: the novelty of long-horizon optimizer switching and the claim that existing L2O methods are scale-specific. We agree that both claims were too broad and have revised the Related Work (Sec.2.2) and Discussion accordingly.

First, we now explicitly discuss prior RL-based optimizer switching, trajectory-feature-based switching, and recent RL-based optimizer scheduling. We therefore do not claim dynamic algorithm selection, long-horizon switching, DAgger, or ranking objectives as standalone novelties. Instead, our contribution focuses on routing between heterogeneous constrained numerical solvers, where solver actions alter subsequent numerical states and optimization trajectories. This is captured by the action-dependent formulation and continuation values in Secs.5.2--5.4, together with explicit solver-state handoff in Sec.4 and AppendixE.

Second, we acknowledge that cross-scale generalization has been demonstrated by prior unrolled-ADMM and GNN-based L2O methods. We have revised our claim accordingly. Methods such as unrolled-ADMM-based approaches (e.g., DeepDistributedQP) are closely tied to specific solver structures, so extending their learned components to heterogeneous numerical algorithms requires substantial solver-specific redesign and engineering effort. Graph-based L2O methods (e.g., HH-MPNN and GNN-based QP solvers) rely on problem-structured representations whose inference costs depend on the number of nodes, edges, and message-passing operations. Moreover, some existing methods rely on target-scale fine-tuning or additional target-distribution adaptation, whereas our single frozen selector achieves zero-shot transfer to unseen structures and large-scale QP instances with up to 2,597 variables, without target-scale retraining. Our claim is therefore that we study a shared fixed-dimensional solver-selection representation without explicit problem-size identifiers, rather than claiming to be the first to achieve cross-scale generalization. Importantly, Secs.5.3--5.4 explicitly characterize representation-induced selection loss and the additional loss from omitting scale information, without assuming representation sufficiency. Cross-scale and unseen-structure experiments in Sec.6 and AppendixG provide the corresponding empirical evaluation.

Thus, our contribution lies in the long-horizon formulation, numerical state handoff, representation-loss analysis, and empirical study of size-agnostic routing between heterogeneous constrained numerical solvers.

AuthorsOpenReview ↗

Response to Weakness 1 – Unclear experiment detials

We thank the reviewer for raising both the ambiguity in the remaining-runtime target and the missing details of the Cheap--Heavy numerical handoff. We have clarified both points in the revised manuscript.

First, the remaining-runtime label is defined by a paired rollout from the same numerical state: the queried action aa is forced only for the first action, and a fixed continuation policy μ\mu is followed thereafter. Thus, the network estimates the long-horizon action value Qtμ(X,a)Q_t^\mu(X,a) rather than the runtime obtained by committing to aa until termination. This is now stated explicitly in Sec. 5.1, Eq. (13), where Qtμ(X,a)=E[∑j=tT−1Cj+g(XT)∣Xt=X, at=a, μ thereafter]Q_t^\mu(X,a)=\mathbb{E}[\sum_{j=t}^{T-1} C_j+g(X_T)\mid X_t=X,\ a_t=a,\ \mu\text{ thereafter}].

Both candidate actions are evaluated from the same state with the same remaining horizon, and the corresponding Bellman relation in Eq. (15) makes explicit that the action value includes both its immediate cost and its downstream effect on the subsequent numerical trajectory.

Second, we have added the detailed solver handoff and initialization procedures in Appendix H.4.3 (QP) and Appendix H.4.5 (ACOPF), with the corresponding implementation excerpts in Appendix I.3.

For QP, the current primal state is passed from ADMM to PD-IPM; inequality slacks are reconstructed as s=max⁡(d−Cx,10−12)s=\max(d-Cx,10^{-12}), valid interior-point multipliers are retained, and the primal/dual boundary steps use the stated 10−1210^{-12} slack and multiplier floors. When a valid PD-IPM state already exists, the updated primal point is used to rebuild positive slacks while the PD-IPM multipliers are retained; the first Heavy action instead uses the configured PD-IPM initialization.

For ACOPF, at the first Cheap-to-Heavy transition we initialize the interior-point state explicitly as s=max⁡(−g(x),1)s=\max(-g(x),1), μ=1/s\mu=1/s, λ=0\lambda=0, and γ=1\gamma=1, and subsequent Heavy calls retain the interior-point state. When switches back to Cheap, the new primal solution replaces the saved primal while the interior-point variables are retained, and the split-ADMM state is reconstructed from the current primal solution. We also explicitly return control to the model after each Heavy action rather than committing to Heavy until convergence.

Finally, we evaluate the complete switched trajectory using the same strict endpoint criteria for reachability, so the reported results include the actual effects of these handoff choices rather than assuming that a partial warm start is automatically beneficial.

AuthorsOpenReview ↗

Response to Weakness 1 – Theorem mismatch

We thank the reviewer for identifying the disconnect between the earlier theoretical formulation and the actual solver-selection dynamics. We substantially revised the theory to address this issue.

In the revised manuscript, the previous trajectory-independent additive runtime formulation and the earlier quadratic-convergence-based theorem are no longer used as the basis of the main guarantee. Instead, Sec. 5.1 explicitly models the numerical process with an action-dependent transition kernel (Xt+1,Ct)∼Kt(⋅∣Xt,at)(X_{t+1},C_t)\sim\mathsf{K}_t(\cdot\mid X_t,a_t), where the kernel includes numerical state conversion and switching costs and captures the effect of the selected solver action on subsequent optimization progress. The long-horizon label is then defined by the continuation value Qtμ(X,a)Q_t^\mu(X,a) in Eq. (13), and its Bellman relation in Eq. (15) explicitly contains the action-dependent next-state distribution. Thus, Cheap versus Heavy can change the subsequent trajectory rather than only contributing an additive runtime term.

The revised main theoretical result is Theorem 1 in Sec. 5.2, which gives the exact runtime identity in Eq. (20) and the corresponding ranking-based sufficient condition in Eqs. (21)--(23), with the complete proof provided in Appendix C.4. In addition, Secs. 5.3--5.4 explicitly decompose the remaining loss into representation aliasing, learning error, and the additional information loss caused by omitting scale.

Therefore, the revised theory is directly aligned with the long-horizon action-dependent routing procedure used by the method, and it no longer relies on a quadratic-convergence region in which Heavy is assumed to be preferable.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.