- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a size-agnostic learning-based framework for dynamic solver selection in constrained optimization. The method learns to switch between a computationally cheap solver (ADMM) and a more expensive solver (PD-IPM) along an optimization trajectory. It employs an eight-dimensional state representation, long-horizon supervision for convergence reachability and remaining-runtime prediction, a ranking loss, and dataset aggregation. An asynchronous speculative execution mechanism is further introduced to reduce inference overhead. The paper also derives a critical condition number and provides conditional theoretical guarantees for runtime improvement. Experiments on QP and ACOPF demonstrate promising convergence performance and zero-shot transfer across problem scales and structures.
Critical strengths
- The paper addresses an important problem in learning-enhanced numerical optimization. Different solvers have distinct computational costs and convergence characteristics, making state-dependent solver selection a meaningful approach to improving end-to-end optimization efficiency. The combination of ADMM and PD-IPM is particularly interesting because these methods exploit different numerical mechanisms.
2.The fixed-dimensional state representation provides a lightweight way to deploy a shared selection model across different problem scales. The use of long-horizon reachability and runtime supervision is also well motivated, as immediate residual reduction does not necessarily reflect the overall computational cost of reaching convergence.
3.The paper reports encouraging experimental results on both convex QP and nonconvex ACOPF. Tables 1–2 show improvements in average runtime while maintaining high convergence reach. The zero-shot experiments in Tables 4–5 provide meaningful evidence of transfer to unseen topologies and matrix structures. In addition, Table 3 demonstrates that asynchronous inference effectively reduces runtime overhead.
Critical weaknesses
- The methodological novelty is not sufficiently established relative to existing learning-based dynamic algorithm selection methods.
Getzelman and Balaprakash (2021), Learning to Switch Optimizers for Quadratic Programming, already investigated reinforcement-learning-based switching among multiple optimizers during QP solving. Guo et al. (2024), Deep Reinforcement Learning for Dynamic Algorithm Selection, further explored state-dependent algorithm selection, algorithmic features, and context restoration for switching between algorithms. Although these works consider different solver families, the central idea of learning a state-dependent policy to dynamically switch algorithms is already established. Furthermore, dataset aggregation is closely related to DAgger, while pairwise ranking losses are standard learning techniques. The proposed framework combines these established components with a fixed-dimensional representation and asynchronous execution. I acknowledge that applying this combination to ADMM/PD-IPM switching and demonstrating cross-scale transfer are meaningful contributions. However, the paper does not sufficiently clarify which components constitute fundamentally new technical contributions beyond existing dynamic algorithm selection frameworks. I therefore view the principal novelty as the task-specific solver integration and size-independent state representation, rather than the overall learning and switching paradigm.
- The experimental comparisons and ablation studies are insufficient to establish the necessity of the learned selection strategy and its key components.
The paper mainly compares the proposed method against static Cheap/Heavy solvers and existing L2O approaches. However, it lacks comparisons with simple dynamic switching strategies, such as a tuned Cheap-to-Heavy schedule, residual-stagnation-based switching, or cost-aware heuristic selection.
This omission is important because the observed improvements may primarily arise from combining complementary solvers rather than from the proposed learning mechanism itself. Moreover, the paper does not provide sufficient ablations for several central components, including long-horizon supervision, ranking loss, reachability prediction, dataset aggregation, and historical solver-usage features. Consequently, it remains unclear how much each component contributes to the final performance. These experiments are particularly important for supporting the methodological contributions and differentiating the proposed approach from simpler alternatives.
- The paper does not establish a theoretical justification for the sufficiency of the proposed eight-dimensional representation across problem scales.
This is a particularly important concern because size-agnostic solver selection is one of the central contributions of the paper. Section 3.2 constructs an eight-dimensional state vector consisting of primal and dual residuals, convergence ratios, optimization progress, and solver-usage history. While its dimensionality is indeed independent of the problem size, this structural property alone does not guarantee that the representation contains sufficient information for reliable solver selection across different scales.
More importantly, Theorem 1 explicitly shows that the critical condition number governing solver preference depends on the problem dimension through the term . However, the proposed state representation does not explicitly include problem dimensions, matrix sparsity, or solver computational costs. Consequently, two optimization problems with substantially different scales may produce nearly identical eight-dimensional features while having different optimal solver preferences. In such cases, a shared selection policy may be unable to distinguish the two states. The existing theoretical results do not resolve this issue. In particular, Theorem 3 establishes a conditional runtime improvement guarantee under assumptions concerning ranking loss, distribution shift, and the solver-selection margin. However, it does not establish that the proposed eight-dimensional representation preserves the information required for optimal or near-optimal solver selection across scales, nor does it provide a scale-uniform bound on the approximation error introduced by this representation.
I believe the paper would benefit from a more rigorous theoretical characterization of the proposed representation. Ideally, the authors should establish sufficient conditions under which these eight features preserve solver preference across problem scales, or derive a nontrivial performance bound, such as a lower bound on achievable selection quality or an upper bound on representation-induced runtime regret under clearly stated assumptions. If such a guarantee is not possible, characterizing the limitations or unavoidable regret caused by state aliasing would also be valuable. Without such analysis, the size-agnostic property is primarily supported at the architectural and empirical levels, rather than by a theoretical guarantee about the adequacy of the selected features.
Questions for authors
1.Could the authors clarify the fundamental differences between the proposed framework and existing learning-based dynamic algorithm selection approaches, particularly Getzelman and Balaprakash (2021) and Guo et al. (2024)? What technical challenges are specifically addressed by the proposed long-horizon supervision and size-agnostic representation that existing approaches cannot handle?
2.Could the authors provide comparisons against well-tuned heuristic dynamic switching strategies, such as fixed Cheap-to-Heavy schedules and residual-based switching? I would also like to see ablations of the ranking loss, reachability prediction, dataset aggregation, and long-horizon supervision. These comparisons would help establish whether the learned selection mechanism provides substantial benefits beyond simply switching between complementary solvers.
3.The paper establishes that these features have a fixed dimension, but does not appear to prove that they are sufficient for effective solver selection across problem scales. Could the authors establish a theoretical guarantee, or at least a nontrivial bound on the achievable selection quality or representation-induced regret, that holds across problem scales under clearly specified assumptions? Alternatively, could the authors characterize the conditions under which the eight-dimensional representation becomes insufficient? I believe such an analysis is important for substantiating the central claim of size-agnostic solver selection, beyond demonstrating that the network input dimension is fixed.
4.How well does the additive runtime assumption approximate the actual trajectories generated by ADMM/PD-IPM switching? Could the authors provide empirical evidence connecting local condition numbers, learned solver preferences, and actual runtime advantages? It would also be useful to investigate whether explicitly incorporating problem-scale or computational-cost features improves performance on these challenging cases.
7 comments
Response to Weakness 2 and Question 2 – About abalation experiment, additional baseline and generalization
We thank the reviewer for emphasizing the need to distinguish the benefit of learned solver selection from that of simply combining complementary solvers. We have substantially expanded our experimental evaluation with controlled switching baselines, training and feature ablations, and additional generalization experiments.
Switching baselines.
As reported in our responses to Reviewer JD7b (Weakness 2) and Reviewer CYLp (Question 2), we added two non-learning baselines: Fixed Prefix, which switches permanently from Cheap to Heavy after a validation-selected number of actions, and Residual Feedback, which makes switching decisions using a validation-tuned feasibility-residual threshold. Both use exactly the same numerical implementations, initialization, convergence criteria, and evaluation budgets as our method. As shown in Tables 1--2 of our response to Reviewer JD7b, on generated ACOPF, our method achieves 0.932 s compared with 1.668 s and 1.724 s for Fixed Prefix and Residual Feedback, respectively. On generated QP, the corresponding runtimes are 24.91, 30.72, and 29.23 ms. These comparisons demonstrate the additional benefit of learned, state-dependent routing beyond heuristic switching.
Training and feature ablations.
We added controlled ablations of ranking loss and dataset aggregation in Table 4. On generated ACOPF, removing ranking loss reduces reach from 99.94% to 61.56%, while removing both ranking loss and aggregation reduces it to 51.00%. On generated QP, removing both components reduces reach from 100% to 88%. These results demonstrate that the two components provide complementary contributions to reliable solver selection.
We further evaluate the eight-dimensional representation in Appendix G.4, Table 9. Removing historical features increases runtime by 12.16%--26.12%, while removing residual-state information increases runtime by 24.09%--56.41% across the four evaluation groups. In particular, residual-state removal increases ACOPF runtime by 47.77% and 56.41% on generated and original instances, respectively. We also clarify in Appendix G.4 the complementary roles of reachability and long-horizon supervision: the former predicts whether an action can reach the prescribed convergence criterion within the remaining budget, while the latter captures its downstream effect on the complete optimization trajectory and remaining runtime. The corresponding ablation results and discussion are also summarized in our response to Reviewer JD7b, Weakness 2.
We further report leave-one-case/topology-out generalization in Appendix G.6, Tables 11--12. Across 500 held-out QP inputs, our method maintains 100% reach and achieves a 1.123× average speedup. Across 400 held-out ACOPF inputs, it improves reach from 85% to 100%, with a 1.646× average speedup on mutually successful inputs. These results complement the frozen cross-scale evaluation by testing generalization to structures explicitly excluded from training and dataset aggregation.
Finally, Appendix G.3, Table 8, reports three-seed runtime results with 95% Student-t confidence intervals. Our method maintains 100% reach on both QP splits and 99.94%/100% reach on generated/original ACOPF, respectively, with relatively tight runtime confidence intervals.
Together, these additions provide controlled evidence that the improvement is not merely due to switching between complementary solvers. The heuristic comparisons isolate the additional benefit of learning, the ablations identify the contributions of the training objectives and state representation, and the cross-scale and leave-one-case-out experiments demonstrate transfer to additional problem sizes and unseen structures. The implementation, trained models, evaluated instances, and evaluation scripts are available at our anonymous repository: https://anonymous.4open.science/status/Solver_Selection-FC06
Response to Question 3 – About generalization ability of 8 feature
We added a single-model cross-scale evaluation in Appendix G.5 (Single-Model Cross-Scale Evaluation), Table 10. We evaluate frozen selectors on newly generated ACOPF operating points from nine additional network topologies, without retraining or modifying the normalization statistics or input representation. We additionally evaluate 26 QP benchmark cases spanning 2 to 2,597 primal variables.
Table 10. Single-model cross-scale evaluation on additional ACOPF topologies. The selector is frozen without retraining. Size denotes the number of buses.
| Unseen case | Size | Instances | Heavy reach (%) | Our reach (%) | Heavy (ms) | Our (ms) | Speedup |
|---|---|---|---|---|---|---|---|
| case3_lmbd | 3 | 4 | 100.00 | 100.00 | 373.05 | 182.54 | 2.044× |
| case24_ieee_rts | 24 | 4 | 100.00 | 100.00 | 248.52 | 245.74 | 1.011× |
| case39_epri | 39 | 4 | 100.00 | 100.00 | 462.27 | 313.90 | 1.473× |
| case57_ieee | 57 | 4 | 100.00 | 100.00 | 214.95 | 201.60 | 1.066× |
| case73_ieee_rts | 73 | 4 | 100.00 | 100.00 | 312.17 | 310.35 | 1.006× |
| case89_pegase | 89 | 4 | 100.00 | 100.00 | 750.72 | 347.53 | 2.160× |
| case118_ieee | 118 | 20 | 100.00 | 100.00 | 373.69 | 362.64 | 1.030× |
| case162_ieee_dtc | 162 | 20 | 85.00 | 100.00 | 1140.25 | 390.76 | 1.531× |
| case200_activ | 200 | 20 | 100.00 | 100.00 | 364.53 | 348.29 | 1.047× |
| Pooled | -- | 84 | 96.43 | 100.00 | 559.71 | 338.58 | 1.260× |
Table 10 (continued). Single-model cross-scale evaluation on QP benchmark cases. Size denotes the number of primal variables.
| Unseen case | Size | Instances | Heavy reach (%) | Our reach (%) | Heavy (ms) | Our (ms) | Speedup |
|---|---|---|---|---|---|---|---|
| HS21 | 2 | 1 | 100.00 | 100.00 | 3.87 | 3.78 | 1.025× |
| QPTEST | 2 | 1 | 100.00 | 100.00 | 4.69 | 3.62 | 1.296× |
| TAME | 2 | 1 | 100.00 | 100.00 | 3.72 | 3.64 | 1.021× |
| ZECEVIC2 | 2 | 1 | 100.00 | 100.00 | 7.25 | 5.70 | 1.272× |
| HS35 | 3 | 1 | 100.00 | 100.00 | 4.19 | 3.92 | 1.068× |
| HS35MOD | 3 | 1 | 100.00 | 100.00 | 7.64 | 7.51 | 1.017× |
| HS76 | 4 | 1 | 100.00 | 100.00 | 4.97 | 4.41 | 1.125× |
| HS51 | 5 | 1 | 100.00 | 100.00 | 2.76 | 2.66 | 1.038× |
| HS52 | 5 | 1 | 100.00 | 100.00 | 2.58 | 2.54 | 1.019× |
| HS53 | 5 | 1 | 100.00 | 100.00 | 3.90 | 3.73 | 1.046× |
| GENHS28 | 10 | 1 | 100.00 | 100.00 | 2.31 | 2.26 | 1.024× |
| KSIP | 20 | 1 | 100.00 | 100.00 | 26.33 | 22.64 | 1.163× |
| QAFIRO | 32 | 1 | 100.00 | 100.00 | 8.90 | 7.40 | 1.204× |
| DUAL1 | 85 | 1 | 100.00 | 100.00 | 10.64 | 10.17 | 1.046× |
| DUAL2 | 96 | 1 | 100.00 | 100.00 | 11.34 | 9.37 | 1.210× |
| DPKLO1 | 133 | 1 | 100.00 | 100.00 | 4.34 | 4.17 | 1.043× |
| VALUES | 202 | 1 | 100.00 | 100.00 | 13.30 | 12.54 | 1.061× |
| QSC205 | 203 | 1 | 100.00 | 100.00 | 11.63 | 10.75 | 1.082× |
| PRIMALC2 | 231 | 1 | 100.00 | 100.00 | 14.42 | 13.92 | 1.036× |
| GOULDQP2 | 699 | 1 | 100.00 | 100.00 | 38.18 | 24.40 | 1.565× |
| GOULDQP3 | 699 | 1 | 100.00 | 100.00 | 68.88 | 34.87 | 1.975× |
| QSCSD1 | 760 | 1 | 100.00 | 100.00 | 30.02 | 29.84 | 1.006× |
| MOSARQP2 | 900 | 1 | 100.00 | 100.00 | 41.28 | 37.33 | 1.106× |
| QSCSD6 | 1350 | 1 | 100.00 | 100.00 | 114.24 | 58.94 | 1.938× |
| MOSARQP1 | 2500 | 1 | 100.00 | 100.00 | 399.52 | 289.90 | 1.378× |
| CONT-050 | 2597 | 1 | 100.00 | 100.00 | 1243.68 | 1100.09 | 1.131× |
| Pooled | -- | 26 | 100.00 | 100.00 | 80.18 | 65.77 | 1.219× |
Our method achieves 100% reach across all 84 additional ACOPF inputs and 26 QP cases. On ACOPF, it improves Heavy reach from 96.43% to 100% and reduces average runtime from 559.71 ms to 338.58 ms, corresponding to a 1.260× speedup. On QP, both methods achieve 100% reach, while ours reduces average runtime from 80.18 ms to 65.77 ms, achieving a 1.219× speedup. Importantly, these results are obtained with a frozen selector, without retraining or modifying the input representation.
Response to Weakness 1 and Question 1 – About related work and novelty
We thank the reviewer for highlighting the need to distinguish our contribution from existing dynamic algorithm selection methods. We agree that learning to switch between algorithms is not new, and we have revised the Related Work (Sec. 2.2 and Appendix E, Discussion about Related Works) to explicitly discuss these connections. We do not claim dynamic algorithm selection, long-horizon switching, DAgger, pairwise ranking, or asynchronous execution as individually novel contributions.
Our approach addresses a different numerical setting and its associated challenges. Specifically, Getzelman and Balaprakash (2021) use PPO to switch among Adam, gradient descent, and random search for QP, with rewards based on immediate objective improvement. Guo et al. (2024) similarly formulate dynamic algorithm selection through reinforcement learning for differential evolution. These works establish the effectiveness of learned algorithm switching, but their formulations do not directly address the numerical-state handoff and strict convergence requirements arising when switching between heterogeneous constrained solvers such as ADMM and PD-IPM. In particular, such transitions require maintaining consistent primal, dual, auxiliary, and interior-point states rather than simply switching between algorithm configurations.
Our first technical contribution is a prediction-based alternative to RL-style policy optimization for this setting. Instead of relying on locally defined rewards and policy exploration, we use paired rollouts from identical numerical states to supervise reachability and remaining wall-clock runtime for each candidate solver action. The queried action is evaluated through a common continuation policy, so its target captures both immediate computational cost and downstream effects on the numerical trajectory. This directly addresses long-horizon credit assignment while retaining the actual convergence criterion as part of the learning objective (Secs. 4 and 5.1).
Our second contribution is a shared fixed-dimensional, size-agnostic representation for routing between these heterogeneous numerical solvers. The eight features combine normalized primal/dual residuals, their recent ratios, optimization progress, and solver-usage history, without explicitly encoding problem dimensions. Unlike claiming that fixed dimensionality alone guarantees sufficiency, the revised theory explicitly characterizes when this representation loses solver-preference information. In particular, Proposition 1 in Sec. 5.3 decomposes selection loss into representation aliasing and excess learning loss, while Corollary 1 in Sec. 5.4 isolates the additional loss associated with omitting problem-scale information. This provides a theoretical characterization of both the capability and limitations of the proposed representation.
Finally, we emphasize that ranking loss and dataset aggregation are established techniques that we employ to learn relative solver preferences and collect supervision on the trajectories induced by the evolving selector. Their contributions are evaluated through the ablations reported in our response to Reviewer JD7b Weakness 2. Thus, our contribution is not a new general paradigm for dynamic algorithm selection, but a long-horizon, prediction-based formulation for heterogeneous constrained numerical solver routing, together with explicit numerical state handoff, a size-agnostic selection representation, and a theoretical characterization of its representation-induced limitations.