- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Search-State Assisted Policy Orchestration (SAPO), a modular framework for budget-constrained black-box optimization. SAPO maps a six-dimensional online search state to three control actions: search-step scaling, relative acceptance slack, and conditional perturbation. These actions are implemented through backbone-specific adapters for Differential Evolution (DE), Particle Swarm Optimization (PSO), and Evolution Strategies (ES). The authors investigate both a handcrafted expert policy and a supervised imitation policy. Experiments on a nine-dimensional delivery-mechanism optimization task demonstrate statistically significant improvements over DE, jDE, and SHADE under matched evaluation budgets. Additional experiments examine channel ablations, cross-backbone applicability, scenario shifts, and numerical optimization benchmarks. However, SAPO does not demonstrate a statistically significant advantage over a deterministic scheduling baseline, and the full three-channel controller does not consistently outperform individual channels. Furthermore, numerical benchmark results reveal substantial backbone-dependent performance, with particularly weak generalization for SAPO-DE. Overall, the paper presents a potentially useful modular optimization interface, but the evidence for its methodological novelty, the necessity of its proposed control mechanisms, and its general effectiveness remains limited
Critical strengths
- Clear modular design and interpretability. The proposed SAPO framework explicitly separates search-state extraction, control-policy decisions, and backbone-specific execution. Its three intervention channels have clear interpretations and provide a structured way to study adaptive optimization behavior across different algorithms.
- Promising improvements in the target application. Expert-SAPO-DE achieves statistically significant improvements over several established DE variants on the delivery-mechanism optimization task. The positive results for PSO and ES in the Standard environment also suggest that the proposed interface can be instantiated across different optimization backbones
Critical weaknesses
- Insufficient coverage of related work and unclear novelty. The paper cites only 16 references, which is surprisingly limited given the extensive literature on black-box optimization, adaptive evolutionary computation, online parameter control, and hyper-heuristics. The related-work discussion primarily summarizes classical algorithms without sufficiently examining more recent developments or closely related approaches. Consequently, the paper does not convincingly establish how SAPO differs from existing adaptive control frameworks or why its proposed interface constitutes a substantial methodological contribution.
- Missing supplementary material and insufficient experimental transparency. A major concern is that the manuscript repeatedly refers to supplementary material that is not included in the provided submission. Essential information, including complete hyperparameter configurations, simulation protocols, detailed statistical comparisons, numerical benchmark results, and additional ablations, is deferred to the missing supplement. For example, Section 6.5 reports aggregated wins, ties, and losses on CEC2020 and BBOB without providing the underlying function-level results. This omission prevents independent verification of several central experimental claims and substantially undermines the reproducibility and completeness of the evaluation.
- Incomplete statistical evidence for the main claims. The manuscript reports statistical significance for selected comparisons, but the complete pairwise results and supporting experimental details are unavailable. More importantly, the proposed method does not significantly outperform the deterministic Schedule-DE baseline, and the full three-channel controller fails to demonstrate a significant advantage over its individual components. Therefore, the experimental evidence does not adequately support the necessity or effectiveness of the proposed state-dependent orchestration mechanism.
- Weak empirical generalization and insufficient comparison with competitive methods. The empirical improvements are largely concentrated in a specific delivery optimization application. On standard numerical benchmarks, SAPO-DE fails to outperform DE on any CEC2020 function and records 19 losses on BBOB. Furthermore, the experimental comparison does not adequately cover competitive alternatives in expensive black-box optimization. These limitations make it difficult to establish whether SAPO offers meaningful advantages over existing optimization methods beyond the particular application and controller configurations considered.
Questions for authors
- The manuscript repeatedly refers to supplementary material, but no supplement is included in the provided submission. Could the authors provide the complete experimental configurations, hyperparameter settings, additional ablations, statistical test results, and function-level benchmark results? Without these materials, how can reviewers independently verify the reported findings?
- Given that SAPO-DE does not significantly outperform the deterministic scheduling baseline and performs poorly on several standard numerical benchmarks, what evidence demonstrates that the proposed framework provides a generalizable methodological advantage rather than application-specific heuristic improvements?
- Could the authors clarify why the reproducibility package is deferred until acceptance and provide sufficient implementation details or executable code during the review process to support independent assessment?