acceptodds
Under review as a conference paper at ICLR 2027

From Search State to Control Actions: A Modular Interface for Budgeted Black-Box Optimization

Abstract

In expensive black-box optimization, the central bottleneck is often how the optimizer spends its scarce oracle budget, rather than how it generates candidate solutions. When each candidate requires a costly rollout, a control rule that mismatches the current search state can consume evaluations without improving the decision. Existing parameter-control methods usually bind such decisions to a particular optimizer or a predefined schedule, leaving the control semantics difficult to separate from their implementation. We introduce Search-State Assisted Policy Orchestration (SAPO), an optimizer-side interface that maps online search states to search-step scale, relative acceptance slack, and conditional perturbation, then executes them through backbone-specific adapters. On a nine-dimensional delivery-mechanism task, the hand-crafted Expert-SAPO-DE instance significantly improves the final objective over DE, jDE, and SHADE under matched oracle, budget, and domain conditions. Its lower median than a deterministic schedule is not statistically significant, and channel ablations do not establish a benefit from joint use of all channels. Backbone-specific SAPO implementations improve on their corresponding baselines in a Standard environment, whereas CEC2020 and BBOB reveal backbone-dependent outcomes. SAPO separates search-control semantics from backbone execution, while the observed benefit remains conditional on the policy, backbone, and task.

Primary area:
reinforcement learning
Keywords:
black-box optimization, simulation-based optimization, adaptive search control, evolutionary computation, policy orchestration

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_FSDGOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Search-State Assisted Policy Orchestration (SAPO), a modular framework for budget-constrained black-box optimization. SAPO maps a six-dimensional online search state to three control actions: search-step scaling, relative acceptance slack, and conditional perturbation. These actions are implemented through backbone-specific adapters for Differential Evolution (DE), Particle Swarm Optimization (PSO), and Evolution Strategies (ES). The authors investigate both a handcrafted expert policy and a supervised imitation policy. Experiments on a nine-dimensional delivery-mechanism optimization task demonstrate statistically significant improvements over DE, jDE, and SHADE under matched evaluation budgets. Additional experiments examine channel ablations, cross-backbone applicability, scenario shifts, and numerical optimization benchmarks. However, SAPO does not demonstrate a statistically significant advantage over a deterministic scheduling baseline, and the full three-channel controller does not consistently outperform individual channels. Furthermore, numerical benchmark results reveal substantial backbone-dependent performance, with particularly weak generalization for SAPO-DE. Overall, the paper presents a potentially useful modular optimization interface, but the evidence for its methodological novelty, the necessity of its proposed control mechanisms, and its general effectiveness remains limited

Critical strengths
  1. Clear modular design and interpretability. The proposed SAPO framework explicitly separates search-state extraction, control-policy decisions, and backbone-specific execution. Its three intervention channels have clear interpretations and provide a structured way to study adaptive optimization behavior across different algorithms.
  2. Promising improvements in the target application. Expert-SAPO-DE achieves statistically significant improvements over several established DE variants on the delivery-mechanism optimization task. The positive results for PSO and ES in the Standard environment also suggest that the proposed interface can be instantiated across different optimization backbones
Critical weaknesses
  1. Insufficient coverage of related work and unclear novelty. The paper cites only 16 references, which is surprisingly limited given the extensive literature on black-box optimization, adaptive evolutionary computation, online parameter control, and hyper-heuristics. The related-work discussion primarily summarizes classical algorithms without sufficiently examining more recent developments or closely related approaches. Consequently, the paper does not convincingly establish how SAPO differs from existing adaptive control frameworks or why its proposed interface constitutes a substantial methodological contribution.
  2. Missing supplementary material and insufficient experimental transparency. A major concern is that the manuscript repeatedly refers to supplementary material that is not included in the provided submission. Essential information, including complete hyperparameter configurations, simulation protocols, detailed statistical comparisons, numerical benchmark results, and additional ablations, is deferred to the missing supplement. For example, Section 6.5 reports aggregated wins, ties, and losses on CEC2020 and BBOB without providing the underlying function-level results. This omission prevents independent verification of several central experimental claims and substantially undermines the reproducibility and completeness of the evaluation.
  3. Incomplete statistical evidence for the main claims. The manuscript reports statistical significance for selected comparisons, but the complete pairwise results and supporting experimental details are unavailable. More importantly, the proposed method does not significantly outperform the deterministic Schedule-DE baseline, and the full three-channel controller fails to demonstrate a significant advantage over its individual components. Therefore, the experimental evidence does not adequately support the necessity or effectiveness of the proposed state-dependent orchestration mechanism.
  4. Weak empirical generalization and insufficient comparison with competitive methods. The empirical improvements are largely concentrated in a specific delivery optimization application. On standard numerical benchmarks, SAPO-DE fails to outperform DE on any CEC2020 function and records 19 losses on BBOB. Furthermore, the experimental comparison does not adequately cover competitive alternatives in expensive black-box optimization. These limitations make it difficult to establish whether SAPO offers meaningful advantages over existing optimization methods beyond the particular application and controller configurations considered.
Questions for authors
  1. The manuscript repeatedly refers to supplementary material, but no supplement is included in the provided submission. Could the authors provide the complete experimental configurations, hyperparameter settings, additional ablations, statistical test results, and function-level benchmark results? Without these materials, how can reviewers independently verify the reported findings?
  2. Given that SAPO-DE does not significantly outperform the deterministic scheduling baseline and performs poorly on several standard numerical benchmarks, what evidence demonstrates that the proposed framework provides a generalizable methodological advantage rather than application-specific heuristic improvements?
  3. Could the authors clarify why the reproducibility package is deferred until acceptance and provide sufficient implementation details or executable code during the review process to support independent assessment?
Reviewer_DX9VOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a random forest-based parameter control framework for DE, PSO and ES for the canonical delivery task. The authors design a state repersentation for the population, an action adapter for different backbones and a supervised random forest model for control. Experimental results on the canonical delivery task shows that the proposed method outperform some DE variants.

Critical strengths

The proposed method is simple but generalizable across different backbones. The experimental results shows that the policy improve the performance of the backbone on the problem.

Critical weaknesses
  1. The novelty of this paper is weak. Using learnable agents to control the parameters of EAs is widely studied, a field known as Meta-Black-Box Optimization (MetaBBO) [1][2]. There are already a large number of works using RL-based or LLM-based agents to control EA parameters. I cannot see any new idea in this paper.

  2. Following the previous weakness, the investigation of related works is limited. Learning-assisted parameter control methods are neither surveyed nor compared. Recent works should be surveyed. Currently, the newest work in the references is from 2023, whereas today is 2026.

  3. The included baselines are also biased. In addition to MetaBBO baselines, advanced EAs such as MadDE, GLPSO, and CMA-ES are not compared. The performance compared with DE and ES on CEC2020 and BBOB makes the generalizability of the proposed method questionable.

  4. Supplementary Material is missing.

[1] Yang, Xu, et al. "Meta-Black-Box optimization for evolutionary algorithms: Review and perspective." Swarm and Evolutionary Computation 93 (2025): 101838.

[2] Ma, Zeyuan, et al. "Toward automated algorithm design: A survey and practical guide to meta-black-box-optimization." IEEE Transactions on Evolutionary Computation 30.2 (2025): 667-687.

[3] Biswas, Subhodip, et al. "Improving differential evolution through Bayesian hyperparameter optimization." 2021 IEEE congress on evolutionary computation (CEC). IEEE, 2021.

[4] Gong, Yue-Jiao, et al. "Genetic learning particle swarm optimization." IEEE transactions on cybernetics 46.10 (2015): 2277-2290.

[5] Hansen, Nikolaus, Sibylle D. Müller, and Petros Koumoutsakos. "Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES)." Evolutionary computation 11.1 (2003): 1-18.

Questions for authors

None

Reviewer_5UzHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Search-State Assisted Policy Orchestration (SAPO) for search control in budget-constrained black-box optimization. SAPO extracts a six-dimensional state from the population and evaluation history. A policy selects the search-step scale, acceptance slack, and conditional perturbation, which adapters apply within differential evolution (DE), particle swarm optimization (PSO), and evolution strategies (ES). The authors implement a hand-crafted con- troller, Expert-SAPO, and train a random forest to imitate discretized expert actions in Imitation- SAPO. Experiments cover nine-dimensional delivery-mechanism optimization, changes in delivery scenarios, and the CEC2020 and BBOB benchmarks, with baseline comparisons, channel abla- tions, implementations across backbones, and rule-imitation tests.

Critical strengths

1.SAPO separates state extraction, control policy, and optimizer updates, making it straightforward to replace policies or compare individual channels while keeping the objective and candidate representation fixed. The state uses only information already recorded by the optimizer, consistent with the black-box setting. The DE instance gives concrete rules, a selection criterion, and an update sequence, so readers can trace how each intervention affects the search.

2.On the Canonical delivery task, Expert-SAPO-DE reaches a median final objective of 6.982 over 20 seeds and 500 evaluations, compared with 7.458, 7.541, and 7.566 for DE, jDE, and SHADE. The authors report statistically significant differences. Within each experiment, the methods share the objective, feasible domain, evaluation-request budget, population size, and seed protocol, with paired tests used for comparison. The DE, PSO, and ES implementations in the Standard environment also show that the interface can be used with different optimizers.

Critical weaknesses

1.Each backbone uses separately designed rules and adapters, so the performance gains primarily support those implementations. The paper defines a shared state and common channel meanings, and the Expert/Imitation substitution and ablations demonstrate uses of the modular design. What the common interface adds beyond conventional parameter-control implementations is less clear. An analysis of the existing implementations would help: which components are reused across backbones, which repeated design work is avoided, and which control comparisons become easier through the interface.

2.The experiments leave open how much of the improvement comes from state feedback. Expert’s effective F approaches 0.9 in the middle and late stages, fixed DE uses F = 0.5, and Expert does not significantly outperform Schedule-DE (raw p = 0.108). In the ablation, step-scale control alone has a median objective of 4.469, compared with 4.579 for the full controller, but none of the full-versus-single-channel differences is significant after correction. The authors acknowledge that an advantage over scheduling and a joint-channel gain remain unestablished. They also state that the ablation uses Standard and the main task uses Canonical, which limits how directly the ablation explains the main-task gains. A more direct test would hold the other channels fixed on Canonical and compare feedback against fixed high-F settings or predetermined trajectories. Trajectories derived from run data should come from separate runs and be evaluated on new paired seeds. A no-intervention baseline and two-channel controls under the same protocol could test the marginal and interaction effects of the additional channels.

3.On BBOB, SAPO-DE records 4 wins, 1 tie, and 19 losses against DE, while SAPO-PSO records 22 wins, 2 ties, and no losses against PSO. The authors acknowledge that gains vary across backbones and tasks. The scale choices in the specified Expert-SAPO-DE rules deserve exami- nation. Diversity is the mean of per-coordinate standard deviations in the original coordinates, but is compared with fixed thresholds of 0.10, 0.05, and 0.02, while the acceptance tolerance β max (1, |fi|) changes with the objective offset. With β = 0.02 fixed, for example, a trial value of 10.3 is rejected against a parent value of 10, but adding 100 to the objective changes the decision to acceptance while preserving solution rankings. The authors should explain these scale choices and how the rules accommodate changes in variable ranges or objective offsets. Experiments on scale transformations, threshold sensitivity, and channel activation frequencies could test their effects; their role in the benchmark losses is not yet established.

4.The authors distinguish rule-prediction accuracy from online performance, but the Ex- pert/Imitation comparison changes both the action representation and the prediction procedure: Expert uses continuous expert actions, whereas Imitation predicts discrete actions. A controller that directly executes discretized expert actions would separate quantization from prediction error. To interpret the high accuracy from the 60%/20%/20% split of 8,000 synthetic states, the main text should describe state sampling, label discretization, class frequencies, and the relationship between model selection and the synthetic test set. It should also explain how the expert rules and model configuration were chosen before online evaluation and which online states the synthetic data cover.

5.Run budgets, population sizes, and seed protocols are matched within each experiment. The com- parability of tuning effort is harder to assess: the main text gives little detail on how the Expert thresholds, Schedule-DE settings, and baseline parameters were chosen. Please identify which came from defaults, the literature, or manual design, and which were tuned through a search. If tuning was performed, report its search ranges, selection criteria, and additional evaluation cost, along with the relationship between development tasks, scenarios, and seeds and those used for final testing. The request budget explicitly includes initialization and cache hits; reporting actual simulator executions would clarify the computational cost of a matched request budget.

6.The paper uses paired Wilcoxon tests and Holm correction, but the main text does not define the comparison families. The main comparison tables give raw p-values, with some adjusted results reported elsewhere in the text. Please provide the adjusted values supporting each significance claim, along with the magnitude of paired improvements and interval estimates, and explain how the intervals were calculated. The randomness settings also need to be specified: how simulator seeds are used in candidate evaluation, how they correspond to optimizer seeds, and how paired runs are constructed. The lowest objective observed within the budget is the authors’ stated metric. Independent-seed reevaluation would be needed if the results are also interpreted as expected performance under stochastic simulation.

Questions for authors

See Critical Weaknesses.

Reviewer_9PdAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SAPO (Search-State Assisted Policy Orchestration), a backbone-agnostic control framework that maps six search-state features to three actions: step-size adjustment, acceptance slack, and conditional perturbation. These actions are adapted to DE, PSO, and ES. Two variants are introduced: Expert-SAPO, using hand-crafted rules, and Imitation-SAPO, using a random forest trained on expert actions. Experiments on a nine-dimensional delivery optimization task and CEC2020/BBOB benchmarks evaluate performance and component contributions, with statistical significance assessed using paired Wilcoxon tests and Holm correction.

Critical strengths
  1. Transparent and statistically rigorous reporting. The paper applies paired Wilcoxon tests with Holm correction and openly reports non-significant results and backbone-dependent failures. This lends credibility to its null and conditional findings, which provide useful insights into the limitations of adaptive parameter control, although they do not establish a broadly generalizable improvement.

  2. Well-controlled ablation design. By fixing the candidate representation and evaluation oracle while independently varying the three control channels, the paper enables meaningful attribution of their individual contributions and supports its conclusions about the limited benefits of combining channels.

Critical weaknesses
  1. Insufficient evidence for cross-backbone generalizability. Although SAPO proposes a shared control interface, each backbone relies on separately designed rules and adapters, with no demonstrated policy transfer across backbones. Consequently, the experiments establish backbone-specific benefits of adaptive parameter control rather than the claimed generalizability of the framework.

  2. Missing surrogate-based optimization baselines. The nine-dimensional optimization task with a limited evaluation budget is well suited to surrogate-based methods such as Bayesian optimization. However, comparisons are restricted to evolutionary optimization baselines. Without evaluating competitive sample-efficient optimizers, the claimed effectiveness of SAPO for budgeted black-box optimization remains insufficiently supported.

Questions for authors

Is any control policy actually shared across backbones, or are the DE/PSO/ES controllers independently hand-tuned? If the control rules differ across backbones, as Section 4.3 suggests, what is reusable beyond the shared state-action interface? Demonstrating that a single expert or imitation policy transfers to an unseen backbone without policy retuning, while outperforming its native baselines, would substantially strengthen the modularity claim and improve my assessment. Otherwise, the contribution should be framed as a common control interface with backbone-specific policies rather than demonstrated cross-backbone policy generalization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.