acceptodds
Withdrawn from ICLR 2027

RIME: Reliable Imagination for Meta-Black-Box Optimization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Meta-black-box optimization (MetaBBO) learns optimizer configuration from optimization trajectories. Surrogate-assisted training reduces the cost of collecting optimization trajectories by replacing expensive objective evaluations with surrogate predictions. However, existing methods use the collected transitions only for direct policy updates and do not learn the optimizer dynamics needed to generate additional training experience. A learned dynamics model can reuse these trajectories through imagined transitions, provided its predictions are reliable. This is challenging because the collected trajectories already reflect surrogate approximation errors, while errors in predicted dynamics can accumulate over imagined rollouts. We propose RIME, a MetaBBO algorithm that learns optimizer configuration through task-conditioned, uncertainty-gated imagination. We first learn a task representation through objective-surrogate pretraining to provide shared context for optimizer control and dynamics prediction. We then learn a task-conditioned world model that reuses collected trajectories through imagined experience, with predictive uncertainty determining which transitions are trusted and the length of imagined rollouts. RIME achieves the best average rank on the evaluated BBOB benchmark across five dimensionalities and outperforms most MetaBBO baselines on UAV path-planning tasks.

Primary area:
optimization
Keywords:
Meta-Black-Box Optimization, Learning to Optimize, World Models, Uncertainty-Gated Imagination.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JZyZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RIME, which integrates surrogate-assisted MetaBBO with a task-conditioned world-model. Global and local reliability gates determine whether predicted optimizer transitions are used for Double-Q updates, while uncertainty controls the rollout horizon. RIME shows competitive average ranks on BBOB and UAV tasks, but its methodological necessity and computational benefits remain to be demonstrated.

Critical strengths
  1. This paper studies an interesting problem of improving the transferability of learned black-box optimizers.
  2. The task encoder, controller, and world model form a pipeline.
Critical weaknesses
  1. The motivation for introducing a world model is partially established. Stage-II interactions already use a learned objective surrogate, so the world model replaces relatively cheap surrogate-based transitions rather than expensive objective evaluations.

  2. RIME introduces additional cost, including surrogate pretraining, a three-member world-model ensemble, world-model updates, uncertainty estimation, and gate tuning. The paper shows only deployment evaluation budgets and does not compare training time, GPU cost, memory, or total optimization-system cost against other MetaBBO methods.

  3. The design principles behind the objective surrogate, 15-action space, nine state features, binary reward, and 50-query support set are insufficiently justified. Most choices are inherited or manually specified, without showing that the resulting state is Markov-sufficient or that these choices are reasonable.

  4. The world-model design is similarly heuristic: it predicts changes in nine aggregate statistics using Huber and binary cross-entropy losses, followed by fixed clipping, loss thresholds, disagreement thresholds, and a hand-designed horizon rule. Since predicted statistics need not correspond to any realizable population, the imagined optimizer states may be internally inconsistent.

  5. The low-dimensional results are insufficient to establish a meaningful optimization advance. Although 20D and 30D are evaluated, RIME loses its aggregate lead on the complete BBOB suite at these dimensions, and strong optimizers such as CMA-ES, SHADE, and L-SHADE are absent.

  6. The comparison should include both learning-based and task-free alternatives, particularly Pretrained Optimization Model for Zero-Shot Black Box Optimization and Task-free Adaptive Meta Black-box Optimization. Their inclusion is important for evaluating whether RIME’s offline training distribution, support-set evaluations, and additional modeling complexity are justified.

Questions for authors
  1. What expensive operation does the world model actually save after Stage II already replaces the original objective with a surrogate?
  2. Can the authors show total training time, GPU hours, model parameters, memory, and performance under matched computation?
  3. What principles determined the action space, state features, binary reward, support-set size, uncertainty thresholds, and rollout rule?
  4. How do the authors verify that an imagined nine-dimensional state corresponds to a realizable DE population?
  5. Does RIME remain competitive against CMA-ES, SHADE/L-SHADE, POM, and task-free MetaBBO on substantially higher-dimensional problems?
Reviewer_rPUDOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a method to learn optimizer dynamics, presumably in a model-based RL setting. The method appears interesting and relevant. However, the presentation lacks clear definitions and sufficient grounding in related work. The title reads like a new optimization algorithm in general, but the introduction focuses on model-based RL. Based on the draft, the method is not clear. Therefore, I recommend resubmission.

Critical strengths
  • The topic is relevant and interesting
  • The method appears to be interesting
  • The results appear strong
Critical weaknesses
  • The optimization problem landscape is huge, and the complexity of solving different optimization algorithms varies a lot based on the class. The authors fail to clearly state the actual optimization problem class.
  • The wording of optimization trajectories is confusing, as this terminology appears in a subfield as trajectory optimization for dynamical systems. Consider using different terminology or explicitly distinguishing it.
  • Similarly, optimizer control is confusing. Are you describing control of the optimizer, or using an optimizer in a dynamical-system control loop? From the abstract and introduction, it is unclear whether you consider stochastic optimal control problems (i.e., RL) or focus purely on solving optimization problems.
  • Another point in this direction: Optimization mathematicians call the whole optimization problem a model, while control engineers call the surrogate of the physical system a model. It is unclear which you mean. I recommend revising the introduction to use precise language and include mathematical definitions. Otherwise, the paper is hard to understand.
  • Which Problem are you actually solving?
  • In the introduction, you write about models of your problem, but then you propose a black-box optimization algorithm. Please clarify why you use a black-box algorithm for a known optimization problem.
  • Contributions: Task embedding and uncertainty-gated imagination were already introduced in related works.
  • Figure 1: Seems to be AI-generated and poorly formatted. It is cluttered with symbols and hard to understand. Consider a simpler plot with the core idea. Please also make sure the font sizes are appropriate.
  • Please define "optimizer configuration"
  • Method: Please first state the problem that you are trying to solve. Unfortunately, too much information is missing to understand this method. You start with "executed trajectories". What are those? Expert demonstrations? Random behavior? Next, you mention "how the search will evolve". Which search are you referring to?
  • Regarding equations, I have similar feedback as above. Please define symbols properly before you introduce them. In Eq. 1, none of the symbols have been defined, except for the support set.
  • Table 4 and Figure 3 would benefit from a more human-readable format
Questions for authors

Unfortunately, the provided work does not allow me to properly understand and discuss the method and results

Reviewer_oRyWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RIME, which augments surrogate-assisted meta-black-box optimization with task-conditioned imagined experience. A pretrained task encoder conditions a Double-Q controller and a world-model ensemble, while readiness checks, ensemble disagreement, and adaptive rollout lengths govern model use. Experiments report improved average ranks on BBOB P-Test tasks and transfer to UAV path planning, with the world model discarded at deployment.

Critical strengths

The paper addresses a relevant problem: reusing optimization trajectories to improve learned optimizer configuration. The evaluation covers five dimensionalities and 14 UAV tasks, with improvements over Surr-RLDE in average rank at every reported P-Test dimension and lower mean objectives on 12 of the 14 UAV tasks. Matching executed-transition and Q-update budgets makes the ablations useful, and discarding the world model at deployment avoids its inference overhead.

Critical weaknesses
  1. Proposition 2 makes the embedding invariant to a common coordinate permutation of the support points, while Equation (2) predicts fitness from the query coordinates and that embedding. Consider fa(x)=∥x−a∥22f_a(x)=\lVert x-a\rVert_2^2 and fΠa(x)=∥x−Πa∥22f_{\Pi a}(x)=\lVert x-\Pi a\rVert_2^2, where Π\Pi permutes coordinates. Their correspondingly permuted support sets have identical labels and embeddings, since fΠa(Πxi)=fa(xi)f_{\Pi a}(\Pi x_i)=f_a(x_i), but their values at the same query generally differ. The stated surrogate therefore cannot represent both objectives correctly. The paper should explain whether additional coordinate information or canonicalization resolves this limitation and report surrogate accuracy on transformed instances.

  2. Appendix C.5 includes a binary improvement flag, while Equations (30)–(33) constrain every predicted state change to magnitude at most 0.250.25. If the flag is encoded as zero or one, its predicted next value is restricted to [0,0.25][0,0.25] when currently zero and [0.75,1][0.75,1] when currently one. The model therefore cannot represent the conditional expected next flag when its conditional probability of switching exceeds 0.250.25. The separate reward head does not alter the stated next-state construction. The feature’s encoding and the effect of this restriction on imagined-state accuracy should be clarified.

  3. Table 2 reports substantially different results for “No world model” and “Real only,” but their precise differences are not explained. Section 5.3 states that “Imagined-only” removes executed replay, while the general procedure falls back to executed replay whenever imagination is unavailable. How this variant handles warm-up and rejected predictions under the matched update budget is therefore unclear. Additionally, Table 19 uses ten paired evaluation seeds with a shared training seed. These results do not quantify variability across independently trained controllers, which is important for assessing the robustness of the gate-ablation conclusions.

  4. Short replay-branched rollouts, uncertainty-based masking, and uncertainty-adaptive rollout lengths appear in MBPO, M2AC, and MACURA, respectively. The paper cites the first two, while MACURA is another directly relevant method to discuss. The existing ablations establish benefits relative to the tested variants, but do not isolate the advantage over established model-use strategies adapted to the same setting. Comparisons using a common encoder, dynamics model, controller, and training budget would clarify the contribution. Adaptive DE baselines such as SHADE or L-SHADE would also strengthen the optimization comparison.

  5. Table 24 provides useful uncertainty–error correlations and multi-step prediction errors at horizons one, five, and fifteen. However, it does not separately characterize errors among accepted and rejected predictions or measure the resulting errors in the controller’s value targets. Appendix B.3 assumes the calibration condition connecting disagreement to return error, and the analysis concerns a fixed model and policy in the surrogate environment. Evaluating prediction and target errors conditional on gate acceptance, together with sensitivity to the gate thresholds, would more directly support the proposed reliability mechanism.

  6. Matching executed-transition and Q-update counts is an appropriate control for the performance comparisons. However, RIME also incurs dynamics-training and rollout-generation costs, and the paper does not report wall-clock training time or the total original-objective evaluations used for encoder and surrogate pretraining. These measurements would help establish the practical cost of the reported gains.

Questions for authors

Please see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.