acceptodds
Under review as a conference paper at ICLR 2027

High-Dimensional Black-Box Optimization via Guidance Sampling of Diffusion Models

Abstract

Many real-world optimization problems involve optimizing black-box objectives for which only a limited number of evaluations are available. Bayesian optimization (BO) is a powerful framework for such problems, but its effectiveness often deteriorates in high-dimensional spaces. Recent work leverages diffusion models for black-box optimization, often through pre-training a reward-conditioned score network to steer generation towards high-reward samples. Alternatively, gradient-based guidance methods assume access to a differentiable reward function from which guidance gradients are queried. However, these approaches either require large amounts of labeled data for offline training or access to exact reward gradients, limiting their applicability in black-box settings. We propose BReaD (Black-box Reward-guided Sampling of Diffusion Model), a guidance sampling framework for optimizing black-box rewards without assuming access to a labeled dataset or reward gradients. The proposed method uses a Gaussian process (GP) as a differentiable surrogate for the unknown objective and actively queries labels for the generated samples. We provide a theoretical analysis of the total-variation distance between the generated distribution and the target distribution. Using Upper Confidence Bound (UCB) as guidance signals, our method balances exploration and exploitation during generation and enables the discovery of high-reward regions in the data space. Numerical experiments on scientific discovery and image generation tasks demonstrate that our framework can effectively optimize black-box rewards in low-data scenarios.

Primary area:
optimization
Keywords:
high-dimensional black-box optimization, guidance sampling, diffusion models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RuqwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed a new method for solving the high-dimensional black-box optimization, by guiding the base diffusion models with either GP likelihood or UCB scores that are induced from a Gaussian process.

Critical strengths
  1. The paper is well written and easy to understand the core techniques
  2. The method is reasonable and the result is promising for several tasks including image generation with style-guided or human-preference, ligand optimization and nuclear fusion
Critical weaknesses

Overall, the core technical contribution of this paper is not novel and the experimental results are too limited.

  1. Omission of diffusion training : the pre-training process for the base diffusion is unclear, how would the author pre-trained this diffusion model, and how would it be adapted to the new observations in each iterations. In the Algorithm 2, the training diffusion was not mentioned at all. I assumed that the author train their diffusion only with the unlabeled dataset DD, but it’s not sufficient since the diffusion model could be fine-tuned with new observations from the DrewardD_{reward} with reweighting techniques similar to [1].
  2. The core technique (sub-routine B) is not novel : the UCB-guided diffusion was introduced in [2]. Basically, [2] also sample their candidates from the weighted distribution pψ∝pθ(x)∗exp(β×UCB(x))p_{\psi} \propto p_{\theta}(x)*exp(\beta\times UCB(x)) where pθ(x)p_{\theta}(x) is their base diffusion. The only notable difference is that [2] fine-tune their diffusion to sample pψp_{\psi} while this paper utilized universal guidance sampling to approximate pψp_\psi.
  3. The choice of universal guidance sampling is not evaluated sufficiently. To sample the weighted posterior diffusion models (e.g pψp_{\psi}), any kind of training-free guidance techniques could be applied. The author need to ablate their design choices with other TFG techniques in [3] for their candidates sampling.
  4. Lacking of crucial sensitivity analysis on core hyperparameters such as the guidance strength η(t)\eta(t), the UCB params α\alpha. These hyperparameter choices would significantly effect to the final performance of their method. Furthermore, the authors should provide additional ablation study on the choice of kernels for the Gaussian process.
  5. Lacking of standard baselines in high dimensional blackbox optimization such as [2],[4],[5],[6],[7]. The author did not provided neither discussion nor empirical results with these methods. Their main experiment results are not sufficient since they mostly compared with only two of their own variants in guidance sampling : fwd uni-guided and unguided. This two setting should be considered as ablation study rather than the main baseline.
  6. Lacking of empirical results on common tasks: how this method would perform on mujoco locomotion tasks, rover trajectory optimization or lasso bench ?

References:

[1] Diffusion Models for Black-Box Optimization

[2] Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization

[3] TFG: Unified Training-Free Guidance for Diffusion Models

[4] Scalable global optimization via local bayesian optimization.

[5] Improving sample efficiency of high dimensional bayesian optimization with mcmc.

[6] High-dimensional bayesian optimization via covariance matrix adaptation strategy.

[7] Diffusion-based inverse modeling for black-box optimization.

Questions for authors

See Weakness

Reviewer_QDUnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a method that optimizes black-box rewards by combining a pretrained diffusion generator with a Gaussian-process surrogate updated from sequential reward queries. One guidance variant differentiates a GP predictive likelihood conditioned on a target reward, while another differentiates a GP upper confidence bound to encourage exploration as well as exploitation. The paper derives a total-variation error decomposition for the reward-conditioned variant and reports improvements on style matching, ImageReward optimization, Vina docking-score optimization, and simulated tokamak control. It also gives an example in which a proposed tokamak actuator change resembles a later successful experimental change.

Critical strengths
  1. The online problem setting is useful and explicit. Algorithm 2 incrementally acquires labels rather than requiring a large task-specific labeled corpus before guidance begins.

  2. The numerical results are averaged over ten trials. The fusion section discloses useful limitations. The error analysis acknowledges several distinct approximation sources.

Critical weaknesses
  1. Experiments have limitations in supporting the claims. Fast Direct is a necessary missing experimental baseline. It is discussed in Appendix but not compared in Section 4. Fast Direct is already an online, query-efficient black-box guidance method for pretrained diffusion models, and its GP variant uses gradients of a posterior-mean surrogate to construct guidance targets. The high-dimensional BO claim is also not adequately tested. The fusion surrogate operates on 21 explicit parameters, a tractable setting for TuRBO, yet no trust-region BO baseline is run there. DM-GP-UCB is restricted to selecting from ten unguided candidates per round, so it is not a strong test of modern acquisition optimization. Query counts and generated-candidate counts are not aligned across reported reference lines. Adaptive methods receive 20 initialization labels and then run 30 rounds, while unguided/exact gradient reference lines use the best of 30 generated images.

  2. Theoretical parts are not rigorous. The theorem covers Subroutine A, not the empirically emphasized UCB optimizer, and does not establish optimization efficiency. The guidance-strength and reward-scaling connection to the proof is incomplete. Algorithms 1–2 allow a general eta(t), but the proof's score decomposition uses the unscaled log-likelihood guidance. The theory also assumes f* and the target lie in [0,1], whereas plotted ImageReward values exceed 1 and Vina values are negative.

  3. Experimental breadth and reproducibility are insufficient for broad claims. Image preference is simulated by a fixed ImageReward model, with only a few displayed prompts/styles.

Questions for authors
  1. How does BREAD compare with GP-based Fast Direct at the same total label, generator, and elapsed-time budgets?

  2. Does the fusion result beat TuRBO on the same 21 actuator parameters, initial points, and query cap?

  3. Is UCB uncertainty calibrated after DKL fitting, and what happens with mean-only guidance or a calibrated ensemble?

  4. Will authors release complete task configurations, and a broader prospective or prespecified retrospective fusion evaluation?

Reviewer_7BvvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BReaD, a diffusion-based framework for high-dimensional black-box optimization under limited reward evaluations. The method combines a pretrained diffusion model with a Gaussian process (GP) surrogate to guide sample generation. Two guidance strategies are developed: reward-directed guidance based on GP predictive likelihood and uncertainty-aware guidance based on the Upper Confidence Bound (UCB). A theoretical bound on the distribution mismatch for reward-directed sampling is also provided.

Critical strengths
  1. The paper addresses high-dimensional black-box optimization with limited function evaluations, an important problem in scientific discovery and generative modeling.

  2. Combining GP-based uncertainty estimation with diffusion guidance provides a practical approach to optimizing non-differentiable objectives while leveraging pretrained generative models.

  3. The paper derives a total-variation distance bound that characterizes several sources of approximation error in reward-directed sampling.

Critical weaknesses
  1. The framework primarily combines established components, including GP surrogate modeling, UCB-based exploration, and forward universal guidance. The two proposed guidance strategies largely replace the original guidance function with GP-derived quantities, without introducing a fundamentally new diffusion sampling or optimization mechanism.

  2. Although the paper emphasizes high-dimensional black-box optimization, its effectiveness appears to rely heavily on the representations learned by pretrained diffusion models and deep kernel learning. The paper does not systematically investigate how performance changes with input dimensionality, intrinsic dimensionality, or the quality of learned representations.

  3. More competitive high-dimensional Bayesian optimization and surrogate-assisted generative optimization approaches are needed. In particular, the GP-UCB baseline evaluates only 10 unguided diffusion candidates per iteration, which may limit its competitiveness.

Questions for authors
  1. What is the fundamental methodological novelty of BReaD beyond combining existing GP-based Bayesian optimization and forward universal guidance?

  2. Since Subroutine B achieves stronger empirical performance in several experiments, can theoretical guarantees regarding its optimization behavior or sample efficiency be established?

  3. How does the proposed method scale with increasing dimensionality?

  4. How sensitive is the performance to GP surrogate quality, kernel selection, and the UCB exploration coefficient? In particular, how reliable are GP uncertainty estimates when deep kernel learning is trained using only a small number of labeled samples?

  5. What is the computational overhead of the proposed guidance procedure compared with conventional GP-UCB and alternative diffusion-based optimization methods? How does this overhead affect the practical advantages of the proposed approach when reward evaluations are not extremely expensive?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.