acceptodds
Under review as a conference paper at ICLR 2027

What Makes a Search Space Effective for Evolution Strategies in Diffusion Fine-Tuning?

Abstract

Evolution strategies (ES) fine-tune generative models from reward evaluations alone, yet successful applications search very different spaces, from all weights of language models to about a hundred scaling factors of diffusion Transformers. Classical analyses of zeroth-order optimization favor small search spaces, but a smaller space also restricts what fine-tuning can change, and spaces of different sizes usually differ in how their parameters act on the network. It therefore remains unclear what makes a search space effective for ES on diffusion models. We separate these two factors in more than one hundred ES runs on SD3.5-Medium, FLUX.1-dev and Wan2.1-T2V, comparing search spaces at matched perturbation energy. Effectiveness depends more on how each parameter acts than on how many parameters there are. On SD3.5-Medium, growing residual scaling from 24 to 36,864 tunable parameters leaves the reward improvement unchanged, even as the agreement between independent update estimates drops by more than an order of magnitude. At 24 parameters, by contrast, scaling whole residual branches improves the reward by 1.68%, dense random mixtures of channels by 0.75% and random channel subsets by only 0.12%. ES search spaces can therefore be large, provided each parameter rescales a whole architectural unit. The improvement generalizes to held-out prompts and reproduces on other backbones, but unoptimized metrics stay flat or decline and optimizing a compositional objective fails, so the reward must encode every property that fine-tuning is meant to improve.

Primary area:
generative models
Keywords:
evolution strategy, diffusion model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_778ROpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper uses evolution strategies (ES) to fine-tune diffusion models from reward evaluations alone and attempts to identify what works well and how the search space impacts learning. It separates two factors: how many tunable parameters there are (dimension d) and how each parameter acts on the network (the map A). This allows comparisons that are otherwise often left unexplored. The authors run, mainly on SD3.5-Medium, with a composite reward (PickScore + HPSv2 + LAION aesthetic). They find interesting results when comparing residual-scaling gains at block, head-group and channel level, dense random subspaces, random channel subsets, and direct weight perturbations, all at matched perturbation energy.

Critical strengths

There are two major strengths of this work: 1) the clear separation of dimensions and parameterization when exploring ES and 2) evidence for the reality that even if estimators should theoretically be expected to have less agreement between updates, they can still perform well.

For the first strength (clear exploration of ES dimensions and parameterization) I would commend this paper's step by step approach, the repetition of seeds, the very clear break down of the search space analysis and overall clarity of the work. For the second (performance) I would commend the authors on further breaking open the field's unintuitive result that solutions can often be found in high dimensional parameter spaces even if alignment with a gradient is rather low.

This paper also provides some practical insights late in the work, specifically related to the use of bf16 as a poor storage method for probes (impact on direction and magnitude of stored probes).

Critical weaknesses

There are a few weaknesses present in this work that I would like to see addressed.

First, the gradient-based comparisons and baselines seem to be lacking. The only gradient-based comparison is in an appendix (Table 22), seemingly trained with a different reward function, and it suggests the ES improvements are sometimes much lower than what could be possible by gradient descent. Without more of a direct comparison (i.e. applying a gradient-based comparison on the same search space), a reader can't tell whether the small improvements (around 1–2% on validation) is due to the limited search spaces or the limitations of ES itself. In general, for this work there is no clear idea of a baseline on performance gain so it is really difficult to know whether the improvements are a serious amount of learning or not.

Second, it seems that the results are quite narrow. Since this paper's contribution is not theoretical in nature but more of an empirical description of effects, it's value really depends on how well the results will generalize. Unfortunately nearly all of the results use the composite reward, and the one alternative objective tested (GenEval) doesn't seem to work well. The cross-backbone results also complicates the main message. On FLUX.1-dev and Wan2.1-T2V, low-dimensional block gains outperform high-dimensional channel gains on every paired seed (Table 6), whereas on SD3.5-Medium the two perform similarly.

Finally, there is a lot of empirical exploration in this work, but unfortunately there is little theoretical contribution to understanding why these effects occur. This compounds with the weaknesses I mentioned above.

Questions for authors

How should readers interpret the size of the improvements? For example, how does +1.68% on the composite compare to the variation between seeds of the base model, or to a human-perceptible difference?

Is the GenEval failure specific to ES or to "gain" approaches? Would a gradient-based or weight-space method succeed on GenEval under the same budget?

What property of a search space do you think actually predicts success based on your results?

Reviewer_SEeXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary
  • The paper investigates various Evolution strategies (ES) for improving generative image models. The authors choose a setup where the generative model (Stable Diffusion 3.5M, Flux.1-dev and Wan2.1-T2V) is perturbed according to the ES, and then a composite reward. This process is repeated for various perturbation and and ES update is calculated.
  • The ES considered are (a) perturbing residual branches in a multiplicative way, (b) randomly rotating the activation vectors - potentially sparsely (c) scaling individual weights (d) perturbing all weights
  • The composite reward is composed of PickScore, HPSv2 and the Laion aesthetic score
  • The ES is performed under a fixed perturbation energy budget, i.e. a maximum deviation between the unperturbed and the perturbed model.
  • The ES optimization can improve the reward by about 1.5%-2% in many setups. However, the control metrics CLIP similarity and ImageReward fall.
  • Sparse matrix multiplications, low rank and full weight perturbations lead to only minor metric improvements.
  • I have trouble completely understand which of the many perturbations lead to exactly which outcome, also because the paper is not very clear and conclusive.
Critical strengths
  • The paper shows, to my knowledge for the first time, that ES can improve on important metrics at the scale of fully developed image generation models up to (SD3.5) 1.5B parameters. While the models are associated with high cost, the derivative-free approach in principle can be executed with GPUs with not enough memory to perform a full backward pass. This makes such optimization possible though I don't believe it will play a big role in practice.
  • The paper collects extensive evidence and performs a systematic and careful analysis. The number of experiments is appropriate, the chosen metrics are sound, the contextual information is helpful. While not providing an in-depth analysis of statistical significance, the repeated experiments give a good idea of statistical uncertainties, which splits the results into to areas: Where the approach "works" (gains >1.3%) vs. where it does not work ( <0.5%). Only few results are in between.
Critical weaknesses
  • I think the paper is weak in explaining its assumptions: What could be the concrete motivations to use ES with generative imaging methods? What are changes in the reward that would qualify as "improvements that are actually visible"? Why the chosen perturbation budget - and what would be different without this?. Paired with a gentle drawback in clarity, especially in Sec. 3 (e.g. CMA-ES is never resolved), this makes the paper's main points hard to grasp for me. Is this an approach that is actually useful? Would it be useful if done at a larger scale?
  • Furthermore, the paper is weak in interpretation. I read: Some methods work, some essentially don't. We don't really know why. Also if the methods work if measured by the reward, points are lost on ImageReward and CLIP score. To me it is not apparent why this would be and how to interpret this. The conclusion section is not very clearly written nor very conclusive.
Questions for authors
  • Does any of the approaches lead to visible improvements in image quality?
Reviewer_Nms8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the effectiveness of different search spaces for evolution strategies (ES) in diffusion fine-tuning. Through controlled experiments across several diffusion models, the authors examine the effects of search-space dimensionality and parameterization structure. The results suggest that how parameters act on the network can be more important than dimensionality alone.

Critical strengths
  1. Clear motivation. The paper addresses an interesting question regarding the relationship between search-space dimensionality and ES optimization performance.
  2. Systematic empirical analysis. The controlled comparisons reveal interesting findings, particularly that higher-dimensional spaces can remain effective despite reduced gradient-estimation consistency.
  3. Cross-model evaluation. Experiments on multiple diffusion backbones provide additional support for the observations.
Critical weaknesses
  1. Incomplete mechanism analysis. The experiments demonstrate differences between search-space parameterizations but provide limited explanations for why certain structures perform better.
  2. Limited generality. The conclusions are primarily supported by residual-scaling search spaces and a relatively narrow set of reward objectives.
Questions for authors
  1. Can the authors further explain why architecture-aligned search spaces outperform random subspaces?
  2. Do the main findings extend to other parameterizations, such as LoRA or adapters, and more diverse reward objectives?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.