- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper uses evolution strategies (ES) to fine-tune diffusion models from reward evaluations alone and attempts to identify what works well and how the search space impacts learning. It separates two factors: how many tunable parameters there are (dimension d) and how each parameter acts on the network (the map A). This allows comparisons that are otherwise often left unexplored. The authors run, mainly on SD3.5-Medium, with a composite reward (PickScore + HPSv2 + LAION aesthetic). They find interesting results when comparing residual-scaling gains at block, head-group and channel level, dense random subspaces, random channel subsets, and direct weight perturbations, all at matched perturbation energy.
Critical strengths
There are two major strengths of this work: 1) the clear separation of dimensions and parameterization when exploring ES and 2) evidence for the reality that even if estimators should theoretically be expected to have less agreement between updates, they can still perform well.
For the first strength (clear exploration of ES dimensions and parameterization) I would commend this paper's step by step approach, the repetition of seeds, the very clear break down of the search space analysis and overall clarity of the work. For the second (performance) I would commend the authors on further breaking open the field's unintuitive result that solutions can often be found in high dimensional parameter spaces even if alignment with a gradient is rather low.
This paper also provides some practical insights late in the work, specifically related to the use of bf16 as a poor storage method for probes (impact on direction and magnitude of stored probes).
Critical weaknesses
There are a few weaknesses present in this work that I would like to see addressed.
First, the gradient-based comparisons and baselines seem to be lacking. The only gradient-based comparison is in an appendix (Table 22), seemingly trained with a different reward function, and it suggests the ES improvements are sometimes much lower than what could be possible by gradient descent. Without more of a direct comparison (i.e. applying a gradient-based comparison on the same search space), a reader can't tell whether the small improvements (around 1–2% on validation) is due to the limited search spaces or the limitations of ES itself. In general, for this work there is no clear idea of a baseline on performance gain so it is really difficult to know whether the improvements are a serious amount of learning or not.
Second, it seems that the results are quite narrow. Since this paper's contribution is not theoretical in nature but more of an empirical description of effects, it's value really depends on how well the results will generalize. Unfortunately nearly all of the results use the composite reward, and the one alternative objective tested (GenEval) doesn't seem to work well. The cross-backbone results also complicates the main message. On FLUX.1-dev and Wan2.1-T2V, low-dimensional block gains outperform high-dimensional channel gains on every paired seed (Table 6), whereas on SD3.5-Medium the two perform similarly.
Finally, there is a lot of empirical exploration in this work, but unfortunately there is little theoretical contribution to understanding why these effects occur. This compounds with the weaknesses I mentioned above.
Questions for authors
How should readers interpret the size of the improvements? For example, how does +1.68% on the composite compare to the variation between seeds of the base model, or to a human-perceptible difference?
Is the GenEval failure specific to ES or to "gain" approaches? Would a gradient-based or weight-space method succeed on GenEval under the same budget?
What property of a search space do you think actually predicts success based on your results?