acceptodds
Withdrawn from ICLR 2027

FASE: Factor-Aware Spectral Evolution for Gradient-Free Continuation of Diffusion Post-Training

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reward-based diffusion post-training methods such as Flow-GRPO and DiffusionNFT rely on backpropagation through the generator. We propose Factor-Aware Spectral Evolution (FASE), a gradient-free continuation stage that starts from a gradient-trained, reward-adapted DiffusionNFT low-rank adaptation (LoRA) and performs no further backpropagation. Our design tests a constrained continuation hypothesis motivated by spectral diagnostics on Stable Diffusion 3.5 Medium (SD3.5-M): measured changes in 's row space and 's column space decrease early while ordered singular values continue to evolve, and the two factors exhibit unequal magnitudes of spectral change under the training parameterization. FASE instantiates this hypothesis by freezing the specific handoff SVD bases and optimizing their singular values using separable CMA-ES. A factor-aware adaptive controller reallocates exploration between the two factors using whitened CMA-ES mean steps while preserving total coordinate variance. Starting from task-specific DiffusionNFT checkpoints, FASE reaches GenEval , OCR , and PickScore ; reference target scores from Flow-GRPO and complete DiffusionNFT runs trained from the base model provide endpoint context, not a matched from-scratch comparison. Controlled ablations isolate the adaptive factor-allocation controller by comparing it with uniform exploration and a fixed non-uniform / allocation from the same warm-start under matched population and generation budgets. Experiments across GenEval, OCR, and PickScore show that FASE effectively continues reward-adapted diffusion models without further backpropagation, improving all three target rewards and achieving the best observed endpoints among matched spectral-continuation variants.

Primary area:
optimization
Keywords:
Evolution Strategies

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_juzsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper analyzes the SVD components of LoRA adapters during gradient-based reward post-training. It reports two observations: (i) the factor subspaces (A’s row space and B’s column space) stabilize early while the singular values keep changing, and (ii) the residual spectral motion is asymmetric between A and B. Motivated by these observations, the authors propose FASE. FASE freezes the singular vectors of each factor and optimizes only the singular values with separable CMA-ES. An adaptive controller reallocates exploration between A and B while preserving the total coordinate variance.

Critical strengths
  1. The paper extends spectral evolution-based search, previously studied for LLM alignment, to text-to-image diffusion models, where each candidate evaluation requires costly multi-step sampling.
Critical weaknesses
  1. Limited methodological novelty. The core framework (SVD of a warm-started LoRA, frozen singular vectors, CMA-ES over singular values) is the same as ESSA. The main methodological difference is the adaptive A/B controller in my understanding.

  2. The observations are overstated and largely known. Observation 1 (stable subspace, changing spectrum) has been reported before: early stabilization of top singular vectors with continued singular-value growth [1,2]. Observation 2 (A/B asymmetry) is well established for gradient training [3,4].

[1] Yunis, David, et al. "Approaching deep learning through the spectral dynamics of weights." arXiv preprint arXiv:2408.11804 (2024).

[2] Tian, Zailong, et al. "Spectral Surgery: Training-Free Refinement of LoRA via Gradient-Guided Singular Value Reweighting." arXiv preprint arXiv:2603.03995 (2026).

[3] Hayou, Soufiane, Nikhil Ghosh, and Bin Yu. "Lora+: Efficient low rank adaptation of large models." arXiv preprint arXiv:2402.12354 (2024).

[4] Zhu, Jiacheng, et al. "Asymmetry in low-rank adapters of foundation models." arXiv preprint arXiv:2402.16842 (2024).

Questions for authors
  1. Why are the ES-based baselines and FASE in Table 1 and Table2 not run under the same budget as FASE? Could you report the comparison under the same budget?

  2. FASE lowers almost every complementary metric compared with its warm-start in Table 1 and 2. Does the OCR gain from 0.950 to 0.980 reflect reward hacking? Could the authors provide an analysis for this situation?

  3. Could the authors report cost in a common unit (GPU hours, number of generated images, or NFEs) for all methods, including the ES wall-clock time per generation?

  4. Fig. 6 only shows FASE starting from DiffusionNFT checkpoints. How does FASE perform when started from the various Flow-GRPO checkpoints?

  5. On GenEval, FASE saturates at about 0.98, the same level that continued DiffusionNFT training reaches, and it needs more GPU hours to get there (Fig. 6). Why does FASE give no gain here? If FASE does not improve reward optimization in general beyond simply continuing gradient training, the motivation for using it is unclear.

  6. How was the raw Static prior = (0.75, 1.25) chosen? Is there a tuned Static ratio that matches or outperforms FASE? A sensitivity sweep over Static prior would answer this. This is needed to separate the effect of the adaptive controller from that of the hand-chosen prior.

Reviewer_JjTcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes FASE, a two phase method for reward finetuning of text to image diffusion models: a short gradient based DiffusionNFT warm-start, followed by gradient free optimisation of the LoRA adapters. The method is motivated by a diagnostic study of LoRA checkpoints from Flow-GRPO and DiffusionNFT on SD3.5. After early training, the singular subspaces of the LoRA factors stop moving while the singular values keep changing, and block B changes more than A.

The main experiments compare FASE with Flow-GRPO, DiffusionNFT, full model ES, PiSSA+ES and Random-LoRA+ES. FASE reaches the same target reward as continued DiffusionNFT without backpropagation in the second phase.

Critical strengths

(i) The paper shows that a gradient-free, singular-value-only ES update can continue reward optimisation on SD3.5-Medium and reach the same target reward as continued DiffusionNFT. This shows that ESSA-style spectral ES works in the diffusion domain, not only on language models. Evidence is provided by sample grids and seed curves.

(ii) Table 2 compares uniform, static and adaptive allocation from the same warm start, which isolates the paper's one methodological addition over ESSA.

Critical weaknesses

(i) The paper never states where FASE is structurally better than continuing DiffusionNFT, since final scores are similar (approx. 0.98 GenEval for both). The authorus claim memory savings are a structural property but I could not find measurements. For me, a method that matches its baseline needs a measured efficiency, memory or robustness gain to justify itself.

(ii) Spectral ES is already established by ESSA. The paper's novelty rests on the diffusion domain and the adaptive A, B controller. The diagnostic that motivates the controller (B moves more than A) has a simple explanation, since B is initialised at zero. The controller itself is supported only by a single-run table, in which every held out metric rises in near equal steps from uniform, to static, to adaptive. Without multi-seed tables, per-seed logs or released code, the claimed benefit of the controller cannot be verified and I do not feel confident enough to give a high score in absence of substantial empirical evidence.

Questions for authors

(1) What exactly is FASE meant to do better than continuing DiffusionNFT from the same handoff point? Please report measured peak GPU memory and GPU hours to reach the same target reward for both.

(2) As I understand it, B is initialised at zero, so how do you rule this out as the reason B moves more than A? For example, does the asymmetry persist when changes are normalised by each block's norm at handoff, or with a non-zero B initialisation?

(3) Seed variance for Table 2. Please report std over the seeds, and ideally provide per-seed logs as well. Are the held-out metrics HPS, Aesthetic, and UnifiedReward computed on the same prompts and seeds for all three variants?

(4) As I understand it, the ES baselines (Full-Model ES, PiSSA+ES, Random-LoRA+ES) start from the base model, while FASE starts from a DiffusionNFT warm-start. What do these baselines reach from the same warm-start? Does FASE’s reported compute include the warm-start, and are all rows evaluated with the same sampler (40-step ODE, no CFG)?

(5) How were the handoff steps chosen, and how sensitive are the results to them? If they were tuned on the target reward, please say so.

(6) Will code and configurations be released?

Reviewer_LhBfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FASE continues diffusion reward optimization from a gradient-trained DiffusionNFT LoRA. It freezes the singular-vector bases of both LoRA factors and optimizes their spectral coefficients using separable CMA-ES. An adaptive controller reallocates exploration between the factors while preserving total spectral-coordinate variance. Experiments on SD3.5-M demonstrate reward improvements on GenEval, OCR, and PickScore without further generator backpropagation.

Critical strengths
  1. Useful and clearly scoped contribution. The paper demonstrates that reward optimization can continue without backward passes after a learned warm-start. It appropriately distinguishes continuation from fully gradient-free training and qualifies the spectral diagnostics.

  2. Relevant controlled comparisons. Uniform, static, and adaptive exploration share the same checkpoint and generation budget. The appendix also compares FASE with continued DiffusionNFT from the same handoff, providing a useful practical reference.

Critical weaknesses
  1. The adaptive controller’s incremental benefit is not established robustly. Frozen-factor spectral search already appears in ESSA, as acknowledged by the authors, this highly limits the novelty of the overall idea being borrowed from language model finetuning. Against static allocation, FASE improves GenEval by 0.007, OCR by 0.004, and PickScore by 0.07 in Table 2. These are single-run comparisons. Figure 8 includes three FASE seeds, but corresponding baseline repetitions and uncertainty on the differences are missing. Repeat the matched comparison across seeds and report both endpoint gains and compute required to reach a fixed reward.

  2. The practical reward–quality tradeoff remains unclear. OCR improves from 0.950 to 0.980 while every reported complementary metric declines from the warm-start. Removing backward passes is useful, but does not itself establish a preferable operating point. Figure 6 reports continuation GPU-hours; extend this comparison with peak memory and held-out quality at matched reward. This would establish when FASE is preferable to simply continuing DiffusionNFT.

  3. Experiments use one backbone, one resolution, and DiffusionNFT warm-starts. Applicability to other architectures or adaptation methods remains untested.

  4. Search prompts, held-out evaluation splits, candidate seed sharing, and checkpoint selection need clearer specification to assess generalization and selection bias.

Questions for authors

What prompt sets are used for search, checkpoint selection, and final evaluation? Are prompts and generation seeds shared across candidates to reduce ranking noise, and are final evaluation prompts held out?

Does adaptive allocation consistently outperform a small, equally budgeted sweep of static A/B ratios, rather than only the chosen ratio of 5/3?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.