- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper analyzes the SVD components of LoRA adapters during gradient-based reward post-training. It reports two observations: (i) the factor subspaces (A’s row space and B’s column space) stabilize early while the singular values keep changing, and (ii) the residual spectral motion is asymmetric between A and B. Motivated by these observations, the authors propose FASE. FASE freezes the singular vectors of each factor and optimizes only the singular values with separable CMA-ES. An adaptive controller reallocates exploration between A and B while preserving the total coordinate variance.
Critical strengths
- The paper extends spectral evolution-based search, previously studied for LLM alignment, to text-to-image diffusion models, where each candidate evaluation requires costly multi-step sampling.
Critical weaknesses
-
Limited methodological novelty. The core framework (SVD of a warm-started LoRA, frozen singular vectors, CMA-ES over singular values) is the same as ESSA. The main methodological difference is the adaptive A/B controller in my understanding.
-
The observations are overstated and largely known. Observation 1 (stable subspace, changing spectrum) has been reported before: early stabilization of top singular vectors with continued singular-value growth [1,2]. Observation 2 (A/B asymmetry) is well established for gradient training [3,4].
[1] Yunis, David, et al. "Approaching deep learning through the spectral dynamics of weights." arXiv preprint arXiv:2408.11804 (2024).
[2] Tian, Zailong, et al. "Spectral Surgery: Training-Free Refinement of LoRA via Gradient-Guided Singular Value Reweighting." arXiv preprint arXiv:2603.03995 (2026).
[3] Hayou, Soufiane, Nikhil Ghosh, and Bin Yu. "Lora+: Efficient low rank adaptation of large models." arXiv preprint arXiv:2402.12354 (2024).
[4] Zhu, Jiacheng, et al. "Asymmetry in low-rank adapters of foundation models." arXiv preprint arXiv:2402.16842 (2024).
Questions for authors
-
Why are the ES-based baselines and FASE in Table 1 and Table2 not run under the same budget as FASE? Could you report the comparison under the same budget?
-
FASE lowers almost every complementary metric compared with its warm-start in Table 1 and 2. Does the OCR gain from 0.950 to 0.980 reflect reward hacking? Could the authors provide an analysis for this situation?
-
Could the authors report cost in a common unit (GPU hours, number of generated images, or NFEs) for all methods, including the ES wall-clock time per generation?
-
Fig. 6 only shows FASE starting from DiffusionNFT checkpoints. How does FASE perform when started from the various Flow-GRPO checkpoints?
-
On GenEval, FASE saturates at about 0.98, the same level that continued DiffusionNFT training reaches, and it needs more GPU hours to get there (Fig. 6). Why does FASE give no gain here? If FASE does not improve reward optimization in general beyond simply continuing gradient training, the motivation for using it is unclear.
-
How was the raw Static prior = (0.75, 1.25) chosen? Is there a tuned Static ratio that matches or outperforms FASE? A sensitivity sweep over Static prior would answer this. This is needed to separate the effect of the adaptive controller from that of the hand-chosen prior.