- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper trains 2 NFE text-to-image generators with an internal DMD, where the real and fake velocities are two condition branches of one network, and , following APEX.
It adds , the squared distance between diagonal Gaussians fitted to mean-pooled features of the generator's EMA copy. This term is motivated by splitting the reverse-KL field into a Gaussian (mean/covariance) part and a residual. Validation: full and LoRA tuning of Z-Image-Turbo 6B and Qwen-Image 20B on about 5.6M mostly synthetic images, reaching 94.25 / 90.20 on LongText-CN/EN at 2 NFE, plus GenEval, DPG, OneIG and an ablation grid.
Critical strengths
- Practical at scale. Extensive experiments at 6B and 20B large-scale models are conducted and proved to be effective at two-stage generation.
- A detailed controlled ablation. Loss removal, regression weighting, and four feature maps (EMA, DINOv2, teacher, live) are ablated, so the advantage and disadvantage of each design in this paper can be read directly from Table 2.
Critical weaknesses
- The empirical contribution of proposed loss is marginal
- The gain of the proposed pop loss is really marginal, according to the ablation studies in Table 2. Most of the gain even exists before the DMD and moment loss, and the base loss alone already is better than the other two-step baselines on the long text bench.
- As the abstract and introduction emphasize a lot for the long-text benchmark, there is no clear explanation on what is the main contributing factor to this excellent long-text benchmark performance, and what is the role of the proposed loss?
- Also the external comparisons are unfair considering the data (5M text-rendering images vs. APEX's 200K) and training steps (10K vs. 2K on Qwen-Image).
- The connection from motivation to theory to practice is not very clear and convincing.
- The Gaussian-plus-residual split is in sample space and used to describe the score function, whereas is in EMA-feature space.
- And in the end, the loss is almost the same as the FD-loss proposed in existing work. While the authors propose to change the feature extractor to an EMA version of the generator, there is no explanation on why this choice is better. Also, for the ema feature extractor, all the features extracted by a moving/changing feature extractor, and we calculate the mean and variance of these features, Which makes it hard to understand.
Questions for authors
- How do matched-data APEX-style and DMD2 baselines perform with and without ?
- Do we use equation 8 somewhere in the paper?
- What's the difference between L-pop and L-pop hat?