- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper introduced a reinforcement learning framework Geo-FSRL for few-step generation. The idea of the proposed method is to minimize the conditional KL divergence (Eq. 8) under the restrictions in Eq. 6. The experimental results demonstrate the effectiveness of the proposed method compared to Flow-GRPO and Flow-CPS under the same budget of sampling steps.
Critical strengths
- The proposed methods achieve the best performance on both image and video generation tasks, compared to Flow-GRPO and Flow-CPS.
- The policy reweighting ablation shows the effectiveness of the policy reweighting strategy.
Critical weaknesses
- The paper seems to be hard to follow (at least for me); e.g., I did not know what the velocity-output update, coefficient-preserving family and scalar geometry are (in the abstract) when I first read the abstract of the paper. Additionally, Section 3.4 is hard to understand and would benefit from a more intuitive explanation of the main steps.
- To compare with the baseline Flow-GRPO, I think it is necessary to use the standard datasets used in Flow-GRPO, such as PickScore, GenEval and text rendering evaluated by OCR (these datasets can be found in the official GitHub repo of Flow-GRPO), which are also commonly used in recent RL papers like "Reinforce Adjoint Matching: Scaling RL Post-Training of Diffusion and Flow-Matching Models" and "On-Policy Self-Distillation in Diffusion Models".
Questions for authors
- I am curious about the performance of the proposed methods under the same time budgets. Some rewards vs. training time figures might show that the proposed methods can achieve higher rewards than Flow-GRPO under the same time budget. Could the authors report the training time for different RL methods?
- For generalization, could the authors present more generalization experiments in the paper? For example, you can train models using Flow-GRPO and your proposed frameworks on PickScore, and evaluate the models' performance on other rewards, similar to the experiments in Flow-GRPO.
- Some terms appear for the first time without sufficient discussion or explanation. Could you add more explanation and discussion for these terms (see Weakness 1)? A more intuitive explanation of Section 3.4 would also be helpful.