- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies the rollout problem in GRPO for flow-based models. To address the large discretization errors of SDE-based rollouts with few sampling steps, the authors propose a method that combines ODE sampling with Langevin correction. Theoretically, Langevin correction can approximate the target rollout distribution more accurately than few-step SDE sampling under certain conditions. Experiments on both image and video generation show that the proposed method outperforms the Flow-GRPO and DanceGRPO baselines.
Critical strengths
This paper addresses an important issue in Flow-GRPO and proposes a seemingly reasonable solution. The proposed algorithm achieves good experimental results, especially in reducing the training–inference mismatch. The paper also provides some theoretical analysis to support the performance advantages of the proposed method.
Critical weaknesses
- I think the baselines considered in this paper are generally reasonable. However, I am curious about another possible baseline. Since ODE + Langevin requires two NFEs per step, would it be possible to compare it with a baseline that alternates between ODE and SDE steps under the same NFE budget, where stochasticity is introduced only through the SDE steps? This seems to be a more closely related alternative to the proposed ODE + Langevin approach.
- I would also like to understand how sensitive LC-GRPO is to the noise parameter . Appendix B shows that different values are used for LC-GRPO across different tasks. How much does this parameter affect performance? If possible, I would also suggest comparing the sensitivity of LC-GRPO, Flow-GRPO, and CPS to , to see whether LC-GRPO is more robust to different noise levels.
- I noticed that the theoretical analysis relies on some assumptions that do not seem particularly straightforward, such as the condition in Eq. (13). I think it would be helpful to quantitatively investigate when this condition holds. For example, a numerical study using a multivariate Gaussian distribution could help illustrate the conditions under which the theoretical results apply.
Questions for authors
I noticed that the Langevin step size is chosen differently from the noise schedule in SDE sampling. Although the footnote states that the total injected noise variance is larger for Langevin under the same , this does not necessarily hold at every individual timestep. Noise injected at different timesteps may have different effects on the final generated samples, I wonder whether comparing the sum of noise variances is sufficient to support this claim. Could the authors provide further clarification or analysis?
4 comments
Response to Reviewer qoLm (2/2)
Q4 (Question 1): I noticed that the Langevin step size is chosen differently ... I wonder whether comparing the sum of noise variances is sufficient to support this claim.
A4: Thank you for the question. We would like to clarify that the claim in Figure 2 is primarily an empirical observation: at comparable levels of injected noise, ODE + Langevin sampling produces samples with better visual clarity than SDE sampling.
To quantify the injected noise levels, we report the noise standard deviation at each timestep for both methods (10 steps):
| Timestep | Ours | SDE |
|---|---|---|
| 1.00 | 0.96 | 1.00 |
| 0.96 | 0.91 | 1.06 |
| 0.91 | 0.86 | 0.77 |
| 0.86 | 0.79 | 0.64 |
| 0.79 | 0.71 | 0.56 |
| 0.71 | 0.60 | 0.50 |
| 0.60 | 0.46 | 0.46 |
| 0.46 | 0.28 | 0.40 |
| 0.28 | 0.01 | 0.32 |
| 0.01 | 0.00 | 0.01 |
| Sum of variances | 4.27 | 4.16 |
We would like to highlight two points:
- Noise levels do not match exactly at each timestep under the respective sampling schedules. Our Langevin step size is derived from the predictor-corrector formulation, whereas SDE sampling follows a different noise schedule. Ideally, we would control the exploration level to isolate differences in sample quality, but exact per-timestep matching is not inherent to these two formulations.
- The summed noise variance provides a reasonable quantitative characterization of trajectory-level exploration. The similar sums of injected noise variances (4.27 vs. 4.16) indicate comparable overall noise-injection budgets. While this metric does not fully establish equivalent exploration, as the timing of noise injection and its interaction with sampling dynamics can influence the resulting trajectories, it provides a meaningful quantitative basis for comparison. Together with the empirical results in Figure 2, this supports our observation that ODE + Langevin sampling achieves better sample clarity at comparable noise levels.
We thank the reviewer for raising this point. Developing a principled quantitative characterization of exploration over the entire sampling trajectory remains an interesting open research question. In this work, the summed noise variance provides a reasonable and informative measure of the overall noise-injection budget, offering quantitative support for our empirical observations.
Thank you for your detailed responses. I have decided to maintain my original score.
We are glad that our responses were helpful, and we sincerely appreciate your continued positive assessment of our work.
Response to Reviewer qoLm (1/2)
Thank you for your thoughtful and valuable feedback! We address your concerns below.
A1: We thank the reviewer for suggesting Alternating ODE/SDE (a MixGRPO-style method) as an additional baseline. Following this suggestion, we conducted experiments with alternating ODE and SDE sampling on SD3.5-M using 20 sampling steps, matching the NFEs of our method to ensure a fair comparison. This is the same experiment as the MixGRPO baseline in our response to Reviewer B4Rv. The results are presented below.
Table: Comparison with the Alternating ODE/SDE baseline on SD3.5-M under matched NFEs.
As shown in the table, LC-GRPO outperforms the newly added Alternating ODE/SDE baseline under matched NFEs, further demonstrating the effectiveness of our approach.
A2: We conducted additional experiments using η=0.7 for SD3.5-M and η=0.9 for FLUX.1-Dev, fixed across tasks within each model. LC-GRPO retains its reward advantage across the evaluated image tasks, showing that the improvements do not depend on task-specific noise selection.
These results are summarized in our common response and reported in Appendix B.1. In particular, in the two settings where the fixed η differs from our original choice, LC-GRPO's task reward changes only slightly and remains the best: SD3.5-M OCR goes from 0.960 (η=0.8) to 0.946 (η=0.7), compared with 0.914 for Flow-GRPO and 0.935 for CPS, and FLUX.1-Dev HPS-v2.1 goes from 0.384 (η=0.8) to 0.379 (η=0.9), compared with 0.378 for Flow-GRPO and 0.373 for CPS.
A3: We thank the reviewer for this valuable suggestion. Please refer to our common response.