acceptodds
Under review as a conference paper at ICLR 2027

Flow-Map GRPO: Reinforcement Learning for Few-Step Flow-Map Generators via Anchored Stochastic Composition

Abstract

Few-step flow-map generators, such as consistency models and MeanFlow, accelerate sampling by learning long-range transport maps between noise and data. However, their deterministic transitions do not directly provide the stochastic trajectories and tractable likelihood ratios required by reinforcement learning (RL) post-training. Existing SDE-based stochasticization techniques target velocity-based samplers and do not directly extend to long-range flow-map transitions. We propose Flow-Map GRPO, an online RL post-training framework for deterministic few-step flow-map generators. Its key component, Anchored Stochastic Flow Map Composition (ASFMC), combines deterministic transport with anchor-based conditional resampling. We establish the conditions under which this construction preserves the marginal probability path and develop tractable local- and endpoint-anchor policies for two-time and single-time flow maps. These policies enable a unified GRPO training procedure. Experiments on FLUX-based MeanFlow and sCM generators demonstrate substantial improvements in OCR, PickScore, and GenEval at different numbers of inference steps, including joint OCR–PickScore gains with mixed rewards. Controlled ablations show that the stochastic transition design is essential for translating training rewards into generation quality. Flow-Map GRPO enables effective RL alignment of pretrained deterministic flow-map generators while retaining their original parameterization, without retraining them as native stochastic models.

Primary area:
generative models
Keywords:
Flow Matching, Few-Step Generative Models, Reinforcement Learning, GRPO, MeanFlow, Text-to-Image Generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_ma2pOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses an important and timely problem: enabling reinforcement-learning post-training for deterministic few-step flow-map generators. The proposed anchored stochastic composition provides a practical way to construct stochastic policies for pretrained MeanFlow and consistency models without changing their underlying parameterizations. The mathematical motivation is interesting, and the substantial improvements in OCR, PickScore, and GenEval demonstrate the method's practical effectiveness. I lean toward a positive evaluation, but I have several questions about the theoretical and methodological justification.

Critical strengths
  1. This paper solves an important challenge for flow-map model-based RL. The deterministic flow map lacks the explicit random term found in an SDE, which hinders direct RL implementation. This paper proposes a way to convert a deterministic flow map back into a stochastic path, thereby enabling RL.

  2. The experimental results clearly demonstrate the validity of the proposed method.

Critical weaknesses

Weaknesses:

  1. Marginal preservation for learned long-range transitions. The theoretical guarantees rely on exact flow maps and exact conditional resampling, whereas practical implementation uses approximate learned maps and, for the local anchor, a discretized reverse transition. Can the authors quantify how well the constructed stochastic policies preserve the intended probability marginals, especially for large transport intervals and after RL adaptation?

  2. Interpretation of normalized likelihood ratios. Appendix A.3 states that the implementation averages log probabilities over latent dimensions, yielding a normalized ratio equal to the dd-th root of the joint policy likelihood ratio. This differs from the importance ratio used in the theoretical GRPO objective. Could the authors clarify the optimization interpretation of this normalization and investigate its influence on training stability and performance?

  3. Comparison with adapted velocity-based RL. Although the comparisons with SFT and DiffusionNFT are useful, an adapted Flow-GRPO baseline using the pretrained MeanFlow model's equal-time velocity predictions would better establish the advantage of directly stochasticizing long-range flow maps. Reporting matched sampling costs and training overhead would further strengthen the practical contribution.

Questions for authors

Please refer to the weaknesses.

To sum up, I think this paper is interesting and tends to be a weak accept. During the rebuttal, the author should clarify the weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer ma2p (1/2)

We sincerely thank you for your positive assessment and your questions. In summary:

  • W1 (marginal preservation): stochastic rollouts and deterministic sampling of the same model reach FID scores that differ by less than 0.8 before RL (26.19 vs. 25.43), and the rollouts are slightly better after RL (15.20 vs. 16.72).
  • W2 (likelihood normalization): following Flow-GRPO, we average the log-probability over latent dimensions. This keeps the direction of the policy gradient at the rollout policy and lets the same ε be used for both FLUX and ImageNet latents. Compared with the unnormalized ratio, it converges faster and reaches a better FID (16.72 vs. 17.93).
  • W3 (equal-time Flow-GRPO baseline): Flow-Map GRPO reaches OCR 0.850 at 247 s per round; Flow-GRPO on the equal-time velocity reaches 0.408 at 441 s, and MeanFlowNFT 0.713 at 309 s.

W1: Can the authors quantify how well the stochastic policies preserve the marginals with learned maps, for large intervals and after RL?

Directly measuring the intermediate marginals of a learned high-dimensional model is impractical, so we measure their effect on the generated distribution: if a stochastic policy preserves the marginals, its rollouts should match deterministic sampling of the same network. We have added this comparison to App. C.6 (ImageNet, 5K images, K=4 stochastic transitions per rollout):

ModelSamplerFID ↓\downarrowPrecision ↑\uparrowRecall ↑\uparrow
BaseDeterministic25.430.5910.340
BaseLocal-anchor policy26.190.5620.378
BaseEndpoint-anchor policy25.690.5670.381
After RLDeterministic16.720.7740.421
After RLLocal-anchor policy15.200.7770.464

Before RL, both anchors reach an FID within 0.8 of deterministic sampling. After RL, local-anchor rollouts reach an FID about 1.5 lower than deterministic sampling, with matching precision and higher recall. This is consistent with RL optimizing the stochastic policy itself.

The table above is the case of an accurate flow map, where the choice of anchor has little effect on the final FID: after RL, the endpoint, midpoint and local anchors reach nearly the same FID (16.68 ± 0.32, 16.51 ± 0.18 and 16.65 ± 0.12 over 3 seeds). The effect of a large transport interval appears when the learned map is less accurate over long ranges, as for FLUX.1-lite MeanFlow.

  • Rollouts before RL (Table 9). Local-anchor rollouts stay close to deterministic sampling in PickScore (20.72 vs. 20.98), whereas endpoint-anchor rollouts, which resample through the longest jump of the map, drop to 17.51.
  • Consequence after RL (Table 5). OCR rises as the anchor time τ moves from the endpoint towards the current step: 0.387 (endpoint, τ = 1), 0.480 (midpoint) and 0.850 (local).
  • Larger steps with the local anchor (Table 10). With fewer stochastic transitions K per rollout, i.e. larger steps, Flow-Map GRPO still raises OCR from 0.349 to 0.762 (K=2) and 0.770 (K=3); the full comparison is in the table under W3.

With a learned map, the quality of the stochastic policy thus depends on how far the anchor's prediction has to reach. The local anchor uses the equal-time velocity at the next state instead of a further long-range prediction, which is why we use it.

AuthorsOpenReview ↗

Response to Reviewer ma2p (2/2)

W2: Interpretation of the normalized likelihood ratio and its influence.

We follow Flow-GRPO (Liu et al., 2025) here, whose public implementation uses the same normalization. It averages the log-probability over the dd latent dimensions, so the ratio is ρ~=ρ1/d\tilde\rho = \rho^{1/d}, the dd-th root of the joint ratio ρ\rho, which is the geometric mean of the per-dimension ratios. This normalization has two effects.

  • The policy gradient keeps its direction. At the rollout policy, ρ=ρ~=1\rho = \tilde\rho = 1 for every sample, so the gradient of the normalized objective equals the gradient of the joint-likelihood objective scaled by 1/d1/d. Each round takes two gradient updates on the same rollouts. In the second update, an unclipped sample contributes (ρ~/d) A ∇θlog⁡πθ(\tilde\rho/d)\,A\,\nabla_\theta \log \pi_\theta: its importance weight is ρ1/d\rho^{1/d} instead of ρ\rho, which stays close to 1.
  • One clipping threshold can be used for any latent size. Clipping ρ~\tilde\rho to [1−ϵ,1+ϵ][1-\epsilon, 1+\epsilon] bounds the joint log-ratio by about ±dϵ\pm d\epsilon, that is, ±ϵ\pm\epsilon per dimension on average. With ϵ=10−4\epsilon = 10^{-4}, the joint bound is ±6.6 nats for FLUX latents (d=65,536d = 65{,}536) and ±0.41 nats for ImageNet (d=4,096d = 4{,}096). We train both models with this ϵ\epsilon, although their latents differ 16-fold in size.

Influence on training. To measure the effect, we retrained the ImageNet model with the joint ratio ρ\rho in place of the normalized ratio ρ1/d\rho^{1/d}, with everything else unchanged:

RatioεFID at round 100 ↓\downarrowFinal FID ↓\downarrow
Normalized (ours)10−410^{-4}18.4516.72
Joint10−410^{-4}23.0917.93
Joint0.4 (≈ dε)18.4418.24

With the joint ratio and the same ε, the clipping is about dd times tighter. In the second update 99.7% of the samples are clipped, and convergence is much slower. Enlarging ε by about dd gives the same clipped fraction as the normalized ratio (19%) and the same early progress, as the clipping bound above predicts. Its importance weights are still ρ\rho rather than ρ1/d\rho^{1/d}, however, and its final FID is higher. The normalized ratio therefore converges at least as fast as the joint ratio and reaches the best FID, and its ε does not have to be retuned when the latent size changes.

We have extended App. A.3 with this derivation and comparison.

W3: An adapted Flow-GRPO baseline on the equal-time velocity, with matched sampling cost and training overhead.

Thank you for proposing this baseline. We ran Flow-GRPO on the equal-time velocity of the same FLUX.1-lite MeanFlow network, using Flow-GRPO's ten-step SDE rollouts, the same LoRA configuration and the same rollout budget (1,152 rollouts per round, 400 rounds). We also compare with MeanFlowNFT, an RL method designed for few-step flow maps. It is concurrent work that appeared on arXiv after our method was publicly released, so the original submission did not include it. We have now added it with its official hyperparameters and the same rollout budget. All methods are evaluated with EMA weights and the same flow-map sampler at K=4, so inference cost is matched. Training cost is reported as wall-clock time per round on 8×H100, which covers rollouts, reward computation and updates:

MethodEval KTime per roundOCR ↑\uparrow
Base4—0.349
Flow-GRPO (equal-time velocity)4441 s0.408
MeanFlowNFT4309 s0.713
Flow-Map GRPO (training K=2)4154 s0.762
Flow-Map GRPO (training K=3)4221 s0.770
Flow-Map GRPO (training K=4)4247 s0.850

At every training K, Flow-Map GRPO reaches a higher OCR than both baselines in less time per round. Flow-GRPO defines its RL objective on the instantaneous velocity, while the K=4 sampler uses the long-range map. Flow-Map GRPO optimizes this map directly.

We have also uploaded a revised version of the paper, with all changes highlighted in red.

Reviewer_zBamOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a framework for reward fine-tuning using policy gradient methods for few-step models, specifically MeanFlow and Consistency Models.

They address the problem that few-step models do not provide likelihoods, and with naive sample strategies they give deterministic transitions. They investigate two strategies to introduce stochasticity and likelihoods: Gaussian re-noising of a predicted one step sample; and Langevin SDE step correction style typical of stochastic sampling.

They experiment over several reward settings on FLUX few step models, showing improvements over the corresponding pretrained FlowMap and sCM.

Critical strengths
  • The proposed method is mathematically motivated, and the local anchor strategy performs well across several reward objectives. This local anchor strategy appears to be the main methodological contribution.
  • The ablation supports the importance of having the correct mathematical derivations for the stochastic transitions.
Critical weaknesses
  • Most of the main results compare only against the pre-trained few-step models. Although the paper discusses the relevant literature, it does not compare against methods addressing related tasks, like MeanFlowNFT. Table 3 compare against a re-adaption of DiffusionNFT, rather than its FlowMap counterpart.
  • The novelty is limited. Endpoint re-noising it is already an established sampling technique and already used for policy gradient training, such as in RLCM. The local anchor strategy appears to be the new technique but its comparison with endpoint re-noising it is limited to pre-trained models.
Questions for authors

I am willing to raise my score if the authors demonstrate that their main novel component offers advantages over current techniques.

  • Can the authors provide a thorough comparison between endpoint anchor and local anchor?
  • Can the authors compare against at least one recent work, such as MeanFlowNFT ?
  • Can the authors discuss similar policy gradients techniques used in methods combining distribution distillation matching with reinforcement learning (see [1,2]) ?
  • Can the authors clarify why they dismiss the mid-point anchor in Proposition 3.4 ? If their goal is to preserve the marginal distribution at every step, the SDE joint law, with λ=1\lambda=1 as in the local anchor derivation, yields the following closed-form Gaussian kernel mapping from pτp_\tau to prp_r: $K_{r\mid\tau}(\cdot\mid x) = \mathcal{N}\left(\frac{a_r}{a_\tau}x,\left(b_r^2-\frac{a_r^2}{a_\tau^2}b_\tau^2\right)I \right), \qquad r<\tau $

[1] https://arxiv.org/abs/2605.26108

[2] https://arxiv.org/pdf/2511.13649

7 comments
AuthorsOpenReview ↗

Response to Reviewer zBam (1/2)

We sincerely thank you for your thorough and constructive review. As you suggested, we compared our new component with existing techniques:

  • Local anchor vs. endpoint anchor after RL: OCR ↑\uparrow 0.850 vs. 0.387 on FLUX.1-lite MeanFlow (Table 9 already showed the same ordering in rollout quality before RL, which is why we chose the local anchor).
  • vs. MeanFlowNFT: OCR ↑\uparrow 0.850 (ours) vs. 0.713 and ImageNet FID ↓\downarrow 16.72 (ours) vs. 22.53, in less time.
  • Midpoint anchor: with the suggested kernel it reaches FID ↓\downarrow 16.67 on ImageNet; on text-to-image it still stays near the endpoint anchor (OCR ↑\uparrow: midpoint 0.480, endpoint 0.387, local 0.850).

Detailed responses on novelty, the midpoint kernel, the analysis of the anchor results and related work follow.

Q4,W2: The midpoint anchor and Proposition 3.4.

Thank you for suggesting the marginal-preserving midpoint kernel. It improves our midpoint anchor and does not contradict Proposition 3.4. Our original midpoint anchor was inspired by endpoint re-noising and used the interpolant's given pair (X0,X1)(X_0, X_1), the standard setting in flow matching. With this pair fixed, Proposition 3.4 requires posterior sampling. The suggested kernel instead injects fresh noise X0′X_0' with the same law as X0X_0, which removes the need for posterior sampling while keeping every marginal exact (Theorem 3.2).

As you noted, the kernel follows from the SDE joint law of our local-anchor derivation rather than from endpoint re-noising. This further separates ASFMC from endpoint re-noising and helps address W2. The anchors form a continuous family indexed by the anchor time τ, in which endpoint re-noising is only the τ = 1 case, and as τ → r the kernel coincides with the local anchor with λ²h = τ − r.

The local anchor reaches its anchor through the equal-time velocity at r instead of a long-range prediction ψr→τ\psi_{r\to\tau}, and this is the property that matters on text-to-image flow maps, where the midpoint anchor remains weaker than the local anchor, so our choice of the local anchor is unchanged (comparison and analysis in Q1). The revised paper states this family as Proposition 3.5 (Sec. 3.2, proof in App. A.4) and acknowledges the reviewers.

Discussion on novelty (W2).

ASFMC gives a unified framework. To our knowledge, Flow-Map GRPO is the first likelihood-ratio RL method for pretrained MeanFlow generators and the first framework spanning both two-time (MeanFlow) and one-time (consistency) flow maps. It was also released before MeanFlowNFT. The consistency re-noising rule used in RLCM is only the endpoint anchor (τ = 1) of ASFMC, which we use for one-time maps. The local anchor (τ → r), which makes two-time maps trainable, and the midpoint anchor place the anchor at τ < 1 and cannot be obtained from RLCM's endpoint re-noising; the anchored construction that yields them is new. The Gaussian SDE discretization used by Flow-GRPO does not extend to a finite flow-map step (Theorem 3.1), whereas Theorem 3.2 gives the marginal-preservation condition, under which the anchors yield tractable Gaussian policies. This gives the anchor family and a criterion for choosing among them (see Q1 below). The new construction also matters in practice: after RL, the local anchor is far better than the endpoint anchor in the text-to-image setting (see Q1 below).

AuthorsOpenReview ↗

Response to Reviewer zBam (2/2)

Q1,W2: A thorough comparison between endpoint, midpoint (suggested kernel) and local anchors.

Table 5 compares all anchors after RL. The three marginal-preserving anchors share the pretrained model, optimization settings and rollout budget, and the FLUX OCR of the last row is from Table 6:

AnchorFLUX OCR ↑\uparrowImageNet 5-step FID ↓\downarrowImageNet FID, 3 seeds ↓\downarrow
Base0.34925.43—
Local (τ → r)0.85016.7216.65 ± 0.12
Midpoint (τ = (r+1)/2, suggested kernel)0.48016.6716.51 ± 0.18
Endpoint (τ = 1)0.38716.6416.68 ± 0.32
Midpoint (non-marginal-preserving scale)0.37226.1826.42 ± 0.77

On text-to-image flow maps, the local anchor far outperforms the endpoint anchor and the midpoint anchor. The OCR after RL decreases as τ moves away from r, because noise is then injected through longer-range predictions, which are not fully accurate in practice for large text-to-image models. Two observations support this: (i) the FLUX.1-lite MeanFlow does not support single-step sampling (Pu et al., 2025), and its quality rises with more sampling steps (Table 13); (ii) rollouts through the endpoint prediction are much worse even before RL (OCR 0.444 local vs. 0.208 endpoint, already reported in Table 9). For one-time maps the situation is different. A model such as sCM is itself trained and sampled through the endpoint prediction ψt→1\psi_{t\to 1}, so the endpoint anchor is the natural choice there, and with it sCM improves OCR from 0.490 to 0.886 (Table 2).

When the flow map is accurate (ImageNet), all three anchors reach nearly the same FID. The local anchor matches the other anchors when the flow map is accurate and is far better when it is not. Without the marginal-preserving scale, the midpoint anchor gives no improvement (26.18 vs. 25.43 for Base).

Q2,W1: Compare against at least one recent work, such as MeanFlowNFT.

We have added MeanFlowNFT to Table 3 and Table 10. MeanFlowNFT is concurrent work that appeared on arXiv after our method was publicly released, so the original submission did not include it. We use its official hyperparameters and the same rollout budget (one adapter per reward):

MethodFLUX OCR ↑\uparrowFLUX PickScore ↑\uparrowImageNet FID (5 / 1 step) ↓\downarrowTime per round (FLUX)
Base0.34920.9825.43 / 29.42—
MeanFlowNFT0.71322.1922.53 / 26.17309 s
Flow-Map GRPO0.85023.0816.72 / 21.81247 s

Flow-Map GRPO is best on every metric and takes the least time per round. Over 3 seeds, the ImageNet 5-step FID is 16.65 ± 0.12 for Flow-Map GRPO and 22.55 ± 0.15 for MeanFlowNFT.

Q3,W2: Policy-gradient techniques in distribution matching + RL [1, 2].

We now discuss both works in Sec. 5 and App. B. Both [1, 2] belong to distribution matching distillation, a different category from trajectory distillation such as flow maps (MeanFlow, consistency models). A distribution matching generator typically takes a single time input (utu_t) and needs an extra fake-score model, whereas trajectory distillation learns a two-time map (ut→ru_{t\to r}, MeanFlow), in some cases with one time fixed (ψt→1\psi_{t\to 1}, consistency models). Moreover, they add rewards while distilling a generator from a multi-step teacher, whereas we post-train an already distilled flow map with no teacher. RTDMD [1] matches a reward-tilted teacher, and DMDR [2] adds a plug-in reward loss (ReFL, DPO, or GRPO) to DMD. The policy-gradient part of RTDMD, like RLCM, re-noises the endpoint prediction ψt→1\psi_{t\to 1}; with full re-noising this is the consistency sampler, i.e., our endpoint anchor. This endpoint policy suits generators trained for endpoint prediction, and it is what we use for sCM (OCR 0.490 → 0.886). For a two-time flow map such as MeanFlow, whose long-range prediction ψt→1\psi_{t\to 1} is less accurate (see Q1 above), it reaches only 0.387 after RL, versus 0.850 with the local anchor.

Flow-Map GRPO needs only the flow map and the reward, and supports two-time maps through the local anchor.

We have also uploaded a revised version of the paper, with all changes highlighted in red.

I thank the authors for the additional experiments, particularly the MeanFlowNFT comparison and the midpoint comparison with the marginal-preserving kernel.

My novelty concerns remain. RLCM already uses likelihood-based RL for consistency models, and the extension to two-time flow maps appears relatively easy. Related re-noising constructions also exist in distribution-matching methods and flow-map samplers [1,2]. Moreover, how does stopping gradients through the local velocity prediction relate to the claimed likelihood-based policy-gradient objective?

I would appreciate a clearer explanation of why local anchors perform better, given confounding factors such as exploration. Could the authors say which predictions are detached for each anchor and whether the midpoint/endpoint implementations use two composed flow map evalutations or one direct prediction?

Nevertheless, the additional results demonstrate the local algorithm’s practical value, and I am raising my score to 2. I remain open to raising it further following:

  • A more precise description of the novelty and its relationship to existing constructions.
  • A more precise exposition of the implemented algorithm.
  • A clearer explanation of the reason for local anchor’s better performance.

[1] https://arxiv.org/abs/2310.02279

[2] https://arxiv.org/abs/2510.27684

Reviewer_pCUMOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces Flow-map GRPO, a reward finetuning method for optimizing few-step flow map models. The authors take inspiration from Flow-GRPO and aim to adapt it to few-step flow maps. This typically requires some form of stochastic sampling from the model, yet the stochastic samplers used in standard diffusion models are not directly compatible with few-step flow maps. To address this, the authors propose a different way of constructing a stochastic sampler using flow maps. Specifically, this is done by using the flow map to denoise slightly beyond the target noise level, followed by re-noising, similar to how multi-step sampling is done with consistency models. Each step of this stochastic process has a Gaussian form, allowing likelihood ratios to be easily computed and used with GRPO. Finally, Flow-map GRPO is evaluated on both ImageNet and text-to-image generation, with results showing that the method can successfully post-train the base flow-map models.

Critical strengths

The core problem the paper aims to address is an important one, and this work appears to be among the first to tackle it. Specifically, while there is a large body of work on reward fine-tuning diffusion and flow models, very few have attempted to adapt these ideas to few-step generators such as flow maps.

Critical weaknesses

The paper's writing is a bit difficult to follow and understand. There were several instances where I had to read a section multiple times to understand the argument, which was made even more challenging by the fact that some of the deductions or conclusions reached by the authors appear to be either incorrect or in need of further clarification. Here are a few examples:

  • As far as I understand, the discussion around Theorem 3.2 revolves around the idea that a proper stochastic sampler must preserve the true marginals, i.e. Xτ∼pτ⇒Xr~∼prX_\tau \sim p_\tau \Rightarrow \tilde{X_r} \sim p_r. One way of achieving this is by sampling from the exact conditional p(Xr∣Xτ)p(X_r | X_\tau). However when τ≠1\tau \neq 1, this is non-gaussian and hard to compute. I believe this is what proposition 3.4 is aiming to establish. However, this isn’t the only way to preserve the marginals, and it’s possible to do so without using the exact conditional. Namely, assuming the flow-map is exact, a simple scaling + gaussian noise injection can preserve the marginals exactly, using slightly different coefficients. Specifically,

    Xr~=araτxτ+br2−ar2bτ2aτ2ϵ\tilde{X_r} = \frac{a_r}{a_\tau} x_\tau + \sqrt{b_r ^ 2 - a_r^2 \frac{b_\tau^2}{a_\tau^2}} \epsilon

    where ϵ\epsilon is gaussian noise, gives the correct marginals. With this in mind, the discussion in lines 262-266 is confusing. Using the midpoint anchor doesn’t necessarily require posterior sampling if the above approach is used. This needs further clarification. I suspect that the midpoint experiments in the appendix may have used the incorrect noise scale cr,τ=br−arbτaτc_{r, \tau} = b_r - a_r \frac{b_\tau}{a_\tau}, which could explain the poor performance.

  • The constant-noise endpoint-anchor ablation in lines 367-371 (with further details in lines 1155-1167) is also confusing. As far as I can tell, in the original variant (referred to as “ours”). the authors perform standard multi-step consistency model sampling, i.e. Xr=arψt→1(Xt)+brϵX_r = a_r \psi_{t \to 1}(X_t) + b_r \epsilon where ϵ∼N(0,I)\epsilon \sim \mathcal{N}(0, I). In the appendix, they specify that 4-step sampling is performed using the schedule (0.7625, 0.525, 0.2875, 0.05). Yet, in the ablation, they only change the noise scale brb_r, replacing it with a constant σCM=1K∑i=1Kbr2≈0.485\sigma_{CM} = \sqrt{\frac{1}{K}\sum_{i=1}^{K} b_r^2} \approx 0.485, while keeping ara_r unchanged. This is clearly going to cause problems, since the resulting intermediate samples will no longer come from the marginals. For example, X~0.7625≁p0.7625\tilde{X}_{0.7625} \not\sim p_{0.7625} since too little noise was added. This makes the samples out-of-distribution for the next step, potentially leading to worse predictions and compounding errors throughout the sampling process. I fail to see what this ablation is trying to show, given that the degradation in performance is an expected consequence of using incorrect noise scales.

  • A similar argument can be made for the local-anchor ablations. If the correct drift and noise scales are replaced with ones that no longer preserve the marginals, the resulting samples will be out-of-distribution for the next step, leading to worse model predictions and potentially compounding errors. As with the previous ablation, I fail to see how outperforming these variants is not an expected outcome, or what conclusions can be drawn from these comparisons.

  • The results for DiffusionNFT in Table 3 are much worse than the ones reported in prior works[1,2]. Appendix C.4 gives further detail, and I believe the reason for the poor performance is that DNFT/SFT are both being trained and evaluated at only 4-steps. Why is this a fair comparison? These methods are diffusion finetuning methods and are not meant to achieve good few-step generation, so obviously evaluating them at this setting will give poor results. This makes the comparison in Table 3 a bit misleading. I would suggest that the authors train and evaluate these methods using their original settings for a fairer comparison. Additionally, including an evaluation NFE column and reporting training wall-clock times would help better show the advantages and disadvantages of each approach.


[1] Zheng, Kaiwen, et al. "Diffusionnft: Online diffusion reinforcement with forward process." International Conference on Learning Representations. Vol. 2026. 2026.

[2] Bergmeister, Andreas, et al. "Reinforce Adjoint Matching: Scaling RL Post-Training of Diffusion and Flow-Matching Models." arXiv preprint arXiv:2605.10759 (2026).

Questions for authors
  • Have the authors tried comparing between flow-map distilling a finetuned model vs. finetuning a flow-map distilled model?
  • The main advantage of flow-maps compared to diffusion/flow models is their faster sampling speed. Therefore, one would expect for these to excel at this OnlineRL-like setting where every step requires generating a large number of samples from the model. Do the authors have any timing results comparing training speeds between Flowmap-GRPO and prior diffusion finetuning methods such as DiffusionNFT?
4 comments
AuthorsOpenReview ↗

Response to Reviewer pCUM (1/2)

We sincerely thank you for your careful reading and for proposing the marginal-preserving midpoint kernel. In summary:

  • Midpoint anchor: the suggested kernel satisfies Theorem 3.2 and does not contradict Proposition 3.4. With it, the midpoint anchor reaches FID ↓\downarrow 16.67 on ImageNet, versus 26.18 with our earlier scale; on text-to-image it still stays near the endpoint anchor (OCR ↑\uparrow: midpoint 0.480, endpoint 0.387, local 0.850).
  • Fair few-step comparison: against MeanFlowNFT, the flow-map counterpart of DiffusionNFT, Flow-Map GRPO reaches OCR ↑\uparrow 0.850 vs. 0.713 at 247 s vs. 309 s per round.
  • Few-step setting: Flow-Map GRPO post-trains the deployed few-step generator itself. With 5 steps it comes close to multi-step RL models at 40 steps (OCR ↑\uparrow 0.912 vs. 0.924 for DiffusionNFT on SD3.5-Medium) at one sixth of the sampling time.

W1: The midpoint kernel of lines 262–266 and its noise scale.

Thank you for suggesting the marginal-preserving midpoint kernel. The suggested kernel improves our midpoint anchor and does not contradict Proposition 3.4. Our original midpoint anchor was inspired by endpoint re-noising and used the interpolant's given pair (X0,X1)(X_0, X_1), the standard setting in flow matching. With this pair fixed, Proposition 3.4 requires posterior sampling, as described in lines 262–266. The suggested kernel instead injects fresh noise X0′X_0' with the same law as X0X_0, which removes the need for posterior sampling while keeping every marginal exact (Theorem 3.2). As Reviewer zBam notes, the same kernel can be derived from the SDE joint law of our local-anchor derivation rather than from endpoint re-noising. This further separates ASFMC from endpoint re-noising: the anchors form a continuous family indexed by the anchor time τ, in which endpoint re-noising is only the τ = 1 case, and as τ → r the kernel coincides with the local anchor with λ²h = τ − r. Table 5 compares the anchors after RL (the FLUX OCR of the last row is from Table 6):

AnchorImageNet 5-step FID ↓\downarrowImageNet FID, 3 seeds ↓\downarrowFLUX OCR ↑\uparrow
Base25.43—0.349
Local (τ → r)16.7216.65 ± 0.120.850
Midpoint (τ = (r+1)/2, suggested kernel)16.6716.51 ± 0.180.480
Endpoint (τ = 1)16.6416.68 ± 0.320.387
Midpoint (non-marginal-preserving scale)26.1826.42 ± 0.770.372

On the noise scale: our earlier midpoint experiment used cr,τc_{r,\tau}, which is derived from the interpolant identity of Proposition 3.4 with X0X_0 replaced by independent noise and does not guarantee the marginals. With the marginal-preserving scale γr,τ\gamma_{r,\tau} of your kernel, the midpoint anchor reaches FID 16.67 instead of 26.18, matching the other two anchors. As you suspected, cr,τc_{r,\tau} is indeed smaller than γr,τ\gamma_{r,\tau}.

The local anchor remains the preferred choice. On ImageNet, where the flow map is accurate, the three anchors perform alike. On FLUX, however, OCR decreases as τ moves away from r. This is because noise is then injected through longer-range predictions, which are not fully accurate in practice for large text-to-image models. Two observations support this: (i) the FLUX.1-lite MeanFlow does not support single-step sampling (Pu et al., 2025), and its quality rises with more sampling steps (Table 13); (ii) rollouts through the endpoint prediction are much worse even before RL (OCR 0.444 local vs. 0.208 endpoint, already reported in Table 9). The midpoint anchor therefore remains weaker than the local anchor on text-to-image. The revised paper states this family as Proposition 3.5 (Sec. 3.2, proof in App. A.4) and acknowledges the reviewers.

W2, W3: What is the purpose of the constant-noise and no-drift ablations?

To make a deterministic flow map stochastic, the most direct approach is to add Gaussian noise to the sampler at each step, and we designed these ablations to test it. The no-drift variants add the noise directly, without the drift correction, and the constant-noise variants add noise of a constant scale, so neither preserves the marginals. After RL, FID is 15.65 for ours versus 42.16 for the constant-noise endpoint anchor on the consistency model (W2), and 16.72 versus 32.71, 70.13 and 49.25 for the local-anchor variants on MeanFlow (W3). As you point out, this degradation is expected. The ablations quantify it and confirm that the drift correction and the noise schedule are both needed, consistent with Theorem 3.2. The most controlled comparison is the midpoint anchor with and without the marginal-preserving scale (W1), where only the noise scale differs. We now state this motivation in Sec. 4.1.

AuthorsOpenReview ↗

Response to Reviewer pCUM (2/2)

W4: Is the DiffusionNFT comparison at 5 NFE fair?

Our paper studies few-step generation. DiffusionNFT and SFT are designed for multi-step models, and our purpose in comparing with them in Table 3 was to show that these multi-step objectives cannot be applied directly to few-step training (on MeanFlow, OCR 0.358 and 0.336, versus 0.349 for Base). Post-training a multi-step model in its original setting does not give it few-step sampling either, since it still needs 28–40 NFE afterwards, which is a different problem from ours. In their original multi-step setting, the public DiffusionNFT checkpoint of SD3.5-Medium (40 steps without CFG) reaches OCR 0.924 at 0.503 s per image on the same prompts, and the public Flow-GRPO checkpoint (40 steps with CFG) reaches 0.930 at 0.956 s. Flow-Map GRPO on SD3.5-Medium AnyFlow reaches 0.912 with 5 steps at 0.081 s, a comparable score 6 to 12 times faster. The right comparison among few-step methods is with MeanFlowNFT, which adapts DiffusionNFT to MeanFlow. It is concurrent work that appeared on arXiv after our method was publicly released, so the original submission did not include it. We agree that a fair comparison is needed, and have now added it with its official hyperparameters and an identical rollout budget and number of rounds (Table 3 and Table 10; one adapter per reward):

MethodEval NFETime per roundFLUX OCR ↑\uparrowFLUX PickScore ↑\uparrowImageNet FID (5 / 1 step) ↓\downarrow
Base5—0.34920.9825.43 / 29.42
MeanFlowNFT5309 s0.71322.1922.53 / 26.17
Flow-Map GRPO5247 s0.85023.0816.72 / 21.81

Over 3 seeds, the ImageNet 5-step FID is 16.65 ± 0.12 for Flow-Map GRPO and 22.55 ± 0.15 for MeanFlowNFT.

Q1: Distilling a fine-tuned model vs. fine-tuning a distilled model.

We did not run the first pipeline, because it needs a full distillation run for every new reward, which is exactly the cost that Flow-Map GRPO avoids. The two pipelines also optimize different targets. Applying RL before distillation optimizes the multi-step teacher, so the student receives the reward only indirectly, through distillation. Flow-Map GRPO instead optimizes the deployed few-step sampler itself, with no teacher and no distillation stage. It trains only a LoRA on the released flow map, which takes about 27 hours on 8×H100 for the OCR reward (400 rounds of two updates each at 247 s per round, roughly 220 GPU-hours). For comparison, the first pipeline reruns distillation for each reward, and the FLUX.1-lite MeanFlow we start from was distilled with 25,000 iterations of the full 8-billion-parameter model on 32 H20 GPUs (Pu et al., 2025), whereas our post-training uses 800 LoRA updates on 8 H100 GPUs. Flow-Map GRPO is therefore more flexible, since a new reward only needs a new LoRA on the same flow map. Reward-aware distillation (as noted by Reviewer zBam) follows that pipeline for distribution matching generators and is a complementary direction.

Q2: The main advantage of flow maps is faster sampling; how does training time compare?

The faster sampling of flow maps is an advantage at inference and in training, and Flow-Map GRPO keeps both.

At inference. A flow-matching model typically needs 28–40 NFE, before and after post-training (see W4 above). A flow map needs ≤ 5 NFE, and it still does after our post-training, with OCR rising from 0.349 to 0.850 at 5 NFE. This is the main benefit, since a model is trained once and sampled many times.

In training. As in W4, the right reference is MeanFlowNFT, since DiffusionNFT trains a multi-step model and its training time measures a different task. With 1,152 rollouts per round on 8×H100, a round takes 247 s for Flow-Map GRPO, compared to 309 s for MeanFlowNFT, while Flow-GRPO with its ten-step SDE rollouts on the same network takes 441 s (OCR 0.408). Rollouts with fewer stochastic transitions K are even faster and still give strong results (all evaluated with 5 NFE after 800 updates):

Flow-Map GRPOTime per roundFLUX OCR ↑\uparrow
K=2154 s0.762
K=3221 s0.770
K=4247 s0.850

Even at K=2 (3 rollout steps), Flow-Map GRPO outperforms MeanFlowNFT with its 5 rollout steps (OCR 0.762 vs. 0.713) in half the time per round (154 s vs. 309 s). In wall-clock terms, the K=2 run passes MeanFlowNFT's final OCR of 0.713 after 15.4 hours (360 rounds), whereas MeanFlowNFT's 400 rounds take 34.3 hours.

Presentation: the writing is difficult to follow.

Following your feedback, Sec. 3.2 now presents the anchor family (endpoint, midpoint, local) and states it as Proposition 3.5, and Sec. 4.1 states the purpose of the ablations before their results and adds a comparison of the anchors.

We have also uploaded a revised version of the paper, with all changes highlighted in red.

I thank the authors for their response. It's very interesting to see the large gap in text-to-image performance remain between the midpoint and local anchors, even after using the corrected midpoint kernel. I also appreciate the authors' additional clarifications and timing analysis. The rebuttal addresses most of concerns, and therefore I raise my score

Thank you for your careful reading, for proposing the marginal-preserving midpoint kernel, and for raising your score. Your suggestions have helped us improve the paper.

Reviewer_txEeOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Flow-Map GRPO, a framework for reward-based post-training of pretrained few-step flow-map generators. Its central component, Anchored Stochastic Flow Map Composition (ASFMC), constructs tractable stochastic policies through deterministic transport and anchor-based resampling. The local-anchor formulation supports MeanFlow, while the endpoint-anchor formulation accommodates consistency models. The paper provides marginal-preservation conditions and local approximation analysis, and evaluates LoRA-based post-training on ImageNet and FLUX-based text-to-image generators.

Critical strengths

(1) This paper studies a relevant problem with a principled methodological solution. Enabling reward-based adaptation of pretrained few-step generators is practically valuable. The local-anchor construction provides a concrete way to introduce stochastic exploration and tractable transition densities for MeanFlow, while retaining its existing parameterization and deterministic inference procedure. (2) The empirical improvements are substantial. The reported gains span text rendering, human-preference scores, and compositional generation.

Critical weaknesses

(1) The comparative evaluation does not fully establish the advantage over well-adapted alternatives. The paper explains that its native SFT and directly adapted DiffusionNFT baselines have objectives that are mismatched to flow-map predictions. While these comparisons demonstrate the limitations of straightforward adaptations, they provide weaker evidence of superiority over methods specifically designed for this setting. A clearer discussion of this distinction, together with a more compatible baseline where feasible, would strengthen the empirical positioning. (2) The scope of the theoretical and methodological contribution could be clearer. The endpoint-anchor formulation is closely connected to existing consistency-sampling and reward-optimization approaches, while the local formulation admits a Langevin-style interpretation. The main novelty therefore appears concentrated in the MeanFlow policy construction and its unified treatment. (3) The evaluation scope remains limited. The text-to-image experiments use two generator variants from the same FLUX family. Furthermore, improvements in the optimized reward sometimes accompany declines in auxiliary quality metrics.

Questions for authors

(1) Could the authors clarify what the baseline comparisons establish given the identified objective mismatches, and discuss how a flow-map-compatible alternative, such as the concurrent MeanFlowNFT, would change the comparison? (2) Could the authors more explicitly separate the new contribution of the local-anchor policy from existing Langevin-style corrections and consistency-sampler policies, and clarify which theoretical properties remain approximate in the implemented algorithm? (3) Could the authors provide variability estimates for representative experiments and discuss the auxiliary-quality degradation observed under some single-reward objectives?

2 comments
AuthorsOpenReview ↗

Response to Reviewer txEe (1/2)

We sincerely thank you for your positive assessment and your questions. In summary:

  • Compatible baselines: Flow-Map GRPO reaches OCR 0.850, versus 0.713 for MeanFlowNFT and 0.408 for Flow-GRPO, at a lower training cost per round than either (Q1).
  • Contribution: the first likelihood-ratio RL method for MeanFlow, derived from a marginal-preserving construction that yields a continuous family of anchors (Q2).
  • Broader scope: on a second model family, SD3.5-Medium AnyFlow, OCR improves from 0.575 to 0.912 (Q3).

Q1,W1: What do the baselines establish, and how would a flow-map-compatible alternative such as MeanFlowNFT change the comparison?

SFT and DiffusionNFT are designed for multi-step models. When we developed Flow-Map GRPO, no RL post-training method existed for MeanFlow, so we compared with them in Table 3 to show that their objectives cannot be applied directly to few-step training, which is the gap Flow-Map GRPO fills.

The flow-map-compatible alternative is MeanFlowNFT. It is concurrent work that appeared on arXiv after our method was publicly released, so the original submission did not include it. We have now added it with its official hyperparameters, together with Flow-GRPO applied to the equal-time velocity of the same MeanFlow network. MeanFlowNFT, Flow-GRPO and Flow-Map GRPO share the model, reward functions, rollout budget, number of rounds and K=4 evaluation (Tables 3 and 10):

MethodTime per roundFLUX OCR ↑\uparrowFLUX PickScore ↑\uparrow
Base—0.34920.98
SFT—0.33620.98
DiffusionNFT—0.35820.54
Flow-GRPO (equal-time velocity)441 s0.408—
MeanFlowNFT309 s0.71322.19
Flow-Map GRPO (K=2)154 s0.762—
Flow-Map GRPO (K=3)221 s0.770—
Flow-Map GRPO (K=4)247 s0.85023.08

On ImageNet, MeanFlowNFT reaches FID 22.53, versus 16.72 for Flow-Map GRPO. At every K, Flow-Map GRPO reaches a higher OCR than both compatible baselines in less time per round, because it makes the deployed long-range map itself the stochastic policy. Even with fewer stochastic transitions in training (K=2), it exceeds MeanFlowNFT in half the time per round.

Q2,W2: Separate the local-anchor policy from Langevin-style corrections and consistency-sampler policies; which properties remain approximate?

We agree that the main novelty lies in the MeanFlow policy construction and its unified treatment. To our knowledge, Flow-Map GRPO is the first likelihood-ratio RL method for pretrained MeanFlow generators and the first framework spanning both two-time (MeanFlow) and one-time (consistency) flow maps.

  • Relation to consistency-sampler policies. The consistency re-noising rule (used in RLCM) is only our endpoint anchor (τ = 1), which we use for one-time maps. The kernel suggested by Reviewers zBam and pCUM connects it to our local anchor through a continuous family indexed by the anchor time τ: values r < τ < 1 give midpoint anchors, and as τ → r the kernel coincides with the local anchor, which uses the equal-time velocity at r instead of a long-range prediction.
  • Relation to Langevin corrections. Three points clarify how our contribution relates to a Langevin-style corrector.
    • (i) From the definition of a flow map, the natural approach is to integrate the SDE over the flow-map step. We write out this integral and show that it does not in general give an additive-Gaussian transition (Theorem 3.1). The direct variants also fail in practice, as adding the same noise without the drift gives ImageNet FID 70.13, against 16.72 for ours and 25.43 for Base (Table 1).
    • (ii) ASFMC gives the correct construction from a unified view. It derives the drift–noise pairing from a marginal-preserving principle (Theorem 3.2), which yields a family of anchors, and transport and correction form a single Gaussian transition whose trainable mean is the long-range flow map.
    • (iii) The local-anchor update admits the Langevin-style interpretation you mention (App. A.5). Flow-GRPO also adds a Langevin-type term through its ODE-to-SDE conversion, and its contribution is to show for the first time that online RL works for flow matching. Likewise, ours is to show for the first time that likelihood-ratio RL works for flow maps: the correction is built into a tractable Gaussian policy around the pretrained long-range transition, so RL optimizes the two-time flow map directly and keeps its few-step sampling.
AuthorsOpenReview ↗

Response to Reviewer txEe (2/2)

  • What is exact and what is approximate. Exact: the likelihood ratio, because each transition is Gaussian by construction (optimization uses its dimension-normalized form, App. A.3), and the marginals of the endpoint anchor for an exact map under the affine interpolant (Theorem 3.2). Approximate: the local anchor matches the exact marginal-preserving transition up to an O(h3/2)O(h^{3/2}) error in L2L^2 (Theorem 3.3). Both results assume an exact flow map, so for learned maps we compare stochastic rollouts with deterministic sampling of the same model (App. C.6). At K=4, local-anchor rollouts reach FID 26.19 before RL and 15.20 after RL, against 25.43 and 16.72 for deterministic sampling.

Q3,W3: Evaluation scope, variability, and auxiliary quality.

Second model family. Following your suggestion to broaden the evaluation, we added SD3.5-Medium AnyFlow, which differs from our FLUX models in both the backbone (SD3.5-Medium) and the flow-map distillation method (AnyFlow instead of MeanFlow or sCM). Results use 5-step sampling and one adapter per reward:

RewardBaseFlow-Map GRPO
OCR0.5750.912
PickScore21.6923.52
GenEval0.7260.868

Variability. Over 3 seeds on ImageNet (5-step FID, mean ± SD), Flow-Map GRPO reaches 16.65 ± 0.12 with the local anchor and 16.68 ± 0.32 with the endpoint anchor, versus 22.55 ± 0.15 for MeanFlowNFT. The gap to MeanFlowNFT is 5.9 FID. The text-to-image results use one training seed per configuration because of their training cost (about 27 hours on 8×H100 per run), and the seed variation on ImageNet is small.

Auxiliary quality. A task reward such as OCR or GenEval scores one specific property (rendered text, object composition), so a trade-off with some untargeted general-quality metrics is normal. We report the auxiliary DrawBench metrics to check whether this trade-off becomes a broad loss of quality. We observe no such loss: the declines are confined to a few metrics.

With the OCR reward, three of the five DrawBench metrics rise on both models (Table 2). The other two decline: Aesthetic 5.54 → 5.51 and DeQA 4.17 → 4.14 on MeanFlow, and ImageReward 0.575 → 0.348 and Aesthetic 5.47 → 5.40 on sCM. The largest decline is under GenEval on MeanFlow (DeQA 4.17 → 3.21, Aesthetic 5.54 → 5.39), where DB-PickScore, ImageReward and UnifiedReward still rise (Table 4).

The trade-off depends on the optimized reward. With the same method, optimizing PickScore alone improves four of five DrawBench metrics on MeanFlow and all five on sCM, and joint OCR–PickScore training with a single adapter improves both targets and all five DrawBench metrics on both models (MeanFlow OCR 0.349 → 0.772, PickScore 20.98 → 22.50; Table 4). Under the joint reward, no auxiliary metric declines.

We have also uploaded a revised version of the paper, with all changes highlighted in red.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response and summary of the revision (1/3)

We sincerely thank all reviewers for their time and constructive suggestions, which have strengthened the paper. We are encouraged that the reviewers find the problem "an important one" and the work "among the first to tackle it" (pCUM), the method "mathematically motivated" with a local anchor that "performs well across several reward objectives" (zBam), "a relevant problem with a principled methodological solution" with "substantial" empirical improvements (txEe), and a paper that "solves an important challenge" (ma2p).

All revisions are highlighted in red in the updated PDF. We summarize the main points below and reply to each reviewer in the individual threads.

A. Novelty (zBam, txEe). To our knowledge, Flow-Map GRPO is the first likelihood-ratio RL method for pretrained MeanFlow generators and the first framework spanning both two-time (MeanFlow) and one-time (consistency) flow maps. It was also released before MeanFlowNFT.

  • From the definition of a flow map, the natural approach is to integrate the SDE over the flow-map step. We write out this integral and show that it does not in general give an additive-Gaussian transition (Theorem 3.1). The direct variants also fail in practice: adding the same noise without the drift gives ImageNet FID 70.13, against 16.72 for ours and 25.43 for Base (Table 1).
  • ASFMC gives the correct construction from a unified view. It derives the drift–noise pairing from a marginal-preserving principle (Theorem 3.2), which yields a family of anchors and a criterion for choosing among them.
  • Flow-GRPO adds a Langevin-type term through its ODE-to-SDE conversion, and its contribution is to show for the first time that online RL works for flow matching. Likewise, ours is to show for the first time that likelihood-ratio RL works for two-time flow maps.
  • The consistency re-noising rule used in RLCM is only the endpoint anchor (τ = 1) of ASFMC, which we use for one-time maps (sCM: OCR 0.490 → 0.886, Table 2). The midpoint anchors (r < τ < 1) and the local anchor (τ → r) place the anchor at τ < 1 and cannot be obtained by re-noising an endpoint prediction. The local anchor, which makes two-time maps trainable, uses the equal-time velocity at r instead of a long-range prediction.

B. A family of anchors, and the choice of the local anchor (zBam, pCUM; Proposition 3.5, Sec. 4.1, App. A.4). We are especially grateful to Reviewers zBam and pCUM for proposing the marginal-preserving midpoint kernel, which improves our midpoint anchor and is now stated, with acknowledgement, as Proposition 3.5 in the paper. It does not contradict Proposition 3.4. Our original midpoint anchor used the interpolant's given pair (X0,X1)(X_0, X_1), and for this pair Proposition 3.4 requires posterior sampling. The suggested kernel instead injects fresh noise X0′X_0' and keeps every marginal exact (Theorem 3.2). The kernel follows from the SDE joint law of our local-anchor derivation rather than from endpoint re-noising. This further separates ASFMC from endpoint re-noising: the anchors form a continuous family indexed by the anchor time τ, in which endpoint re-noising is only the τ = 1 case, and as τ → r the kernel coincides with the local anchor with λ²h = τ − r.

Table 5 compares the anchors after RL. The three marginal-preserving anchors share the pretrained model and hyperparameters, and the FLUX OCR of the last row is from Table 6:

AnchorImageNet 5-step FID ↓\downarrowImageNet FID, 3 seeds ↓\downarrowFLUX OCR ↑\uparrow
Base25.43—0.349
Local (τ → r)16.7216.65 ± 0.120.850
Midpoint (τ = (r+1)/2, suggested kernel)16.6716.51 ± 0.180.480
Endpoint (τ = 1)16.6416.68 ± 0.320.387
Midpoint (non-marginal-preserving scale)26.1826.42 ± 0.770.372

On ImageNet, where the flow map is accurate, the three marginal-preserving anchors perform similarly, while our original midpoint anchor, without the marginal-preserving scale, does not improve over Base. On text-to-image flow maps, OCR decreases as τ moves away from r, because noise is then injected through longer-range predictions, which are not fully accurate in practice for large text-to-image models (supported by the FLUX.1-lite MeanFlow not supporting single-step sampling, Pu et al., 2025, and by its quality rising with more sampling steps, Table 13). The same ordering is already visible before RL (Table 9), where local-anchor rollouts reach OCR 0.444 and PickScore 20.72, against 0.208 and 17.51 for the endpoint anchor. The midpoint anchor remains weaker than the local anchor on text-to-image, so our choice of the local anchor is unchanged.

AuthorsOpenReview ↗

General response and summary of the revision (2/3)

C. Comparison with few-step baselines (all reviewers; Tables 3 and 10, App. C.4). MeanFlowNFT is concurrent work that appeared on arXiv after our method was publicly released, so the original submission did not include it. We have added it with its official hyperparameters, together with Flow-GRPO applied to the equal-time velocity of the same network. The three methods share the FLUX.1-lite MeanFlow model, reward functions, LoRA setting, rollout budget, number of rounds and K=4 evaluation. Time is wall-clock per round on 8×H100:

MethodTime per roundOCR ↑\uparrowPickScore ↑\uparrow
Base—0.34920.98
Flow-GRPO (equal-time velocity)441 s0.408—
MeanFlowNFT309 s0.71322.19
Flow-Map GRPO (training K=2)154 s0.762—
Flow-Map GRPO (training K=3)221 s0.770—
Flow-Map GRPO (training K=4)247 s0.85023.08

At every training K, Flow-Map GRPO reaches a higher OCR than both baselines in less time per round, and with training K=2 it exceeds MeanFlowNFT in half the time. On ImageNet, the 5-step FID is 16.72 for Flow-Map GRPO versus 22.53 for MeanFlowNFT, and the 1-step FID is 21.81 versus 26.17.

SFT and DiffusionNFT are designed for multi-step models. When we developed Flow-Map GRPO, no RL post-training method existed for MeanFlow, so we compared with them in Table 3 to show that their objectives cannot be applied directly to few-step training (on MeanFlow, OCR 0.336 and 0.358, versus 0.349 for Base). In their original setting they still need 28–40 NFE, whereas Flow-Map GRPO post-trains the deployed few-step generator and keeps its sampling at ≤ 5 NFE. The fair few-step comparison is therefore the one with MeanFlowNFT. As a multi-step reference on SD3.5-Medium, the public DiffusionNFT and Flow-GRPO checkpoints reach OCR 0.924 and 0.930 with 40 steps, while Flow-Map GRPO on SD3.5-Medium AnyFlow reaches 0.912 with 5 steps, 6 to 12 times faster. Flow-Map GRPO also trains only a LoRA on the released flow map, so a new reward needs a new LoRA rather than a new distillation run.

AuthorsOpenReview ↗

General response and summary of the revision (3/3)

D. A second model family and variability (txEe; Sec. 4.2, App. C.5). To show that Flow-Map GRPO transfers beyond the FLUX models, we added SD3.5-Medium AnyFlow, which differs from them in both the backbone (SD3.5-Medium) and the flow-map distillation method (AnyFlow instead of MeanFlow or sCM). On this model (5-step sampling, one adapter per reward), Flow-Map GRPO improves OCR from 0.575 to 0.912, PickScore from 21.69 to 23.52, and GenEval from 0.726 to 0.868. On auxiliary quality, optimizing a single task reward can lower some untargeted metrics, and this trade-off depends on the optimized reward: with the same method, joint OCR–PickScore training with a single adapter improves both targets and all five DrawBench metrics on both FLUX MeanFlow and sCM (MeanFlow: OCR 0.349 → 0.772, PickScore 20.98 → 22.50; Table 4). On variability, the ImageNet 5-step FID over 3 seeds is 16.65 ± 0.12 for Flow-Map GRPO and 22.55 ± 0.15 for MeanFlowNFT, a gap of 5.9 FID. The text-to-image results use one training seed per configuration because of their training cost.

E. Further clarifications and revisions.

  • Purpose of the ablations (pCUM; Sec. 4.1). The ablations test the most direct way to make a deterministic flow map stochastic, which is to add Gaussian noise to the sampler at each step. The variants that drop the drift correction or use a constant noise scale do not preserve the marginals, and after RL the gap is large: FID 16.72 versus 32.71, 70.13 and 49.25 on MeanFlow, and 15.65 versus 42.16 on the consistency model. As Reviewer pCUM points out, this degradation is expected. The ablations quantify it and confirm that the drift correction and the noise schedule are both needed, consistent with Theorem 3.2. The most controlled comparison is the midpoint anchor with and without the marginal-preserving scale (see B above; Table 5), where only the noise scale differs.
  • Stochastic rollouts vs. deterministic sampling (ma2p, txEe; App. C.6). Directly measuring the intermediate marginals of a learned high-dimensional model is impractical, so we compare stochastic rollouts with deterministic sampling of the same network. On ImageNet (K=4), before RL both anchors reach an FID within 0.8 of deterministic sampling of the same model (26.19 local, 25.69 endpoint, 25.43 deterministic). After RL, local-anchor rollouts reach 15.20 against 16.72, with matching precision and higher recall.
  • Exact and approximate properties (txEe; Theorems 3.2 and 3.3). The likelihood ratio is exact, because each transition is Gaussian by construction. For an exact flow map, the endpoint anchor preserves the marginals exactly, and the local anchor matches the exact marginal-preserving transition up to an O(h3/2)O(h^{3/2}) error in L2L^2. For learned maps, see the rollout comparison above.
  • Normalized likelihood ratio (ma2p; App. A.3). Following Flow-GRPO, training uses the ratio ρ1/d\rho^{1/d}, which keeps the direction of the policy gradient at the rollout policy and lets the same ε be used for any latent size. Compared with the joint ratio ρ\rho at the same ε, it converges faster and reaches a better FID (16.72 vs. 17.93).
  • Related work (zBam; Sec. 5, App. B). We now discuss RTDMD and DMDR. Both belong to distribution matching distillation, a different category from trajectory distillation such as flow maps. Their task also differs: they add rewards while distilling a generator from a multi-step teacher, whereas we post-train an already distilled flow map with no teacher.
  • Presentation (pCUM, txEe). Sec. 3.2 now presents the anchor family (endpoint, midpoint, local), Sec. 4.1 states the purpose of the ablations before their results, and App. A.5 gives the Langevin form of the local anchor.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.