acceptodds
Under review as a conference paper at ICLR 2027

FlowCTS: On-policy Continuous Trajectory Supervision of Flow Models

Abstract

While on-policy distillation (OPD) effectively addresses sparse rewards and exposure bias in large language model post-training, its extension to flow models remains underexplored. To this end, we propose Flow Continuous Trajectory Supervision (FlowCTS), which matches subsequent student and reference trajectories initialized from the same student-visited state. Using the integral relation between trajectories and velocity fields, we derive a temporally weighted velocity-matching upper bound and discretize it into practical objectives parameterized by the number of supervision steps. Under a multi-reference setup, FlowCTS-OPD outperforms vanilla KL-based OPD with faster convergence. Compared with the SD3.5-M base model, it improves GenEval from to , OCR from to , and PickScore from to , matching task experts and surpassing the GenEval expert under multi-step supervision, while outperforming mixed-reward RL across all metrics. Further analysis reveals a clear temporal supervision mismatch in vanilla KL-based OPD arising from its auxiliary SDE transition kernels. Beyond on-policy setting, FlowCTS also consistently outperforms vanilla SFT especially on OCR, while increasing supervision steps exhibit a trade-off between richer trajectory information and greater optimization difficulty.

Primary area:
generative models
Keywords:
Flow Matching, Text-to-Image Generation, On-Policy Distillation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pWPEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FlowCTS formulates distillation for flow models as matching student and reference trajectories from a shared starting state. A velocity-based upper bound yields weighted local losses over K subsequent states. K=1 reduces to velocity MSE at a student-visited state. The paper evaluates multi-reference distillation on SD3.5-Medium and an off-policy extension using straight image-noise paths. It reports better GenEval, OCR, and PickScore than a controlled transition-KL baseline. It also analyzes temporal weighting and finds that earlier supervision helps GenEval. The OPD student starts from averaged expert LoRA weights, as specified in Appendix C.1.

Critical strengths
  1. The paper connects a simple trajectory objective to an implementable family of velocity losses. Appendices A and B derive the temporal weights explicitly. Appendix F also shows that the compared transition KL is a time-weighted velocity MSE. The controlled early/middle/late supervision experiment in Appendix G supports the practical relevance of the temporal allocation analysis for GenEval.

  2. The controlled K=1 comparison is promising. Table 1 improves GenEval from 0.90 to 0.93, OCR from 0.90 to 0.92, and PickScore from 22.75 to 23.06 under a shared OPD pipeline. Table 3 also improves all four reported image-quality and preference metrics. The authors disclose that MAR is disabled for both objectives in Appendix C.1. This is a useful objective comparison, although it is not a comparison against the complete Flow-OPD system. The off-policy OCR improvement provides additional evidence beyond the main setting.

Critical weaknesses
  1. The multi-step method is not specified precisely enough to reproduce or interpret. Section 3.3 defines h=(t_s-t_e)/K. For fixed endpoints, increasing K refines the discretization over the same interval. However, Sections 3.3 and 4.3 repeatedly interpret K as extending the supervision span. The paper does not specify how t_e is selected or how the interval changes with K. Appendix C.1 freezes reference predictions, but does not state whether the starting state and intermediate student states are detached. Backpropagating through x_i^theta in Equation 10 optimizes a different gradient from treating each state as fixed. These choices directly affect both the trajectory interpretation and the optimization-difficulty explanation for larger K.

  2. The main results establish the value of local velocity matching more clearly than the added value of the trajectory construction. Most gains already occur at K=1, which removes the KL temporal weight. A related local velocity-MSE objective on routed student states also appears in DanceOPD, which the paper cites as concurrent work: https://arxiv.org/abs/2606.27377 . The multi-step extension remains a plausible contribution, but Table 1 shows only a 0.01 GenEval gain at K=2, unchanged OCR, and lower PickScore. Table 4 matches supervised evaluations, but does not isolate the derived weights from uniform multi-step weighting. Uncertainty for these small differences is not reported. Moreover, the headline gains over the pretrained model include an averaged expert-LoRA initialization whose standalone performance is missing. The controlled KL comparison remains informative, but it does not establish how much of the headline capability gain comes from FlowCTS itself.

Questions for authors
  1. Clarification: How are t_s, t_e, and h chosen for each K? Which starting and intermediate states are detached, and which losses backpropagate through the student rollout? Please give the actual update algorithm for both OPD and SFT.

  2. Assessment-relevant control: What are the scores of the averaged expert-LoRA initialization before OPD? How much improvement does each objective achieve from that common starting point?

  3. Assessment-relevant control: Does trajectory-derived weighting outperform uniform weighting over identical multi-step states? Please report uncertainty and total compute for this comparison and the matched-evaluation K=1/K=2 control. Appendix D already averages generation seeds, but neither their count nor uncertainty is given.

  4. Clarification: What exact normalization and unrounded values produce Table 1's Overall scores? Why does Figure 5's base-model reference appear near 0.64 rather than the 0.63 reported in Table 1?

3 comments

Dear Reviewer pWPE

Thank you for your thoughtful review . We appreciate your concrete suggestions for clarifying the contributions of multi-step supervision, the weighting scheme, and the student initialization.

W1 & Q1 : Clarifying time selection and gradient paths

We clarify the time selection and gradient paths below and provide the update procedures for both OPD and SFT.

Time selection. Both OPD and SFT use the fixed step size hh prescribed by the ten-step sampling schedule, independently of the supervision step count KK. Starting from tst_s, we advance KK steps with this spacing, giving te=ts−Kht_e=t_s-Kh. Thus, h=(ts−te)/Kh=(t_s-t_e)/K describes the selected interval, rather than refining a fixed interval as KK increases. Before reaching the time boundary, increasing KK extends the supervision span. The difference lies in how the starting state is obtained: OPD uses an online student rollout to reach tst_s, whereas SFT constructs the state by adding noise to a data sample at a randomly selected tst_s. We recognize that the step size affects supervision resolution and coverage; jointly exploring step size and supervision length is a promising direction for future work.

Gradient paths. Under the unified stop-gradient scheme, both OPD and SFT detach the starting state and all intermediate student states, while retaining gradients through the student velocity predictions at each supervised state. Each loss therefore updates the corresponding student prediction without backpropagating through preceding rollout transitions. OPD reference targets are computed without gradients using the frozen teacher; SFT uses the fixed data-derived target ϵ−z\epsilon-z.

In our experiments, both OPD and SFT use the fixed step size h prescribed by the ten-step sampling schedule, independently of K.

OPD

  1. Obtain the starting state x by rolling out the student to time t_s without gradients.
  2. Initialize the reference state x_ref ← x and loss L ← 0.
  3. For i = 0, …, K−1:
    • Set t ← t_s − i·h.
    • Compute v ← Student(stop_gradient(x), t, condition), retaining gradients.
    • Compute u ← Teacher(x_ref, t, condition) without gradients.
    • Accumulate L ← L + αᵢ · ‖v − u‖₂², where the weights αᵢ sum to one.
    • If i < K−1, advance both states without gradients:
      • x ← StudentStep(x, v, t, h).
      • x_ref ← ReferenceStep(x_ref, u, t, h).
  4. Update the student parameters using the gradient of L.

SFT

  1. Sample a data–condition pair (z, c) and Gaussian noise ε.
  2. Sample a starting time t_s such that the K-step interval remains within the valid time range.
  3. Construct x ← stop_gradient(AddNoise(z, ε, t_s)).
  4. Set the fixed target u ← stop_gradient(ε − z) and initialize L ← 0.
  5. For i = 0, …, K−1:
    • Set t ← t_s − i·h.
    • Compute v ← Student(stop_gradient(x), t, c), retaining gradients.
    • Accumulate L ← L + αᵢ · ‖v − u‖₂², where the weights αᵢ sum to one.
    • If i < K−1, update x ← stop_gradient(x − h·v).
  6. Update the student parameters using the gradient of L.

K denotes the number of supervised velocity predictions over K integration intervals. Computing these losses requires only K−1 state updates, since the final endpoint is not itself supervised. We will add these details and revise the corresponding descriptions in the updated manuscript. Thank you for helping us make the method clearer and more reproducible.

W2 & Q2: Initialization scores and the benefits of multi-step supervision

Thank you for highlighting the need to distinguish the gains from initialization, distillation, and multi-step supervision. We have evaluated the averaged adapter before distillation and report its scores below:

InitializationGenEval ↑OCR ↑PickScore ↑
Averaged adapter0.75590.776922.226

Both objectives substantially improve upon this common initialization, bringing the student to approximately expert-level performance. Beyond these gains from distillation, the additional value of multi-step supervision lies in both training efficiency and final performance.

In our distillation setting, the teacher’s capabilities impose a practical bottleneck on performance gains, leaving limited room for further improvement once the student approaches expert performance, and K=1 already approaches expert performance on GenEval. Nevertheless, K=2 reaches expert performance at 660 updates, compared with 1,440 updates for K=1 in the reported comparison, and surpasses the expert by 1,020 updates. Reaching expert performance requires approximately 50 H200 GPU-hours for K=2, compared with 70 H200 GPU-hours for K=1. K=2 also achieves a higher final overall score, although the gains differ across metrics.

Supervision steps660 updates1,020 updates1,440 updates
K=10.92310.92500.9304
K=20.93130.94210.9419

We acknowledge that more trajectory supervision is not always better, and its benefits may depend on optimization difficulty, model capacity, and training data. Our primary focus is to identify a natural form of OPD supervision for flow models. The auxiliary Gaussian distributions used for KL matching encode teacher information through their velocity-dependent means, without providing the rich distributional supervision naturally available from next-token vocabulary distributions in text OPD. This motivates us to start from trajectory discrepancy and derive a tractable velocity objective. While DanceOPD derives the Gaussian KL–weighted MSE connection and supports unweighted velocity matching through ablations, we additionally analyze how KL-induced temporal weighting affects supervision allocation. We see understanding how these factors influence longer trajectory supervision as a compelling direction for future research, and hope our findings inspire further exploration of scaling supervision steps.

W2 & Q3 : Weighting ablation and evaluation uncertainty

The results in Appendix D are averaged over three generation seeds. For clarity, we report the mean and standard deviation for the two requested controlled comparisons below, and will provide the corresponding statistics for the remaining results in the revised appendix. Given the available compute during the rebuttal period, we conducted this experiment with a single GenEval expert and 320 training updates, evaluating each configuration using three generation seeds.

The trajectory-derived weights achieve higher GenEval scores at both K=2 and K=3 in this setting, providing empirical support for the weighting scheme. More broadly, our main focus is how to formulate supervision for flow-model OPD; the weights follow from the trajectory-based formulation. This comparison helps clarify their contribution within that framework.

Supervision stepsTrajectory-derived weightsUniform weights
K=20.920 ± 0.00260.914 ± 0.0032
K=30.926 ± 0.00350.917 ± 0.0038

Q4: Clarifying the Overall score and Figure 5 reference line

Following the fixed rescaling convention in the Flow-GRPO, we divide the raw PickScore by 26 and compute:

Overall = (GenEval + OCR + PickScore / 26) / 3.

In Figure 5, the base-model reference line is positioned at 0.63, consistent with Table 1. Its dashed rendering and line width make it visually appear closer to 0.64. We will adjust the plotting style to make its position clear in the revised manuscript.

Best,

The Authors

Dear Reviewer pWPE

As we are approaching the last day of the rebuttal period, we would like to thank you again for your time and constructive feedback on our paper.

We have provided detailed responses to your concerns, including clarifications of time selection and gradient paths in OPD and SFT, evaluations of the averaged LoRA initialization and multi-step training efficiency, and weighting ablations with means and standard deviations across three generation seeds. We have also clarified the Overall score calculation and the reference-line placement in Figure 5.

We understand that this is a busy period. When convenient, we would greatly appreciate it if you could review our responses and let us know whether they have addressed your concerns. If any questions remain, we would be happy to provide further clarification or discussion before the rebuttal period closes.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_Wo78OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2(−1)
Read review
Paper summary

This paper introduces FlowCTS, which is a method of On Policy Distillation for Flow Matching. The authors propose a new formulation of the flow matching distillation problem, where rather that trying to minimise the KL divergence between the student and the reference, the work frames the problem as aiming to minimise the distance between the student and reference trajectory.

The authors derive a velocity-based upper bound on this distance-based loss, and provide a discrete-time-based objective function. Unlike the standard FlowOPD loss, FlowCTS allows for the consideration of multi-step loss, where the divergence of the student with respect to the reference is considered over multiple future steps.

The main contributions of this work are the reformulation of the policy distillation equation, the consideration of multi-step flow loss, and empirical results showing that this reformulation outperforms existing baselines.

Critical strengths

The core strength of this paper are the empirical results presented. FlowCTS clearly outperforms Vanilla OPD, shown clearly in Table 1 and Figure 3. In GenEval and OCR, FlowCTS basically matches the expert performance, and in PickScore, it significantly improves over the base model, and outperforms Vanilla OPD.

The trajectory distance based loss formulation also represents an interesting comparison to KL-divergence, and considering multi-step trajectory discrepancy seems a sensible way of aiming to train the student to match the reference most accurately in states where this will have the most effect.

I also found this work to be very well presented - the authors clearly introduce the relevant topics necessary to understand their contribution, and present their work clearly, both with a thorough description of the reformulation and method, but also clear and informative figures.

Critical weaknesses

I would summarise the main weakness of this paper as a divergence between the claimed justification of why FlowCTS is effective, and the empirical results presented in the paper. The core idea of this work is that, for OPD with flow models, we shouldn't use KL-divergence, but should instead consider trajectory divergence, using an upper bound loss function that is derived from the distance between two trajectories. However, for K=1, this practically manifests as a very similar loss function to Vanilla OPD, just without the time-dependent weight, and I think this does highlight two main weaknesses with the work.

Minimal Multi-Step Performance Improvements

The first is that the multi-step performance improvements seem very marginal. For K=2, there is a very slight improvement, and for K=3, the performance drops below K=1. As multi-step supervision is one of the claimed contributions of this work, I don't think these empirical results support the benefit of the method. Whilst it is a compelling idea, the results indicate it isn't actually particularly effective. There is a clear performance improvement over Vanilla OPD, but if most of the benefit of the method is induced by the case where K=1, then practically, the actual implementation is remains very similar to Vanilla OPD.

Unconvincing Justification of the Trajectory Formulation

This leads to the second main weakness of the paper, which is that I don't think the empirical results convincingly support the trajectory-based formulation. The practical implementation differences between this work and baselines are very small compared to existing work, for the scenario when K=1 - simply the removal of the time-dependent weight. The authors highlight that the advantages of FlowCTS can be attributed to greater emphasis on training in the early stages of denoising, whereas Vanilla OPD using KL-divergence instead places greater weight on the latter stages of denoising. This seems like the main takeaway of this work, and Figure 6 supports this, showing that actively upsampling early timesteps improves performance over sampling middle or late timesteps. However, the main narrative of the paper centres around the trajectory-based reformulation of the problem (rather than using KL-divergence), but I'm not convinced this reformulation is actually the source of the performance improvement - the reformulation just happens to not overweight the later stages of denoising like how KL-divergence does.

The performance improvements of removing the time-based weighting function are clear, but I do not think this small implementation change is necessarily a large enough contribution for a paper. However, given the minimal benefit of the multi-step trajectory supervision, and the results shown in Figure 6, I remain unconvinced that the trajectory-based reformulation actually facilitates the performance improvements shown in the results.

Questions for authors
  1. Am I mischaracterising the minimal performance benefits of multi-step trajectory supervision? Can you provide stronger empirical result to show the benefits of this? It would greatly strengthen the justification behind the trajectory-based reformulation.

  2. Have you examined different time-based weighting functions? Specifically upweighting the early stages of denoising rather than the later stages (as KL-divergence does)

  3. Table 3 is used to show that FlowCTS outperforms Vanilla OPD in visual quality and human preference. However, in the FlowOPD[1] paper, Manifold Anchor Regularization (MAR) is used to improve this as well. Would you be able to test how MAR affects FlowCTS?

[1] Fang, Zhen, et al. "Flow-opd: On-policy distillation for flow matching models." arXiv preprint arXiv:2605.08063 (2026).

4 comments

Dear Reviewer Wo78:

Thank you for your thoughtful review and for recognizing the empirical improvements, the potential of trajectory-based supervision, and the clarity of our presentation. We appreciate your constructive feedback, which has helped us clarify our contributions and strengthen the supporting evidence.

W1 & Q1 : Clarifying the Performance Benefits of Multi-Step Trajectory Supervision

Thank you for encouraging us to clarify the practical benefits of multi-step supervision. In our distillation setting, the teacher’s capabilities impose a practical bottleneck on performance gains, leaving limited room for further improvement once the student approaches expert performance, and K=1 already approaches expert performance on GenEval. In practice, K=2 reaches expert performance with fewer training updates and surpasses it with less GPU time, while also achieving a higher final overall score than K=1

Specifically, K=2 reaches the GenEval expert’s performance at 660 updates, whereas K=1 reaches it at 1,440 updates in the reported comparison. With further training to 1,020 updates, K=2 improves beyond the expert. To reach expert performance, K=2 requires approximately 50 H200 GPU-hours, compared with 70 H200 GPU-hours for K=1. These results highlight an efficiency benefit alongside the final performance improvement, whose magnitude may be limited by the capabilities of the expert teachers. We will include the training dynamics and compute details in the revised manuscript.

Supervision steps660 updates1,020 updates1,440 updates
K=10.92310.92500.9304
K=20.93130.94210.9419

W2 : Motivation for Trajectory Supervision

Our goal is to develop a natural OPD formulation for flow models with a rich teacher supervision signal. In text OPD, the teacher directly provides a predictive distribution over vocabulary tokens, conveying meaningful relative preferences among possible continuations. Vanilla OPD for flow models instead constructs auxiliary Gaussian distributions with a shared, prescribed covariance. Their teacher information is entirely carried by the velocity-dependent means: the Gaussian representation adds no information beyond the velocity predictions, and its KL divergence reduces to weighted velocity matching. This construction also introduces a timestep-dependent weight that biases supervision toward later denoising stages.

We therefore consider the teacher’s generation dynamics directly. A single velocity describes instantaneous motion, whereas a trajectory segment captures how generation evolves across successive states. Ideally, if the student fully reproduces the teacher’s continuation from a shared starting state, it has learned the teacher’s generation behavior over that segment. This makes trajectory matching a natural target for knowledge distillation, providing richer supervision than a single local velocity. Starting from this trajectory discrepancy, we derive a family of tractable velocity-based objectives.

Empirically, K=1 already achieves strong performance while avoiding the SDE-induced temporal weighting issue. In a distillation setting, the teacher’s capabilities constrain the available supervision, leaving limited room for improvement when the student already approaches expert performance. Nevertheless, K=2 reaches expert performance more efficiently and surpasses the GenEval expert with further training, as reported above. Longer supervision is not always better and may introduce optimization challenges. Our focus is on establishing this supervision formulation, and we hope it inspires further exploration of scaling supervision steps.

Q2: Evaluating early-stage weighting

We have additionally evaluated an early-weighted K=1 variant, adapting the noise-aware weighting schedule of TempFlow-GRPO[1] to the velocity-matching loss to place greater emphasis on the early stages of denoising.

As shown below, this weighting improves some intermediate checkpoints but does not provide a consistent advantage. The final reported GenEval score is slightly lower than that of the original unweighted objective.

One possible explanation is that direct velocity matching already removes the late-stage emphasis induced by the KL weight. Further upweighting the early stages may shift supervision away from other useful parts of the generation process. These results favor retaining unweighted velocity supervision in our current setting, without introducing additional temporal weighting.

Method660 updates1,020 updates1,440 updates
Unweighted0.92310.92500.9304
Early-weighted0.92490.92040.9274

[1]He, Xiaoxuan, et al. "Tempflow-grpo: When timing matters for grpo in flow models." International Conference on Learning Representations. Vol. 2026. 2026.

Q3: Evaluating FlowCTS with and without MAR

Thank you for suggesting this comparison. Given the available compute during the rebuttal period, we could only conduct an early-stage comparison of FlowCTS with and without MAR. The preliminary results show improved scores with MAR, suggesting that its additional anchor supervision can complement FlowCTS. The extent of this benefit may depend on the expert models and target tasks, while our main focus is how to formulate OPD supervision.

MethodImageReward ↑Aesthetic ↑HPS-v2.1 ↑UnifiedReward ↑
FlowCTS w/o MAR1.225.510.30213.2842
FlowCTS w/ MAR1.245.560.30473.3004

Best,

The Authors

Thank you for your comprehensive response to my review, it has definitely been helpful to see the additional results you have been able to produce.

In terms of the weaknesses I initially raised, I think they have been at least generally satisfied.

W1 and Q1

The benefit of multi-step trajectories are seen with reducing the number of supervision steps, and GPU hours required for training. I think this does provide a clear use case for where this is beneficial

W2 and Q2

I think there are a few additional results that have supported the justification for the trajectory-based formulation. The unweighted vs early weighted experiment does indicate that unweighted is optimal, which is justified by the trajectory formulation. I also think the results presented to the other two reviewers about trajectory-derived vs uniform weighting for the time-based weighting also supports the trajectory reformulation.

However, the responses provided by the author have somewhat made me question the overall soundness of the paper.

There are two main reasons for this

  1. The authors explanation of the benefits of multi-step trajectory have been in relation to more efficient training, e.g. reducing the number of training step, and the length of training time. They have been able to produce some results here during the rebuttal period, and it is understandable that a full suite of tests could not be performed, given the limited time. However, if this is the headline result that justifies multi-step trajectories, it is very unclear why this was not presented in the original paper. If Figure 5 was presented with comparisons of K=1,2,3 etc., it would be very clear to see the performance improvements of the multi-step trajectory. Again, I understand that this may be infeasible to produce in the rebuttal period, but if it is a core justification for multi-step trajectory training, it should have been in the paper to begin with.
  2. The second is the portrayal of what model was used to initialise the distilled models. In my initial reading of the paper, I had assumed that the initial model before distillation would be the pretrained model. Both other reviewers identified that in Appendix C, it is specified that it was an averaged Flow-GRPO model. This on its own is not an issue, I don't have a huge amount of hands-on experience with this area of work, so that may be standard practice, and as long as the baselines used the same initialisation, the results presented are perfectly sound. I appreciate that you now have provided results for this averaged model, but the paper would have benefited in clarity if the results of the averaged model were shown in Table 1. More importantly however, if the initial model was the averaged model, rather than just the pretrained model, Figure 3 is a blatant misrepresentation of the results. Figure 3 shows that the initial results of the model were the pretrained model, which appears to not be the case at all. I cannot understand how this representation of the results can be justified, and it seriously makes me question whether there are other similarly unsound parts of the methodology of this work that have not been identified in the review process. What is especially confusing is, based on the results presented to the other reviewers on the performance of the averaged model, using the correct datapoint for the 0 step in Figure 3 would not change the insight gained from this plot. I am genuinely baffled by this representation of the results, and it does raise concerns about the general soundness of the work.

If I have have misunderstood this plot and the overall methodology of the results, I deeply apologise, and I would very much appreciate the authors correcting me. However, based on my current understanding, I cannot justify increasing my score.

Thank you for your follow-up. We are glad that the additional evidence has helped address your initial concerns. Below, we respond to the two additional concerns you raised.

Q1.

We really appreciate your reviews during rebuttal period, which has helped improve the quality of our manuscript. In fact, your comments prompted us to revisit the motivation for multi-step supervision and re-examine our experimental results, helping us better understand its benefits and how to demonstrate them more clearly.

The original manuscript included an ablation study that provided indirect evidence of the training-efficiency benefits of multi-step supervision. This study compared performance at a limited number of training checkpoints. Specifically, Table 4 in Appendix E compared (K=2) trained for (N) updates with (K=1) trained for (2N) updates, matching the total number of supervised state evaluations. (K=2) achieved higher GenEval performance with half as many training updates. We discussed this comparison in Lines 348–350 as a control for the supervision budget, and we should have highlighted the associated efficiency advantage more explicitly.

We have added a direct comparison of training dynamics to the current version of the manuscript. We agree that this provides clearer evidence of convergence behavior. Figure 4 now presents the GenEval training curves for (K=1,2,3), with the corresponding discussion updated in Section 4.3. These curves show that (K=2) improves more rapidly than (K=1) and reaches and surpasses the expert reference within the evaluated training range. Increasing the supervision step to (K=3) does not provide a clear additional advantage. This reflect a trade-off between richer trajectory supervision and increased optimization difficulty.

Together, these results demonstrate two complementary benefits of (K=2) on GenEval: higher final performance and more efficient progress during training. Table 1 shows that extending supervision from (K=1) to (K=2) improves the final GenEval score from 0.93 to 0.94. The newly added training curves show that this improvement is accompanied by higher scores earlier in training, consistent with the original budget-controlled comparison. Your feedback has helped us distinguish and present these two benefits more clearly.

Q2.

Thank you for pointing out the issue in Figure 3. We acknowledge that the original figure incorrectly used the pretrained model’s performance as the zero-update point. In our experiments, the student’s LoRA adapter is initialized by averaging the LoRA adapter parameters of the three task-specific Flow-GRPO experts, following the model-merging cold-start strategy used in Vanilla OPD, We sincerely apologize for the misleading presentation.

We have corrected the starting points in Figure 3 and added the averaged initialization results to Table 1. Both Vanilla OPD and FlowCTS-OPD use the same pretrained backbone and averaged LoRA initialization. The correction separates the improvement provided by initialization from the subsequent improvement achieved through OPD; From the same initialization, FlowCTS-OPD still shows faster convergence and higher final scores across all three metrics in the corrected Figure 3(a)–(c). Since the two methods differ only in their supervision objectives, these controlled results continue to support the advantage of trajectory-derived velocity matching over Vanilla OPD.

You can review these updates in the latest submitted version of our manuscript. Thank you again for your careful review and constructive feedback.

Reviewer_QWhEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes FlowCTS for training text-to-image flow models. The student and teacher start from the same noisy image state reached by the student. FlowCTS then matches their velocities, meaning the directions they predict for denoising the image. The method can supervise one step or follow both models for several steps. The authors derive weights that give more importance to errors near the start of each short path. They test the method with three expert teachers and also extend it to offline fine-tuning.

Critical strengths
  • The main comparison uses the same teachers, starting model, and training settings. With one supervision step, FlowCTS improves GenEval from 0.90 to 0.93, OCR from 0.90 to 0.92, and PickScore from 22.75 to 23.06 over KL-based distillation.

  • The paper tests when teacher feedback helps most. In the GenEval experiment, training more often at early, high-noise states reaches 0.92, compared with 0.86 when training focuses on later states.

  • The approach also works with offline image–text pairs. Three-step supervision improves OCR from 0.70 to 0.75 while keeping GenEval and PickScore close to standard fine-tuning.

Critical weaknesses
  • Most gains already appear with one-step velocity matching, which the concurrent DanceOPD paper also uses. Adding a second step raises GenEval from 0.93 to 0.94, leaves OCR unchanged, and lowers PickScore from 23.06 to 22.96. The extra steps therefore offer a narrow benefit. DanceOPD

  • The experiments leave open whether the proposed weights help. Appendix E compares runs with the same number of supervised states, but the paper does not compare its weights with equal weights on the same trajectories.

  • The headline gains use plain SD3.5-Medium as the starting score. Appendix C says the student starts from an averaged Flow-GRPO adapter. Reporting that adapter’s scores would show how much improvement comes from distillation itself.

Questions for authors
  • At two or three supervision steps, how do the proposed weights compare with giving every step equal weight?

  • With the same GPU-hour budget, does the small GenEval gain from two steps hold across independent training runs?

  • What scores does the averaged adapter achieve before training, and which checkpoint do the starting points in Figure 3 represent?

1 comment

Dear Reviewer QWhE,

Thank you for your thoughtful review and for recognizing the controlled comparisons, the temporal analysis, and the extension to offline fine-tuning. We appreciate your constructive suggestions and would like to clarify the contributions of multi-step supervision, the weighting scheme, and the student initialization below.

W1 : Clarifying the Performance Benefits of Multi-Step Trajectory Supervision

Thank you for encouraging us to clarify the practical benefits of multi-step supervision. In our distillation setting, the teacher’s capabilities impose a practical bottleneck on performance gains, and K=1 already approaches expert performance on GenEval. In practice, K=2 reaches expert performance with fewer training updates and surpasses it with less GPU time, while also achieving a higher final overall score than K=1

Specifically, K=2 reaches the GenEval expert’s performance at 660 updates, whereas K=1 reaches it at 1,440 updates in the reported comparison. With further training to 1,020 updates, K=2 improves beyond the expert. To reach expert performance, K=2 requires approximately 50 H200 GPU-hours, compared with 70 H200 GPU-hours for K=1. These results highlight an efficiency benefit alongside the final performance improvement. We will include the training dynamics and compute details in the revised manuscript.

Supervision steps660 updates1,020 updates1,440 updates
K=10.92310.92500.9304
K=20.93130.94210.9419

We also acknowledge that more trajectory supervision is not always better: there may be a trade-off between its benefits and optimization difficulty, with model capacity and training data also playing a role. Our primary focus is to identify a more natural form of OPD for diffusion/flow models, starting from trajectory discrepancy and deriving a tractable velocity-supervision objective. While DanceOPD supports unweighted velocity matching through ablations, we additionally analyze how KL-induced temporal weighting affects supervision allocation, providing a theoretical rationale for this choice. Understanding how these factors influence the benefits of longer trajectory supervision is not the main focus of this paper, but we see it as a compelling direction for future research and hope our findings inspire further exploration of scaling supervision steps.

W2 & Q1:Weighting ablation: trajectory-derived vs. uniform weights at K=2 and K=3

We have added an ablation comparing the trajectory-derived weights with uniform weighting under the same trajectory construction and supervision settings. Given the available compute during the rebuttal period, we conducted this experiment with a single expert and 320 training updates.

Supervision stepsTrajectory-derived weightsUniform weights
K=20.9200.914
K=30.9260.917

The trajectory-derived weights achieve higher GenEval scores at both K=2 and K=3 in this setting, providing empirical support for the weighting scheme. More broadly, our main focus is how to formulate supervision for flow-model OPD; the weights follow from the trajectory-based formulation. This comparison helps clarify their contribution within that framework.

Q2 : Better performance at the same compute budget

We have added a comparison under the same budget of 70 GPU-hours across independent runs, where K=2 achieves a higher GenEval score than K=1, as shown below:

SettingGenEval
K=10.9295
K=20.9402

Furthermore, K=2 surpasses expert performance with approximately 50 H200 GPU-hours, whereas K=1 requires approximately 70 H200 GPU-hours to reach expert performance. These results highlight both the stronger performance at a matched compute budget and the lower compute required to reach expert-level performance.

W3 & Q3 : Performance of the averaged adapter before distillation

We have evaluated the averaged adapter before distillation and report its scores below:

InitializationGenEval ↑OCR ↑PickScore ↑
Averaged adapter0.75590.776922.226

Both objectives substantially improve upon this common initialization, bringing the student to approximately expert-level performance. Beyond these gains from distillation, the additional value of multi-step supervision lies in both training efficiency and final performance. We will add these initialization scores and correct the starting points in Figure 3 in the revised manuscript. Thank you for pointing out this issue.

Best,

The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

Dear Reviewers and Area Chair,

As we approach the end of the rebuttal period, we would like to provide a consolidated summary of our contributions, manuscript revisions, and the additional evidence presented in response to the reviewers’ concerns. We sincerely appreciate your constructive feedback. Below, we first address the shared concerns, followed by the additional experiments and clarifications.

Our work introduces FlowCTS , a trajectory-based formulation of on-policy distillation for flow models. We derive tractable single- and multi-step velocity-supervision objectives from continuous trajectory discrepancies and identify how KL-induced temporal weighting can misallocate supervision. Empirically, FlowCTS combines task-specific teacher capabilities, matching expert performance on GenEval and OCR and surpassing the GenEval expert with multi-step supervision. We further demonstrate the benefits of multi-step supervision in both final GenEval performance and training efficiency, and extend the formulation to off-policy fine-tuning.

GR1. Benefits of multi-step supervision

Our distillation results demonstrate strong performance across supervision variants. In distillation setting, FlowCTS variants can reach or even surpass teacher-level performance. In particular, K=1 already approaches the GenEval expert and matches the reported OCR expert score, while K=2 further surpasses the GenEval expert. Table 1 shows that extending supervision from K=1 to K=2 improves the final GenEval score from 0.93 to 0.94, while maintaining OCR performance.

The training dynamics further demonstrate the efficiency benefits of multi-step supervision. Figure 4 directly compares the GenEval training curves for K=1,2,3. Compared with K=1, K=2 achieves higher scores earlier in training and reaches and surpasses the expert reference within the evaluated range. The original supervision-budget-controlled comparison in Appendix E, Table 4, provides complementary evidence: K=2 achieves higher GenEval performance with half as many training updates, while matching the total number of supervised state evaluations.

Together, these results highlight two complementary benefits of K=2: stronger final GenEval performance and more efficient progress in terms of training updates. We have updated Section 4.3 to clarify these findings.

GR2. Initialization and comparison fairness

Both methods use the same pretrained backbone and averaged expert LoRA initialization. Specifically, the student’s LoRA adapter is initialized by averaging the LoRA adapter parameters of the three task-specific Flow-GRPO experts; the pretrained backbone remains frozen. The initialization scores are:

InitializationGenEval ↑OCR ↑PickScore ↑
Averaged expert LoRA adapter0.75590.776922.226

We acknowledge that the original Figure 3 incorrectly used the pretrained model’s performance as the zero-update point. We have corrected these starting points, added the initialization results to Table 1, and clarified the experimental setup.

The corrected presentation separates initialization gains from subsequent distillation gains. The subsequent training measurements and final results remain unchanged. From this shared initialization, FlowCTS-OPD continues to show faster convergence and higher final scores than Vanilla OPD across the three metrics in Figure 3.

GR3. Empirical support for trajectory-derived weighting

The trajectory-derived weights outperform uniform weighting in the controlled comparisons. We compared the two weighting schemes using the same trajectory construction and supervision settings, with a single GenEval expert and 320 training updates. The following results report means and standard deviations across three generation seeds:

Supervision stepsTrajectory-derived weightsUniform weights
K=20.920 ± 0.00260.914 ± 0.0032
K=30.926 ± 0.00350.917 ± 0.0038

These comparisons provide empirical support for the weighting scheme at both supervision steps and clarify its contribution within the trajectory-based formulation.

Additional experiments and clarifications

  • Early-stage weighting. We evaluated an early-weighted K=1 variant. It improved some intermediate evaluations but provided no consistent advantage and achieved a slightly lower final GenEval score than unweighted supervision, supporting our choice in the reported setting.
  • Compatibility with MAR. We provided a preliminary early-stage comparison with and without MAR. Adding MAR improved all four evaluated metrics, suggesting that its anchor supervision can complement FlowCTS.
  • Training procedures. We clarified the fixed sampling step size, how K determines the supervision span, and the stop-gradient treatment. We also provided the OPD and SFT update procedures, specifying that gradients pass through supervised student velocity predictions without backpropagating through preceding rollout transitions.
  • Motivation and relation to prior work. We clarified how trajectory discrepancy motivates a family of velocity-supervision objectives and how our temporal analysis explains the supervision allocation induced by the auxiliary Gaussian KL objective.
  • Evaluation and presentation details. We clarified that we follow Flow-GRPO’s scoring convention to normalize PickScore by dividing its raw value by 26. We then average the three metrics to obtain Overall = (GenEval + OCR + PickScore / 26) / 3. We also clarified the placement of the pretrained-model reference line.

Please refer to the latest revised manuscript and our detailed responses for these updates and additional results. Thank you again for helping us clarify the contributions, strengthen the experimental evidence, and improve the manuscript.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.