acceptodds
Under review as a conference paper at ICLR 2027

ReplanFlow: Adaptive Flow-Map Scheduling for Few-Step Video Generation

Abstract

Few-step video generation depends on both the accuracy of learned transitions and their composition under a limited inference budget. Finite-time flow maps allow flexible sampling endpoints, but preset endpoints during on-policy distillation and inference limit video quality when only a few sampling steps are available. We present ReplanFlow, which combines broader on-policy transition coverage with adaptive flow-map scheduling. During distillation, we sample nonuniform internal endpoints to prepare the generator for more varied transition compositions. Meanwhile, we train a lightweight endpoint predictor offline using quality scores of fully generated videos. At inference, it selects the next sampling endpoint based on the current video latent, text condition, and remaining step budget, using exactly the specified number of sampling steps. This requires neither candidate-video search nor online quality evaluation. On VBench, ReplanFlow outperforms the AnyFlow baseline at the same sampling budget under matched generator-training data. With only three sampling steps, it also surpasses the four-step baseline while achieving a measured 1.21× end-to-end speedup. Component ablations further show that broader training coverage increases the gains from adaptive scheduling.

Primary area:
generative models
Keywords:
video generation, flow matching, flow map, diffusion distillation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HL9VOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ReplanFlow studies timestep selection for few-step text-to-video generation. It builds on AnyFlow (Wan2.1-T2V-1.3B), whose flow maps allow transitions between arbitrary endpoints but whose Stage-2 on-policy distillation and default inference use preset schedules. The paper changes both the training distribution of transitions and the way inference endpoints are selected.

During distillation, the uniform internal endpoints ai=1−i/Na_i=1-i/N are replaced with sorted i.i.d. U(0,1)\mathcal{U}(0,1) samples (Eq. 3). The teacher, training data, Stage-1 initialization, step-count distribution and scheduler shift are shared across the two branches; the shortcut simulation and DMD update are retained. A separate, lightweight predictor then learns to choose the next endpoint from the current latent, text, endpoint history and remaining budget. Its supervision is constructed offline: approximately 1K GPT-generated prompts and five seeds are used to enumerate schedules on an eight-point grid, giving 6, 15 and 20 schedules for two, three and four steps. Each candidate endpoint receives the highest VBench-derived score among its evaluated continuations (Eq. 5), which is converted into a soft target over 64 relative-progress bins. Predictors are trained separately for each generator checkpoint and total budget. At inference, the last call is forced to zero, so the method uses exactly the requested NFE without candidate search or online scoring.

Both generator branches are trained on approximately 4.3K Wan-Syn pairs for 1,200 LoRA updates, as AnyFlow's original 256K pairs are unavailable. On VBench full, ReplanFlow improves Total by 0.50 / 0.45 / 0.39 over the reproduced AnyFlow at two / three / four NFE (Table 1). Three-step ReplanFlow scores 83.34, compared with 83.11 for four-step AnyFlow, at a measured 1.21× end-to-end speedup. Table 2 separates the coverage and scheduling changes, while Table 3 uses a shared-first-transition control to examine the contribution of intermediate-state feedback. The predictor adds about 0.20% to sampling time (Table 4).

Critical strengths
  1. The component comparisons are well designed. The training branches share the data, teacher, initialization, step-count distribution and shift (p4 L203–205; Sec. 4.1). This makes Table 2's 2×2 comparison a useful test of whether broader coverage and sequential scheduling reinforce each other. I particularly like the Shared-first control in Table 3: it uses Sequential's first endpoint and plans the remaining transitions without observing intermediate latents. The resulting comparison is more informative about feedback than comparing Sequential with a planner that also chooses a different first step. The paper is also clear that Eq. 5 represents the best enumerated continuation, rather than the value of the deployed policy.

  2. The inference cost is small and carefully measured. The method uses B−1B-1 predictor calls for a budget of BB generator calls and requires no search or reward evaluation online. The measured overhead is 11.39 / 14.96 ms per video at three / four NFE. Replaying the same endpoint sequences with the predictor disabled isolates this cost; all 60 paired runs retain identical final latents (p8 L428–431). If the quality gains are reliable, the three-step configuration would offer a useful reduction in generation time without an expensive search procedure. The appendices provide label construction, architecture, optimization settings and per-dimension scores, which makes the approach reasonably easy to follow and reproduce.

Critical weaknesses
  1. The quality gains need stronger validation.

    The improvements are small: 0.50 / 0.45 / 0.39 Total points at matched NFE and 0.23 for three-step ReplanFlow versus four-step AnyFlow (Table 1). Table 2 reports a 0.23 coverage–scheduling interaction, and the feedback gains in Table 3 are 0.0957 / 0.1605 at four / six NFE. Main results use five generation seeds, but the paper gives neither standard deviations nor confidence intervals. The feedback study uses only 192 prompts and generation seed 0 (p6 L297–301). With differences of this size, paired uncertainty estimates are needed to judge whether the gains are reproducible. The setup also does not show how sensitive they are to retraining the generators or predictors.

    The four-NFE result in Table 3 illustrates why this matters. Sequential exceeds Initial-only by 0.2327, but the difference falls to 0.0957 when the first transition is shared. The unadjusted difference therefore cannot all be credited to intermediate-state feedback. I would like to see the shared-first comparison repeated over additional seeds. The two planners use different training inputs at later depths (App. D), so the remaining 0.1370 cannot be read as an estimate of training-seed variability.

    There is also no evaluation independent of the scheduler's reward. Eq. 5 uses VBench-derived scores, with normalized quality and applicable semantic dimensions weighted 0.8 and 0.2 (App. C, p18 L927–930). The reported results are again VBench scores. Although the per-video reward and benchmark-level Total are aggregated differently, they rely on the same scoring models. A higher held-out VBench score therefore leaves open whether the videos also improve under another evaluator or human preference. The qualitative examples are useful, but do not resolve this question. A small blinded comparison of three-step ReplanFlow and four-step AnyFlow would be particularly informative, since that is the paper's main efficiency claim.

  2. A tuned fixed schedule is needed to establish the benefit of adaptivity.

    Fixed inference follows AnyFlow's default schedule (p6 L292–293). The natural next comparison is with the best single schedule selected on the offline training prompts, separately for each generator and budget. App. C already contains scores for all enumerated grid schedules, so choosing this baseline requires no additional generation; it would then need to be evaluated on held-out prompts. This would separate the gain from replacing the default schedule from the further gain achieved by adapting decisions to each sample. Table 3 compares learned planners, but does not include this fixed baseline.

    Fig. 6(a) makes this comparison especially relevant: first endpoints are concentrated at high noise levels. A fixed schedule with a better first step may already recover part of the improvement. The existing results do not show how much Sequential adds over such a schedule. The shared-first experiment is a useful control for intermediate feedback, but it answers a different question from whether sample-dependent scheduling outperforms a well-chosen fixed schedule.

    I also have a question about how well the offline targets predict performance at deployment. Eq. 5 labels each endpoint with the maximum score among its continuations. At the four-step root, the highest of six internal grid points leaves (52)=10\binom{5}{2}=10 continuations, whereas the fourth-highest leaves one. This is an optimal-continuation target: it credits an endpoint for the best subsequent choices in the enumerated set. The deployed predictor must make those choices itself. More downstream choices can genuinely improve the optimum; the important missing measurement is the gap between this optimum and what the learned policy achieves. Reporting achieved continuation scores, or a small sensitivity check with equal-size continuation subsets, would show whether the labels translate into the expected inference gains.

    Finally, the controlled comparison is confined to a reduced-data reproduction: 4.3K pairs versus the 256K reported for AnyFlow (Sec. 4.1). The matched setup is appropriate for comparing the two branches, but it is unclear whether the advantage persists in the original training regime. Placing the published AnyFlow scores alongside the reproduction would help readers assess this limitation.

I lean toward rejection. The method is practical and the controls are thoughtful, but the current results leave two important questions open: whether the small gains are reliable beyond VBench's own scorers, and whether they exceed those of a tuned fixed schedule. Paired results, an independent quality comparison and the best-fixed baseline would be the most useful additions.

Questions for authors
  1. Could you report paired differences and confidence intervals over the five VBench-full generation seeds, particularly for three-step ReplanFlow versus four-step AnyFlow and the Table 2 interaction? Are results from additional generation seeds available for the Shared-first comparison in Table 3? A further training run, if feasible, would also help assess sensitivity to retraining.

  2. Which fixed schedule has the highest mean reward on the offline training prompts for each generator and budget? How does it compare with the default and Sequential on held-out VBench prompts at matched NFE?

  3. How close are the continuation scores achieved by the learned predictor to the targets in Eq. 5? Does the first-endpoint preference persist when labels use an equal-size subset of continuations, or their mean instead of their maximum?

  4. Do you have an independent automatic evaluation or a small blinded human preference comparison for three-step ReplanFlow versus four-step AnyFlow? Also, how were the GPT-generated training prompts checked for overlap with VBench?

  5. How do the reproduced AnyFlow scores compare with the published results? For the six-NFE planners in Table 3, how was the selected 3,000-update configuration chosen, and which validation prompts were used (p19 L1009–1010)?

As a reporting suggestion, App. C already gives about 60 A800 GPU-hours for the four-step labels. Adding the two-/three-step costs and the Stage-2 training cost would make the offline expense easier to compare with the inference savings.

5 comments
AuthorsOpenReview ↗

Response to Reviewer HL9V (Part 1/3)

Part 1/3: A1–A2

We sincerely appreciate your feedback.

We address five points: A1. reliability of the gains A2. tuned fixed schedules A3. target–execution gap and first-endpoint preference A4. independent evaluation and training prompts A5. training scale, configuration selection, and costs

A1. Reliability of the gains (Q1/W1–W3)

Q1: “Could you report paired differences and confidence intervals over the five VBench-full generation seeds, particularly for three-step ReplanFlow versus four-step AnyFlow and the Table 2 interaction? Are results from additional generation seeds available for the Shared-first comparison in Table 3? A further training run, if feasible, would also help assess sensitivity to retraining.”

We reused the existing VBench Full results to compute paired score differences and 95% confidence intervals. The results support consistent gains at matched budgets (Table R1).

All five score differences are positive. At two, three, and four steps, the confidence intervals are entirely above zero. For the three-versus-four-step comparison and the Table 2 interaction, most of each interval lies above zero.

The first four rows compare ReplanFlow Sequential with AnyFlow Fixed. The last row measures how much broader training coverage increases the gain from switching from Fixed to Sequential scheduling.

Table R1. Paired Total score differences and 95% confidence intervals on VBench Full.

Comparison (NFE)Δ TotalPaired 95% CI
2 vs. 2+0.5002[+0.2246, +0.7789]
3 vs. 3+0.4480[+0.1596, +0.7578]
4 vs. 4+0.3851[+0.0675, +0.7177]
3 vs. 4+0.2364[−0.0815, +0.5712]
Interaction (4)+0.2278[−0.0442, +0.5043]

Additional generation seed. Sequential still scores higher than Shared-first with additional generation seed 2. We used the original four-step predictors. The Total gain is 0.0539 points.

Sensitivity to retraining. We retrained both Sequential and Initial-only predictors with training seed 20261111. We used the same evaluation settings as in Table 3 of the paper. Sequential still scores higher than Shared-first, with a Total gain of 0.0376 points. Both checks retain a positive feedback gain (Table R2).

Table R2. Shared-first and Sequential with an additional generation seed and retrained predictors.

ExperimentTraining seedPolicyQualitySemanticTotal
Additional generation seed20260903Shared-first85.116676.082883.3098
Additional generation seed20260903Sequential85.101476.413083.3637
Retrained predictors20261111Shared-first85.571878.408184.1391
Retrained predictors20261111Sequential85.647178.295384.1767

A2. Comparison with tuned fixed schedules (Q2/W5–W7)

Q2: “Which fixed schedule has the highest mean reward on the offline training prompts for each generator and budget? How does it compare with the default and Sequential on held-out VBench prompts at matched NFE?”

Sequential scores higher than Tuned Fixed at both three and four steps on the ReplanFlow generator. The Total gains are 0.3943 and 0.2591 points (Table R3). The gains therefore come from more than just replacing the default schedule.

For each generator and budget, we chose the fixed schedule with the highest mean reward on the offline training data. We then froze the schedule and tested it on VBench Full. Tuned Fixed also improves on the default schedule, but Sequential still scores higher.

The selected schedules are [1, 6/7, 4/7, 2/7, 0] for four-step AnyFlow, [1, 6/7, 3/7, 0] for three-step ReplanFlow, and [1, 6/7, 4/7, 1/7, 0] for four-step ReplanFlow. These are normalized endpoints before the scheduler shift. Each schedule is fixed across all evaluation samples.

Table R3. Tuned fixed schedules and sequential scheduling on VBench Full.

GeneratorNFEScheduleQualitySemanticTotal
AnyFlow4Default Fixed85.026675.419183.1051
AnyFlow4Tuned Fixed85.235875.389783.2665
ReplanFlow3Tuned Fixed84.799775.537182.9472
ReplanFlow3Sequential85.141976.139983.3415
ReplanFlow4Default Fixed85.077875.839583.2302
ReplanFlow4Tuned Fixed85.117475.686283.2311
ReplanFlow4Sequential85.281376.325983.4902
AuthorsOpenReview ↗

Response to Reviewer HL9V (Part 2/3)

Part 2/3: A3–A4

A3. Target–execution gap and first-endpoint preference (W8/Q3)

Q3: “How close are the continuation scores achieved by the learned predictor to the targets in Eq. 5? Does the first-endpoint preference persist when labels use an equal-size subset of continuations, or their mean instead of their maximum?”

Actual continuation scores are close to the training targets on average. Eq. 5 uses the best score among the evaluated candidate continuations as a training target. The learned predictor may choose a different continuation, so its score need not match this target exactly.

We tested the three-step predictor on 32 randomly sampled training prompts, with one seed per prompt. We tested five first endpoints for each sample, giving 160 state–action pairs. After each first transition, the predictor completed the video. The mean achieved score was 0.77714, compared with a mean target of 0.78585. The mean gap was 0.00872 on the normalized training reward.

The preference for high-noise first endpoints remains under both alternative labels. We recomputed the labels using either an equal-size random subset of continuations or their mean score instead of their maximum. For equal-size subsets, we sampled one continuation per endpoint and repeated this 1,000 times.

Table R4. First-endpoint preference under different label rules.

Label ruleFirst-endpoint preference for 6/7 or 5/7
Maximum over all candidate continuations84.4%
Maximum over equal-size random subsets74.7%
Mean over all candidate continuations81.3%

The preference remains in all three cases (Table R4). This suggests that unequal continuation counts do not fully explain it. The percentages change, so the label rule still affects endpoint choices.

A4. Independent evaluation and training prompts (W4/Q4)

Q4: “Do you have an independent automatic evaluation or a small blinded human preference comparison for three-step ReplanFlow versus four-step AnyFlow? Also, how were the GPT-generated training prompts checked for overlap with VBench?”

Blinded human preference study. Three-step ReplanFlow received more preference votes than four-step AnyFlow (Table R5). We randomly sampled 30 video pairs. Seven participants rated all pairs in a blind survey, giving 210 votes. We hid method names and randomized left/right positions. Participants chose a preferred video or a tie.

Table R5. Blinded human preference: three-step ReplanFlow vs. four-step AnyFlow.

ReplanFlow winsTiesAnyFlow wins
42.4% (89)24.8% (52)32.9% (69)

VLM evaluation. The results were close to even, with a small lead for ReplanFlow. We evaluated the same 30 pairs with gpt-6-astra, gpt-6.1-sol, and gpt-5.6-sol through Codex. Method names were hidden. ReplanFlow won 51.5% of judgments after excluding ties (Table R6).

Table R6. VLM preference: three-step ReplanFlow vs. four-step AnyFlow.

JudgeWin / Tie / LossWin rate excluding ties
gpt-6-astra8 / 13 / 947.1%
gpt-6.1-sol11 / 9 / 1052.4%
gpt-5.6-sol16 / 0 / 1453.3%
Combined35 / 22 / 3351.5%

Training prompts. We used GPT to generate training prompts based on the scene distribution of VBench.

AuthorsOpenReview ↗

Response to Reviewer HL9V (Part 3/3)

Part 3/3: A5

A5. Training scale, configuration selection, and costs (W9/Q5)

Q5: “How do the reproduced AnyFlow scores compare with the published results? For the six-NFE planners in Table 3, how was the selected 3,000-update configuration chosen, and which validation prompts were used (p19 L1009–1010)?”

Reporting suggestion: “Adding the two-/three-step costs and the Stage-2 training cost would make the offline expense easier to compare with the inference savings.”

Training scale. ReplanFlow achieves a Total score of 83.49 with about 4.3K training pairs. Published AnyFlow reports 83.48 with 256K pairs. The Total scores are nearly identical, despite the large difference in data size. The training settings differ, so our controlled comparison uses the reproduced AnyFlow baseline.

Six-step configuration. We used a small set of GPT-generated prompts and VBench scores to select checkpoints.

Offline costs. We measured data generation, scoring, and predictor training costs on A800 GPUs (Tables R7 and R8). The four-step data cost is 86.23 GPU-hours per seed, rather than the roughly 60 GPU-hours stated in the paper. We will correct this value.

Table R7. Offline data generation and scoring costs on A800 GPUs.

BudgetVideo generationScoringTotal GPU-hours per seed
2 steps1.57 h1.42 h23.96
3 steps4.18 h3.47 h61.13
4 steps6.05 h4.73 h86.23

Table R8. Predictor training and data-loading time on A800 GPUs.

PredictorTraining and exportInput loading and checksTotal time
2 steps171.31 s195.82 s6.12 min
3 steps166.31 s78.07 s4.07 min
4 steps191.00 s678.68 s14.49 min

Stage 2 training. Stage 2 training took about 33 hours on eight A800 GPUs for each model, or about 265 GPU-hours. This covers 1,200 training updates and excludes checkpoint saving and evaluation.

Reviewer_obfZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ReplanFlow combines broader on-policy transition coverage with adaptive endpoint selection for few-step video generation. Starting from AnyFlow, the method replaces preset internal endpoints during second-stage distillation with sorted random endpoints. A lightweight predictor is then trained offline from terminal-quality scores of enumerated continuations. At inference, it selects the next endpoint using the current latent, text condition, endpoint history, and remaining budget; the last call is forced to the clean endpoint. Under matched generator-training settings on Wan2.1-T2V-1.3B, the paper reports improvements at two, three, and four generator calls. Three-step ReplanFlow scores best on VBench. It seems like a good way for few step video generation. but the improvement appears to be quite minor.

Critical strengths
  1. The paper clearly distinguishes the number of generator evaluations (NFE) from predictor overhead and ensures that each trajectory uses exactly BB generator evaluations by forcing the final step to reach 00. This provides a cleaner basis for fair comparison than many adaptive-step methods.

  2. The method connects the training distribution to the decisions made by the inference scheduler. Broadening transition coverage is a well-motivated complement to choosing nonstandard endpoints at inference. The four-way component comparison in Table 2 supports this interaction: sequential scheduling yields little improvement with the baseline generator, but a larger improvement after coverage is broadened. A budget-conditioned predictor that selects flow-map endpoints sequentially under an exact generator-call budget is effective. Overall, the methodology and approach in the paper are quite clear and easy to understand.

Critical weaknesses

1、The generated videos appear to exhibit very limited motion, typically with a static camera and only slight movement of the subjects or scene elements. The visual quality also appears relatively low. The method appears to combine several implementation heuristics and achieves some improvement over the AnyFlow baseline. However, the gains are not readily apparent in the video examples. The speed improvement is relatively small, and there is also additional overhead (endpoints predictor, and I do not think it is effective every time).

2、At its core, this is a distillation method for few-step generation. Beyond comparisons with AnyFlow, a central question is whether the method can match the video quality of the original Wan model using fewer sampling steps. In the “07-acoustic” example, the generated video appears to have noticeable quality issues, including hand distortions. The absence of a corresponding result from the original Wan baseline makes this comparison difficult. Based on the video examples alone, I do not see a clear improvement in visual quality. The slightly higher VBench scores may stem from the training reward combining VBench quality dimensions with applicable semantic dimensions, rather than from perceptible improvements in the generated videos.

Questions for authors

1、Could you evaluate the method on a baseline that produces higher-quality videos, such as Wan2.1-14B, and extend the evaluation to image-to-video (I2V) generation? Evaluating only text-to-video (T2V) generation at 480p makes it difficult to assess the method’s broader value to the community.

2、Was the VBench evaluation conducted using only 192 videos? This seems insufficient to support strong conclusions. Table 1 suggests that the distilled model outperforms the original Wan2.1-T2V-1.3B model with classifier-free guidance (CFG). I am skeptical about this result; the distilled version performs even better than the original model, which runs somewhat counter to most empirical experience. Could you expand the evaluation to include more videos and additional models to establish the robustness and generality of this paper?

4 comments
AuthorsOpenReview ↗

Response to Reviewer obfZ

We sincerely appreciate your feedback.

We address four points: A1. evaluation size and comparison with Wan A2. visual quality beyond VBench A3. inference speed and scheduling overhead A4. additional models and tasks

A1. Evaluation size and comparison with Wan (Q2/W2)

Q2: “Was the VBench evaluation conducted using only 192 videos?”

Q2: “Table 1 suggests that the distilled model outperforms the original Wan2.1-T2V-1.3B model with classifier-free guidance (CFG). I am skeptical about this result.”

Our main results and component ablations use VBench Full. We evaluate 944 prompts with five seeds, giving 4,720 regular videos per configuration. The 192-prompt subset is used only for the feedback analysis in Table 3.

The Wan2.1-T2V-1.3B score of 83.24 in Table 1 comes from the AnyFlow paper. We explicitly state this source in Section 4.1. Table 1 also separates literature results from our controlled comparisons.

We have now re-evaluated the original Wan model directly. Wan achieves 83.5377, slightly above four-step ReplanFlow at 83.4902 (Table R1). Thus, our distilled model does not exceed Wan's Total score in this comparison.

Table R1. Re-evaluation of original Wan and comparison with ReplanFlow.

MethodNFEQualitySemanticTotal
Original Wan, re-evaluated50×285.629175.172183.5377
ReplanFlow Sequential485.281376.325983.4902

Four-step ReplanFlow achieves a Total score close to Wan’s, while Wan’s logged runtime is 13.44× longer.

A2. Visual quality beyond VBench (W1/W2)

W1: “The generated videos appear to exhibit very limited motion, typically with a static camera and only slight movement of the subjects or scene elements.”

W2: “The slightly higher VBench scores may stem from the training reward combining VBench quality dimensions with applicable semantic dimensions, rather than from perceptible improvements in the generated videos.”

We added a human preference study and a VLM evaluation to assess visual quality beyond VBench.

Human evaluation. ReplanFlow received more preference votes while using fewer steps (Table R2). We randomly sampled 30 video pairs. Each pair compares three-step ReplanFlow with four-step AnyFlow. Seven participants rated all pairs in a blind survey, giving 210 votes. We hid method names and randomized left/right positions. Participants chose a preferred video or a tie.

Table R2. Blinded human preference: three-step ReplanFlow vs. four-step AnyFlow.

ReplanFlow winsTiesAnyFlow wins
42.4% (89)24.8% (52)32.9% (69)

VLM evaluation. ReplanFlow won 51.5% of non-tied judgments across three models, a result close to even (Table R3). We evaluated the same 30 pairs with gpt-6-astra, gpt-6.1-sol, and gpt-5.6-sol through Codex. Method names were hidden.

Table R3. VLM preference: three-step ReplanFlow vs. four-step AnyFlow.

JudgeWin / Tie / LossWin rate excluding ties
gpt-6-astra8 / 13 / 947.1%
gpt-6.1-sol11 / 9 / 1052.4%
gpt-5.6-sol16 / 0 / 1453.3%
Combined35 / 22 / 3351.5%

Paired statistical analysis. The matched-budget gains have positive 95% confidence intervals (Table R4). We reused the existing VBench Full scores. We resampled prompts, kept all seeds for each prompt, and recomputed the VBench scores. The three-versus-four-step gain is also positive. Each row compares ReplanFlow Sequential with AnyFlow Fixed. Differences are in Total score points.

Table R4. Paired Total score differences on VBench Full.

NFE (ReplanFlow vs. AnyFlow)Δ TotalPaired 95% CI
2 vs. 2+0.5002[+0.2246, +0.7789]
3 vs. 3+0.4480[+0.1596, +0.7578]
4 vs. 4+0.3851[+0.0675, +0.7177]
3 vs. 4+0.2364[−0.0815, +0.5712]

A3. Inference speed and predictor overhead (W1)

W1: “The speed improvement is relatively small, and there is also additional overhead (endpoints predictor, and I do not think it is effective every time).”

ReplanFlow achieves a 1.21× end-to-end speedup with three steps over the four-step AnyFlow baseline, as reported in Table 4 of the paper. Online scheduling adds only about 0.20% to sampling time. Our goal is to improve average quality under a fixed budget, though individual samples may not always improve.

A4. Additional models and tasks (Q1/Q2)

Q1: “Could you evaluate the method on a baseline that produces higher-quality videos, such as Wan2.1-14B, and extend the evaluation to image-to-video (I2V) generation?”

Q2: “Could you expand the evaluation to include more videos and additional models to establish the robustness and generality of this paper?”

Our current experiments just focus on T2V generation task. We agree that testing larger models and I2V would strengthen the evidence. Due to limited compute, we have not tested these settings. We therefore limit our claims to the evaluated model and task.

We would like to share two additional results on adaptive scheduling and visual quality.

1. Comparison with tuned fixed schedules. For each generator and budget, we selected the fixed schedule with the highest mean reward on the offline training data. We then evaluated it on VBench Full. On the same ReplanFlow generator, Sequential improves Total by 0.3943 points at three steps and 0.2591 points at four steps (Table R5). The gains therefore come from more than replacing the default schedule.

Table R5. Tuned Fixed vs. Sequential on VBench Full.

NFEScheduleTotal
3Tuned Fixed82.9472
3Sequential83.3415
4Tuned Fixed83.2311
4Sequential83.4902

2. Human preference on new prompts. We tested 50 prompts randomly selected from EvalCrafter (CVPR 2024) after filtering close text matches to VBench prompts. We used the existing checkpoints without further tuning. Both methods used four steps. Five additional participants rated all 50 pairs in a blind study, giving 250 votes. ReplanFlow received more preference votes than AnyFlow (Table R6). Its win rate excluding ties was 58.67%.

Table R6. Blinded human preference on EvalCrafter prompts.

ReplanFlow winsTiesAnyFlow wins
35.2% (88)40.0% (100)24.8% (62)

VLM judges also showed a slight preference for ReplanFlow. We used gpt-6-astra, gpt-6.1-sol, gpt-5.6-sol, gpt-6-sol, and gpt-6.1-terra. Each model rated the same 50 pairs under a blinded protocol, giving 250 judgments in total. ReplanFlow won 52.32% of non-tied judgments.

1、"The generated videos appear to exhibit very limited motion, typically with a static camera and only slight movement of the subjects or scene elements." This concern is not solved.

2、As for video generation, Wan2.1-T2V-1.3B is relatively small. I think its performance wasn't quite good. And I still do not see a video with large motion. Maybe you can attach in supp.

3、If computational resources are limited, you could consider using LoRA to fine-tune the 5B version of Wan-I2V. Since your focus is on adaptive flow-map scheduling, I would expect the computational cost to be manageable. You do not need to pretrain a video. If your current setup uses Wan-1.3B with rank-256 LoRA for 1,200 updates at a global batch size of 16, maybe you can just try reducing the batch size to 2 or 4 for the 5B model. I believe LoRA fine-tuning should be sufficient for this purpose. However, these experiments appear to be missing.

4、In my opinion, it is not easy to compare performance with t2v models. Demonstrating improvements only on the text-to-video (T2V) version of Wan-1.3B is not sufficiently convincing. As for t2v models, there are so many tricks toboosting T2V models’ VBench scores.

Thank you for the suggestion.

1–2. Subject and camera motion

We have added a Motion-case folder to the updated supplementary material. It contains six examples. Each includes four-step AnyFlow and four-step ReplanFlow videos with the full prompt.

The examples show several types of motion: a dog running toward the camera, a camera following a woman with an umbrella, a moving boat and a flying bird, falling coffee beans, two people approaching and kissing, and a forward-moving highway view. We place the dog and umbrella examples first to make the subject and camera motion easy to find.

These examples allow a direct comparison of motion and video quality at the same four-step budget.

3. Larger models and I2V

Our main constraint is the available GPU resources and total compute time, rather than GPU memory alone. A smaller batch may make the model fit, but adapting to a new backbone still requires a compatible flow map and new offline predictor training data. The reported 1,200 updates cover Stage 2 only and start from a trained Stage 1 flow map. Given the remaining rebuttal time and our available resources, we cannot complete this extension with a properly matched comparison. We agree that larger-model and I2V experiments would strengthen the paper, and will state this limitation clearly.

4. Evidence beyond VBench

We agree that VBench scores alone are not enough. We therefore added a blinded human study on EvalCrafter prompts, using existing checkpoints without further tuning. As reported in Table R6 of our previous response, participants preferred ReplanFlow over AnyFlow. These results support generalization to prompts from another benchmark, beyond gains measured by VBench.

Reviewer_SELUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is mainly built on AnyFlow for few-step video generation. It makes two changes:

  1. During training, the interval endpoints are sampled randomly instead of from a fixed uniform grid.
  2. During inference, a lightweight predictor picks the next endpoint at each step based on the current latent and the number of remaining steps. This module is trained offline and adds negligible inference cost.

Experiments on Wan2.1-1.3B show improved VBench scores over AnyFlow at the same number of steps, and 3-step ReplanFlow slightly outperforms 4-step AnyFlow.

Critical strengths
  • The method is simple and practical. It does not change the generator and adds negligible inference cost.
  • The ablation shows that the two changes work together: the endpoint predictor helps much more when combined with the randomized training (+0.26 vs. +0.03 on VBench).
Critical weaknesses
  • Limited technical novelty. My main concern is the limited technical novelty. This work is mainly built on AnyFlow, and randomizing the interval endpoints is a small modification of AnyFlow. The endpoint predictor is also similar to the idea of Schedule On the Fly (TPDM, Ye et al., CVPR 2025), which also predicts the next timestep from the current latent and is trained with a final-quality reward. The paper does not clearly explain what is fundamentally new beyond applying these ideas to flow maps for video.
  • Marginal gains and questionable evaluation. The performance gain is marginal: it improves VBench by only +0.2 to +0.5, and no variance across runs is reported. Moreover, the predictor is trained with a VBench-derived reward and then evaluated on VBench. This setting is also questionable, since the gains may partly come from fitting the evaluation metric.
Questions for authors
  • What is the key technical difference from TPDM, and how does ReplanFlow compare with it experimentally?
  • How does the method perform on metrics other than VBench, or in a user study?
  • How does it compare with the best fixed schedule found from the same offline data?
5 comments
AuthorsOpenReview ↗

Response to Reviewer SELU

We sincerely appreciate your feedback.

W/Q labels follow review order.

We address three points: A1. differences from TPDM A2. comparison with tuned fixed schedules A3. a user study, VLM evaluation, and paired confidence intervals

A1. Differences from TPDM (W1/Q1)

Q1: “What is the key technical difference from TPDM, and how does ReplanFlow compare with it experimentally?”

TPDM uses state feedback and a discounted reward to balance quality and sampling cost. We instead allocate a fixed number of generator calls. Our predictor uses the current state and remaining budget to choose each flow-map endpoint. Generation ends after exactly the given number of calls.

We tested this budget input by replacing it with a constant and retraining the predictor. Both versions still use four calls. The Total score drops from 83.4902 to 83.3682 (Table R1). This suggests that the remaining budget helps the predictor choose endpoints.

Table R1. Effect of explicit remaining-budget input.

ConfigurationExplicit remaining budgetQualitySemanticTotalChange
Original, with the true remaining budget✓85.281376.325983.4902—
Retrained with budget input fixed at 4✗85.167476.171283.3682−0.1220

We also broaden on-policy training coverage to support adaptive transitions. This increases the four-step scheduling gain from 0.0322 to 0.2600 points.

A2. Comparison with tuned fixed schedules (Q3)

Q3: “How does it compare with the best fixed schedule found from the same offline data?”

We selected the best fixed schedule for each generator and budget using offline training scores. We then evaluated that schedule on VBench Full. Compared with Tuned Fixed on the same generator, Sequential improves Total by 0.3943 points at three steps and 0.2591 points at four steps (Table R2). Three-step ReplanFlow also exceeds four-step AnyFlow Tuned Fixed by 0.0750 points. The gains therefore do not come solely from a poor default fixed schedule. Adaptive scheduling provides additional gains.

Table R2. Tuned fixed schedules and sequential scheduling on VBench Full.

GeneratorNFEScheduleQualitySemanticTotal
AnyFlow4Default Fixed85.026675.419183.1051
AnyFlow4Tuned Fixed85.235875.389783.2665
ReplanFlow3Tuned Fixed84.799775.537182.9472
ReplanFlow3Sequential85.141976.139983.3415
ReplanFlow4Default Fixed85.077875.839583.2302
ReplanFlow4Tuned Fixed85.117475.686283.2311
ReplanFlow4Sequential85.281376.325983.4902

A3. User study, VLM evaluation, and paired confidence intervals (W2/Q2)

Q2: “How does the method perform on metrics other than VBench, or in a user study?”

W2: “The performance gain is marginal: it improves VBench by only +0.2 to +0.5, and no variance across runs is reported.”

Taken together, the results support reliable gains. We added a user study, VLM evaluation, and paired statistical analysis. Our GPT-generated training prompts were constructed with reference to the scene distribution of VBench.

1. User study. Seven participants blindly rated 30 random video pairs of three-step ReplanFlow versus four-step AnyFlow, giving 210 votes. Participants preferred ReplanFlow more often than AnyFlow (Table R3).

Table R3. Blinded human preference: three-step ReplanFlow vs. four-step AnyFlow.

ReplanFlow winsTiesAnyFlow wins
42.4% (89)24.8% (52)32.9% (69)

2. VLM evaluation. We evaluated the same 30 pairs through Codex using gpt-6-astra, gpt-6.1-sol, and gpt-5.6-sol. Method names were hidden. Excluding ties, ReplanFlow's aggregate win rate was 51.5% (Table R4).

Table R4. VLM preference: three-step ReplanFlow vs. four-step AnyFlow.

JudgeWin / Tie / LossWin rate excluding ties
gpt-6-astra8 / 13 / 947.1%
gpt-6.1-sol11 / 9 / 1052.4%
gpt-5.6-sol16 / 0 / 1453.3%
Combined35 / 22 / 3351.5%

3. Paired statistical analysis. We paired scores by prompt and seed and bootstrapped over prompts. Each draw retained all seeds and recomputed the original VBench aggregation. All point estimates are positive. The 95% confidence intervals for matched-budget gains at two, three, and four steps are entirely above zero (Table R5).

The first four rows compare ReplanFlow Sequential with AnyFlow Fixed. Interaction measures the extra scheduling gain from broader training coverage. Differences are in Total score points.

Table R5. Paired differences on VBench Full.

Comparison (NFE)Δ TotalPaired 95% CI
2 vs. 2+0.5002[+0.2246, +0.7789]
3 vs. 3+0.4480[+0.1596, +0.7578]
4 vs. 4+0.3851[+0.0675, +0.7177]
3 vs. 4+0.2364[−0.0815, +0.5712]
Interaction (4)+0.2278[−0.0442, +0.5043]

Thanks the authors for the response and the new experiments. Regarding the questions:

  • The new results show the scheduling is better than the tuned fixed scheduling. The concern is addressed
  • Regarding the gain, the concern is partially addressed: The VLM evaluation seems pretty random and dose not show a clear advantage. The user study shows clear improvement but the evaluation set is relatively small. In addition, as the training prompt follow the VBench scene distribution, and uses VBench as rewards, I am still concern about the potential overfitting issue when using VBench is the main evaluation metric.
  • I still have question about the novelty: a fixed budget of TPDM is a difference in setting while the core idea of predicting the next timestep from the current latent to optimize the reward, is the same as TPDM.

Thank you for the follow-up comments.

A1. Differences from TPDM

The budget is not just a fixed setting outside the model. We encode the remaining budget with a learned embedding and combine it with time features. This forms a time/budget token that interacts with the latent, text, and history tokens through attention. Replacing the budget input with a constant and retraining reduces Total by 0.1220 points, while both versions still use exactly four calls.

The training also differs. TPDM uses policy gradients with a reward for the whole trajectory. We train endpoint selection with offline labels from candidate continuations under the same remaining budget. This form of supervision more closely matches the next-endpoint decision made at inference.

A2. Overfitting and evaluation size

To address your concerns about overfitting and the small evaluation set, we tested 50 prompts from EvalCrafter (CVPR 2024). We started with its official 700 prompts and removed duplicate prompts. We also removed prompts with word-set Jaccard similarity ≥ 0.8 to VBench prompts. We randomly selected 50 without replacement using seed 20261117. Selection took place before generation. We kept the original wording and did not select prompts based on generated results. Both methods used four steps and the existing checkpoints, without further tuning. This tests whether the gains extend beyond VBench at the same budget.

Blinded human preference. Five additional participants rated all 50 pairs under the same blinded protocol, giving 250 votes. Human preference for ReplanFlow also appeared on these prompts (Table R6). Its win rate excluding ties was 58.67%.

Table R6. Blinded human preference: four-step ReplanFlow vs. four-step AnyFlow on EvalCrafter prompts.

ReplanFlow winsTiesAnyFlow wins
35.2% (88)40.0% (100)24.8% (62)

VLM evaluation. We expanded the evaluation to five models: gpt-6-astra, gpt-6.1-sol, gpt-5.6-sol, gpt-6-sol, and gpt-6.1-terra. Each model rated the same 50 pairs under the same blinded protocol. Across 250 judgments, ReplanFlow's win rate excluding ties was 52.32% (Table R7). VLM preferences remained close, with a slight preference for ReplanFlow.

Table R7. VLM preferences: four-step ReplanFlow vs. four-step AnyFlow on EvalCrafter prompts.

JudgeWin / Tie / LossWin rate excluding ties
gpt-6-astra13 / 25 / 1252.00%
gpt-6.1-sol12 / 26 / 1250.00%
gpt-5.6-sol19 / 14 / 1752.78%
gpt-6-sol18 / 16 / 1652.94%
gpt-6.1-terra17 / 18 / 1553.13%
Combined79 / 99 / 7252.32%

Comments on the paper

1 comment

Dear Reviewers,

It is important that we thoroughly discuss our perspectives before reaching a final recommendation. Now that the authors' rebuttal is available, I kindly ask each of you to respond in the discussion and clarify whether your original concerns have been addressed or still remain.

Thank you in advance for your timely engagement before the discussion period ends.

Best, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.