- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ReplanFlow studies timestep selection for few-step text-to-video generation. It builds on AnyFlow (Wan2.1-T2V-1.3B), whose flow maps allow transitions between arbitrary endpoints but whose Stage-2 on-policy distillation and default inference use preset schedules. The paper changes both the training distribution of transitions and the way inference endpoints are selected.
During distillation, the uniform internal endpoints are replaced with sorted i.i.d. samples (Eq. 3). The teacher, training data, Stage-1 initialization, step-count distribution and scheduler shift are shared across the two branches; the shortcut simulation and DMD update are retained. A separate, lightweight predictor then learns to choose the next endpoint from the current latent, text, endpoint history and remaining budget. Its supervision is constructed offline: approximately 1K GPT-generated prompts and five seeds are used to enumerate schedules on an eight-point grid, giving 6, 15 and 20 schedules for two, three and four steps. Each candidate endpoint receives the highest VBench-derived score among its evaluated continuations (Eq. 5), which is converted into a soft target over 64 relative-progress bins. Predictors are trained separately for each generator checkpoint and total budget. At inference, the last call is forced to zero, so the method uses exactly the requested NFE without candidate search or online scoring.
Both generator branches are trained on approximately 4.3K Wan-Syn pairs for 1,200 LoRA updates, as AnyFlow's original 256K pairs are unavailable. On VBench full, ReplanFlow improves Total by 0.50 / 0.45 / 0.39 over the reproduced AnyFlow at two / three / four NFE (Table 1). Three-step ReplanFlow scores 83.34, compared with 83.11 for four-step AnyFlow, at a measured 1.21× end-to-end speedup. Table 2 separates the coverage and scheduling changes, while Table 3 uses a shared-first-transition control to examine the contribution of intermediate-state feedback. The predictor adds about 0.20% to sampling time (Table 4).
Critical strengths
-
The component comparisons are well designed. The training branches share the data, teacher, initialization, step-count distribution and shift (p4 L203–205; Sec. 4.1). This makes Table 2's 2×2 comparison a useful test of whether broader coverage and sequential scheduling reinforce each other. I particularly like the Shared-first control in Table 3: it uses Sequential's first endpoint and plans the remaining transitions without observing intermediate latents. The resulting comparison is more informative about feedback than comparing Sequential with a planner that also chooses a different first step. The paper is also clear that Eq. 5 represents the best enumerated continuation, rather than the value of the deployed policy.
-
The inference cost is small and carefully measured. The method uses predictor calls for a budget of generator calls and requires no search or reward evaluation online. The measured overhead is 11.39 / 14.96 ms per video at three / four NFE. Replaying the same endpoint sequences with the predictor disabled isolates this cost; all 60 paired runs retain identical final latents (p8 L428–431). If the quality gains are reliable, the three-step configuration would offer a useful reduction in generation time without an expensive search procedure. The appendices provide label construction, architecture, optimization settings and per-dimension scores, which makes the approach reasonably easy to follow and reproduce.
Critical weaknesses
-
The quality gains need stronger validation.
The improvements are small: 0.50 / 0.45 / 0.39 Total points at matched NFE and 0.23 for three-step ReplanFlow versus four-step AnyFlow (Table 1). Table 2 reports a 0.23 coverage–scheduling interaction, and the feedback gains in Table 3 are 0.0957 / 0.1605 at four / six NFE. Main results use five generation seeds, but the paper gives neither standard deviations nor confidence intervals. The feedback study uses only 192 prompts and generation seed 0 (p6 L297–301). With differences of this size, paired uncertainty estimates are needed to judge whether the gains are reproducible. The setup also does not show how sensitive they are to retraining the generators or predictors.
The four-NFE result in Table 3 illustrates why this matters. Sequential exceeds Initial-only by 0.2327, but the difference falls to 0.0957 when the first transition is shared. The unadjusted difference therefore cannot all be credited to intermediate-state feedback. I would like to see the shared-first comparison repeated over additional seeds. The two planners use different training inputs at later depths (App. D), so the remaining 0.1370 cannot be read as an estimate of training-seed variability.
There is also no evaluation independent of the scheduler's reward. Eq. 5 uses VBench-derived scores, with normalized quality and applicable semantic dimensions weighted 0.8 and 0.2 (App. C, p18 L927–930). The reported results are again VBench scores. Although the per-video reward and benchmark-level Total are aggregated differently, they rely on the same scoring models. A higher held-out VBench score therefore leaves open whether the videos also improve under another evaluator or human preference. The qualitative examples are useful, but do not resolve this question. A small blinded comparison of three-step ReplanFlow and four-step AnyFlow would be particularly informative, since that is the paper's main efficiency claim.
-
A tuned fixed schedule is needed to establish the benefit of adaptivity.
Fixed inference follows AnyFlow's default schedule (p6 L292–293). The natural next comparison is with the best single schedule selected on the offline training prompts, separately for each generator and budget. App. C already contains scores for all enumerated grid schedules, so choosing this baseline requires no additional generation; it would then need to be evaluated on held-out prompts. This would separate the gain from replacing the default schedule from the further gain achieved by adapting decisions to each sample. Table 3 compares learned planners, but does not include this fixed baseline.
Fig. 6(a) makes this comparison especially relevant: first endpoints are concentrated at high noise levels. A fixed schedule with a better first step may already recover part of the improvement. The existing results do not show how much Sequential adds over such a schedule. The shared-first experiment is a useful control for intermediate feedback, but it answers a different question from whether sample-dependent scheduling outperforms a well-chosen fixed schedule.
I also have a question about how well the offline targets predict performance at deployment. Eq. 5 labels each endpoint with the maximum score among its continuations. At the four-step root, the highest of six internal grid points leaves continuations, whereas the fourth-highest leaves one. This is an optimal-continuation target: it credits an endpoint for the best subsequent choices in the enumerated set. The deployed predictor must make those choices itself. More downstream choices can genuinely improve the optimum; the important missing measurement is the gap between this optimum and what the learned policy achieves. Reporting achieved continuation scores, or a small sensitivity check with equal-size continuation subsets, would show whether the labels translate into the expected inference gains.
Finally, the controlled comparison is confined to a reduced-data reproduction: 4.3K pairs versus the 256K reported for AnyFlow (Sec. 4.1). The matched setup is appropriate for comparing the two branches, but it is unclear whether the advantage persists in the original training regime. Placing the published AnyFlow scores alongside the reproduction would help readers assess this limitation.
I lean toward rejection. The method is practical and the controls are thoughtful, but the current results leave two important questions open: whether the small gains are reliable beyond VBench's own scorers, and whether they exceed those of a tuned fixed schedule. Paired results, an independent quality comparison and the best-fixed baseline would be the most useful additions.
Questions for authors
-
Could you report paired differences and confidence intervals over the five VBench-full generation seeds, particularly for three-step ReplanFlow versus four-step AnyFlow and the Table 2 interaction? Are results from additional generation seeds available for the Shared-first comparison in Table 3? A further training run, if feasible, would also help assess sensitivity to retraining.
-
Which fixed schedule has the highest mean reward on the offline training prompts for each generator and budget? How does it compare with the default and Sequential on held-out VBench prompts at matched NFE?
-
How close are the continuation scores achieved by the learned predictor to the targets in Eq. 5? Does the first-endpoint preference persist when labels use an equal-size subset of continuations, or their mean instead of their maximum?
-
Do you have an independent automatic evaluation or a small blinded human preference comparison for three-step ReplanFlow versus four-step AnyFlow? Also, how were the GPT-generated training prompts checked for overlap with VBench?
-
How do the reproduced AnyFlow scores compare with the published results? For the six-NFE planners in Table 3, how was the selected 3,000-update configuration chosen, and which validation prompts were used (p19 L1009–1010)?
As a reporting suggestion, App. C already gives about 60 A800 GPU-hours for the four-step labels. Adding the two-/three-step costs and the Stage-2 training cost would make the offline expense easier to compare with the inference savings.
5 comments
Response to Reviewer HL9V (Part 2/3)
Part 2/3: A3–A4
A3. Target–execution gap and first-endpoint preference (W8/Q3)
Q3: “How close are the continuation scores achieved by the learned predictor to the targets in Eq. 5? Does the first-endpoint preference persist when labels use an equal-size subset of continuations, or their mean instead of their maximum?”
Actual continuation scores are close to the training targets on average. Eq. 5 uses the best score among the evaluated candidate continuations as a training target. The learned predictor may choose a different continuation, so its score need not match this target exactly.
We tested the three-step predictor on 32 randomly sampled training prompts, with one seed per prompt. We tested five first endpoints for each sample, giving 160 state–action pairs. After each first transition, the predictor completed the video. The mean achieved score was 0.77714, compared with a mean target of 0.78585. The mean gap was 0.00872 on the normalized training reward.
The preference for high-noise first endpoints remains under both alternative labels. We recomputed the labels using either an equal-size random subset of continuations or their mean score instead of their maximum. For equal-size subsets, we sampled one continuation per endpoint and repeated this 1,000 times.
Table R4. First-endpoint preference under different label rules.
| Label rule | First-endpoint preference for 6/7 or 5/7 |
|---|---|
| Maximum over all candidate continuations | 84.4% |
| Maximum over equal-size random subsets | 74.7% |
| Mean over all candidate continuations | 81.3% |
The preference remains in all three cases (Table R4). This suggests that unequal continuation counts do not fully explain it. The percentages change, so the label rule still affects endpoint choices.
A4. Independent evaluation and training prompts (W4/Q4)
Q4: “Do you have an independent automatic evaluation or a small blinded human preference comparison for three-step ReplanFlow versus four-step AnyFlow? Also, how were the GPT-generated training prompts checked for overlap with VBench?”
Blinded human preference study. Three-step ReplanFlow received more preference votes than four-step AnyFlow (Table R5). We randomly sampled 30 video pairs. Seven participants rated all pairs in a blind survey, giving 210 votes. We hid method names and randomized left/right positions. Participants chose a preferred video or a tie.
Table R5. Blinded human preference: three-step ReplanFlow vs. four-step AnyFlow.
| ReplanFlow wins | Ties | AnyFlow wins |
|---|---|---|
| 42.4% (89) | 24.8% (52) | 32.9% (69) |
VLM evaluation. The results were close to even, with a small lead for ReplanFlow. We evaluated the same 30 pairs with gpt-6-astra, gpt-6.1-sol, and gpt-5.6-sol through Codex. Method names were hidden. ReplanFlow won 51.5% of judgments after excluding ties (Table R6).
Table R6. VLM preference: three-step ReplanFlow vs. four-step AnyFlow.
| Judge | Win / Tie / Loss | Win rate excluding ties |
|---|---|---|
| gpt-6-astra | 8 / 13 / 9 | 47.1% |
| gpt-6.1-sol | 11 / 9 / 10 | 52.4% |
| gpt-5.6-sol | 16 / 0 / 14 | 53.3% |
| Combined | 35 / 22 / 33 | 51.5% |
Training prompts. We used GPT to generate training prompts based on the scene distribution of VBench.
Response to Reviewer HL9V (Part 3/3)
Part 3/3: A5
A5. Training scale, configuration selection, and costs (W9/Q5)
Q5: “How do the reproduced AnyFlow scores compare with the published results? For the six-NFE planners in Table 3, how was the selected 3,000-update configuration chosen, and which validation prompts were used (p19 L1009–1010)?”
Reporting suggestion: “Adding the two-/three-step costs and the Stage-2 training cost would make the offline expense easier to compare with the inference savings.”
Training scale. ReplanFlow achieves a Total score of 83.49 with about 4.3K training pairs. Published AnyFlow reports 83.48 with 256K pairs. The Total scores are nearly identical, despite the large difference in data size. The training settings differ, so our controlled comparison uses the reproduced AnyFlow baseline.
Six-step configuration. We used a small set of GPT-generated prompts and VBench scores to select checkpoints.
Offline costs. We measured data generation, scoring, and predictor training costs on A800 GPUs (Tables R7 and R8). The four-step data cost is 86.23 GPU-hours per seed, rather than the roughly 60 GPU-hours stated in the paper. We will correct this value.
Table R7. Offline data generation and scoring costs on A800 GPUs.
| Budget | Video generation | Scoring | Total GPU-hours per seed |
|---|---|---|---|
| 2 steps | 1.57 h | 1.42 h | 23.96 |
| 3 steps | 4.18 h | 3.47 h | 61.13 |
| 4 steps | 6.05 h | 4.73 h | 86.23 |
Table R8. Predictor training and data-loading time on A800 GPUs.
| Predictor | Training and export | Input loading and checks | Total time |
|---|---|---|---|
| 2 steps | 171.31 s | 195.82 s | 6.12 min |
| 3 steps | 166.31 s | 78.07 s | 4.07 min |
| 4 steps | 191.00 s | 678.68 s | 14.49 min |
Stage 2 training. Stage 2 training took about 33 hours on eight A800 GPUs for each model, or about 265 GPU-hours. This covers 1,200 training updates and excludes checkpoint saving and evaluation.
Response to Reviewer HL9V (Part 1/3)
Part 1/3: A1–A2
We sincerely appreciate your feedback.
We address five points: A1. reliability of the gains A2. tuned fixed schedules A3. target–execution gap and first-endpoint preference A4. independent evaluation and training prompts A5. training scale, configuration selection, and costs
A1. Reliability of the gains (Q1/W1–W3)
We reused the existing VBench Full results to compute paired score differences and 95% confidence intervals. The results support consistent gains at matched budgets (Table R1).
All five score differences are positive. At two, three, and four steps, the confidence intervals are entirely above zero. For the three-versus-four-step comparison and the Table 2 interaction, most of each interval lies above zero.
The first four rows compare ReplanFlow Sequential with AnyFlow Fixed. The last row measures how much broader training coverage increases the gain from switching from Fixed to Sequential scheduling.
Table R1. Paired Total score differences and 95% confidence intervals on VBench Full.
Additional generation seed. Sequential still scores higher than Shared-first with additional generation seed 2. We used the original four-step predictors. The Total gain is 0.0539 points.
Sensitivity to retraining. We retrained both Sequential and Initial-only predictors with training seed 20261111. We used the same evaluation settings as in Table 3 of the paper. Sequential still scores higher than Shared-first, with a Total gain of 0.0376 points. Both checks retain a positive feedback gain (Table R2).
Table R2. Shared-first and Sequential with an additional generation seed and retrained predictors.
A2. Comparison with tuned fixed schedules (Q2/W5–W7)
Sequential scores higher than Tuned Fixed at both three and four steps on the ReplanFlow generator. The Total gains are 0.3943 and 0.2591 points (Table R3). The gains therefore come from more than just replacing the default schedule.
For each generator and budget, we chose the fixed schedule with the highest mean reward on the offline training data. We then froze the schedule and tested it on VBench Full. Tuned Fixed also improves on the default schedule, but Sequential still scores higher.
The selected schedules are [1, 6/7, 4/7, 2/7, 0] for four-step AnyFlow, [1, 6/7, 3/7, 0] for three-step ReplanFlow, and [1, 6/7, 4/7, 1/7, 0] for four-step ReplanFlow. These are normalized endpoints before the scheduler shift. Each schedule is fixed across all evaluation samples.
Table R3. Tuned fixed schedules and sequential scheduling on VBench Full.