- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies the denoising process of video models through maze example. It finds that trajectories extracted from early predictions are already close to those in the final generated videos. Based on this observation, it proposes Early Planning Beam Search (EPBS) to screen seeds before completing generation, and Chaining with Early Planning (ChEaP) to extend solutions across multiple video segments. Experiments show improvements in sampling efficiency and task success rates.
Critical strengths
-
The paper is well organized, with a clear connection between its empirical observations and proposed methods.
-
The experiments are extensive, covering trajectory analysis, hyperparameter selection, efficiency, and transfer to other tasks.
-
The proposed screening strategy is simple and requires no model retraining. The efficiency analysis also considers intermediate decoding overhead.
Critical weaknesses
-
Section 3.2 / Figure 2: The analysis relies on extracting a single trajectory from each intermediate prediction. If an early prediction contains multiple motion branches—a phenomenon I have frequently observed in maze-solving videos generated by video models—it is unclear how the tracker selects or combines these branches.
-
Section 5 / Figure 6: The correlation between path length and failure does not establish whether failures are due to insufficient video duration or increased planning difficulty. A direct test with longer videos on the same mazes is missing. Furthermore, ChEaP uses approximately 630 NFEs versus 400 for the main baselines and additionally uses maze-layout information, so the comparison does not fully isolate the benefit of chaining.
-
Section 6: The irregular-maze and LIBERO experiments extend the evaluation beyond grid mazes, but still rely on explicit goals and task-specific visual scoring. How can the approach be extended to a broader range of tasks, and how would suitable verifiers be designed for those tasks?
Questions for authors
-
How does the trajectory extractor handle multiple motion branches in early predictions, and have you checked its reliability in such cases?
-
Does generating longer videos improve success on the same long-path mazes?
-
How can the approach be extended to a broader range of tasks, and what would be required to design suitable verifiers?
2 comments
Thank you for the detailed response and the additional experiments.
Multiple branches. Table 7 resolves my concern about Fig. 2. My question on whether the endpoint-based score prefers wall-crossing or discontinuous trajectories remains unanswered; a brief note on how EPBS handles such cases would help.
Video duration and controls. Good job. Please make sure to state the revised interpretation clearly in Section 5.
Verifier design. I truly understand the limitation and I suggest explicitly discussing it in the limitations section, ideally with any preliminary evidence on whether a VLM-based verifier can reliably score noisy early decoded predictions.
Overall, the response addresses most of my concerns, and I maintain my positive assessment.
Reply to Reviewer URJx
Dear Reviewer URJx,
Thank you for appreciating the organization of our paper alongside our extensive experiments and efficient method. Your concerns are about (1) multiple motion branches, (2) the effect of video duration on long-path mazes, and (3) generalization of verifier design. We address each below.
W.1/ Q.1: Multiple motion branches in early predictions
Early predictions sometimes contain multiple motion branches. The Section 3.2 extractor maps the centroid of moving pixels in each frame to a grid cell. With two moving regions, both contribute to the centroid, which can fall between branches.
To test whether averaging explains the convergence in Fig. 2, we added a branch-preserving measure (Appendix E.2, Table 7). We remove isolated specks smaller than 5% of a grid cell or 50 pixels, whichever is larger. In each frame, we measure the fraction of moving pixels within one cell of the final video’s motion in the corresponding frame. We then average this fraction over frames. A competing branch that moves away from the final route lowers the score in proportion to its share of the predicted motion.
On the same 747 trajectories used in Fig. 2, we obtain the following results across all four maze sizes.
At step 1, the motion overlaps the two final videos to a similar extent. By step 5 of 40, 88.6% of the predicted motion lies near the same seed’s final motion, compared with 53.5% for an independent seed. The early convergence we report in Fig. 2 therefore remains visible when all branches contribute to the measurement, and is not caused by potential averaging. We also include example frames so readers can see how the extractor handles a prediction with multiple branches in Appendix E.2, Figure 12 (L1312).
W.2/Q.2: Longer videos and chaining controls
We directly tested whether longer videos improve success on the same long-path mazes. We evaluated Wan2.2-14B on all 64 Frozen Lake mazes requiring 10–13 moves, using four matched seeds per maze and increasing generation length from 81 to 121 frames. We also report ChEaP’s result on the same 64 mazes.
Longer videos do improve success on these mazes. Increasing the duration raises the number of mazes solved from 4 to 14, although 50 of 64 remain unsolved at 121 frames. ChEaP solves 42 of the same 64 mazes by chaining shorter generations. We agree that path length alone cannot distinguish insufficient video duration from increased planning difficulty. The longer-video experiment shows that duration contributes to success, while leaving open how much of the remaining difficulty comes from planning the route or executing it within a single video. We revise Section 5 to clarify this interpretation (L430).
We also added a separate comparison to address your concern about ChEaP’s access to the maze layout and its higher NFE budget (Appendix C, Table 5). On 145 Frozen Lake mazes, we gave both non-chained baselines the same obstacle information and matched their budgets to ChEaP’s mean cost at each grid size, rounded up to the next 40 NFEs. This gave budgets of 800, 840, and 640 NFEs on 6×6, 8×8, and 10×10 mazes, respectively. Under these settings, ChEaP solved 48 mazes, obstacle-aware EPBS solved 23, and obstacle-aware best-of-N solved 12.
Finally, we kept the chaining and pivot-selection rules fixed and generated every candidate in full. This increased the number solved from 48 to 51, while raising the mean cost from 731 to 1,842 NFEs per maze. Full generation thus required 2.52x as many NFEs per maze for three additional solves.
W.3/Q.3: Generalization to broader tasks and verifier design.
EPBS requires a verifier that can rank early decoded predictions from different seeds according to task-relevant progress. We chose simple verifiers that measure progress directly. For regular and irregular mazes, we use spatial distance to the goal. For LIBERO, we use a SAM 2-based object-relation score. These verifiers are inexpensive enough to apply across many seeds before deciding which generations to complete.
For broader tasks when such relations are difficult to specify by hand, a learned or VLM-based verifier conditioned on the task instruction could provide a more flexible score. These verifiers would need to reliably evaluate intermediate predictions and the computational cost must remain low to preserve EPBS’s efficiency gains. We use lightweight verifiers for this reason, while we acknowledge that tasks without readily verifiable progress remain challenging.