acceptodds
Under review as a conference paper at ICLR 2027

Goodbye Drift: Anchored Tree Sampling for Long-Horizon Video-to-Video Generation

Abstract

Long-horizon video generation suffers from two intertwined issues. First, there is drift, where video quality degrades over time. Second, there are continuity issues which manifest as object permanence issues, or improperly rendering transient content (e.g., an object that appears in non-consecutive frames changing color/style). Recent autoregressive distillation methods seek to improve both, but retain sequential rollouts that can accumulate errors over long horizons. We introduce **Anchored Tree Sampling (ATS)**: a training-free inference-time scheduler that addresses this accumulation by replacing left-to-right rollout with sparse-to-dense, anchor-bounded imputation organized as a tree. A root call produces sparse anchors over the full horizon, recursive refinement generates intermediate anchors, and final leaf spans are synthesized between neighboring anchors. For a horizon requiring autoregressive chunks, ATS uses a balanced hierarchy with sequential generation levels, replacing open-ended rollout with anchor-bounded infilling. We focus on V2V generation in the *static-camera* regime, where sparse anchors over the horizon are well approximated by the dense conditioning signal, and the base model can produce them without retraining. We evaluate ATS against two contemporary autoregressive baselines on Wan VACE, across five conditioning modalities (inpainting, outpainting, edge, pose, depth). We show that ATS outperforms both competitors in overall quality, as well as in drift prevention. We additionally demonstrate stable -minute generation on LTX- across the same five modalities.

Primary area:
generative models
Keywords:
long video generation, video-to-video generation, diffusion models, hierarchical sampling, training-free inference, parallel generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Hx6pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Anchored Tree Sampling (ATS), a training-free inference strategy for long-horizon video-to-video generation. Instead of sequential autoregressive rollout, ATS hierarchically generates sparse anchors and performs bidirectional infilling between neighboring anchors, reducing sequential dependency and enabling parallel generation. The method is evaluated on static-camera V2V tasks across five conditioning modalities, showing improved frame quality and reduced temporal quality variation over the evaluated AR baselines, with demonstrations of over 40-minute video generation.

Critical strengths
  1. The proposed ATS framework is conceptually simple and practically appealing. It reorganizes long-horizon generation into hierarchical anchor construction and bounded bidirectional infilling without modifying or retraining the underlying video generator. The resulting tree-structured dependency also naturally exposes substantial parallelism.
Critical weaknesses
  1. Limited quantitative evaluation of continuity. The main metrics (AQ/IQ and their temporal variations) measure frame-level quality rather than identity, motion, or object consistency, while the central continuity claim is supported largely by qualitative examples

  2. The contribution of tree scheduling is not isolated. ATS simultaneously introduces bidirectional endpoint conditioning and hierarchical tree scheduling, making it unclear which component is responsible for the improvements. The paper explicitly acknowledges this confounding factor.

  3. The experimental scale is limited. The main evaluation uses only five source videos (one per modality) and a single random seed, making the robustness and generalizability of the reported improvements difficult to assess.

  4. Baseline coverage and fairness could be improved. Several relevant training-free long-video methods are not experimentally compared, and the LongLive AR/ATS comparison is not weight-matched, preventing the observed difference from being attributed solely to the proposed sampling strategy.

Questions for authors

Please see the Critical Weaknesses.

Reviewer_ZL34OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets long-sequence video-to-video (V2V) generation scenarios and proposes an inference-stage, training-free scheduling strategy called Anchored Tree Sampling. Through a tree-structured "sparse-to-dense" hierarchical filling mechanism, it replaces the traditional left-to-right autoregressive generation approach. ATS adopts a bidirectional generation primitive: it first generates sparse anchors across the entire sequence, then recursively refines intermediate anchors and fills dense segments between adjacent anchors, thereby constraining error drift within anchor intervals and reducing the critical path latency from K serial segments to L+1 parallelizable levels. In the fixed-camera V2V task based on Wan 2.1 + VACE, ATS demonstrates excellent long-sequence generation quality.

Critical strengths
  1. The research problem proposed in this paper is clearly important and offers a solution approach different from existing autoregressive methods. Unlike existing work that employs autoregressive distillation or historical context management to mitigate error accumulation, this paper attempts to start from the structural design of the generation task, replacing unidirectional extrapolation with local completion under bidirectional constraints.
  2. ATS modifies the sequential structure into a tree structure, adjusting anchor organization and generation scheduling to accomplish long video generation without requiring modifications to the model architecture or retraining of parameters. This reduces the cost of method deployment and adaptation, carrying practical engineering significance.
  3. The tree dependency structure has clear parallelization advantages. The authors do not merely improve video quality by adding more anchors; rather, they reconstruct the entire generation pipeline into a tree-shaped computation graph with limited dependency depth. Sibling nodes at the same level depend only on already-generated parent-level anchors, and therefore can in principle be generated in parallel. This design gives ATS the potential to reduce critical path latency in offline long video generation scenarios.
Critical weaknesses
  1. The experimental scale of the paper is limited. Its main quantitative experiments on Wan 2.1 + VACE include only five source videos, with one video per conditional modality. Although the videos are extended to 30 minutes, this still cannot substitute for the diversity of video content. Diverse videos differ substantially in motion complexity, number of subjects, degree of occlusion, and visual details. The current experiments make it difficult to determine whether the model is adapting to specific videos, nor can they demonstrate the stability of the proposed method across different data distributions. The authors should add test videos with varying motion and scene complexity.
  2. The paper uses VBench's Aesthetic Quality (AQ) and Imaging Quality (IQ) metrics, which represent aesthetic level and single-frame generation quality. To demonstrate the "long-range consistency" claimed in the paper, additional metrics such as object identity, appearance consistency, motion continuity, or cross-temporal semantic consistency are needed. The current experiments can only show improvement in frame-level quality but are insufficient to directly prove improvement in long-timescale content drift.
  3. The paper lacks key ablation experiments. Although the method section describes in detail the three core components of sparse anchors, recursive refinement, and bidirectional segment completion, there are not enough ablation experiments to validate these design choices. The missing content is mentioned in the Questions section.
  4. The current experimental setup uses static camera positions for all videos, making it difficult to judge whether consistency can be guaranteed when objects outside the frame in the first frame later appear in the frame. It is even more difficult to guarantee generation quality under dynamic camera settings (which the paper also acknowledges).
Questions for authors

see the weakness above

Reviewer_mtDfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Anchored Tree Sampling (ATS), a training-free inference scheduler for long-horizon, offline video-to-video generation. ATS replaces sequential autoregressive rollout with hierarchical, anchor-bounded generation: it first generates sparse anchors across the full timeline, recursively inserts intermediate anchors, and synthesizes dense video segments conditioned on their boundary anchors and per-frame controls. This structure aims to reduce accumulated drift while enabling parallel generation of sibling segments. Validation uses Wan2.1 with VACE and LongLive- and Reward-Forcing-based checkpoints on five 30-minute static-camera videos covering inpainting, outpainting, edge, pose, and depth conditioning. Results show improved average frame-quality scores and reduced quality variation relative to the evaluated autoregressive baselines. The paper also reports a 5.3× latency speedup using eight H100 GPUs and presents LTX-2.3 demonstrations exceeding 40 minutes.

Critical strengths
  1. A practical, reusable design. ATS extends pretrained video generators through anchor-bounded infilling and parallelizable scheduling, making it practical for long-horizon V2V without additional training.
  2. Long-duration validation. Thirty-minute comparisons across five conditioning modalities and 40-minute demonstrations establish feasibility in static-camera settings. The weight-matched Reward Forcing comparison shows improved average frame quality and reduced quality variation.
Critical weaknesses
  1. The contribution of tree scheduling is not isolated. ATS changes both the generation order and the use of bidirectional endpoint conditioning relative to the AR baselines. Without comparison against the same bidirectional model using a left-to-right anchor-and-infill schedule, the improvements cannot be clearly attributed to the tree structure, limiting evidence for the central contribution.
  2. The headline speedup lacks a clearly hardware-matched comparison. ATS uses eight H100 GPUs, while the AR baselines’ GPU allocation and multi-GPU parallelization are not clearly documented. Consequently, the reported 5.3× speedup does not establish a latency advantage under an equal hardware budget.
  3. Insufficient validation of discontinuous temporal packing. ATS packs temporally distant source windows into a short clip and applies a generator trained on continuous footage. This mismatch may compromise anchor quality, conditioning adherence, and motion continuity, but sensitivity to temporal gaps and motion changes is not systematically evaluated. Moreover, no supplementary videos were available in the reviewed materials, limiting independent assessment of motion smoothness and stitching artifacts from the frame montages alone.
Questions for authors
  1. Controlled evaluation. How does ATS compare with a left-to-right anchor-and-infill scheduler using identical model weights, boundary conditioning, per-call context limits, and hardware resources?
  2. Hardware clarification. How many GPUs were used for each AR baseline, and was model or sequence parallelism enabled? Does the reported speedup persist when both methods receive eight H100 GPUs, and what are their respective GPU-hour costs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.