- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper accelerates video diffusion by combining perceptual-semantic token scheduling with learned reconstruction and aligned K/V context. It reports up to 2.64x acceleration on Wan and 2.48x on LTX while largely preserving native output quality.
Critical strengths
-
The work is substantial, with experiments across two backbones, baseline comparisons, component ablations, and runtime analysis. Although some targeted ablations are missing, the overall technical development and experimental evaluation are thorough.
-
The method demonstrates practical acceleration on both backbones, reaching up to 2.64x on Wan and 2.48x on LTX with limited changes in aggregate VBench scores. Measured wall-clock latency and multiple operating points provide concrete evidence of its speed-fidelity trade-off.
Critical weaknesses
-
Overstated conceptual framing. The method is essentially feedback-driven token scheduling coupled with learned reconstruction and noise-aligned K/V context: new DiT observations guide subsequent refresh decisions (Eqs. (7), (A7), and (A8)). Equation (2) defines the "compute code" as an execution-mask sequence. While technically substantial, the work does not clearly establish why this constitutes a new computational paradigm beyond relabeling the scheduling and reconstruction loop. The authors should clarify the additional design principle provided by the coding perspective or moderate the paradigm-level claims.
-
Reliability of early importance estimates. In the evaluated configuration, visual and semantic cues are extracted once after step 10 and used throughout the remaining 40 steps. Final-output results support the overall approach, but there is no targeted analysis of whether these initial importance estimates remain reliable as finer details emerge. Dynamic refresh decisions alone do not isolate the contribution of the fixed cues. An analysis across denoising stages or a comparison with periodic cue extraction would strengthen this design choice. (Section 3.4; Algorithm A2; Appendices A and G.)
-
Missing warm-up length ablation. The 10-step dense warm-up affects the reliability of the entrance estimates and the remaining opportunity for acceleration, but the paper does not compare alternative switching points. The sensitivity study varies tau and beta while keeping warm-up fixed. An ablation at a fixed total sampling budget should report final quality and end-to-end latency, clarifying whether the auxiliary networks are adapted to each entrance state. This would establish how sensitive the reported quality-latency trade-off is to the switching point. (Section 4.1; Appendices B.4-B.5, C.1, and I.)
-
Boundary consistency. Full-context attention and neighborhood safeguards may help maintain consistency, and Table 3 supports the decoder's overall effectiveness. However, no targeted evaluation checks for spatial or temporal artifacts at interfaces between refreshed, unrefreshed, and directly reconstructed regions. A brief analysis or close-up video comparison around these interfaces would be useful. (Table 3; Appendix B.2-B.5.)
-
Missed candidates. The method includes offline risk calibration and neighborhood safeguards, but the candidate set remains fixed after warm-up. The router cannot subsequently admit tokens excluded due to underestimated risk, and their states follow the direct reconstruction path. It would be useful to discuss the potential impact of these errors and whether candidate membership could be reconsidered. (Appendix B.2, B.4, and B.5.)
-
Attribution within the complete router. Table 2 supports visual and semantic guidance under coverage-based selection, but several factors change together when comparing those strategies with the complete router. Removing visual guidance, semantic guidance, reconstruction risk, and innovation feedback individually under matched execution budgets would clarify their respective contributions. (Table 2; Equation (A7); Appendix G.)
-
Training and resource details. The paper describes teacher-trajectory generation, but does not clearly state the training prompt sources or the total numbers of prompts and trajectories. The provenance of Wan's pretrained reconstruction checkpoint also needs clarification. Reporting training compute, including teacher-data generation, auxiliary parameter counts, and additional peak GPU memory would complement the existing inference-timing results and improve reproducibility. (Appendices B-C and J.)
Questions for authors
See weakness.
5 comments
R3 · Reviewer X9ET (Part2)
R3-W3: Dense warm-up length and adaptation to the entrance state
Thank you for suggesting a comparison of warm-up lengths. In Table 14 of Appendix F.1, we added a controlled experiment with K = 5, 10, 20, and 30 on 100 VBench prompts using the 41-frame Wan configuration, with the total sampling budget fixed at 50 steps.
-
We train a separate decoder for each entrance state. All four decoders, including K = 10, use identical initialization, training trajectories, sample order, and 900 optimizer updates. Each decoder takes the clean-state prediction at its corresponding entrance as input and learns to recover the final Native latent. The backbone, VAE, visual and subject auxiliary branches, and original step-10 calibration are shared and frozen. This design compares entrance adaptation under the same training budget; Appendix F.1 specifies the training and freezing settings for our new experiment.
-
Increasing warm-up from 5 to 10 steps substantially improves the basis for reconstruction. The results of our new warm-up experiment in Table 14 show a 55.3% reduction in LPIPS and a 5.30 dB increase in PSNR, with 5.09 s of additional latency. We also added an analysis of direct latent-prediction error in Appendix F.1: the decoder's prediction MSE decreases by 54.3%, supporting the benefit of a more informative entrance state for reconstruction.
-
Longer warm-up further improves fidelity while increasing latency. At K = 10, LPIPS is 0.0527 and latency is 29.18 s, retaining a 2.55× speedup over Native. At K = 20/30, LPIPS decreases to 0.0164/0.0086 and latency increases to 40.82/52.99 s. Our newly added Table 14 reports final fidelity, end-to-end latency, and actual DiT execution for each configuration.
-
Ten warm-up steps provide a practical default under the evaluated conditions. Compared with an earlier switch, this setting obtains substantial reconstruction gains while preserving 40 steps for efficient coded computation. The results characterize the fidelity–latency trade-off associated with the switching point.
R3-W4: Boundary consistency between computation regions
Thank you for suggesting a targeted evaluation of boundary consistency. In Appendix F.2, we added an analysis of three types of region interfaces across 100 Wan videos. Our new Table 15 and Fig. 12 report the boundary statistics, and we added aligned visualizations in Fig. 13 covering five spatial-error quantiles.
-
Shared reconstruction and context maintain coupling between regions. Dense reconstruction, noise-aligned context references, and neighborhood-aware risk estimates keep selectively evaluated tokens connected to the complete video representation and incorporate reconstruction needs near boundaries into computation allocation.
-
Matched controls support spatial compatibility. We match Native gradients and entrance prediction risk, using candidate-region interiors as controls for refreshed/unrefreshed interfaces. Our new spatial analysis in Table 15 and Fig. 12(a) reports mean spatial excess errors of −0.657 × 10⁻³ for refreshed/directly reconstructed interfaces, −0.205 × 10⁻³ for unrefreshed/directly reconstructed interfaces, and −0.064 × 10⁻³ for refreshed/unrefreshed interfaces. The last has a confidence interval of [−0.138, +0.010] × 10⁻³ and a mean close to zero. These statistics agree with the preserved local structures and color transitions in our added visualizations in Fig. 13, supporting spatial compatibility between reconstructed content and selectively computed content.
-
The temporal statistics characterize errors at the interfaces. Our new temporal analysis in Table 15 and Fig. 12(b) also reports positive temporal excess errors of 0.276 × 10⁻³ and 0.824 × 10⁻³ at refreshed/unrefreshed and unrefreshed/directly reconstructed interfaces, respectively, indicating measurable temporal excess error near unrefreshed regions. Together, the spatial and temporal statistics characterize reconstruction behavior at the three types of interfaces.
R3 · Reviewer X9ET (Part3)
R3-W5: Whether fixed candidates miss tokens that later require computation
Thank you for the comment. In Appendix E.3, we added candidate-coverage analysis and matched-budget experiments on membership updates, examining the roles of entrance selection and subsequent dynamic routing.
-
The entrance candidate set covers the main reconstruction demands. We added coverage statistics over 100 videos in Table 11: the candidate set covers 99.96% of the top-10% endpoint-error tokens and 99.70% of the top-10% final-output-error tokens, while retaining approximately 97.6% coverage of the top-20% importance tokens at later steps. This provides a candidate pool with broad coverage, within which actual computation can adapt throughout denoising.
-
We compare membership updates at the same actual execution budget. On 20 paired videos with nonempty excluded regions, all policies retain the same candidate capacity, per-step refresh quotas, and actual DiT token execution. Our new membership-update experiment in Table 12 shows that the fixed candidate set achieves global LPIPS 0.045902, reducing LPIPS by 6.06%, 4.75%, and 14.00% relative to score-based, conservative, and random membership updates, respectively. Our added paired statistics in Table 13 show that all three improvements remain significant after Holm correction. The conservative policy also protects scheduled refreshes and applies an admission margin to test more selective membership changes.
-
Regional results explain the global trade-off. The regional metrics from our new experiment in Table 12 show that membership updates improve reconstruction in the originally excluded regions but increase error in the original candidate regions. Entrance selection and dynamic routing therefore serve complementary roles: the former maintains coverage of the main reconstruction demands, while the latter adjusts the limited refresh budget within this stable set to preserve overall video fidelity.
R3-W6: Individual contributions within the complete router
Thank you for the suggestion. In Appendix E.1, we added matched-budget ablations on 100 Wan prompts, individually removing visual guidance, semantic guidance, reconstruction risk, and innovation feedback. Our new Tables 8–9 report the final-output and shared-state update results, respectively.
-
Each ablation changes only the corresponding signal's contribution to routing. All variants in our new ablations in Table 8 share the reconstruction network, initial candidate count, per-step refresh counts, and actual DiT token execution, directly comparing the signal's effect on routing decisions at the same computation budget.
-
Visual and semantic guidance improve update effectiveness in detail and subject regions, respectively. In our newly added shared-state experiment in Table 9, actual updates start from identical states, candidate locations, and observation histories at matched budgets. Visual guidance increases detail-error removal from 23.21% to 29.11%; semantic guidance increases subject-error removal from 22.61% to 27.93%, with subject evaluation using the 60 prompts with independent valid masks. Our new ablations in Table 8 further report changes in the final videos: removing visual guidance increases detail MAE from 0.03678 to 0.03728; removing semantic guidance increases subject LPIPS from 0.0486 to 0.0495 while reducing global LPIPS from 0.0479 to 0.0454. This trade-off reflects the allocation of limited computation toward subject regions by semantic guidance.
-
Risk and innovation feedback affect different error metrics. The other two new ablations in Table 8 show that removing risk guidance reduces PSNR from 29.910 dB to 29.690 dB and increases detail MAE from 0.03678 to 0.03788. Removing innovation feedback increases subject LPIPS from 0.0486 to 0.0497 and temporal-difference MAE from 0.01204 to 0.01211.
R3 · Reviewer X9ET (Part1)
R3-W1: Design principle and conceptual framing of coded computation
Thank you for the comment. Coded computation uses the reconstructability of omitted generative information to guide selective execution, incorporating reconstruction reliability directly into computation allocation. We have replaced “paradigm” with “framework” in the introduction's first contribution statement and explicitly emphasized a reconstruction-aware formulation: jointly deciding which generative computations to execute and how to recover omitted information throughout the diffusion trajectory.
Our design incorporates learned reconstruction reliability into computation allocation. TeaCache estimates temporal output changes to guide caching; ProfilingDiT uses foreground/background importance for adaptive feature reuse; and SenCache makes caching decisions based on output sensitivity. We jointly consider perceptual-semantic importance, reconstruction risk, and observation history. Computation is therefore allocated according to both content importance or change and the value of explicit evaluation relative to what the decoder can recover.
Computation and reconstruction are coupled throughout generation. The compute code in Eq. (2) specifies the observation pattern along the diffusion trajectory. Learned dense prediction recovers omitted information and constructs K/V references aligned with the diffusion state for subsequent sparse DiT computation. Fresh observations then update routing decisions. Reconstruction thus participates in the ongoing generation process rather than operating as independent post-processing.
Experiments with matched budgets and execution masks support this design. Fig. 2(c) shows that different computation schedules produce different reconstruction errors at the same budget. Table 2 verifies the benefits of adaptive computation allocation. Table 3 shows that learned reconstruction and diffusion-state alignment substantially improve fidelity even with identical execution masks. Appendices E–F provide the full experimental settings and analysis.
R3-W2: Reliability of early importance estimates during later denoising
Thank you for raising the question of early importance reliability. In Appendix E.2, we added an analysis of stability across denoising stages and controlled refresh experiments using 100 VBench prompts with the 41-frame Wan configuration. The new Table 10 reports the refresh results and paired confidence intervals.
The step-10 cues remain highly consistent with later estimates. Our newly added stage-wise statistics in Appendix E.2 show that fused step-10 importance has mean Spearman correlations of 0.980, 0.974, and 0.970 with estimates at steps 20, 30, and 40, respectively, and an 88.05% recall of the top-20% importance tokens at step 40. These results support the continued usefulness of the initial cues during later denoising.
Periodic extraction does not improve average fidelity or latency in the compared configurations. The configurations in our new refresh experiment in Table 10 share the encoder, decoder, reconstruction, risk map, candidate set, and dynamic feedback mechanism. One-time extraction achieves LPIPS 0.04794 at 29.19 s. One or two additional refreshes yield LPIPS 0.04810/0.04822 at 30.55/31.98 s, respectively. One-time extraction achieves the best average fidelity and lowest latency among these configurations; the additional refreshes increase end-to-end cost by 4.65%/9.55%.
Control experiments separate cue updates from additional probing. In Appendix E.2, we added a control that performs the probes while retaining the original importance map, producing outputs identical to those obtained with fixed cues. We also added offline Oracle refreshes to Table 10 using later Native cues at matched per-step sparse budgets, further supporting the effectiveness of step-10 guidance. Early visual-semantic importance provides a stable content prior, while subsequent observations and dynamic feedback adjust actual token updates.