- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes EcoVideo++, a training-free large–small model collaboration framework for video DiTs that allocates large-model computation across denoising stages, temporal segments, and spatial regions using cheap online proxies, namely the clean-latent change for switching, neighboring-frame latent differences for segment selection, and the small model's CFG gap for spatial cores, combined with context-expanded crops, global 3D RoPE offsets, and smooth fusion, and reports 2.29–2.93× speedups on Wan2.1, Wan2.2, and CogVideoX with VBench within 0.0027 of large-only inference.
Critical strengths
- Extending large–small collaboration from step-level switching to spatiotemporal regions is well-motivated and practically relevant for expensive video DiTs.
- The online routing signals add negligible overhead, and the CFG-gap signal reuses computations already required for classifier-free guidance.
- The joint-denoising design, including context expansion, global RoPE alignment, and smooth fusion, is carefully engineered and each component is ablated.
- The evaluation is broad, covering three model pairs, the full VBench prompt set with five seeds, signal-replacement ablations, sensitivity sweeps, and compatibility with TeaCache.
- The paper is clearly written, with an explicit algorithm and detailed implementation settings for the Wan models.
Critical weaknesses
- EcoVideo++ outperforms large-only inference on many dimensions such as aesthetic quality, imaging quality, overall consistency, multiple objects, and scene, even though most post-switch computation comes from the small model, and no explanation is given.
- The main claims rest on VBench gaps of at most 0.0027 but are not supported by confidence intervals, human studies, or reference-based fidelity metrics against large-only outputs.
- The capability analysis uses only 40 prompts and discusses the proxy–gap correlations only qualitatively.
- Potential crop-induced boundary artifacts and global-coherence failures, especially for high-motion or multi-object videos, are not analyzed.
- Lack of Failure Case Analysis. For instance, how does the method handle scenes with rapid, complex motion spanning the entire frame (where spatial selection might miss crucial background dynamics)? How does it handle long videos where the small model's temporal consistency inherently degrades?
Questions for authors
- How does EcoVideo++ surpass large-only quality on multiple dimensions, and can you provide confidence intervals and reference-based fidelity against large-only outputs?
- What switch step does the step-only variant in Table 3 use, and is it the same as in the full method?
- What are the average switch step, the actual fraction of tokens processed by the large model after context expansion, and the per-component latency breakdown?
- How were the hyperparameters chosen for Wan2.2 and CogVideoX, and were they tuned on the evaluation prompts?
- How does EcoVideo++ compare with region-adaptive sampling or sparse-attention methods at matched speedup?
- Can you provide human preference results and a failure-case analysis, for example when spatial cores miss relevant objects?
7 comments
Response to Reviewer T3W1 (2/5)
A1 (continued). Reference-based fidelity against large-only outputs. We compute PSNR, LPIPS and SSIM between the videos generated by each method and the large-only videos under the same prompts and seeds on Wan2.1-14B + Wan2.1-1.3B.
Table 2: Similarity to Wan2.1-14B outputs and speedup.
| Method | PSNR ↑ | LPIPS ↓ | SSIM ↑ | Speedup ↑ |
|---|---|---|---|---|
| T-Stitch | 11.3046 | 0.6531 | 0.3747 | 1.66× |
| HybridSD | 8.4820 | 0.8249 | 0.1829 | 1.50× |
| EC-Diff | 8.6250 | 0.8288 | 0.1814 | 1.55× |
| SRDiffusion | 16.2905 | 0.4486 | 0.5469 | 1.96× |
| EcoVideo++ | 14.3902 | 0.4815 | 0.4599 | 2.93× |
EcoVideo++ ranks second on all three metrics while being the fastest. All collaborative methods show limited similarity, because large–small collaboration inherently changes the generation trajectory: once the small model participates in denoising, the difference at each step is carried into all subsequent steps, so no collaborative method can reproduce the large-only output pixel by pixel.
Human evaluation. We conduct a blind Good/Same/Bad (GSB) user study on Wan2.1. We randomly sample 100 VBench prompts and generate videos with large-only inference, SRDiffusion, EC-Diff and EcoVideo++ under the same seed. For each prompt, EcoVideo++ is paired with each of the three baselines, giving 300 pairs. 24 participants each rate 50 pairs, so every pair is rated by 4 participants and each row of Table 3 collects 400 valid votes. Videos are shown side by side in random order without method names, and participants judge prompt adherence, visual quality and temporal consistency.
Table 3: GSB preference on Wan2.1 (%, G / S / B = EcoVideo++ is better / same / worse; 400 votes per row).
| EcoVideo++ vs. | Prompt adherence (G/S/B) | Visual quality (G/S/B) | Temporal consistency (G/S/B) |
|---|---|---|---|
| Large-only | 28.25 / 37.00 / 34.75 | 37.25 / 30.00 / 32.75 | 21.75 / 32.75 / 45.50 |
| SRDiffusion | 41.75 / 34.50 / 23.75 | 49.25 / 29.00 / 21.75 | 42.00 / 37.00 / 21.00 |
| EC-Diff | 41.25 / 36.00 / 22.75 | 69.50 / 21.00 / 9.50 | 50.50 / 32.75 / 16.75 |
Against large-only inference, EcoVideo++ achieves a 2.93× speedup and is rated better or tied in 65.25% of votes on prompt adherence, 67.25% on visual quality (37.25% better, 30.00% tied), and 54.50% on temporal consistency. Against SRDiffusion, 49.25% of votes favor EcoVideo++ on visual quality and 42.00% on temporal consistency, rising to 78.25% and 79.00%, respectively, when ties are included. Against EC-Diff, 69.50% of votes favor EcoVideo++ on visual quality and 21.00% rate the two methods equally, giving 90.50% better or tied.
W3: The capability analysis uses only 40 prompts and discusses the proxy–gap correlations only qualitatively.
A2: Thank you for the comment. We clarify the scale and the quantitative evidence.
Scale. The 40 prompts are split into 20 simple and 20 complex prompts, each with three seeds, giving 120 trajectories per model pair. At every step, both models are evaluated on the same latent, timestep and text condition, so each trajectory provides paired large–small predictions at every step, frame and spatial position.
Quantitative analysis. The proxy–gap correlation is quantified by recall. In Figure 3(b) of our paper, ranking frames by the neighboring-frame latent difference is evaluated by Top-3 recall of the frames with the largest model gap, against uniform and random selection. In Figure 3(c–d), the small model's conditional–unconditional response is evaluated by Top-8% recall of the positions with the largest large–small gap, with 73% and 70% spatial overlap on the shown examples. Recall directly quantifies what routing needs: whether the selected positions are those where the large model matters most. At the generation level, the ablation in Table 2 of our paper tests each proxy on the full VBench under matched budgets: replacing the spatial signal with uniform token selection lowers VBench from 0.8348 to 0.8102, replacing the temporal signal with uniform frame selection gives 0.8235, and replacing the switching signal with fixed switching gives 0.8108.
Response to Reviewer T3W1 (3/5)
W4: Potential crop-induced boundary artifacts and global-coherence failures, especially for high-motion or multi-object videos, are not analyzed.
A3: Thank you for this point. EcoVideo++ is designed so that cropping does not break the global prediction.
Design. The small model always predicts the complete video, so every location receives a continuous full-frame update. The large model reads each crop together with surrounding context, uses the crop's global coordinates for 3D RoPE, and only its core prediction is smoothly blended back with a weight that tapers toward the boundary. Therefore, there is no hard seam between large- and small-model regions.
Quantitative evidence. The cumulative ablation in Table 4 of our paper isolates each component. Core cropping with hard replacement scores 0.8127; adding context expansion gives 0.8216; global-coordinate 3D RoPE gives 0.8294; and smooth fusion gives 0.8348. The full design recovers 0.0221 VBench over hard replacement with only 34 s additional latency (674.33 s vs. 640.18 s).
Visual evidence. At https://anonymous.4open.science/w/ecovideo-beta/, we provide videos with the crop boxes overlaid, including an object moving quickly across a crop boundary and two objects separating within the scene, which show no visible seams and no missed objects.
Q2: What switch step does the step-only variant in Table 3 use, and is it the same as in the full method?
A4: Thank you for the question. The step-only variant in Table 3 of our paper uses a stricter threshold (τ = 0.045) and hands the entire video over to the small model at around step 39 on average. The full method uses the same stability rule with τ = 0.15 and starts joint denoising at step 7.53 on average, after which the 14B model processes only the selected crops.
Q3: What are the average switch step, the actual fraction of tokens processed by the large model after context expansion, and the per-component latency breakdown?
A5: At the default Wan2.1 settings (P = 2, CFG = 5), the full method starts joint denoising at step 7.53 on average across 128 videos. After context expansion, the 14B model processes 6.60% of the full-frame video tokens per joint step on average, and the full 50-step trajectory, including the initial full-frame steps, uses 18.80% of the large-only 14B token count. A joint step uses 1.742 crops on average (at most two, median expanded size 4 × 48 × 62 in latent frames × height × width) and takes 8.9983 s, with 7.9374 s for the full-frame small-model pass, 0.8969 s for the 14B crop passes, 0.1553 s for region selection, 0.0064 s for fusion, and 0.0023 s for the scheduler.
Response to Reviewer T3W1 (1/5)
We sincerely thank you for the valuable feedback and for recognizing our work. We address each question below, and full videos are available at https://anonymous.4open.science/w/ecovideo-beta/.
A1: Thank you for raising these points. We address them in four parts: why EcoVideo++ scores higher on some dimensions, confidence intervals, reference-based fidelity, and human evaluation.
Why EcoVideo++ scores higher on some dimensions. We first clarify that EcoVideo++ does not surpass large-only inference overall. As shown in Table C1 of our paper, on Wan2.1 both its Quality score and Semantic score are slightly lower than those of large-only inference, and the gains on aesthetic quality, imaging quality, overall consistency, multiple objects and scene are offset by small drops on dimensions such as subject consistency, temporal flickering, human action and color. The paired prompt-level analysis in Table 1 makes this explicit. With the same prompts and seeds, Aesthetic Quality, Multiple Objects and Scene become slightly negative (−0.00852, −0.00819, −0.01221), Overall Consistency is +0.00484, and every per-dimension difference is below 0.02 in magnitude. We therefore regard these differences as fluctuations around the large-only quality rather than a systematic improvement. The consistent gain is on Imaging Quality (+0.01780 [+0.00719, +0.02848]). One reason is that collaboration changes the generation trajectory: the spatial cores are selected by the small model's conditional–unconditional response, which concentrates the large-model computation on prompt-related objects, while the remaining regions follow the small model's prediction. Full videos of large-only inference and EcoVideo++ are available at https://anonymous.4open.science/w/ecovideo-beta for direct comparison.
Confidence intervals. The official 16-dimension VBench prompt list contains 944 distinct prompts, and both methods use the same five seeds for every prompt. We pair EcoVideo++ with large-only inference by prompt, resampling the five seeds and all applicable dimensions of each prompt together, and compute 95% intervals of the paired difference with 10,000 prompt-stratified bootstrap resamples on normalized dimension scores.
Table 1: Paired differences (EcoVideo++ − large-only) and 95% intervals. Per-dimension rows: Wan2.1-14B + Wan2.1-1.3B.
On all three model pairs, the paired VBench difference is at most 0.0041, and the whole interval lies within 0.0083 of large-only inference (below 1% of the score), which quantifies that EcoVideo++ is on par with large-only inference.