acceptodds
Under review as a conference paper at ICLR 2027

LongLive-Plug: Once-for-All Distillation for Video Generation

Abstract

Video diffusion models are increasingly developed into specialized models for diverse downstream tasks, and this development often includes a distillation stage, for example to accelerate sampling or to improve long-video generation. This stage is typically repeated for every specialized model. We introduce LongLive-Plug, a once-for-all distillation framework that learns reusable capabilities as LoRAs on a base model for training-free, plug-and-play deployment to compatible downstream models. These capabilities include single-pass classifier-free guidance, few-step sampling, and long-context error correction for autoregressive generation. The adapters remain reusable even when downstream models add conditioning branches, expand output channels. Despite training at a fixed guidance scale, our dedicated CFG LoRA provides text guidance control through its inference weight. Combining it with a few-step LoRA simultaneously preserves few-step generation and CFG controllability on downstream tasks. We verify training-free deployment on 54 downstream models across three backbone families and eight task categories, including world modeling, robotics, editing, and multimodal generation. The approach may support additional compatible models. Each capability can thus be distilled once per backbone family and reused without per-target retraining.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Gen, LoRA, Distillation, Acceleration

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2CoDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper distills three capabilities into LoRAs on a base video diffusion model: CFG distillation, four-step sampling, and long-context error correction. These LoRAs are then plugged into downstream fine-tunes of that base without target training. CFG and few-step distillation are kept in separate LoRAs, so guidance strength can be tuned through the CFG LoRA weight. On SCOPE and Wan2.2-Fun-5B-Control, the transferred LoRAs clearly beat naive four-step sampling and come close to per-target distillation. The long-context LoRA improves long rollouts on ReWorld and Matrix-Game 3.0. Transfer to 54 downstream models across three backbone families is shown qualitatively.

Critical strengths
  1. Practical value and breadth. Repeating distillation for every downstream video model is a real cost. The paper tries its recipe on an unusually wide range of downstream models: full fine-tunes, task LoRAs, ControlNet branches, and models with extra output channels. Where quantitative results are given, the transferred adapters come close to per-target distillation.
  2. Decoupled guidance control. Distilling CFG separately from few-step sampling lets each downstream task adjust guidance without breaking few-step generation. This simple design makes a single adapter set reusable across tasks with different guidance preferences.
Critical weaknesses
  1. Limited novelty, and no comparison with closely related work. Distilling acceleration into a LoRA once and plugging it into fine-tunes of the same base was already done for images by LCM-LoRA [1]. Guidance distillation with continuous control over guidance strength already exists: Meng et al. [2] condition the student on ww, and AGD [3] conditions adapters on the guidance scale and composes them with ControlNet and IP-Adapter. Public four-step, CFG-free distillations also exist for both Wan backbones used in the paper: the lightx2v StepDistill-CfgDistill LoRA for Wan2.1-14B [4], and Wan2.2-TI2V-5B-Turbo [5], whose codebase FlashMotion (listed in Tab. 5) builds on. The long-context LoRA applies LongLive's Streaming Long Tuning [6]. None of these is used as a baseline.
  2. The 54-model claim is mostly qualitative. Tab. 5 states that each entry "is evaluated" but gives no criterion or metric for success. Apart from the H3 camera-control attenuation discussed in App. C.3, the paper does not say whether any attempted model failed. The demo shows 18 of the 54 models, with one selected example each. Quantitative results cover only two acceleration targets and two long-context targets, all in the Wan2.2-5B family. Wan2.1-14B and MiniMax-H3 have no quantitative results.
  3. Tab. 2 has no locally run native 40-step result. Tabs. 1/2 do not include the transferred coupled CFG+step LoRA, which is the direct ablation of the decoupling design. λcfg\lambda_{cfg} is 3 in Tab. 1 and 1 in Tab. 2, and the paper does not say how it was chosen.
  4. The long-context evaluation does not isolate the long-context LoRA. Tab. 3 has no four-step few-step-only row, even though such outputs appear in Figs. 19–21. The ReWorld group also omits ReWorld's official 4-step DMD LoRA [7, Sec. 2.4], while the Matrix-Game group does include an official distilled baseline. In addition, the long-context LoRA's training setup is not given: which causal AR Wan base, what data, and what rollout length (see Q2).
  5. The intro claims that the CFG LoRA weight adjusts guidance "in a near-linear manner, as we observe empirically", but the evidence is limited to qualitative examples (Figs. 5–6, 14–18), with no quantitative measurement of the relationship. Fig. 6A and 6B use different coupled checkpoints, so the two panels cannot be compared directly.

[1] Luo et al., LCM-LoRA: A Universal Stable-Diffusion Acceleration Module, arXiv:2311.05556, 2023. [2] Meng et al., On Distillation of Guided Diffusion Models, CVPR 2023. [3] Perez Jensen & Sadat, Efficient Distillation of Classifier-Free Guidance using Adapters (AGD), TMLR 2025. [4] lightx2v, Wan2.1-I2V-14B-480P-StepDistill-CfgDistill-Lightx2v and Wan2.1-T2V-14B-StepDistill-CfgDistill-Lightx2v, Hugging Face model releases, 2025. [5] Wan2.2-TI2V-5B-Turbo, GitHub: quanhaol/Wan2.2-TI2V-5B-Turbo, 2025. [6] Yang et al., LongLive: Real-time Interactive Long Video Generation, ICLR 2026. [7] Chen et al., ReWorld: An Interactive World Model with Long-Horizon Memory, arXiv:2608.23565, 2026.

Questions for authors

Clarifications (answers could change my assessment):

  1. What exactly does "verified" mean for each of the 54 models in Tab. 5? Were any models tried and then excluded or found to fail?
  2. Where did the long-context adapter come from, and how was it trained? In the supplementary material, the ReWorld "+Long" video is long-reworld-official.mp4 (curve id official), and the Matrix-Game "+Long" video is long-matrix-reworld.mp4 (curve id reworld). ReWorld's paper describes an official 4-step DMD LoRA trained with streaming self-forcing rollouts. Is the "+Long" adapter in Tab. 3 the one distilled on an AR Wan base as described in Sec. 4.5, or ReWorld's official LoRA, possibly transferred to Matrix-Game? Please give its training details (base model, data, rollout length, iterations, rank), and add the official ReWorld LoRA as a baseline in Tab. 3.
  3. How was λcfg\lambda_{cfg} chosen for Tab. 1 (3) and Tab. 2 (1)? Was the test set used? Which adapter variant appears in Figs. 4 and 9, and how does it differ from the one used in the tables?

Additional evidence requested (needed to support claims already made):

  1. Please add the following under the same protocol:
    • a local native 40-step row for Tab. 2;
    • the transferred coupled CFG+step LoRA in Tabs. 1/2;
    • a four-step few-step-only row in Tab. 3.
  2. For a subset of the 54 models that includes Wan2.1-14B and H3 targets, please report per-model quantitative results (one quality metric plus one task-fidelity metric, e.g., camera or action following) for native, naive four-step and transferred inference.
  3. Please give a quantitative curve of text alignment and quality versus λcfg\lambda_{cfg}, compared with native CFG at w~=1+λcfg(wtrain−1)\tilde w = 1 + \lambda_{cfg}(w_{train}-1), to support the "near-linear" guidance claim.

Suggestions (would not change my rating by themselves):

  1. Compare against public Wan step/CFG-distilled releases (lightx2v StepDistill-CfgDistill LoRA for Wan2.1-14B, Wan2.2-TI2V-5B-Turbo), against AGD-style ww-conditioned guidance adapters, and against a short target fine-tune that starts from the transferred LoRA. Report wall-clock latency, and analyze when transfer fails, e.g., as a function of how far the fine-tuned weights have drifted from the base.
8 comments

We thank the reviewer for the precise questions. We answer each Weakness together with the Questions that refer to it.

W1: Limited novelty; no comparison with LCM-LoRA, Meng et al., AGD, lightx2v, Wan2.2-TI2V-5B-Turbo; the long-context LoRA applies LongLive's Streaming Long Tuning.

  • LCM-LoRA. We acknowledge it as the precedent for reusing a distilled LoRA across fine-tunes and have expanded Sec. 2.2 (marked in blue). It targets image models whose variants differ mainly in style, with no separate CFG branch. Our scope differs: diverse video downstream tasks with additional conditioning modules (depth, camera, action, audio, RGBA), guidance kept adjustable after transfer, a long-context LoRA for AR models, and task-capability measurements across three families (Tabs. 6–9; see Q5).
  • Meng et al. / AGD. Both treat the guidance scale as an extra input of the student. This has two drawbacks that our design avoids. (1) Training cost. A scale-conditioned student must be trained over a wide range of guidance scales, whereas our CFG LoRA is trained at a single scale (w = 5) with no additional input; the guidance strength is set afterwards by its merge weight λ_cfg, which keeps training simple. (2) Transfer. A scale input adds a new pathway that the downstream model must accept; AGD in particular introduces new parameters to embed the guidance scale, which further limits its transfer to downstream models. Our CFG LoRA is a plain weight update that is merged into any compatible descendant by weight addition. We verify this transfer on 54 downstream models, 10 of them quantitatively, with guidance remaining adjustable after transfer (Tab. 10). Meng et al. do not study transfer to other models, and AGD evaluates it only on image-model derivatives; neither covers video models with new conditioning branches or output channels. We expanded Sec. 2.2 accordingly (marked in blue).
  • lightx2v (Wan2.1-14B; new Fig. 22). Its only control is a global weight; lowering it also weakens step distillation (hazy, blurry videos below weight 1), whereas our λ_cfg ∈ [0, 1] adjusts the prompted attributes at unchanged four-step quality. At very large weights, both over-guide and degrade (Sec. 4.3).
  • Streaming Long Tuning. We use LongLive's SLT unchanged and do not claim it as a contribution; our claim concerns transferring the resulting LoRA to other AR models. We made this explicit in Sec. 3.4 ("from LongLive", marked in blue).

W2 and Q1: What does "verified" mean for each of the 54 models? Were any tried and excluded, or found to fail? Wan2.1-14B and H3 have no quantitative results.

  • Verified. Ten of the 54 models are evaluated quantitatively (four in the paper, six new: Wan2.1-Fun-14B-Control, H3 ControlNet-Union, SolarWM-H3, Wan-Alpha, Fun-Control-Camera, TalkVerse-5B), six of them with task-capability metrics. Because of time constraints, the remaining 44 are verified by visual inspection against native inference (Sec. 4.2, marked in blue). No attempted model was excluded.
  • Failures and degradations. Besides the H3 camera case in App. C.3 (over 120 SolarWM-H3 test clips our median translation amplitude is 1.04× the ground truth, native 1.03×, so it is an isolated case), we observe: TalkVerse needs the re-noising sampler (Sync-C 4.50 with UniPC vs. 5.07); H3 needs its training-matched sampler for depth fidelity; AR rollouts need the long-context LoRA; a large λ_cfg raises FVD in the four-step composition.
  • Wan2.1-14B and H3: now quantified in Tabs. 6–8 (see Q5).

Q3 (and the λ_cfg part of W3): How was λ_cfg chosen? Was the test set used? Which adapter variant appears in Figs. 4 and 9?

  • λ_step is fixed to 1. λ_cfg is chosen by visual inspection of 10 prompts outside the test set; the test set is never used. Tab. 2 uses (1, 1). The new Wan2.1-14B experiment selected λ_cfg = 1 with the same procedure (Tab. 6).
  • Figs. 4 and 9. The transferred outputs use the same LongLive-Plug LoRAs distilled on the base model as in the tables; no other adapter is involved. The caption sentence "uses an additional base-distilled adapter variant" was ambiguous, and we removed it from both captions.

W3 and Q4 (rows 1–2): No local native 40-step row for Tab. 2; Tabs. 1/2 lack the transferred coupled CFG+step LoRA.

  • Native 40-step row for Tab. 2. The native reference in Tab. 2 is the official PAI-Bench-C leaderboard entry, which is already reported for 40-step native inference under the official protocol, so a local rerun would duplicate it. The new Wan2.1-14B and H3 tables include locally run native rows (50-step + CFG 6; 40-step).
  • Coupled CFG+step LoRA. Tabs. 1 and 2 report video-quality metrics, whereas a coupled and a separately weighted adapter differ mainly in text alignment, i.e., in how guidance can be controlled. At the default operating point both give similar video quality, so a coupled row in these tables would not reveal the difference, and we did not add it. The relevant ablation is Fig. 6: the coupled LoRA alone does not provide control over text guidance, and scaling it to strengthen guidance degrades video quality, whereas our separately weighted CFG LoRA provides both adjustable guidance and preserved four-step quality.

W4, Q2 and Q4 (row 3): Where does the long-context adapter come from and how was it trained? Add the official ReWorld LoRA and a four-step few-step-only row to Tab. 3.

  • Provenance. ReWorld's official 4-step LoRA is our +Long adapter; the file and curve names in the supplement (official, reworld) refer to this one file. Adding "official ReWorld LoRA" as a separate Tab. 3 row would duplicate +Long. The same file is applied unchanged to Matrix-Game, with no Matrix-Game training.
  • Training. The long-context LoRA is trained on a frozen causal AR Wan base with LongLive's Streaming Long Tuning configuration and training prompts, for 2,200 iterations at rank 128 (Sec. 3.4 and App. A, marked in blue).
  • Few-step-only row. Added to Tab. 3 (blue rows): ReWorld 73.05 vs. 75.77 for +Long (base 73.51); Matrix-Game 83.12 vs. 84.34 (native 83.73). With 3 seeds per scene, +Long is ahead by 1.19 on average, in 9/10 scenes. The long-rollout gain therefore comes from the long-context LoRA.

Q5: Per-model results for Wan2.1-14B and H3 targets (one quality and one task-fidelity metric; native, naive four-step, transferred).

Native / naive four-step / ours:

  • Wan2.1-Fun-14B-Control (Tab. 6, 600 cases): Aesthetic / Imaging / DOVER 0.507 / 0.732 / 10.29, 0.439 / 0.669 / 7.46, 0.505 / 0.734 / 9.90 (retrained task-specific distillation 0.512 / 0.710 / 9.78).

Tab. 6 (new App. F.1): Wan2.1-Fun-14B-Control (depth), 600 PAI-Bench-C cases, 480p, 81 frames.

MethodAesthetic ↑Imaging ↑DOVER ↑NFE
Native 50 step + CFG 60.5070.73210.29100
Naive 4 step0.4390.6697.464
ControlNet-specific distillation (retrained)0.5120.7109.784
LongLive-Plug (λ_step=1, λ_cfg=1)0.5050.7349.904
  • H3-Fun-ControlNet-Union (Tab. 7, 600 cases): depth si-RMSE 1.608 / 1.521 / 1.489, F1 0.068 / 0.069 / 0.077, DOVER 9.04 / 9.06 / 9.78.

Tab. 7 (new App. F.1): MiniMax-H3-Fun-ControlNet-Union (depth) and adapter/sampler ablation, 600 PAI-Bench-C cases, 480p, 121 frames.

MethodSSIM ↑F1 ↑si-RMSE ↓mIoU ↑DOVER ↑LPIPS ↑NFE
Native 40 step0.6150.0681.6080.5569.04–40
Naive 4 step0.6520.0691.5210.5739.060.4584
LongLive-Plug adapter + Euler update (ablation)0.6240.0731.5510.5649.620.4964
LongLive-Plug (re-noising sampler)0.6230.0771.4890.5709.780.5144
  • SolarWM-H3 (Tab. 8, 120 clips, camera following): RotErr 83.8 / 89.3 / 71.1, imaging 69.6 / 62.1 / 69.7.

Tab. 8 (new App. F.2): SolarWM-H3 (camera-controlled world model on MiniMax-H3), 120 official test clips, 158 frames.

MethodNFERotErr ↓TransErr ↓Trans. ratio (→1)LPIPS ↓Aesthetic ↑Imaging ↑
Native (30 points)2983.82.721.0300.3720.52969.6
Naive 4 step489.33.101.0220.3940.49462.1
SolarWM official distillation (Stage 2)4/chunk107.73.580.8990.4160.53171.1
LongLive-Plug (4 step, no SolarWM training)471.12.531.0350.3470.54069.7
  • Wan-Alpha (Wan2.1-14B) (Tab. 9): alpha IoU 0.857 / 0.540 / 0.838.

Tab. 9 (new App. F.3), Wan-Alpha (Wan2.1-14B), RGBA generation, 40 prompts.

MethodNFEMatte IoU ↑Matte MAD ↓CLIP-T ↑Imaging ↑
Native (50 steps + CFG)1000.8573.8026.310.507
Naive 4-step80.54010.6725.300.286
LongLive-Plug (transferred)40.8385.2927.130.548

W5 and Q6: Quantitative curve of text alignment and quality vs. λ_cfg against native CFG, to support "near-linear"; Fig. 6A/6B use different coupled checkpoints.

  • Guidance sweep (App. F.4, marked in blue). On the full SCOPE CrossFPS test set (1,378 clips), for the CFG LoRA alone and composed with the few-step LoRA, Tab. 10 shows that text alignment increases monotonically with λ_cfg, and compares it with native CFG at the nominally matched scale w = 1 + 4λ_cfg. Native CFG is also monotone; the CFG LoRA follows the same trend with a weaker response (CLIP 19.30 vs. 20.54 at λ_cfg = 1, i.e., w = 5), and λ_cfg = 2 matches the FVD of native CFG 5 (379.9 vs. 382.0) with one pass per step instead of two. We therefore describe λ_cfg as a monotone guidance control and no longer claim it is near-linear (Introduction, marked in blue). In the four-step setting, a large λ_cfg trades distribution fidelity for alignment (FVD 486.8 → 649.6), which we state as a limitation.

Tab. 10 (new App. F.4): CLIP text alignment (↑) vs. λ_cfg on SCOPE, 1,378 CrossFPS test clips. λ_cfg = 0 removes the CFG LoRA; native CFG uses two forward passes per step at the matched scale w = 1 + 4λ_cfg.

Settingλ_cfg=00.51235
CFG LoRA (30 steps)18.0318.5519.3020.3720.91–
Native CFG (30 steps, w = 1+4λ_cfg)18.0319.6420.5421.4521.89–
Few-step + CFG LoRA (4 steps)22.22–22.3822.5022.5622.57
  • Fig. 6A/6B. Both panels use the same coupled checkpoint, prompt, seed and inputs; panel B varies only that checkpoint's weight. We revised the caption to "(B) uses different checkpoint weights" (marked in blue).

Q7: Compare with public Wan step/CFG-distilled releases, AGD-style adapters and a short target fine-tune from the transferred LoRA; report wall-clock latency; analyze when transfer fails, e.g., as a function of weight drift from the base.

  • lightx2v StepDistill-CfgDistill LoRA: compared in new Fig. 22 (see W1).
  • Wall-clock latency. On Wan2.1-Fun-14B-Control (480p, 81 frames, one H100), generating one video takes 411 s with native 50-step + CFG inference (100 NFE) and 19 s with our four-step transfer (4 NFE), a 21× speed-up.
  • When transfer fails. Transfer breaks when the adapter is used outside the regime it was distilled in: a different solver (TalkVerse UniPC: Sync-C 4.50 vs. 5.07), long AR rollouts without the long-context LoRA (Tab. 3), or a large CFG-LoRA weight in the four-step composition (FVD 486.8 → 649.6 at λ_cfg = 5).
Reviewer_55X6OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The main motivation of this work is to train reusable distillation LoRA adapters on a base video model and then transfer them to compatible downstream models, without retraining for each target model. It includes three LoRAs: CFG distillation, few-step distillation, and distillation that corrects accumulated error in long videos. The paper shows that the distilled LoRAs can be deployed on 54 downstream models across three backbone families. It reports quantitative comparisons on SCOPE and the depth-conditioned Wan2.2-Fun, together with long-sequence generation experiments on ReWorld and Matrix-Game 3.0, and thereby demonstrates that the method is effective.

Critical strengths

Repeatedly distilling a separate adapter for every downstream model is a real pain point, and this paper addresses it reasonably well. Starting from three common distillation needs, the paper trains three general-purpose LoRAs and thoroughly checks that they transfer to downstream models and remain effective. They are also competitive with task-specific distillation. The list of 54 models further shows a fairly broad range of applicability, although the models are not validated to the same degree.

Critical weaknesses
  1. The novelty is limited, and the increment over existing practice is thin. At its core, the work trains three LoRAs with different functions on top of a base model. Reusing a distillation LoRA on fine-tuned models of the same base is already the premise of LCM-LoRA, yet the paper passes over this in a single sentence. This work mainly verifies that point further. It neither explains why the transfer works, nor makes clear where it fails.

  2. The current experiments do not clearly separate the contribution of each capability. The few-step LoRA is itself trained with guided teacher supervision, so adding a CFG-only adapter shows that there is one more control knob. It does not show that guidance and the reduction in sampling steps have been factored apart. The long-context experiments have a similar problem. Table 3 and Figure 7 do not quantitatively report a few-step-adapter-only baseline under the corresponding setting, so it is hard to tell how much of the numerical gain comes from the long-context LoRA itself.

  3. Some of the metrics do not reflect the capabilities of video generation itself, and the evidence for the main conclusions is qualitative. Most of the reported metrics, including FVD, Dover, and LIPSP, are visual-quality metrics. Attractive frames alone cannot confirm that motion control, audio synchronization, or alpha-channel capability are preserved, and the paper lacks quantitative metrics for these. In addition, although the method is tested on 54 models, only two models from a single base have per-model numbers. Wan2.1-14B and H3 have none.

Questions for authors
  1. Scope and conditions of the transfer. By what criteria was each of the 54 models judged a successful transfer? How were the adapter weights chosen for each task? Please summarize the degradations already observed, especially those that involve a newly added conditioning branch or output channel, and distinguish “visually confirmed to run” from “task capability actually measured.” This does not ask for a new evaluation of all 54 models.

  2. Experimental details. Does Figure 6 use the same coupled checkpoint, and the same prompt, random seed, and conditioning inputs, to compare overall scaling against additionally inserting the CFG-only adapter? Why does Table 3 omit ReWorld-distill? How many rollouts were evaluated in the reported results, and what is the variance? If the few-step adapter alone already matches the +Long result, the long-context conclusion does not hold.

  3. Suggested experiments. On a subset that covers the three families, please report the native multi-step-plus-CFG numbers and, where feasible, the per-target distillation numbers. An open-source “steps + CFG” distillation LoRA for Wan2.1-14B is already in circulation and is routinely loaded onto fine-tuned Wan models. A comparison against unpublished code is not required, but if you already have this comparison, it would make the claim that the contribution goes beyond packaging more convincing.

6 comments

We thank the reviewer for the detailed questions. Most of the requested experiments are now done; the new results are in App. F (marked in blue).

W1: Novelty is limited (LCM-LoRA already reuses a distillation LoRA on fine-tunes); the paper neither explains why transfer works nor where it fails.

We agree that LCM-LoRA is the key precedent and have expanded Sec. 2.2 (marked in blue): LCM-LoRA targets image models whose downstream variants differ mainly in style, and it has no separate CFG branch for adjusting guidance. Our contribution is to show, and now measure, how far reuse extends for video generation:

  1. Targets that change the model's interface. Downstream models add conditioning branches or output channels (depth/camera ControlNets, audio-driven animation, an RGBA alpha decoder) or are action-conditioned AR world models. We now measure task capability after transfer, not only visual quality (Tabs. 7–9, reproduced in our reply to W3).
  2. Separately weighted capabilities. A separate CFG LoRA keeps guidance adjustable after transfer, on top of the few-step LoRA, and a long-context LoRA targets AR rollouts. They are composed by weight addition, with no joint retraining.
  3. Scale and cost. 54 models in three families, now 10 with numbers. Against per-target distillation on the same backbone, transfer is on par in visual quality on Wan2.1-14B (Aesthetic / Imaging / DOVER 0.505 / 0.734 / 9.90 vs. 0.512 / 0.710 / 9.78) and better in camera control on SolarWM-H3 (RotErr 71.1 vs. 107.7). The cost is one base distillation per family (≈80 H100-h on Wan2.2-5B vs. 376.8 additional H100-h for four per-target runs; ≈860–890 H100-h on Wan2.1-14B shared by 24 models vs. 684.6 H100-h for one per-target run).

Why transfer works. Our recipe distills acceleration once on the base model for reuse across tasks. We hypothesize that its transferability is related to reduced entanglement with task-specific content. On Wan2.2-TI2V-5B, we analyze three rank-128, four-step LoRAs: (i) our recipe with diverse prompts; (ii) a prompt ablation on the same base model using a single Minecraft text prompt; and (iii) per-task distillation on the Minecraft-specialized model using the same prompt as (ii). We measure cross-attention principal-subspace overlap with the Minecraft model’s weight difference from the base model as a shared content-related proxy. Our recipe has lower overlap (0.114) and good qualitative transfer to Minecraft. The prompt ablation has higher overlap (0.197) and visible content bias. Per-task distillation yields the highest overlap (0.233) and visibly degraded transfer back to the base model. This correspondence supports lower content entanglement as a possible explanation for the transferability of our recipe.

Where it fails. (1) A sampler that differs from the one used in distillation (TalkVerse with UniPC: Sync-C 4.50 vs. 5.07; H3 needs its re-noising sampler for depth fidelity). (2) Long AR rollouts without the long-context LoRA (Tab. 3). (3) A large CFG-LoRA weight in the four-step composition (FVD 486.8 → 649.6 at λ_cfg = 5). (4) Residual gaps, e.g. TalkVerse FID/FVD 24.4/128 vs. 21.8/112 native.

W2: The experiments do not separate guidance from step reduction; Tab. 3 / Fig. 7 lack a few-step-only baseline.

Guidance vs. few-step. We agree that the few-step LoRA, distilled from a guided teacher, already carries guidance, and we do not claim the two are factored apart. Our claim is narrower: with the few-step weight fixed, the separately weighted CFG LoRA is an adjustable knob that survives transfer. On the full SCOPE test set, text alignment increases monotonically with λ_cfg, both with and without the few-step LoRA (Tab. 10, new App. F.4). A coupled LoRA offers only a global weight, and globally scaling it collapses (Fig. 6B). The public lightx2v step+CFG LoRA has the same limitation: below weight 1 its step distillation is weakened too and videos turn hazy and blurry, whereas with our few-step LoRA fixed, λ_cfg ∈ [0, 1] keeps four-step quality and layout while gradually strengthening the prompted attributes (new Fig. 22). We replaced "decoupled" with "separately weighted" throughout (marked in blue).

Tab. 10 (new App. F.4): CLIP text alignment (↑) vs. λ_cfg on SCOPE, 1,378 CrossFPS test clips. λ_cfg = 0 removes the CFG LoRA; native CFG uses two forward passes per step at the matched scale w = 1 + 4λ_cfg.

Settingλ_cfg=00.51235
CFG LoRA (30 steps)18.0318.5519.3020.3720.91–
Native CFG (30 steps, w = 1+4λ_cfg)18.0319.6420.5421.4521.89–
Few-step + CFG LoRA (4 steps)22.22–22.3822.5022.5622.57

Long-context. Tab. 3 now includes a few-step-only row on both models (blue rows). On Matrix-Game at 62.18 s, few-step only reaches 83.12, below native (83.73), task-specific distillation (84.30) and +Long (84.34). On ReWorld at 64 s it reaches 73.05, versus 73.51 (base) and 75.77 (+Long). With three noise seeds per scene, +Long is ahead by 1.19 on average, in 9/10 scenes, and at all four rollout lengths. The long-rollout gain therefore comes from the long-context LoRA.

W3: Metrics are mostly visual quality; no quantitative evidence that motion control, audio sync or alpha are preserved; no per-model numbers for Wan2.1-14B and H3.

New task metrics (Tab. 9, new App. F.3), 4 steps, no CFG, no downstream training:

  • Wan-Alpha (40 prompts). Alpha-matte IoU against a BiRefNet matte of the generated RGB: 0.838 (ours) vs. 0.857 (native 50-step + CFG) vs. 0.540 (naive four-step).

Tab. 9 (new App. F.3), Wan-Alpha (Wan2.1-14B), RGBA generation, 40 prompts.

MethodNFEMatte IoU ↑Matte MAD ↓CLIP-T ↑Imaging ↑
Native (50 steps + CFG)1000.8573.8026.310.507
Naive 4-step80.54010.6725.300.286
LongLive-Plug (transferred)40.8385.2927.130.548
  • Fun-Control-Camera (64 image–trajectory pairs, VGGT poses). Translation error 0.82 vs. 1.16 (native) and 1.40 (naive); direction accuracy 100% / 100% / 97.9%.

Tab. 9 (new App. F.3), Fun-5B-Control-Camera (Wan2.2-5B), 64 image–trajectory pairs.

MethodNFERotErr ↓TransErr ↓Dir. acc ↑Imaging ↑
Native (50 steps + CFG)10026.51.16100.0%0.739
Naive 4-step826.21.4097.9%0.573
LongLive-Plug (transferred)420.90.82100.0%0.741
Task-specific DMD434.00.91100.0%0.742
  • TalkVerse-5B (30 HDTF clips). Sync-C 5.07 vs. 5.20 native vs. 2.59 for naive four-step; FID/FVD 24.4/128 vs. 21.8/112. This holds with the adapter's re-noising sampler; with TalkVerse's default UniPC solver transfer is only partial (Sync-C 4.50), which we report as a limitation.

Tab. 9 (new App. F.3), TalkVerse-5B (Wan2.2-5B), audio-driven, 30 HDTF clips.

MethodNFESync-C ↑Sync-D ↓FID ↓FVD ↓
Native (50 steps + CFG)1005.209.6821.8112.0
Naive 4-step (UniPC)84.6410.1829.1148.8
LongLive-Plug (UniPC)44.5010.4629.2166.0
Naive 4-step (re-noise)85.089.7126.2152.3
LongLive-Plug (re-noise)45.079.9224.4128.0
Task-specific DMD (re-noise)45.219.6233.0712.5
  • Wan2.1-14B and H3. Tab. 6 (Wan2.1-Fun-14B-Control: Aesthetic / Imaging / DOVER 0.505 / 0.734 / 9.90 vs. 0.439 / 0.669 / 7.46 naive), Tab. 7 (H3 ControlNet-Union: best F1, si-RMSE and DOVER of all rows) and Tab. 8 (SolarWM-H3 camera control: RotErr 71.1 vs. 83.8 native).

Tab. 6 (new App. F.1): Wan2.1-Fun-14B-Control (depth), 600 PAI-Bench-C cases, 480p, 81 frames.

MethodAesthetic ↑Imaging ↑DOVER ↑NFE
Native 50 step + CFG 60.5070.73210.29100
Naive 4 step0.4390.6697.464
ControlNet-specific distillation (retrained)0.5120.7109.784
LongLive-Plug (λ_step=1, λ_cfg=1)0.5050.7349.904

Tab. 7 (new App. F.1): MiniMax-H3-Fun-ControlNet-Union (depth) and adapter/sampler ablation, 600 PAI-Bench-C cases, 480p, 121 frames.

MethodSSIM ↑F1 ↑si-RMSE ↓mIoU ↑DOVER ↑LPIPS ↑NFE
Native 40 step0.6150.0681.6080.5569.04–40
Naive 4 step0.6520.0691.5210.5739.060.4584
LongLive-Plug adapter + Euler update (ablation)0.6240.0731.5510.5649.620.4964
LongLive-Plug (re-noising sampler)0.6230.0771.4890.5709.780.5144

Tab. 8 (new App. F.2): SolarWM-H3 (camera-controlled world model on MiniMax-H3), 120 official test clips, 158 frames.

MethodNFERotErr ↓TransErr ↓Trans. ratio (→1)LPIPS ↓Aesthetic ↑Imaging ↑
Native (30 points)2983.82.721.0300.3720.52969.6
Naive 4 step489.33.101.0220.3940.49462.1
SolarWM official distillation (Stage 2)4/chunk107.73.580.8990.4160.53171.1
LongLive-Plug (4 step, no SolarWM training)471.12.531.0350.3470.54069.7
Reviewer_ykMpOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies whether the acceleration and long-video capabilities that are usually distilled separately for every specialized video model can instead be distilled once on a base model and reused. The authors train three functional LoRAs on a frozen base model — a CFG-only LoRA that folds two-pass guidance into one conditional forward, a few-step LoRA built on DMD2 for four-step sampling, and a long-context LoRA for error correction in causal autoregressive rollouts — and then attach them to compatible downstream models while keeping the downstream task weights. The key design is decoupling: the CFG LoRA is trained and weighted separately from the few-step LoRA, so its inference weight acts as a guidance dial without disturbing the four-step behavior. The authors also report two transfer-oriented findings, that larger adapter rank and broader prompt coverage improve transfer even when source fit looks fine. They verify training-free deployment on 54 downstream models across three backbone families and eight task categories, and report quantitative comparisons on SCOPE and Wan2.2-Fun-5B-Control plus long-rollout results on ReWorld and Matrix-Game 3.0.

Critical strengths
  1. The problem is practical and the framing is clean.

Specialized video models keep multiplying, and each one currently repeats its own distillation run; distilling the capability once per backbone family and reusing it is a sensible way to cut that repeated cost, and the paper makes the contrast explicit in Figure 1 and in the cost accounting of Figure 3.

  1. The decoupled CFG design targets a real limitation of joint distillation.

Existing few-step methods bake guidance in at a fixed scale, which is awkward when downstream tasks want different guidance strengths; separating a CFG-only LoRA and using its weight as a dial lets the authors keep the few-step weight fixed and still adjust text guidance, and they show both the working case and the failure case of globally scaling a coupled LoRA.

  1. The transfer study goes beyond a single benchmark.

The paper reports two transfer-oriented ablations (rank and prompt diversity) that isolate design choices under matched protocols, and it separates three levels of evidence — the two main quantitative comparisons, the long-context quantitative results, and the broad 54-model coverage — rather than claiming uniform validation everywhere.

Critical weaknesses
  1. Make the experimental setup and its constraints clearer.

In the appendix, the authors note that the H3 comparison changes both the adapter and the sampler from E4 to S4 and its effect. I suggest the authors could choose to add a short paragraph in the main text to clarify what the experiment demonstrates and highlight any important details, preventing readers from misunderstanding.

  1. Discussion of the metric trade-off.

In Table 1, the proposed method perform better on FVD and Photo, but worse on Smooth and Depth. In Table 2, its results also perform different across tasks. I suggest the authors could add more discussion of why the model win or lose on particular tasks and metrics — are there any special characteristics of these tasks which may have effect on the performance of the model?

  1. Discussion of why the LoRA trained on the foundation model could be used directly on downstream models.

The whole framework rests on an implicit assumption: that some of the few-step / CFG capability distilled on the base model is task-agnostic, and can therefore be transferred to specialized models. This assumption is reasonable, but the validations are most based on "the results are not bad", not on why this could happen. I suggust the authors could add more quantitative or qualitative experiments and discuss this point.

Questions for authors

See the Weakness.

2 comments

We thank the reviewer for the positive assessment and the concrete suggestions.

W1: The H3 comparison changes both the adapter and the sampler (E4 → S4); clarify what the experiment demonstrates.

We added a quantitative ablation that separates the two changes (Tab. 7, new App. F.1) and state its result in the H3 transfer section (App. C.3), both marked in blue. In the revision we no longer use the E4/S4 labels and write "naive four-step sampling" and "LongLive-Plug" instead.

  • What differs. Naive four-step sampling uses no adapter. LongLive-Plug adds our H3 few-step LoRA and its re-noising sampler (x0 prediction followed by fresh re-noising, matching its DMD training). In the appendix examples, naive four-step sampling used each downstream model's own scheduler; the ablation fixes the σ grid to remove this difference.
  • Ablation (600 PAI-Bench-C cases, same σ grid and initial noise). A row "adapter + Euler" changes only the adapter: DOVER 9.06 → 9.62, F1 0.069 → 0.073, but si-RMSE 1.521 → 1.551. Switching only the sampler then gives LongLive-Plug: si-RMSE 1.551 → 1.489, F1 0.073 → 0.077, DOVER 9.62 → 9.78.

Tab. 7 (new App. F.1): MiniMax-H3-Fun-ControlNet-Union (depth) and adapter/sampler ablation, 600 PAI-Bench-C cases, 480p, 121 frames.

MethodSSIM ↑F1 ↑si-RMSE ↓mIoU ↑DOVER ↑LPIPS ↑NFE
Native 40 step0.6150.0681.6080.5569.04–40
Naive 4 step0.6520.0691.5210.5739.060.4584
LongLive-Plug adapter + Euler update (ablation)0.6240.0731.5510.5649.620.4964
LongLive-Plug (re-noising sampler)0.6230.0771.4890.5709.780.5144
  • What it demonstrates. The adapter brings the visual-quality gain; the training-matched sampler is needed to recover control fidelity. The appendix examples therefore evaluate the full deployment recipe (adapter + its sampler), not the isolated effect of adding a LoRA.

W2: Explain the metric trade-offs (Tab. 1: better FVD/Photo, worse Smooth/Depth; Tab. 2: mixed across tasks).

  • Comparable quality at no per-target training cost. Task-specific distillation is trained for each downstream model (Fig. 3: 376.8 additional H100-h for four Wan2.2-5B tasks, plus task-specific data such as 5,000 depth-conditioned videos for Tab. 2), whereas our adapters are distilled once on the base model and transferred without any downstream training or data. Matching task-specific distillation in quality is therefore already a substantial practical advantage; the per-metric differences below are small in comparison.
  • Tab. 1. The losses to SCOPE-specific distillation are small: Smooth 2.399 vs. 2.415 (−0.7%) and Depth 1.230 vs. 1.203 (+2.2%). The wins are larger: FVD 478.7 vs. 502.1 (−4.7%) and Photo 4.246 vs. 5.695 (−25.4%). Our Depth error is also lower than native 30-step inference (1.287).
  • Tab. 2, si-RMSE (1.641 vs. 1.515). The ControlNet-specific distillation is trained on 5,000 depth-conditioned videos, so it directly optimizes depth adherence; our adapter has never seen depth data. It still improves si-RMSE over naive four-step sampling (2.135) and over the leaderboard score of the original model (1.819).
  • Task characteristics. SSIM to a blurred ground truth and smoothness/consistency scores reward low-contrast, static outputs. On H3, naive four-step sampling therefore keeps the highest SSIM/mIoU (0.652/0.573) despite lower DOVER (9.06 vs. 9.78; Tab. 7), and in Tab. 3 the ReWorld rows without the long-context LoRA collapse into near-uniform frames (see the late frames in Fig. 19) that score high on background consistency. For this reason the new Wan2.1-14B table reports visual quality (Tab. 6).

Tab. 6 (new App. F.1): Wan2.1-Fun-14B-Control (depth), 600 PAI-Bench-C cases, 480p, 81 frames.

MethodAesthetic ↑Imaging ↑DOVER ↑NFE
Native 50 step + CFG 60.5070.73210.29100
Naive 4 step0.4390.6697.464
ControlNet-specific distillation (retrained)0.5120.7109.784
LongLive-Plug (λ_step=1, λ_cfg=1)0.5050.7349.904

W3: Why can the LoRA trained on the base model be used directly on downstream models? Validation is mostly "results are not bad".

To investigate why our recipe transfers, we hypothesize that reusable acceleration updates are less entangled with task-specific content. We examine this through weight-space measurements and qualitative transfer comparisons.

We analyze three rank-128 LoRAs targeting four-step generation within the Wan2.2-TI2V-5B family: (i) our recipe, distilled on the base model with diverse prompts; (ii) a prompt ablation, distilled on the same base model with a single Minecraft text prompt; and (iii) per-task distillation, performed on the Minecraft-specialized model using the same prompt as (ii). For all three, we measure cross-attention principal-subspace overlap against the same reference: the Minecraft model’s weight difference from the base model, used as a content-related proxy.

The overlaps are 0.114, 0.197, and 0.233, respectively. Our recipe preserves visual quality when transferred to Minecraft. The prompt ablation introduces visible content bias, while the per-task LoRA produces blurred outputs when transferred back to the base model.

These geometric measurements and qualitative observations are consistent with our hypothesis that reduced content entanglement contributes to transferability, providing a possible explanation for why our recipe produces reusable acceleration updates.

Beyond this analysis, the task signal of downstream models with new branches or output channels survives transfer (Tabs. 8–9, new App. F.2–F.3; e.g., alpha IoU 0.838 vs. 0.857 native, Sync-C 5.07 vs. 5.20, camera translation error 0.82 vs. 1.16 native).

Tab. 8 (new App. F.2): SolarWM-H3 (camera-controlled world model on MiniMax-H3), 120 official test clips, 158 frames.

MethodNFERotErr ↓TransErr ↓Trans. ratio (→1)LPIPS ↓Aesthetic ↑Imaging ↑
Native (30 points)2983.82.721.0300.3720.52969.6
Naive 4 step489.33.101.0220.3940.49462.1
SolarWM official distillation (Stage 2)4/chunk107.73.580.8990.4160.53171.1
LongLive-Plug (4 step, no SolarWM training)471.12.531.0350.3470.54069.7

Tab. 9 (new App. F.3), Wan-Alpha (Wan2.1-14B), RGBA generation, 40 prompts.

MethodNFEMatte IoU ↑Matte MAD ↓CLIP-T ↑Imaging ↑
Native (50 steps + CFG)1000.8573.8026.310.507
Naive 4-step80.54010.6725.300.286
LongLive-Plug (transferred)40.8385.2927.130.548

Tab. 9 (new App. F.3), Fun-5B-Control-Camera (Wan2.2-5B), 64 image–trajectory pairs.

MethodNFERotErr ↓TransErr ↓Dir. acc ↑Imaging ↑
Native (50 steps + CFG)10026.51.16100.0%0.739
Naive 4-step826.21.4097.9%0.573
LongLive-Plug (transferred)420.90.82100.0%0.741
Task-specific DMD434.00.91100.0%0.742

Tab. 9 (new App. F.3), TalkVerse-5B (Wan2.2-5B), audio-driven, 30 HDTF clips.

MethodNFESync-C ↑Sync-D ↓FID ↓FVD ↓
Native (50 steps + CFG)1005.209.6821.8112.0
Naive 4-step (UniPC)84.6410.1829.1148.8
LongLive-Plug (UniPC)44.5010.4629.2166.0
Naive 4-step (re-noise)85.089.7126.2152.3
LongLive-Plug (re-noise)45.079.9224.4128.0
Task-specific DMD (re-noise)45.219.6233.0712.5
  • Where it fails. Transfer breaks when the adapter is used outside the regime it was distilled in: a different solver (TalkVerse with its default UniPC solver: Sync-C 4.50), long autoregressive rollouts without the long-context LoRA (Matrix-Game 83.12 < 83.73 native), or a large CFG-LoRA weight in the four-step composition (FVD 486.8 → 649.6 at λ_cfg = 5). We list these as limitations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.