acceptodds
Under review as a conference paper at ICLR 2027

DART: Distillation-Aware Reparameterization for Training-Free LoRA Reuse in Few-Step Video Diffusion Models

Abstract

Few-step distillation reduces the inference cost of image-to-video generation, but directly reusing LoRA adapters trained for long denoising trajectories can weaken their intended effects and degrade video quality. We observe that adapters with similar measured static parameter geometry can behave differently under the shortened target schedule, motivating response-aware transfer. We propose DART, a training-free reparameterization method that transports source LoRAs into aligned coordinates through a low-rank distillation bridge. Paired forward evaluations measure channel-level incremental responses under the target schedule to fit coefficients that calibrate response direction, source-relative magnitude, and timestep allocation. Coordinate transport establishes update directions, while calibration adapts their contributions, requiring neither source training videos nor backpropagation. Experiments across multiple distilled I2V models demonstrate improved generation quality and aggregate functional retention over direct reuse. Further analyses show that coordinate transport complements response calibration, with adapter-level benefits encompassing both functional preservation and reduced negative transfer.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Diffusion Models, LoRA, Few-Step Diffusion, Model Distillation, Adapter Transfer, Training-Free Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_C459OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates the reuse of LoRA from the original multi-step model in a few-step distilled video model. The authors point out that geometric compatibility in the weight space is insufficient to predict the functional performance after transfer, and propose DART. LoRA is transferred via a low-rank distillation bridge and singular subspace alignment; then, channel responses are estimated using paired forward computations to solve a regularized coefficient calibration problem. The conversion requires neither the original training video nor backpropagation, and the final output can be deployed as a fixed LoRA.

Critical strengths
  1. The motivation has practical value. Few-step distillation may compromise the reusability of existing LoRA models, and retraining requires data and computational resources. This paper proposes a transformation method to address this deployment challenge, and the motivation for its application is clear.

  2. The methods are designed to complement each other in a logical manner. Coordinate translation adjusts the available update directions, while response calibration adjusts the relative contributions of these directions.

Critical weaknesses
  1. There is no direct validation between thebridge-based coordinate transport and the deployment of the target model. Appendix D.7 analyzes the impact of residual drift but does not quantify the actual response error.

  2. Performance gains vary across different LoRAs. Table 13 shows that DART-F does not improve performance metrics on all LoRAs. For example, the FGR of LoRA 07 drops from 0.3397 (when directly reused) to 0.2780; while LoRA 01, 03, and 06 showed some improvement but remained negative. Therefore, the improvement in average metrics is insufficient to demonstrate that the method can consistently preserve performance across various LoRA variants. We recommend analyzing these cases of failure or degradation to clarify the method’s applicability boundaries.

Questions for authors

See weakness.

Reviewer_5NGwOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies reusing LoRA adapters trained on an original video diffusion model when they are attached to its distilled, few-step counterpart. It proposes DART, a two-stage training-free transfer method. The authors first examine the prior method CASA and find that its spectral-arbitration transfer produces mixed results. DART improves on this line of work by (i) projecting the distillation-induced weight change onto its leading low-rank component to form a bridge, (ii) aligning the clustered singular subspaces of the source and bridge weights and transporting the LoRA update through the resulting maps, and (iii) calibrating the per-channel coefficients of the transported LoRA via paired forward evaluations under the target schedule. Experiments on three distilled targets show that DART outperforms both direct attachment and CASA in generation quality and LoRA functional retention.

Critical strengths

Reusing existing LoRAs after distillation is a practically useful problem, especially when the original customization data are unavailable.

Critical weaknesses
  1. The writing is extremely poor and makes reviewing this paper painful. The text is full of defensive statements (e.g., the ends of the first paragraph of Sec. 2.3, of Sec. 3.2, and of Sec. 4.2): abrupt disclaimers against over-extensions that no reader would make, and recurring "X, rather than Y" constructions that spell out self-evident caveats. What is genuinely non-obvious is left unexplained: for instance, how CASA, the paper's central comparison method, actually works, or on what the "clustered singular subspace alignment" operates. Readability suffers further because the illustration of the method is repeatedly interrupted by dense listings of experimental configuration (e.g., the second paragraph of Sec. 3), much of which contributes little to understanding the analysis or the conclusions. Fig. 2 does not help either: it is crowded with symbols that cannot be interpreted from the figure alone.

  2. Insufficient discussion of the work the paper builds on. Although this partially overlaps with the previous point, I still think it is worth listing separately: the analysis in Sec. 3 relies heavily on the method introduced by CASA, yet the paper does not provide a self-contained description of that method.

  3. DART takes an "incremental improvement over CASA" route without justification. The analysis in Sec. 3.2 does not close the logical loop: it establishes that the CASA-style static method does not work, and the paper's own ablation shows that coordinate transport alone (DART-W) underperforms even direct attachment. Yet the transport stage is retained without explanation. In particular, the low-rank bridge construction, which is what actually makes transport useful, is never explained or justified in Sec. 3. Its plausibility rests solely on the post-hoc ablation differences in Sec. 5.3.

  4. The calibration stage is not lightweight, and may be even more expensive than simply retraining the LoRA on the distilled model. The calibration requires 528 forward evaluations per adapter and takes 226.5 minutes on average (Table 17). This is a substantial overhead for a method whose motivation is to avoid retraining.

Questions for authors

See "Critical Weaknesses".

Reviewer_kYJGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies training-free transfer of LoRA adapters from video diffusion models with long denoising trajectories to few-step distilled models. It proposes DART, a two-stage approach that:

  • uses a low-rank distillation bridge and singular-subspace alignment to transport the source LoRA into target-compatible parameter coordinates;
  • decomposes the LoRA into rank-one channels, measures their incremental responses through paired forward evaluations under the target denoising schedule, and calibrates their coefficients by solving a regularized linear system.
Critical strengths
  • The problem is practically important, as reusing existing LoRA adapters without access to their original training data or retraining can substantially reduce adaptation costs.
  • The proposed method relies only on forward evaluations of the target model and a closed-form linear solve, requiring neither the original LoRA training videos nor backpropagation. This makes the approach simple and effective.
Critical weaknesses
  • The evaluation includes only six LoRA adapters, limiting the diversity and representativeness of the experiments.
  • The method lacks direct cross-target generalizability. Each distilled model and denoising schedule requires separate bridge construction and response calibration, and a converted LoRA cannot be directly transferred to unseen models.

Minor concern

  • The robustness analysis is relatively toy-like, relying on synthetic inputs with random Gaussian latents, empty text, and no reference images. It therefore provides limited evidence of robustness under realistic prompts and reference-image conditions.
Questions for authors

See Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.