- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Reveris, an efficient visual effect transfer method based on video diffusion models. It aims to transfer dynamic visual effects from a reference video to a target scene while preserving the target subject and scene structure and preventing leakage of irrelevant reference content. The authors construct a large-scale dataset containing 49,194 target videos and 350 types of visual effects, and design a four-stage training framework. First, adaptive reference conditioning modeling is adopted to extract effect information. Second, modulation calibration is used to control effect strength and spatial coverage. Third, multi-source conditional guidance is incorporated to boost generation quality. Finally, distribution-matching distillation compresses diffusion sampling from 50 steps to only 4 steps. In addition, the authors propose a multi-dimensional evaluation system combining VLMs and evidence-based validation. Experiments show that Reveris outperforms existing methods overall in visual effect transfer quality, subject preservation and reference content isolation, while achieving approximately 9.4× inference speedup.
Critical strengths
- The research problem carries clear practical significance. This work focuses on reference-video-driven visual effect transfer. Its goal is to achieve dynamic effect transfer across different subjects and scenes while preserving the original subject and scene structure. Existing methods suffer from issues such as reference content leakage, destruction of subject structure, and low generation efficiency. This paper proposes a relatively systematic solution addressing these challenges.
- It presents a complete and targeted four-stage training framework. Reference Adaptation improves the accuracy of reference effect extraction via content matching and temporal variation modeling. Modulation Calibration introduces signed modulation magnitudes and target-conditional gating mechanisms, enabling visual effect transfer to adapt to diverse target scenes. Multi-source Guidance leverages different conditional information comprehensively to enhance the controllability of generation outputs. Few-step Distillation compresses diffusion sampling into only 4 steps, boosting inference efficiency while maintaining generation quality.
- A large-scale dataset with rich visual effect types is constructed. The dataset contains 49,194 target videos covering 350 kinds of visual effects across multiple effect categories and diverse scenes. Built through automated generation, filtering and validation pipelines, the dataset provides data support for training reference-conditioned visual effect transfer models. Dataset construction itself constitutes an important contribution of this paper.
- The experiments are comprehensive, including extensive comparisons and ablation analyses. Quantitative and qualitative comparisons are conducted against VAP, VFXMaster, ReVFX and their variants. Ablation studies are performed on key components including reference modulation, multi-source guidance and distillation. The appendix further analyzes the impacts of different guidance weights, evaluator selections and reference conditions, helping clarify the function of each module.
- The method exhibits notable advantages in generation efficiency. Few-step distillation reduces sampling steps from 50 to 4, achieving approximately 9.4× inference speedup. Compared with conventional methods requiring numerous diffusion steps, this improvement enhances usability for practical video editing tasks.
- A relatively comprehensive automated evaluation system is proposed. It considers multiple dimensions beyond visual quality, including effect fidelity, subject preservation, temporal consistency and reference content leakage. An evaluation mechanism combining VLM judgments and evidence-based validation is introduced, attempting to reduce uncertainty introduced by relying solely on subjective VLM scores.
Critical weaknesses
- The core method exhibits relatively limited novelty and mainly consists of the combination and optimization of existing techniques. The four-stage training framework adopts reference conditioning adaptation, feature modulation, multi-condition CFG, distribution-matching distillation and other techniques. Most of these underlying ideas have been explored in prior works. Although each module is adapted for visual effect transfer, the paper does not sufficiently elaborate the fundamental methodological innovations of these designs compared with prior arts. In particular, it remains to be clarified whether the main performance gains stem from the novel reference conditioning modeling mechanism, or from the multi-stage training strategy and larger-scale dataset.
- The evaluation over-relies on automated VLM assessment and lacks human subjective evaluation. The paper mainly uses GPT-5.6 and other VLMs to evaluate generation outputs. Despite the evidence-based validation mechanism, evaluation results may still be affected by model bias, prompt design and visual understanding capabilities. The naturalness, visual appeal and perceptual realism of video visual effects are highly subjective and difficult to fully measure solely via VLMs. The absence of human evaluation constitutes a notable limitation in experimental validation, especially given the relatively modest quantitative improvements.
- The dataset is built primarily via an automated synthesis pipeline; its authenticity and diversity remain to be verified. The authors construct large-scale training data using generative models and automatic filtering pipelines, yet synthesized visual effects may carry shared generative biases. Although there are many effect categories, it is unclear whether the data possesses sufficient diversity in visual styles, motion patterns and physical interactions. The paper lacks systematic generalization experiments on real-captured visual effects, complex physical interactions and out-of-distribution scenarios, making it hard to fully assess its real-world applicability.
- There is a lack of systematic analysis of failure cases and the applicability boundary of the method. The paper demonstrates numerous successful examples but provides limited discussion of failure scenarios. For instance, it remains unclear whether the method works stably for visual effects involving complex occlusions, drastic camera motion, multi-subject interactions, precise spatial constraints or sophisticated physical changes. The lack of systematic failure-case analysis hinders a comprehensive understanding of the method’s limitations and directions for future improvements.
- Minor formatting issues exist. Several subtle formatting problems are present in the manuscript. For example, the vertical spacing between the captions of some tables and the top borders of the tables is noticeably too small, which deviates from the standard layout specified in the official ICLR template. The authors are advised to further check the formatting throughout the paper to ensure that details such as table captions and spacing comply with the requirements of the official ICLR template.
Questions for authors
- Regarding methodological novelty: The four-stage framework of this paper combines multiple existing techniques. Could the authors further clarify the key innovations of Reference Adaptation and Modulation Calibration compared with existing reference conditioning modeling and feature modulation methods? Could the authors further specify which performance gains mainly come from reference conditioning adaptation, and which come from multi-source guidance or distillation? Under a fixed data scale and training computation budget, do the relative contributions of each module still hold?
- Regarding dataset quality and generalization ability: Training data is mainly constructed via an automated generation pipeline. How do the authors ensure that the dataset does not inherit systematic biases from the base generative model? For reference visual effects from real-captured videos that exhibit a clear distribution shift from the training data, can Reveris maintain comparable transfer quality?
- Regarding evaluation reliability: The current evaluation relies primarily on VLMs. Have the authors considered adding human user studies and analyzing the correlation between human preferences and automatic evaluation metrics? Especially for small performance gaps among different methods, can VLM scores accurately reflect human perceptual experience?
- Regarding the applicability boundary of the method: How does Reveris perform on visual effects with complex occlusions, multi-subject interactions, drastic motion, or scenarios requiring precise physical interactions? Can representative failure cases be supplemented, together with analysis of the primary causes of failure?
6 comments
Response to Q1, Subquestion 2: Sources of Performance Gains
The following ablation examines where the teacher’s quality gains come from. All four configurations are evaluated before distillation.
| Training configuration | EF ↑ | SP ↑ | TC ↑ | RD ↑ | VQ ↑ | EL ↑ |
|---|---|---|---|---|---|---|
| Stage I only | 2.927 | 3.363 | 3.343 | 4.972 | 3.167 | 3.572 |
| Stage I + II | 3.023 | 3.474 | 3.263 | 4.971 | 3.123 | 3.650 |
| Stage I + III, without adaptive modulation | 3.022 | 3.484 | 3.260 | 4.974 | 3.111 | 3.499 |
| Full teacher: Stage I + II + III | 3.073 | 3.628 | 3.407 | 4.977 | 3.249 | 3.734 |
Reference Adaptation establishes the reference-conditioned model represented by Stage I. Adding Modulation Calibration in Stage II raises EF from 2.927 to 3.023, SP from 3.363 to 3.474, and EL from 3.572 to 3.650. These improvements in effect fidelity, source preservation, and edit locality are obtained through reference-conditioning training, before multi-source guidance or DMD.
Multi-source guidance training further improves how the transferred effect is balanced with the target content. Compared with Stage I + II, the full teacher increases SP from 3.474 to 3.628, TC from 3.263 to 3.407, and VQ from 3.123 to 3.249, while EF rises from 3.023 to 3.073. The Stage I + III model also improves EF and SP over Stage I, but the complete teacher achieves the highest scores across all six metrics. This shows the benefit of combining calibrated reference conditioning with multi-source guidance, which improves both effect fidelity and output quality.
DMD primarily contributes inference efficiency. The quality improvements above are already present in the 50-step teacher. Distillation reduces inference to four steps and provides a measured 22.8× speedup (Table 2 in the manuscript). It retains comparable effect fidelity, with EF of 3.110 versus the teacher’s 3.073, while further improving SP by 0.072 and VQ by 0.143 (Table 1).
For the continued teacher-training comparisons, we fixed the training data and batch size and matched the total number of updates after the same saved Stage I checkpoint, counting both Stage II and Stage III toward the full teacher’s budget. The Stage I-only row is the starting-point baseline. The full teacher’s advantage over both Stage I + II and Stage I + III therefore persists under this matched data and update budget.
Response to Q2: Dataset Quality and Generalization Ability
We evaluated our final four-step model on a manually curated set of real-captured effect references from Kinetics-700-2020, paired with source images from our dataset, without additional fine-tuning. We also evaluated it on 500 cases from the external EffectData dataset (Appendix D).
| Evaluation set | EF ↑ | SP ↑ | TC ↑ | RD ↑ | VQ ↑ | EL ↑ |
|---|---|---|---|---|---|---|
| Real-captured references | 2.979 | 4.083 | 3.625 | 4.979 | 3.375 | 3.291 |
| Our test set | 3.110 | 3.700 | 3.468 | 4.982 | 3.392 | 3.892 |
| EffectData | 3.176 | 3.428 | 3.370 | 4.992 | 3.066 | 3.392 |
With real-captured references, Reveris maintains comparable effect fidelity and visual quality: EF is 2.979 versus 3.110 on our original test set, and VQ is 3.375 versus 3.392. Source preservation and temporal consistency reach 4.083 and 3.625, respectively. Appendix F shows real fire-breathing and welding effects transferred to different target subjects and scenes. The generated videos reproduce the flame or welding arc and sparks while retaining the target subjects and surroundings. These results complement the EffectData evaluation and show successful transfer under a change in reference distribution.
To reduce reliance on a single generator, our training data combine videos from multiple video generators and cover 350 effect types across 13 families, with diverse subjects and scenes. We check both effect realization and visual quality, rejecting absent or incorrect effects, shot changes, severe structural artifacts, and dominant unrelated transformations. Captions describe the effects actually observed in the generated videos, including their onset, temporal progression, and final appearance, improving correspondence between the visual and textual training signals.
Reference–target pairs combine different videos within each effect category, exposing the model to the same effect across different subjects and backgrounds. These construction choices encourage transferable effect representations, while the external and real-reference evaluations assess generalization beyond the training corpus.
Response to Q1, Subquestion 1: Methodological Novelty
Reference Adaptation and Modulation Calibration address different aspects of reference conditioning.
Reference Adaptation establishes reference–target interaction within a shared pretrained I2V backbone. Target and reference tokens interact through selected self-attention blocks while retaining their respective text and image conditions. Compared with VAP’s separate reference expert coupled to a frozen backbone, our design reuses the existing backbone parameters and adapts its interaction self-attention without introducing new parameters in this stage.
Modulation Calibration augments content-based reference matching with an explicit temporal-variation prior computed from clean reference latents. The prior enters target-to-reference attention as a position-dependent bias. Learned signed amplitudes allow different attention heads to favor or downweight stronger reference variation, while gates conditioned on the source image and target prompt set its strength for each target. Reference-caption-conditioned features provide semantic context for matching.
Compared with a scalar conditioning-strength bias, such as that used in OminiControl, our modulation uses the spatial and temporal distribution of observed reference changes and adapts their influence to the target. Stage II calibrates this mechanism by freezing the adapted backbone and training only the added amplitudes and gates, with the attention bias initialized to zero. Reference Adaptation learns the reference–target interactions; Modulation Calibration learns how reference-change evidence should influence those interactions for different targets.