- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents a real-time streaming video stylization framework that combines source-initialized flow matching, style-specific LoRAs, causal teacher forcing, DMD, and rollout fine-tuning. It achieves impressive inference efficiency with reasonable visual quality.
Critical strengths
- Impressive inference efficiency, achieving over 100 FPS on an H100.
- A practical framework that combines one-step generation with temporally consistent streaming stylization.
- The ability to adapt new style LoRAs without repeating the full distillation pipeline is interesting.
Critical weaknesses
- Lack of theoretical analysis for Rectified Style Flow. While the paper argues that source initialization shortens the transport path, it lacks a theoretical analysis of how this affects trajectory straightness and distillation efficiency. The authors are encouraged to discuss its relationship to existing source-to-target bridge formulations, such as Bridge Transformers, and clarify the theoretical distinction.
- The method is restricted to style-specific LoRAs rather than general video editing. It is somewhat time-consuming to train a separate LoRA for each style.
- Most training components, including interleaved teacher forcing, causal adaptation, and DMD, are established techniques. The technical novelty is therefore somewhat limited.
Questions for authors
- Could the authors compare against a simpler baseline combining a pretrained causal video model, style LoRA, and SDEdit-style source initialization? It would be helpful to understand whether the proposed multi-stage training pipeline provides substantial improvements over this simpler approach.
- Why does multi-style training collapse, and could a unified model support more general editing tasks?
- How much data and training time are needed to adapt an unseen style? The recovery experiments report 100 updates but do not clearly specify the dataset size.
- How does the proposed recovery strategy compare with including a new style in the full four-stage training pipeline? Is there a noticeable difference in stylization quality, temporal consistency, or training cost?
1. What is the theoretical distinction from bridge formulations, and does RSF make one-step distillation easier?
RSF differs from a Brownian bridge in where it introduces randomness. We corrupt the source endpoint, Zs=(1−s)X+sϵ, and use a deterministic linear interpolation from that sampled endpoint to its paired style target Y. For geometry-preserving stylization, this lets the student retain source structure rather than regenerate it from noise. Our sweep also shows why retaining more source information is not always better: teacher CLIP-T peaks at s=0.2 for American Cartoon and 0.3 for Flat Vector Illustration, but at 0.6 for Van Gogh and 3D Chibi. For 3D Chibi, increasing s from 0.2 to 0.6 improves style alignment while reducing source similarity. Thus, the useful starting point depends on what the edit must change. This motivates distilling a source-conditioned correction into one step.
2. Compared with a pretrained causal video model + style LoRA + SDEdit initialization?
Here is the comparison results of SDEdit + Style Lora (the initial training start at s comes from transferred domain) against RSF + Style Lora (the initial training start at s comes from original domain). RSF wins is the actual clips that RSF achieves higher CLIP-T against SDEdit.
The results show that RSF is effective for the degree of stylization.
3. Why train a LoRA per style? Does a unified model necessarily collapse?
No. We chose per-style LoRAs because of a failure mode we observed, not because a unified model must collapse. When we trained the full RSF DiT jointly across styles, changing the requested style often produced much the same blended appearance, even though we sampled the style bins approximately equally. The outputs looked like a visual average of the styles rather than a convincing example of the requested one. Even strong classifier-free guidance did not reliably restore the distinguishing features.
Our hypothesis is that a shared DiT can reduce the flow-matching loss across many styles by first learning the changes they have in common. Updates for features specific to one style may instead compete. A separate LoRA gives each style a small set of parameters dedicated to its own source-to-target flow, without updates from other styles overwriting it. The trade-off is one adapter per style. Our observations motivate that choice under the setting of an 1.3B-level lightweight models, which may change with larger model (since the have more parameters for each specific style).
4. How much data and compute does recovery need, and how does it compare with full retraining?
Given an existing streaming backbone and a Stage-1 LoRA for the new style, our default recovery uses 10 bidirectional teacher trajectories of 81 frames each per style. We freeze both the backbone and the style LoRA and train only a rank-64 residual LoRA for 100 updates.
Using nested sets of 1, 3, and 10 trajectories per style gave nearly identical mean CLIP-T on Pexels-47: 0.2465, 0.2462, and 0.2464, respectively, compared with 0.2361 without recovery. The clearest benefit of using ten rather than one was lower warped LPIPS (0.01973 vs. 0.02065).
For the full-retraining reference, we reused the same four Stage-1 style LoRAs but retrained the full AR teacher and one-step student on the four styles together. This required 1,800 teacher trajectories (450 per style), 12,000 AR updates, and 2,000 DMD outer iterations: