acceptodds
Withdrawn from ICLR 2027

FACT: FREQUENCY AND ATTENTION CONTROL FOR TRAINING-FREE VIDEO STYLIZATION

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reference-guided video stylization transfers the appearance of a reference image to a source video while preserving source structure and temporal consistency. Recent training-free approaches leverage pretrained diffusion models to avoid costly training, yet they can lose fine-grained source details during style transfer and produce inconsistent appearance across frames. We analyze the frequency components of intermediate denoising latents and find that low-frequency components evolve gradually while high-frequency components change more rapidly across steps. Motivated by this finding, we propose FACT, a training-free framework that combines latent frequency modulation with attention control. At each denoising step, frequency modulation attenuates the low-frequency component of the current latent while retaining its high-frequency component, preserving fine spatial details throughout stylization. Logarithmic Length Attention Scaling (LAS) adaptively adjusts content-to-style attention strength across U-Net feature scales, balancing style injection with source-structure preservation. Confidence-Weighted Temporal Attention (CTA) guides cross-frame attention along tracked point correspondences, weighting interactions by tracking confidence and temporal distance to improve frame-to-frame consistency. Experiments show that FACT outperforms existing methods in reference-style transfer, source-structure preservation, and temporal consistency.

Primary area:
generative models
Keywords:
video stylization, diffusion models, frequency modulation, attention scaling, temporal consistency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SZADOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FACT stylizes videos with a frozen SD1.5. It attenuates low frequencies of the latent at every step, rescales content-to-style attention per U-Net scale (LAS), and edits temporal attention along confidence-weighted CoTracker tracks (CTA), evaluated on 2,400 video-style pair.

Critical strengths
  1. It is training-free on one frozen backbone, and the examples keep subjects recognizable while picking up the reference colors and textures.

  2. The evaluation is large, and the ablation shows CTA matters. Removing it drops CLIP-F from 0.986 to 0.974, far more than removing the other two parts.

Critical weaknesses
  1. The frequency step is close to StyleSSP's low-frequency attenuation. The paper says the difference is doing it at every step (lines 123-125), but there is no comparison with init-only filtering.

  2. CTA is a weightted version of OmnimatteZero's track-based temporal attention guidance. Table 2 only removes CTA entirely, so the weighting is never tested against plain averaging, and β is not given.

  3. Since n ≤ m_ref at every scale, LAS gives λ ≤ 1, which softens attention, the opposite of the paper's diffuse-attention motivation (lines 204-207). The gain may just be weaker style injection.

Questions for authors
  1. Where do the foreground masks come from? If they are dataset annotations, FACT gets an input the baselines don't.

  2. StyleMaster (CVPR 2025) addresses the same task. Since you already compare with StyleCrafter, which also needs training, could you add it or explain why not?

  3. SD1.5 has no cross-frame attention. How is the temporal attention built, and at which layers and steps is CTA applied?

Reviewer_1pogOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents FACT, a training-free framework for reference-guided video stylization built on Stable Diffusion v1.5. The method combines three components: latent frequency modulation to preserve spatial details, Logarithmic Length Attention Scaling (LAS) to control style injection across feature scales, and Confidence-Weighted Temporal Attention (CTA) to improve cross-frame consistency using tracked correspondences. Experiments on 2,400 video–style pairs show competitive results against six baselines, with ablations demonstrating the contribution of each component.

Critical strengths
  1. Practical training-free framework. The paper addresses two relevant challenges in video stylization: preserving fine-grained content details and maintaining temporal consistency. The proposed components operate entirely at inference time without modifying the pretrained diffusion model.

  2. Promising experimental results. FACT achieves the best ArtFID (31.96), CLIP-F (0.986), and VBench (75.43) among the compared methods. The qualitative results show reasonable preservation of object boundaries and reference appearance, while the component ablations indicate consistent improvements.

Critical weaknesses
  1. The benefits of the proposed designs are not sufficiently isolated. In Eq. (5), LAS produces scaling factors below 1 at typical U-Net feature resolutions. It is therefore unclear whether the improvement comes from the proposed length-dependent scaling or simply weaker style attention. Figure 7 lacks comparisons with fixed scaling factors in a similar range. Likewise, CTA builds on trajectory-guided attention, but the individual contributions of confidence weighting and temporal decay are not evaluated against uniform trajectory-based guidance. These missing controls make it difficult to assess the incremental novelty.

  2. The experimental protocol leaves important questions about fairness and temporal consistency. FACT uses a supplied foreground mask for CoTracker point sampling, while UniVST is evaluated with its mask module disabled. The source of these masks and their impact on performance are not reported. Moreover, the evaluation relies heavily on CLIP-F for temporal consistency, which does not directly measure motion-aligned correspondence or flickering. The absence of a comparison with FreeViS, despite its relevance to training-free video stylization, further limits the empirical assessment.

Questions for authors
  1. LAS: Does LAS outperform fixed attention scaling factors of comparable magnitude? A controlled comparison would help establish whether adapting the scaling across feature resolutions provides benefits beyond reducing attention strength.

  2. CTA: How are the cross-frame attention maps constructed within the SD1.5 U-Net, and are attention weights renormalized after the replacement in Eq. (12)? Can the authors also compare CTA against uniform trajectory-guided attention to isolate the contribution of confidence and temporal weighting?

  3. Evaluation: Where do the foreground masks used by FACT come from, and how does the method perform without them? Can the authors provide a motion-aware temporal consistency metric, such as flow-guided warping error, to complement the CLIP-F results?

Reviewer_wekUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FACT, a training-free framework for reference-guided video stylization, aiming to preserve fine-grained spatial details and temporal consistency during style transfer. The method combines latent frequency modulation, Logarithmic Length Attention Scaling, and Confidence-Weighted Temporal Attention to improve detail preservation, style injection, and cross-frame consistency, respectively. Experiments on over 2k video–style pairs demonstrate improvements over existing methods on several evaluation metrics.

Critical strengths
  1. The paper addresses two relevant challenges in video stylization: preserving fine-grained spatial details and maintaining temporal consistency.

  2. The proposed method integrates frequency modulation and attention control into a pretrained diffusion model without requiring additional training.

  3. The paper is well-organized and easy to follow.

Critical weaknesses
  1. I wonder whether spatial frequency modulation sufficient for video stylization? The authors should clarify why spatial-only filtering is adopted instead of spatiotemporal frequency modulation (e.g., 3D FFT). Jointly considering spatial and temporal frequencies might offer a better balance between detail preservation and temporal consistency.

  2. According to the method part, CTA relies on CoTracker, but the potential model bias introduced by the pretrained tracker is not sufficiently discussed. A robustness study regarding tracking accuracy or a discussion of potential error propagation would be helpful.

  3. To the best of my knowledge, frequency-based manipulation has been widely explored in prior image and video generation methods. Although some related methods are discussed, the authors should more precisely position their contributions and clarify the technical distinctions from existing frequency-based approaches.

  4. In this paper, each module proposed involves several heuristic design choices, but the current ablations mainly focus on comparisons with and without each module. It is recommended to add more fine-grained ablations, particularly comparisons among representative choices of weighting factors and other key design parameters.

Questions for authors

Please refer to the weakness part.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.