acceptodds
Withdrawn from ICLR 2027

FlowVVTON: Flow-Guided Mask-Free Video Virtual Try-On

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video virtual try-on aims to transfer a target garment onto a moving person across video frames. Existing methods often rely on human parsing masks or pose keypoints that can become unreliable under large motions and occlusions, causing boundary artifacts and temporal inconsistency. A further limitation is that most approaches rely solely on attention mechanisms for temporal modeling, providing no explicit motion supervision. To address both limitations, we propose FlowVVTON, a mask-free framework that eliminates parsing mask dependency while introducing explicit motion supervision. Our key innovation is a flow-warped latent loss that uses optical flow solely during training to align features across frames at multiple layers of the generation model, enforcing multi-scale temporal consistency. We incorporate this loss into a two-stage training strategy that first establishes mask-free spatial alignment and then introduces flow-guided temporal supervision. Together, these designs improve temporal consistency and reduce boundary artifacts, enabling robust garment transfer under large body motions. Experiments on TikTokDress show that FlowVVTON outperforms baselines by substantial margins, including a 5.7 reduction in unpaired VFID-R compared with SwiftTry, while requiring no segmentation masks, pose keypoints, or region annotations as model inputs.

Primary area:
generative models
Keywords:
Video Virtual Try-On, Mask-free Architecture, Optical Flow Warping, Temporal Consistency

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_CokZOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

FlowVVTON is a mask-free video try-on framework built on an SD1.5 dual-UNet backbone. A frozen reference UNet injects garment features via mutual self-attention, and newly inserted temporal attention layers are trained with a flow-warped latent loss computed on decoder features. Training proceeds in two stages on pseudo-paired TikTokDress data, and flow is discarded at inference.

Critical strengths
  1. Optical flow is training-time only, which keeps deployment simple.
Critical weaknesses
  1. Limited novelty. Flow-warped feature consistency losses are long established in video generation and video translation, and flow-based temporal supervision has existed in video try-on since FW-GAN and ClothFormer, both already cited in Section 2.2. The contribution reduces to applying a known loss at several decoder levels with multiple strides. Section 2.2 also mischaracterizes this prior work: FW-GAN and ClothFormer explicitly estimate and exploit flow rather than applying post-hoc smoothing, and ViViD and Tunnel Try-on are SD1.5 plus temporal-module methods, not SVD-based. (RealVVT is genuinely SVD-initialized; the blanket attribution in Section 2.2 should be corrected to apply only to it.)
  2. Mask dependency is displaced, not eliminated. Section 3.2 never names the off-the-shelf model that synthesizes source-side videos. If it is mask-based, the pipeline inherits precisely the mask-based teacher dependency criticized in Section 2.3, and the pseudo-pair construction then closely resembles PEMF-VTO, which builds its paired pseudo-data with the mask-based ViViD.
  3. Non-standard and unfair evaluation. On TikTokDress, CatV2TON and MagicTryOn are evaluated from released checkpoints without fine-tuning while the proposed model is trained in-domain; on ViViD the bias reverses, since training on TikTokDress and testing on ViViD is not an accepted protocol. VVT and the standard ViViD training split are missing.
  4. Missing representative baselines. No mask-free video baseline is compared, which is the central claim. ViViD, Tunnel Try-on, WildVidFit, RealVVT, PEMF-VTO and TripVVT are all cited in Section 2 yet never evaluated; PEMF-VTO in particular is the closest mask-free competitor.
  5. Gains are not attributable to the claimed contribution. Table 4 shows that removing L_flow still yields U-VFID-R of 1.24 against SwiftTry's 3.42, i.e. about 2.8x of the headline 5.7x gap is already obtained without the proposed loss, so most of it originates from the undisclosed pretrained mask-free image base model. The large VFID-R movement alongside a 4% change in VFID-I is unexplained and suggests dominance by low-level appearance copying, consistent with the admitted sleeve-length failure.
  6. Technical inconsistencies. The abstract and Figure 2 claim all layers whereas Section 3.3 restricts the loss to decoder levels. Eq. 2 warps frame t+1 with F_{t+1->t} under W(F,f)(p)=F(p+f(p)), which requires the forward field F_{t->t+1}. Flow magnitudes must be rescaled when downsampled per layer, and tau is given without resolution normalization. The main text states noise-prediction MSE while the appendix states v-prediction.
  7. Incomplete analysis. The clean t=0 forward pass is off the denoising trajectory and doubles compute, yet is neither justified nor ablated. No sensitivity study for lambda, level weights, or joint versus two-stage training. Ablation gaps of 0.04 VFID-I lack seeds or variance, and only 13 raters participated.
Questions for authors

See above.

1 comment

We thank the reviewer for the detailed assessment.

1. Novelty and related work.

Our contribution concerns where and how flow supervision is applied. FW-GAN, ClothFormer, and MV-TON exploit flow through generation, tracking, or refinement. We impose multi-resolution, confidence-weighted, multi-stride consistency on internal UNet features during video diffusion training (Sec. 3.1, Eqs. (2)–(3)). Flow supervises the trainable temporal modules and is discarded at inference. We will correct Sec. 2.2's descriptions of prior flow-based methods and distinguish SD1.5-based temporal-module architectures from SVD-initialized models.

2. Scope of mask independence.

The image backbone is DS-VTON; SwiftTry generates synthetic source-side videos (Sec. 3.2). Offline data construction therefore depends on a mask-based model, as in PEMF-VTO. Our mask-free claim concerns FlowVVTON's conditioning and deployment: it receives source frames and a garment image without mask or pose inputs, and does not require SwiftTry, mask estimation, or optical flow at inference.

We will remove “eliminates masks entirely” from Sec. 2.3. Synthetic videos are inputs and original videos are targets; Appendix A's contrary statement will be corrected.

3. Evaluation protocol and cross-dataset generalization.

The SwiftTry checkpoint in Table 2 also undergoes Stage 2 training on TikTokDress. Both are therefore trained in-domain, although backbones and data construction differ. Table 4 isolates the loss's incremental benefit.

Table 3 evaluates transfer from TikTokDress to ViViD. Such evaluation has precedent: PEMF-VTO trains on VITON-HD, DressCode, and ViViD, and evaluates on VVT and TikTok (Sec. IV-A, Table I). Our ViViD experiment similarly assesses generalization without dataset-specific fine-tuning.

CatV2TON and MagicTryOn use public checkpoints trained with ViViD data. We will clarify these training domains and label Table 3 as cross-dataset evaluation for FlowVVTON. VVT and in-domain ViViD training remain outside our evaluation.

4. Missing baselines.

Runnable implementations and pretrained checkpoints for PEMF-VTO and TripVVT were unavailable at submission and remain unavailable as of this rebuttal. PEMF-VTO's official repository contains only a README; TripVVT explicitly excludes training code, inference code, and checkpoints from its public release. This constrained reproducible comparisons. We acknowledge PEMF-VTO as a close comparator and TripVVT as a relevant baseline, and claim no superiority over unevaluated methods. We will clarify the release status and incomplete baseline coverage.

5. Attribution and interpretation of VFID improvements.

With the backbone, synthetic data, and temporal architecture retained, adding Lflow\mathcal{L}_{\mathrm{flow}} improves U-VFID-I/R from 32.48/1.24 to 30.99/0.60 (Table 4):

\quad \frac{1.24-0.60}{1.24}\times100\\%\approx51.6\\%.$$ Both metrics use spatiotemporal features; their relative changes neither establish appearance copying nor strictly separate spatial and temporal effects. **Both improve**, supporting the loss's additional benefit. Supplementary videos provide complementary qualitative evidence. We will distinguish this gain from the full $5.7\times$ gap to SwiftTry. **6. Technical consistency.** Features are captured from down, mid, and up blocks; “all layers” and decoder-only descriptions will be replaced by the precise supervised block set. With backward flow $b_l=F_{t+1\to t,l}$ and $\mathcal{W}(z,b)(p)=z(p+b(p))$, the discrepancy is: $$\left\|z_l(t+1,p)-\mathcal{W}\left(z_l(t),b_l\right)(p)\right\|_2^2.$$ **Eq. (2)** and **Appendix A's algorithm** will be corrected accordingly. Resizing scales displacements as $u_l=(W_l/W)u$ and $v_l=(H_l/H)v$. Confidence is computed at the training-image resolution with $\tau=3$ pixels, then resized to feature grids. The supplied zero-SNR configuration uses $v$-prediction; we will correct the main-text objective. **7. Clean-feature supervision and analysis scope.** Flow is estimated from clean frames. The clean branch at diffusion timestep $0$ avoids directly aligning independently sampled noise, which generally does not follow the estimated motion. This motivates the design without asserting exact feature equivariance or that noisy-feature supervision necessarily harms denoising. The extra forward and associated backward computation occur **only during training**, with **zero additional inference cost**. Table 4 demonstrates effectiveness at the evaluated setting; $\lambda=1$ was selected empirically. Sensitivity and training-strategy studies would further characterize its operating range. We will avoid treating the **0.04 VFID-I difference** as statistically reliable and present the **13-rater study** as supplementary evidence.
Reviewer_UMYSOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FlowVVTON proposes a mask-free video virtual try-on framework that combines temporal attention with a confidence-weighted, multi-scale flow loss on UNet features during training. It reports strong results on TikTokDress and evaluates performance on ViViD, alongside component ablations and a user study. However, the current evidence does not sufficiently demonstrate the advantage of the specific loss formulation or support the broader state-of-the-art claims.

Critical strengths
  1. The core technical idea is clearly presented. The explanation of how flow-guided feature alignment complements temporal attention makes the overall approach easy to follow.

  2. The method performs well on TikTokDress and provides a useful empirical reference for applying optical-flow supervision to video try-on. Its improvements over the evaluated baselines, together with the flow-loss ablation, support the practical value of this approach.

Critical weaknesses
  1. The specific advantage of the proposed loss formulation is insufficiently demonstrated. Flow-based temporal regularization has already been explored in prior work [3]. The ablation demonstrates a benefit from the proposed loss, but the variant without it already substantially outperforms SwiftTry on U-VFID-R. The roles of the underlying image model and synthetic training data therefore need clearer separation.

  2. The evaluation does not sufficiently support the claimed robustness to challenging motions or accurate preservation of garment attributes. Recent work investigates richer motion, interaction, and garment-detail challenges [4, 5, 6], while the current evaluation provides limited evidence on these aspects. Figure 9 also shows a long-sleeved reference garment rendered with short sleeves, highlighting a substantive limitation in garment fidelity.

  3. Important details needed to reproduce and interpret the results are missing or inconsistent. The pretrained image try-on model and the external model used to generate training data are not clearly identified. Moreover, lines 241–242 describe synthetic videos as inputs and original videos as targets, whereas lines 756–758 describe the synthetic outputs as training targets.

  4. The reported baseline training settings differ from the original papers, making the comparisons difficult to interpret. Appendix B, lines 776–779, describes CatV2TON training on TikTokDress, whereas its original paper [1, Section 4.2, p. 5] lists DressCode, VITON-HD, and ViViD for the second training stage. Similarly, lines 782–783 describe TikTokDress training for MagicTryOn and evaluation of public checkpoints without additional fine-tuning, whereas [2, Section 4.2, p. 7] states that both training stages use VITON-HD, DressCode, and ViViD.

  5. The figure layouts and font sizes need improvement. Figure 1 is overly crowded and uses excessively small text, while the text in Figure 7 is disproportionately large relative to the main text.

References

[1] CatV2TON: Taming Diffusion Transformers for Vision-Based Virtual Try-On with Temporal Concatenation [2] MagicTryOn: Harnessing Diffusion Transformer for Garment-Preserving Video Virtual Try-on [3] MV-TON: Memory-based Video Virtual Try-on network [4] The devil is in the details: Enhancing Video Virtual Try-On via Keyframe-Driven Details Injection [5] iTryOn: Mastering Interactive Video Virtual Try-On with Spatial-Semantic Guidance [6] TripVVT: A Large-Scale Triplet Dataset and a Coarse-Mask Baseline for In-the-Wild Video Virtual Try-On

Questions for authors

In addition to addressing the concerns above, could the authors provide more convincing comparisons and results on more challenging videos to substantiate the claimed state-of-the-art performance? I have concerns about the performance of the proposed method, particularly in light of the results reported in Table 3.

2 comments

We thank the reviewer for recognizing the clarity and practical value of our approach. We clarify the contributions, evaluation protocol, and planned revisions below.

1. Loss formulation and contribution attribution.

Flow regularization has precedents. MV-TON applies flow consistency in its memory refinement stage (Sec. 3.4, Eq. (9)); our loss directly regularizes internal features of the main video diffusion network. Sec. 3.1 (Eqs. (2)–(3)) describes its confidence weighting, feature resolutions, and temporal strides.

Tables 2 and 4 distinguish two sources of improvement. Our no-flow variant achieves a U-VFID-R of 1.24, compared with SwiftTry's 3.42, reflecting the combined DS-VTON backbone, synthetic video supervision, and temporal training. This comparison does not isolate the individual contributions of the image prior and data pipeline.

With that backbone, data, and temporal architecture retained, adding Lflow\mathcal{L}_{\mathrm{flow}} improves U-VFID-I/R from 32.48/1.24 to 30.99/0.60, including a 51.6% additional reduction in U-VFID-R, with no inference-time computation added. Table 4 further supports confidence weighting (0.67 to 0.60 VFID-R) and strides {1,2,4}\{1,2,4\} over stride 1 (0.72 to 0.60), although stride 1 has slightly better VFID-I. We will distinguish these gains from the overall gap to SwiftTry.

2. Garment fidelity, challenging videos, and quality–efficiency balance.

We acknowledge the sleeve-length failure in Figure 9. Cross-frame consistency does not guarantee correct garment attributes. Stage 2 freezes spatial weights (Sec. 3.3), and improving spatial garment transfer remains necessary.

Table 3 shows stronger ViViD quality results for MagicTryOn, whose backbone has 14B parameters. Our lightweight UNet-based system offers competitive quality with efficient single-GPU inference, requiring no masks, pose, or flow inputs at deployment. In Table 1's single-A6000 setup, ours takes 1.60 s/frame, versus 13.8 s/frame for MagicTryOn with CPU offloading—approximately 8.6×8.6\times faster under this configuration.

Against CatV2TON, Table 3 shows better results on all four paired metrics and U-VFID-I, while CatV2TON achieves better U-VFID-R and U-CLIP. Table 1 reports lower latency (1.60 vs. 2.44 s/frame) and GPU memory usage (7.6 vs. 21.0 GB). These results demonstrate a favorable balance of quality, inference cost, and conditioning requirements, including the fastest measured inference among the evaluated methods under the reported configuration. We will refine the Abstract, Introduction, and Conclusion accordingly.

In response to the reviewer's question, the supplementary videos already include large-amplitude body movements. Continuous playback reveals cross-frame garment stability, flicker, and texture drift more clearly than static figures. We will add explicit pointers in Sec. 4.3. These videos complement the controlled quantitative comparison in Table 4.

3. Model identities and synthetic-data direction.

The image backbone is DS-VTON, and SwiftTry generates the synthetic source-side inputs. We will name them in Secs. 4.2 and 3.2. Sec. 3.2, lines 241–242, gives the correct direction: synthetic videos are inputs; original videos are ground-truth targets. Appendix A, lines 756–758, incorrectly calls synthetic outputs training targets and will be corrected. SwiftTry is used only for offline data preparation.

4. Baseline evaluation protocol.

Appendix B, lines 776–783, incorrectly describes CatV2TON and MagicTryOn as trained on TikTokDress. We evaluated their public checkpoints without additional fine-tuning. Their original Sec. 4.2 descriptions specify DressCode, VITON-HD, and ViViD for CatV2TON's second stage and MagicTryOn's two stages. We will correct Appendix B.

Training domains matter in both directions: ours is trained in-domain for TikTokDress, whereas ViViD appears in these baselines' training data but not ours. Table 3 therefore evaluates these baselines in-domain while testing our cross-dataset transfer. Our competitive results on several metrics and Table 1's efficiency measurements demonstrate practical value in this setting. We will document training histories and distinguish these comparisons from matched-training experiments.

5. Presentation.

We will simplify Figure 1, enlarge its small annotations, and reduce Figure 7's oversized text for consistent readability.

Thanks for the author's reply. Since my concerns have not been resolved, I will keep my rating.

Reviewer_Ve8COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a flow-warped latent loss for mask-free video virtual try-on. Following mask-free spatial alignment in the first training stage, the loss is incorporated into temporal attention training in the second stage to improve garment fidelity and temporal consistency. The framework requires no additional conditioning inputs like segmentation masks at inference and reduces unpaired VFID-R by a factor of 5.7 compared with SwiftTry.

Critical strengths

The method is simple, intuitive, and deployment-friendly. The model requires only a source video and a garment image, avoiding the need for parsing masks, pose annotations, or optical-flow estimation at inference.

Critical weaknesses
  1. In the supplementary code, the features used to compute the flow-warped latent loss are detached from the computation graph, meaning that this loss cannot backpropagate gradients to the trainable modules and therefore does not contribute to their optimization. This raises concerns about whether the reported improvements can be attributed to the proposed loss, which is the paper’s central contribution.
  2. In the Introduction, the paper states its central question as “Can optical flow, used purely as a training-time supervision signal, enable robust mask-free video try-on?” However, it primarily addresses this question by introducing a flow-warped latent loss into a SwiftTry-like framework built upon an image diffusion model. Pretrained video diffusion models already benefit from stronger spatiotemporal representations and motion priors, enabling them to model coherent motion without explicit optical-flow supervision. Therefore, a more compelling question is whether the proposed loss provides additional gains for pretrained video diffusion models.
Questions for authors
  1. The authors should clarify whether the supplementary code matches the implementation used for the reported experiments.
  2. I suggest that the authors provide controlled experiments demonstrating that the proposed loss offers additional benefits for a video virtual try-on model built upon a pretrained video diffusion backbone with strong motion priors, rather than primarily compensating for limitations of an image-based backbone extended with temporal modules.
  3. How sensitive is performance to the flow-warped latent loss weight λ, and how is its value selected? This question is not decisive for my rating.
2 comments

We thank the reviewer for the careful assessment and constructive suggestions. We address the implementation discrepancy, pretrained motion priors, and the loss weight below.

1. Supplementary code and gradient propagation.

The supplementary code does not match the implementation used for the reported experiments. We inadvertently submitted an older version that detached the features used by the flow-warped latent loss. We agree that this prevents the loss from updating the intended trainable modules, and we apologize for this submission error.

In the implementation used for our experiments, the clean-feature UNet forward pass retains its computation graph, and the feature hooks store the original tensors without tensor-level detachment. The flow-warped discrepancy is computed through differentiable bilinear sampling, weighted by λ\lambda, and added to the denoising objective before backpropagation. Consequently, its gradients propagate to the trainable motion modules, while the spatial parameters remain frozen.

Two implementation details deserve clarification. First, UNetFeatureExtractor.detach() only removes hook handles; it does not detach the captured feature tensors. Second, the feature-variance normalization statistic is intentionally detached, while the feature reconstruction error remains differentiable. The corrected implementation also includes a runtime check that rejects a detached flow loss when its weight is positive.

The relevant empirical evidence is Table 4 in Sec. 4.5, “Ablation Study”: removing the flow-warped latent loss changes VFID-I/VFID-R from 30.99/0.60 to 32.48/1.24. These results support its contribution within our framework, although we agree that an ablation alone cannot verify code correctness. We will replace the supplementary implementation with the experimental version and document the discrepancy explicitly.

2. Complementarity with pretrained video motion priors.

Our framework already incorporates a pretrained video motion prior: its temporal modules are initialized from AnimateDiff (mm_sd_v15_v2.ckpt), rather than learned from scratch. AnimateDiff learns transferable motion representations from videos and is an established initialization used by representative video try-on methods, including SwiftTry, ViViD, and Tunnel Try-on. We will explicitly document this initialization in Sec. 4.2. AnimateDiff, SwiftTry, ViViD, Tunnel Try-on

Table 4 (Sec. 4.5) therefore evaluates the loss in a framework that already benefits from pretrained temporal representations. The λ=0\lambda=0 variant retains the same architecture and pretrained initialization. Adding the flow loss improves unpaired VFID-I/VFID-R from 32.48/1.24 to 30.99/0.60, including a 51.6% reduction in VFID-R. This supports an additional benefit beyond the motion prior in our evaluated framework.

Pretrained video priors and explicit flow supervision address complementary aspects of generation. Stronger video backbones can improve motion coherence, but do not inherently enforce accurate correspondence of garment textures across frames. Fine-detail preservation remains an active challenge in video try-on, motivating dedicated garment-preservation mechanisms even in modern DiT-based systems. MagicTryOn

Our loss supplies an explicit geometric constraint on corresponding UNet features across resolutions and temporal strides, while confidence weighting reduces the influence of unreliable flow. Its role is therefore complementary to learned motion priors. We agree that extending this evaluation to a complete pretrained video DiT backbone would further establish its applicability; our present evidence demonstrates this complementarity within an AnimateDiff-initialized framework.

3. Choice and sensitivity of λ\lambda.

As reported in Sec. 4.2, “Implementation Details,” under “Flow computation,” we use λ=1.0\lambda=1.0, corresponding to flow_loss_weight: 1.0 in the training configuration. This value was chosen empirically; it was not established as optimal through a systematic sensitivity study.

The comparison of λ=0\lambda=0 with the evaluated λ=1.0\lambda=1.0 configuration in Table 4, Sec. 4.5, supports the benefit of enabling the loss at this weight. It does not establish insensitivity to λ\lambda or optimality of the selected value. We will clarify the empirical selection rationale and acknowledge this limitation explicitly.

Thank you for your response. However, my concern remains unresolved, so I maintain my original rating. I hope this work continues to improve in future revisions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.