acceptodds
Withdrawn from ICLR 2027

FusedVTON: Early Image Fusion for Efficient Virtual Try-On

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing image-based virtual try-on methods typically use explicit garment warping or model garment–person correspondence through reference networks, cross-attention, or self-attention over concatenated tokens. Although effective, these designs introduce architectural complexity and computational overhead, especially in multi-garment try-on, where additional garments require additional warping operations or garment representations. We view garment transfer as coarse spatial placement followed by non-rigid generative refinement and propose to implement this formulation. Reference garments are resized and placed within the clothing-agnostic person image before VAE encoding. The resulting composite is processed through a single, fixed-length image-token stream without a separate garment branch or appended garment tokens. Experiments on VITON-HD, DressCode, DressCode-MR, and Garments2Look show competitive quality. Separate profiling at without classifier-free guidance measures 53.9% and 51.7% fewer model-inference FLOPs than the corresponding SD1.5 and FLUX.1 CatVTON variants. At , counted model-inference computation remains constant across one to four garment inputs in the evaluated scaling setup.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Virtual Try-on, Lightweight Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_MRdwOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

FusedVTON investigates whether virtual try-on requires garment representations separate from the person’s spatial token grid. It crops, resizes, and places reference garments into a clothing-agnostic person image before VAE encoding, then conditions an SD1.5 or FLUX.1 backbone on the fused image and DensePose. Multiple garments are jointly represented within the same canvas, keeping the denoiser’s spatial token count fixed. Experiments cover VITON-HD, DressCode, DressCode-MR, and Garments2Look, with quantitative comparisons, qualitative examples, and component ablations. The paper reports competitive quality and approximately 52–54% lower model-inference FLOPs than corresponding CatVTON variants under a separate profiling protocol.

Critical strengths
  1. A simple and consequential conditioning design. Representing garments within the person’s existing spatial grid removes the need for separate garment branches or additional garment tokens. This provides a clear computational advantage, particularly when conditioning on multiple garments. The demonstrated reduction in model-inference cost makes the design a useful contribution to efficient virtual try-on.

  2. Evidence across different backbones and try-on settings. The approach is instantiated with both SD1.5 and FLUX.1 and evaluated on four single- and multi-garment benchmarks. The fusion and placement ablations provide additional support for the proposed design choices. Together, these experiments suggest that early image fusion is a viable conditioning strategy beyond a single architecture or dataset.

Critical weaknesses
  1. Heuristic placement and dependence on masks. The method relies on heuristic garment placement guided by clothing-agnostic masks and human parsing. Although Table 5 compares fusion spaces and placement strategies, robustness to inaccurate masks remains untested. How do missing regions, incorrect boundaries, or parsing errors affect placement and generation? Controlled perturbation experiments are needed. Furthermore, the current pipeline still requires mask-based preprocessing, and its applicability to mask-free methods is not established.

  2. Incomplete efficiency evaluation. Since efficiency is a central contribution, FLOPs and latency alone provide an incomplete assessment. The paper should also report total inference parameters and peak GPU memory, together with preprocessing costs. Including compact diffusion and representative GAN-based baselines, such as Mobile-VTON and SD-VITON, under comparable settings would better establish the practical quality–efficiency trade-off.

  3. Missing human evaluation. The paper does not include a user study. Aggregate image metrics cannot fully establish garment fidelity or perceptual realism, particularly for text, intricate textures, and densely composed outfits. A blinded comparison assessing garment similarity, person preservation, and overall realism would provide stronger evidence that the computational savings preserve perceptually important details.

  4. Insufficient visualization of multi-garment inputs. Although joint generation is described, the paper should show the actual fused conditioning images used during training and inference for two, three, and four garments. These examples should illustrate area allocation, overlap resolution, and the effect of garment dropout during training. Packing multiple items into one canvas may produce crowded compositions or excessive shrinking; without inspecting these inputs, it is difficult to assess how much garment information survives preprocessing.

  5. Limited quantitative validation at high resolution. The quantitative quality evaluation is restricted to 512 × 384, despite the availability of higher-resolution baselines and the paper’s own 1024 × 768 efficiency measurements and qualitative examples. Downsampling may obscure precisely the texture and detail losses introduced by fixed-canvas fusion. The authors should justify this choice and provide matched 1024 × 768 quality, latency, and memory comparisons to establish whether the reported trade-off persists at higher resolution.

Questions for authors

My questions are incorporated into the five points in Critical Weaknesses. Please respond to them individually. Clarifications concerning mask dependence, multi-garment input construction, and the evaluation resolution could materially affect my assessment.

Reviewer_kzqqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FusedVTON investigates whether virtual try-on requires garment representations separate from the person’s spatial grid. It crops, resizes, and places reference garments within category-dependent regions of the clothing-agnostic person image before VAE encoding. The fused-image latent and the DensePose condition are concatenated channel-wise with the noisy target latent and processed by a fine-tuned SD1.5 or FLUX.1 backbone. Because all garments share one composite image, the denoiser’s input size remains fixed at a given resolution regardless of the number of garments. Experiments on VITON-HD, DressCode, DressCode-MR, and Garments2Look show competitive, though not uniformly superior, quality. With CFG disabled for all compared methods and external preprocessing excluded, FusedVTON requires roughly half the model-inference FLOPs of the corresponding CatVTON variants.

Critical strengths

A simple conditioning design with a clear computational benefit. FusedVTON incorporates reference garments into the person’s existing image canvas before VAE encoding, avoiding separate garment branches and appended garment tokens. This keeps denoiser computation independent of garment count at a fixed resolution. Profiling at 512 × 384, with CFG disabled for all compared methods and external preprocessing excluded, reports 53.9% and 51.7% fewer model-inference FLOPs than the corresponding SD1.5 and FLUX.1 CatVTON variants. The FLUX.1 denoiser breakdown in Table 1 further explains the sources of computational savings. This property is particularly valuable for multi-garment try-on: in the scaling experiment in Figure 6, FusedVTON maintains constant counted model-inference FLOPs, whereas all three evaluated baselines incur increasing costs as garments are added.

Critical weaknesses
  1. The source of the quality gains is not sufficiently established. Despite its simpler conditioning and reduced spatial reference budget, FusedVTON outperforms concatenation-based baselines on several metrics, including 11 of the 12 single-garment quality metrics reported for FLUX.1 in Table 2. However, the experiments do not establish whether these gains arise from early fusion or other configuration differences. The exact baseline checkpoints and their training or adaptation procedures are insufficiently specified, and the training procedure for the authors’ FLUX.1 port of FastFit is not described. FusedVTON fully fine-tunes its denoiser on combined VITON-HD and DressCode data with DensePose, but whether the evaluated concatenation baselines use comparable training and conditioning is unclear. In particular, the paper should clarify whether its CatVTON baselines retain the original methods’ parameter-efficient fine-tuning and conditioning configurations or were adapted for these comparisons. A same-backbone concatenation baseline with matched training data, budget, fine-tuning policy, and pose conditioning would help isolate the contribution of early fusion.

  2. The quality–efficiency trade-off is insufficiently characterized. Fitting garments into restricted canvas regions may discard fine appearance information, a limitation acknowledged in Appendix D but not directly quantified. In Figure 9 (first row), FusedVTON appears to preserve the reference print’s fine structure less faithfully than CatVTON. On Garments2Look, it achieves stronger paired metrics but worse unpaired FID/KID than FastFit (16.29/1.410 versus 15.48/0.968). These observations do not establish downscaling as the cause, but motivate evaluating garment-detail fidelity across downsampling factors, allocated areas, and garment counts, alongside a concatenation baseline at comparable computation. Additionally, profiling disables CFG for all methods, whereas FusedVTON’s reported single-garment quality uses two denoiser evaluations per step; the baselines’ corresponding guidance settings are not specified. Reporting cost under the configurations that produce the quality scores would clarify the practical trade-off.

Questions for authors
  1. Attribution of quality gains. Please specify the baseline checkpoints and their training or adaptation procedures, particularly the FLUX.1 port of FastFit and the multi-garment CatVTON and RefTon variants. A same-backbone concatenation baseline with matched training data, budget, fine-tuning policy, and pose conditioning would help isolate the contribution of early fusion.

  2. Garment-detail preservation. Can you quantify garment-specific fidelity across reference downsampling factors, allocated canvas areas, and garment counts? This would contextualize the print differences in Figure 9 and the weaker unpaired FID/KID than FastFit on Garments2Look. How does early fusion compare with concatenation using reduced reference resolution at a comparable computation budget?

  3. Multi-garment placement. Could you visualize the fused conditioning images for two-, three-, and four-item outfits, including shoes and bags? Beyond the overlap-resolution rule in Appendix A, please specify how the placement regions for these categories are constructed.

  4. Cost at the reported quality. Beyond the standardized profiling without CFG, please report inference costs under the configurations used to obtain the quality scores, including each baseline’s guidance settings and sampling steps. Reporting preprocessing time separately would also clarify the complete pipeline cost.

  5. Qualitative comparison protocol. Figures 8–11 predominantly identify FusedVTON outputs as 1024 × 768, whereas Appendix C states that all outputs are generated at 512 × 384. Please clarify the generation resolution, resizing procedure, and checkpoint used for each method in the qualitative comparisons, and explain the “selected 512 × 384 output” in Figure 11.

  6. Garments2Look split. Appendix A states that Garments2Look provides no official split, whereas the dataset paper’s Appendix A.2 and Table S1 describe a train/test division. Please explain the relationship between that division and your processed subset and product-disjoint split, and specify the training and evaluation splits used for each baseline.

  7. Metric consistency. The paired/unpaired KID values for FusedVTON (FLUX.1) on VITON-HD differ between Table 2 (0.207/0.239) and Table 4 (0.1992/0.2205). Please clarify whether the tables use the same checkpoint, generated outputs, and evaluation settings, and explain the discrepancy.

Additional suggestions: Provide implementation details for the latent-space fusion and random-placement variants in Table 5, report variability across seeds for close comparisons, and add the missing references for LaDI-VTON, OOTDiffusion, and RefTon.

Reviewer_FVbYOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

FusedVTON resizes and pastes reference garments into a clothing-agnostic person image before VAE encoding. The resulting fused image provides garment and person conditioning through a single spatial grid, avoiding separate garment branches or additional reference tokens. The method combines this representation with DensePose conditioning and is instantiated with SD1.5 and FLUX.1. Experiments cover single- and multi-garment try-on on four datasets. The paper reports approximately 52–54% lower model-inference FLOPs than the corresponding CatVTON variants under profiling without classifier-free guidance, and constant model computation as garment count increases at fixed resolution.

Critical strengths
  1. Reusing the masked person-image region for garment conditioning provides a practical way to reduce reference-token overhead, particularly for multi-garment try-on.

  2. Careful computational analysis. Table 1 distinguishes token-linear projections from quadratic attention products and shows that most measured savings at the evaluated resolution come from the former. This provides a more precise explanation than attributing all savings to quadratic attention.

  3. Ablations examine fusion space, placement, DensePose, and explicit masks. The Garments2Look split separates connected groups of looks and products, reducing leakage through reused garment assets.

Critical weaknesses
  1. I have tryied similar methods before. Early fusion imposes a fundamental bottleneck on garment-detail preservation. Resizing reference garments to fit within the person’s editable region substantially reduces their effective spatial resolution, potentially discarding fine logos, text, and textures before VAE encoding. The problem becomes more severe when multiple garments share the same canvas. Increasing reference resolution independently is a direct way to preserve these details, but the proposed formulation couples garment resolution to the person-image grid, making it impossible.

  2. The method depends on accurate garment-agnostic masks, limiting robustness and applicability. Early fusion requires a garment-agnostic region to remove the original clothing and place the reference garment. Errors in this region can leave residual clothing, remove relevant person details, or constrain garment placement incorrectly, particularly under large silhouette changes and complex occlusions. The proposed pipeline therefore does not directly support mask-free try-on, and its reliance on external parsing and mask construction introduces additional preprocessing and failure modes.

  3. The fidelity cost and robustness of fitting all garments into one canvas remain insufficiently evaluated. Constant model computation follows from the fixed token grid, but does not establish that garment fidelity remains stable as more items share its limited area. Figure 6 profiles nested inputs from a single outfit, without corresponding quality measurements by garment count, garment size, or pattern complexity. Aggregate image metrics can also obscure errors in small shoes, bags, logos, and textures. Furthermore, placement depends on source-person parsing and category regions, while the Garments2Look unpaired protocol transfers only shared garment slots. Per-garment fidelity evaluation and tests involving large silhouette changes, overlapping layers, difficult poses, and perturbed masks would better establish the limits of the proposed representation.

Questions for authors

Please refer to Weaknesses 1 2 and 3

Reviewer_AMnwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper considers virtual try-on. The proposed approach fuses garment and masked model image in a simple manner by just pasting a sized down garment into the masked body area without any further processing. The This allows for cross-attention between garment and model without increasing the number of tokens or using parallel networks. The placement of the garment into the mask is done to retain as much garment resolution as possible.The fused image is combined with DensePose and passed into the image generating model (used here are StableDiffusion and FLUX.1). DensePose encoding depends on the chosen generative model.

Critical strengths

[S1] Simple idea for simplifying VTON and making it more resource efficient. Results do indicate that this is the most efficient generative setup out of those considered

[S2] Experiments done on a wide array of publicly available VTON datasets, which is good for future comparison.

Critical weaknesses

[W1] Results in terms of generative quality are mixed, with generative models analyzed being relatively old (Stable Diffusion from 2022 and FLUX from 2024). Furthermore, generative model metrics are quite unreliable, so having a user study would make the quality analysis more convincing.

[W2] The main idea of this paper is to maximally simplify garment and model information fusing. In that sense, the main competitors are similarly efficient pre-warp methods, but they do not have results on most of the datasets. It would be valuable to give more insight into what exactly is lost in terms of quality (texture, pattern, shape) with each simplification decision (e.g. could a more complex pasting strategy that does not fit into the smallest box. but into a more complex shape be used to preserve more information)

[W3] If the goal is efficiency, it would make sense to also consider training efficiency in terms of which parts of the network need to be finetuned in this efficient setup, or if PEFT could be applied.

Questions for authors

In figure /, what is the explanation for the far right image and the magenta dot location? Are there any examples with more complex textures?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.