- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents ProxyFit, which uses proxy point mappings between garment-side source points and person-side target points to enable precise wearing-style control in virtual try-on, including front opening, sleeve placement, hem placement, and fit. The authors develop an automated data pipeline that combines image editing, mesh-based correspondence matching, and quality filtering to construct paired images and point mappings for different wearing states of the same person and garment. Built on an SDXL-based dual-UNet backbone, ProxyFit introduces a dedicated proxy representation branch that performs Fold–Attention–ReFold operations to retrieve local features around garment source points and write them to specified person-side target locations. The model uses paired denoising training with shared noise and timesteps, while mapping dropout supports ordinary VTON. Experiments evaluate spatial control accuracy, point-placement error, and standard VTON quality, with ablation studies examining the contributions of the proxy branch and paired-training strategy.
Critical strengths
-
The authors develop an automated data pipeline that produces paired images and point correspondences for learning wearing-style control. The pipeline combines image editing, mesh-based correspondence matching, rule-based filtering, and multimodal pair review to construct samples with different wearing states under shared person and garment conditions, while identifying the corresponding locations of garment parts in person images. It provides supervision for both appearance changes and spatial mappings, making it well suited to training precise wearing-style control.
-
The authors introduce ProxyFit to enable precise control over how garments are worn while also supporting ordinary virtual try-on. Users can drag target points to coordinate changes in front opening, sleeve placement, and hem placement. Its proxy representation branch uses Fold–Attention–ReFold to extract queries at person-side target locations, retrieve local features around the corresponding garment-side source points, and write the retrieved features back at the specified locations. This well-motivated design directly incorporates garment–person correspondences into feature routing. Matched-noise paired training helps the model learn responses to wearing-state changes, while mapping dropout enables the same model to perform ordinary VTON without point mappings.
-
The experiments demonstrate strong control precision and generation quality, with ablations supporting the effectiveness of the main design choices. Under the same backbone, training data, and optimization budget, ProxyFit improves mean spatial control accuracy from 0.733 to 0.828 and reduces point-placement error from 51.40 to 40.13 pixels compared with WAW-style. The proxy branch also outperforms RGB- and embedding-based injection, while paired sampling and shared noise provide further improvements. Qualitative results demonstrate coordinated control of multiple garment parts. On standard VTON, the model achieves the best LPIPS, FID, and KID among the compared methods, indicating that improved controllability is compatible with strong appearance fidelity.
Critical weaknesses
-
The baseline comparisons are not sufficiently comprehensive. Limited qualitative comparisons and implementation details for WAW-style and MOFA-style make it difficult to assess the fidelity of these reimplementations. The evaluation also lacks comparisons with stronger closed-source commercial image editing models. Moreover, providing general-purpose editors with text instructions alone, without visual drag annotations such as those in Figure 4, leaves their ability to follow explicit spatial guidance underexplored.
-
Table 2 evaluates control accuracy, while Table 3 measures image quality only for ordinary VTON. Thus, appearance fidelity under point-based control lacks sufficient quantitative validation.
Questions for authors
-
Could the authors provide more qualitative comparisons and additional implementation details for WAW-style and MOFA-style? I believe these additions would strengthen the credibility of the paper’s findings.
-
Could the authors compare ProxyFit with stronger closed-source commercial models? Additionally, could general-purpose image editing models be given both text prompts and visual drag instructions, similar to the point-movement annotations shown in the source images of Figure 4?
-
Could the authors evaluate both control accuracy and image quality on the same point-controlled test set to assess whether ProxyFit preserves appearance fidelity during wearing-state changes?
Response 2/3
The core operations of Proxy Branch are reading, matching, and writing back within spatial features and do not depend on UNet-specific convolutional structures. In the new Qwen experiments, we connect it to 15 DiT layers, with each person proxy point still querying only the 3×3 neighborhood of its paired garment point. Compared with the same LoRA settings, adding this branch improves mAcc by approximately 3.2 percentage points and reduces point error by 9.91 pixels (21.55%). Thus, SDXL is an implementation choice for establishing controlled comparisons in this paper; the new Qwen experiments support transferring the proxy routing design to another backbone.
Q1: Edits of Different Parts and Different Difficulty Levels
Editing difficulty is mainly determined jointly by displacement magnitude, visibility changes, and the distribution of similar scenarios in the training set, rather than solely by the name of the garment part. Small movements within a visible surface are relatively easy, such as slightly adjusting the visible hem height, cuff placement, or front-opening spacing. In these cases, the source part and target location are clear, and the model mainly needs to adjust local structure while preserving the existing appearance.
Edits involving large movements, occlusion changes, or the generation of previously invisible content are usually more challenging. For example, rolling sleeves up substantially to the upper arms requires forming new folds and exposing the arms. We examine this difference in editing difficulty by comparing different sleeve-roll magnitudes.
For this purpose, we manually selected 50 images from the dataset and used SDXL-ProxyFit to perform two edits: rolling sleeves up to the forearms and rolling sleeves up to the upper arms. Each state produces 50 evaluation images, giving 100 images in total. Accuracy is calculated as the number of successful images for each state divided by 50, without averaging accuracies over multiple random seeds.
Table R6. Control precision at different sleeve-roll magnitudes.
In addition, we manually selected 50 images from the dataset, used SDXL-ProxyFit to perform small- and large-magnitude front-opening edits, and report their control precision.
Table R7. Control precision at different front-opening magnitudes.
In Table R6, rolling sleeves up to the upper arms has an accuracy 8 percentage points lower than rolling them up to the forearms, and point error increases by 12.38 pixels (approximately 37.00%). In Table R7, both front-opening magnitudes have an accuracy of 0.84, while the larger front opening increases point error by 2.05 pixels (approximately 4.77%). This shows that, in these selected examples, the effect of displacement magnitude on control results varies with the editing task.
Q2: Stability across Random Seeds
Content outside the intended editing region is consistently well preserved, with differences mainly occurring in the editing results within the target region. We conducted an uncertainty test to report its performance. We fix the person, garment, proxy point mappings, and inference parameters and change only the random seed. On top of the original seed of 42, we add 43, 44, 45, and 46, for a total of five random seeds (42–46) to evaluate control stability. Table R3 first computes the aggregate test-set metrics for each seed separately and then reports the mean and standard deviation across the five seeds.
Table R3. Control stability with fixed inputs and point mappings and varying random seeds. Each entry is the mean ± standard deviation under random seeds 42, 43, 44, 45, and 46. MOFA-style reports only Hem Acc on the 113-pair hem subset; this column is reported separately from the four-task mAcc on the full test set.
In the results reported in Table R3, ProxyFit's cross-seed mAcc standard deviation is 0.013, lower than WAW-style's 0.017; its point-error standard deviation is 2.389 pixels, lower than WAW-style's 2.983 pixels. This supports that ProxyFit exhibits smaller cross-seed fluctuations in aggregate test-set control metrics.