acceptodds
Under review as a conference paper at ICLR 2027

Beyond Holistic Realism: Dimension-wise Garment Fidelity Assessment for Virtual Try-On Evaluation and Optimization

Abstract

Virtual try-on (VTON) demands not only plausible visual synthesis but, crucially, faithful preservation of reference garment characteristics. However, conventional evaluation metrics—such as PSNR, SSIM, KID, and FID—primarily capture superficial or global realism, failing to reliably reflect multi-dimensional garment consistency. To address this limitation, we introduce DAT, a Dimension-wise Assessment framework for virtual Try-on. DAT disentangles garment fidelity into seven interpretable and critical dimensions: silhouette, color, neckline and sleeve shape, major decoration and structure, material texture, fine-detail fidelity, and logo preservation, formulating each as a dedicated attribute-level prediction task. To effectively train this assessment model, we implement a two-stage learning paradigm comprising scalable weak supervision on 50K paired samples, followed by high-precision refinement on 10K curated annotations distilled via multi-model consensus. Furthermore, a dimensional-weighted cross-entropy loss is introduced to counter severe label imbalances inherent across evaluation categories. Beyond benchmarking, we establish a closed-loop optimization by seamlessly integrating DAT into the reinforcement learning pipeline of Qwen-Image-Edit for VTON, where dimension-wise rewards are adaptively aggregated to prioritize under-optimized visual attributes. Extensive experiments demonstrate that our 8B model achieves state-of-the-art performance across balanced accuracy, SROCC, and PLCC—surpassing prominent frontier proprietary models including Gemini-3.1, Qwen3.7-plus, and GPT-5.5—while serving as an exceptionally effective supervision signal for reward-guided garment generation.

Primary area:
generative models
Keywords:
VTON, assessment, post training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5vG1OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DAT, a framework for evaluating virtual try-on performance. It relies on fine-tuning a VLM to conduct assessments across multiple dimensions. Scores from these dimensions can also serve as training supervision for other virtual try-on models, with the focus on improving the weak aspects of generative models. The paper validates its effectiveness via human rating comparisons and blind evaluation experiments.

Critical strengths
  1. The research problem has practical application value, and the evaluation dimensions are relatively comprehensive. 2. It verifies that improvements in the evaluator model can positively guide downstream generative models.
Critical weaknesses
  1. The source of test labels is not clearly stated. Besides, only three teacher models are adopted in the paper, and their capabilities are questionable. Thus, the correctness of such labels is hard to guarantee. 2. The work is more of a comprehensive application of large models. It essentially fine-tunes one VLM using several other VLMs, lacking distinct innovations. 3. The improvement of generative model performance is not sufficiently demonstrated. The score gains need to be interpreted with supporting independent evidence. Moreover, only one generative model is optimized, which is insufficient to confirm that this method generalizes to other models.
Questions for authors
  1. It is suggested to supplement the annotation pipeline for each dataset. In addition, stronger models may be used for auxiliary annotation to make the results more convincing. 2. The generative model tested in the paper is still a VLM. It would be worthwhile to explore other types of virtual try-on approaches.
Reviewer_AWiwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Dimension-wise Assessment framework (DAT) for a more thorough evaluation virtual-try on models. DAT assesses VTON quality along seven dimensions: 1) silhouette, 2) color, 3) neckline and sleeve shape, 4) major decoration and structure, 5) material texture, 6) fine-detail fidelity and 7) logo preservation. The proposed approach models each dimension as a prediction task. Dimension may not be applicable (e.g. there is no logo visible in the garment).

The training data is created with foundation models. For 50K image triplets (garment, model, try-on results) Gemini 3.1 is used to produce 7 scores along chosen dimension . For 10K triplets, Gemini 3.1 , Qwen 3.7 and GPT 5.5 are used to to for quality score by aggregating with the median of the three results. A model is trained to correctly predict the 7 quality scores, first on the less reliable 50 K dataset and then on the more reliable 10K dataset. As not all scores are equally present, the loss is weighted to amplify the rare scores.

DAT is subsequently used as a reward signal for RL optimization of Qwen-Image-Edit for VTON. During training, different dimensions are dynamically weighted to encourage focus on currently underperforming aspects.

Critical strengths

[S1] This paper considers a timely and important topic, that is to say proper evaluation of VTON. Shape, color, pattern are all important aspects of garment fidelity and correctness.

[S2] For the test, ground truth annotations were collected by using 5 expert annotators to show that DAT captures human VTON quality assessment better than individual foundation models.

[S3] A user study was done to show that DAT may be used to improve generation quality.

Critical weaknesses

[W1] The ground truth annotations are done with foundation models (Gemini, Qwen, GPT) which means that there is no control of annotation quality. This is also acknowledged by the authors. It is not clear how the evaluation is supposed to be adopted by the field. Will a DAT evaluation need to be retrained with a new batch of annotations as foundation models improve? Is the current one supposed to become the standard that everyone uses? Could DAT have been trained with a multi-head regression model instead of Qwen?

[W2] Following on W1, I would appreciate more insight into the current semantics of the score. What does a 0.7 for silhouette mean (good sleeve reconstruction, bad ration of width and height)? What were the differences in scoring between different models? Should the weighting of garment aspects be done according to human priority (e.g. color more important than fine detail)? Can the score be used to localize mistakes? Are all mistakes in shape or texture equally important? How were the human annotators instructed to handle these ambiguities?

[W3] The problem of generation quality assessment in fashion has been raised by VTOFF [a], where there is an added benefit of a clear ground truth. The paper would be stronger if it applied DAT to both tasks.

[a] Velioglu et al., TryOffDiff: Virtual-Try-Off via High-Fidelity Garment Reconstruction using Diffusion Models, BMVC 2025

Questions for authors
  1. Color and material and texture seem to be particularly difficult to achieve agreement with annotators. Is there an explanation for why that is?
Reviewer_PahzOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

DAT adapts Qwen3-VL-8B to assess garment fidelity along seven dimensions, including silhouette, structure, texture, fine details, and logos. It combines 50K weakly labeled examples with 10K multi-model-consensus refinement annotations and weighted cross-entropy training. The assessor is tested against five-expert labels on 1.6K examples from six generators, including three held-out sources. Its scores also guide Flow-GRPO optimization of Qwen-Image-Edit through adaptive dimension weights, with quantitative and blind human evaluations.

Critical strengths
  1. Useful and well-motivated evaluation target. Separating garment identity into interpretable dimensions addresses failures that global image metrics can obscure. How to evaluate the AIGC content in specific domain remains valuable.

  2. Independent human test labels. Five-expert assessment and held-out generator sources provide meaningful evidence beyond agreement with the automatic training annotators.

  3. The staged supervision scheme, explicit not-applicable categories, and dimension-specific reporting support diagnostic use.

Critical weaknesses

The methodological insight beyond task-specific data construction remains limited. Domain LoRA fine-tuning, consensus pseudo-labeling, class-weighted cross-entropy, and deficit-based reward weighting are relatively straightforward components.

Reproducibility and practical accessibility remain insufficiently established. The experiments rely on proprietary e-commerce data, an internally developed try-on generator, and annotations produced by commercial models. However, the submission provides no explicit commitment to release the dataset, human-annotated test set, DAT weights, or training and evaluation code. This limits independent verification of the reported human alignment and downstream improvements, as well as practical adoption of the proposed evaluator. Given the substantial data-driven contribution, a concrete release plan covering at least a representative benchmark subset, annotations, evaluation scripts, and model weights is important.

Annotation subjectivity and scoring stability are insufficiently examined. Direct human ratings on a five-point scale are subjective, and annotators may differ substantially in their preferences, interpretation of the rubric, and use of score levels. Majority voting alone does not establish label reliability; inter-rater agreement, score dispersion, and adjudication of disagreements should be reported. VLM-based annotations may also contain hallucinated judgments or vary across decoding seeds and repeated evaluations of the same image pair. The paper does not characterize these failure modes or quantify their impact on consensus labels, DAT predictions, and downstream rewards. Repeated-scoring experiments and sensitivity analyses are needed to establish the robustness of the assessment pipeline.

The median aggregation rule is undefined when annotators disagree on NA versus numeric scores.

Questions for authors

Please refer to weakness 2 3 and 4. I’d like to reconsider my rating upon a concrete open-source plan.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.