RealFit: Measurement-Conditioned Virtual Try-On

The core value of real-world try-on lies in allowing consumers to both a) visualize how a garment looks on themselves, as well as b) see how a garment will fit on them: whether any given size will stretch tightly over body contours, or drape loosely with excess slack. Prevailing Virtual Try-On (VTO) models address the first use-case, but ignore the second: by conditioning solely on 2D visual inputs, they default to synthesizing deceptively well-fitted try-on results regardless of garment dimensions. Data scarcity is a central bottleneck for developing fit-aware VTO systems. Standard public benchmarks consist of catalog imagery lacking both garment dimensions and body measurements. While recent synthetic approaches leverage 3D physics simulation to generate exact measurement labels, they suffer from sim-to-real domain gaps and remain constrained to simplistic garment topologies. In this work, we address the issue of fit through RealFIT, the first framework for scalable, real-world measurement-conditioned virtual try-on across diverse and structurally complex apparel categories. To bridge the data gap, we introduce a multi-tier data hierarchy that bootstraps geometric priors from large-scale web retail metadata and VLM-assisted measurement annotations, anchored by a newly curated real-world dataset with tape-measured metric ground truth. Using this data, we propose a fit-aware virtual try-on model based on flow matching that integrates a dedicated measurement encoder to condition generation on human and garment dimensions. To enhance the model's sensitivity to metric conditions, we also introduce a Localized Metric Contrastive (LMC) loss, which explicitly boosts the conditioning signal by contrasting velocity fields across paired sizing perturbations within targeted spatial boundaries. Furthermore, we establish a systematic evaluation protocol to benchmark fit accuracy. Extensive experiments demonstrate that RealFIT achieves state-of-the-art photorealism and fit accuracy and controllability across varied body geometries and complex garment styles. We plan to release our benchmark dataset, evaluation suite, and code upon acceptance.