acceptodds
Under review as a conference paper at ICLR 2027

SartorAI: Multimodal Garment Pattern Generation and Editing with Tokenized Geometry

Abstract

Garments and their sewing patterns play an important role in fashion, gaming, film, and virtual reality. However, designing garments that can be sewn together still requires considerable expertise, making it difficult to meet the growing demand for personalized clothing. To bridge this gap, we propose SartorAI, a framework for garment generation and editing. Rather than representing garment panels as sets of parametric edges, we model their boundaries as arbitrary closed 2D curves and define stitching relationships as connections between boundary points. This representation avoids dependence on predefined panel-edge parameterizations and stitching schemes, which are commonly used in existing works. Our key insight is that panel shapes and their 3D layout provide strong cues about how the panels should be sewn together. We therefore separate panel geometry generation from stitching prediction, using a vision-language model (VLM) to generate panel geometry and a flow model to predict stitching conditioned on that geometry. To improve geometric precision, we encode panel geometry into discrete tokens using residual vector-quantized variational autoencoders (RQ-VAEs) and train the VLM to predict these tokens. This preserves the VLM's symbolic reasoning capabilities and interactivity while alleviating the precision limitations of autoregressive floating-point prediction. To train SartorAI, we construct a dataset of 100k garments, each draped over human bodies with diverse shapes and rendered with realistic PBR textures. Trained on this dataset, SartorAI achieves state-of-the-art performance in garment sewing pattern generation and editing.

Primary area:
generative models
Keywords:
Digital Avatar, Garment Generation, vision language models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZoRmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces SartorAI, a framework for garment generation with multimodal conditioning. It proposes a two-stage approach that combines a trained autoregressive model for generating tokenized panel geometry with geometry-conditioned flow modeling for generating stitching relationships. The paper also introduces GCD-Verse, a dataset containing text annotations and instruction-based editing pairs.

Critical strengths
  • A novel two-stage framework for garment generation that combines language-driven geometry generation with flow-based topology modeling. The experimental results demonstrate that the proposed method outperforms existing approaches.
  • A large dataset that provides a useful resource for the research community.
Critical weaknesses
  • While the results are promising, the motivation for generating panels and stitching information in two separate stages remains unclear. Since the second stage takes only the panels generated by the first stage as input, could the model instead generate stitching information immediately after the panels within a single stage? Further discussion of this design choice or additional ablation studies would strengthen the paper.
Questions for authors
  • Could the authors provide more garment editing results? For example, it would be useful to see successive edits to sleeve length through repeated instructions such as "make the sleeves longer."
  • Could the authors provide more complex generation examples, such as those using a real image as input, similar to the example in Figure 7(f)?
5 comments

We thank the reviewer for recognizing the strengths of our algorithm design and dataset construction, and for raising constructive questions. We address the weaknesses identified by the reviewer and respond to their questions point by point.

W1: Effectiveness of Decoupling Geometry and Topology

In response to the reviewer's request, we introduce a joint modeling variant, Joint-VLM, that predicts both panel geometry and stitching relationships. The new comparison shows that SartorAI's decoupled framework achieves more accurate stitching topology prediction. We have added qualitative and quantitative experiments to the revised main text and reproduced the quantitative results in Table 1 of the General Response.

Joint-VLM uses the same Qwen3-VL-8B backbone and token representations for panel shape, scale, and 3D pose as SartorAI, and reuses the same frozen shape and RT RQ-VAEs. Both models are trained and evaluated on GCD-Verse. Within a single autoregressive sequence, Joint-VLM first generates the geometry of all panels and then their stitching relationships. It also discretizes panel boundaries into points, but explicitly tokenizing every pointwise connection would produce excessively long sequences (approximately 2k stitching tokens per garment on average). We therefore use a simple lossless compression scheme that represents each seam as a pair of contiguous boundary intervals. Each interval is described by three tokens: the panel index, starting-point index, and number of consecutive points. A single direction token specifies the traversal directions of both intervals; together with a seam marker, each seam requires eight tokens. These tokens, along with markers delimiting the complete stitching sequence, are added to the vocabulary. During decoding, the two intervals are expanded into stitching point pairs by matching their relative positions. Like our flow-based representation, this encoding defines stitching relationships between sampled boundary points without imposing priors on panel-edge parameterizations or stitching topology.

Joint-VLM and SartorAI achieve nearly identical Panel CD scores of 1.823 and 1.824 cm, respectively, but their Stitch F1 scores are 0.699 and 0.896, giving SartorAI an absolute improvement of 0.197. This substantial difference in stitching accuracy, despite a shared backbone and geometry representation and nearly identical boundary reconstruction CD, supports the effectiveness of a dedicated geometry-conditioned topology predictor and provides more direct evidence for our decoupled design.

Q1: Successive Garment Editing

We appreciate this insightful question. SartorAI supports multi-round garment editing. We present some results in the supplementary material successive_edits.png.

The supplementary figure presents five examples, each with two successive editing rounds. The first addresses the reviewer's request by demonstrating progressive sleeve lengthening through successive instructions to make the sleeves longer. The second demonstrates editing across garment components: first shortening the top, then lengthening the skirt. The third and fourth examples show successive increases in skirt length and trouser length, respectively. The fifth demonstrates stable results across successive changes to the neckline.

In future work, we plan to extend SartorAI to an agentic loop supporting multi-round editing, collaboration among multiple agents, and automated iterative refinement. Although the current model design and training do not yet implement this full agentic workflow, the model already supports basic multi-round editing.

Q2: Generalization to Challenging Real-World Garments

SartorAI generalizes to a range of complex real-world garments and stitching structures and achieves higher geometry and topology prediction accuracy than existing methods, but can fail on particularly challenging poses or topologies.

We collected additional Internet images covering a design combining shoulder straps with off-the-shoulder bands (case 4), addressing the reviewer's request for examples similar to Figure 7(f), as well as challenging poses (cases 8, 9, and 12) and complex garment structures (cases 6, 7, and 11). We performed inference, simulation, and rendering with SartorAI, AIpparel, ChatGarment, GarmentGPT, and Joint-VLM (the GarmentGPT comparison was added at reviewer PnhL's request). These comparisons are shown in challenging_wild_img_comparison.png in the supplementary material. These qualitative comparisons expose limitations on particularly difficult examples. In particular, none of the models successfully reconstructed the design in case 4 highlighted by the reviewer. Similar designs account for too small a proportion of the training dataset, so the models have not adequately learned this stitching pattern. Nevertheless, SartorAI retains advantages over the other models.

Further improving the accuracy of panel generation and garment design requires more diverse training data that better reflect real-world garments. Collecting such data is an important direction for our future work, and we expect increasing both data scale and diversity to further improve generalization. Our representation supports this direction because it does not impose priors on panel-edge parameterizations or stitching topology, allowing it to accommodate a broad range of sewing-pattern data.

Dear Reviewer ZoRm,

Thank you for your thoughtful review and your positive assessment of our framework and dataset. With the discussion deadline approaching, we wanted to briefly follow up on our rebuttal.

In response to your suggestions, we have included a Joint-VLM comparison to evaluate our two-stage design, successive editing demonstrations, and additional comparisons on challenging real-world images. We would welcome your feedback on whether these results sufficiently address your questions, and we remain available to clarify any outstanding concerns.

We appreciate your time and the opportunity to improve our work through this discussion.

Best regards,
The Authors

Reviewer_PnhLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SartorAI, a two-stage framework for garment sewing-pattern generation and editing from text or image inputs. The proposed framework first uses a VLM to generate the shapes and layouts of garment panels, and then predicts how these panels should be stitched together with a geometry-conditioned flow model. The authors also build a large dataset, GCD-Verse, containing around 100K garments with different body shapes, rendered images, text descriptions, and editing pairs.  Experiments show that SartorAI performs well on garment generation and editing compared with existing methods.

Critical strengths

SartorAI aim to generate and edit sewing patterns from an image or a text prompt, or an edit instruction. The pipeline fine tuned Qwen3-VL-8B to predict discrete tokens for every panel. Two residual-quantized VAEs are used toproduce these tokens. A conditional flow model, which follows Nexus, can generate one latent per boundary point given the placed panels. The authors also build GCD-Verse with about 100k GarmentCode garments draped on varied bodies, with PBR textures, text from GPT-5.5, and template editing pairs.

Critical weaknesses

The main methodological design of SartorAI is to decouple panel geometry generation from stitching topology prediction. However, the current experiments do not directly verify whether this decomposition is actually beneficial. Although AIpparel performs joint modeling, it differs from SartorAI in the backbone, geometry representation, and stitching representation. Therefore, the comparison with AIpparel cannot isolate the effect of the proposed decoupled design. Since this decomposition is one of the central contributions of the paper, the lack of a controlled comparison makes it difficult to assess how much of the performance gain comes from this design itself.

GarmentGPT(ICLR2026) has been cited with a similar method using RQ-VAE and VLM, but was not be compared. In addition, there are SewingLDM, GarmageNet, Garment Particles and DressCode.

Questions for authors
  1. Could the authors provide a controlled comparison between the proposed decoupled framework and a joint modeling variant? For example, the VLM could be extended to predict both panel geometry and stitching relationships while keeping the other settings as consistent as possible.

  2. In the topology prediction stage, each panel boundary is sampled with 256 points regardless of panel size. What is the motivation for choosing 256 points? Since this may result in different sampling densities for large and small panels, could the authors provide an ablation on the number of boundary samples and report its effect on stitching accuracy?

5 comments

We thank the reviewer for recognizing the strengths of our algorithm design and dataset construction, and for raising constructive questions. We address the weaknesses identified by the reviewer and respond to their questions point by point below:

W1/Q1: Effectiveness of Decoupling Geometry and Topology

In response to the reviewer's request, we introduce a joint modeling variant, Joint-VLM, that predicts both panel geometry and stitching relationships. The new comparison shows that SartorAI's decoupled framework achieves more accurate stitching topology prediction. We have added qualitative and quantitative experiments to the revised main text and reproduced the quantitative results in Table 1 of the General Response.

Joint-VLM uses the same Qwen3-VL-8B backbone and token representations for panel shape, scale, and 3D pose as SartorAI, and reuses the same frozen shape and RT RQ-VAEs. Both models are trained and evaluated on GCD-Verse. Within a single autoregressive sequence, Joint-VLM first generates the geometry of all panels and then their stitching relationships. It also discretizes panel boundaries into points, but explicitly tokenizing every pointwise connection would produce excessively long sequences (approximately 2k stitching tokens per garment on average). We therefore use a simple lossless compression scheme that represents each seam as a pair of contiguous boundary intervals. Each interval is described by three tokens: the panel index, starting-point index, and number of consecutive points. A single direction token specifies the traversal directions of both intervals; together with a seam marker, each seam requires eight tokens. These tokens, along with markers delimiting the complete stitching sequence, are added to the vocabulary. During decoding, the two intervals are expanded into stitching point pairs by matching their relative positions. Like our flow-based representation, this encoding defines stitching relationships between sampled boundary points without imposing priors on panel-edge parameterizations or stitching topology.

Joint-VLM and SartorAI achieve nearly identical Panel CD scores of 1.823 and 1.824 cm, respectively, but their Stitch F1 scores are 0.699 and 0.896, giving SartorAI an absolute improvement of 0.197. This substantial difference in stitching accuracy, despite a shared backbone and geometry representation and nearly identical boundary reconstruction CD, supports the effectiveness of a dedicated geometry-conditioned topology predictor and provides more direct evidence for our decoupled design.

W2: Additional Comparisons with GarmentGPT

We have added qualitative and quantitative comparisons with GarmentGPT to the revised manuscript, with the quantitative results also reported in Table 1 of the General Response. The results show that SartorAI outperforms our reproduction of GarmentGPT. Since GarmentGPT also combines an RQ-VAE with a VLM, it is an important baseline closely related to our design. We likewise trained GarmentGPT on GCD-Verse using eight NVIDIA H200 GPUs until its evaluation metrics stopped improving.

As GarmentGPT's official implementation was not publicly available when we conducted these experiments, we reproduced the method based on its paper and compared it under the same evaluation protocol. The reported results are from our reproduction, which may differ from the authors' original implementation because some implementation details were not disclosed.

We acknowledge the relevance of SewingLDM, GarmageNet, Garment Particles, and DressCode, and have discussed them in the related work section. A fair quantitative comparison requires adapting their data pipelines, retraining them on GCD-Verse, and evaluating them under a common protocol, which entails additional implementation and training costs. Given the time and computational resources available for this rebuttal, we have not been able to cover all relevant methods. We therefore prioritized GarmentGPT as a recent method closely aligned with our task formulation and technical approach, complementing our existing comparisons with AIpparel and ChatGarment.

W3: Choice of Boundary Sample Count in Topology Prediction

Using 256 sample points per panel offers a reasonable trade-off between the accuracy of the stitching representation and computational cost. We explain this choice below, and our existing prediction results demonstrate its practical viability. Nevertheless, we agree with the reviewer that further ablation experiments are needed. These experiments require substantial time and computational resources. We are working to obtain additional quantitative evidence and will update our response as soon as results become available, although we cannot guarantee completion before the rebuttal deadline.

  1. Our 256-point Topology VAE achieves a test reconstruction F1 of 0.9999, indicating nearly lossless encoding. We further compared 128, 256, and 512 samples in an overfitting experiment on 4,096 training garments, using the same 40.82M-parameter VAE and optimization settings for 3,000 steps on eight H200 GPUs (global effective batch size: 4,096). As Table 2 shows, higher sample counts yield lower reconstruction F1 under this fixed training budget.

    Table 2. Topology VAE reconstruction F1 after 3,000 training steps.

    NReconstruction F1 ↑
    1280.9877
    2560.9179
    5120.6085
  2. The number of point pairs involved in attention within each panel and pairwise topology classification grows approximately quadratically with the number of samples. Higher sampling densities may therefore yield limited accuracy gains while substantially increasing computational cost.

  3. A fixed sample count does result in larger average physical spacing between boundary samples on panels with longer perimeters, but adaptive sampling introduces additional training complexity. Our Topology VAE uses GraphSAGE and attention to encode panel boundaries and stitching graphs into per-point latents, while Flow generates the corresponding latents conditioned on each point's 3D coordinates. Both use attention within each panel and exchange information across panels through mean-pooled panel features. A fixed sample count allows pointwise features to be organized into tensors with a uniform point dimension for parallel computation. Allocating different numbers of samples to panels according to their perimeters would require changes to data batching, cyclic boundary indexing, and stitch indexing, together with valid-point masks for attention, pooling, pairwise topology classification, and loss computation, or an implementation that directly handles variable-length inputs. Padding would introduce additional unnecessary computation. Thus, the fixed sample count is primarily a choice for implementation and computational efficiency; variable sample counts are feasible in principle. In addition, after prediction, we recover continuous stitching intervals from the predicted point pairs and refine their endpoints using corners on the predicted contours. We then resample these intervals along arc length according to their physical lengths and the mesh resolution, assigning both sides of each seam the same number of points, determined by the longer side. This removes the constraint of 256 points per panel on the seam sampling density used in the final simulation and ensures simulation stability.

  4. Too few samples increase positional errors in stitching intervals. During inference, we sample each panel boundary uniformly. If the sample count is too small, the spacing may not be sufficiently small relative to the length of a stitched edge, leading to large discretization errors at seam endpoints or even missed seams. We quantified this effect on the training set, with results in Table 3. N denotes the number of boundary samples per panel. Mean and P95 are the mean and 95th percentile of the endpoint displacement along the boundary divided by the length of the corresponding GT stitched edge on that side, expressed as a percentage. The collapsed seam rate is the percentage of GT seams for which the start and end points on at least one side are quantized to the same sample point.

    Table 3. Relative seam-endpoint discretization errors on the training set.

    NMean (%)P95 (%)Collapsed seam rate (%)
    647.43817.4851.862
    1283.7128.7251.112
    2561.8614.3620.524
    5120.9302.1830.167
  5. We will make every effort to complete the ongoing end-to-end topology prediction ablations with 128, 256, and 512 samples, but cannot guarantee that results will be available before the rebuttal deadline. We will report their effects on stitching accuracy and training efficiency as results become available.

Reviewer_PnhLOpenReview ↗

Thank you for rebuttal

Thank you for the authors providing the careful revision for my review comment. This rebuttal basically solved my previous concerns. Therefore, I have increased my rating accordingly.

Reviewer_3oHwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SartorAI, a framework for multimodal garment pattern generation and editing. It uses an RQ-VAE to tokenize panel geometry, a VLM to generate panel shapes and layouts, and a conditional flow model to predict stitching relationships. The authors also introduce a garment dataset and demonstrate improvements over existing methods.

Critical strengths

The separation of panel geometry generation and stitching prediction is a reasonable design, which enables flexible garment representations without predefined panel edges.

The method supports multimodal instruction-based editing, with encouraging quantitative improvements over existing approaches.

Critical weaknesses
  1. Insufficient qualitative evaluation. To better support the claims about generating physically plausible and simulation-ready garments, video demonstrations showing garment dynamic simulation under body motion are needed. The current static renderings are insufficient to assess the quality and reliability of the generated sewing patterns and stitching relationships.

  2. The evaluation is mainly conducted on a synthetic dataset. The generalization to more complex real world garments and stitching structures remains insufficiently demonstrated.

Questions for authors

How robust is the generated stitching topology for complex garments and challenging poses?

How well does the method generalize to garment structures and stitching patterns that are not well represented in the synthetic training dataset, e.g., on in-the-wild input images?

2 comments

We thank the reviewer for recognizing the rationale behind our design and the improvements achieved by our method, and for the constructive feedback. We address the concerns below.

W1/Q1: Dynamic Simulation Robustness

Our generated stitching topology supports robust garment simulation under complex and challenging body poses. To demonstrate this, we provide two videos in the supplementary material: simulation_synthetic.mp4 compares the ground-truth (GT) and predicted sewing patterns for the same synthetic example using the same simulation and rendering pipeline, while simulation_wild.mp4 shows a garment predicted by SartorAI from a real-world image. Both use Warp’s XPBD solver. The synthetic comparison shows dynamic behavior close to the GT result, while the real-world example demonstrates plausible garment motion.

W2/Q2: Generalization to Challenging Real-World Garments

SartorAI generalizes to a range of complex real-world garments and stitching structures and achieves higher geometry and topology prediction accuracy than existing methods, but can fail on particularly challenging poses or topologies. Although Figure 5 already presents results on real-world images, we understand the concern that these examples may be too simple. We therefore collected additional Internet images covering challenging poses (cases 8, 9, and 12) and complex garment structures (cases 6, 7, and 11). We performed inference, simulation, and rendering with SartorAI, AIpparel, ChatGarment, GarmentGPT, and Joint-VLM; the latter two comparisons were added in response to reviewers ZoRm and PnhL. These comparisons are shown in challenging_wild_img_comparison.png in the supplementary material. These qualitative comparisons expose limitations on particularly difficult examples while showing that SartorAI retains advantages over the compared methods.

Further improving the accuracy of panel generation and garment design requires more diverse training data that better reflect real-world garments. Collecting such data is an important direction for our future work, and we expect increasing both data scale and diversity to further improve generalization. Our representation supports this direction because it does not impose priors on panel-edge parameterizations or stitching topology, allowing it to accommodate a broad range of sewing-pattern data.

Dear Reviewer 3oHw,

Thank you again for your constructive feedback. As the discussion period approaches its end, we would be grateful if you could take a look at our responses and additional results when you have a chance.

To address your concerns, we have added dynamic simulation videos and comparisons on challenging real-world garments. We would appreciate your thoughts on whether these additions address your main concerns and would be happy to clarify any remaining questions.

Thank you for your time and consideration.

Best regards, The Authors

Comments on the paper

2 comments

We thank all reviewers for their constructive feedback on our paper. We also appreciate the reviewers' recognition of the rationale and novelty of our framework, its multimodal instruction-based editing capabilities and quantitative improvements, and the contribution of GCD-Verse to the community. The main concerns relate to the effectiveness of our geometry-topology decomposition, the scope of our experimental evaluation, and the choice of boundary sample count for topology prediction. In response, we have added a controlled Joint-VLM comparison, an additional GarmentGPT baseline, dynamic simulation videos, challenging real-world examples, and successive editing demonstrations. We also explain the rationale for using a fixed count of 256 boundary samples per panel and present quantitative experimental results to support this choice. We summarize our responses below and provide further explanations in our individual replies to the reviewers.

1. Quantitative Comparisons and Decoupling Ablation

To directly examine the effectiveness of decoupling geometry and topology, we added Joint-VLM, which predicts both within a single autoregressive sequence while sharing SartorAI's Qwen3-VL-8B backbone, geometry token representation, and frozen shape and RT RQ-VAEs. We also added a comparison with GarmentGPT, trained on GCD-Verse and evaluated under the same protocol. Since its official implementation was unavailable when we conducted the experiments, these results are from our paper-based reproduction and may differ from the authors' original implementation. We have revised the main text to include additional quantitative and qualitative comparisons, along with further details on Joint-VLM and GarmentGPT (no other changes have been made to the main text).

Table 1. Quantitative comparison on GCD-Verse, reproduced from the revised manuscript. Best results are in bold, and second-best results are in italics.

MethodPanel Acc. (%) ↑Rot L2 ↓Transl L2 (cm) ↓Panel IoU ↑Panel CD (cm) ↓Stitch F1 ↑
ChatGarment87.520.01925.910.7401.8930.871
AIpparel86.700.02546.820.7362.0270.830
GarmentGPT86.630.02799.250.7172.5010.804
Joint-VLM87.290.02587.630.7801.8230.699
SartorAI (ours)88.470.02486.380.8141.8240.896

SartorAI achieves the highest Panel Accuracy, Panel IoU, and Stitch F1. Joint-VLM and SartorAI have nearly identical Panel CD scores (1.823 and 1.824 cm), while SartorAI achieves a substantially higher Stitch F1 (0.896 versus 0.699). This comparison provides direct experimental support for a dedicated geometry-conditioned topology predictor.

2. Dynamic Garment Simulation

We provide two supplementary videos demonstrating garment dynamics under body motion using Warp's XPBD solver. simulation_synthetic.mp4 compares GT and predicted sewing patterns for the same synthetic example under an identical simulation pipeline, showing similar dynamic behavior; simulation_wild.mp4 demonstrates plausible dynamics for a garment predicted from a real-world image.

3. Generalization to Challenging Real-World Garments

We provide comparisons on 12 additional real-world images covering challenging poses and complex garment structures, including a design combining shoulder straps with off-the-shoulder bands similar to Figure 7(f). These comparisons are shown in challenging_wild_img_comparison.png in the supplementary material. These examples show SartorAI's relative advantages while also exposing limitations. However, we acknowledge that all evaluated methods, including SartorAI, face generalization challenges on particularly difficult real-world inputs. More diverse training data remain an important direction for improving generalization.

4. Successive Garment Editing

Five additional examples demonstrate two successive editing rounds involving sleeve length, top length, skirt length, trouser length, and neckline shape. The second round directly uses the panel tokens generated in the first round, demonstrating that the current model already supports basic multi-round editing.

5. Choice of Boundary Sample Count in Topology Prediction

We provide additional quantitative evidence for choosing 256 boundary samples per panel. A fixed sample count allows efficient batching and parallel computation in our Topology VAE and Flow; assigning different counts according to panel perimeter would require variable-length processing or padding and masking, together with changes to graph indexing. A controlled VAE overfitting experiment shows that increasing the sample count makes fitting more difficult. Statistics on the training set show that increasing the sample count reduces the mean seam-endpoint discretization error. Together with the approximately quadratic computational cost of pairwise processing, these results support 256 samples as a practical accuracy–cost trade-off. We will make every effort to complete the ongoing end-to-end topology prediction ablations with 128, 256, and 512 samples, but cannot guarantee that results will be available before the rebuttal deadline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.