acceptodds
Under review as a conference paper at ICLR 2027

Sketch2BRep: Boundary-First Diffusion for Sketch-to-BRep CAD Generation

11 views

Abstract

Generating B-Rep CAD models from sketches is challenging because sparse 2D drawings provide only partial boundary cues, while B-Reps require both continuous geometric primitives and discrete topological relations. Existing methods typically infer coupled geometry and topology from sparse sketches, often failing to produce B-Reps that are both geometrically precise and topologically valid. We observe that boundary curves naturally bridge sketches and B-Reps: sketches provide intuitive cues for object boundaries, while boundaries reveal faces of a B-Rep. Motivated by this, we propose Sketch2BRep, a boundary-first hierarchical latent diffusion framework for sketch-conditioned B-Rep generation. Sketch2BRep decomposes the generation process into three progressive stages: sketch-conditioned boundary generation, boundary-conditioned surface generation, and explicit boundary-surface relation modeling. Specifically, a Shape-Layout Coupled Sketch Diffusion employs interacting curve-shape and spatial-layout branches to infer a plausible structured 3D boundary conditioned on the partial boundary cues in the sketch. A Boundary-Conditioned Joint Surface Diffusion enables direct interaction between face and edge tokens to synthesize surfaces consistent with the recovered boundaries. A Geometry-Aware RelationNet models pairwise boundary-surface relationships to explicitly assemble the B-Rep topology. Experiments demonstrate that the proposed method, by following the intrinsic boundary–surface–topology construction logic of B-Reps, outperforms state-of-the-art methods in geometric fidelity, topological validity, and sketch consistency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Sketch-to-BRep Generation, Boundary-First Generation, Latent Diffusion Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_MzszOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Sketch2BRep generates B-Rep CAD models from a single sketch through a boundary-first pipeline. It first generates 3D boundary curves using shape–layout coupled diffusion, then synthesizes boundary-conditioned surfaces, and finally predicts face–edge relationships to assemble the B-Rep. Experiments on DeepCAD and ABC evaluate geometric accuracy, topology, sketch consistency, and validity, with ablations examining the proposed components and generation order.

Critical strengths

Strengths

  1. Clear presentation and well-motivated design. The paper clearly explains the challenges of recovering geometry and topology from sparse sketches. Using explicit boundaries as an intermediate representation provides a coherent connection between sketch cues and surface generation.
  2. Promising results with supporting ablations. The reported comparisons show improvements in geometry, topology, and sketch consistency. Component ablations and the controlled boundary-first versus surface-first comparison provide useful evidence for the proposed design.
Critical weaknesses

Weaknesses

  1. Unclear handling of primitive counts. The paper defines variable-sized edge and face sets but does not clearly explain how their sizes are determined at inference time. Since diffusion requires an initial noise tensor of a specified shape, the authors should clarify how edge and face counts are selected, how variable-length sets are handled, and whether padding, duplication, or filtering is used. This is important for understanding whether generation operates from the sketch alone or requires additional count information.
Questions for authors

As weakness.

2 comments

Questions: Unclear handling of primitive counts. The paper defines variable-sized edge and face sets but does not clearly explain how their sizes are determined at inference time. Since diffusion requires an initial noise tensor of a specified shape, the authors should clarify how edge and face counts are selected, how variable-length sets are handled, and whether padding, duplication, or filtering is used. This is important for understanding whether generation operates from the sketch alone or requires additional count information.

Response: Thank you for pointing out this important implementation detail. We apologize for the insufficient clarification of how variable-sized edge and face sets are handled during inference.

To select edge and face counts at inference, our Sketch2BRep follows a strategy similar to BrepGen (Xu et al., 2024). Specifically, it adopts fixed-capacity diffusion followed by geometric deduplication and validity filtering, rather than explicitly predicting primitive counts. The boundary diffusion stage initializes a predefined number of candidate edge tokens from Gaussian noise, where each token contains an edge point latent, corner coordinates, and a bounding box. These geometric attributes are jointly denoised under the guidance of the input sketch. After denoising, redundant edge candidates are identified by comparing their predicted bounding boxes and removed using a predefined similarity threshold. The remaining edges are further examined through endpoint matching and connectivity analysis to determine whether they can form closed boundary loops. Geometrically inconsistent candidates are filtered out, yielding a variable-sized boundary set that provides structural guidance for subsequent surface generation. Conditioned on the retained boundaries, the surface diffusion stage similarly initializes a fixed number of candidate face tokens from Gaussian noise. Each face token consists of a surface point latent and a bounding box, which are jointly generated through the Boundary-Conditioned Joint Surface Diffusion to maintain consistency between surface geometry, spatial layout, and the recovered boundaries. After denoising, redundant face candidates are removed based on bounding-box similarity. The remaining candidates are further validated using the face-edge relationships predicted by Geometry-Aware RelationNet. Specifically, the associated boundary edges are examined to determine whether they can form closed trimming loops. For faces with insufficient incident edges, additional high-confidence edge candidates are considered to complete the boundary. Faces that still fail the boundary consistency checks are excluded from the final B-Rep reconstruction.

To handle variable-length primitive sets, we employ fixed-capacity representations, padding masks, and geometric filtering rather than duplicating valid primitives. This scheme ensures that padded or filtered tokens do not interfere with subsequent feature interactions. The predefined capacities specify only the shapes of the initial Gaussian noise tensors, whereas the effective numbers of edges and faces are determined by geometric deduplication, connectivity analysis, and validity filtering.

In the rebuttal version, we have added a dedicated section Appendix A.2 (Inference and Post-Processing), providing more implementation details on the initialization capacities, deduplication criteria, closed-loop validation, and padding-mask strategy.

Reference:

Xiang Xu, Joseph George Lambourne, Pradeep Kumar Jayaraman, Zhengqing Wang, Karl D. D. Willis, and Yasutaka Furukawa. Brepgen: A b-rep generative diffusion model with structured latent geometry. ACM Transactions on Graphics, 43(4):1–14, 2024.

Thanks for your response, I will maintain my score.

Reviewer_xEkuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents Sketch2BRep, a framework for generating B-Rep CAD models from single 2D sketches. The main challenge is that sketches only provide partial boundary information, while constructing a valid B-Rep requires accurate 3D geometry and consistent topological relationships. To address this, the authors propose a boundary-first generation pipeline that separates the task into three stages. First, a sketch-conditioned diffusion model reconstructs complete 3D boundary curves by jointly modeling curve shapes and spatial layouts. These boundaries are then used to guide surface generation through a second diffusion model. Finally, a geometry-aware relation network predicts face–edge connectivity, which is used to assemble the final B-Rep.

The method is evaluated on the DeepCAD and ABC datasets, covering both sketch-conditioned and unconditional generation. Compared with HoLa, Sketch2BRep achieves better geometric accuracy on most metrics, improved sketch consistency, and higher B-Rep validity. Ablation studies further show the benefits of the proposed components and the boundary-first generation order.

Critical strengths

The paper presents an interesting boundary-first approach to sketch-to-BRep generation. The idea of generating 3D boundaries before surfaces is well motivated and may help preserve geometric details from sparse sketches.

The experimental evaluation is fairly comprehensive, covering both DeepCAD and ABC datasets. The results are generally promising, and the ablation studies provide reasonable support for the effectiveness of the proposed components.

Critical weaknesses
  1. The evaluation relies entirely on synthetic sketches rendered from CAD models. Since real hand-drawn sketches often contain irregular or missing strokes, testing on such inputs would help better assess the method's robustness and practical applicability.

  2. A single 2D sketch may correspond to multiple plausible 3D shapes, but the paper provides limited discussion of this ambiguity. It would be useful to examine whether the model can generate diverse and reasonable B-Reps from the same sketch.

Questions for authors

Compared with previous boundary-first methods such as BrepForge, what is the main advantage of predicting topology after surface generation, particularly for sketch-conditioned inputs?

3 comments

Questions 1: The evaluation relies entirely on synthetic sketches rendered from CAD models. Since real hand-drawn sketches often contain irregular or missing strokes, testing on such inputs would help better assess the method's robustness and practical applicability.

Response 1: Thank you for your valuable suggestion. To assess Sketch2BRep’s robustness and practical applicability, we have conducted additional experiments on real hand-drawn sketches following your suggestion. The results are listed in the newly added Section 4.5 and Fig. 8 of the rebuttal version.

Specifically, we collect some hand-drawn sketches from four users and generate corresponding B-Rep models, with each result visualized from two different viewpoints. The results show that Sketch2BRep can recover CAD structures from manually drawn sketches, preserving the intended shapes and key geometric features despite variations in irregular or missing strokes. We also acknowledge that reconstruction quality may degrade for highly abstract or ambiguous sketches, where irregular strokes and insufficient geometric cues make the underlying 3D structures difficult to infer. This limitation is partly attributed to the domain gap between the CAD-rendered sketches used for training and unconstrained hand-drawn inputs. We have explicitly discussed this observation in Section 4.5 of the rebuttal version.

Questions 2: A single 2D sketch may correspond to multiple plausible 3D shapes, but the paper provides limited discussion of this ambiguity. It would be useful to examine whether the model can generate diverse and reasonable B-Reps from the same sketch.

Response 2: Thank you for your helpful comment. We totally agree that the ambiguity of 2D sketches should be considered in sketch-conditioned B-Rep generation. Accordingly, we have added a diversity study in Section 4.6 of the rebuttal version, with the results presented in Fig. 9. Specifically, we select eight input sketches and generate five B-Rep models for each sketch using different random seeds. Although the generated models exhibit variations in hole sizes, wall thicknesses, and local geometric profiles, they maintain similar overall shapes and key structural arrangements consistent with the input sketches. These results demonstrate that Sketch2BRep preserves the inherent diversity of diffusion-based generation while producing multiple reasonable B-Rep interpretations that remain consistent with the input sketches.

The added experiments and corresponding discussions have been included in Section 4.6 of the rebuttal version.

Questions 3: Compared with previous boundary-first methods such as BrepForge, what is the main advantage of predicting topology after surface generation, particularly for sketch-conditioned inputs?

Response 3: Thank you for your insightful comment. Unlike BrepForge, our method is designed specifically for sketch-conditioned inputs and therefore must account for the sparse characteristics of 2D sketches during generation.

In our view, the main advantage of our strategy is that it avoids determining uncertain topological relationships too early, thereby reducing the risk of incorrect connectivity affecting subsequent surface generation. Specifically, sparse 2D sketches provide incomplete geometric information, making it difficult to determine which boundaries belong to each face. If topology is predicted before surface generation, incorrect Face–Edge relationships may impose inappropriate constraints on subsequent surface reconstruction. In contrast, our method first recovers 3D boundaries and uses them as geometric guidance to generate surfaces, without requiring the Face–Edge relationships to be determined in advance. Once the surfaces are generated, the model can jointly consider boundary and surface geometries to establish their topological relationships. This allows the model to make topology predictions based on more complete geometric evidence, rather than relying solely on ambiguous boundary information. Therefore, our topology-last strategy is particularly suitable for sketch-conditioned B-Rep generation.

Reviewer_eVdtOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Proposes to use an intermediate 3D boundary-curves prediction as an additional step in sketch-to-B-rep modeling. Rather than directly predicting patches and connectivity, the first use diffusion to generate a completed 3D boundary (inferring plausible hidden lines) and then use that guidance for the generation of the 3D surface elements.

Critical strengths

The method is explained in sufficient detail and clearly described.

Although in the appendix, the argument that boundary curves as an initial step helps is shown by experiment

Directly addressing topology alongsied the geometry is good.

On DeepCAD data, it seems to improve over HoLa on most metrics.

Critical weaknesses

-The main sketch-conditioned comparison is against HoLa on DeepCAD only

  • There don't seem to me many sketch-conditioned methods to compare to, which is surprising. They only compare to HoLa on DeepCad data. Of course the win for ABC in a table with only one row..... (Table 1, second part of the table). The authors explain why, but some comparison against some adapted baseline is needed there.

** Post Rebuttal** Thank you for the additional experiments. I maintain my previous rating of week-accept.

Questions for authors

Input sketches contain both projected B-rep edges and view-dependent silhouettes, which need not correspond to B-rep boundary curves. How does the method handle this distinction? You show in the ablation that predicting B-rep boundaries is a more effective intermediate task than predicting surfaces directly—but why? It is unclear to me why sequential rather than joint prediction would be beneficial.

In the A3 ablation, how is topology recovered without Geometry-Aware RelationNet?

3 comments

Question 1: The main sketch-conditioned comparison is against HoLa on DeepCAD only. There don't seem to me many sketch-conditioned methods to compare to, which is surprising. They only compare to HoLa on DeepCad data. Of course the win for ABC in a table with only one row..... (Table 1, second part of the table). The authors explain why, but some comparison against some adapted baseline is needed there.

Response 1: Thank you for your valuable suggestion. To provide a more comprehensive evaluation of sketch-conditioned B-Rep generation, we have extended the comparison by including GenCAD (Alam & Ahmed, 2025), an image-conditioned CAD generation method that can be adapted to sketch inputs. Since GenCAD's original input preparation differs from ours, we retrained it on our constructed DeepCAD and ABC sketch datasets following its original experimental settings, ensuring a consistent input setting for comparison. HoLa is evaluated only on DeepCAD because its implementation and pretrained weights for ABC are unavailable.

Table 1(a) Sketch-conditioned generation\mathbf{\text{Table 1(a) Sketch-conditioned generation}}

\hline \text{Dataset} & \text{Method} & \text{CD-V}\downarrow & \text{CD-E}\downarrow & \text{CD-F}\downarrow & \text{Geo-V}\uparrow & \text{Geo-E}\uparrow & \text{Geo-F}\uparrow & \text{FE}\uparrow & \text{EV}\uparrow & \text{CLIP}\uparrow & \text{SSIM}\uparrow & \text{Val.}\uparrow \\\\ \hline \text{DeepCAD} & \text{GenCAD} & 1.93 & 1.66 & 1.57 & 0.61 & 0.63 & 0.60 & 0.62 & 0.57 & 0.69 & 0.65 & 63.50\\% \\\\ & \text{HoLa} & 0.77 & 0.22 & 0.13 & 0.77 & 0.79 & \mathbf{0.84} & 0.79 & 0.78 & 0.81 & 0.73 & 86.28\\% \\\\ & \text{Ours} & \mathbf{0.76} & \mathbf{0.18} & \mathbf{0.11} & \mathbf{0.79} & \mathbf{0.82} & 0.83 & \mathbf{0.81} & \mathbf{0.80} & \mathbf{0.86} & \mathbf{0.81} & \mathbf{88.71\\%} \\\\ \hline \text{ABC} & \text{GenCAD} & 2.16 & 1.71 & 1.62 & 0.59 & 0.56 & 0.52 & 0.58 & 0.55 & 0.66 & 0.60 & 56.35\\% \\\\ & \text{Ours} & \mathbf{0.83} & \mathbf{0.25} & \mathbf{0.14} & \mathbf{0.75} & \mathbf{0.79} & \mathbf{0.80} & \mathbf{0.76} & \mathbf{0.73} & \mathbf{0.83} & \mathbf{0.79} & \mathbf{66.84\\%} \\\\ \hline \end{array}$$ *Note: CD denotes Chamfer Distance, Geo denotes Geometry F1. V, E, and F represent Vertex, Edge, and Face, respectively.* The additional results are reported in Table 1(a) of the rebuttal version. On the ABC dataset, Sketch2BRep outperforms the adapted GenCAD baseline across all evaluation metrics. Specifically, Face--Edge and Edge--Vertex Topology F1 increase from 0.58 and 0.55 to 0.76 and 0.73, respectively, while CLIP Similarity and SSIM improve from 0.66 and 0.60 to 0.83 and 0.79. B-Rep validity also increases from 56.35\% to 66.84\%. Similar improvements are observed on DeepCAD, where Sketch2BRep achieves higher geometric accuracy, topology consistency, and sketch alignment than GenCAD. These additional comparisons provide a more comprehensive evaluation of Sketch2BRep, demonstrating consistent improvements in geometric accuracy, topological consistency, and sketch alignment across both DeepCAD and ABC datasets. The corresponding experiments and discussions have been added to **Section 4.2** and **Table 1(a)** of the rebuttal version. **Reference:** Md Ferdous Alam and Faez Ahmed. GenCAD: Image-conditioned computer-aided design generation with transformer-based contrastive representation and diffusion priors. Transactions on Machine Learning Research, 2025. URL https://openreview.net/forum?id=e817c1wEZ6.

Question 2: Input sketches contain both projected B-rep edges and view-dependent silhouettes, which need not correspond to B-rep boundary curves. How does the method handle this distinction? You show in the ablation that predicting B-rep boundaries is a more effective intermediate task than predicting surfaces directly—but why? It is unclear to me why sequential rather than joint prediction would be beneficial.

Question 2-1: Input sketches contain both projected B-rep edges and view-dependent silhouettes, which need not correspond to B-rep boundary curves. How does the method handle this distinction?

Response 2-1: Thank you for important comment. We handle the distinction between input sketches and B-Rep boundary curves through supervised diffusion. Specifically, the input sketches are rendered from CAD models using hidden-line removal, including visible projected edges and silhouettes, while the corresponding complete 3D B-rep boundary curves serve as supervision. Through sketch-conditioned diffusion, the network learns to infer the underlying 3D boundary structures from these partial 2D observations, rather than directly converting individual sketch strokes into B-rep edges. This enables the model to recover visible and hidden B-Rep boundaries while implicitly learning the distinction between sketch contours and actual B-Rep edges.

Question 2-2: You show in the ablation that predicting B-rep boundaries is a more effective intermediate task than predicting surfaces directly—but why? It is unclear to me why sequential rather than joint prediction would be beneficial.

Response 2-2: Jointly predicting boundaries and surfaces from a sparse 2D sketch is challenging because the model must infer both continuous geometric primitives and their structural relationships from incomplete observations. To address this challenge, we follow the progressive generation paradigm adopted by previous B-rep methods (Xu et al., 2024; Li et al., 2026), decomposing the complex generation process into several manageable stages.

We adopt the boundary-first strategy because B-Rep boundaries are more similar to sketches than surfaces. Both sketches and B-Rep boundaries are curve-based representations that describe important structural features of an object. This makes boundary prediction from sketches more straightforward than direct surface generation. Therefore, our method first recovers complete 3D boundaries from the input sketch and then uses them as explicit geometric constraints to guide subsequent surface generation. This progressive strategy reduces the difficulty of generating complex surface geometry directly from sparse 2D sketches.

To further verify the effectiveness of this generation order , we have added a controlled comparison between boundary-first and surface-first generation in Appendix A.7, with the results reported in Table 3. Both variants use the same sketch encoder, geometric representations, training data, and topology reconstruction procedure. On DeepCAD, boundary-first generation reduces Face Chamfer Distance from 0.027 to 0.011, improves Face–Edge Topology F1 from 0.65 to 0.81, and increases B-Rep validity from 80.33% to 88.71%. CLIP Similarity and SSIM also improve from 0.71 and 0.70 to 0.86 and 0.81, respectively. These results demonstrate that the boundary-first strategy provides effective structural guidance and improves geometric accuracy, topological consistency, and sketch alignment.

Reference:

Xiang Xu, Joseph George Lambourne, Pradeep Kumar Jayaraman, Zhengqing Wang, Karl D. D. Willis, and Yasutaka Furukawa. Brepgen: A b-rep generative diffusion model with structured latent geometry. ACM Transactions on Graphics, 43(4):1–14, 2024.

Jing Li, Yihang Fu, and Falai Chen. BrepForge: Factorized B-rep synthesis via wireframe composition and boundary-conditioned surface instantiation. arXiv preprint arXiv:2605.19411, 2026.

Question 3: In the A3 ablation, how is topology recovered without Geometry-Aware RelationNet?

Response 3: In the A3 ablation, inspired by BrepGen (Xu et al., 2024), a geometry-based topology reconstruction strategy is adopted to recover the topology without relying on a learned relation predictor. The main difference between this ablation strategy and the full Sketch2BRep lies in how the Face–Edge relationships are determined. Without A3, these relationships are inferred from geometric proximity and connectivity. In contrast, the full Sketch2BRep employs Geometry-Aware RelationNet to explicitly learn the compatibility between each edge and face by considering their geometric features, enabling more reliable topology prediction.

Specifically, in the A3-ablated variant, geometric correspondences between the generated boundary and surface sets are first established. For each generated face, candidate boundary edges are identified according to their spatial proximity to the surface, using the predicted bounding boxes and decoded geometric information. The selected edges are then connected by matching nearby endpoints and organized into closed trimming loops. Once a valid loop is formed, its constituent edges are assigned to the corresponding face, thereby establishing the Face–Edge adjacency relationships. For adjacent faces, matching boundary edges are identified and merged into shared edges, indicating that they belong to both faces. Meanwhile, spatially coincident edge endpoints are merged into shared vertices, from which the Edge–Vertex adjacency relationships are recovered. Finally, the reconstructed topology is used to trim the generated surfaces and assemble the complete B-Rep.

Comments on the paper

1 comment

We sincerely thank all reviewers for their constructive and detailed feedback. We have carefully addressed every concern through clarification and additional experiments in our point-by-point responses below. We have revised the paper by incorporating all the new results, analyses, and revisions, and updated the PDF for your reference. All the edits are marked with blue text. We greatly appreciate the reviewers’ time and insightful comments, which have helped us substantially improve our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.