acceptodds
Under review as a conference paper at ICLR 2027

Contrastive Learning to Align Drawings and CAD Models for Retrieval

Abstract

Retrieving computer-aided design (CAD) models from drawings can support the discovery and reuse of existing designs in engineering workflows. However, differences between visual depictions and parametric model representations make it challenging to identify the relevant CAD model, particularly when the query image has been altered. This paper proposes a contrastive learning framework that aligns CAD drawings with CAD model representations in a shared embedding space. The framework learns correspondences between drawings and model construction sequences, enabling retrieval through embedding similarity. Beyond matching an original drawing to its corresponding CAD model, the approach also retrieves relevant models from altered query images. We evaluate the framework on a dataset of parametric CAD command sequences derived from the ABC dataset, assessing retrieval performance under both original and altered query conditions. Experimental results show improved retrieval performance compared with baseline methods, demonstrating that contrastive learning can effectively align CAD drawings with parametric CAD model representations. These findings support the applicability of contrastive learning beyond image–text alignment to image–CAD alignment, enabling CAD model retrieval directly from visual queries.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
CAD Retrieval, Contrastive Learning, Cross-Modal Retrieval, Parametric CAD Models, Representation Learning, Image-to-CAD Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_URFVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a dual-encoder framework for image-to-CAD retrieval, aligning rendered CAD images and parametric command sequences using CLIP RN50, a Transformer-based CAD encoder, and symmetric contrastive learning. On a gallery of 7,629 CAD models, it achieves 69.60% Recall@1 for canonical queries and 67.40% for mixed canonical/altered queries, demonstrating promising cross-modal retrieval performance.

Critical strengths
  1. Retrieving editable CAD models directly from visual queries is a meaningful engineering task. The framework is straightforward and computationally suitable for embedding-based retrieval.

  2. The evaluation uses a reasonably large held-out gallery and reports strong Recall@K performance. Sequence-equivalent matching appropriately accounts for duplicate CAD command representations.

Critical weaknesses
  1. The core image-CAD contrastive alignment closely resembles GenCAD, which already supports image-based CAD retrieval. The paper primarily reformulates this approach for retrieval without substantial methodological innovation. Moreover, Table 2 compares results obtained under different datasets and evaluation protocols, making the claimed improvements inconclusive. Controlled comparisons and ablations are missing.

  2. The altered-query experiment mixes canonical and modified images, without reporting altered-only performance or modification severity. Consequently, the small performance drop does not conclusively establish robustness to geometric changes. Furthermore, evaluation is restricted to synthetic CAD renderings, leaving generalization to real drawings and sketches unverified.

Questions for authors
  1. Can the authors provide controlled comparisons against GenCAD and representative image-based retrieval baselines using identical query sets, galleries, and evaluation metrics? What improvements can be attributed specifically to the proposed architecture?

  2. What proportion of queries in Experiment 2 are geometrically altered? Could the authors report Recall@K separately for altered-only queries, ideally stratified by modification type or severity?

  3. How many duplicate quantized CAD sequences occur in the test gallery, and how does exact-instance Recall@K compare with the reported sequence-equivalent metric?

1 comment
  1. We do not claim contrastive learning itself as our contribution. Sharing a dual-encoder contrastive setup does not make our method a reformulation of GenCAD; the contribution must be assessed through the differences in training, inputs, and representation design.

  2. Our framework jointly optimizes the CLIP RN50 image encoder and Transformer CAD sequence encoder for retrieval. GenCAD first trains its CAD encoder for autoregressive sequence reconstruction and then freezes it during contrastive alignment. Consequently, our framework allows both modalities to adapt to the retrieval objective, whereas GenCAD aligns image representations to a fixed CAD representation.

  3. Our method maps single shaded renders and CAD sequences into a shared embedding space optimized for bidirectional similarity matching. GenCAD instead uses multi-view line drawings and a CAD latent space shaped by sequence reconstruction within a generative pipeline. These constitute substantive differences in the learning formulation.

  4. On the same dataset, our framework achieves 69.60% Recall@1 with a substantially larger gallery, compared with GenCAD's reported 60.77% on 2,048 candidates. These results provide relevant evidence of our framework's retrieval capability. We acknowledge, however, that differences in evaluation protocols prevent these figures alone from establishing controlled superiority.

  5. The limited prior work and differing experimental setups make a common evaluation setting challenging to establish. GenCAD's own retrieval comparison is also limited to a ResNet baseline. We agree that controlled experiments against GenCAD would strengthen the comparison; nevertheless, we believe the reported results support our framework's strong retrieval performance.

  6. As explained in the paper, the mixed-query evaluation was deliberately designed to approximate a practical retrieval scenario in which queries may include both canonical and altered images. The mixture is therefore part of the evaluation design.

  7. Each query view is sampled uniformly from indices 0–4, with only index 0 corresponding to the canonical view. Consequently, the expected proportion of altered queries is 80%. The reported performance therefore predominantly reflects retrieval with altered inputs, rather than a mainly canonical evaluation.

  8. The decrease from 69.60% to 67.40% Recall@1 provides evidence that retrieval performance remains stable under this predominantly altered-query setting. We agree that this aggregate result does not establish altered-only performance or robustness across all levels of modification severity, and our conclusions should be interpreted within the tested conditions.

  9. As explained in Table 1, the test set contains 7,629 images corresponding to 7,609 CAD models, so a small number of models have two valid query images.

Reviewer_zF5EOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the retrieval of corresponding parametric modeling command sequences from CAD rendered images. Our method employs a pre-trained CLIP RN50 image encoder and a Transformer sequence encoder trained from scratch, aligning the two modalities via symmetric contrastive learning. On a test set containing 7,629 models, the raw image query achieves an R@1 of 69.60% and an R@10 of 93.86%.

Critical strengths

The research problem is closely aligned with the requirements of engineering design reuse. We conduct retrieval over the complete test library, explicitly reporting both the number of queries and the scale of the candidate pool. Acknowledging that different model IDs may correspond to identical command sequences, we introduce a sequence equivalence evaluation metric. Furthermore, we present failure cases and openly discuss limitations regarding cross-domain generalization and comparison protocols.

Critical weaknesses
  1. The novelty of the proposed method is limited compared to GenCAD
  2. The absence of baselines under a unified protocol undermines the claims of performance advantage made in the abstract.
  3. The experimental design for 'modified queries' is insufficient to solely verify modification robustness.
  4. The sequence equivalence evaluation is reasonable, but it requires a more complete analysis of data duplication and data splits.
  5. The paper lacks sufficient ablation experiments, making it difficult to attribute the performance gains to specific components.
  6. Statistical reliability and reproducibility details still need to be supplemented
Questions for authors
  1. The novelty of the proposed method is limited compared to GenCAD

As noted in Sections 2.3 and 5.2, GenCAD also aligns rendered images with parametric CAD programs via contrastive learning. The dual-encoder architecture, normalized embeddings, symmetric cross-entropy loss, and cosine similarity-based retrieval employed in this work are all well-established designs. While shifting the research focus toward retrieval and expanding the candidate pool holds experimental value, it is currently insufficient to constitute a clear methodological contribution.

If the primary contribution is a new benchmark or empirical findings, the experiments should be strengthened around this positioning.

  1. The absence of baselines under a unified protocol undermines the claims of performance advantage made in the abstract.

Table 2 explicitly states that results from different works are not directly comparable, yet the abstract still claims performance improvements over baseline methods. In particular, the 60.77% reported for GenCAD and the 69.60% achieved in this work cannot be directly compared given the differences in data, candidate sets, and correctness criteria. A larger candidate library does not eliminate these discrepancies.

  1. The experimental design for 'modified queries' is insufficient to solely verify modification robustness.

The second set of experiments in Section 4.2 draws queries from indices 0–4, comprising both original and modified images; thus, Table 1 reports results under mixed conditions. Consequently, the observed 2.20 percentage point drop in R@1 cannot be directly attributed to performance degradation caused solely by modified queries. Furthermore, since modified images were already sampled during the training phase, the current results do not demonstrate generalization capability to unseen modification types.

  1. The sequence equivalence evaluation is reasonable, but it requires a more complete analysis of data duplication and data splits.

Section 4.2 acknowledges that different model IDs may correspond to identical quantized command sequences. Although the training, validation, and test sets do not share overlapping IDs, this does not automatically guarantee that their sequence content or geometric content is disjoint. The paper should report the number of unique command sequences in each split, the extent of cross-split duplication, and the number of equivalent targets per query. This is a risk that needs to be verified, and the current materials are insufficient to conclude that no data leakage has occurred.

  1. The paper lacks sufficient ablation experiments, making it difficult to attribute the performance gains to specific components.

The current experiments primarily compare two query conditions and do not validate the individual contributions of pretrained visual features, visual encoder fine-tuning, the CAD Transformer, or the modified training data.

Additionally, please clarify how duplicate sequences are handled within a batch: if two CAD sequences are identical, Equation (5) treats only the diagonal pair as a positive sample, potentially treating other valid pairs as negatives. The impact of this design should be analyzed, or a multi-positive training strategy should be adopted.

  1. Statistical reliability and reproducibility details still need to be supplemented

The paper reports results from only a single training run with random seed 42, lacking statistics over multiple runs. For the small differences observed at R@5 and R@10, confidence intervals or paired statistical tests should be provided over the matched query set.

We recommend supplementing the mean and standard deviation across multiple training seeds, as well as details on the parameter embedding combination method, feed-forward layer configuration, image preprocessing and augmentation, learning rate scheduling, weight decay, long-sequence truncation ratio, training resources, and retrieval latency.

https://iclr.cc/Conferences/2027/AuthorGuidelines Recommended: Reproducibility statement It is important that the work published in ICLR is reproducible. Authors are strongly encouraged to include a paragraph-long Reproducibility Statement at the end of the main text (before references) to discuss the efforts that have been made to ensure reproducibility.

1 comment
  1. We agree that dual encoders, normalized embeddings, symmetric cross-entropy, and cosine similarity are established techniques; we do not claim these individually as novel. However, their use alone does not establish that our framework is a reformulation of GenCAD. Our work develops a retrieval-focused framework that jointly optimizes a pretrained CLIP RN50 image encoder and a Transformer CAD sequence encoder for cross-modal matching. Its contribution must be assessed through this formulation together with the empirical investigation of full-gallery retrieval and geometrically altered queries.

  2. On the same underlying dataset, our framework achieves 69.60% Recall@1 against 7,629 candidates, compared with GenCAD's reported 60.77% against 2,048 candidates. Achieving a higher score with a substantially larger gallery represents an improvement in reported retrieval performance, although it does not isolate the effect of methodological differences under a unified protocol. Table 2 explicitly acknowledges this limitation. The comparison should also be understood in the context of limited directly relevant prior work: GenCAD itself evaluates retrieval against a ResNet backbone baseline. We included additional works from the broader retrieval literature to give readers an intuition for retrieval performance in general, rather than to claim direct, controlled superiority over each method. The Results section explains the rationale and limitations of these comparisons individually. We will clarify that the abstract refers to improvements in reported scores, while preserving the distinction from a controlled comparison.

  3. The mixed-query setting is deliberate: it evaluates whether the model can retrieve the original CAD model from both canonical and modified inputs. Queries are sampled uniformly from indices 0–4, with only index 0 being canonical, so approximately 80% are expected to be modified. Crucially, the test images are never shown during training. Including modified images from the training split does not expose the model to the held-out test queries; it teaches the model to accommodate variations that may occur at inference. This introduces a potential trade-off: greater tolerance to modifications may improve modified-query retrieval while reducing discrimination between exact designs. Evaluating both canonical and mixed queries therefore assesses whether the model maintains performance across these two requirements. The results support generalization to held-out instances under the tested modification distribution.

  4. The split procedure separates duplicate models jointly: duplicates are assigned to the same split rather than independently distributed across training, validation, and test sets. Our separation therefore goes beyond checking for overlapping model IDs. Geometric similarity between different CAD models can still occur, as shared shapes and construction patterns are inherent to CAD datasets; similarity alone does not establish data leakage.

  5. We agree that the current experiments do not isolate the contributions of pretrained visual features, visual encoder fine-tuning, the CAD Transformer, or modified training data. Component-level ablations would strengthen the explanation of which choices drive performance. The reviewer is also correct about duplicate sequences within a batch: Equation (5) uses diagonal pairings as positives, so sequence-equivalent CAD models with different IDs are treated as negatives. A sequence-aware, multi-positive objective would address this limitation and is a concrete direction for improving the framework.

  6. We agree that a single training run with seed 42 does not establish variability across runs. Evaluation across multiple training seeds, reporting mean and standard deviation, together with confidence intervals or paired tests on matched queries, is the appropriate direction. We will clarify the single-run limitation and expand the implementation details and reproducibility statement, distinguishing the reported findings from statistical validation that remains to be conducted.

Reviewer_6q1cOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a contrastive learning framework for retrieving parametric CAD models from images. It aligns rendered images and CAD command sequences in a shared embedding space using a pretrained CLIP image encoder, a Transformer-based CAD encoder, and a symmetric contrastive loss. Experiments on the ABC/GenCAD dataset evaluate retrieval from both original and geometrically modified designs, achieving Recall@1 scores of 69.60% and 67.40%, respectively, over a gallery of 7,629 CAD models.

Critical strengths

(1) A practically relevant retrieval task. Retrieving parametric CAD models directly from images can support design reuse and reduce the effort required to locate existing designs. The paper also considers geometrically altered queries, where the visual input differs from the original design. This is a useful extension beyond retrieving CAD models from their canonical renderings.

(2) Evaluation on a full retrieval gallery. The experiments rank queries against all 7,629 held-out CAD models and report Recall@K under both canonical and altered-query conditions. This provides a meaningful assessment of retrieval performance in a relatively large candidate pool. The results offer a useful empirical reference for image-to-CAD retrieval, even though comparisons with existing methods under a common evaluation protocol are still lacking.

Critical weaknesses

(1) Limited methodological novelty. The proposed framework combines a pretrained CLIP image encoder, a Transformer-based CAD sequence encoder, and a standard symmetric contrastive objective. These are established components, and the paper does not introduce a substantially new architecture, learning objective, or CAD-specific representation mechanism. Given that prior work such as GenCAD has already explored contrastive alignment between rendered images and parametric CAD programs, the main contribution appears to be an extension of the evaluation setting rather than a significant methodological advance.

(2) Insufficient evidence of an advantage over prior work. Although the reported retrieval results are promising, comparisons with existing methods are contextual rather than controlled. Differences in query modality, gallery size, and relevance criteria make it difficult to determine whether the proposed framework genuinely improves upon the closest alternatives. The paper does not evaluate competing methods under a common test split, gallery, and correctness criterion. Consequently, the experiments demonstrate that the approach is feasible, but do not establish a clear advance over existing CAD retrieval methods.

Questions for authors

(1) Methodological contribution: Given the similarity to prior contrastive image-to-CAD retrieval methods such as GenCAD, what specific methodological advance does the proposed framework offer beyond combining a pretrained image encoder, a Transformer-based CAD encoder, and a standard symmetric contrastive objective?

(2) Comparison with prior work: Can the authors clarify what evidence supports an improvement over existing CAD retrieval methods, given the differences in query modalities, gallery sizes, and relevance criteria? What conclusions about the method's relative performance can be drawn without controlled comparisons under a common evaluation protocol?

1 comment
  1. We do not claim novelty in contrastive learning, image encoders, or Transformers individually. Our contribution lies in their integration and joint optimisation for image-to-parametric-CAD retrieval. The use of established components does not, by itself, establish that the resulting framework is merely a reformulation of prior work. Novelty should be assessed in terms of the proposed training strategy, representation alignment, and task formulation rather than the individual components in isolation.

  2. Our training strategy differs fundamentally from GenCAD. Our framework jointly fine-tunes the pretrained CLIP RN50 image encoder and Transformer-based CAD sequence encoder using a symmetric contrastive objective. In contrast, GenCAD first trains its CAD encoder through autoregressive sequence reconstruction and subsequently freezes it during contrastive alignment. Consequently, GenCAD aligns image representations with a fixed, reconstruction-trained CAD embedding space, whereas our approach allows both modalities to adapt jointly to the retrieval objective. This distinction directly affects how cross-modal representations are learned and cannot be dismissed as a minor architectural variation.

  3. The frameworks also differ in their visual representations and primary objectives. Our method uses single shaded renders and learns a shared embedding space explicitly optimised for bidirectional image-to-CAD similarity matching. GenCAD employs multi-view line drawings within a broader generative framework. These differences concern the information available to the image encoder, the learning objectives, and the intended application of the resulting representations. Reducing these methodological distinctions to an extension of the evaluation setting does not accurately characterise our contribution.

  4. The reviewer’s criticism regarding the breadth of comparisons should also be considered in the context of the existing literature. There are relatively few directly comparable frameworks addressing image-to-parametric-CAD retrieval. Notably, GenCAD itself evaluates retrieval against a ResNet-based baseline rather than an extensive collection of competing image-to-parametric-CAD retrieval methods. The absence of numerous directly comparable baselines is therefore not unique to our work, but reflects the current state of research in this specialised domain.

  5. We included additional studies from the broader retrieval literature to provide context for retrieval performance, not to suggest that their reported results constitute controlled, directly comparable benchmarks. The Results section explicitly discusses the motivation for including each study. We agree that differences in datasets, retrieval modalities, and evaluation protocols limit direct numerical comparisons. However, acknowledging these limitations does not invalidate the contextual relevance of the cited studies, nor does it justify disregarding them entirely.

  6. Our experimental results demonstrate substantial retrieval capability in a considerably larger candidate space. On the same underlying dataset, our framework achieves 69.60% Recall@1 across 7,629 gallery candidates, compared with GenCAD’s reported 60.77% across 2,048 candidates. We acknowledge that differences in query representation and relevance criteria prevent a strictly controlled comparison or an attribution of the numerical difference exclusively to our methodology. Nevertheless, the results demonstrate that our framework achieves strong retrieval performance under a substantially larger gallery configuration. These findings are meaningful empirical evidence of the proposed approach's effectiveness, even if they should not be interpreted as proof of superiority under identical evaluation conditions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.