acceptodds
Under review as a conference paper at ICLR 2027

Exploiting Uncertainty Aware Reasoning Ability of Large Language Models for CAD Drawing Completion

Abstract

2D CAD drawings are essential documents for the design of industrial products. Their drawing process requires rigorous reliability assurance. However, traditional deep learning methods fail to provide convincing explanations when dealing with complex topologies. In this work, we propose a novel paradigm that leverages the powerful reasoning capabilities of LLM to drive cross-modal drawing completion, while enhancing the interpretability of LLM by designing a novel uncertainty-aware module. CAD drawings are encoded as text symbol sequences, and are subsequently integrated into instructional prompts to fine-tune LLM. Greedy Coordinate Gradient (GCG) is used to train adversarial suffixes, which are then used to execute adversarial attacks against LLM. Following this, we calibrate the LLM by utilizing the consistency metric of model outputs under perturbation. In order to quantify uncertainty, we devised a feature that incorporates a variety of metrics for evaluating intrinsic confidence to train the classifier. Our system achieves 93.6% accuracy in primitive classification and 70.9% accuracy in attribute prediction. In uncertainty analysis, our method first achieves efficient calibration, requiring only six perturbations and being applicable to both white-box and black-box models, subsequently achieving an AUC score of 0.919 in predicting model reliability. Experiments confirm our method's adaptability to mainstream models, and efficacy in addressing limited-sample learning scenarios and long-tail data distributions. Furthermore, we contribute a sketch dataset composed of instruction-prompt pairs, which includes 200,000 high-quality sketches, with the aim of contributing to research in this field.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
CAD, Large Language Model, Cross-modal, Instructional Prompts, Greedy Coordinate Gradient, Logistic Regression

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_fYDMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies 2D CAD drawing completion, where the goal is to infer missing primitives and their attributes from an existing sketch with geometric constraints. The authors construct a 200K-sketch instruction-prompt dataset from SketchGraphs, encode sketches as text sequences, and fine-tune LLMs with instruction prompts in a two-stage pipeline: first predicting the missing primitive type, then generating its attributes such as coordinates or radius. They also introduce an uncertainty-aware module based on adversarial perturbations and confidence features to estimate prediction reliability. Experiments show strong primitive classification, more limited but meaningful attribute prediction, and improved reliability estimation over several confidence baselines.

Critical strengths
  1. The paper addresses a useful problem: applying LLMs to 2D CAD drawing completion. Representing CAD sketches as text sequences and fine-tuning LLMs with instruction prompts is a relevant direction for connecting structured CAD data with language-model reasoning.
  2. The uncertainty-aware component is a valuable addition. In CAD applications, it is not enough to generate a completion. The system should also estimate whether the prediction is reliable. The paper’s use of adversarial perturbations and confidence features for reliability estimation is interesting and meaningful for building more reliable CAD assistance systems.
Critical weaknesses
  1. The CAD completion results are not fully convincing. Primitive-type classification is strong, but attribute prediction is much weaker, especially at the sketch level. Since usable CAD completion depends on accurate attributes, this limits the strength of the paper’s main claim.
  2. The experimental validation does not fully support the broader claim about LLM reasoning for CAD. The main comparisons are mostly with generic graph-learning baselines, rather than methods more directly designed for CAD sketch completion, parametric sketch inference, or CAD generation. As a result, it is unclear whether the proposed LLM-based formulation is stronger than existing CAD-focused approaches.
Questions for authors
  1. The main comparisons are mostly with generic graph-learning baselines for primitive classification. Could the authors clarify why more directly related CAD sketch completion, parametric sketch inference, or CAD generation methods are not included as main baselines?
  2. How is the train/test split constructed from SketchGraphs? In particular, is there any check for near-duplicate sketches or highly similar templates across the split? This is important for assessing whether the reported performance reflects generalization rather than memorization of common sketch patterns.
Reviewer_JYgbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a scheme to use LLM for 2D CAD drawing. The drawing procedure is first converted into NLP sequences describing geometric elements and constraints. A LLM is then fine-tuned using LoRA on such NLP descriptions. For the drawing, the pipeline consists of two stages: 1. classification on missing geometrc components to be added; 2. estimating the coordinates and parameters for such components. The authors also propose reliability estimation on the LLM's drawing component predictions using a variety of uncertainty estimators or indicators.

Critical strengths
  1. The proposed approach is practical, with promising empirical performance. Specifically, as shown by the results, the LLM based approach substantially outperforms graph neural network baselines in component classification.
  2. Instead of relying on LLM's only uncertainty estimation, which is often not calibrated, the authors propose the framework to quantitatively estimate the uncertainty, mainly based on adversarial perturbation based consistency, along with several other uncertainty metrics.
Critical weaknesses
  1. The method novelty is very limited. Most components of the method, including text serialization of task description, LoRA fine-tuning, adversarial perturbations, logistic regressions, are all well-established and adapted techniques. The method is purely application and practical in nature that it asembles these components for CAD completion. This fundamental nature of the paper severly limits its competitiveness to be considered for a top ML research venue.
  2. Experiments are limited. For one, the baselines are primarily based on graph-neural-net architectures. How about LLM-based baselines? Secondly, for uncertainty calibration, besides AUC, there are other metrics that should be included (such as ECE, Brier scores, etc.).
Questions for authors

Besides classification and attribute accuracy, as well as the uncertainty quantification, are there other metrics or tests that can evaluate the completed drawings entirely for its overall constraint satisfaction or practicality? The metrics all focus on individual aspects of the drawing pipeline.

Reviewer_VJCUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces an LLM-supported approach to enabling the completion of 2D CAD drawings, aiming to improve interpretability and reliability in complex topological design. In particular, CAD drawings are converted into text symbol sequences and fine-tuned using instructional prompts; Greedy Coordinate Gradient (GCG) is employed for adversarial training, while an uncertainty component is used to assess the confidence of the LLM output. The approach has been evaluated on a subset of the SketchGraphs dataset, which the authors calibrated, yielding interesting results.

Critical strengths
  • The paper tackles an interesting problem with potential practical value

  • The calibrated dataset is an asset that could be used by the community

Critical weaknesses
  • Motivation: The motivation underpinning the paper is not clearly presented, leaving many questions about the overall value and potential impact of the approach. This needs to be strengthened significantly.

  • Rigour: While the use of GCG and LLMs is, to some extent, interesting, the former has been used (as the authors correctly state) for jailbreaking LLMs; thus, their use and relevance in CAD drawing completion scenarios is not sufficiently substantiated. Similarly, there is confusion about the use of LLMs rather than geometric deep learning paradigms, which natively preserve spatial and topological relationships without loss (e.g., sequential analysis).

  • Soundness: The core part of the approach primarily combines existing LLM fine-tuning pipelines with some standard confidence estimation formulae. A formal foundation to support the theoretical base of the approach is lacking, failing to substantiate the soundness of the approach. Furthermore, no theoretical justification is provided regarding why six textual perturbations are mathematically sufficient to bound or quantify model uncertainty across complex, multi-modal topological spaces.

  • Evaluation: Although the evaluation is quite informative, it has several limitations. First, no comparison against SoTA CAD completion baselines is provided (e.g., Text2CAD in NeurIPS 2024); hence, the evaluation results fail to corroborate the text for the benefits of the approach. Likewise, while interpretability is highlighted as a contribution of the paper, its evaluation focuses on performance metrics, lacking a user study to assess whether the interpretable feature is actually meaningful.

Questions for authors
  • What is the motivation and novelty of the work?

  • Is there any theoretical foundation that can be used to support the soundness and rigour of the approach?

  • How do you assess the interpretability of the approach?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.