Exploiting Uncertainty Aware Reasoning Ability of Large Language Models for CAD Drawing Completion

2D CAD drawings are essential documents for the design of industrial products. Their drawing process requires rigorous reliability assurance. However, traditional deep learning methods fail to provide convincing explanations when dealing with complex topologies. In this work, we propose a novel paradigm that leverages the powerful reasoning capabilities of LLM to drive cross-modal drawing completion, while enhancing the interpretability of LLM by designing a novel uncertainty-aware module. CAD drawings are encoded as text symbol sequences, and are subsequently integrated into instructional prompts to fine-tune LLM. Greedy Coordinate Gradient (GCG) is used to train adversarial suffixes, which are then used to execute adversarial attacks against LLM. Following this, we calibrate the LLM by utilizing the consistency metric of model outputs under perturbation. In order to quantify uncertainty, we devised a feature that incorporates a variety of metrics for evaluating intrinsic confidence to train the classifier. Our system achieves 93.6% accuracy in primitive classification and 70.9% accuracy in attribute prediction. In uncertainty analysis, our method first achieves efficient calibration, requiring only six perturbations and being applicable to both white-box and black-box models, subsequently achieving an AUC score of 0.919 in predicting model reliability. Experiments confirm our method's adaptability to mainstream models, and efficacy in addressing limited-sample learning scenarios and long-tail data distributions. Furthermore, we contribute a sketch dataset composed of instruction-prompt pairs, which includes 200,000 high-quality sketches, with the aim of contributing to research in this field.