Measure Before You Code: Tool-Augmented CAD Program Synthesis from Point Clouds

Synthesizing executable CAD programs from point clouds requires recovering both discrete modeling sequences and continuous geometric parameters. Recent point-to-CAD methods rely on multimodal large language models (MLLMs) to compress point clouds into latent tokens and infer parametric programs end-to-end. While these methods show impressive progress in recovering overall shape structure, this paradigm overlooks the primary advantage of point clouds over 2D images: direct access to explicit 3D coordinates. Dimensions are therefore predicted rather than measured, which makes them prone to error. We propose QM-CAD (Query-and-Measure CAD), an agentic framework guided by a simple principle: measuring before coding. An MLLM drafts a modeling plan that directs geometric queries to the raw point cloud. A query-driven segmenter identifies the surface points relevant to each planned operation. A geometry fitter returns measured parameters and point projections. The MLLM uses this numerical and visual feedback to generate the final CAD program. To support this paradigm, we construct an automated pipeline for synthesizing plan-measure-generate trajectories with real tool executions. Supervised fine-tuning on these trajectories teaches the model to formulate queries and use their feedback. Reinforcement learning then refines the model, using the solid Intersection-over-Union (IoU) of executed CAD models as the reward. Experiments on DeepCAD and Fusion360 show that QM-CAD achieves higher geometric reconstruction accuracy than recent point-to-CAD methods while keeping a low invalidity ratio.