acceptodds
Under review as a conference paper at ICLR 2027

IndustrialCAD-1400: Benchmarking Detailed Industrial Assemblies From Text

Abstract

Industrial assemblies require more than a recognizable shape: functional parts, interfaces, dimensions and service features must satisfy a detailed request together. We introduce IndustrialCAD-1400, a text-to-CAD benchmark for this industrial setting. Its 1,400 requests combine 400 equipment categories across 40 domains with 250 four-case design families. Family requests specify a median of 17 named component or configuration roles and six explicit numeric targets; controlled variants test whether dimensions change while shared structure is retained. Task construction draws on documented manufacturer information, owner specifications and engineering guidance. Models generate geometry programs from requests, and a common Blender environment executes them to produce scenes, optional measurements, and standardized views for evaluation. A request assessment of front, isometric, and top views measures visible component, arrangement, and interface fulfillment without requiring a single reference geometry. Five stratified panels preserve complete families while covering the benchmark exactly once, enabling balanced comparisons and separating industrial breadth from controlled numerical responsiveness. In conclusion, IndustrialCAD-1400 extends text-to-CAD from recognizable part geometry to industrial assembly synthesis under joint component, interface, spatial, and numerical specifications.

Primary area:
datasets and benchmarks
Keywords:
Benchmark, CAD

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2KHzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a benchmark of industrial parts and systems for CAD. It includes 1400 text descriptions of these parts intended for examining the ability of generative models to create 3D geometries that follow the text descriptions. The benchmark includes 40 categories and evaluates two tasks: category based generation and editing. The authors offer evaluation metrics and evaluates well known foundation models on these generation tasks.

Critical strengths

The benchmark and metrics presented are clear and can be useful for evaluating the ability of generative models to generate reliable designs and edit them in various industrial categories. The descriptions in the supplementary material seem detailed. I found the paper easy to follow and the benchmark seems to have good potential in my view.

As I am not very familiar with the field its hard for me to assess how big of a contribution of such benchmark is, and how much it is actually needed. The paper does provide a comparison to a previous benchmark which seems much less robust. As this is a benchmark paper technical correctness does not seem to be an issue and the proposed evaluation metrics seems reasonable to me. For these reasons, and assuming that there is an actual need for such benchmarks I was left with a positive impression.

Critical weaknesses

As this is a benchmark paper I did not find any critical weaknesses.

More explanation on how the examples were gathered and why (motivation, complexity, materials, etc.) is needed.

Adding more models to the presented evaluation could prove useful for future work.

Questions for authors

Can you explain the rationale of the selection and choices of the models in the benchmark.

Can you give a measure for the complexity of CAD models in the benchmark.

Reviewer_1xJwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

the authors propose a text-to-CAD benchmark for industrial assemblies. the benchmark consists of 1400 text prompts from 40 industrial domains, across 400 equipment categories. It is aimed at extending text-to-CAD from part geometry to industrial assembly synthesis under joint, interface, spatial, and numerical specifications. Two tasks are evaluated - category-based generation and controlled-variant generation. The models generate Blender scripts to render viewpoints, which are scored for conformity. Deepseek V4 Pro achieves the highest overall score.

Critical strengths
  1. the benchmark is well motivated and industry-relevant as moving from single geometry to an assembly is challenging

  2. the task design is strong choice as separating equipment diversity from controlled variation allows for more informative analysis than a single heterogeneous collection and shows a user the different capabilities being tested as these are two fundamentally different tasks.

Critical weaknesses
  1. its hard to follow the text at times and transitions between concepts are abrupt. the relationships among requests, categories, design families are not communicated as intuitively as they could be and makes it difficult to understand the benchmark structure. a clearer delineation of the various components would improve readability and make the experimental results easier to follow. this is consequential because if i cannot thoroughly understand, i may not be able to judge the paper completely.

  2. using visual conformity and blender geometry generation as proxies for engineering-valid CAD assemblies is limiting. the benchmark may reward visually plausible models that would not satisfy an industrial design requirement. Tighter claims about the scope are required as the current evaluation does not verify whether the generated assemblies have their specified dimensions, unless limitations are provided.

Questions for authors
  1. why is GLM-5.3 appearing in tables 3 to 6 but not mentioned anywhere in experimental setup description?

  2. how is functional correctness of the generated assemblies computed? because visual assessment alone does not guarantee this in practice

  3. are there any checks for component interface connectivity or joint mating relationships?

Reviewer_LS4qOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces IndustrialCAD-1400, a text-to-CAD benchmark containing 1,400 requests across 40 industrial domains. It evaluates category-based equipment generation and controlled numerical variations. Models generate Blender geometry programs, and outputs are primarily evaluated using VLM-based visual scores from three rendered views. The authors compare four language models and report their performance across equipment categories and design families.

Critical strengths
  1. Broad task coverage. The benchmark includes diverse industrial equipment categories and detailed assembly specifications.
  2. Structured evaluation setup. The controlled-variant families and standardized evaluation protocol support consistent model comparisons.
Critical weaknesses
  1. Lack of validated industrial CAD ground truth. The benchmark is constructed from industrial descriptions rather than validated real-world CAD designs. The paper does not establish whether the generated specifications correspond to feasible and functional engineering assemblies.

  2. Fundamentally insufficient evaluation. The primary metric relies on VLM judgments of three Blender renderings. Visual similarity cannot reliably verify dimensions, geometric constraints, mechanical connectivity, or functional correctness. Despite targeting complex industrial equipment, the benchmark lacks systematic engineering validation or simulation. Consequently, it is unclear whether the reported scores measure meaningful industrial CAD generation capabilities rather than visual plausibility.

Questions for authors
  1. How were the benchmark specifications validated against real-world industrial designs? Is there evidence that the requested assemblies are physically and functionally feasible?

  2. Why are geometric measurements and engineering constraints excluded from the primary evaluation metric?

  3. Without mechanical or functional validation, how can the benchmark distinguish usable industrial CAD assemblies from visually plausible but nonfunctional Blender models?

Comments on the paper

1 comment

i appreciate the rebuttal and thank you for agreeing to make the results consistent. i have raised my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.