acceptodds
Withdrawn from ICLR 2027

From Observation to Action: Structured Engineering Inference for Agent-Driven Design on Real CAD/CAE Infrastructure

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Engineering design optimization requires iterative simulation and modification, where each change must be grounded in reasoning about the simulation behavior it addresses. That cycle is exposed to software through computer-aided design and computer-aided engineering (CAD/CAE) tools, which recent tool-augmented agents can now operate end-to-end. However, operating tools is not the same as making informed design decisions. We present an engineering inference framework that structures the agent's reasoning from simulation observation to design action, with a 157-category controlled vocabulary of engineering failure modes, design modifications, and formal validation. We evaluate both the observation and the action step on real infrastructure. On observation, we extend prior benchmarks to tool-augmented and multi-view settings across 560 Finite Element Analysis questions, finding that interpretation accuracy ranges from 38-65% regardless of model tier, tool access, or viewpoints. On action, we compare an unstructured Baseline agent against an Ontology-guided agent on three self-terminating design tasks with live CAD/CAE solvers, spanning single-objective and coupled multi-physics settings. On the single-objective design, the ontology agent removes 9.32% of the starting mass compared with 3.72% for the baseline agent, with all stress and displacement limits satisfied. On the coupled multi-physics case, it reaches its best design at 91% of the starting mass, while the baseline finishes 23% heavier than the part it began with. The two agents differ in strategy rather than in effort: the Ontology-guided agent adds structure through ribs, while the Baseline agent adds bulk. Together, these experiments establish where in the inference loop current frontier models are effective, and where an explicit engineering vocabulary changes what the agent searches for.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
applied engineering design, agentic reasoning, structured reasoning, tool learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_W2A1OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work present an engineering inference framework that structures the agent's reasoning from simulation observation to design action. Specifically, previous benchmark OpenSeeSimE-Structural-Mini was used with extended setting, such as tool-augmented and multi-view settings. Evaluation results show that interpretation accuracy ranges from 38-65%, and for actions, authors compare an unstructured baseline agent with an ontology-guided agent on three self-terminating design tasks with live CAD/CAE solvers.

Critical strengths
  • Evaluating VLMs with external tools on engineering tasks are generally interesting research topic.
  • This paper focuses on both perception level and design level capabilties.
Critical weaknesses

My key concern is the novelty, the paper reads more like a report that extends an existing benchmark using different inference methodology. In addition, the evaluation in the "actions" consists primarily of case studies, which limits the broader insights that can be drawn from it.

Questions for authors

NA

Reviewer_MVvMOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper divides the design reasoning of CAD/CAE agent into five steps: Observe, Orient, Decide, Design, and Act, and connects simulation observation with geometric modification using an engineering vocabulary containing 157 classes and structured output. On the one hand, the author compared the image input and tool-assisted performance of 17 models on 550 FEA problems; on the other hand, the structural linkage, helicopter pitch arm and coupled T-bracket design were studied on Siemens NX/Simcenter 3D. The complete conditions achieved a better trade-off between quality and performance in some Opus 4.6 single runs, but repeated experiments, glossary ablation, and cross-model results did not give consistent design quality gains.

Critical strengths

The real CAD/CAE closed loop has practical value. The modification requires geometric reconstruction, remeshing, reapplication of loads and constraints, as well as actual solution, rather than merely comparing the generated text or rendering appearance. Appendix D explains how to rematch loads and supports based on stable geometric properties after geometric changes. Such engineering implementation details are helpful for understanding the real difficulties faced by agents. The paper publicly presented significant negative results. Appendix F reports that when the load is not locked, the Ontology reduces the load 5 times out of 10 runs, while the Baseline reduces it once. After locking the load, there is no longer a significant difference in the design results. The author did not retain the FoS improvement obtained by changing the task requirements as effective design benefits. This is a useful failure case and also reveals that the vocabulary may change the agent's understanding of feasible actions. The observation and evaluation covered 17 models, 550 questions, and five responses to each question. Appendix C reported the question categories, tool usage, and costs, and paired bootstrap and Holm corrections were used for comparisons between models. Another model experiment in Appendix I and the clear definition of the interpretation range of a single trajectory in Appendix L also provide important materials for evaluating the applicable boundaries of the method. These disclosures are commendable, although the main text needs to reflect them more fully.

Critical weaknesses
  • The helicopter pitch arm results in Table 1 and the coupled T-bracket results in Tables 2 and 15 have only one run per condition. The mass reduction of 9.32% versus 3.72%, as well as the superior design of 1,160kg versus 1,579kg, can prove these operational differences, but the average benefit, success rate or trajectory variance between conditions cannot be estimated. In contrast, there were no obvious design differences in the 10 seed experiments of the structural link (Table 13); In the 10 iterations and 10 seed experiments of the Helicopter pitch arm, the average mass changes of Baseline, Information and Ontology were -3.24%, -3.26% and -2.27%. The GPT-5.6 Sol five-seed experiment in Appendix I also did not report any significant differences in the quality of results across tasks (Table 16), and its coupled task is still a variant with the strict mass limit removed and cannot be used as a direct cross-model replication of the main text restricted task.

  • Section 3.3 states that the conditions only differ in the system prompt prompts, while the model and tools are the same. However, Table 3 in Appendix A divides Ontology into vocabulary, coupled, and full, with full adding equations, examples, parameter mappings, and tools. Appendix D clearly states that the Ontology condition has additional tools for querying and navigating the ontology. Therefore, it is necessary to distinguish between "the same CAD/CAE execution interface" and "exactly the same toolset". The use of vocabulary in single-target tasks and full in main text coupling cases should not be interpreted as a uniform processing condition either. More directly, the peak scores of Baseline, Information, vocabulary, coupled and full in Table 15 are 0.679, 0.759, 0.440, 0.457 and 0.778 respectively. The controlled vocabulary performs even worse on this trajectory after being added alone. full is only 0.019 higher than Information without labels, and each condition is run only once. The mass reduction advantage of full is indeed valuable, but these data cannot determine whether the improvement comes from engineering knowledge, equations and examples, knowledge queries, or the 157 types of labels themselves.

Questions for authors
  • Is the 20-iteration cap in Appendix H also applied to Table 2 of the main text? What is the actual reason for termination for each trajectory? Are the mass values in Table 2 and the abstract peak design or termination design?

  • What prompt words, equations, examples and tools do the Ontology conditions of the three cases respectively provide? Are there any results that have the same conditional knowledge content and query capabilities as full but do not require 157 types of labels?

  • Could you provide the independent repetition of the main text variable pitch arm and the strict quality upper limit T-bracket, as well as the best feasible design for each budget point? Has the restricted coupling task of GPT-5.6 Sol actually run, or have only variants been reported?

  • Where exactly is the data for multi-view conditions? How many independent images or simulation states do the 550 questions come from? How do the tool data correspond to the images? How to handle the failure of answer parsing during scoring?

Reviewer_c5H2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents an engineering inference framework in a real CAD/CAE environment. It structures reasoning from simulation observation to design action, using a 157-category controlled vocabulary of engineering failure modes, design modifications, and formal validation. It evaluates both the observation and the action step.

Critical strengths
  • It is a promising research problem. The paper distinguishes between simulation, tool execution, and engineering decision-making. Focusing on the quality of design decisions aligns more closely with real-world applications.

  • The use of CAD/CAE environments is a significant experimental contribution, not just simple text-based reasoning.

  • The paper provides an analysis of failure modes. For example, it shows when an agent change its approach or stop searching. The authors try to analyze how engineering strategies are formulated.

Critical weaknesses
  • One of my major concerns comes from the results of the ablation studies. They do not clearly demonstrate the ontology's contribution. In Table 15, for the T-Bracket task, including only the ontology vocabulary even reduces performance compared to the baseline. The best results, achieved using the full ontology, also incorporate additional engineering formulas and examples. The performance gains from the full ontology are comparable to those observed in the "information" case. It is hard to determine whether the performance gains stem from the reasoning or from the additional knowledge.

  • In Tables 1 and 2, the improvements are observed in the "Helicopter Pitch Arm" and "T-Bracket" tasks. However, each was evaluated with only one experimental run. Relying on one raises concerns regarding the robustness of the reported performance.

Questions for authors

In Section 4.3, the paper reports on metrics of chain completeness and vocabulary fidelity. These assess format completeness and category alignment. Can the authors evaluate the engineering correctness of each reasoning step? For example, whether the simulated interpretations in the "Observe" phase and the modification strategies in the "Design" phase are consistent with actual physical mechanisms? If not, do the generated inference traces truly reflect reliable engineering reasoning?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.