- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This work present an engineering inference framework that structures the agent's reasoning from simulation observation to design action. Specifically, previous benchmark OpenSeeSimE-Structural-Mini was used with extended setting, such as tool-augmented and multi-view settings. Evaluation results show that interpretation accuracy ranges from 38-65%, and for actions, authors compare an unstructured baseline agent with an ontology-guided agent on three self-terminating design tasks with live CAD/CAE solvers.
Critical strengths
- Evaluating VLMs with external tools on engineering tasks are generally interesting research topic.
- This paper focuses on both perception level and design level capabilties.
Critical weaknesses
My key concern is the novelty, the paper reads more like a report that extends an existing benchmark using different inference methodology. In addition, the evaluation in the "actions" consists primarily of case studies, which limits the broader insights that can be drawn from it.
Questions for authors
NA