acceptodds
Withdrawn from ICLR 2027

aDSL: Agentic 3D Creation via Joint Agent-Program Design

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Programmatic representations provide a compelling paradigm for 3D content creation, enabling fine-grained edits, interpretability, and explicit structural control. Yet, agentic workflows that rely on large language models (LLMs) to author 3D programs remain brittle, often failing to translate high-level intent into consistent low-level geometry. We attribute this fragility to a mismatch between existing programmatic interfaces and the reasoning strengths of LLMs, which favor semantic structure and spatial relations over fragile numeric choices. In this paper, we jointly design an Agent-centric Domain-Specific Language (aDSL) and a role-specialized multi-agent system to close this gap. aDSL bridges semantic logic and geometric constraints by emphasizing composability and spatial reasoning; it enables agents to manipulate geometry through relational operators instead of brittle absolute coordinates. Building on aDSL, our training-free multi-agent system follows a Plan-Execute-Critic loop to decompose requests, synthesize code, and iteratively repair errors and constraint violations using execution feedback. Experiments show that this co-design improves robustness, controllability, and faithfulness to user intent. Our method outperforms prior LLM-based baselines on text-to-shape and image-to-shape tasks while preserving explicit structure, editability, and interpretability. It also enables downstream applications such as articulated object creation and structured scene composition.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM Agents, 3D Generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vZCuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ADSL, a structured language and multi-agent workflow for generating editable 3D assets from text or images. Its main idea is to represent object parts and spatial relations explicitly, allowing the agents to generate and iteratively correct 3D programs. Experiments on text-to-shape and image-to-shape tasks show improvements over existing code-generation baselines, and ablations support the proposed spatial operators and refinement loop. The paper also demonstrates editing, articulation, memory reuse, and scene composition, although these capabilities are evaluated mainly through qualitative examples.

Critical strengths

The paper’s main strengths are its coherent system design and solid empirical performance.

  1. Joint agent–program design: The explicit DSL gives all agents a shared representation for planning, generation, verification, and repair, directly addressing the brittleness of low-level 3D code generation.
  2. Strong empirical results: The method consistently improves over code-generation baselines across several datasets, and the ablations show that both the spatial operators and iterative refinement contribute to these gains.
Critical weaknesses

The main weaknesses are the mismatch between the paper’s broad capability claims and its evaluation, and the limited isolation of the framework’s contribution.

  1. Insufficient structural and functional evaluation: Although the paper claims reliable support/contact relations, structural validity, mechanical function, editing, and scene composition, nearly all quality metrics are computed from rendered images. Execution success only measures whether a renderable mesh is produced. The paper provides no direct evaluation of collisions, physical stability, articulation, spatial-constraint satisfaction, or edit preservation.
  2. Base-model and baseline confounds: Most experiments rely on Gemini 3.1 Pro, while the cross-model analysis is small and qualitative. It is therefore unclear how much of the reported capability comes from the proposed framework rather than the underlying model.
Questions for authors

My questions are as follows: Clarification:

  1. Meaning of structural validity: The paper discusses support/contact, structural validity, mechanical function, functional correctness, and reducing floating or interpenetrating parts. Are these intended as geometric-placement claims or as claims of physical correctness?
  2. Physical verification: Beyond inspecting rendered views, does the system perform collision detection, stability analysis, physics simulation, or joint-motion validation? If not, please clarify the scope of the physical and functional claims.
  3. Scene-generation scope: Is scene generation a core capability of the method or only a qualitative downstream application? Figure 6 demonstrates scene composition, but the main evaluation focuses on object shapes.
  4. Editing mechanism: For a follow-up instruction such as adding a new object or part, does the Planner update its decomposition and constraints, or does the Coder directly rewrite the existing program? How are affected dependencies identified?
  5. Failure behavior: What are the most common semantic or structural failures, including cases where the system produces a renderable mesh that does not satisfy the prompt? How often does the system reach the ten-round refinement limit?

Evidence needed for current claims:

  1. Evaluation metrics: CLIP, VQAScore, FID, and the human study primarily evaluate rendered appearance, while execution success only checks whether a renderable mesh is produced. Can the authors provide direct measurements of spatial-relation accuracy, support/contact correctness, collision or interpenetration, articulation, and constraint satisfaction?
  2. Editing evaluation: Can the authors quantify whether an edit is correctly applied while preserving unaffected geometry and constraints? The current examples demonstrate parameter changes, but do not establish reliable insertion of new semantic objects or parts.
  3. Scene-level comparison: If scene generation is a substantive claim, how does the method compare with more recent work such as I-Scene or other agentic methods, particularly in spatial-relation accuracy, collision, stability, and scene completeness? If the tasks are considered incomparable, please explain the intended boundary of the claim.
  4. Backbone dependence: Since all main agents use Gemini 3.1 Pro and the GPT-6 Astra comparison is qualitative, how much of the performance comes from the proposed framework rather than the base model? Are results available using multiple backbones under matched inference budgets?
Reviewer_mqoGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a training-free framework for programmatic 3D generation by co-designing an Agent-centric Domain-Specific Language (ADSL) and a multi-agent system. To avoid brittle coordinate arithmetic in LLMs, ADSL wraps basic CSG primitives into a hierarchical Python Asset class and provides declarative spatial operators. The agents iteratively generate, execute, and repair ADSL programs using multi-view renders and code-level verification. Experiments on text-to-shape and image-to-shape benchmarks show improvements over existing LLM-based code generation baselines, alongside qualitative demonstrations on articulated objects, shape editing, and scene layout.

Critical strengths
  1. Replacing manual coordinate calculations with declarative AABB and anchor-based operators aligns well with LLM capabilities. The ablation study (Tab. 3) clearly shows that this abstraction reduces self-correction rounds while improving semantic alignment and execution success rate.

  2. Embedding the DSL in Python classes provides a clean hierarchical representation that unifies part assembly, URDF joint definition, and multi-object scene composition within a single interpretable and editable format.

Critical weaknesses
  1. The geometric expressiveness of ADSL is overly restricted, relying solely on three basic primitives (Cube, Sphere, Cylinder) and CSG boolean operations. Unlike parametric CAD paradigms (e.g., B-Rep/CadQuery, Text2CAD, CAD-Coder) that support sketch-and-extrude, revolve, loft, or fillets, ADSL produces coarse, blocky meshes that cannot be directly used in industrial or production pipelines. Although Section C.4 uses ADSL meshes to condition Trellis via SpaceControl for better visual quality, doing so turns the output back into a monolithic mesh and loses the part hierarchy, programmatic editability, and articulation that motivate ADSL.

  2. The method sits in an awkward middle ground across its target tasks. On single objects, its geometric detail falls behind CAD-based or specialized procedural systems. Meanwhile, its downstream extensions to scene layout and articulated objects remain superficial compared to dedicated methods in those fields. Specifically, the scene composition is only shown qualitatively on two simple rooms using basic AABB alignment, lacking the global constraint solving, collision optimization, and walkability reasoning established in dedicated scene generation works (e.g., Holodeck, LayoutVLM, SceneCraft). Similarly, articulated object generation relies only on static code inspection of coordinate frames without physics simulation validation or self-collision refinement (as done in ArtLLM or URDF-Anything), offering no guarantee that complex joints can physically move without interpenetration.

Questions for authors
  1. In Section C.4 (Fig. 12), when ADSL is combined with SpaceControl + Trellis to obtain high-fidelity meshes, is there any way to retain the programmatic part hierarchy, editability, or URDF joints in the final mesh?

  2. Since the spatial operators primarily rely on world-space AABBs, how does the system handle tight contacts between rotated, non-axis-aligned parts? Furthermore, have the generated URDF assets been tested in a physics simulator (e.g., Isaac Sim or PyBullet) to measure self-collision or joint validity?

Suggestions for Revision:

  1. Please expand the discussion comparing ADSL with parametric CAD workflows. It would also be helpful to discuss how this work relates conceptually to concurrent procedural modeling work such as Procedura (Agentic 3D Modeling with Procedural Control), which uses incremental part-by-part OpenSCAD generation, mechanical mate solving, and mesh-level contact/connectivity verification gates. (Note: As Procedura is concurrent work, not comparing against it does not negatively affect my rating.)
Reviewer_5D1rOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the challenge of creating programmatic 3D assets using Large Language Models. In particular, The authors introduce a python based agent-centric-DSL, a DSL designed for agentic creation, which provide various geometric primitives such as booleans, 3D primitive shapes, symmetry groups, and shape assembly (or partitioning). Additionally, the authors introduce an LLM-harness, a "Plan-Execute-Critic" loop, to improve the LLM's ability to plan the task, as well as to improve design via verification, and iterative feedback loops. The authors compare this method to other LLM-based programmatic 3D asset generators, as well as to generative models which directly output 3D implicit fields (rather than programs). While, the proposed method achieves higher metric score on fidelity to input text (via clip score) vs the other methods, the qualitative results remain unimpressive.

Critical strengths

The paper's exposition is smooth. It's easy to read. The paper's appendix provides a lot of informative content, including the prompts used in the system.

Critical weaknesses

There are two major weaknesses that make it difficult for me to recommend acceptance: (1) insufficient experimental evaluation and (2) poorly justified contributions.

  1. Insufficient experimental evaluation

Comparison with direct Blender usage. The paper compares direct Blender usage against the proposed DSL and harness using Gemini 3.1 Pro. However, stronger and potentially cheaper frontier models (e.g., GPT-6, Claude Sonnet 5.5) may be able to use Blender directly without either component. It is therefore unclear whether the proposed approach provides meaningful benefits across the current spectrum of model capabilities and costs. If the intended advantage is for smaller or open-weight models, this should be demonstrated experimentally.

Missing comparisons with recent agentic approaches. The paper does not adequately compare against recent methods that use AI agents to generate programmatic 3D representations, such as Re-thinking Inverse Graphics with Large Language Models (Kulits et al.), VIGA (Yin et al.). These methods explore closely related ideas involving program synthesis, iterative refinement, and agentic 3D generation. Without such comparisons, it is difficult to establish the advantages of the proposed approach over existing work.

Outdated 3D generation baselines. The choice of TRELLIS is difficult to justify when TRELLIS.2 and several stronger 3D generative models have been available for months. Furthermore, the reported CLIPScore and VQAScore improvements over TRELLIS are surprising given the qualitative results in Figure 3, where TRELLIS produces considerably more realistic and detailed assets. The authors should investigate this discrepancy and evaluate against stronger contemporary baselines.

  1. Poorly justified contributions

The paper proposes two main contributions: a DSL for agentic 3D generation and a multi-agent orchestration framework. However, neither is sufficiently motivated relative to existing approaches.

The DSL. Several prior works have explored specialized languages for 3D generation (e.g., ShapeAssembly, Jones et al.; ShapeCoder, Jones et al.; 3D Shape Programs, Tian et al.). Meanwhile, increasingly capable models can directly use established and expressive modeling environments such as Blender and CAD scripting languages.

The proposed DSL appears primarily to provide wrappers around common modeling operations, while lacking basic procedural capabilities such as instancing objects along curves or procedural noise. What does this language enable that existing scripting environments do not? More importantly, what generalizable insight about language design for agentic modeling does it offer?

The multi-agent harness. Modern frontier models can increasingly decompose tasks, invoke tools, and organize iterative workflows themselves. It is therefore unclear whether a manually designed orchestration framework is necessary. A strong baseline would simply instruct a capable model to follow a plan–execute–critique loop using Blender directly. Without such comparisons, it is difficult to distinguish the benefits of the proposed harness from those of generic agentic tool use.

Broader significance. My central concern is that the paper does not adequately establish why these specialized abstractions remain valuable as foundation models become more capable and less expensive.

The current evidence primarily establishes that Gemini 3.1 Pro benefits from the proposed DSL and harness. This is a useful observation, but insufficient to establish a broadly relevant methodological contribution.

Questions for authors
  1. Does the proposed DSL or harness provide similar benefits when coupled with a more capable model (yet cheaper) model choice - Chat GPT Sol 6 or Claude Sonnet 5.5 or a open source model such as Qwen 3.8 Max?

  2. Can the authors position this contribution in the context of the current frontier models which have become exceptionally capable of using blender, and recreating high quality 3D scenes and objects?

  3. Do the authors believe that the proposed harness is useful in the new paradigm where a frontier level model does the planning and a lower tier model potentially fills in the details. I.e. custom harness free approaches which simply allow the a master agent to create sub-agents and perform the task directly?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.