acceptodds
Withdrawn from ICLR 2027

3DCodeVerse: Exploring 3D Worlds as Code

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We introduce 3DCodeVerse, a research ecosystem that enables coding agents to create editable and verifiable 3D content at scale. 3DCodeVerse comprises large-scale, high-quality datasets of executable 3D code, a coding harness for spatial reasoning and automated verification, and practical guidance for 3D coding. First, we construct 3DCodeVerse-2M, a large-scale curated dataset of standalone 3D executable code by collecting open-source data, converting specialized representations, and generating new programs with coding agents. Each program is packaged with the source files and associated assets required for execution in its declared environment, along with available descriptions and verification records. Second, we develop the 3DCodeVerse harness, a multi-agent directed acyclic graph framework for generating and refining 3D content. It uses execution checks, spatial measurements, and visual feedback to automatically evaluate and revise generated programs without human feedback during generation. Third, we conduct systematic empirical studies of 3D coding, establishing baselines with fine-tuned open models and deriving practical guidance through component-level analyses of our harness. Extensive experiments demonstrate that fine-tuning on our dataset significantly improves the 3D coding capabilities of open-source models. Notably, our 3DCodeVerse harness achieves a 90% win rate against Claude Design in pairwise human evaluations when both are utilized by Claude Opus 5.5.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Generation, Code Generation, Procedural Modeling, Agentic Vision, Agentic Generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZCVBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

3DCodeVerse combines a cross-runtime executable 3D program corpus (CAD, Blender, URDF, GLSL, Three.js, OpenSCAD and others), a multi-agent harness with planning, execution, spatial inspection and visual feedback, and open-model fine-tuning on the corpus. It reports large execution-rate gains from SFT/DPO (e.g., 0% to 84.9% for a 9B model and 51.9% to 92.9% for a 27B model on 3DCodeBench), paired object-quality improvements for the complete workflow, and a 90% win rate against Claude Design in pairwise human evaluation when both use Claude Opus 5.5. The resource integration and conversion validation are potentially valuable; the manuscript, however, contains inconsistent headline results and compares workflows with very different generation budgets.

Critical strengths

S1. A broad executable 3D-code resource with concrete conversion validation. The collection spans CAD, Blender, URDF, GLSL and Three.js, and it validates conversions rather than only aggregating sources: mean DeepCAD voxel IoU is 0.985 and 6,878/6,882 Articraft conversions pass the stated geometry criteria. A cross-dialect, provenance-tracked executable corpus is a genuine resource contribution even where the underlying mechanisms are not new.

S2. Most consistent results show substantial gains, and limitations are disclosed. Execution improves markedly after training (0% to 84.9% for 9B and 51.9% to 92.9% for 27B on 3DCodeBench), and paired intervals support object-quality gains for the complete workflow. The appendix openly reports representation duplication, quality tiers, unmatched budgets, limited ablations, oracle state and negative results, which makes the claims auditable.

Critical weaknesses

W1. Headline results are internally inconsistent and cannot be audited from the paper. Table 2 reports 74.5% Three.js execution for 9B +SFT, whereas the corresponding 270K run in Table 20 reports 32.5%, and the other five columns of that row (84.9/90.3/98.5/86.0/56.0) match Table 2 exactly, so the 270K run is the intended configuration. With a single-seed evaluation on 40 tasks and binary execution outcomes, rates should move in 2.5-point steps, so 74.5% also needs an explanation of its denominator or aggregation. Table 3 reports OpenSCAD F=.416 for 27B SFT+DPO against .314 in Table 20 (the other columns agree), which changes the negative-transfer conclusion. The abstract's quality/executability wording also blurs units: the 2,028,286 export rows multiply captions, representations and shader passes and are not independent programs (the catalogue has 269,597 records including variants), and the mapping from each headline to its 60/30/100-prompt subset and run snapshot is incomplete. Tables should be regenerated from a single manifest of checkpoints, mixtures, prompt IDs, seeds, runtimes and per-example predictions.

W2. The comparisons do not isolate the benefit of the harness, and novelty is not positioned against the closest mechanisms. Native baselines generate in a single pass without tools or feedback; generation cost is about $0.035 versus $3.312 per task (roughly 95x, 3.312/0.035 = 94.6), and Claude Design usage is not measured. The evidence therefore shows that a more expensive complete workflow improves quality, not that it beats budget-matched execute-repair, best-of-N or tool-enabled agents; none of the paired artifact-quality ablations in Table 26 separates from zero, and the scene/graphics studies use only 7-11 paired prompts with a negative mean for Sonnet graphics. The 90% preference result needs raw prompt/rater counts, W/L/T, tie handling and prompt-bootstrap intervals. SceneCraft (planning, spatial constraints, visual critique), BlenderLLM (script data, SFT, filtered self-improvement), 3DCodeBench (API migration, geometric validation, multi-turn feedback) and VIGA (editable programs with visual feedback) already cover the core mechanisms, so the defensible contribution is the cross-dialect resource and validated conversion; the independently added programs/sources and a modern-API adaptation baseline are needed to show what remains beyond them.

Questions for authors
  1. Which denominators, runs or checkpoints explain the Table 2/3/20 discrepancies and the 74.5% Three.js rate with N=40? Can corrected tables and a unified manifest be provided?
  2. What are the raw evaluation units and W/L/T counts behind the 90% preference result, and which prompt subset and version underlie each headline?
  3. Does the complete harness outperform ordinary execute-repair, SceneCraft-style workflows or best-of-N under equal generation cost and tool access?
  4. Relative to 3DCodeBench, BlenderLLM and the source datasets, which independently added programs, conversions and training gains remain, and how much geometric improvement survives a modern-API baseline?
Reviewer_qrQuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces 3DCodeVerse, a framework that models 3D content creation through executable program generation. The proposed framework consists of two main components. The first component is the 3DCodeVerse Harness, where a high-level LLM planner breaks down a task into a directed acyclic graph of sub-agents. During each iteration, the harness executes the generated code, runs geometric measurements along with connectivity checks, and renders multi-view images. A rubric-based VLM judge evaluates the rendered output, a repair agent refines the code based on the feedback, and the system retains the execution round with the highest score. This harness supports several coding backends, including Claude Code, Codex, and Gemini CLI. The second component is 3DCodeVerse 2M, a dataset covering six 3D programming languages, namely Blender, CadQuery, OpenSCAD, GLSL, Three.js, and URDF. The authors fine-tune Qwen3.5 9B and Qwen3.8 27B using LoRA on this corpus, while applying DPO to the 27B variant. Experimental results show that the proposed harness consistently improves human preference and automated judge scores across diverse domains, including static objects, articulated objects, scenes, and graphics. When paired with Claude Opus 5.5, the harness achieves a 90% win rate against Claude Design in human evaluations. Furthermore, fine-tuning open weights significantly boosts execution rate and geometric accuracy on 3DCodeBench and related evaluations.

Critical strengths
  1. The paper addresses a timely problem with strong motivation, as combining code execution, multi-view rendering, and geometric verification directly aligns with recent advances in agentic workflows and verifiable generation.

  2. The engineering execution is commendable, featuring an end-to-end pipeline for task planning, program execution, geometric measurement, rendering, automatic evaluation, and iterative repair across multiple agentic backends.

  3. The 3DCodeVerse 2M dataset covers an extensive range of 3D programming languages and domain tasks, providing valuable resources for the research community.

Critical weaknesses
  1. The DPO experiments are confusing. The main text does not explain how DPO is done or how preference pairs are selected, and the SFT vs SFT+DPO comparison is incomplete. Taken together with the appendix, DPO shows essentially no gain. Two main-text numbers also conflict with the appendix, in both cases turning a decline into a gain, so I cannot tell whether the fine-tuning conclusions hold. My confuse is detailed in Questions.
Questions for authors
  1. The filtering mechanism for DPO should really be explained in the main text instead of being left out. Also, the paper misses two very important baseline comparisons, which are Qwen 3.5 9B SFT vs Qwen 3.5 9B SFT+DPO and Qwen 3.5 27B SFT vs Qwen 3.5 27B SFT+DPO. The main text only shows Qwen 3.5 9B SFT and Qwen 3.5 27B SFT+DPO, and the authors didn't give any explanation for this. On top of that, these key results are scattered across the main body and the appendix, and taken together, the actual gain from DPO looks pretty small, which doesn't match the paper's claims.

  2. There are also two conflicts between the main text and the appendix. In Table 3, the OpenSCAD [email protected] value for 27B SFT+DPO shows that performance got better after training, but in Table 20 of the appendix, the same experiment shows a performance drop. I am not sure which result to trust. The same issue happens with the Three.js execution rate for 9B SFT between Table 3 and Table 20, so please check if these are typos.

  3. What model was used as the evaluator during the harness iterations? This key detail is missing from the main text. Is it the same model used for the final evaluation? Is there any risk of overfitting to the evaluator's preferences and losing generalization capability?

  4. The experimental setup for human evaluation is not very transparent either.

  5. In Figure 4, using the exact same Opus 5.5 model, why is there a 40-point gap in Native Harness performance between Claude Code and Claude Design? Meanwhile, the appendix mentions that the baseline "generates code in a single pass without tools or feedback", which is just a single-pass API call with Opus 5.5 getting 77.2. So my question is whether the Claude Code framework was actually used in Figure 4 or not. Either way, this huge gap is confusing, so please clarify this.

Reviewer_GasbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

3DCodeVerse combines a corpus of executable 3D programs with a multi-agent generation and repair framework. The framework integrates planning, execution checks, geometric measurements, rendering, and VLM feedback across several coding environments. The paper evaluates fine-tuned models and compares the complete workflow with one-shot generation and Claude Design.

The submission demonstrates substantial implementation effort. However, it does not clearly establish the research question beyond building a broader integrated system, or demonstrate which new insight or mechanism accounts for the reported improvements.

Critical strengths
  • The integration of multiple 3D coding environments, executable assets, provenance, and geometric checks could be a useful community resource. The implementation appendix provides substantial practical detail.
  • The fine-tuning experiments provide evidence that domain-specific training improves program executability on several evaluated suites. The appendix also usefully discusses failure modes and the distinction between execution success and geometric fidelity.
Critical weaknesses
  • The research contribution and its significance are not convincingly established. The paper presents a broad collection of engineering components, but does not formulate and answer a sufficiently focused research question about them. Combining planning, tools, execution feedback, and visual feedback is practically useful; the experiments need to establish what is learned from this particular combination beyond the usefulness of a more elaborate workflow.

The primary baselines are described in Appendix C.3 as single-pass generation without tools or feedback, despite being labeled by native agent backends. The proposed workflow receives substantially more resources: Table 16 reports approximately 3.3123.312 versus 0.0350.035 per Gemini output. This does not establish an advantage over tool-enabled native agents or simpler repair loops at comparable budgets. The component studies likewise do not establish which proposed mechanisms improve artifact quality.

A systems or dataset contribution can be sufficient for publication, but its significance must be demonstrated through a reliable comparison or a clearly established resource advantage. The current evidence does not adequately establish either. The qualitative showcases and breadth of implementation do not resolve this gap.

  • The central evaluation is not reliable enough to support the main claims. There are substantive conflicts between the main results and the detailed appendix. The fine-tuned 9B Three.js execution rate is 74.5% in Table 2 but 32.5% in the corresponding Table 20 row. The 27B SFT+DPO OpenSCAD F-score is 0.416 in Table 3 but 0.314 in Table 20. These differences change whether the results indicate improvement or regression.

The human-preference results are also insufficiently documented. The manuscript does not provide the evaluator counts, ratings per pair, aggregation details, and comparison-specific denominators needed to assess the reported percentages. The relationship between the main evaluation sizes and the smaller subsets in the appendix is unclear.

These are central concerns because human preference is used to validate an automated evaluator that shares its model and rubric family with the generation workflow. Until the results are reconciled and the human study is auditable, the headline conclusions cannot be assessed confidently.

Questions for authors
  • What specific research conclusion does the submission establish beyond the practical usefulness of combining existing generation and verification components? Which controlled result supports that conclusion?
  • Are there existing comparisons against tool-enabled native agents or simpler repair workflows at comparable resource budgets? If not, the current results appear to establish improvement over one-shot generation, rather than the specific advantage of the proposed harness.
  • Which values are correct for the conflicting Three.js execution and OpenSCAD fidelity results? Please reconcile the main tables and appendix.
  • What are the exact human-evaluation sample sizes, evaluator counts, ratings per pair, aggregation rules, W/L/T counts, and confidence intervals for the headline comparisons?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.