- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces InfiniDream, a VLM-driven framework for generating editable procedural assets and composing them into 3D worlds. The asset side relies on Symbolically Constrained Editing (SCE), which starts by retrieving an Infinigen factory, builds a structured code summary and editing boundary, and then extends the factory through planning and render-guided refinement. For scene generation, the key idea is to separate semantic planning from deterministic geometric placement, using frustum-guided candidate search together with constraint filtering.
Critical strengths
1.The authors cleverly address the limitation of existing VLM-guided procedural generation. Instead of forcing the VLM to directly predict continuous 3D coordinates (which is error-prone), they introduce a Deterministic Frustum-Guided Grid Placement mechanism. By delegating semantic reasoning (where to put it, what to put) to the VLM and continuous geometric computation (exact 3D coordinates, collision checking) to a deterministic algorithm, they achieve much more reliable object placement.
2.The proposed Symbolically Constrained Editing (SCE) represents a meaningful step forward for open-ended asset generation. By allowing the VLM to edit the underlying procedural code while maintaining the original factory's function signatures and seed-based randomness, InfiniDream successfully generates novel, photorealistic assets that were previously impossible with predefined factories.
Critical weaknesses
I find the pipeline quite heavy. It relies on multiple VLM calls (Planner, Director, Reviser) plus up to six rounds of iterative code refinement with Blender rendering, but the paper doesn't say much about the actual computational cost or wall-clock time. If it takes tens of minutes or hours to generate a single scene, the practical scalability is severely limited.
Questions for authors
1.Can you test InfiniDream without the 70-scene retrieval library and compare it against a baseline using the same initial coarse scenes? I want to know how much of the improvement is genuinely generated versus just picked from Infinigen.
2.Can you break down the benchmark by difficulty (parameter changes, new structures, major extensions) and report success rates for each? This would clarify where the "open-ended" claim actually holds up.
3.What's the end-to-end generation time and model-call cost, split across library prep, asset code generation, placement, and rendering? With multiple VLM calls and up to six rendering rounds, a breakdown is important to assess practical scalability.
If the authors can address these questions, I would be happy to raise my score.
5 comments
Response to Q1: Scene Picker Ablation and Same-Initialization (2/3)
To distinguish the contributions of scene retrieval and actual generation, we conducted two additional experiments on the same 20 test scenes.
1. Scene Picker Ablation: We compare three initializations: (1) Full, selecting the initial scene from the 70-scene library; (2) Random, generating scenes of the same biome on the fly with five new random seeds outside the library; and (3) Minimal Biome, keeping only the terrain and environment of the selected scene. InfiniDream remains robust in both alternative settings (CLIP 29.45 and 28.60 vs. 29.64 for Full), showing that the richness and diversity of the final scenes come from the high-quality assets that InfiniDream generates, rather than from what is picked from Infinigen.
2. Same-Initialization Comparison: We provide VLM Harness and Code2Worlds with the exact same initial scenes selected by InfiniDream, including their Blender files and procedural code. InfiniDream outperforms both baselines across all four metrics, so its advantage holds under identical initialization and is not attributable to the Scene Picker alone. A general VLM agent has difficulty handling the large procedural code of the initial scene, and Code2Worlds is restricted to adjusting scene-level and object-level parameters, which limits what it can edit.
Detailed experimental settings and visual comparisons are included in Appendix O (Table 11, Figs. 21–23).
Scene Picker Ablation
| Method | Initialization | CLIP ↑ | SigLIP2 ↑ | DINOv3 ↑ | IR ↑ |
|---|---|---|---|---|---|
| Ours (Minimal) | Basic biome only | 29.45 | 0.142 | 0.350 | −0.736 |
| Ours (Random) | 5 new random seeds | 28.60 | 0.139 | 0.339 | −1.042 |
| Ours (Full) | 70-scene library | 29.64 | 0.143 | 0.361 | −0.877 |
Same-Initialization Comparison
| Method | Initialization | CLIP ↑ | SigLIP2 ↑ | DINOv3 ↑ | IR ↑ |
|---|---|---|---|---|---|
| VLM Harness | Matched initial scenes | 28.26 | 0.136 | 0.351 | −1.157 |
| Code2Worlds | Matched initial scenes | 24.15 | 0.082 | 0.157 | −1.980 |
| Ours (Full) | Matched initial scenes | 29.64 | 0.143 | 0.361 | −0.877 |
Response to Q2: Difficulty-Stratified Evaluation of Open-Ended Generation (3/3)
To further evaluate InfiniDream's open-ended procedural generation capability, we categorize the 100 assets from our original evaluation into three difficulty levels, based on the modifications required to extend the original Infinigen factories:
- D1 (38 assets): Parameter adjustments to existing factories.
- D2 (40 assets): Addition of local geometric structures.
- D3 (22 assets): Major extensions to the original geometry generation rules.
Importantly, the difficulty levels are defined by the expressive limitations of the original Infinigen factories, rather than the outputs of individual methods. For baselines that generate code or geometry without relying on Infinigen factories, these levels do not necessarily reflect their intrinsic generation difficulty. Code2Worlds is evaluated with its factory coverage manually extended to all 100 objects (Appendix M).
We use a single evaluator, GPT-Sol-5.6, to perform binary feature checks and report Feature Pass Rate to quantify generation performance across all methods.
Table 9 (Appendix M). Feature Pass Rate across three levels of procedural generation difficulty.
| VLM Harness | Infinigen | Code2Worlds | VIGA | BlenderRAG | InfiniDream (Ours) | |
|---|---|---|---|---|---|---|
| D1 | 0.84 | 0.02 | 0.26 | 0.66 | 0.74 | 0.97 |
| D2 | 0.85 | 0.00 | 0.00 | 0.78 | 0.65 | 0.93 |
| D3 | 1.00 | 0.00 | 0.00 | 0.82 | 0.82 | 1.00 |
| Overall | 0.88 | 0.01 | 0.10 | 0.74 | 0.72 | 0.96 |
The results show that InfiniDream achieves Feature Pass Rates of 97%, 93%, and 100% on D1, D2, and D3, respectively, with the highest overall score of 96%. Notably, InfiniDream maintains high performance on D2/D3, which require new geometric structures or major extensions to generation rules, whereas Code2Worlds achieves 0% on both. These results indicate that InfiniDream can overcome the expressive limitations of existing procedural factories, rather than being restricted to predefined parameter adjustments.
The evaluation procedure and visual comparisons across difficulty levels are included in Appendix M (Table 9, Fig. 18).
Response to W1 & Q3: Computational Cost and Scalability (1/3)
We thank the reviewer for the suggestion. To clarify the computational overhead and practical scalability of InfiniDream, we provide a breakdown of wall-clock time, VLM calls, and API costs across different pipeline stages.
1. Asset Generation: Generating a new procedural asset takes approximately 38 minutes, 15.3 VLM calls, and $10.24 on average (including up to six refinement rounds). Generated assets are stored in a reusable procedural asset library, making this a one-time construction cost that does not need to be repeated for subsequent scenes.
2. Scene Generation: When all required assets are available in the library, scene selection, planning, object placement, and final rendering take approximately 24 minutes, 5 VLM calls, and $1.07.
3. Library Preparation: Preparing a coarse scene takes approximately 29 minutes on average. The 70-scene library is built once offline and its scenes are directly retrieved at generation time, so this cost is not incurred when generating a scene.
4. End-to-End: Each asset is generated independently, so new assets can be generated in parallel across GPUs, and assets already in the library are simply retrieved. The wall-clock time of a full scene is therefore about 24 minutes when its assets are retrieved, and about one hour (38 + 24 minutes) when new assets are generated in parallel, rather than growing with the number of assets.
The per-stage breakdown is included in Appendix Q (Table 13). All timings are measured on a single NVIDIA RTX 4090 GPU.