acceptodds
Under review as a conference paper at ICLR 2027

RelShape: Program-Mediated Generation of Realistic 3D Assets with Controllable Relations

Abstract

Text- and image-conditioned 3D generation could supply games, simulation and design with objects on demand. Yet 3D generative models yield realistic assets either as monolithic shapes or as parts with no relations between them, while program-based methods expose structure but rarely realism or reliable variation, so neither suits workflows that revise and vary assets. We therefore change what 3D generation produces. Program-mediated generation, a paradigm for creating realistic 3D assets with controllable relations, writes a procedural model of the object from the input and materializes every executed instance into a complete asset with a pretrained 3D generator. To make the relations controllable, we represent the procedural model as a Relational Shape Program (RSP), which assembles parts through interfaces computed from their geometry and derives part parameters from object-level controls. To synthesize an RSP that varies correctly from a single input, a 3D coding agent designs its variation before its shape: it declares the object-level controls with the effects and invariants each must satisfy, and keeps only those that executions at non-default values confirm. Each asset keeps the part structure and configuration of its instance, so the assets are realistic, explicitly structured and reliably variable. Experiments show that our assets rival 3D generative models and surpass program-based methods, while following their controls and edits at unseen settings.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D generation, procedural modeling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_W748OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RelShape, which generates realistic articulated 3D objects with movable structures while emphasizing structural control and editing. The paper achieves this goal by developing a variation-first coding agent based on DeepSeek-V4.1-Flash, that generates intermediary programs to construct the specified objects part by part, and uses a visual critic to revise the generated output. The agent refines the output for up to two rounds, stopping early if it satisfies all specified control requirements and passes the rule-based checks, which makes the program can establish articulation relationships consistent with the instructions and produce a preliminary visual appearance. The paper then refines the details and materials of the preliminary geometry using a materialization procedure and the TRELLIS.2 generator. Experiments show that RelShape achieves the best performance among program-synthesis methods, attains reconstruction quality comparable to that of pretrained models such as TRELLIS.2 and Hunyuan3D 2.1, and outperforms other program-synthesis methods in controllability tests.

Critical strengths
  1. This paper proposes a practical algorithm for constructing and refining a 3D model generation program, while checking geometric residuals, visual similarity, and changes in control settings. This gives the method the potential to reconstruct many real-world articulated objects with complex interaction logic and conditional constraints on part movement. It also demonstrates superior performance in structural control compared with other methods.
  2. The proposed algorithm also appears to be extensible, as it is not tied to a specific agent or visual critic. As foundation models become more capable, the algorithm can be expected to handle more complex and diverse modeling tasks and produce more realistic results.
Critical weaknesses
  1. The emphasis on variation is mostly on structural control, but the specific range of motion along each control axis is not explored in depth. Within the authors’ algorithmic framework, controllable relations could support more dimensions, including not only the number, positions, or types of parts, but also how far they can open or close, interaction directions (e.g., for buttons or sliding switches), and occlusion conditions. These aspects are particularly important for detailed modeling of certain objects, such as capturing the interaction logic of buttons, levers, and knobs on a toaster. If these goals were achieved, my confidence in the method’s extensibility would increase substantially.
  2. The paper does not test the algorithm’s extensibility, for example by using different foundation models to assess its robustness to the choice of foundation model and substantiate its independent contribution. Given that foundation models are becoming increasingly capable, with some already able to perform 3D reconstruction independently, I believe that such tests of the algorithm’s extensibility are still necessary.
Questions for authors
  1. As mentioned in W1, in the examples shown in Figures 1 and 5, the range of variation along each control axis does not appear to be determined by the input. Are these ranges determined by the agent and then checked, or are they set empirically at a later stage?
  2. Figure 9 shows a toaster model generated by RelShape, as well as models of other intricate objects, such as a washing machine and a watch. Are the buttons, knobs, and other small components on these models interactive? If so, could the authors provide GIFs demonstrating their ranges of motion?
  3. As mentioned in the previous point, Figure 9 provides only static images, but motion is central to articulated objects. Could the authors provide GIFs of these examples to demonstrate their ranges of motion?
  4. Could the authors test the algorithm with other foundation models, particularly those capable of performing modeling tasks independently, to isolate the performance gains attributable to the algorithm? This would be very helpful for assessing its extensibility.
2 comments
AuthorsOpenReview ↗

Response to Reviewer W748 (1/2)

We sincerely thank you for your positive assessment of our work. We are encouraged that you find our algorithm practical, with its checks of geometric residuals, visual similarity, and changes in control settings. We are also glad that you recognize its superior performance in structural control. We value your observation that the algorithm appears to be extensible, as it is not tied to a specific agent or visual critic. Your suggestions point to two tests of this extensibility, and we ran new experiments for both. For W1, we swept controls over their full declared ranges and made the buttons and the dial knob of the Figure 9 objects interactive through edits. For W2, we ran the method with a second foundation model. The GIFs you asked for are on the anonymous supplementary webpage.

[W1, Q1] Range of variation along each control axis

The ranges are determined by the agent and then checked by execution, and none is set empirically at a later stage. As you rightly point out, the input does not determine the range of a control. It depicts the object at one setting, which gives each control its default value. Before construction, our agent declares each control with the range that the object's function allows (Section 3.2, page 5). For the watch of Figure 9, its declaration sets the strap hole count from 3 to 9 so that the whole row of holes stays inside the rib band of the strap. It sets the bezel ridge count from 12 to 72 because the gaps between the teeth would close near 96. The agent then executes each control at non-default values and keeps only the controls whose declared effects hold.

To explore these ranges in depth, we ran new sweeps. For one control in each of 8 generated programs of Table 1, we executed the unchanged program over the full range that the agent declared. All 60 settings pass the rule-based checks.

ObjectControlDeclared rangeSettings
LaptopLid opening0 to 135°10
HeadphonesEar cup swivel0 to 90°10
ToasterLever position along its track0.15 to 0.8512
Washing machineButton count3 to 75
BedEnd panel bar count2 to 168
Barred doorBar count1 to 106
Desk fanBlade count2 to 65
WatchStrap hole count3 to 94

The supplementary webpage's "Control sweeps" section shows a GIF of each sweep.

AuthorsOpenReview ↗

Response to Reviewer W748 (2/2)

[W1, Q2, Q3] Buttons, levers, and knobs

As generated, the toaster's lever slides. Following your suggestion, four new instruction-guided edits make the buttons and the dial knob interactive on the Figure 9 toaster, washing machine, and watch, with the system unchanged. These edits use the instruction-guided editing of the original submission (Section 4.3 and Figure 5, page 9). Each request names the parts that should be pressable or rotatable and gives no range. The agent gives each part its own control, declares the range of that control, and checks it by execution.

For the dimensions you list in W1, the table below gives the interaction direction of each part, how far it moves, and the limit that neighboring parts place on it:

Object and partInteraction directionHow far it movesLimit from neighboring parts
Toaster, lever (as generated)slides along its track0.15 to 0.85 of the trackthe grip stays over the track and clear of the button row
Toaster, four buttonseach presses in along its own axis0 to 3 mma fully pressed button stays 3 mm proud of the panel
Washing machine, dial knobturns about its own axis-180° to 180°none, it turns a full circle
Washing machine, five buttonseach presses into the panel along its guide bore0 to 6 mmthe front panel skin is 8 mm thick, and a 2 mm margin is kept
Watch, three buttonseach presses into the case along its own axis0 to 0.5 mmthe button neck is 0.6 mm long, and a 0.1 mm clearance is kept

All settings executed over these ranges pass the rule-based checks. In each setting of the four edits, the pressed or turned part moves and every other part stays in place. On the toaster of your example, the lever slides and each of the four buttons presses independently.

The supplementary webpage's "Pressable buttons and a rotatable dial through edits" section shows GIFs of these motions for the toaster, washing machine, and watch of Figure 9. The sliding lever is in the "Control sweeps" section.

[W2, Q4] Other foundation models

Following your suggestion, we ran RelShape with a second foundation model, GPT-6.1 Sol. The method runs unchanged, and on the same model it leads aDSL by 31.3 points with DeepSeek and by 56.2 points with Sol. aDSL is one of the program-based methods of Table 2 that run on the same LLM as our agent. Each row below therefore holds the foundation model fixed and changes only the method.

Foundation modelRelShapeaDSLLead
DeepSeek-V4.1-Flash (Table 2 of the original submission, page 8)71.940.631.3
GPT-6.1 Sol (new, Appendix F.5, page 26)60.44.256.2

The table reports Success on the 16 objects of Table 2 under the same held-out protocol. In the new run, every stage of our agent uses Sol, including the visual critic. With Sol, RelShape's programs expose a control for 75.0% of the held-out goals, against 93.8% with DeepSeek. On the settings with a control, RelShape succeeds on 80.6% with Sol, against 76.7% with DeepSeek (Appendix F.5).

The ablations of Table 3 of the original submission (page 9) isolate the performance gains attributable to the algorithm. They keep the foundation model and remove one step of the algorithm. Success falls from 71.9 to 45.8 without variation planning and to 51.0 without variant feedback.

For models capable of performing 3D generation independently, Table 1 of the original submission (page 7) compares our assets with those of TRELLIS.2 and Hunyuan3D 2.1. These generators output an asset without a program. Our assets answer more structural questions on text and achieve the highest Elo of all methods on images.

Reviewer_LKXXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RelShape synthesizes procedural models from text or a single image and materializes their instances using a pretrained 3D generator. It combines geometry-derived attachment interfaces, shared parameter dependencies, and a variation-first agent that tests controls at non-default settings. Experiments show improvements over evaluated program-based methods in generation quality and held-out control success.

Critical strengths

The paper addresses a useful problem: generating realistic assets that support subsequent structural changes. Planning variation before construction is a meaningful design choice, and the ablations support the effectiveness of both variation planning and execution-based feedback. Freezing programs during held-out evaluation also provides useful evidence that the generated controls work beyond the settings tested during synthesis.

Critical weaknesses

My concern is primarily with the breadth of the conceptual claim, rather than the reported improvements. The first contribution describes “a new paradigm that shifts the output of 3D generation from a static shape to a procedural model.” However, ShapeAssembly already explicitly generates procedural programs representing families of related shapes [1], while ShapeCraft and CADCodeVerify establish generation and refinement of 3D programs through feedback [2, 3].

The related-work discussion acknowledges these precedents and identifies more specific distinctions: geometry-derived interfaces, variation-first planning, testing at non-default settings, and materialization. These distinctions are meaningful and may constitute a valuable contribution. However, they support advances within program-based generation rather than establish the broader program-versus-shape shift itself as new.

The same distinction matters for the claimed scope and the effort required to extend it. The demonstrated variation depends on the specific relations, parameter dependencies, and verification criteria encoded for each generated object. Extending to objects or behaviors outside this scope may require additional construction logic, relation types, or checks, whether supplied by the agent or the system designer. The paper does not clearly establish how much of this work is reusable and how much must be supplied for new cases. Consequently, the improvement appears grounded in particular modeling and validation choices, while the broader framing leaves their applicability boundaries and extension costs insufficiently characterized.

I would find the contribution substantially more convincing if the paper centered its claims on the specific representation and variation-aware synthesis workflow, explained their advance over prior work, and clarified what transfers to new cases versus what requires additional modeling and verification. I want to clarify that I am not asking for universal applicability, but better scoped claims that match the demonstrated contribution.

Questions for authors
  1. What specific new capability warrants the “new paradigm” characterization, given existing work on generating procedural shape families and feedback-driven refinement?
  2. What generalizes through the existing representation and verification machinery, and what requires new object-specific logic or checks? Could you clarify the additional effort needed to expand the supported scope?
2 comments
AuthorsOpenReview ↗

Response to Reviewer LKXX (1/2)

We appreciate your careful reading and constructive suggestions. We are glad that you find the problem useful and regard planning variation before construction as a meaningful design choice, supported by the ablations and the frozen-program evaluation. Your review asks us to center our claims on the specific representation and variation-aware synthesis workflow. It also asks what transfers to new cases versus what requires additional modeling and verification. Following these suggestions, we revised the first contribution and the related work, and we state the scope of the representation in the conclusion and the new Appendix F.3.

[Weakness, Q1] Breadth of the conceptual claim

Thank you for finding the distinctions of our method meaningful. Following your suggestion, the first contribution of the revised manuscript now states the specific new capability of our framework in place of the "new paradigm" characterization. From a single text or image, we synthesize a procedural model with each retained control verified at non-default settings, and its instances materialize into a family of realistic assets.

The novelty of this capability lies in what a single input yields. Infinigen's procedural models are also realistic and vary reliably, and they are written by hand, one category at a time. Our framework synthesizes such a model from a single text or image.

The three works you cite are important precedents. We would like to humbly clarify that none of them yields, from a single input, a family of realistic assets with controls verified beyond the input. To explain our advance over these works, we describe two differences. The first is what the family consists of. ShapeAssembly represents shape families through the continuous free variables of cuboid parts, so its families consist of cuboid structures. Our families consist of realistic assets, materialized from instances that differ in sizes, counts, and joint states. The second is what the feedback checks. ShapeCraft and CADCodeVerify use feedback to refine programs so that their outputs match the input description. This feedback checks what a program builds at the input's setting. Our feedback extends to non-default settings, where execution checks the variation effects declared before construction. Table 2 of the original submission (page 8) compares RelShape with ShapeCraft on the same LLM. ShapeCraft's programs expose a control for 9.4% of the held-out goals and RelShape's for 93.8%, and Success is 3.1 and 71.9. Table 3 (page 9) removes execution at non-default settings from our agent, and Success falls from 71.9 to 51.0.

Following your request for better scoped claims, we have made three revisions to the manuscript:

  • Rescoped the first contribution (page 2): The sentence you quote now reads "We propose a framework that synthesizes, from a single text or image, a procedural model that yields a family of realistic assets." The abstract and introduction refer to our method as a framework accordingly.
  • Linked the first contribution to the second and third (page 3): The first contribution adds "We realize this framework through Relational Shape Programs and a variation-first 3D coding agent, introduced below." Relational Shape Programs and this agent are the specific representation and the variation-aware synthesis workflow, our second and third contributions.
  • Added comparisons with the cited works (page 3): The related work now contrasts our method with ShapeAssembly, ShapeCraft, and CADCodeVerify.

You also asked that the claims match the demonstrated contribution. The scoped claim has three parts, and an experiment of the original submission supports each. The realism of the assets is supported by Table 1 (page 7), which compares our assets with those of direct 3D generators. Our assets answer more structural questions on text and achieve the highest Elo of all methods on images. The controls that hold at settings beyond the input are supported by the frozen-program evaluation above (Table 2). The contributions of the representation and synthesis workflow are supported by the ablations above (Table 3), where Success falls when any one of their elements is removed.

The scope of this claim is objects made of rigid parts joined through four relation types. [Weakness, Q2] below and Appendix F.3 (page 25) describe this scope.

AuthorsOpenReview ↗

Response to Reviewer LKXX (2/2)

[Weakness, Q2] What generalizes and what requires object-specific logic

Thank you for raising the applicability boundary and extension cost of our method. Within the supported scope, the existing representation and verification machinery are reused unchanged for a new object, and the object-specific logic it requires is its own program, which our agent writes in one run. The supported scope is objects made of rigid parts joined through four relation types, with controls checked by five kinds of measurement. Beyond it, the system itself has to be extended, as described below.

We would like to humbly clarify that the relations and verification criteria of every object draw on shared types. Every relation is one of four relation types (Appendix A, page 15), and every declared effect of a control is checked by one of five kinds of measurement (Appendix B, page 16).

To answer what generalizes, what has to be supplied, and the additional effort to expand the supported scope, we separate three kinds of new case:

New caseExisting machinery, reusedNewly suppliedChange to the system
A new object within the supported scopefour relation types, five kinds of measurement, instructions and manualsits parts, relations, dependencies, control declarations, and checks, written by the agentnone
A new behavior that the existing types can expressthe above and the object's programthe changed parts and relations, the new control, and its checks, written by the agentnone
A motion or variation that the existing types cannot expressall remaining machinerya new relation type or kind of measurementimplemented once by the system designer, then available to every later object

The first two cases extend the objects and behaviors covered while the system stays unchanged. For new objects, the 96 objects of Table 1 span 32 text categories and 32 ABO products (page 6), and one version of the system generated all of them, with no relation type or kind of measurement added for any object. What the agent writes for an object averages about five parts, four relations, and three verified controls (Table 7, page 26). For new behaviors, Figure 5 of the original submission (page 9) shows edits in which the agent revises a program that lacks the requested control. Following Reviewer W748's suggestion, we ran four new edits of this kind, which make buttons and a dial on the toaster, washing machine, and watch movable, again with the system unchanged (see our reply to Reviewer W748). The anonymous supplementary webpage's "Pressable buttons and a rotatable dial through edits" section shows GIFs of these four edits.

The third case marks the applicability boundary. Variation that none of the five kinds of measurement checks needs a new kind of measurement, and a motion that the four relation types cannot compose needs a new relation type. Adding one means implementing that computation in the system: a relation type computes the pose of a part from interfaces, and a kind of measurement is a quantity that execution returns.

On the additional effort, the first two cases take one run or one edit by the agent, and the paper reports about 22 minutes for a run (page 8). None of the 96 objects required the third case. The revised manuscript states this scope in Appendix F.3 (page 25), and the conclusion (page 10) names the bounds of the families.

Reviewer_hRzKOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a program-mediated approach to 3D asset generation, combining the controllable structure of procedural 3D programs with the realism of pretrained 3D generative models. Specifically, given a text or image, an agent constructs a Relational Shape Program (RSP) with part relations and controls verified by execution. Executing the program produces an instance consisting of explicit object parts. This instance is used as input to the first coarse geometry generation stage of a pre-trained 3D generation model (TRELLIS2) for detailed materialization. The authors report quantitative and qualitative results on text-to-3D and image-to-3D generation on Objaverse and ABO benchmarks and outperform the examined baselines.

Critical strengths
  1. Strong quantitative and qualitative results. The method achieves strong quantitative performance on the evaluated benchmarks. The qualitative results are also convincing.
  2. Thoughtful design of the Shape Program. Defining controls and their intended effects upfront, evaluating them across parameter settings and configurations beyond those observed in the input, repairing failures, and retaining only controls that behave as intended are effective design choices.
Critical weaknesses
  1. Shape preservation. It is unclear why the materialization stage introduces noise that can alter the structure and potentially violate verified constraints. SpaceControl applies the SDEdit technique to TRELLIS, adding noise to the voxelized input up to a timestep τ, which controls adherence to the initial shape. As a result, the generated geometry may drift from the scaffold, shifting contacts or merging parts. It is unclear how τ is chosen in the experiments or how it affects the results. Moreover, nearest-part assignment may fail where the scaffold and final mesh differ. These failures are not adequately examined.
  2. Editing. TRELLIS2 regenerates the asset rather than directly editing the existing geometry. Thus, although the scaffold can be edited procedurally, the resulting asset may differ because texture and geometric details are resampled. Editing is also presented as a contribution but is not evaluated on an editing benchmark, nor compared quantitatively or qualitatively against recent asset- and CAD-editing methods, such as Nano3D [1] and Voxhammer [2].
  3. Novelty. Conceptual novelty appears limited. aDSL [3] similarly combines LLM-generated structured programs with SpaceControl and TRELLIS. Relational shape programs also build on established CAD assembly practices, while LLM-based systems such as Articraft [4] already generate structured shape programs. References [1] Ye et al., “Nano3D: A Training-Free Approach for Efficient 3D Editing Without Masks”, ICLR ‘26 [2] Li et al., “VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Space”, 3DV ‘26 [3] Wang et al., “aDSL: Agentic 3D Creation via Joint Agent-Program Design”, arXiv ‘26 [4] Zhou et al., “Articraft: An Agentic System for Scalable Articulated 3D Asset Generation”, NeurIPS ‘26
Questions for authors
  1. Could the authors clarify the motivation for using noise added by the SDEdit technique, as in SpaceControl, instead of directly initializing the sparse structure and running the second stage of TRELLIS for material and detail generation, which is already supported in TRELLIS? It would be useful to see experiments comparing these two approaches, along with an explanation of how the timestep is chosen and why.
  2. Another concern is what happens to part assignment if the generated shape diverges from the scaffold, as thin parts, for example, might disappear due to noise. How is part assignment handled in such cases?
  3. It would be valuable to add editing benchmarks and quantitatively report comparisons with at least one of the state-of-the-art 3D editing baselines (mentioned in Weakness 2).
3 comments
AuthorsOpenReview ↗

Response to Reviewer hRzK (1/3)

Thank you for recognizing our quantitative and qualitative results and the design of the Shape Program. Following your suggestions and questions, we ran three new experiments. They are a comparison of direct initialization with SDEdit, an evaluation of part coverage that includes thin parts, and an editing comparison with VoxHammer and Nano3D. The corresponding images are on the anonymous supplementary webpage.

[W1, Q1] Shape preservation and direct initialization

Thank you for raising this question about shape preservation. We would like to humbly clarify that RelShape used direct initialization in the original submission, the approach that your Q1 proposes, without adding SDEdit noise. We use the scaffold's occupied cells to carry the verified program's spatial layout into subsequent shape and texture generation (Section 3.3 of the original submission, p. 6). There is therefore no τ to choose. Section 3.3 of the revised manuscript (p. 6) now states explicitly that no noise is added to the sparse structure.

Following your suggestion, we compared this approach with SDEdit, the guidance used to materialize aDSL in Table 1, on all 96 instances. The appearance image, seed, and subsequent generation stages were held fixed.

Structure initializationParts lost (of 476) ↓Far surface (%) ↓Sparse-structure IoU with direct initialization
Direct initialization (RelShape)00.551.000
SDEdit, noise 0.83 (SpaceControl default)66.250.746
SDEdit, noise 0.6300.850.972
SDEdit, noise 0.5000.660.995
SDEdit, noise 0.3100.630.998

Far surface denotes the percentage of asset surface beyond the distance threshold from every scaffold part. At lower tested noise levels, SDEdit approaches direct initialization but does not improve part retention or far surface. Appendix F.1 of the revised manuscript (p. 25) provides the metric definitions and experimental settings.

The supplementary webpage's "Direct initialization and SDEdit" section shows these differences. For the rolling stool, default SDEdit tilts the seat, merges the legs, and moves the front casters, whereas direct initialization preserves the seat orientation and support layout. These results support direct initialization for limiting structural drift without tuning τ.

[Q2] Part assignment

Thank you for raising this concern about thin parts and their labels. As noted in our response to [W1, Q1], we would respectfully clarify that our method directly initializes the sparse structure without adding SDEdit noise. Under direct initialization, 0.55% of the asset surface lies far from every scaffold part.

For part assignment, we 1) assign each mesh face to its nearest program part and 2) smooth labels over adjacent faces (Appendix C of the revised manuscript, p. 17). Thin parts therefore need not be disconnected from neighboring surfaces to receive part labels.

For the thin parts that you mention, our additional evaluation found that none of the 151 thin parts is lost (Appendix F.2 of the revised manuscript, p. 25). The supplementary webpage's "Part labels" section presents side-by-side views of program geometry, final assets, and part labels. The shelving-unit example shows the retained layout and corresponding labels of its thin uprights and shelves.

AuthorsOpenReview ↗

Response to Reviewer hRzK (2/3)

[W2, Q3] Editing

Thank you for raising this question about editing and for suggesting the two baselines. We would like to humbly note that both the rationale behind our method and our experimental results support effective structural editing while retaining the source asset's overall material and color scheme.

To explain how we combine structural changes with appearance preservation, we describe the two steps of an edit:

  1. Apply the structural edit. The program changes the target part, and part relations propagate the necessary changes to dependent parts. The other parts of the scaffold keep their shape.
  2. Reuse the source appearance conditions. To preserve the source asset's materials and colors, we materialize the edited scaffold using the same appearance image, part materials, and random seed.

The barred-door width edit illustrates the effect on a final asset. The brown wooden finish and dark hinges and lock are retained despite variations in the fine wood-grain pattern. The frame and bars widen together, while the door height and bar count stay the same.

To test whether edits succeed and preserve the specified properties, we added quantitative and qualitative comparisons with VoxHammer and Nano3D, following your suggestion. As the editing benchmark, these new comparisons use the 32 structural edit goals from Table 2 of the original submission, covering changes in dimensions, counts, and angles. Each goal specifies the target change, the changes that must follow in dependent parts, and the properties that must stay fixed. Our agent did not see these goals when it generated the programs. The three methods are compared on the 27 goals for which our program provides an executed edit, starting from the same source assets. Each baseline is guided by a 2D image of the source asset edited according to the goal text. VoxHammer also receives the corresponding 3D edit region.

27 goals where all methods runRelShapeVoxHammerNano3D
Edit achieved2211
Target change2321
Dependent changes2497
Properties kept fixed by the goal212220
Goals that rotate a part (of 6)601
Goals that change a single part (of 6)400

An edit is achieved when both the target and dependent changes succeed. We separately evaluate whether the properties specified as fixed are preserved, such as the door height and bar count above. Over all 32 goals, RelShape achieves 22 edits.

The comparison shows that RelShape achieves more structural edits while preserving the properties specified by the goals about as often as the two editors. Appendix F.4 of the revised manuscript (p. 26) provides the experimental settings and evaluation. The supplementary webpage's "Editing comparison" section shows the source asset and all methods' results side by side for all 27 goals. The "Barred door" row for "door width ×1.3" shows the example above.

AuthorsOpenReview ↗

Response to Reviewer hRzK (3/3)

[W3] Conceptual novelty

Thank you for your feedback on conceptual novelty and for the prior work that you discuss. The conceptual novelty lies in what a single input yields. From a single text or image, we synthesize a procedural model with each retained control verified at non-default settings, and its instances materialize into a family of realistic assets (first contribution, p. 2 of the revised manuscript). These controls change dimensions, part counts, and joint states. Generation and structural editing therefore pass through the same procedural model.

To clarify how this differs from the works you cite, we compare with each in your order:

  1. aDSL. aDSL plans and verifies an output that satisfies the input. Before construction, our agent declares what each control must change, what must follow, and what must stay fixed. It then retains only the controls that hold when executed at settings beyond the input. Table 2 of the original submission (p. 8) compares the two methods on the same LLM, with goals that neither method sees. It reports Success, the share of held-out settings that reach the goal value and keep every required relation, and RelShape leads by 31.3 points. The new Appendix F.5 (p. 26) repeats this comparison with a second LLM, GPT-6.1 Sol, where RelShape leads by 56.2 points. With Sol, the aDSL programs write most dimensions and counts as literal values, and 5 of the 32 goals have a named input to set. For materialization, we initialize the sparse structure directly and do not use SpaceControl (see [W1, Q1]).

  2. CAD assembly practices. CAD assemblies place parts through mates. What we add is the synthesis of a whole Relational Shape Program from a single input. Our agent writes the parts, interfaces, dependencies, relations, and checks of an object together, and it verifies the declared effects at non-default settings.

  3. Articraft. Each Articraft program describes one articulated asset, which varies through its joints. Our controls also change dimensions and part counts. Of the 320 verified controls in the 96 programs of Table 1, 242 set continuous quantities such as dimensions, 53 set counts, and 25 set motions (Table 7, p. 26). One program therefore yields instances that differ in dimensions and part counts as well as in pose.

The comparison with aDSL shows that a structured program written by an LLM does not by itself vary reliably under its controls. This reliability comes from the design choices that you recognized in Strength 2, and Table 3 of the original submission (p. 9) tests them. Success falls from 71.9 to 45.8 without variation planning. It falls to 51.0 when the agent is still told to design controls but never executes them at non-default values.

The evidence above concerns the programs and their controls. The two new experiments that you suggested test whether the program's parts and structural edits carry over to the final assets after materialization. None of the 476 program parts is lost ([W1, Q1]), and RelShape achieves 22 of the 27 structural edits, against 1 each for VoxHammer and Nano3D ([W2, Q3]).

The corresponding places in the revised manuscript are:

  • First contribution (p. 2): "We propose a framework that synthesizes, from a single text or image, a procedural model that yields a family of realistic assets."
  • Related work (p. 3): We added the comparison with Articraft.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for recognizing the strong results (hRzK), the design of controls planned before construction and evaluated beyond the input (hRzK, LKXX), and the practical, extensible algorithm (W748). Following the reviewers' suggestions, we revised the manuscript and ran new experiments. The items below give our main answers, with details in the reply to each reviewer.

  • hRzK [W3] and LKXX [Weakness, Q1] (pages 2 and 3): Following Reviewer LKXX's suggestion, the first contribution now states the specific new capability of our framework. From a single text or image, we synthesize a procedural model with each retained control verified at non-default settings, and its instances materialize into a family of realistic assets. This capability differs from the works that the two reviewers cite in two ways. First, ShapeAssembly's families consist of cuboid structures, and each Articraft program describes one articulated asset, which varies through its joints. Our families consist of realistic assets that differ in dimensions, part counts, and joint states. Second, aDSL, ShapeCraft, and CADCodeVerify check what a program builds at the input's setting, whereas our agent retains only the controls that hold when executed at settings beyond the input. On the same LLM, RelShape leads aDSL in held-out Success by 31.3 points (Table 2 of the original submission, page 8).
  • hRzK [W1, Q1] and [Q2] (Section 3.3, page 6, and Appendix F.1 and F.2, page 25): RelShape used direct initialization of the sparse structure in the original submission, without adding SDEdit noise, so there is no τ to choose. Following the reviewer's suggestion, we added a comparison with SDEdit, in which direct initialization loses none of the 476 program parts, including the 151 thin parts.
  • hRzK [W2, Q3] (Appendix F.4, page 26): Following the reviewer's suggestion, we added a comparison with VoxHammer and Nano3D. On the 27 structural edit goals where all three methods run, RelShape achieves 22 edits, and VoxHammer and Nano3D achieve 1 each. Our edits change the structure while retaining the source asset's overall material and color scheme.
  • LKXX [Weakness, Q2] (Appendix F.3, page 25): Within the supported scope, the existing representation and verification machinery are reused unchanged for a new object, and the object-specific logic it requires is its own program, which our agent writes in one run. The supported scope is objects made of rigid parts joined through four relation types, and one version of the system generated all 96 objects of Table 1.
  • W748 [W1, Q1, Q2, Q3] and [W2, Q4]: The agent determines the range of each control and then checks it by execution. New sweeps execute 8 controls over their full declared ranges, and all 60 settings pass the rule-based checks. New instruction-guided edits make the buttons and the dial knob interactive on the Figure 9 toaster, washing machine, and watch. With a second foundation model, GPT-6.1 Sol, the method runs unchanged, and its lead over aDSL on the same model is 56.2 points (Appendix F.5, page 26).

The revised manuscript marks the following changes in dark red:

  • Abstract and introduction (pages 1 to 3): We present our method as a framework and state its specific new capability in the first contribution.
  • Related work (page 3): We add comparisons with the works that the reviewers cite.
  • Section 3.3 (page 6): We state explicitly that materialization adds no noise.
  • Conclusion (page 10): We state the scope of the representation.
  • Appendix F (pages 25 and 26): We add the SDEdit comparison, part coverage, the scope of the representation, the editing comparison, and the second foundation model.

The images for Reviewer hRzK and the GIFs for Reviewer W748 are on the anonymous supplementary webpage. We hope that these revisions and new experiments address the reviewers' concerns, and we are glad to answer further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.