acceptodds
Withdrawn from ICLR 2027

Rollback-Free Stable Brick Generation via Reinforcement Learning from Physical Rewards

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

While autoregressive models have advanced 3D generation, creating physically stable brick structures remains a challenge due to the strict requirements of gravity and interconnectivity. Existing approaches rely on external physical simulators during inference to perform rejection sampling and brick-by-brick rollbacks, which severely bottlenecks efficiency. To address this, we propose a reinforcement learning paradigm that shifts physical validity enforcement from test-time correction to training-time policy optimization. By utilizing assembly-level rewards, the model optimizes for collision avoidance, global connectivity, structural interlocking, and shape conformity. This paradigm allows the model to internalize physical priors, enabling the first rollback-free generation of stable brick structures. Experimental results demonstrate that our approach achieves state-of-the-art generation quality while accelerating inference speed by orders of magnitude. Our code is available at https://anonymous.4open.science/r/STABLE.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Brick-Construction, Large Language Model, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7QEWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces a reinforcement-learning framework that uses assembly-level physical rewards to shift stability enforcement from costly test-time correction to training-time optimization for efficient, rollback-free brick generation.

Critical strengths
  1. The paper is well written.
  2. The paper demonstrates strong empirical performance.
Critical weaknesses
  1. The novelty of this work appears limited. While the paper emphasizes “rollback-free brick generation” as its core contribution, this property emerges naturally from reinforcement-learning policy optimization and finds counterparts in other autoregressive generation tasks. It does not seem to stem from task-specific insights or methodological innovations designed for brick generation. Rather, the main contribution consists of adapting an existing reinforcement-learning framework to this target task.

  2. The method aggregates multiple reward terms via a simple weighted sum. However, these rewards can differ markedly in optimization difficulty, magnitude, and convergence behavior, and some objectives may even conflict, leading to unfavorable optimization trade-offs. More principled mechanisms for balancing and combining multi-objective rewards are therefore recommended, instead of naive weighted summation.

  3. Figure 5 mainly presents training reward curves, rather than physical stability metrics measured on a held-out test set. While these curves verify that the model optimizes the predefined reward signals, they are inadequate to demonstrate that the model learns generalizable physical principles.

Questions for authors

See weakness

Reviewer_NM5QOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes STABLE, a framework combining supervised fine-tuning (SFT) and reinforcement learning (GRPO) with structure-level rewards to generate physically valid brick assemblies without test-time simulator rollbacks or rejection sampling. The method uses rewards for collision avoidance, shape similarity, connectivity, and interlocking. Results show improvements over SFT-only generation and reduced inference time compared with rollback-based BrickGPT.

Critical strengths
  • The paper addresses an important problem: generating physically valid structures efficiently without expensive inference-time correction.

  • The improvements over SFT-only generation are substantial, particularly in collision-free rate and structural connectivity.

  • The ablation studies provide useful evidence for the contributions of individual structural rewards.

Critical weaknesses
  • The handcrafted rewards for collision avoidance, connectivity, and interlocking do not necessarily guarantee mechanical stability. The authors should report the stability rate across all generated structures, including invalid outputs, and provide stronger independent validation.

  • Methodological novelty is unclear. The framework combines supervised fine-tuning (SFT), handcrafted rewards, and GRPO, an existing reinforcement learning method. The authors should clarify what is fundamentally new in this combination and how it differs from prior RL-based generation and assembly methods.

  • Although Table 1 reports a high connectivity ratio of 0.97, Table 3 shows that the strict connected rate is only 0.49. This suggests that around half of the generated assemblies may still contain floating or disconnected bricks, raising concerns about structural validity. The authors should clarify the difference between these metrics and explain how often the generated structures are fully connected.

Questions for authors
  • Could the authors briefly discuss how this coordinate-based RLPR/GRPO approach compares conceptually with contemporaneous work like BrickAnything [1], which uses structure-aware tree tokenization and DPO to eliminate rollbacks?

  • What is the total end-to-end training budget (in GPU hours) required for the GRPO phase?

  • In Table 3, could the authors explain the gap between the 0.97 connectivity ratio and the 0.49 strict connected rate? Does this indicate that approximately half of the generated assemblies contain disconnected components?

References

  1. Ni, Zhengyang, et al. "BrickAnything: Geometry-Conditioned Buildable Brick Generation with Structure-Aware Tokenization." NeurIPS (2026).
Reviewer_vF3MOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents STABLE, a two-stage framework for generating brick assemblies without inference-time rejection sampling or rollback. It first performs supervised fine-tuning of an autoregressive language model, then applies GRPO using assembly-level rewards for collision avoidance, shape conformity, connectivity, and interlocking. The main contributions are the construction-aware reward formulation and its application to coordinate-based and graph-based assembly generation. The authors also derive PointCloud2Brick from StableText2Brick to study point-cloud-conditioned brick reconstruction. Validation includes comparisons with general-purpose and specialized generators, geometric and simulator-based stability evaluations, reward ablations, training-dynamics analyses, and experiments on BrickNet. The reported results show substantial improvements over the supervised model in collision avoidance and structural quality, longer valid generation sequences on BrickNet, and approximately 94% lower inference latency than rollback-based BrickGPT.

Critical strengths

(1) Automatic reward computation. Deterministic, rule-based rewards eliminate the need for human preference annotations or a learned reward model, reducing supervision costs. (2) Strong gains from RL post-training. Collision-free rate improves from 0.00 to 0.99, while voxel IoU increases from 0.813 to 0.907, supporting the effectiveness of assembly-level feedback. (3) Efficient inference. Rollback-free decoding avoids repeated correction, with a reported latency reduction of approximately 94% relative to rollback-based BrickGPT.

Critical weaknesses

W1. Limited Exploration of Physics-Aware Reward Design.

While incorporating physical constraints into brick generation is interesting and intuitive, the exploration of physics-aware rewards remains somewhat limited. The proposed rewards mainly focus on collision, connectivity, interlocking, and shape conformity, while more complex physical factors, such as balance under gravity and resistance to collapse, are not explicitly considered. Since reward design is a central contribution, a deeper investigation would strengthen the paper.

W2. Insufficient Qualitative Evaluation.

The qualitative evaluation is relatively limited. Figure 4 mainly compares the proposed method with its SFT-only variant, without visual comparisons with existing methods. More diverse generation results, especially for complex structures, would help better demonstrate the effectiveness of the proposed approach.

W3. Insufficient Analysis of Physical Stability.

Although the paper reports impressive stability scores, some results deserve further clarification. For instance, STABLE achieves a connectivity ratio of 0.97, while its connected rate is only 0.49 in Table 3. Further analysis of these metrics and representative failure cases would help better substantiate the claim of physically stable generation.

Questions for authors

Q1. Could the authors further discuss the limitations of the current physics-aware rewards in more complex stability scenarios?

Q2. Could the authors provide more qualitative results, including challenging structures and visual comparisons with existing methods where feasible?

Q3. How should the perfect stability scores in Table 1 be interpreted given the connected rate of only 0.49 in Table 3? Could the authors provide further analysis or representative failure cases?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.