- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ASTER turns indoor point clouds into simulation-ready worlds. It combines SpatialLM-based layout and physical-attribute prediction, progressive object generation with ShapeR, and simulator-based refinement. Evaluation includes reconstruction and stability metrics, policy transfer across three matched simulation scenes, and a small real-robot experiment using additional synthetic data.
Critical strengths
-
The pipeline recovers room structure and separate object assets for simulation. This is useful for embodied data generation, and the reported 5.5-minute reconstruction example is encouraging.
-
Table 4 shows clear improvements over SpatialLM + fine-tuned ShapeR: collision decreases from 25.8 to 18.0, and stability increases from 69.9 to 78.6. Figure 5 also checks policy performance, giving useful evidence beyond reconstruction quality.
Critical weaknesses
-
The benefit of predicted physical attributes is unclear. Appendix B measures agreement with VLM labels, while refinement can make unstable objects static. A comparison with simple parameter defaults and separate results for movable objects would help establish whether the method improves useful interactions.
-
The downstream evidence is promising but limited. Figure 5 uses three synthetic scenes, and Figure 6 reports one workspace with six hard trials. In the real study, task objects are added separately, and there is no matched synthetic-data baseline. This makes it difficult to identify the contribution of ASTER’s proposed stages.
Questions for authors
-
Clarification: What fraction of objects become static during refinement, and what is the stability score for objects that remain movable?
-
Requested evidence: With the same refinement, how do category-default mass and friction compare with the predicted attributes?
-
Requested evidence: With equal amounts of synthetic training data, how does ASTER compare with SpatialLM + fine-tuned ShapeR using the same task objects? Does the real-robot improvement hold across repeated training runs?
7 comments
Questions
Q1. Static conversions and remaining-object stability
“What fraction of objects become static during refinement, and what is the stability score for objects that remain movable?”
Refinement converts 7.2% of evaluated objects from dynamic to static, mainly generated large furniture with uneven bases, especially bookshelves. Initial model assignments separately classify 23.7% as static, mainly fixed lighting and other scene elements.
After excluding objects that should be fixed, original-output room assembly achieves 98.7% stability and 3.2% collision for the remaining objects. W1 distinguishes these group-specific results from overall scene stability and reports the categories affected by refinement.
Q2. Defaults versus predicted physical attributes
“With the same refinement, how do category-default mass and friction compare with the predicted attributes?”
The added comparison in W1 evaluates category-default mass/friction with mobility-only prediction against joint mass/friction/mobility prediction. The mobility predictor changes between these configurations.
Collision/stability improves from 22.1%/75.9% to 18.0%/78.6%, while mean task success is nearly unchanged: 54.1/49.3/76.0% versus 54.3/49.3/76.0% for Pick & Place, Dish Stacking, and Object Pushing.
The main measured benefit is the improvement in mobility agreement from 93.5% to 98.9%. In the manipulation scenes, the principal observed difference is a distant falling painting outside the robot workspace. These results support joint physical supervision for mobility prediction and scene configuration.
Q3. Equal-data comparison and repeated training
“With equal amounts of synthetic training data, how does ASTER compare with SpatialLM + fine-tuned ShapeR using the same task objects?”
The comparison in W2 matches the scenes, task objects, demonstration budgets, and training conditions, with policies evaluated in the same GT environments. ASTER achieves 59.9% versus 23.2% end-to-end success.
The component baseline is comparable in Scene 1. A falling ceiling light prevents training in Scene 2, while table–chair intersections displace the table in Scene 3. Across the two scenes where both methods support training, ASTER still achieves 62.7% versus 34.8%. This provides evidence for the value of ASTER's environment construction independently of increasing the prescribed training-data budget.
“Does the real-robot improvement hold across repeated training runs?”
The original real-robot learning experiment evaluates one trained policy per data condition. The expanded evaluation tests a complementary form of transfer: ten successful simulated trajectories per task are strictly replayed on the physical robot across five tasks, achieving 41/50 successes (82%).
These 50 executions measure trajectory transfer across task types. The matched-budget comparison separately controls synthetic-data volume and training conditions when evaluating the reconstructed training environments.
The added results strengthen the evidence that ASTER supports reliable environment construction, robot learning, and automated data generation across tasks. We would appreciate your reconsideration in light of this additional evidence.
Response to the rebuttal
Thanks for the quick rebuttal from the authors. I think those added experiments of matched budget comparison are helpful, and the result from scene 3 can show that collisions can affect the training outcome.
I still have some quick follow-up questions:
- In Q2, I asked
With the same refinement, how do category-default mass and friction compare with the predicted attributes?
now you have the comparison and show that stability slightly increase but the successful rate is similar with different mobility prediction results. so I'm not sure whether those new predictions really help the downstream tasks. if not, maybe the claim should be revised. Also, since the mobility predictor changes together with mass and friction in this comparison, it doesn't really isolate the effect of the predicted mass and friction. Could you keep the jointly predicted mobility fixed and only replace mass and friction with category defaults?
- in the matched-budget baseline comparison experiment, you show results from three scenes. the first scene comes out similar result, and the second scene you didnt really train a policy for the baseline due to the failure of the ceiling light prediction, but that should be easily fixable.. and only scene 3 has clear comparison. I think the conclusion can be much stronger if there are some more experiment results showing positive signals in more scenes. Also, what mobility setting does the baseline use? If it simply has no static assignment, fixing scene 2 with a category rule, e.g., setting ceiling lights as static, and rerunning the comparison would make the result looks more convincing to me.
Thanks!
Critical Weaknesses
Thank you for the thoughtful review. We add controlled downstream comparisons, physical-prediction results, and five-task real-world evaluation to address the concerns below.
W1. Physical attributes and useful interactions
The main measured benefit of joint physical prediction is improved mobility prediction and scene configuration. We compare category-default mass/friction with mobility-only prediction against joint mass/friction/mobility prediction.
Joint prediction improves mobility agreement by 5.4 percentage points, evaluated against the same VLM annotations on the same held-out set and matched objects. Mass/friction prediction provides additional physical supervision for identifying fixed and movable objects.
Task success is nearly unchanged between these configurations. The principal observed difference in the manipulation scenes is a painting falling far from the robot in Scene 3, affecting scene stability much more than task execution in the robot workspace. The downstream comparison in W2 evaluates the complete pipeline's effect on useful robot interactions.
We distinguish two stages. Initial model assignments classify 15,425/65,003 objects (23.7%) as static across 10,514 valid rooms, mainly ceiling lights and other fixed elements. Refinement subsequently converts 7.2% of evaluated objects, mainly generated large furniture with uneven bases, especially bookshelves. These corrections include normally movable furniture whose generated geometry prevents stable placement.
After excluding objects that should be fixed, rooms assembled from the original outputs achieve 98.7% stability and 3.2% collision for the remaining objects. This group-specific result is reported separately from overall scene stability. Most remaining overlaps arise from small bounding-box errors.
W2. Downstream evidence and matched-data baselines
We add SpatialLM + fine-tuned ShapeR using the same scenes, task objects, demonstration budgets, and training conditions. Policies trained in the reconstructed environments are evaluated in the same GT scenes.
Entries report Pick & Place / Dish Stacking / Object Pushing success (%).
† A ceiling light falls onto the table in Scene 2, preventing baseline training; this counts as zero end-to-end success. In Scene 3, table–chair intersections displace the table and reduce success.
ASTER improves overall success by 36.7 percentage points. Across Scenes 1 and 3, where baseline training is possible, success remains 62.7% versus 34.8%. The advantage therefore persists among trainable scenes and demonstrates the value of constructing reliable robot-training environments at the same data budget.
We expand real-world evaluation to five tasks and 50 trajectory executions. A robot VLM agent generates ten successful simulated trajectories per task in the reconstructed laboratory. We instantiate the corresponding real tasks and strictly replay the trajectories.
The matched-budget experiment evaluates the reconstructed training environments; the real-world experiment evaluates direct transfer of successful simulated trajectories. Together, they demonstrate environment usability and reuse across task types. Physical experiments use the laboratory where our robot arm is available.
This enables varied object identities, positions, and task combinations within each reconstructed room. Robot and task-object placement belong to data synthesis, and both reconstruction methods use identical task objects in the matched-budget comparison.