- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces TACER, a framework for generating verified 3D tabletop scenes targeted at simulation and robotic interaction. Unlike prior tabletop generation approaches that mainly focus on visual similarity or static geometry, TACER formulates scene generation as a simulator-in-the-loop construction problem. The proposed framework represents scenes using hierarchical executable programs, including asset-level programs for constructing physics-ready objects and scene-level programs for assembling layouts. A verification module evaluates physical validity, spatial constraints, stability, and articulation clearance, and uses these failures to guide iterative program refinement. The paper further introduces TabletopAudit, a benchmark with 500 cases covering input fidelity, 3D scene quality, simulation validity, and embodied robot execution. The authors report improvements over existing baselines on both text- and image-conditioned generation, and demonstrate better downstream robot manipulation performance in Isaac Sim.
Critical strengths
This is an important shift from "generate plausible scenes" to "generate executable environments." The hierarchical executable program representation is a reasonable design choice
Critical weaknesses
Limited evidence that TACER generalizes beyond tabletop manipulation
The paper positions TACER as a general framework for simulation-ready scene generation, but almost all experiments focus on tabletop environments. However, it remains unclear whether the proposed representation and verification framework would extend to: larger indoor/out door environments, deformable and fluid objects. The current evidence mainly supports "verified tabletop scene generation" rather than general embodied world generation.
The downstream application is promising but relatively limited in scale.
Embodied data generation and policy learning is an interesting validation direction. A broader evaluation involving multiple manipulation skills or comparison against other synthetic data generation pipelines would strengthen this claim.
Questions for authors
- How would TACER scale to environments where physical constraints are not limited to tabletop support/contact relationships?
- The refinement loop repeatedly executes simulation and verification. What is the average computational cost of generating one verified scene? average number of refinement iterations, simulation time, VLM/asset-generation cost, success rate before and after refinement.
- The paper demonstrates downstream policy learning in simulation. Are there plans or preliminary results showing whether TACER-generated scenes improve transfer to real robots?
4 comments
Response to Reviewer gAtz (2/2): Generation cost and real-robot transfer
Q2. What is the average computational cost of generating one verified scene, and how does refinement change the success rate?
A2. Refinement raises the all-check pass rate from 36.8% to 72.0% with 3.2 rounds per input on average. A passing scene takes 21.37 min, of which 9.38 min is asset generation and about 32 s is Drake simulation, and USD 1.89 in model cost.
R3-Table 2. Pass rate and cost on all 500 image-track inputs. Failures and timeouts (40-min budget) count as non-passes.
| Metric | Result |
|---|---|
| All checks passed: before / after refinement | 184/500 (36.8%) / 360/500 (72.0%) |
| Mean refinement rounds per input | 3.2 |
| Mean end-to-end time: passing scenes / all inputs | 21.37 / 25.21 min |
| Mean asset-generation time (passing scenes) | 9.38 min |
| Drake checks per passing scene: placement / stable pose | 71 calls, 17.6 s / 10 calls, 14.3 s |
| Input / output tokens per input | 283.3K / 38.4K |
| Model cost per input / per passing scene | USD 1.36 / 1.89 |
Costs include failures and retries and use Kimi-K3 prices (CNY 20/100 per MTok input/output, CNY 7/USD), excluding local compute. In new comparisons on a 50-case text-track subset, TACER costs USD 1.29 per case, less than half of SceneSmith (3.41) and GPT-6 Astra (3.22), two agentic baselines we added. We will add this table to the appendix.
Q3. Do TACER scenes improve transfer to real robots, and can the downstream evaluation be broadened?
A3. Yes. On a real Franka FR3, adding TACER simulation data raises π0.5 success from 6/10 and 4/10 (real data only) to 10/10 and 9/10 on two tasks, and TACER data alone, without real demonstrations, reaches 5/10 and 4/10.
Setup. We teleoperated the robot with GELLO [2] on two tasks: banana pick (the banana is lifted off the plate and held for at least 0.5 s) and apple-to-plate (the apple is placed, released, and left resting in the plate). We collected 30 and 50 demonstrations (24 and 43 for training). For each task, TACER built a twin of the real workspace without manual alignment, and we generated 125 trajectories (105 for training) with ±1 cm and ±10° object perturbations. We fully fine-tune π0.5 [3], sampling real and simulated data 1:1 per batch, with identical settings across conditions; simulation-only training uses 60 and 100 TACER trajectories. Each condition is tested once on the same 10 real layouts per task, which deviate by up to 2.5 cm and 30°, beyond the simulated range.
R3-Table 3. Real-robot success of fine-tuned π0.5 (successes / 10).
| Training data | Banana pick | Apple-to-plate |
|---|---|---|
| None (π0.5 base) | 0/10 | 0/10 |
| Real only | 6/10 | 4/10 |
| TACER simulation only | 5/10 | 4/10 |
| Real + TACER simulation | 10/10 | 9/10 |
Results. The base model fails both tasks, and real-only fine-tuning reaches 6/10 and 4/10. Without any real demonstrations, TACER data alone reaches 5/10 and 4/10, comparable to real-only fine-tuning, even though the twins are generated without manual alignment and the rendered images are not registered to the real ones. Co-training performs best, solving 10/10 banana and 9/10 apple layouts, all of which deviate beyond the simulated perturbation range. TACER data can thus partly substitute for real demonstrations and clearly complements them.
Videos. The anonymous project page now includes videos of teleoperated data collection, simulated demonstrations in the TACER twins, and real-robot rollouts for every training condition.
Broader downstream evaluation. With the simulated screwdriver-to-tray study, the downstream evaluation now covers three tasks, two of them on a real robot. We have not yet trained policies on scenes from other generators, but scenes must first support successful execution before demonstrations can be collected: under a stricter protocol on Embodied-100, with clutter collisions enabled and goals held for 1 s, TabletopGen and GPT-6 Astra scenes support 9/60 and 13/60 successful rigid tasks, versus 38/60 for TACER. The study remains small (two real tasks, 10 trials each); we will add it as App. E.6 and leave policy-level comparisons across generators and more skills to future work.
We thank the reviewer again for the encouraging assessment and the suggestions, which have strengthened the paper.
References
[2] Wu et al. GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators. IROS, 2024.
[3] Physical Intelligence et al. π0.5: A Vision-Language-Action Model with Open-World Generalization. 2025.
Thank you for the detailed response and additional experiments. These additions strengthen the paper, and I maintain my Weak Acceptance rating.
Dear Reviewer gAtz,
Thank you for reading our response and for your continued support. We are glad that the additional experiments strengthen the paper. Following your suggestions, the revision states the tabletop scope explicitly, discusses extensions to larger environments, deformable objects, and fluids, and adds a real-robot study. If you feel these additions further strengthen the work, we would be grateful if you would consider reflecting this in your final assessment.
Best regards, The Authors
Response to Reviewer gAtz (1/2): Scope beyond tabletop scenes
We thank the reviewer for the thoughtful assessment. Q1 addresses Weakness 1 and Question 1, Q2 addresses Question 2, and Q3 addresses Weakness 2 and Question 3.
Q1. How would TACER scale to environments where physical constraints are not limited to tabletop support and contact relationships?
A1. We agree that our evidence supports verified tabletop scene generation rather than general embodied world generation, and we will state this scope explicitly. TACER's program hierarchy, dependency-ordered assembly, and failure-attributed refinement are not tabletop-specific; its placement initialization and benchmark are, and its checks would need environment-level additions.
Beyond a single support plane. Within the tabletop setting, App. E.5 already places objects on multiple support surfaces at different heights (bookshelves, multi-level desks, storage workbenches, and lab racks), including nested supports such as a desk supporting a monitor riser that supports a monitor. The same dependency graph orders these supports, and the same checks verify them.
Larger environments. Room-scale systems such as SceneSmith [1] first lay out furniture and then populate surfaces with small objects; in new comparisons, we also ran SceneSmith's small-object stage as a tabletop baseline. TACER could serve as this second stage, treating each workspace as a subscene within region-level programs. Verification would add environment-level checks, and after a local edit it would re-verify only the affected objects and the constraints linked to them, so cost grows with the edit rather than the scene. For example, moving an object to resolve a collision must also keep the robot's passage clear.
Deformable objects and fluids. These require material and dynamic-state representations, solvers that simulate them, and time-dependent criteria, such as whether liquid stays in a container. The program structure can carry such properties, but we have not built or evaluated them. We see chemistry-lab tabletops as an especially meaningful next step: TACER already generates laboratory benches with rigid and articulated instruments, and adding liquids in beakers, flexible tubing, and gloves would enable verified scenes for tasks such as pouring, pipetting, and sample handling.
We will state the tabletop scope in the abstract and introduction and discuss these extensions in the limitations.
References
[1] Pfaff et al. SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes. 2026.