acceptodds
Under review as a conference paper at ICLR 2027

TACER: Verified 3D Tabletop Scene Generation for Simulation and Interaction

Abstract

Generating 3D tabletop scenes from images or text supports robot learning, but visual and geometric plausibility alone do not establish simulation validity or interaction usability. Existing methods largely operate on static meshs, with limited support for functional structures. Evaluation also emphasizes static geometry and visual judgments, offering limited evidence of dynamic validity and task executability. To address these limitations, we present TACER, which connects asset construction, scene assembly, and verification-guided refinement through hierarchical executable programs. Asset-level programs construct geometry and physical representations, including articulation; scene-level programs organize placements and spatial relations. Geometric checks and simulation tests guide asset or layout revisions, which are re-executed and verified before acceptance. To independently assess scene capabilities, we introduce TabletopAudit, a 500-case benchmark covering input fidelity, 3D scene quality, simulation validity, and robot task execution. TACER leads input fidelity on both input tracks. On Embodied-100, it achieves 71.67 rigid-manipulation and 60.00% articulated task success, compared with 20.00% for TabletopGen and 5.00% for TabletopGen + Articraft, respectively. To demonstrate scalability, we construct TACER-11K: 11,399 scenes across nine environments with physical representations, interactive assets, and available verification evidence. A downstream study demonstrates their utility for demonstration collection and task-specific policy learning. Together, these results support TACER's effectiveness and establish TabletopAudit as a basis for assessing simulation readiness and interaction usability.

Primary area:
applications to robotics, autonomy, planning
Keywords:
3D Scene Generation, Programmatic Scene Modeling, Simulation-Ready Environments, Embodied AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gAtzOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces TACER, a framework for generating verified 3D tabletop scenes targeted at simulation and robotic interaction. Unlike prior tabletop generation approaches that mainly focus on visual similarity or static geometry, TACER formulates scene generation as a simulator-in-the-loop construction problem. The proposed framework represents scenes using hierarchical executable programs, including asset-level programs for constructing physics-ready objects and scene-level programs for assembling layouts. A verification module evaluates physical validity, spatial constraints, stability, and articulation clearance, and uses these failures to guide iterative program refinement. The paper further introduces TabletopAudit, a benchmark with 500 cases covering input fidelity, 3D scene quality, simulation validity, and embodied robot execution. The authors report improvements over existing baselines on both text- and image-conditioned generation, and demonstrate better downstream robot manipulation performance in Isaac Sim.

Critical strengths

This is an important shift from "generate plausible scenes" to "generate executable environments." The hierarchical executable program representation is a reasonable design choice

Critical weaknesses

Limited evidence that TACER generalizes beyond tabletop manipulation

The paper positions TACER as a general framework for simulation-ready scene generation, but almost all experiments focus on tabletop environments. However, it remains unclear whether the proposed representation and verification framework would extend to: larger indoor/out door environments, deformable and fluid objects. The current evidence mainly supports "verified tabletop scene generation" rather than general embodied world generation.

The downstream application is promising but relatively limited in scale.

Embodied data generation and policy learning is an interesting validation direction. A broader evaluation involving multiple manipulation skills or comparison against other synthetic data generation pipelines would strengthen this claim.

Questions for authors
  1. How would TACER scale to environments where physical constraints are not limited to tabletop support/contact relationships?
  2. The refinement loop repeatedly executes simulation and verification. What is the average computational cost of generating one verified scene? average number of refinement iterations, simulation time, VLM/asset-generation cost, success rate before and after refinement.
  3. The paper demonstrates downstream policy learning in simulation. Are there plans or preliminary results showing whether TACER-generated scenes improve transfer to real robots?
4 comments
AuthorsOpenReview ↗

Response to Reviewer gAtz (1/2): Scope beyond tabletop scenes

We thank the reviewer for the thoughtful assessment. Q1 addresses Weakness 1 and Question 1, Q2 addresses Question 2, and Q3 addresses Weakness 2 and Question 3.


Q1. How would TACER scale to environments where physical constraints are not limited to tabletop support and contact relationships?

A1. We agree that our evidence supports verified tabletop scene generation rather than general embodied world generation, and we will state this scope explicitly. TACER's program hierarchy, dependency-ordered assembly, and failure-attributed refinement are not tabletop-specific; its placement initialization and benchmark are, and its checks would need environment-level additions.

R3-Table 1. Which TACER components transfer beyond tabletops.

ComponentTransfers?What larger scenes would need
Asset- and scene-level program hierarchyYesA region level that groups workspaces as subscenes
Dependency graph from support, containment, and attachmentYesFloor → furniture → objects; attachment covers wall-mounted items
Asset routes, including articulated assetsYesFurniture-scale articulation such as cabinet doors and drawers
Support, collision, stability, and clearance checks with failure attributionYesNavigation connectivity, robot reachability, terrain traversability
Placement initialization (Eq. 3)PartlyDepth-based reconstruction instead of offsets relative to a support in image space
Deformable objects and fluidsNoMaterial and dynamic-state representations, suitable solvers, temporal criteria
TabletopAuditNoRoom-level requirements and tasks

Beyond a single support plane. Within the tabletop setting, App. E.5 already places objects on multiple support surfaces at different heights (bookshelves, multi-level desks, storage workbenches, and lab racks), including nested supports such as a desk supporting a monitor riser that supports a monitor. The same dependency graph orders these supports, and the same checks verify them.

Larger environments. Room-scale systems such as SceneSmith [1] first lay out furniture and then populate surfaces with small objects; in new comparisons, we also ran SceneSmith's small-object stage as a tabletop baseline. TACER could serve as this second stage, treating each workspace as a subscene within region-level programs. Verification would add environment-level checks, and after a local edit it would re-verify only the affected objects and the constraints linked to them, so cost grows with the edit rather than the scene. For example, moving an object to resolve a collision must also keep the robot's passage clear.

Deformable objects and fluids. These require material and dynamic-state representations, solvers that simulate them, and time-dependent criteria, such as whether liquid stays in a container. The program structure can carry such properties, but we have not built or evaluated them. We see chemistry-lab tabletops as an especially meaningful next step: TACER already generates laboratory benches with rigid and articulated instruments, and adding liquids in beakers, flexible tubing, and gloves would enable verified scenes for tasks such as pouring, pipetting, and sample handling.

We will state the tabletop scope in the abstract and introduction and discuss these extensions in the limitations.

References

[1] Pfaff et al. SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes. 2026.

AuthorsOpenReview ↗

Response to Reviewer gAtz (2/2): Generation cost and real-robot transfer

Q2. What is the average computational cost of generating one verified scene, and how does refinement change the success rate?

A2. Refinement raises the all-check pass rate from 36.8% to 72.0% with 3.2 rounds per input on average. A passing scene takes 21.37 min, of which 9.38 min is asset generation and about 32 s is Drake simulation, and USD 1.89 in model cost.

R3-Table 2. Pass rate and cost on all 500 image-track inputs. Failures and timeouts (40-min budget) count as non-passes.

MetricResult
All checks passed: before / after refinement184/500 (36.8%) / 360/500 (72.0%)
Mean refinement rounds per input3.2
Mean end-to-end time: passing scenes / all inputs21.37 / 25.21 min
Mean asset-generation time (passing scenes)9.38 min
Drake checks per passing scene: placement / stable pose71 calls, 17.6 s / 10 calls, 14.3 s
Input / output tokens per input283.3K / 38.4K
Model cost per input / per passing sceneUSD 1.36 / 1.89

Costs include failures and retries and use Kimi-K3 prices (CNY 20/100 per MTok input/output, CNY 7/USD), excluding local compute. In new comparisons on a 50-case text-track subset, TACER costs USD 1.29 per case, less than half of SceneSmith (3.41) and GPT-6 Astra (3.22), two agentic baselines we added. We will add this table to the appendix.


Q3. Do TACER scenes improve transfer to real robots, and can the downstream evaluation be broadened?

A3. Yes. On a real Franka FR3, adding TACER simulation data raises π0.5 success from 6/10 and 4/10 (real data only) to 10/10 and 9/10 on two tasks, and TACER data alone, without real demonstrations, reaches 5/10 and 4/10.

Setup. We teleoperated the robot with GELLO [2] on two tasks: banana pick (the banana is lifted off the plate and held for at least 0.5 s) and apple-to-plate (the apple is placed, released, and left resting in the plate). We collected 30 and 50 demonstrations (24 and 43 for training). For each task, TACER built a twin of the real workspace without manual alignment, and we generated 125 trajectories (105 for training) with ±1 cm and ±10° object perturbations. We fully fine-tune π0.5 [3], sampling real and simulated data 1:1 per batch, with identical settings across conditions; simulation-only training uses 60 and 100 TACER trajectories. Each condition is tested once on the same 10 real layouts per task, which deviate by up to 2.5 cm and 30°, beyond the simulated range.

R3-Table 3. Real-robot success of fine-tuned π0.5 (successes / 10).

Training dataBanana pickApple-to-plate
None (π0.5 base)0/100/10
Real only6/104/10
TACER simulation only5/104/10
Real + TACER simulation10/109/10

Results. The base model fails both tasks, and real-only fine-tuning reaches 6/10 and 4/10. Without any real demonstrations, TACER data alone reaches 5/10 and 4/10, comparable to real-only fine-tuning, even though the twins are generated without manual alignment and the rendered images are not registered to the real ones. Co-training performs best, solving 10/10 banana and 9/10 apple layouts, all of which deviate beyond the simulated perturbation range. TACER data can thus partly substitute for real demonstrations and clearly complements them.

Videos. The anonymous project page now includes videos of teleoperated data collection, simulated demonstrations in the TACER twins, and real-robot rollouts for every training condition.

Broader downstream evaluation. With the simulated screwdriver-to-tray study, the downstream evaluation now covers three tasks, two of them on a real robot. We have not yet trained policies on scenes from other generators, but scenes must first support successful execution before demonstrations can be collected: under a stricter protocol on Embodied-100, with clutter collisions enabled and goals held for 1 s, TabletopGen and GPT-6 Astra scenes support 9/60 and 13/60 successful rigid tasks, versus 38/60 for TACER. The study remains small (two real tasks, 10 trials each); we will add it as App. E.6 and leave policy-level comparisons across generators and more skills to future work.

We thank the reviewer again for the encouraging assessment and the suggestions, which have strengthened the paper.

References

[2] Wu et al. GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators. IROS, 2024.

[3] Physical Intelligence et al. π0.5: A Vision-Language-Action Model with Open-World Generalization. 2025.

Thank you for the detailed response and additional experiments. These additions strengthen the paper, and I maintain my Weak Acceptance rating.

Dear Reviewer gAtz,

Thank you for reading our response and for your continued support. We are glad that the additional experiments strengthen the paper. Following your suggestions, the revision states the tabletop scope explicitly, discusses extensions to larger environments, deformable objects, and fluids, and adds a real-robot study. If you feel these additions further strengthen the work, we would be grateful if you would consider reflecting this in your final assessment.

Best regards, The Authors

Reviewer_zatYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TACER generates simulation-ready 3D tabletop scenes using hierarchical executable programs and verification-guided refinement. The paper introduces TabletopAudit for evaluating scene quality and robot interaction and TACER-11K with 11,399 scenes. Experiments demonstrate substantial improvements in scene fidelity and manipulation success, along with preliminary downstream policy-learning results.

Critical strengths

The paper addresses an important gap between geometrically plausible 3D scenes and scenes that are usable for robotic interaction. TabletopAudit provides a structured evaluation of input fidelity, scene quality, simulation validity, and robot task execution. This is a meaningful contribution because static geometric metrics alone do not adequately characterize whether generated environments support the intended robotic operations.

Critical weaknesses

The ablations show that refinement helps, but it is unclear whether the proposed method works better than simply allowing more retries. Some evaluation settings are lenient. Clutter collisions are disabled in certain stages, and articulated tasks can succeed by reaching the goal for just one frame.

Questions for authors
  1. How does TACER decide whether to modify an asset or adjust the scene layout when both could resolve a verification failure?

  2. Could the authors provide a small ablation comparing asset-level refinement only and scene-level refinement only?

3 comments
AuthorsOpenReview ↗

Response to Reviewer zatY (1/2): Choosing between asset and layout repairs, and a stricter protocol

We thank the reviewer for the constructive questions. Q1 addresses Question 1, Q2 addresses the lenient evaluation settings in the Weakness, and Q3 addresses the comparison with additional retries in the Weakness and Question 2.


Q1. How does TACER decide whether to modify an asset or adjust the layout when both could resolve a verification failure?

A1. TACER chooses the repair level from the failed check and what it identifies, rather than generating and comparing both repairs. For example, when two individually valid objects overlap, it keeps both assets and moves one of them.

Mechanism. An asset that fails its own checks is repaired or replaced at the asset level; a failure between objects, in their positions or support, is repaired by editing placement instructions in the scene program. Fixed rules map each failure type to the edit tried first (R2-Table 1), and re-execution decides whether the edit is kept: it must improve verification without regressions, or it is rolled back. Scenes still failing when the budget runs out remain unverified; layout failures are not automatically escalated to asset edits.

R2-Table 1. Failure types and the edit tried first.

Failure reported by verificationEdit tried first
Invalid geometry or physical representationAsset repair or replacement
No stable pose in isolated single-asset testsAsset repair or replacement
Floating above or penetrating the declared supportLayout: adjust placement height
Placement outside the support regionLayout: move into the region
Overlap between individually accepted objectsLayout: move one object
Violated spatial relation or articulation clearanceLayout: revise the placement instruction

Example. When two individually accepted objects overlap, swapping in a smaller asset could also remove the overlap, but the check reports a boundary overlap under the current placement, not evidence that either asset is mis-sized. TACER therefore moves one object and keeps both geometries; the edit is accepted only if the new layout still satisfies the original spatial relations and passes scene stability. Across the 500 image-track runs, verification triggered 1,174 layout edits and 329 asset edits, and 101 failures were such overlaps.

We will add this mapping to the failure-attribution step of Algorithm 1 and use the overlap example to separate two decisions: which edit is tried first, and what evidence accepts it.


Q2. How do the results change under a stricter protocol that enables clutter collisions and requires sustained goal attainment?

A2. Success drops for all methods and TACER still leads, but its margin narrows, especially on articulated tasks.

We thank the reviewer for pointing out this limitation. The original protocol disabled clutter collisions during target settling and rigid grasp replay, and passed an articulated task once its goal was reached for a single frame; it thus showed that the goal was reached, not that it was held.

Strict protocol. On the same 60 rigid and 40 articulated tasks, we enable clutter collisions throughout execution for all methods and keep the original controllers, goals, tolerances, and retry budgets. Task completion requires 1.0 s continuously within goal tolerance. For placement tasks, containment or support and stability are checked at every frame of the post-release window, not only at the final frame. Articulated goals must be held while force is applied; for example, Press must stay at the target while pressed but may rebound normally after release.

R2-Table 2. Task success on Embodied-100 under the original and strict protocols. Articulated tasks use TabletopGen + Articraft. GPT-6 Astra: an added baseline that writes Blender scene code; N/S: no articulated assets.

MethodRigid, originalRigid, strictArticulated, originalArticulated, strict
TabletopGen (+ Articraft)12/60 (20.0%)9/60 (15.0%)2/40 (5.0%)1/40 (2.5%)
GPT-6 Astra18/60 (30.0%)13/60 (21.7%)N/SN/S
TACER43/60 (71.7%)38/60 (63.3%)24/40 (60.0%)19/40 (47.5%)

Under the strict protocol, TACER's lead over TabletopGen (+ Articraft) narrows from 51.67 to 48.33 percentage points on rigid tasks and from 55.00 to 45.00 on articulated tasks. TACER's own articulated success drops by 12.50 points, which confirms the reviewer's concern about the single-frame criterion. Because the strict protocol changes collision handling and the holding requirement together, it measures their combined effect rather than either change alone. We will report both protocols, with their criteria, counts, and rates, in the evaluation section.

AuthorsOpenReview ↗

Response to Reviewer zatY (2/2): Contribution of each refinement level versus additional retries

Q3. What does each refinement level contribute, and can additional retries alone achieve the same gains?

A3. The two levels are complementary: layout-only refinement is stronger on spatial relations, placement, and rigid tasks, and asset-only refinement on articulated tasks. Full TACER exceeds the better single-level variant on both task types and outperforms retry-only under the same additional budget.

Protocol. We compare five configurations on 50 cases drawn from Embodied-100 (20 rigid and 30 articulated tasks), allocated across the nine domains in proportion and stratified by object count. No local refinement, asset-only, layout-only, and full TACER start from the same initial scene for each case. Retry-only regenerates candidates from the same input specification and keeps the best one under the same verifier (best-of-N), but never feeds failure reports into the next attempt; it thus isolates the effect of targeted edits from that of extra attempts. Asset-only freezes placement transforms, layout-only freezes assets, and full TACER may edit both. Retry-only, both single-level variants, and full TACER share the same additional per-case limits: 120 model calls, 80 asset operations, 4 candidate evaluations or replay validations, and 3 hours of execution. Failed attempts count toward these limits.

R2-Table 3. Input fidelity and scene quality on the 50-case subset (pass rate, %).

MethodCNTRELPLCATRCOL
No local refinement98.9862.3080.4877.5478.00
Retry-only99.1867.0284.8280.2182.00
Asset-only98.9867.0283.9583.9684.00
Layout-only99.3976.4492.6278.6192.00
Full TACER99.5985.3497.8384.4998.00

R2-Table 4. Simulation validity and task success under the original protocol (successes / tasks).

MethodRigid PHYSRigid STABArtic. STABRigid TASKArtic. TASK
No local refinement16/2010/2011/305/202/30
Retry-only17/2012/2015/307/206/30
Asset-only18/2014/2020/3010/2012/30
Layout-only18/2016/2018/3011/208/30
Full TACER19/2019/2026/3014/2018/30

Complementary levels. Layout-only achieves higher REL and PLC than asset-only (76.44%/92.62% vs. 67.02%/83.95%) and slightly higher rigid success (11/20 vs. 10/20). Asset-only better preserves attributes (ATR 83.96% vs. 78.61%) and succeeds on 12/30 articulated tasks versus 8/30, as many articulated failures arise from joints or collision proxies that layout edits cannot fix. Full TACER succeeds on 14/20 rigid and 18/30 articulated tasks, exceeding the better single-level variant by 15 and 20 percentage points.

Retries versus refinement. Retry-only improves over no local refinement, from 5/20 and 2/30 to 7/20 and 6/30, so additional candidates do help. Under the same limits, however, full TACER exceeds retry-only by 35 and 40 percentage points, showing that edits guided by failure reports are more effective than resampling whole candidates. With 20 and 30 tasks, one task corresponds to 5.0 and 3.3 points, so small differences between single-level variants should be read with care.

We will add these variants and the budget protocol to the ablation section. We thank the reviewer again for the constructive questions, which have clarified both the method and its evaluation.

Dear Reviewer zatY,

With the discussion period closing soon, we would be grateful if you could let us know whether our response and the added ablations address your questions. Thank you very much for your time.

Best regards,

The Authors

Reviewer_eexBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes TACER, an agentic framework for generating simulation-ready tabletop scenes from text or images. It combines asset retrieval and generation, hierarchical scene construction, and simulation-based verification and refinement. The authors also introduce TabletopAudit for evaluating physical validity and robot interactions, and release TACER-11K, a dataset of generated scenes.

Critical strengths
  • The paper addresses an important problem of generating simulation-ready manipulation environments for robot learning
  • A synthetic dataset of diverse physically plausible tabletop scenes with articulated objects could be useful for the community
Critical weaknesses
  • Poor qualitative results: The examples on the website and in the supplement show multiple physical inconsistencies: floating objects (office desk, gaming table, kitchen counter), disconnected components (hourglass), and implausible placements (upright test tubes). Generally, scenes look contrived due to simplistic object placement, low-quality geometry, and lack of stacking. Interaction examples also show collisions and strange physics, such as the gripper sticking to the handle in "Open" and collisions in "Place Into"/"Place Onto". This raises significant concerns about the effectiveness of the verification pipeline and the usefulness of the generated dataset.

  • Downstream evaluation: The authors collect 303 demonstrations for a simple pick-and-place task and show improved policy performance in generated scenes. However, it is unclear how this demonstrates the usefulness of the generated environments, especially with only one task and one scene. A more meaningful evaluation would test real-data-trained policies in generated environments or demonstrate sim-to-real transfer of policies trained on generated tasks.

  • Baseline comparisons: The qualitative results for SAM3D and TabletopGen look unusually poor and potentially unrepresentative. Relevant methods such as SceneSmith and stronger recent VLM-based approaches, such as GPT 6 Astra, are missing.

Questions for authors
  1. How do the authors explain the floating objects, collisions, and disconnected components in the qualitative results despite the proposed verification pipeline?
  2. What percentage of generated scenes pass all verification checks, and what is the average generation time and cost?
  3. Why are the baseline qualitative results so poor, and how would TACER compare against SceneSmith and stronger recent VLM-based approaches (GPT 6 Astra)?
  4. How representative are the qualitative examples of the released TACER-11K dataset?
6 comments
AuthorsOpenReview ↗

Response to Reviewer eexB (1/3): Sources of the visual defects and a stricter evaluation protocol

We thank the reviewer for carefully inspecting our project page and videos. Q1 addresses Weakness 1 and Question 1, Q2 addresses Question 4, Q3 addresses Weakness 3 and Question 3, Q4 addresses Question 2, and Q5 addresses Weakness 2.


Q1. Why do floating objects, disconnected components, and interaction artifacts appear despite the verification pipeline?

A1. The artifacts were real but came from three separate sources: a project-page renderer that is not used in verification or evaluation, a few asset defects now repaired across the whole dataset, and a lenient interaction protocol, which we have tightened. The interaction sequences themselves contain no collision or adhesion.

(i) Project-page rendering. The importer used only to render the project page omitted some URDF geometry, overwrote internal component transforms, and placed objects by declared rather than loaded dimensions, which left some objects floating. Verification (Drake) and all TabletopAudit evaluation (Isaac Sim and mesh metrics) load scenes directly from the released asset files, so no quantitative result passed through this importer. In released scenes, object heights come from the loaded meshes' bounding boxes, and poses are exported after settling.

R1-Table 1. Geometry completeness of 90 sampled TACER-11K scenes before and after fixing the page importer. Assets and layouts are unchanged; sampling is described in A2.

CheckPrevious importerCorrected importer
Scenes with omitted geometry61/900/90
Object instances with omitted geometry1190

After the fix, 89/90 scenes have no static screening flag (no triangle–surface intersections, below-table geometry, or boundary violations beyond 2 mm); one support-gap flag remains.

(ii) Asset defects. The hourglass's missing connector and the unsupported tapered test tubes were genuine asset defects: verification checks collision geometry and stability, which a disconnected visual part, or a stable collision proxy with an implausible visual shape, can pass. We added visual-mesh connectivity and visual–collision consistency checks and ran them on all 1,029 unique assets in TACER-11K; the 39 flagged assets have been repaired and the affected scenes re-verified.

(iii) Interaction sequences. In "Open," the finger and handle move almost in sync, which looks like adhesion. Physically, one finger lifts the handle from below as the lid rotates, with the gripper open; the minimum visual-mesh gap is about 0.22 mm, and no finger or palm vertex enters the handle or lid. The lid reached the goal only transiently, which the original single-frame criterion accepted; this case fails under the strict protocol below. In "Place Into/Onto," the die stays at least 9.6 mm from the cup wall, and the flat-bottomed bottle, lowered near-vertically, stays upright because its center of mass lies over its base. These cases still pass under the strict protocol, consistent with no collision.

Strict protocol. The original protocol disabled clutter collisions during target settling and rigid grasp replay, and accepted articulated goals reached for a single frame. We re-evaluated Embodied-100 with clutter collisions enabled throughout, a continuous 1.0 s hold within goal tolerance (for articulated tasks, while force is applied), and per-frame placement checks over the post-release window; controllers, goals, tolerances, and budgets are unchanged.

R1-Table 2. Task success on Embodied-100 under the original and strict protocols. Articulated tasks use TabletopGen + Articraft; N/S: no articulated assets.

MethodRigid, originalRigid, strictArticulated, originalArticulated, strict
TabletopGen (+ Articraft)12/60 (20.0%)9/60 (15.0%)2/40 (5.0%)1/40 (2.5%)
GPT-6 Astra18/60 (30.0%)13/60 (21.7%)N/SN/S
TACER43/60 (71.7%)38/60 (63.3%)24/40 (60.0%)19/40 (47.5%)

Stricter conditions lower success for all methods, and TACER remains the best on both subsets by a wide margin. We have regenerated all project-page renders with the corrected importer, and we will add the asset checks to App. A.3 and report both protocols in App. D.

AuthorsOpenReview ↗

Response to Reviewer eexB (2/3): Dataset representativeness and comparison with SceneSmith and GPT-6 Astra

Q2. How representative are the qualitative examples of TACER-11K?

A2. On a stratified sample frozen before inspection, the corrected scenes are free of the reported defects, and the new asset checks cover the whole dataset.

We took 10 scenes per environment from the 11,399-scene release list, ranking scenes within each environment by SHA256(seed | layout_type | scene_id) with fixed seed 20261107. The list was frozen before any geometry was inspected, no case was replaced, and it can be recomputed byte for byte from the release list. On these 90 scenes, the corrected renders show no omitted geometry and 89/90 have no static screening flag (R1-Table 1). Every TACER-11K scene passed all final verification checks and retains its programs, bindings, and verification records. We will revise the wording "available verification evidence" and add this protocol to the appendix.


Q3. Why are the baseline results poor, and how does TACER compare with SceneSmith and stronger VLM-based approaches such as GPT-6 Astra?

A3. Baselines use official implementations with models aligned to ours, and SAM3D additionally receives benchmark-assisted masks. On a 50-case subset, GPT-6 Astra builds realistic, well-arranged scenes that are far less usable for robot interaction, which is the gap TACER targets.

Baseline configurations. TabletopGen [1] uses its official implementation and defaults (including Hunyuan3D), with its VLM set to Kimi-K3 and Text-track reference images from GPT-image-2, matching TACER. SAM3D [2] receives the RGB image plus 6,483 instance masks and categories derived from benchmark annotations, more input than TACER uses. Its adapter places each GLB at its predicted pose and scale, then drops the scene's lowest point onto a fixed 110×110 cm table. We do not snap each object to a support, as that would import part of our own pipeline; this causes the floating and interpenetration seen in SAM3D results.

New baselines. The 50 cases are drawn from Embodied-100 (20 rigid and 30 articulated tasks), allocated across the nine domains in proportion and stratified by object count. SceneSmith [3] runs its official small-object stage on an empty 1.2×0.7 m table, with its native GPT-5.2 agents, asset routing and generation, and Drake physics; stacked and contained objects are evaluated against their actual support. GPT-6 Astra runs one independent high-effort session per case from an equally sized empty table, writing Blender/Python code and checking renders and MuJoCo contacts before submitting, with no external image or 3D generation (at most 32 tool actions and 3,600 s).

R1-Table 3. Text track on the 50-case subset (pass rate, %) and model cost per case.

MethodCNTRELPLCATRCOLSUPEDGECLRCost (USD)
TabletopGen67.0824.0025.0056.6568.0098.5796.7262.000.67
SceneSmith67.0874.5047.5062.4378.0081.5098.6936.003.41
GPT-6 Astra93.4684.5092.9267.0598.0099.22100.0076.003.22
TACER99.5981.5096.4667.63100.0098.64100.0078.001.29

R1-Table 4. Image track on the 50-case subset. Pass rates (%) and appearance similarity.

MethodCNTRELPLCATRCOLSUPLPIPS ↓DINO ↑CLIP ↑
GPT-6 Astra68.7150.7949.8972.1988.0096.120.28720.85560.9044
TACER99.5985.3497.8384.4998.0099.400.25030.84740.9269

Astra produces realistic scenes, leading text-track REL and image-track DINO. Yet under the strict protocol on RigidManip-60, it reaches 60.0% target stability and 13/60 task success, versus 93.3% and 38/60 for TACER, and it produces no articulated assets. Realistic appearance and plausible layout alone do not yield simulation-ready, interactable scenes; this gap motivates TACER and TabletopAudit. TACER also costs less than half as much per case as SceneSmith and Astra (USD 1.29 vs. 3.41 and 3.22); TabletopGen is cheaper (USD 0.67) but far less faithful to the input. We will document these settings, add both new baselines to the appendix, and revise the claim that TACER leads all image-track metrics.

References

[1] Wang et al. TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulation. 2025.

[2] SAM 3D Team. SAM 3D: 3Dfy Anything in Images. 2025.

[3] Pfaff et al. SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes. 2026.

AuthorsOpenReview ↗

Response to Reviewer eexB (3/3): Pass rate, generation cost, and real-robot transfer

Q4. What fraction of scenes pass all checks, and what are the generation time and cost?

A4. Refinement raises the all-check pass rate from 36.8% to 72.0%, at 21.37 min and USD 1.89 in model cost per passing scene.

We ran all 500 image-track inputs under the generation criteria (Sec. 3.5, App. A.3–A.4), counting failures and timeouts (40-min budget) as non-passes.

R1-Table 5. Verification pass rate and generation cost.

MetricResult
All checks passed: before / after refinement184/500 (36.8%) / 360/500 (72.0%)
Mean refinement rounds per input3.2
Mean end-to-end time: passing scenes / all inputs21.37 / 25.21 min
Mean asset-generation time (passing scenes)9.38 min
Drake checks per passing scene: placement / stable pose71 calls, 17.6 s / 10 calls, 14.3 s
Input / output tokens per input283.3K / 38.4K
Model cost per input / per passing sceneUSD 1.36 / 1.89

Rounds and tokens average over all 500 inputs. Costs include failures and retries and use Kimi-K3 prices (CNY 20/100 per MTok input/output, CNY 7/USD), excluding local compute. Main-table results use each method's final scene; TACER outputs that failed checks were submitted unfiltered. We will add this table to the appendix.


Q5. Do the generated environments help beyond one simulated task, for example in sim-to-real transfer?

A5. On a real Franka FR3, adding TACER simulation data raises π0.5 success from 6/10 and 4/10 (real data only) to 10/10 and 9/10 on two tasks, and TACER data alone, with no real demonstrations, reaches 5/10 and 4/10.

Setup. We teleoperated a Franka FR3 with GELLO [4], recording fixed and wrist cameras, joint states, and commands at 25 Hz. Banana pick succeeds when the whole banana is lifted off the plate and held for at least 0.5 s; apple-to-plate succeeds when the robot places the apple, releases, and retreats with the apple resting in the plate. Outcomes are judged from both views.

Data and training. We collected 30 banana and 50 apple demonstrations (24 and 43 for training). For each task, TACER built a twin of the real workspace with its standard pipeline and no manual alignment, and we generated 125 trajectories (105 for training) with ±1 cm and ±10° object perturbations, using simulated cameras with the real calibration. We fully fine-tune π0.5 [5] with 16-step action chunks, sampling real and simulated data 1:1 per batch, for 2,000 (banana) and 2,500 (apple) steps; all conditions share these settings. Simulation-only training uses 60 and 100 TACER trajectories, twice the real demonstrations.

Evaluation. Each condition is tested once on the same 10 real layouts per task, which deviate by up to 2.5 cm and 30°, beyond the simulated range.

R1-Table 6. Real-robot success of fine-tuned π0.5 (successes / 10).

Training dataBanana pickApple-to-plate
None (π0.5 base)0/100/10
Real only6/104/10
TACER simulation only5/104/10
Real + TACER simulation10/109/10

Videos. The anonymous project page now includes videos of teleoperated data collection, simulated demonstrations in the TACER twins, and real-robot rollouts for every training condition.

Without real demonstrations, TACER data alone performs comparably to real-only fine-tuning, and co-training performs best. Together with the simulated screwdriver study, this shows that TACER scenes support policy learning that transfers to a real robot. The study is small (two tasks, 10 trials each); we will add it as App. E.6 with full alignment details.

We hope these responses and the new results address the reviewer's concerns, and we would be grateful if the reviewer would take them into account when reconsidering the score.

References

[4] Wu et al. GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators. IROS, 2024.

[5] Physical Intelligence et al. π0.5: A Vision-Language-Action Model with Open-World Generalization. 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.