- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes WorldMirror, an agentic real-to-sim framework for constructing interactive digital twins from in-the-wild robot demonstrations. A router agent coordinates specialized reconstruction, calibration, and simulation tools while maintaining an editable world state. A camera-pose-anchored representation aligns the reconstructed scene, interactive objects, and robot, while verification feedback guides iterative refinement. The framework is evaluated on DROID demonstrations in terms of visual fidelity, spatial accuracy, and physical interaction consistency. WorldMirror achieves strong perceptual reconstruction quality, improved motion replay accuracy, and a reported 100% trajectory replay success rate. The main contribution is an integrated agentic framework for reconstructing simulation-ready environments through tool orchestration and verification-driven refinement.
Critical strengths
- A relevant problem with a coherent real-to-sim framework. Automatically constructing interactive simulation environments from real-world robot demonstrations is an important problem for scalable robot learning. WorldMirror integrates scene reconstruction, object modeling, camera calibration, and physical alignment within a unified agentic workflow. The structured world representation and verification-driven refinement provide a reasonable approach to constructing simulation-ready digital twins.
- Promising results across multiple reconstruction metrics. The paper evaluates visual quality, spatial consistency, and trajectory replay, covering several important aspects of real-to-sim reconstruction. WorldMirror achieves the best LPIPS and DISTS scores among the evaluated methods and substantially reduces motion replay errors. The reported improvements in contact fidelity and trajectory replay success are encouraging, although their interpretation depends on the evaluation protocol.
Critical weaknesses
- Insufficient validation of the core agentic framework. The central contribution of WorldMirror is its agentic construction framework, featuring adaptive tool orchestration and verification-driven refinement. However, the paper lacks controlled ablations isolating these mechanisms. In particular, there is no comparison between agentic orchestration and a fixed reconstruction pipeline using the same tools, or between reconstruction with and without verification. It therefore remains unclear whether the improvements arise from agentic decision-making, iterative refinement, the underlying reconstruction tools, or additional optimization effort. Sections 3.3 and 3.4 also provide limited implementation details regarding tool selection, verification criteria, refinement decisions, and termination conditions. The paper does not report reconstruction time, agent iterations, tool-call counts, or manual intervention requirements, making it difficult to assess the efficiency and scalability of the framework. Without these analyses, the contribution of the proposed agentic mechanisms remains insufficiently demonstrated.
- Insufficiently specified evaluation protocols for key quantitative claims. Several important aspects of the experimental evaluation are unclear. The visual evaluation covers only ten DROID scenes, while the spatial evaluation includes 17 objects and eight object pairs. More importantly, Table 4 reports a 100% trajectory replay success rate without specifying the number of evaluated trajectories, their selection criteria, or the definition of successful replay. The treatment of failed reconstructions is also unclear, making it difficult to assess the significance of this result. The construction of ground-truth measurements is another concern. Tables 3 and 4 report 3D object positions, motion errors, and contact F1, but the paper does not sufficiently explain how the corresponding real-world 3D states, object trajectories, and contact annotations are obtained from DROID demonstrations. These details are essential for interpreting the reported metrics. Furthermore, the comparison protocols are not fully specified, particularly regarding differences in input information, reconstruction tools, and computational budgets across methods. These omissions make the quantitative results difficult to independently verify and weaken the evidence supporting the paper's main claims.
- Limited evidence of physical interaction fidelity beyond demonstration replay. The paper aims to construct physically consistent interactive environments, but the evaluation primarily focuses on replaying the original robot demonstrations. Section 3.4 describes verification-driven refinement using consistency with the observed demonstrations, while Section 4.4 evaluates trajectory replay using those demonstrations. This raises concerns about whether the reported replay performance reflects accurate physical modeling or reconstruction specifically optimized for the demonstrated interactions. Successful replay provides useful evidence of geometric alignment and interaction feasibility, but does not necessarily establish that the reconstructed environments can accurately simulate different actions or unseen interaction outcomes. Evaluations with alternative robot trajectories, perturbed object configurations, or independent policy rollouts would provide stronger evidence of general physical consistency. Without such experiments, the paper demonstrates fidelity to the original demonstrations more convincingly than general-purpose physical simulation capabilities.
Questions for authors
Q1. Agentic framework and ablations. Could the authors compare WorldMirror against a fixed reconstruction pipeline using the same tools, and provide an ablation without verification-driven refinement? How are tool selection, refinement decisions, and termination conditions determined? Q2. Evaluation protocol and ground truth. How are the ground-truth 3D object positions, motion trajectories, and contact labels obtained? How many trajectories are evaluated in Table 4, and what criteria define the reported 100% replay success rate? Are failed reconstructions included? Q3. Physical interaction generalization. Are the trajectories used for verification-driven refinement also used for the replay evaluation? Could the authors evaluate different trajectories or independent policy rollouts to demonstrate physical consistency beyond the original demonstrations? Q4. Efficiency and reproducibility. What are the average reconstruction time, number of agent iterations, tool-call counts, and manual intervention requirements? How do these compare with the baselines? Could the authors also clarify the input information and computational budgets used for each method?