- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
OrbiSim is an asset- and parameter-conditioned world model that separates explicit state dynamics from visual generation. The differentiable pipeline supports prediction, physical-parameter identification, and gradient-based policy or action optimization. The paper evaluates these capabilities in simulation and on a real cable-manipulation task.
Critical strengths
- Separating explicit dynamics from visual generation makes physical parameters available for direct optimization. Connecting this interface to both system identification and control is practically interesting.
- The paper includes downstream evidence beyond prediction metrics, notably the Newton initial-velocity experiment and Cable Lift.
Critical weaknesses
- The Push success criteria require reconciliation. Appendix A.1 describes reaching a designated region while satisfying constraints against objects falling off the table, whereas Table 9 and Appendix C.11 count success when the second cube moves more than 5 cm. The reported 42.71% versus 25.00% therefore supports the latter metric, but cannot directly establish completion of the former task. Pretraining data, environment interactions, and update budgets should also be accounted for explicitly when interpreting the control comparison.
- The evidence for useful gradients has a narrower scope than a general replacement for a differentiable physics solver. Appendix C.10 optimizes a ball's initial velocity: Newton reaches zero loss within 60 steps, whereas OrbiSim reduces its loss below 10⁻⁴ within 800 steps. This demonstrates useful optimization in that example, rather than equivalent optimization efficiency or generally accurate contact gradients. Cable Lift provides further evidence, but its alternating identification and optimization should be distinguished from fixed-policy baselines in terms of test-time adaptation. Equation 12 also uses geometrically shaped rewards delivered at termination, which differs from binary success feedback.
Questions for authors
- Are the different success criteria in Appendix A.1 and Table 9 intentional? If trajectories are available, please report both criteria on the same trajectories and clarify each method's data, interaction, update, and adaptation budgets.
- For the existing Push or Cable Lift experiments, can the authors relate improvements predicted during model-based optimization to improvements after execution, including failure cases? This would directly strengthen the central claim about useful gradients without requiring a new benchmark suite.
4 comments
Response to Reviewer wWGk (Part 2)
W2.1: The Newton comparison leaves optimization efficiency and contact-gradient reliability unclear.
We agree that the ball experiment alone cannot establish comparable efficiency across tasks or gradient accuracy at all contact events. In the revised paper, we distinguish optimization outcomes from computational cost in Appendix C.13 and examine gradients near contact separately in Appendix C.9.
Optimization outcomes. The official Newton diffsim_ball task tests whether OrbiSim can optimize a task supported by an existing differentiable simulator using learned gradients. With the same learning rate, Newton reaches zero loss within 60 steps, while OrbiSim reduces the loss below within 800 steps. OrbiSim thus reaches a comparably low loss, although it converges more slowly on this task. We additionally implement a differentiable Newton baseline for Cable Lift and compare state-based system identification and action-sequence optimization on the same test cases. Figure 23 and Table 11 of the revised paper report the full results:
| Method | Target distance sum: before after (cm) | Physical-parameter error: before after | Average time per iteration (s) |
|---|---|---|---|
| Newton (ID) | 24.62 3.56 | 0.2086 0.1044 | 100.3 |
| OrbiSim (ID) | 24.62 1.22 | 0.2086 0.0293 | 77.5 |
| Newton (OOD) | 18.96 4.07 | 0.4518 0.1666 | 100.3 |
| OrbiSim (OOD) | 18.96 1.53 | 0.4518 0.0673 | 77.5 |
Both methods improve the target distance and parameter estimates under ID and OOD conditions. In this Cable Lift experiment, OrbiSim achieves lower final errors with less average time per optimization iteration.
Differentiable computation and runtime. Table 12 of the revised paper details the computation and gradient path within each integration substep across three tasks; Table 13 reports the corresponding rollout-update times. An update includes the forward rollout, backpropagation, and optimizer step.
| Task | Method | Control steps | Integration substeps | Rollout update time (s) | Amortized time per substep (ms) |
|---|---|---|---|---|---|
diffsim_ball | Newton | 36 | 288 | 0.23 | 0.81 |
| OrbiSim | 36 | 36 | 0.49 | 13.50 | |
Cable Lift | Newton | 40 | 2,560 | 2.00 | 0.78 |
| OrbiSim | 40 | 40 | 1.55 | 38.75 | |
Push | Newton | 80 | 4,000 | 182.81 | 45.70 |
| OrbiSim | 80 | 80 | 2.90 | 36.31 |
On diffsim_ball, Newton advances one particle with contact against static surfaces, yielding a lighter differentiable computation and a faster update than OrbiSim. Cable Lift requires gradients through a flexible particle chain, while Push additionally involves robot kinematics, control, rigid-body contacts, and their gradients. The cost of Newton's differentiable rollouts increases substantially across these implementations. OrbiSim instead optimizes through learned Dynamics transitions. The Cable Lift comparison provides optimization results; for Push, we report a runtime analysis, with an approximately 63-fold difference in single-rollout update time under the current implementations.
Gradients near contact. We do not interpret these optimization results as evidence that learned gradients are accurate at every contact event. Appendix C.9 and Figure 14 separately compare OrbiSim Jacobians with MuJoCo finite-difference estimates over a representative 40-step trajectory containing contact phases. Action-Jacobian cosine similarity is positive at 35 of 40 steps, and state-Jacobian cosine similarity is positive at all 40 steps, although local fluctuations occur near contact-mode transitions. Our claim is that the learned gradients provide useful local directions in the evaluated settings, not that they exactly reproduce derivatives at every instantaneous contact.
Revision: We clarify the purpose of the ball experiment and add the Cable Lift optimization comparison in Appendix C.13.1, the three-task computational analysis in C.13.2, and the contact-gradient analysis in C.9.
Response to Reviewer wWGk (Part 3)
W2.2: Distinguishing alternating test-time adaptation from fixed-policy baselines.
We agree that adaptation budgets should be matched. In the real-robot experiments in Sec. 4.3.1 of the revised manuscript, OrbiSim uses a single adaptation round: we collect eight real-robot RGB/action trajectories to identify the cable’s physical parameters, fix the inferred parameters, and optimize an action sequence for deployment. No additional observations are used to update the model or actions during deployment.
For a matched comparison, we additionally fine-tuned the baseline on the same eight trajectories and kept the resulting policy fixed during deployment. Both methods therefore receive the same real-robot data and one round of adaptation before evaluation. Using the same mean pixel-distance metric averaged over the two cable endpoints as in Table 10, we obtain:
| Method | Mean distance to target (px) ↓ |
|---|---|
| (original) | 62.2 |
| (fine-tuned on the same real-robot data) | 45.1 |
| OrbiSim | 17.5 |
Fine-tuning improves from 62.2 to 45.1 px, while OrbiSim retains a lower mean distance of 17.5 px under the matched adaptation-data budget.
The multi-round alternating procedure is evaluated separately in the controlled simulation diagnostics in Sec. 4.3.2 of the revised manuscript, with eight rounds of physical-parameter identification and action optimization under matched ID and OOD conditions.
Revision: We clarify the single-round real-robot adaptation in Sec. 4.3.1 and report the fine-tuned result in Appendix C.12 and Table 10.
W2.3: Limited evidence for OrbiSim under binary sparse rewards.
We thank the reviewer for raising this concern. We added an experiment on the robosuite Push task in which all three reward components, , were changed from the original dense rewards to sparse binary rewards defined by annotation thresholds. We relabeled the existing world-model training dataset with these sparse rewards and trained a reward predictor on the resulting labels. We then repeated the same policy optimization experiments, replacing the original reward signal with the predictor’s estimate of .
Figure 18 of the revised manuscript shows the training curves. Metric 1, Metric 2, and Metric 3 in the figure are computed using the original definitions of , , and , respectively, and Total is their sum.
As expected, policy performance is lower under sparse reward supervision than in the original dense-reward setting. Thresholding removes fine-grained reward information, while the reward predictor introduces approximation error and an additional stage in gradient propagation. Nevertheless, OrbiSim still achieves policy improvement in this setting: the learned predictor provides a smooth, differentiable surrogate for the sparse rewards, enabling gradient-based optimization through OrbiSim-Dynamics. These results demonstrate that OrbiSim can support policy learning with sparse binary reward supervision, with reduced performance relative to dense rewards.
Revision: Policy optimization results with binary rewards are updated in Appendix C.11.3.
Q2: Relating model-predicted gains to execution outcomes.
We compare the rewards of policies trained by model-based method in OrbiSim-Dynamics between rollouts in robosuite and in OrbiSim-Dynamics. The training curves of evaluation rollouts in robosuite and OrbiSim-Dynamics are shown in Figure 16 in the revised manuscript. Although detailed rewards calculated in OrbiSim-Dynamics have deviation comparing to rewards in robosuite, they have same growing trends and similar total rewards. While model-based policy optimization in OrbiSim do not requires accurate rewards, policies trained with gradients from OrbiSim-Dynamics gain improvements in robosuite evaluation.
Revision: Comparison settings and results are updated to Appendix C.11.2.
Thanks for the response. I’ve raised my score.
Response to Reviewer wWGk (Part 1)
1. Success criteria and new success rate results
The original manuscript stated inconsistent success criteria. We redesign the success criteria and re-evaluate success rates to fit our initial task objective. In our evaluation, a robosuite
Pushtask rollout is successful when r3≥0.2, where r3 is the relative reduction in the distance from the second cube to the table edge over the rollout, corresponding to a reduction of at least 20%. Under this criterion, OrbiSim achieves a success rate of 29.17%, compared with 4.92% for DreamerV3 and 0% for PPO, SAC, and behavior cloning.Revision: New success criteria and success reports are updated in paper's Appendix A.1, C.11 sections and Table 8.
2. Pretraining data, environment interaction and update budget statistics
Regarding pretraining data, collecting the robosuite dataset of approximately 234k frames (11,740 seconds of video) takes about one hour, and training OrbiSim-Dynamics takes about 30 hours on a single NVIDIA A800 GPU. We use the world model as a simulator and report this construction cost separately from policy training, analogous to setting up a simulation environment for downstream learning.
The table below distinguishes ground-truth simulator samples, imagined transitions, and policy optimizer updates. The model-free methods, PPO and SAC, collect their policy-training samples directly from robosuite and use no imagined transitions. OrbiSim uses the pre-collected dataset to train its world model, then trains its policy entirely within that model without further interaction with robosuite. DreamerV3 receives the same pretraining sample budget, but additionally collects simulator samples during training and uses imagined rollouts for policy learning. Optimizer-update counts are algorithm-specific and do not imply equal data or computational budgets.
Revision: We added computational costs and training budgets in Appendix C.14 and Table 14.