- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper introduce the explosion problem with physics simulation in the context of simulated robotics benchmarks.
Critical strengths
The paper introduces a solution to the explosion problem, where it looks for specific huge deltas with manipulated objects and it limits the explosion.
Critical weaknesses
To me this paper is missing a key benchmark, robolab, which since its publication in June 2026 is now the standard for evaluating robotics policies. Moreover the explosion problem is a well known problem and thus not necessarily novel even though the applications to robotics simulation is important. I would have liked the authors to address this problem directly into the simulation rather than introducing an extra layer. Moreover the paper is hard to follow, the abstract and introduction need considerable amount of work.
Questions for authors
Here are my raw notes as I was reading.
ICLR 12374 Why is robolab not there? Ah not even cited! Three pages for an intro? Already I feel like without robolab it is hard to take this work seriously, I know it is built on top of PhysX or newton. “Testing a vision language action model and a world action model in simulation, we find that failures in which the policy has placed the object and the simulator then ejects it are frequent, and that the benchmark’s own scripted demonstrations fail the same way.” hard to parse. Ejecting like it is an airplane pilot or rejecting the condition for success? abstract is quite hard to follow and read. So I think the problem is physics explosion. The abstract does not serve its purpose well, I did not get some of the main findings, I did not read anything regarding how it improved things, just the problem and its technical solution.
Introduction. I think the problem observed is physics explosion or clipping explosion. This happens when clipping occurs in rigid body simulation. It is a well known problem in simulation and authors should refer to it as a known problem not as a surprise. The second paragraph should focus on related work at high level and what you are proposing as a solution. there is a lot of numbers for an intro, keep that for the experiments please. So reading the intro the authors propose a solution to the physics explosion problem.
I like the contact simulation in the related work.
7 comments
Dear Reviewer Hxnn,
Thank you again for your review. We ran two policies on the 120 RoboLab tasks, 4,800 episodes in total. We also fixed the simulator itself by making a threshold in PhysX's collision detection smaller: LingBot-VA rises from 89% to 98%, and no episode has an artifact.
If you have other questions, we are happy to discuss them.
Thank you for looking into my comments into such details. "numerical instability in physics simulation" is the problem name, I did a little bit more research on it using chatgpt, I only prompt about the problem no information about your submission was shared.
in soft body simulations https://publications.ri.cmu.edu/storage/publications/pub_files/pub1/baraff_david_1998_1/baraff_david_1998_1.pdf https://dl.acm.org/doi/10.1145/311535.311548
in rigid bodies https://cims.nyu.edu/gcl/papers/2021-RigidIPC.pdf "guarantees intersection-free configurations at every time step." I think the proposed benchmark here would be interesting to check against your proposed method.
https://www.cs.ubc.ca/labs/sensorimotor/projects/sp_sigasia08/ "accurate frictional contact resolution"
I am inclined to increase my score. I appreciate the effort that was put into the paper. Sorry for the late reply, CVPR took a lot of my attention. But I feel like the body of work that was put into rigid body simulation is very present, I am not an expert but I feel like a better coverage would help the language of the paper. I just did a 10 min research and found older relevant work, I am specially curious about the benchmark as well.
Response to Reviewer Hxnn
Thank you for your review. We reply to each point below. All changes in the revised PDF are in blue.
W1. RoboLab is missing.
We agree that RoboLab is now a standard benchmark for robot policies, and we apologize for missing it. The revised paper cites RoboLab and evaluates on it: two policies, each on 120 tasks × 20 episodes, 4,800 episodes in total. Success rates before and after the correction with SimuGuard and a human check:
SimuGuard can now identify and detect three classes of artifacts:
Videos of one example of each class are in the supplementary material.
With RoboLab, we cover 5 benchmarks on the two main physics engines, PhysX and MuJoCo: RoboLab, RoboTwin 2.0, RoboDojo, LIBERO and RoboCasa. All 5 show artifacts.
W2. Physics explosion is a well-known problem, so the work is not novel.
An object flying off in simulation, often called a physics explosion, is a known effect. Building on this, we measure how often it happens in policy evaluation and how much it changes the scores, and we find its cause in the engine. Prior work compared with this paper:
W3. The problem should be fixed in the simulator itself, not with an extra layer.
We agree. After the submission, we went on to fix the simulator itself, and the revised paper reports this. Take RoboTwin as an example. We used SimuGuard to find the source of the false contact in three steps:
The cause: when PhysX computes the distance between two shapes, it uses a fixed threshold, 2^-23, to decide whether a triangle is degenerate, that is, so flat that it is almost a line. This value is too large. Normal triangles on thin container walls are treated as degenerate, and the engine reports a contact that does not exist.
The PhysX docs limit how fast bodies are pushed apart. The false contact stays, and objects just fly off more slowly. Our fix only makes the threshold above seven orders of magnitude smaller, and the false contact is gone. The speed limit is tested on Place Object Basket, and our fix on five tasks:
W4. The abstract and introduction are hard to follow.
We rewrote both. The abstract now gives the main results: the cause of the artifacts, the fix, and the gain after the fix. We replaced "ejects" with "throws it out". The introduction now says that such ejections, often called physics explosions, are a known effect of rigid body solvers, and most of its numbers moved to the experiments. We will keep polishing the whole paper.