- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces NegativeSpace-WM, which skips world-model generation for candidate futures that seem physically impossible. Each candidate comes with a symbolic description of its expected outcome called a commitment. A Cosmos3-Nano model is fine-tuned with LoRA to write a structured explanation of why each candidate might be impossible such as which objects are involved, which physical rule is broken, and when. Then these are used to decided which hand-written physical checks to run first. A candidate is skipped only if one of those checks confirms the problem. In MuJoCo environment, this saves about 33% of GPU time compared with generating every candidate while preserving task success.
Critical strengths
I like the part that the learned model does not remove a candidate on its own, and that cancellation is limited to the scope of the clear (or verified) conflict, so other outcomes of the same action are kept. The structured explanation is also nicely designed.
Critical weaknesses
I see some clear limitations: First, the method shows narrow applicability. The rule out process needs three things: 1) The future must be written as a typed symbolic claim, 2) the physical premises must be explicit (rigid bodies, closed obstcles, etc.), and 3) the object geometry and poses must be calibrated. The checks also cover only five hand-written constraint types. These conditions seem easy to meet in MuJoCo, where the simulator knows every object's shape and position and the rules of physics are fixed. However, they are hard to meet in real scenes, which is where we need learned video world models. There is no simulator, object geometry and physical properties are unknown, and candidate outcomes are not written as symbolic statements. In those cases the checks would mostly return "unknown", and the advantage becomes decreased.
Second, I don't get the practical meaning of this method. When these conditions do hold, geometry and physics are already known, so the candidate could be checked or simulated directly, probably at lower cost than video generation. The learned part of the method also does not seem to matter much. A baseline that runs the same checks in a fixed order, with no learned model, performs about the same (Table 1B). For me, the saving seems to mainly comes from the hand-written checks, not from learning.
Lastly, evaluation seems unfair to test the method's novelty or superiority. The evaluation setup was largely designed by the authors such as the violation types, the symbolic commitment that describe each candidate's expected outcome, the available checks, and the proposer. Since the proposer decides the number of impossible candidates, the amount of compute saved also depends on this author-defined proposer. With this experiment, the advantage of using the proposed method is proved narrowly. It cannot show how the things going on in various scenarios can be seend in real scene.
Questions for authors
Q1. Can the method bring advantage (or can skip the candidate) when a setting was not designed for? For example, a case with real or uncalibrated video, or with a proposer that was not tuned for this setup?
Q2. When the geometry and object poses are known, we can roll out each candidate action with a physics simulator. Or, we could check directly whether the commitment is physically possible or not (without a learned model or video generation). I think these are simpler ways compared to the use of the proposed method. Does the proposed method bring advantages in terms of cost and accuracy compared to these simpler options?
Q3. Since the learned model only changes the order of the checks, the same result could be obtained by using fixed rule-out order with a larger budget. If so, the advantage of learned model seems quite limited. Is there a case where the learned explanation allows a candidate to be ruled out that the fixed-order checks cannot rule out, even with more checks?