- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposed STRIKE to help video generation models handle physical interactions by explicitly predicting how a scene changes step-by-step rather than guessing entire motions at once. E.g., when picking up a cube, the model maps out the timing of events like contact and lifting, then predicts the visual state after each interaction, and finally use such states to guide the diffusion model to synthesize the final sequence. This approach allows STRIKE to significantly improve physical consistency and video fidelity across benchmarks, e.g., Physics-IQ Verified and RoboTwin 2.0.
Critical strengths
-
Explicitly modeling interaction outcomes before generating the video is a reasonable approach. The separation between state-transition prediction and video dynamics is also a clear structure.
-
STRIKE improves performance across physical-consistency, object-motion, and robotic-manipulation evaluations, e.g., 76.91 on PhyGenBench (vs. 54.79 for the CogVideoX baseline).
-
Good ablation studies, e.g., training the transition model improves its state-prediction quality vs. using the pretrained image-editing model directly. I also appreciate the temporal conditioning, first-frame anchoring, and synthetic training data studies.
-
Also exhaustive results on both real-world and simulated data.
Critical weaknesses
-
Weak novelty because the idea of planning events, predicting intermediate visual states, and conditioning video generators on those states is already proposed in papers like Key World and CausalMotion. The counterargument could be the introduction of transition model, however, it is not clear on how much gain does is introduce from learning transitions rather than simply providing better intermediate visual conditions to the video generator.
-
Unfair comparison from model size perspective. STRIKE combines a 20B transition model with a 5B video generator (25B+ in total), whereas several baselines use much smaller models.
-
The error from each autoregressive state prediction could accumulate. While the paper acknowledges this limitation, a systematic rollout-horizon study or length-dependent evaluation would confirm how severely these errors impact performance over longer sequences.
-
The role and reliability of the event planner requires further evaluation. Even a strong transition model may generate incorrect videos if the event sequence or timing is wrong, and an evaluation comparing predicted event plans against reference plans would confirm the planner's reliability.
Questions for authors
-
How much improvement comes from the transition model vs. simply conditioning on intermediate states?
-
How does performance degrade as the number of predicted transitions increases?
-
How sensitive is the transition model to the train/test domain gap. E.g., if the training is purely synthetic data, would the result be much worse?
5 comments
Response to Reviewer BLCA (2/4)
W3: We that two facts bound the issue on our benchmarks:
- Rollouts are short. On Physics-IQ the planner specifies 1, 2, or 3 future states for 39, 138, and 21 of 198 videos (mean 1.91).
- The design limits drift. The first-frame sink keeps the input image as a persistent reference (without it, Physics-IQ drops to 40.60 and Pisa-IoU to 0.1528, Table 7), and states are soft conditions.
We added two analyses on our validation dataset:
- Step-wise state accuracy on validation dataset (VAL). We find that adding first-frame sink in the transition model during can significantly mitigate the step-wise error accumulation.
| States Count | PSNR Rec. | PSNR TF | PSNR Rec. w/o sink | SSIM Rec. | SSIM TF | SSIM Rec. w/o sink | LPIPS Rec. | LPIPS TF | LPIPS Rec. w/o sink | PSNR Rec. | PSNR Rec. w/o sink |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 22.33 | 22.33 | 22.35 | 0.783 | 0.783 | 0.782 | 0.217 | 0.217 | 0.221 | 0.00 | +0.02 |
| 2 | 21.21 | 23.22 | 20.85 | 0.752 | 0.781 | 0.744 | 0.267 | 0.220 | 0.279 | −2.01 | −2.37 |
| 3 | 19.57 | 22.13 | 18.57 | 0.712 | 0.758 | 0.690 | 0.328 | 0.254 | 0.355 | −2.56 | −3.56 |
| 4 | 15.62 | 18.65 | 14.91 | 0.627 | 0.679 | 0.578 | 0.431 | 0.308 | 0.473 | −3.04 | −3.75 |
- In addition, we ran the following analyses. The first three columns use our validation videos (VAL) with annotated reference plans and states. We report S-IoU, WS-IoU, ST-IoU against the real videos inspired by Physics-IQ verified benchmark. For the benchmark dataset, we extract state frames from original video clips by running our data curation pipeline.
| State condition supplied to the video model | S-IoU | WS-IoU | ST-IoU | Physics-IQ |
|---|---|---|---|---|
| (a) Reference states | 0.693 | 0.524 | 0.336 | 63.23 |
| (b) Generated, teacher-forced | 0.593 | 0.394 | 0.221 | 45.59 |
| (c) Generated, recursive w/o sink | 0.497 | 0.318 | 0.201 | 40.60 |
| (d) Generated, recursive (STRIKE) | 0.530 | 0.341 | 0.211 | 41.93 |
Here Reference States means using the ground truth state images as condition for video generation, which is set as an upper bound of the system. Our conclusion is that error accumulation is measurable but accounts for a minority of that headroom.Teacher-forced (b) states improve over our recursive (d) ones by +0.063 S-IoU, about 39% of the oracle gap; the rest is single-step error. For Physics-IQ, the teacher-forced states only advance ours +3.66 on benchmark score, which only accounts for 17% of the difference compared with reference states. In addition, adding first frame sink (comparing (c) and (d)) can effectively mitigate the error accumulation problem.
Response to Reviewer BLCA (3/4)
W4: We thank the reviewer for this question. We answer it in two parts: (1) how accurate GPT-5.5's plans are, measured against reference plans and against the real videos; (2) how much its errors cost in the generated videos.
1. How accurate is the planner on our own validation dataset?
(a) Against the reference plan . For every pair of predicted and reference states of our validation dataset, a judging VLM (Claude Opus 5.5) scores whether the two describe the same physical state (same entity, same phase of the process, same outcome), with the matching rule is that pairs scoring ≥ 0.75 with no incompatibility are matched one-to-one. The evaluation results are shown below:
| Metric | Planner with GPT 5.5 |
|---|---|
| Event-count (exact-K rate) | 87.5% |
| Event precision | 0.73 |
| Pairwise order agreement on matched events | 0.99 |
| Mean timestamp error on matched events | 0.45 s |
(b) Against the real video. A judging VLM (Opus 5.5) sees the real clip and one list of states, without times or source. For each state it decides (1) whether the state occurs (yes / partial / no); (2) when it first becomes visible. It also lists clearly visible state changes that the list misses. The results are as follows:
| Validation clips | GPT-5.5 plan | Reference annotation | Δ |
|---|---|---|---|
| State occurs: yes / partial / no | 64.5% / 25.4% / 10.1% | 83.9% / 13.7% / 2.4% | "yes" −19.4 pts |
| Timing error vs. observed onset (states judged "yes") | 1.11 s | 0.55 s | +0.57 s |
| Visible state changes not covered, per clip | 1.83 | 1.63 | +0.20 |
Our conclusions are that GPT-5.5 gets the order of events right, and its states are realized fully or partly 90% of the time; 10% do not occur. It covers nearly as many visible events as the annotation with its timing is about 0.6 s coarser. However, the planner sees only the first frame, so these are prediction errors, not perception errors.
2. How do planning errors affect generation?
(a) Predicted vs. reference plan, everything else fixed. On the same validation clips we generate each video twice: once from the reference plan and once from GPT-5.5's plan. The transition model, video model and seed are identical. We conduct comparison experiments with the real continuation:
- Motion-mask IoU, using Physics-IQ's mask algorithm. S-IoU measures where motion occurs; WS-IoU also weights how often it occurs.
- A VLM judge (Claude Opus 5.5, blind to which plan was used). It compares real and generated frames at the same times and scores event, final-state and timing agreement in [0, 1].
For scale, the table includes a run that uses real frames as states (the best score can be achieved).The results are shown in the following table:
| Plan | States | S-IoU | WS-IoU | VLM judge |
|---|---|---|---|---|
| GPT-5.5 (predicted) | generated | 0.511 | 0.316 | 0.326 |
| Reference | generated | 0.530 | 0.341 | 0.364 |
| Reference | real frames | 0.693 | 0.525 | – |
Reading from the table, our conclusions are as follows:
- Planning gap (GPT-5.5 plan − reference plan): S-IoU −0.019, WS-IoU −0.025, VLM judge −0.038.
- State gap (real frames − generated states): S-IoU +0.163, about 8× the planning gap. The remaining headroom lies mainly in state generation, not in planning.
- Per clip. Clips with worse plans get slightly worse videos.
(b) Swapping the planner on Physics-IQ.
- Only the plans change; the transition model, video model, seed and scorer are fixed.
- Δ is paired vs. GPT-5.5 over the 66 scenarios.
| Planner | Physics-IQ | Δ vs. GPT-5.5 |
|---|---|---|
| None (CogVideoX-5B-I2V backbone) | 30.5 | – |
| GPT-5.5, (paper) | 41.93 | – |
| Qwen3-VL-8B | 39.94 | −1.99 |
| Qwen2.5-VL-7B | 35.52 | −6.41 |
Both Qwen2.5-VL and Qwen3-VL perform worse than GPT5.5 on physics-IQ benchmark
To summarize:
- Accuracy. GPT-5.5 orders events correctly (0.99 pairwise agreement), and 90% of its planned states actually occur, fully or partly, in the real video.
- Effect on generation. These errors cost little. Replacing the reference plan with GPT-5.5's plan lowers spatial IoU by only 0.019, one-eighth of what imperfect state generation costs (0.163). Controlled timing, order and drop errors change Physics-IQ by at most 1.7 points, and none of these changes is significant.
- Choice of planner. Large losses (6 points) appear in other planner (Qwen2.5-VL) compared with more advanced VLM (GPT 5.5)
Response to Reviewer BLCA (1/4)
We thank the reviewer for finding the explicit modeling of interaction outcomes reasonable and the decomposition clear, and for appreciating the ablations and the breadth of real and simulated evaluations. We address each point below.
W1: We do not claim keyframe-guided generation itself as new. Our contribution is a learned transition unit: one shared model, trained on event-aligned state pairs, that maps (current state, local transition specification, elapsed time) → next state. It is composed recursively, and its timed outputs condition a video model that generates the full window jointly.
How much comes from learning transitions: Table 6 isolates exactly this. With planner, video model, parameters, and FLOPs held fixed, intermediate states from the pretrained editor vs. our learned transition model give:
Comparing (a) and (b), (c) and (d), which (b) and (d) provide intermediate states using pretrained models, our learned transition model perform better in all three benchmarks.
W2: We thank the reviewer for raising this point. We agree that model size matters for a fair comparison; this is why Table 1 reports both parameter counts and generation cost (EFLOPs) for every method. We believe the comparison does not favor STRIKE, for three reasons.
1. With the same parameters, other systems do much worse. CoECT [1] uses the same composition as STRIKE, a 20B image editor plus a 5B video generator. Yet it scores 19.9 on Physics-IQ, vs. 41.9 for STRIKE, and needs about 3× the generation cost (29.6 vs. 10.02 EFLOPs). The gain therefore comes from what the transition model learns, not from its parameter count.
2. Much larger systems are not clearly better. Cosmos3-Super has 64B parameters, 2.6× STRIKE's 25B, yet scores only 0.8 points higher (42.7 vs. 41.9). MiniMax-H3 has 33B parameters and scores 39.8, 2.1 points below STRIKE. STRIKE is thus on par with systems well above its size.
3. Smaller baselines put their capacity into the video model; STRIKE puts it into an image model, which is cheaper to train and to run. Most baselines below 25B parameters rely on a large video backbone: Wan2.2 (14B), Cosmos3-Nano (16B), LTX-Video (13B), Open-Sora v2 (11B). STRIKE instead places most of its parameters in an image-space transition model and keeps the video model small (5B, or 2B).
Within the same model family, STRIKE with Wan2.2-5B outperforms Wan2.2-14B (39.7 vs. 34.2) at 1/12 of the generation cost (4.19 vs. 49.1 EFLOPs). On VideoPhy, STRIKE with CogVideoX-2B obtains the best physical-commonsense score (32.9). This is above its own 2B backbone (27.6) and above all 11–16B video models: Open-Sora v2 20.2, LTX-Video 20.9, WISA 18.0, Cosmos3-Nano 19.5. Its semantic adherence is comparable (70.4 vs. 70.5 for WISA-14B).
In addition, An image model trains on single frames instead of long clips, so sequences are far shorter, memory is lower and batches are larger. A 20B image model is thus much cheaper to train than a video model of similar size, and it runs only once per state at inference.
In short, STRIKE does not win by being larger. Systems of the same size do much worse, much larger systems are not clearly better, and the smaller baselines spend their capacity on the video model, which is the more expensive place to put it.