- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2(+1)
Read review
Paper summary
This paper focuses on the task of physically grounded video generation. The authors propose to use an agent-based system for the single image scene construction and the following physical simulation intialization. Moreover, a refinement loop via agent is further leveraged to refine the written simulation / dynamics for better physics guide. Following the refined simulation, the authors employ a pre-trained VACE model to take in various forms of physical conditions, to generate the final video following the physically plausible and desired dynamics and interactions. Authors mainly compare the experiment results from a holistic pipeline perspective, comparing with pure Wan video model to showcase the physical correctness brought by the whole system.
Critical strengths
The motivation of simulation refinement sounds interesting and useful. Although agent these days can write plausible dynamics or simulation, it can always improve by refining on top of its existing output.
Critical weaknesses
The main weakness of the paper is the lack of detailed explanation of the overall "agent system", that takes in one image + interaction, and construct underlying 3D scene with initialized simulation, and refine the simulation results. Also the underlying DSL, which is key to the whole system, seems also lack of any detailed explanation. I even didn't find the actual model of the so called agent here, is it an open-sourced model, or calling Claude Code / GPT API? What's the harness of this agent? What's the customized harness (e.g. DSL), and its own harness (e.g. Claude Code, Codex).
These days the state-of-the-art agent just need to specify the desired output condition format for VACE, they can definitely output reasonable dynamics, and even did the simulation construction, loop refinement all by their own without need for further instructions. So is the paper's contribution for this harness? If so the authors should explain it thoroughly and use corresponding experiment to prove its effectiveness comparing to a bare codex agent.
Questions for authors
- It seems unnecessary to compare to Wan-T2V baseline, which fundamentally isn't using the same task input as the proposed system here (using only text but not image).
- It's also meaningful to compare to at least 1 or 2 physics-grounded video generation baseline, just comparing to Wan is meaningless. The pipeline enables per-frame condition through VACE, which will almost always lead to better result comparing to pure image to video model's result.
- The experiment section is lack of the results on the cost for the pipeline, in both time-wise and token / money cost. This can also potentially show the usefulness of specializing the pipeline, instead of directly call a, e.g. codex agent.
4 comments
3. Physics-Grounded Baselines and Controlled Video Comparisons (Q2)
We agree that it is important to distinguish improvements in physical simulation from those introduced during video synthesis. We therefore provide comparisons at both stages of our text-to-physics-to-video pipeline.
(a) Physical simulation: PhysAgent comparison. We compare 21 pairs of rendered simulations against PhysAgent [1], which receives the event text, the corresponding first frame, and shared meshes. Our method achieves an Astra mean of 4.58 vs. 4.24 overall, including 4.75 vs. 4.33 on 12 action-search cases and 4.36 vs. 4.11 on nine passive-dynamics cases (Table 24, Appendix F.2). Supplementary S2 provides examples, while S3 reports all pairwise scores.
Importantly, these comparisons evaluate the simulated outcomes before final appearance synthesis, demonstrating that the observed improvements are not solely attributable to VACE conditioning.
(b) Final video: Same-checkpoint comparison. The gains are concentrated in the requested event and its physical interaction. With Wan2.1-VACE-14B fixed, our captioned configuration scores 4.23/3.44 on prompt alignment/physical plausibility, compared with 2.77/3.10 for direct T2V and 3.18/2.36 for direct I2V. Direct T2V leads in visual realism and temporal consistency. These results measure the full simulation-conditioned workflow at a fixed checkpoint (Table 1, Appendix C).
Taken together, the direct simulator coding comparison (already included in the original submission) demonstrates the benefits of structured agentic authoring, while the additional PhysAgent and fixed-checkpoint comparisons show improvements in simulated dynamics and final video generation, respectively. Our method shows aggregate gains in authoring and simulation quality, and gains in final-video prompt alignment and physical plausibility.
4. Refinement Effort and End-to-End Cost (Q3)
We agree that computational cost matters, but task completion is equally important. As shown in Table 3, direct simulator coding succeeds in only 7/13 scenarios, compared with 13/13 for our structured framework under the same agent model. Moreover, all 13 scenarios reach acceptance within 9 rounds (median 3), demonstrating that successful refinement requires relatively few iterations (Table 11). While we did not separately measure runtime or token costs, we agree that these would provide a more complete evaluation of computational efficiency.
Summary
We sincerely thank the reviewer for the constructive feedback.
Our framework addresses text-to-video generation through explicit physical world construction, rather than image-conditioned reconstruction or animation. Starting from the same text input as direct T2V methods, our agent constructs and refines an executable physical world, whose simulated rollout then guides appearance synthesis.
The additional comparisons further evaluate the contributions of structured scene authoring, physical simulation, and video synthesis at their respective stages. We hope these clarifications and results help address the reviewer's concerns, and we welcome further questions and discussion.
References
[1] Q. Li et al. PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generation. arXiv, 2026.
Thank the authors for detailed responses and large part rewriting of the paper to better state the methods, contributions and detail designs.
I can agree with authors' claim about that task is T2V and accept the current comparisons, but I still think to better show the "physic plausibility" of the proposed system (by the combination of agents, simulation, per-frame control, etc), a more informative comparison should prioritize the I2V baseline, which can clearly show the performance improvement comparing to a pure video model.
The part I still find hard to follow or can't agree much is the comparison of "direct genesis". Does this mean asking gpt-5.6-luna in one-shot to output genesis code in each round? I think a more straightforward comparison to actually measure the current harness is to directly describe the input / output settings of each round to a codex with gpt-5.6-luna, which I think should have much better performance.
For now I'll improve my scores but still put it at 2.
We thank the reviewer for the follow-up and for suggesting the direct Codex comparison.
I2V comparison. Table 1 of the revised paper already reports comparisons with Wan2.2 I2V and Wan2.1-VACE I2V across all 13 scenarios. Both I2V baselines receive the event description and the first frame of our corresponding captioned video.
Final-video physical plausibility across 13 scenarios (Table 1).
| Method | Mean physical plausibility (1–5) |
|---|---|
| Wan2.1-VACE I2V | 2.36 |
| Wan2.2 I2V | |
| Ours with rollout-derived captions | 3.44 |
Codex comparison. Following your suggestion, we ran Codex with GPT-5.6-Luna on all 13 scenarios. Codex edited Genesis Python directly and iterated with execution and visual feedback. It used the same task descriptions, assets, and ten-submission budget as Direct Genesis.
Scene-authoring results across 13 scenarios under each workflow's recorded protocol.
| Method | Online acceptance | Mean simulation-preview score (1–5) |
|---|---|---|
| Codex + GPT-5.6-Luna | ||
| Our structured DSL workflow | 13/13 | 4.23 |
Table 3 reports the DSL results. Supplementary Section S4 provides side-by-side Codex and DSL simulation previews for all 13 scenarios, along with per-scenario results and the evaluation protocol.
Conclusion. The final-video gain in physical plausibility holds at the same Wan2.1-VACE checkpoint. In scene authoring, our structured DSL workflow outperforms direct Genesis authoring with Codex and the same GPT-5.6-Luna model on both online acceptance and simulation-preview quality under the evaluated protocols.
We thank the reviewer for the feedback. We would first like to clarify our task formulation, which is directly relevant to the reviewer's concerns about our experimental comparisons. We then address the questions regarding agent implementation, the DSL, physics-grounded baselines, and computational cost.
1. Clarification of the Task Formulation (W1, Q1).
Our method takes only a text prompt as input. No input image, reference video, or preconstructed 3D scene is provided. The agent autonomously constructs, executes, and iteratively refines a physical world from text, and the accepted simulation subsequently guides video synthesis.
We would like to clarify this distinction because several statements in the review appear to describe a different task setting:
These descriptions do not reflect our task formulation. The images inspected by our VLM are internally rendered previews of the agent-generated simulations, not external inputs. The first-frame images used by I2V baselines are provided only for those baseline evaluations.
Consequently, Wan-T2V and our method receive the same task input: the event text. The direct T2V comparison is therefore task-matched and addresses our central research question: Does constructing and simulating an explicit physical world before synthesizing pixels improve text-to-video generation?
We hope this clarification helps resolve the concern in Q1 and provides the appropriate context for evaluating our methodology and experiments.
2. Agent Model, Controller, and DSL (W1)
The scene author and the rollout reviewer both use
gpt-5.6-lunavia API calls orchestrated by our Python controller. Our contribution lies not in developing a new agent model, but in designing an agentic framework for constructing and iteratively refining executable physical worlds from text through simulation and visual feedback.The controller exposes asset tools, deterministic scene validation, simulation, indexed rollout frames, and edits to named DSL fields. The DSL specifies geometry, materials, physical roles, initial states, and motion commands. The VLM reviewer can request more frames and propose edits, which are validated before being committed.
For e.g., in Fig. 3, the initial basketball trajectory misses the rim. The agent iteratively revises the launch state and hoop configuration, achieving the intended bank shot in round 6. The accepted simulation provides aligned depth and contours for VACE conditioning, along with material masks for combining denoising updates (Sections 3.2–3.4, Appendices A and D.3).
We also compare structured DSL authoring against direct simulator coding using the same agent model and event descriptions, with both workflows receiving execution and visual feedback. Direct Genesis achieves acceptance on 7/13 cases, versus 13/13 for our structured workflow, with last-complete-preview mean scores of 3.58 vs. 4.23, respectively (Table 3, Appendix D.1). These results support the effectiveness of structured scene authoring over direct simulator coding under the evaluated asset histories and budgets.