acceptodds
Under review as a conference paper at ICLR 2027

World Before Pixels: Agentic Physical Simulation for Text-to-Video Synthesis

Abstract

Generative text-to-video synthesis has achieved striking visual realism, yet current models still struggle to ensure that scene geometry supports the requested interaction and that the resulting motion follows physical laws. Faithfully modeling the world requires understanding not only visual appearance, but also object geometry and material properties, physical states and interactions, and the causal dynamics that govern how these states evolve—knowledge that is difficult to acquire implicitly from even massive collections of video data. We present an agentic framework that addresses these challenges by constructing, validating, and simulating an explicit physical world before synthesizing its appearance. Given a text prompt, our agent constructs a structured and executable scene whose geometry, materials, physical roles, and simulation parameters expose the variables governing the requested event. Simulation then turns these coupled choices into observable consequences, including rigid-body contact, deformation, and fluid motion, enabling a vision-language controller to assess the simulated outcome and revise the scene through validated edits and re-execution. This closed loop progressively aligns the constructed world with the intended event. Finally, as the requests from the initial text prompt are met, verified geometric priors are rendered from the simulated rollouts, supplying aligned depth, contour, and material controls for a pretrained video model. This combines simulated dynamics with generative appearance synthesis without additional model training. Experiments across diverse physical scenarios show improved physical fidelity over direct text-to-video generation and ablated variants.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
text-to-video generation, agentic graphics, physical simulation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zs8cOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

This paper focuses on the task of physically grounded video generation. The authors propose to use an agent-based system for the single image scene construction and the following physical simulation intialization. Moreover, a refinement loop via agent is further leveraged to refine the written simulation / dynamics for better physics guide. Following the refined simulation, the authors employ a pre-trained VACE model to take in various forms of physical conditions, to generate the final video following the physically plausible and desired dynamics and interactions. Authors mainly compare the experiment results from a holistic pipeline perspective, comparing with pure Wan video model to showcase the physical correctness brought by the whole system.

Critical strengths

The motivation of simulation refinement sounds interesting and useful. Although agent these days can write plausible dynamics or simulation, it can always improve by refining on top of its existing output.

Critical weaknesses

The main weakness of the paper is the lack of detailed explanation of the overall "agent system", that takes in one image + interaction, and construct underlying 3D scene with initialized simulation, and refine the simulation results. Also the underlying DSL, which is key to the whole system, seems also lack of any detailed explanation. I even didn't find the actual model of the so called agent here, is it an open-sourced model, or calling Claude Code / GPT API? What's the harness of this agent? What's the customized harness (e.g. DSL), and its own harness (e.g. Claude Code, Codex).

These days the state-of-the-art agent just need to specify the desired output condition format for VACE, they can definitely output reasonable dynamics, and even did the simulation construction, loop refinement all by their own without need for further instructions. So is the paper's contribution for this harness? If so the authors should explain it thoroughly and use corresponding experiment to prove its effectiveness comparing to a bare codex agent.

Questions for authors
  1. It seems unnecessary to compare to Wan-T2V baseline, which fundamentally isn't using the same task input as the proposed system here (using only text but not image).
  2. It's also meaningful to compare to at least 1 or 2 physics-grounded video generation baseline, just comparing to Wan is meaningless. The pipeline enables per-frame condition through VACE, which will almost always lead to better result comparing to pure image to video model's result.
  3. The experiment section is lack of the results on the cost for the pipeline, in both time-wise and token / money cost. This can also potentially show the usefulness of specializing the pipeline, instead of directly call a, e.g. codex agent.
4 comments

We thank the reviewer for the feedback. We would first like to clarify our task formulation, which is directly relevant to the reviewer's concerns about our experimental comparisons. We then address the questions regarding agent implementation, the DSL, physics-grounded baselines, and computational cost.

1. Clarification of the Task Formulation (W1, Q1).

Our method takes only a text prompt as input. No input image, reference video, or preconstructed 3D scene is provided. The agent autonomously constructs, executes, and iteratively refines a physical world from text, and the accepted simulation subsequently guides video synthesis.

We would like to clarify this distinction because several statements in the review appear to describe a different task setting:

Summary: "single image scene construction"

W1: the agent "takes in one image + interaction"

Q1: Wan-T2V "fundamentally isn't using the same task input as the proposed system here (using only text but not image)"

These descriptions do not reflect our task formulation. The images inspected by our VLM are internally rendered previews of the agent-generated simulations, not external inputs. The first-frame images used by I2V baselines are provided only for those baseline evaluations.

Consequently, Wan-T2V and our method receive the same task input: the event text. The direct T2V comparison is therefore task-matched and addresses our central research question: Does constructing and simulating an explicit physical world before synthesizing pixels improve text-to-video generation?

We hope this clarification helps resolve the concern in Q1 and provides the appropriate context for evaluating our methodology and experiments.

2. Agent Model, Controller, and DSL (W1)

The scene author and the rollout reviewer both use gpt-5.6-luna via API calls orchestrated by our Python controller. Our contribution lies not in developing a new agent model, but in designing an agentic framework for constructing and iteratively refining executable physical worlds from text through simulation and visual feedback.

The controller exposes asset tools, deterministic scene validation, simulation, indexed rollout frames, and edits to named DSL fields. The DSL specifies geometry, materials, physical roles, initial states, and motion commands. The VLM reviewer can request more frames and propose edits, which are validated before being committed.

For e.g., in Fig. 3, the initial basketball trajectory misses the rim. The agent iteratively revises the launch state and hoop configuration, achieving the intended bank shot in round 6. The accepted simulation provides aligned depth and contours for VACE conditioning, along with material masks for combining denoising updates (Sections 3.2–3.4, Appendices A and D.3).

We also compare structured DSL authoring against direct simulator coding using the same agent model and event descriptions, with both workflows receiving execution and visual feedback. Direct Genesis achieves acceptance on 7/13 cases, versus 13/13 for our structured workflow, with last-complete-preview mean scores of 3.58 vs. 4.23, respectively (Table 3, Appendix D.1). These results support the effectiveness of structured scene authoring over direct simulator coding under the evaluated asset histories and budgets.


3. Physics-Grounded Baselines and Controlled Video Comparisons (Q2)

We agree that it is important to distinguish improvements in physical simulation from those introduced during video synthesis. We therefore provide comparisons at both stages of our text-to-physics-to-video pipeline.

(a) Physical simulation: PhysAgent comparison. We compare 21 pairs of rendered simulations against PhysAgent [1], which receives the event text, the corresponding first frame, and shared meshes. Our method achieves an Astra mean of 4.58 vs. 4.24 overall, including 4.75 vs. 4.33 on 12 action-search cases and 4.36 vs. 4.11 on nine passive-dynamics cases (Table 24, Appendix F.2). Supplementary S2 provides examples, while S3 reports all pairwise scores.

Importantly, these comparisons evaluate the simulated outcomes before final appearance synthesis, demonstrating that the observed improvements are not solely attributable to VACE conditioning.

(b) Final video: Same-checkpoint comparison. The gains are concentrated in the requested event and its physical interaction. With Wan2.1-VACE-14B fixed, our captioned configuration scores 4.23/3.44 on prompt alignment/physical plausibility, compared with 2.77/3.10 for direct T2V and 3.18/2.36 for direct I2V. Direct T2V leads in visual realism and temporal consistency. These results measure the full simulation-conditioned workflow at a fixed checkpoint (Table 1, Appendix C).

Taken together, the direct simulator coding comparison (already included in the original submission) demonstrates the benefits of structured agentic authoring, while the additional PhysAgent and fixed-checkpoint comparisons show improvements in simulated dynamics and final video generation, respectively. Our method shows aggregate gains in authoring and simulation quality, and gains in final-video prompt alignment and physical plausibility.


4. Refinement Effort and End-to-End Cost (Q3)

We agree that computational cost matters, but task completion is equally important. As shown in Table 3, direct simulator coding succeeds in only 7/13 scenarios, compared with 13/13 for our structured framework under the same agent model. Moreover, all 13 scenarios reach acceptance within 9 rounds (median 3), demonstrating that successful refinement requires relatively few iterations (Table 11). While we did not separately measure runtime or token costs, we agree that these would provide a more complete evaluation of computational efficiency.


Summary

We sincerely thank the reviewer for the constructive feedback.

Our framework addresses text-to-video generation through explicit physical world construction, rather than image-conditioned reconstruction or animation. Starting from the same text input as direct T2V methods, our agent constructs and refines an executable physical world, whose simulated rollout then guides appearance synthesis.

The additional comparisons further evaluate the contributions of structured scene authoring, physical simulation, and video synthesis at their respective stages. We hope these clarifications and results help address the reviewer's concerns, and we welcome further questions and discussion.


References

[1] Q. Li et al. PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generation. arXiv, 2026.

Thank the authors for detailed responses and large part rewriting of the paper to better state the methods, contributions and detail designs.

I can agree with authors' claim about that task is T2V and accept the current comparisons, but I still think to better show the "physic plausibility" of the proposed system (by the combination of agents, simulation, per-frame control, etc), a more informative comparison should prioritize the I2V baseline, which can clearly show the performance improvement comparing to a pure video model.

The part I still find hard to follow or can't agree much is the comparison of "direct genesis". Does this mean asking gpt-5.6-luna in one-shot to output genesis code in each round? I think a more straightforward comparison to actually measure the current harness is to directly describe the input / output settings of each round to a codex with gpt-5.6-luna, which I think should have much better performance.

For now I'll improve my scores but still put it at 2.

We thank the reviewer for the follow-up and for suggesting the direct Codex comparison.

I2V comparison. Table 1 of the revised paper already reports comparisons with Wan2.2 I2V and Wan2.1-VACE I2V across all 13 scenarios. Both I2V baselines receive the event description and the first frame of our corresponding captioned video.

Final-video physical plausibility across 13 scenarios (Table 1).

MethodMean physical plausibility (1–5)
Wan2.1-VACE I2V2.36
Wan2.2 I2V2.92‾\underline{2.92}
Ours with rollout-derived captions3.44

Codex comparison. Following your suggestion, we ran Codex with GPT-5.6-Luna on all 13 scenarios. Codex edited Genesis Python directly and iterated with execution and visual feedback. It used the same task descriptions, assets, and ten-submission budget as Direct Genesis.

Scene-authoring results across 13 scenarios under each workflow's recorded protocol.

MethodOnline acceptanceMean simulation-preview score (1–5)
Codex + GPT-5.6-Luna8/13‾\underline{8/13}3.46‾\underline{3.46}
Our structured DSL workflow13/134.23

Table 3 reports the DSL results. Supplementary Section S4 provides side-by-side Codex and DSL simulation previews for all 13 scenarios, along with per-scenario results and the evaluation protocol.

Conclusion. The final-video gain in physical plausibility holds at the same Wan2.1-VACE checkpoint. In scene authoring, our structured DSL workflow outperforms direct Genesis authoring with Codex and the same GPT-5.6-Luna model on both online acceptance and simulation-preview quality under the evaluated protocols.

Reviewer_8xiJOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

World Before Pixels proposes an agentic framework for text-to-video synthesis that constructs, validates, and simulates an explicit physical world before synthesizing pixels: a scene author translates the prompt into a structured, executable scene DSL (geometry, materials, physical roles, sim settings); deterministic validation gates execution; a vision-language controller inspects rendered rollouts, requests extra frames to resolve ambiguity, and proposes validated edits in a closed refinement loop; the accepted rollout then supplies temporally aligned depth/contour/material controls (plus an optional rollout-grounded caption) to a pretrained Wan2.1-VACE-14B without any additional training. On 13 physical benchmark scenarios, evaluated by gpt-5.6-sol under a four-criterion rubric (OP/CD/SG/PR, 1-5, three calls per video, equal scenario weighting), the method achieves mean 3.69 (no caption) and 3.56 (with caption) vs 3.14 (Wan 2.2 T2V) and 3.09 (Wan 2.2 I2V), with the largest gains in contact/event-direction (CD) and physical response (PR); qualitative examples (ball collision 2.92 to 4.42; vase 2.08 to 3.08) support the aggregate.

Critical strengths
  1. The "world before pixels" idea is a timely, coherent synthesis of agentic scene construction and physics simulation for T2V, addressing a well-documented failure (physical implausibility) of direct T2V with a training-free pipeline.
  2. The refinement loop is well-engineered: deterministic validation before execution, explicit asset interfaces (cavities/outlets), and a VLM controller with the right to request more frames and validate edits before committing - principled answers to simulation-hallucination and edit-regret problems.
Critical weaknesses
  1. The headline comparison is not backbone-matched: the method uses Wan2.1-VACE-14B while baselines use Wan 2.2 official checkpoints - as the paper acknowledges, this confounds method and video-backend improvements; the marginal contribution of the simulation pipeline given the same VACE backbone is therefore not isolated in the main table.

  2. Evaluation rests on a single LLM judge (gpt-5.6-sol) with a rubric; there is no reported human-correlation or judge-variance analysis in the read portion, which matters for a methodology whose headline delta is ~0.5 points.

Questions for authors
  1. What do the ablations show: how much of the 0.55-point mean gain over T2V comes from structured scene authoring vs the refinement loop vs the VACE conditioning itself?
  2. Is the simulation-in-the-loop stage ever harmful (accepted rollout that later fails appearance synthesis), and what is the per-scenario acceptance/refinement cost?
  3. How was gpt-5.6-sol calibrated against human ratings, and how stable are ratings across judge calls (SDs in Table 1 are 0.02-0.22)?
2 comments

We sincerely thank the reviewer for the thoughtful assessment and constructive suggestions. We particularly appreciate the concerns regarding baseline fairness and evaluation reliability. In response, we have added same-checkpoint comparisons and a blinded human study, alongside further analyses of our ablations and refinement effort. Additions are marked in blue in the revised manuscript. Supplementary S3 (Open index.html in Supplementary_Material.zip) provides detailed ratings and evaluation protocols.


1. Same-Checkpoint Final-Video Comparison (W1)

We agree that the original comparison, while using the stronger Wan2.2 direct-generation baselines, does not fully control for differences in video backbones. We therefore added direct T2V and I2V baselines using the same Wan2.1-VACE-14B checkpoint as our method.

Table R2. Same-checkpoint Astra scores on four 1–5 scales. Bold and underlining mark the best and second-best scores.

ConfigurationPrompt alignmentPhysical plausibilityVisual realismTemporal consistency
Direct T2V2.77±0.082.77 \pm 0.083.10±0.163.10 \pm 0.163.92±0.15‾\underline{3.92 \pm 0.15}4.51±0.09\mathbf{4.51 \pm 0.09}
Direct I2V3.18±0.123.18 \pm 0.122.36±0.162.36 \pm 0.163.18±0.273.18 \pm 0.273.15±0.403.15 \pm 0.40
Wan2.2 T2V3.38±0.153.38 \pm 0.152.85±0.002.85 \pm 0.004.13±0.04\mathbf{4.13 \pm 0.04}4.03±0.254.03 \pm 0.25
Wan2.2 I2V3.64±0.193.64 \pm 0.192.92±0.082.92 \pm 0.083.41±0.193.41 \pm 0.194.10±0.164.10 \pm 0.16
Ours w/o caption3.97±0.09‾\underline{3.97 \pm 0.09}3.15±0.15‾\underline{3.15 \pm 0.15}3.56±0.123.56 \pm 0.123.90±0.313.90 \pm 0.31
Ours w/ caption4.23±0.23\mathbf{4.23 \pm 0.23}3.44±0.04\mathbf{3.44 \pm 0.04}3.69±0.133.69 \pm 0.134.23±0.00‾\underline{4.23 \pm 0.00}

Our method still outperforms both direct baselines in prompt alignment and physical plausibility, achieving 4.23/3.44 with captions, compared with 2.77/3.10 for direct T2V and 3.18/2.36 for direct I2V. Direct T2V performs better in visual realism and temporal consistency. These results support the benefits of our simulation-conditioned pipeline under a shared video backbone. The revised Astra table (Table 1) and Appendix C provide the full six-method comparison and generation settings.


2. Human Ratings and Judge Repeatability (W2, Q3)

We thank the reviewer for highlighting the limitations of relying on a single LLM judge. We agree that independent human evaluation is important for validating our conclusions. We therefore re-evaluated the final videos using GPT-6 Astra over three scoring rounds (reporting mean ± sample SD) and conducted a blinded human study, updating the main results table accordingly.

Human evaluation. All 16 participants passed attention checks and rated all 78 videos in randomized order. The human results confirm our main findings, with our methods maintaining their advantages in prompt alignment and physical plausibility. Compared with direct Wan2.1-VACE T2V, our no-caption system scores 4.11 vs. 2.51 for prompt alignment and 3.33 vs. 2.81 for physical plausibility. Both variants outperform all four baselines in human-rated prompt alignment after Holm correction, while the no-caption variant also leads in physical plausibility (Table 2, Appendix F.1).

Table 2. Blinded human evaluation of final videos (16 participants, 78 videos, 1–5 scales). Entries are mean ± sample SD across 16 participant-level means. Bold and underlining indicate the best and second-best scores.

ConfigurationPrompt alignmentPhysical plausibilityVisual realismTemporal consistency
Direct T2V2.51±0.432.51 \pm 0.432.81±0.482.81 \pm 0.483.22±0.463.22 \pm 0.463.91±0.543.91 \pm 0.54
Direct I2V2.78±0.452.78 \pm 0.452.08±0.422.08 \pm 0.422.67±0.542.67 \pm 0.543.38±0.723.38 \pm 0.72
Wan2.2 T2V3.15±0.353.15 \pm 0.352.88±0.272.88 \pm 0.273.46±0.47\mathbf{3.46 \pm 0.47}3.88±0.573.88 \pm 0.57
Wan2.2 I2V3.17±0.483.17 \pm 0.482.56±0.342.56 \pm 0.342.92±0.492.92 \pm 0.493.58±0.673.58 \pm 0.67
Ours, without caption4.11±0.39\mathbf{4.11 \pm 0.39}3.33±0.39\mathbf{3.33 \pm 0.39}3.30±0.353.30 \pm 0.354.11±0.57\mathbf{4.11 \pm 0.57}
Ours, with caption4.05±0.45‾\underline{4.05 \pm 0.45}3.25±0.40‾\underline{3.25 \pm 0.40}3.37±0.36‾\underline{3.37 \pm 0.36}4.08±0.60‾\underline{4.08 \pm 0.60}

Judge reliability. We further assess the agreement between Astra and human judgments, obtaining Spearman correlations of 0.720 (prompt alignment), 0.486 (physical plausibility), 0.515 (visual realism), and 0.324 (temporal consistency) with human ratings across the four dimensions. These indicate stronger correspondence in prompt alignment than temporal consistency. The three Astra scoring rounds yield sample SDs of 0.07–0.15 for the overall mean and 0.24 for Wan2.2 T2V object preservation. These characterize the repeatability of aggregate Astra scores (Table 1, Appendices C.2 and F.1, Supplementary S3).


3. Contributions of Individual Components (Q1)

We appreciate the reviewer's question about the contributions of structured scene authoring, iterative refinement, and video conditioning. Our ablations examine their effects at different stages of the pipeline, rather than assigning an exact fraction of the final-video gain to each component.

  • Structured authoring. Compared with direct Genesis coding, our DSL improves acceptance from 7/13 to 13/13 and last-complete-preview quality from 3.58 to 4.23 (Table 3a, Appendix D.1).
  • Iterative refinement. Acceptance improves from 2/13 to 13/13, while preview quality increases from 4.00 to 4.31 across the nine scenarios with rateable previews at both endpoints (Table 3b, Appendix D.2).
  • Video conditioning. The same-checkpoint comparison measures the complete simulation-conditioned configuration at the final-video stage.

The updated four-method Sol comparison gives +0.49 for our no-caption system (3.78) versus Wan2.2 T2V (3.28, Supplementary S3). The ablations show gains from structured authoring and refinement before VACE synthesis. Together, these experiments support the benefits of structured authoring and refinement before video synthesis, while a precise attribution of the final-video gain would require further matched component-level comparisons.


4. Synthesis Artifacts and Refinement Effort (Q2)

The refinement loop increased rollout acceptance from 2/13 to 13/13 and raised the mean preview score from 4.00 to 4.31 across the nine scenarios with scoreable previews at both endpoints. (Table 3b, Appendix D.2). More physically plausible rollouts provide more faithful motion guidance for synthesis, helping produce better final videos.

Per-scenario refinement cost. Table R3 reports the number of rounds to first acceptance for each scenario. Counting the initial attempt as round 1, the median was 3 rounds, with a range of 1–9. This corresponds to a median of 2 refinement rounds after the initial attempt (Table 11, Appendix D.2). Complete per-scenario timing, token, and billing records are unavailable.

Table R3. Rounds to first acceptance by scenario.

ScenarioRoundsScenarioRounds
Vase fracture1Newton’s cradle5
Dominoes2Ball and mattress5
Hanging lanterns2Ball impacts glass tower5
Sand rabbit2Basketball bank shot6
Water wheel3Champagne tower2
Two-ball collision1Pillows9
Bowling5

Summary

We sincerely thank the reviewer for the constructive feedback. The additional same-checkpoint comparisons and human evaluations further support our main findings, while the ablation and refinement analyses clarify the contributions and limitations of our framework. We hope these results address your concerns and welcome further discussion.

Reviewer_mm7DOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an agentic framework for physics-grounded text-to-video generation. The authors design a DSL for the agent to use to author and iteratively refine executable physical scenes based on visual feedback from simulated rollouts. The accepted simulation then provides geometric guidance for a pretrained video generator. Experiments show improvements over direct text-to-video/image-to-video generation and ablations indicate that the DSL is effective over direct code for agents to operate on for this task.

Critical strengths
  • The agentic framework provides a practical and interpretable approach for bridging physics simulation with ambiguous text for physics-grounded video generation.

  • Ablations support that the defined DSL and iterative refinement loop improve agent execution reliability and task completion.

Critical weaknesses
  • I think it would be better to include some comparison with existing agentic simulation methods such as PhysAgent. The authors exclude this from their scope due to differences in inputs, but would it be possible to compare certain portions of the frameworks, e.g. share the material/geometry from your framework (which PhysAgent estimates from image input) and only compare the dynamics optimization portion? This would motivate the proposed DSL more.

  • Some accepted simulations appear physically questionable (e.g., the refined basketball example in Figure 3 shows that the basketball penetrates through the net).

Questions for authors
  • Some simulated results appear physically unrealistic, such as overly elastic responses for pillows or the ball–hoop/net interaction in Figure 3. While the paper describes numerical validation and VLM-based review, could authors discuss why such issues may occur? Were there ever issues in simulation stability?

  • Could the authors provide side-by-side videos of the underlying simulations and final generated videos? This would help distinguish artifacts originating from simulation from those introduced during video synthesis.

  • Section 3.2 includes motion as part of the object representation. Could the authors clarify how this motion is specified and how much it constrains the simulation?

3 comments

Thank you for suggesting a shared-asset PhysAgent comparison and for pointing out artifacts in the accepted rollouts. We address the following points.

  • Shared-Asset Comparison with PhysAgent (W1)
  • Ball–Net Contact and Pillow Rebound (W2, Q1)
  • Side-by-Side Simulation and Final Videos (Q2)
  • Motion Specification in the Scene DSL (Q3)

W and Q follow the order of weaknesses and questions in the review. Additions are marked in blue in the revised manuscript. Open index.html in Supplementary_Material.zip to view S1–S3.


1. Shared-Asset Comparison with PhysAgent (W1)

Scope. As noted by the reviewer and discussed in our original paper, PhysAgent requires an input image, whereas our framework constructs physical scenes directly from text. Our framework also covers broader phenomena, including fluid motion and fracture, which are not explicitly addressed in PhysAgent. Nevertheless, we appreciate the reviewer's suggestion to compare the simulation and refinement stages under shared conditions. We therefore focus on rigid-body and deformable-object dynamics supported by both.

Comparison. We provide PhysAgent with the event text, corresponding first frame, and shared meshes and material parameters. PhysAgent further estimates object poses from the first frame. We evaluate 21 paired simulated rollouts before appearance synthesis, with our method scoring 4.58 vs. 4.24 overall and outperforming PhysAgent across all four criteria (Table R1).

Table R1. Simulation comparison with PhysAgent. Bold and underlining mark the best and second-best scores.

Simulation subsetPairsMethodOPCDSGPRMean
Action search12Ours4.834.674.924.584.75
Action search12PhysAgent4.58‾\underline{4.58}4.33‾\underline{4.33}4.50‾\underline{4.50}3.92‾\underline{3.92}4.33‾\underline{4.33}
Passive dynamics9Ours4.784.564.114.004.36
Passive dynamics9PhysAgent4.56‾\underline{4.56}4.22‾\underline{4.22}4.00‾\underline{4.00}3.67‾\underline{3.67}4.11‾\underline{4.11}
All cases21Ours4.814.624.574.334.58
All cases21PhysAgent4.57‾\underline{4.57}4.29‾\underline{4.29}4.29‾\underline{4.29}3.81‾\underline{3.81}4.24‾\underline{4.24}

Metrics. Action search (12 pairs) evaluates whether the agent can identify actions that produce the requested event; passive dynamics (9 pairs) evaluates scene evolution under zero action input. Astra scores object preservation (OP), contact and event direction (CD), support and gravity (SG), and physical response (PR), with equal weighting.

Takeaway. The results show that our method achieves better simulated outcomes than PhysAgent (visual comparisons in Supplementary S2). The direct-Genesis comparison in our original submission (Table 3, Appendix D.1) further demonstrates more reliable scene authoring (13/13 vs. 7/13 accepted cases, Table 3), supporting the effectiveness of our simulation and agentic framework.


2. Ball–Net Contact and Pillow Rebound (W2, Q1)

We thank the reviewer for carefully examining the simulations and identifying these artifacts. We acknowledge that accepted rollouts may still contain physical inaccuracies. Rollout acceptance combines targeted numerical checks with VLM review of sampled frames, but does not guarantee complete physical correctness.

Basketball. We confirmed a small intersection between the ball and net in Fig. 3. The thin net strands are not fully resolved by Genesis's collision grid (approximately 10–24 mm spacing), allowing localized penetration that was overlooked during VLM review of sampled frames.

Pillows. Our MPM model represents each pillow as a homogeneous elastic body. Although refinement reduced the Young's modulus from 30 kPa to 5 kPa, the resulting rebound may still appear overly elastic compared with real pillows.

Simulation stability. We did not observe major simulation stability issues in our experiments. When a simulation run fails, the controller passes error diagnostics to the scene-authoring agent, which revises the scene specification and retries.

Limitations. Our framework balances physical fidelity against the cost of iterative simulation and review. Correcting subtle artifacts may require additional refinement rounds, while improvements in simulation fidelity may not always be preserved in the final video synthesis. We will clarify these limitations and trade-offs in the revised manuscript.


3. Side-by-Side Simulation and Final Videos (Q2)

We appreciate the reviewer's suggestion to provide side-by-side visualizations. Supplementary S1 contains all 13 scenarios, showing the underlying simulations alongside the synthesized videos with and without rollout captions. The videos are aligned by frame index, allowing readers to distinguish artifacts originating from simulation from those introduced during video synthesis.


4. Motion Specification in the Scene DSL (Q3)

Thank you for raising this question. In our DSL, motion specifies an object’s initial linear and angular velocities and any scheduled velocity commands. For dynamic bodies, Genesis computes subsequent motion and contact responses between commands. For actuators, the commands prescribe their motion, while Genesis computes the responses of interacting dynamic bodies.


Summary

We sincerely thank the reviewer for the constructive feedback. We hope the shared-asset comparison with PhysAgent, the side-by-side videos, and the clarifications on rollout acceptance and motion specification address your concerns. We welcome further discussion.


References

[1] Q. Li et al. PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generation. arXiv, 2026.

Thank you to the reviewers for the detailed response, I do not have any further questions and will maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.