acceptodds
Under review as a conference paper at ICLR 2027

Gen-RSI: Recursive Self-Improvement of Physical Video Generators through World Proxies

Abstract

Despite producing visually realistic videos, video generation models often violate physical laws in the motion and interactions they depict. Existing post-training methods usually rely on offline physics datasets to encode physical priors which may not address the model’s current failures, or learned reward models that assess physical plausibility without providing explicit correction targets. We introduce **Gen-RSI**, a recursive self-improvement framework that constructs physical corrections from the model’s own generations through executable world proxies. Specifically, a coding agent reconstructs the generated scene and motion as a Blender program and uses physics simulation to diagnose and repair physical violations. Physically verified repairs are rendered to condition a frozen video teacher, whose predictions provide on-policy distillation targets for the student generator at its own denoising states. Across rounds, Gen-RSI rebuilds correction targets from the updated generator and refines its agentic diagnosis and repair procedures using feedback from prior attempts. After four-round training on Wan2.1-14B, Gen-RSI improves the PhyGenBench average score from 0.369 to 0.558 and the mean score across the mechanics, thermotics, and material dimensions of VBench-2.0 from 73.09% to 80.00%. View the supplementary material video at: https://anonymous-submission-21194.github.io/

Primary area:
generative models
Keywords:
Video Generation, Recursive Self Improvement

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_EUxzOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Gen-RSI, an OPD method which leverages Blender as world proxy to improve physical fidelity. Unlike approaches using additional physics datasets or plausibility rewards, Gen-RSI builds correction targets from the generator's current failures. At first, it generates video with student model, reconstruct the scene with Blender, repair the physical error, and use it for VACE conditioning for teacher prediction. The student generates new denoising trajectories and learns to match the teacher's predictions at the same noisy states. Across rounds, it updates both the student and the tools used to reconstruct and repair scenes. To evaluate the method, they use PhyGenBench and VBench, along with human ratings and ablations of the self-improvement loop.

Critical strengths
  1. Connecting OPD with Blender teacher for physical plausibility looks interesting. It seems to help reduce exposure bias from naive fine-tuning video generative models with additional physics datasets.

  2. The paper is well written and easy to follow.

Critical weaknesses
  1. Unclear practicality and scalability. Each step requires heavy Blender reconstruction, repeated repair and rendering, and verification. These steps are expensive, but the normalized GPU-hour curves do not provide a clear breakdown of the total cost, including failed attempts. It is also unclear how reliably the pipeline handles complex scenes with many objects or interactions that are difficult to represent in Blender. The experiments use VACE teachers, leaving applicability to other stronger video models unclear (LTX, Minimax ...).

  2. Uncertain quality of the teacher supervision. Querying the teacher at the student's noisy states does not ensure that the reconstructed reference matches the student's scene. Reconstruction errors may therefore affect the teacher targets, and distillation may transfer the proxy's appearance along with its corrected motion. Additionally, some examples shows Blender-like artifacts, raising concerns about visual realism. Since some project-page examples are conditioned VACE outputs, the final text-only student outputs should be clearly identified and evaluated for both physics and visual quality. Additionally, there are real-sim (Blender) gap in physical plausibility. Although Table 3 shows proposed method is better than SFT, SFT could use diverse synthetic videos refined with V2V, and do not require complicated reconstructing pipeline, so it is unclear whether its gains justify its added complexity.

Questions for authors
  1. Which displayed examples and benchmark scores come from the final text-only student? It can be confusing.
  2. See weaknesses.
4 comments
AuthorsOpenReview ↗

Response to Reviewer EUxz (1/3)

W1. Total cost, complex scenes, and teachers other than VACE

Unclear practicality and scalability. Each step requires heavy Blender reconstruction, repeated repair and rendering, and verification. These steps are expensive, but the normalized GPU-hour curves do not provide a clear breakdown of the total cost, including failed attempts. It is also unclear how reliably the pipeline handles complex scenes with many objects or interactions that are difficult to represent in Blender. The experiments use VACE teachers, leaving applicability to other stronger video models unclear (LTX, Minimax ...).

Thank you for the feedback. We provide a cost breakdown, complex-scene examples, and the teacher interfaces needed to adapt Gen-RSI beyond VACE.

1. Total cost breakdown. The table below reports absolute GPU usage, agent API expenditure and wall-clock time for a 3,000-sample round, including failed attempts. Gen-RSI training includes OPD training, and the agent pipeline for reconstruction, diagnosis and repair. The configuration uses 64 A800 GPUs and an API limit of 250 requests per minute.

Total (3,000 samples)1.3B student14B student
OPD training (incl. teacher queries)32.9 h (2,107 GPU-h)72.4 h (4,631 GPU-h)
Agent reconstruction + repair + Blender rendering (API, overlapped with training)6.0 h, $14,1906.0 h, $14,190
Total wall-clock time (pipelined / sequential)34.3 h / 40.3 h80.0 h / 86.0 h

2. Reliability on complex scenes. In the complex-scene evaluation conducted during the rebuttal, all 18 scenes reached the VLM judge requirements, 16 of them within the 8-round budget. These included a branching spiral of about 200 dominoes, dozens of bouncing balls descending stairs, and a glass tower filling tier by tier. Representative examples are shown at https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-euxz/#complex.

The two remaining scenes—the pendulum wave and the chain-reaction machine—passed after 22 and 25 iterations, respectively. These tests show that the pipeline can handle large object counts, while long interaction chains and precise synchronization require more refinement. In the training pipeline, candidates that fail the acceptance checks within the allowed budget are excluded from supervision.

3. Teachers other than VACE. Gen-RSI can be adapted to another video backbone by using a frozen, proxy-conditioned version of that backbone as its teacher. The teacher and student only need to have compatible latent representations, noise schedules and prediction parameterizations, so that the teacher can provide correction targets at the student’s denoising states.

Concrete conditioning interfaces also exist for other backbones. LTX-2 [1] provides official IC-LoRA adapters for depth, edge and pose conditioning, allowing Blender-derived control sequences to serve as inputs. Code World Model [2] demonstrates LoRA adaptation of MiniMax-H3 for proxy-conditioned video generation. These provide concrete starting points for constructing compatible teachers. Our supplementary examples also show the same coding agent reconstructing and repairing videos generated by MiniMax-H3 and LTX-2.5 (https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-euxz/#h3 and https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-euxz/#ltx25), demonstrating that the reconstruction and repair stage transfers across generators.

References

[1] HaCohen, Yoav, et al. "LTX-2: Efficient joint audio-visual foundation model." arXiv preprint arXiv:2601.03233 (2026).

[2] Chen, Yiwen, Guosheng Lin, and Chi Zhang. "Code world model: Coding agent as world brain." arXiv preprint arXiv:2608.25927 (2026).

AuthorsOpenReview ↗

Response to Reviewer EUxz (2/3)

W2. Teacher-target fidelity, appearance transfer, the sim-to-real gap, and comparison with SFT

Uncertain quality of the teacher supervision. Querying the teacher at the student's noisy states does not ensure that the reconstructed reference matches the student's scene. Reconstruction errors may therefore affect the teacher targets, and distillation may transfer the proxy's appearance along with its corrected motion. Additionally, some examples shows Blender-like artifacts, raising concerns about visual realism. Since some project-page examples are conditioned VACE outputs, the final text-only student outputs should be clearly identified and evaluated for both physics and visual quality. Additionally, there are real-sim (Blender) gap in physical plausibility. Although Table 3 shows proposed method is better than SFT, SFT could use diverse synthetic videos refined with V2V, and do not require complicated reconstructing pipeline, so it is unclear whether its gains justify its added complexity.

Querying the teacher at the student's noisy states does not ensure that the reconstructed reference matches the student's scene.

1. Student output and Reconstruction Alignment. Our method uses an agent to write code to reconstruct a Blender proxy from a student-generated video. A separate VLM judge compares the proxy's rendered frames with the source student frames across five dimensions, including visible events, process, causality, continuity, and physical behavior. We make sure the reconstruction score is at latest 8/10 before proceeding to the next repair step. This makes sure the alignment between the reconstructed proxy and the source video, including its physical errors.

Reconstruction errors may therefore affect the teacher targets,

2. Reconstruction Error Propagation. Reconstruction errors may affect teacher targets, but our verification mechanism reduces this risk by filtering out unreliable proxies.

Additionally, we introduce a controlled ablation study examining how proxy quality and rendering appearance affect teacher predictions. Specifically, we evaluate the frozen teacher under three reference conditions. Physics correction (A → B) improves teacher predictions in both PhyGenBench (0.33 → 0.47) and DINO similarity (0.68 → 0.82), while appearance changes (B → C) have only minor effects (0.47 → 0.43 and 0.82 → 0.80, respectively).

Reference supplied to teacherPhyGenBench [1] ↑DINO similarity vs. source [2] ↑LPIPS, C vs. B [3] ↓
A. Observed proxy before repair0.330.68—
B. Verified repaired proxy, using A's rendering settings0.470.82— (reference)
C. Same repaired proxy as B, with alternative rendering appearance0.430.800.14

and distillation may transfer the proxy's appearance along with its corrected motion. Additionally, some examples shows Blender-like artifacts, raising concerns about visual realism.

3. Appearance Leakage and Visual Realism. The final text-only student maintains visual quality comparable to the base model. The student learns the frozen video teacher’s denoising predictions, rather than directly fitting Blender-rendered pixels. The anchor loss also regularizes its predictions toward the initial model (Sec. 3.3). To assess the visual effect of training, we compare the base model and the final student below under the same prompts and sampling settings. Aesthetic quality, imaging quality, and subject consistency remain comparable, with no decrease in these measured dimensions.

Wan2.1-1.3BAesthetic qualityImaging qualitySubject consistency
Base65.265.193.1
+ Gen-RSI (text-only student)66.566.294.0

Since some project-page examples are conditioned VACE outputs, the final text-only student outputs should be clearly identified

4. Identification of text-only student outputs. All scores and demos come from the text-only student during inference.

Additionally, there are real-sim (Blender) gap in physical plausibility.

5. The physical gap between simulation and the real world. We use Blender only for basic motion and event relations: how objects move, when they make contact, how they support each other, and the initial and final state of an event. The realism of the final video comes from a generator pretrained on real videos. What we correct are constraints that hold in both simulation and reality, such as gravity, non-penetration, object permanence, and causal order. A gap remains at the parameter level, in friction, deformable objects, and fluids. This is why we supervise at the event level rather than at the parameter level.

AuthorsOpenReview ↗

Response to Reviewer EUxz (3/3)

Although Table 3 shows proposed method is better than SFT, SFT could use diverse synthetic videos refined with V2V, and do not require complicated reconstructing pipeline, so it is unclear whether its gains justify its added complexity.

6. Comparison with V2V-refined SFT. The additional control is SFT-Synthetic: Blender scenes are generated directly from the training prompts, refined with the same VACE teacher, and then used for SFT, without reconstructing student failures.

Training method (Wan2.1-14B)PhyGenBench ↑VBench-2.0 physics avg. ↑
SFT-Synthetic (without reconstruction)0.49276.32
Gen-RSI (Table 3)0.55880.00

We will label every figure and page video by source and add the SFT-Synthetic baseline to Table 3.

[1] Meng, Fanqing, et al. "Towards world simulator: Crafting physical commonsense-based benchmark for video generation." Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:43781-43806, 2025.

[2] Sheynin, Shelly, et al. "Emu edit: Precise image editing via recognition and generation tasks." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

[3] Zhang, Richard, et al. "The unreasonable effectiveness of deep features as a perceptual metric." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.

[4] Jiang, Dengyang, et al. "D-OPSD: On-policy self-distillation for continuously tuning step-distilled diffusion models." arXiv preprint arXiv:2605.05204 (2026).

Q1. Source of displayed examples and benchmark scores

Which displayed examples and benchmark scores come from the final text-only student? It can be confusing.

Thank you for pointing out the ambiguity. The Gen-RSI benchmark results are obtained from the student alone. We clarify the sources of the displayed videos separately below.

Displayed videos: student outputs and proxy renders are distinct. The Gen-RSI demonstrations on the original project page and the Gen-RSI columns in Figures 9 and 10 are text-only student outputs. Figure 8 instead shows Blender proxy renders used to illustrate reconstruction and repair.

Thank you for the detailed response. Some of my concerns have been addressed, but I remain concerned about practicality, heavy and expensive agent calls considering limited generalizability (simple interactions in Blender space).

Additional qualitative examples from the trained prompt-only student would further demonstrate its effectiveness, particularly for in-the-wild prompt generation, such as "playing basketball". Additionally, given the largely engineered VLM-as-judge pipeline, I will increase my score to weak accept.

Reviewer_y6C7OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose a recursive self-improvement framework to improve physical plausibility in video generators. They use coding agents to reconstruct generated videos via Blender and base on simulation to repair physical violations. Then, the fixed simulations are used as privileged conditioning information for the teacher generator and optimize the student video denoiser using on-policy distillation. The results on PhyGenBench and some dimensions of VBench 2.0 show the effectiveness of Gen-RSI.

Critical strengths

In general, the idea of using coding agents to simulate physical laws has been proven to be effective, and this paper designs a framework to incorporate it in a self-improving loop. In my opinion, the framework is novel and can be useful for the community.

The results on both small and large-scale t2v models show the effectiveness of the proposed method over other post-training and distillation techniques.

Critical weaknesses

I think the most important component in this framework is from the generated scene to the code and to the simulation. The authors should discuss more in this part, for example, how correctly the code can simulate the scene, and whether it gets bugs in llms coding agent, the cost of each sample, and whether it can simulate all physical laws?

I would love to see more qualitative results in the simulation, for example, the simulated flame in Figure 5, besides the generated videos from the denoiser.

From my personal experience, the coding agent can not always generate the simulation in a single shot, and sometimes the simulation is not at least similar to the scene. Do you filter them automatically or use a human in the loop?

Questions for authors

Initially i put weak rejection as the simulated videos are not given. I would love to see if it works from easy to hard physical cases. I would definitely raise my score if the authors could include them in the rebuttal.

5 comments
AuthorsOpenReview ↗

Response to Reviewer y6C7 (1/3)

W1. Simulation accuracy, agent bugs, per-sample cost, and physics coverage

I think the most important component in this framework is from the generated scene to the code and to the simulation. The authors should discuss more in this part, for example, how correctly the code can simulate the scene, and whether it gets bugs in llms coding agent, the cost of each sample, and whether it can simulate all physical laws?

Thank you for the detailed questions. We report the success rate of the reconstruction pipeline, program failures, per-sample cost, and physics coverage.

Figure R4. Of 3,000 candidate samples, 2,985 (99.5%) have programs that execute and render the full sequence successfully, and 2,430 (81.0%) pass quality verification and enter training. Both percentages use all 3,000 candidates as the denominator. Rejected samples are excluded from training.

Figure R4: https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/figures/figure-r4.png

1. How accurately does the simulation reproduce the source video? Iterative revision raises the source-matching score from 7.88 to 8.71 out of 10. The pass rate increases from 51.5% to 71.2%. We evaluate whether the rendered simulation reproduces the source video's events, rather than requiring an exact copy of its 3D geometry or rendering style.

Reconstruction stageVideos evaluatedMean source-matching score ↑ (out of 10)Pass rate ↑
Initial reconstruction667.8851.5%
After iterative revision668.7171.2%

The table compares the same 66 videos before and after iterative revision. Source matching is evaluated on five dimensions: visibility, process, causality, continuity, and physical plausibility, each scored out of 2. A reconstruction passes if its total source-matching score is at least 8 out of 10. After revision, the respective mean scores are 1.85, 1.67, 1.73, 1.79, and 1.68. These pass rates are separate from execution success and final training acceptance in Figure R4.

Quality verification has two steps: the reconstruction must pass a quality check against the source frames, and the repaired simulation must be re-executed and pass an independent audit with no regression (Eq. 2, Appendix B).

2. Bugs in the agent's code. There are two kinds, and each is caught by a separate check.

Crashes and API errors. When an API call fails, we retry the request. When code execution or rendering fails, the agent uses the error message to revise the program and runs it again. With retries, 99.5% of the programs run successfully, and 0.5% fail.

Executable Programs with Incorrect Scene or Physical Dynamics. These are caught by quality verification. The agent identifies which physical rule is violated and when the violation occurs. It updates the Blender program only for that time interval and reruns the simulation. We then check whether the violation has been resolved in the simulated states, while an independent VLM checks the physical consistency of the rendered video.

3. Cost per sample. We measured 100 samples of varying difficulty: on average 107 minutes, about 810K tokens, and $4.73 per sample (Table R2, in 8FuM W3). This cost is offline and paid once per round. Under the reported pipelined configuration, reconstruction and repair run asynchronously with student training (see 8FuM W3).

4. Coverage of physical laws. Blender does not simulate all physical laws. Rigid-body mechanics, including gravity, collision, friction and support, are computed by Blender's solver. Reflection and refraction come from the renderer. Our current Blender-based pipeline cannot directly solve the underlying thermal and chemical equations for processes such as combustion and acid-base reactions. The agent therefore uses explicit programs to describe how these events start, evolve, and end. Turbulence, reaction kinetics, and similar phenomena are out of scope, and finite checks certify only the tests they declare. The laws Blender does cover compose into a wide range of complex scenes, for example, a spiral of about 200 dominoes (examples: https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-8fum/#complex).

We will add these measurements and Figure R4 to Sec. 3.2 and Appendix B.

AuthorsOpenReview ↗

Response to Reviewer y6C7 (2/3)

W2. Qualitative simulation results

I would love to see more qualitative results in the simulation, for example, the simulated flame in Figure 5, besides the generated videos from the denoiser.

Thank you for the suggestion. We provide side-by-side videos to show the generated scenes, their Blender reconstructions and the repair process.

https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-y6c7/

1. The flame in Figure 5. We add more examples, including simulated flame in Figure 5, besides the generated videos from the denoiser.

2. What the reconstruction shows. The blender carries the timing of the burn travelling from the bristle tips to the ferrule and the charring process, while the shape of the flame comes from the generator (see 8FuM W1, point 1). Combustion is one of the harder types to reconstruct (see 8FuM Q3).

We will add these proxy renders next to the generated videos in Appendix D.

W3. Automatic filtering and human involvement

From my personal experience, the coding agent can not always generate the simulation in a single shot, and sometimes the simulation is not at least similar to the scene. Do you filter them automatically or use a human in the loop?

Thank you for raising this concern. We clarify the automatic filtering procedure and the stages at which humans participate.

1. Outcome of the automatic filter. Our training dataset contains 3,000 training prompts. 99.5% of 3000 samples can execute and render the full sequence successfully and 81% of the whole datatset pass quality verification and are used for training.

2. Detail of reconstruction. The agent first improves the reconstruction by repeatedly writing code, rendering and comparing against the source frames, guided by motion descriptors and VLM feedback (Sec. 3.2). The decision is made by an auditor separate from the reconstruction agent, GPT-5.6-sol in the main experiments. It scores the render against the source video on five dimensions: visibility, process, causality, continuity, and physical plausibility.

3. Acceptance criteria. Filtering is automatic and has two stages. Before physical diagnosis, the reconstruction must score at least 8/10 for agreement with the source video. After repair, the program must execute and render successfully, pass the media and quality checks, and leave no violation under the applicable executable checks. The independent VLM’s total score must meet the acceptance threshold, set one grade below the maximum, and no dimension may score lower than in the immediately preceding program version. Candidates that still fail after the allowed revisions are discarded (Sec. 3.2; Eq. 2; Appendix B).

4. Human effort. During pipeline development, humans assisted the process and reviewed the overall correctness of candidate samples. Samples accepted through this review were included in training. During subsequent RSI training rounds, candidates are filtered automatically using the acceptance checks described above.

We will state the auditor, the rating dimensions and the acceptance threshold in Appendix B.

AuthorsOpenReview ↗

Response to Reviewer y6C7 (3/3)

Q1. Simulated videos from easy to hard cases

Initially i put weak rejection as the simulated videos are not given. I would love to see if it works from easy to hard physical cases. I would definitely raise my score if the authors could include them in the rebuttal.

Thank you for the suggestion. We provide additional videos demonstrating reconstruction and physical repair from easy to challenging cases. “Generated” denotes the original video, “Rebuild” its Blender reconstruction, and “Repaired” the attempted physical correction.

1. Reconstruction: Easy to Hard. We show four categories, ranging from simple multi-body drops and refraction to challenging combustion and natural convection, under an eight-iteration budget.

https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-y6c7/#build

For complex scenes, all 18 cases were successfully reconstructed, with 16 completed within eight iterations. The remaining two required 22 and 25 iterations. We provide eight representative examples, including a scene with approximately 200 dominoes.

https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-y6c7/#complex

2. Physical Repair. We evaluate whether reconstructed scenes can be physically corrected. Newton's cradle (1 iteration) and block splash and float (8 iterations) pass verification, while iodine–starch colour change (6 iterations) and flour balloon burst (12 iterations) remain unsuccessful. Repairs are verified through re-execution, executable checks, and visual inspection.

https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-y6c7/#repair

3. Failure Cases. We also provide eight unsuccessful reconstruction examples, including dense smoke, dust, and moving hair, to illustrate current limitations.

https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-y6c7/#fail

We will add these examples and their reconstruction and repair outcomes to Appendix D.

Reviewer_8FuMOpenReview ↗
Rating
4(+1)
Confidence
3(+1)
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors argue that existing approaches rely on offline datasets to encode physical priors or learned reward models to assess physical consistency in generated videos. However, these approaches may struggle to identify the current generator's physical errors or provide explicit guidance on how to correct them.

To address this problem, the authors propose Gen-RSI, a recursive self-improvement framework that uses a coding agent for 3D scene reconstruction and Blender-based physics simulation to diagnose and correct physical violations. The resulting corrections are then used for on-policy distillation of the student video generator.

Critical strengths
  1. The idea is very interesting. Combining coding agents and recursive self-improvement aligns with current research trends in improving video generation and physical consistency, with potential applications in robotics and Physical AI.

  2. I really like this idea. If the authors can adequately address my concerns, I would be willing to increase my rating.

Critical weaknesses
  1. If the authors already use Blender and a coding agent to reconstruct the entire scene, why is it necessary to fine-tune a video generator? Why not directly use the Blender rendering results for video generation?

  2. How can the authors ensure that the physics simulated by the engine is correct? Furthermore, how can they ensure that the coding agent provides an accurate 3D/4D reconstruction that faithfully matches the original video scenes?

  3. Given that the method relies on on-policy distillation, how efficiently can the coding agent evaluate the physical correctness of generated videos?

  4. I am concerned about the generalizability of the approach. Can the coding agent successfully reconstruct arbitrary scenarios? The paper mainly demonstrates relatively simple reconstruction scenarios. However, real-world video generation often involves much more complex scenes, and physical consistency is not always the primary objective of every prompt.

Questions for authors
  1. Could the authors include comparisons with more recent state-of-the-art physics-aware video generation baselines?

  2. Could the authors evaluate the method on additional physics benchmarks, such as PhyGround: Benchmarking Physical Reasoning in Generative World Models?

  3. Can Blender successfully simulate all types of complex physical phenomena? I am skeptical that it can reliably handle every complex physical scenario.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 8FuM (1/4)

W1. Role of the fine-tuned generator versus direct Blender rendering

If the authors already use Blender and a coding agent to reconstruct the entire scene, why is it necessary to fine-tune a video generator? Why not directly use the Blender rendering results for video generation?

Thank you for the feedback. Direct Blender rendering produces a video for a particular scene; fine-tuning makes the physical corrections useful beyond that scene. It provides following benefits:

  1. Retain the pretrained generator’s visual capabilities. The Blender proxies use simplified scenes to represent physical events, for example, rigid-body physics. However, it struggles to simulate non-rigid physical sense, such as fluid dynamics . Fine-tuning lets the video model learn from the corrected dynamics while retaining its ability to generate varied material and detailed appearances, rather than using the proxy renders as final outputs (Secs. 3.2–3.3).

  2. More Efficient in Inference. At inference time, our fine-tuned video generator is more efficient than the coding-agent pipeline VideoCoCo [1]. We reproduced VideoCoCo on the same Wan2.1-14B backbone. Gen-RSI achieves higher physical-consistency scores and runs 2.8× as fast, as shown in the table below.

MethodInference componentsPhyGenBenchVBench-2.0 physics avg.Time per video (A800)
VideoCoCo [1]agent + Blender + video editor0.46774.131637 s (2.8x)
Gen-RSIgenerator only0.55880.00585 s (1x)

[1] Li, Haodong, et al. "VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System." arXiv preprint arXiv:2607.27380 (2026).

W2. Physical validity and 3D/4D reconstruction fidelity

How can the authors ensure that the physics simulated by the engine is correct? Furthermore, how can they ensure that the coding agent provides an accurate 3D/4D reconstruction that faithfully matches the original video scenes?

Thank you for raising these two questions. We check the physical validity of the simulated result and the reconstruction's fidelity to the source video separately. A candidate is used for training only after passing both checks.

1. How do we ensure the simulated physics is correct? We verify each repaired simulation with executable checks and an independent visual audit. The agent uses diagnostic feedback to repair detected violations. We check the simulated state trajectories for penetration, support, and object persistence. An independent VLM evaluates the rendered events across five dimensions: visibility, process, causality, continuity, and physical plausibility. A repaired candidate enters training only if it passes the executable checks and meets the audit threshold. Otherwise, it is revised within the allowed budget or discarded.

2. How do we check that the reconstruction matches the source video? We accept a reconstruction only after its rendering scores at least 8/10 for agreement with the source video, before any physical diagnosis or repair. The coding agent repeatedly renders and revises the Blender program using feedback from a separate VLM evaluator. The evaluator compares visible events, process, causality, continuity, and physical behavior, scoring each from 0 to 2. This measures agreement with the observed behavior: if a ball passes through a table in the source video, a reconstruction in which it bounces is not faithful. Reconstructions that remain below the threshold within the revision budget are discarded. We verify event-level correspondence, without claiming exact recovery of scene geometry or physical parameters (Sec. 3.2).

Source videos and their reconstructions are shown side by side at https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-8fum/#complex.

Candidates that fail these checks are excluded from training. In a round of 3,000 samples, 99.5% of programs execute and render successfully, while 81% pass quality verification and enter training.

AuthorsOpenReview ↗

Response to Reviewer 8FuM (2/4)

W3. Agent efficiency under on-policy distillation

Given that the method relies on on-policy distillation, how efficiently can the coding agent evaluate the physical correctness of generated videos?

Thank you for raising the efficiency question. The agent evaluates candidate videos during each RSI round; OPD updates reuse the accepted references. The measurements below cover the full agent pipeline, including reconstruction, diagnosis, and repair. They are not standalone timings of correctness evaluation.

1. Agent cost per sample. We measured 100 samples of varying difficulty. The table reports elapsed time, token usage, and API cost for each candidate's complete agent pipeline.

Table R2. Time and cost per sampleLatencyInput tokensOutput tokensAgent API cost (gpt-6.0-sol pricing)
Blender agent: reconstruct one video38 min122K121K$1.45
Blender agent: repair69 min299K268K$3.28
Blender agent total107 min421K389K$4.73

The complete agent pipeline averages 107 minutes and $4.73 per candidate, corresponding to $14,190 for 3,000 candidates, including failed attempts.

2. Candidates can be evaluated in parallel. Each candidate has its own video, program and checks, so the agent can process different candidates concurrently. The agent's throughput depends on the API request rate and the resources available for Blender execution and rendering.

  1. The Agentic pipeline is asynchronous with the training pipeline as shown below.

Figure R3. Per-sample asynchronous schedule for three samples (S1-S3), using a 14B student on one A800 (585 s per inference and 1,303 s per OPD step). Agent costs follow Table R2. Processing samples sequentially leaves the GPU idle during agent and Blender work; interleaving samples allows that work to overlap with generation and training on other samples.

Figure R3: https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/figures/figure-r3.png

W4. Generalization to complex scenes and diverse prompts

I am concerned about the generalizability of the approach. Can the coding agent successfully reconstruct arbitrary scenarios*? The paper mainly demonstrates relatively simple reconstruction scenarios. However,* real-world video generation often involves much more complex scenes, and physical consistency is not always the primary objective of every prompt.

Thank you for the feedback. Our coding agent can reconstruct rigid-body interactions and other supported physical events, but it does not yet generalize to arbitrary physical scenarios. Its current scope is scenes whose objects and physical events can be represented and checked in Blender. We elaborate reconstruction coverage, complex scenes, and non-physical generation goals separately below.

1. Which scenarios can the coding agent reconstruct? The agent targets support, contact, object permanence, causal order, and the progression of optical, thermal, and material processes. It reconstructs these events using simplified scenes rather than recovering every geometric and visual detail.

2. Can it reconstruct complex scenes within this range? Yes. A scene can combine several physical processes, either simultaneously or in sequence, and include many interacting objects. For example, a branching domino spiral contains about 200 objects whose collisions propagate through the scene, while a chain-reaction machine connects successive interactions. Examples are shown at https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-8fum/#complex.

3. What if physical consistency is not the main objective? Our focus is video generation where physically coherent motion and interactions matter, as in simulation for robotics and world-model research. General prompts that intentionally request fantasy or non-physical visual effects have a different objective and fall outside physical consistency.

AuthorsOpenReview ↗

Response to Reviewer 8FuM (3/4)

Q1. Recent physics-aware baselines

Could the authors include comparisons with more recent state-of-the-art physics-aware video generation baselines?

Thank you for suggesting these comparisons. We have added WISA comparisons on both Wan2.1-1.3B and Wan2.1-14B.

We evaluate WISA under the same protocol on the full PhyGenBench test set and all VBench-2.0 physics prompts.

Method1.3B: PhyGenBench ↑1.3B: VBench-2.0 physics avg. ↑14B: PhyGenBench ↑14B: VBench-2.0 physics avg. ↑
Base0.34270.560.36973.09
+ PhysVid0.33870.690.36367.07
+ PhyGDPO0.38073.090.41473.00
+ PhyT2V0.44066.840.47775.60
+ WISA [1]0.42872.800.46275.20
+ Gen-RSI0.51478.010.55880.00

Gen-RSI outperforms WISA at 0.086 and 0.096 for 1.3B and 14B, respectively, with corresponding VBench-2.0 physics margins of 5.21 and 4.80 percentage points.

We will include these comparisons in the revised Tables 1 and 2.

[1] Wang, Jing, et al. "WISA: World simulator assistant for physics-aware text-to-video generation." Advances in Neural Information Processing Systems 38 (2025): 5388-5416.

Q2. Additional benchmark: PhyGround

Could the authors evaluate the method on additional physics benchmarks, such as PhyGround: Benchmarking Physical Reasoning in Generative World Models?

Thank you for suggesting a more comprehensive evaluation. We provide the PhyGround results and explain the evaluation protocol for two model scales.

Protocol. We use PhyGround’s text-plus-first-frame setting. For two Wan2.1 scales, the benchmark-provided first frame is passed through VACE’s first-frame conditioning interface, identically for the base model and Gen-RSI. VACE supplies only first-frame conditioning; no Blender proxy video or teacher-generated output is used. The released PhyJudge-9B scores all 250 benchmark prompts. Overall is the equal-weight mean of the General and Domain views.

ModelGeneral SAGeneral PTVGeneral Persist.Domain Solid-BodyDomain FluidDomain OpticalOverall
Veo-3.1 (closed)3.033.103.112.903.333.703.05
LTX-2-19B2.502.622.792.493.013.092.62
Wan2.1-1.3B2.442.642.762.512.973.252.62
Wan2.1-1.3B + Gen-RSI2.542.762.902.843.083.222.82
Wan2.1-14B2.682.922.922.813.003.242.85
Wan2.1-14B + Gen-RSI2.783.003.122.893.083.272.95

Gen-RSI raises Overall from 2.62 to 2.82 on Wan2.1-1.3B and from 2.85 to 2.95 on Wan2.1-14B. The largest gains are on object persistence (+0.14 and +0.20) and solid-body physics (+0.33 and +0.08), showing the effectiveness of our method.

We add these results to Sec. 4.2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.