- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces Gen-RSI, an OPD method which leverages Blender as world proxy to improve physical fidelity. Unlike approaches using additional physics datasets or plausibility rewards, Gen-RSI builds correction targets from the generator's current failures. At first, it generates video with student model, reconstruct the scene with Blender, repair the physical error, and use it for VACE conditioning for teacher prediction. The student generates new denoising trajectories and learns to match the teacher's predictions at the same noisy states. Across rounds, it updates both the student and the tools used to reconstruct and repair scenes. To evaluate the method, they use PhyGenBench and VBench, along with human ratings and ablations of the self-improvement loop.
Critical strengths
-
Connecting OPD with Blender teacher for physical plausibility looks interesting. It seems to help reduce exposure bias from naive fine-tuning video generative models with additional physics datasets.
-
The paper is well written and easy to follow.
Critical weaknesses
-
Unclear practicality and scalability. Each step requires heavy Blender reconstruction, repeated repair and rendering, and verification. These steps are expensive, but the normalized GPU-hour curves do not provide a clear breakdown of the total cost, including failed attempts. It is also unclear how reliably the pipeline handles complex scenes with many objects or interactions that are difficult to represent in Blender. The experiments use VACE teachers, leaving applicability to other stronger video models unclear (LTX, Minimax ...).
-
Uncertain quality of the teacher supervision. Querying the teacher at the student's noisy states does not ensure that the reconstructed reference matches the student's scene. Reconstruction errors may therefore affect the teacher targets, and distillation may transfer the proxy's appearance along with its corrected motion. Additionally, some examples shows Blender-like artifacts, raising concerns about visual realism. Since some project-page examples are conditioned VACE outputs, the final text-only student outputs should be clearly identified and evaluated for both physics and visual quality. Additionally, there are real-sim (Blender) gap in physical plausibility. Although Table 3 shows proposed method is better than SFT, SFT could use diverse synthetic videos refined with V2V, and do not require complicated reconstructing pipeline, so it is unclear whether its gains justify its added complexity.
Questions for authors
- Which displayed examples and benchmark scores come from the final text-only student? It can be confusing.
- See weaknesses.
4 comments
Response to Reviewer EUxz (2/3)
W2. Teacher-target fidelity, appearance transfer, the sim-to-real gap, and comparison with SFT
Uncertain quality of the teacher supervision. Querying the teacher at the student's noisy states does not ensure that the reconstructed reference matches the student's scene. Reconstruction errors may therefore affect the teacher targets, and distillation may transfer the proxy's appearance along with its corrected motion. Additionally, some examples shows Blender-like artifacts, raising concerns about visual realism. Since some project-page examples are conditioned VACE outputs, the final text-only student outputs should be clearly identified and evaluated for both physics and visual quality. Additionally, there are real-sim (Blender) gap in physical plausibility. Although Table 3 shows proposed method is better than SFT, SFT could use diverse synthetic videos refined with V2V, and do not require complicated reconstructing pipeline, so it is unclear whether its gains justify its added complexity.
Querying the teacher at the student's noisy states does not ensure that the reconstructed reference matches the student's scene.
1. Student output and Reconstruction Alignment. Our method uses an agent to write code to reconstruct a Blender proxy from a student-generated video. A separate VLM judge compares the proxy's rendered frames with the source student frames across five dimensions, including visible events, process, causality, continuity, and physical behavior. We make sure the reconstruction score is at latest 8/10 before proceeding to the next repair step. This makes sure the alignment between the reconstructed proxy and the source video, including its physical errors.
Reconstruction errors may therefore affect the teacher targets,
2. Reconstruction Error Propagation. Reconstruction errors may affect teacher targets, but our verification mechanism reduces this risk by filtering out unreliable proxies.
Additionally, we introduce a controlled ablation study examining how proxy quality and rendering appearance affect teacher predictions. Specifically, we evaluate the frozen teacher under three reference conditions. Physics correction (A → B) improves teacher predictions in both PhyGenBench (0.33 → 0.47) and DINO similarity (0.68 → 0.82), while appearance changes (B → C) have only minor effects (0.47 → 0.43 and 0.82 → 0.80, respectively).
| Reference supplied to teacher | PhyGenBench [1] ↑ | DINO similarity vs. source [2] ↑ | LPIPS, C vs. B [3] ↓ |
|---|---|---|---|
| A. Observed proxy before repair | 0.33 | 0.68 | — |
| B. Verified repaired proxy, using A's rendering settings | 0.47 | 0.82 | — (reference) |
| C. Same repaired proxy as B, with alternative rendering appearance | 0.43 | 0.80 | 0.14 |
and distillation may transfer the proxy's appearance along with its corrected motion. Additionally, some examples shows Blender-like artifacts, raising concerns about visual realism.
3. Appearance Leakage and Visual Realism. The final text-only student maintains visual quality comparable to the base model. The student learns the frozen video teacher’s denoising predictions, rather than directly fitting Blender-rendered pixels. The anchor loss also regularizes its predictions toward the initial model (Sec. 3.3). To assess the visual effect of training, we compare the base model and the final student below under the same prompts and sampling settings. Aesthetic quality, imaging quality, and subject consistency remain comparable, with no decrease in these measured dimensions.
| Wan2.1-1.3B | Aesthetic quality | Imaging quality | Subject consistency |
|---|---|---|---|
| Base | 65.2 | 65.1 | 93.1 |
| + Gen-RSI (text-only student) | 66.5 | 66.2 | 94.0 |
Since some project-page examples are conditioned VACE outputs, the final text-only student outputs should be clearly identified
4. Identification of text-only student outputs. All scores and demos come from the text-only student during inference.
Additionally, there are real-sim (Blender) gap in physical plausibility.
5. The physical gap between simulation and the real world. We use Blender only for basic motion and event relations: how objects move, when they make contact, how they support each other, and the initial and final state of an event. The realism of the final video comes from a generator pretrained on real videos. What we correct are constraints that hold in both simulation and reality, such as gravity, non-penetration, object permanence, and causal order. A gap remains at the parameter level, in friction, deformable objects, and fluids. This is why we supervise at the event level rather than at the parameter level.
Response to Reviewer EUxz (3/3)
Although Table 3 shows proposed method is better than SFT, SFT could use diverse synthetic videos refined with V2V, and do not require complicated reconstructing pipeline, so it is unclear whether its gains justify its added complexity.
6. Comparison with V2V-refined SFT. The additional control is SFT-Synthetic: Blender scenes are generated directly from the training prompts, refined with the same VACE teacher, and then used for SFT, without reconstructing student failures.
| Training method (Wan2.1-14B) | PhyGenBench ↑ | VBench-2.0 physics avg. ↑ |
|---|---|---|
| SFT-Synthetic (without reconstruction) | 0.492 | 76.32 |
| Gen-RSI (Table 3) | 0.558 | 80.00 |
We will label every figure and page video by source and add the SFT-Synthetic baseline to Table 3.
[1] Meng, Fanqing, et al. "Towards world simulator: Crafting physical commonsense-based benchmark for video generation." Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:43781-43806, 2025.
[2] Sheynin, Shelly, et al. "Emu edit: Precise image editing via recognition and generation tasks." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.
[3] Zhang, Richard, et al. "The unreasonable effectiveness of deep features as a perceptual metric." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
[4] Jiang, Dengyang, et al. "D-OPSD: On-policy self-distillation for continuously tuning step-distilled diffusion models." arXiv preprint arXiv:2605.05204 (2026).
Q1. Source of displayed examples and benchmark scores
Which displayed examples and benchmark scores come from the final text-only student? It can be confusing.
Thank you for pointing out the ambiguity. The Gen-RSI benchmark results are obtained from the student alone. We clarify the sources of the displayed videos separately below.
Displayed videos: student outputs and proxy renders are distinct. The Gen-RSI demonstrations on the original project page and the Gen-RSI columns in Figures 9 and 10 are text-only student outputs. Figure 8 instead shows Blender proxy renders used to illustrate reconstruction and repair.
Thank you for the detailed response. Some of my concerns have been addressed, but I remain concerned about practicality, heavy and expensive agent calls considering limited generalizability (simple interactions in Blender space).
Additional qualitative examples from the trained prompt-only student would further demonstrate its effectiveness, particularly for in-the-wild prompt generation, such as "playing basketball". Additionally, given the largely engineered VLM-as-judge pipeline, I will increase my score to weak accept.
Response to Reviewer EUxz (1/3)
W1. Total cost, complex scenes, and teachers other than VACE
Thank you for the feedback. We provide a cost breakdown, complex-scene examples, and the teacher interfaces needed to adapt Gen-RSI beyond VACE.
1. Total cost breakdown. The table below reports absolute GPU usage, agent API expenditure and wall-clock time for a 3,000-sample round, including failed attempts. Gen-RSI training includes OPD training, and the agent pipeline for reconstruction, diagnosis and repair. The configuration uses 64 A800 GPUs and an API limit of 250 requests per minute.
2. Reliability on complex scenes. In the complex-scene evaluation conducted during the rebuttal, all 18 scenes reached the VLM judge requirements, 16 of them within the 8-round budget. These included a branching spiral of about 200 dominoes, dozens of bouncing balls descending stairs, and a glass tower filling tier by tier. Representative examples are shown at https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-euxz/#complex.
The two remaining scenes—the pendulum wave and the chain-reaction machine—passed after 22 and 25 iterations, respectively. These tests show that the pipeline can handle large object counts, while long interaction chains and precise synchronization require more refinement. In the training pipeline, candidates that fail the acceptance checks within the allowed budget are excluded from supervision.
3. Teachers other than VACE. Gen-RSI can be adapted to another video backbone by using a frozen, proxy-conditioned version of that backbone as its teacher. The teacher and student only need to have compatible latent representations, noise schedules and prediction parameterizations, so that the teacher can provide correction targets at the student’s denoising states.
Concrete conditioning interfaces also exist for other backbones. LTX-2 [1] provides official IC-LoRA adapters for depth, edge and pose conditioning, allowing Blender-derived control sequences to serve as inputs. Code World Model [2] demonstrates LoRA adaptation of MiniMax-H3 for proxy-conditioned video generation. These provide concrete starting points for constructing compatible teachers. Our supplementary examples also show the same coding agent reconstructing and repairing videos generated by MiniMax-H3 and LTX-2.5 (https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-euxz/#h3 and https://anonymous-submission-21194.github.io/anonymous-submission-21194-rebuttal/reviewer-euxz/#ltx25), demonstrating that the reconstruction and repair stage transfers across generators.
References
[1] HaCohen, Yoav, et al. "LTX-2: Efficient joint audio-visual foundation model." arXiv preprint arXiv:2601.03233 (2026).
[2] Chen, Yiwen, Guosheng Lin, and Chi Zhang. "Code world model: Coding agent as world brain." arXiv preprint arXiv:2608.25927 (2026).