acceptodds
Withdrawn from ICLR 2027

Agentic Physics-Grounded Video Generation via Memory-Augmented In-Context Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Physical plausibility is essential for realistic video generation. Existing methods impose physical guidance before synthesis via simulations or refined prompts, but this can limit dynamic diversity, miss errors that emerge only after generation, and require repeated regeneration when inconsistencies remain. We therefore study a generate–diagnose–repair paradigm that preserves the original generation capability and selectively repairs physical inconsistencies afterward, raising two key questions: what to repaired, and how to repair? To address them, we introduce PhysRepair, an agentic video generation framework that combines causal reasoning over the observed physical failure with prior repair experience to enable localized and adaptive correction. Specifically, Spatiotemporal Causal Constraints identify the first invalid physical state and its causally affected states, producing a repair specification that guides physically plausible local regeneration while preserving unaffected content. Experience-augmented In-context Learning combines the ongoing repair context with relevant experience from previous repairs, allowing the agent to adapt its repair decisions across diverse physical failures and accumulate completed trajectories for future cases. Experiments on Physics-IQ Verified across solid mechanics, fluid dynamics, optics, thermodynamics, and magnetism demonstrate that PhysRepair consistently improves the physical plausibility of generated videos over existing methods.

Primary area:
generative models
Keywords:
Video Generation, Physical Consistency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_RFCKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PhysRepair, a method for repairing physical errors in generated videos following a new paradigm of repairing videos after they have been generated. The contributions lie in two modules: 1) spatiotemporal causal constraints – identify the first frame where an error occured and connecting it with errors in the following frames to fix them locally and consistently; 2) in-context learning augmented by experience – use previous successful fixes to guide the current fix.

PhysRepair can be thought of as a component adapted to a pretrained video generator to improve its physical plausibility. Experiments on the Physics-IQ Verified benchmark show that PhysRepair improves the video generator's physical plausibility metrics. However, there are concerns about the advantages of PhysRepair compared to existing physical plausibility methods for video generation, as well as whether the extra computation imposed by PhysRepair is justified.

Critical strengths
  1. PhysRepair can fix physical errors in generated videos after they have been generated (which can be thought of as post-processing). This is somewhat different from previous works that attempt to constrain physical plausibility before or during generation. This has two pros: it can fix all errors in the video if detected; it can fix errors locally without having to regenerate the whole video multiple times.

  2. The proposed method is intuitive and reasonable. Identifying spatiotemporal causal errors and fixing them together makes sense, and leveraging past experience to guide the current fix helps reduce solution exploration.

  3. The results seem pretty good, especially when compared to the baseline generator PhysRepair is attached to.

Critical weaknesses
  1. The comparison with physics-aware methods is not clear: which paradigms (as illustrated in Fig. 1) do the two physics-aware methods in Table 1 belong to? PhyCo in Table 1 does not seem to be described in Related Work (or any other section of the paper). A more detailed comparison with them is needed to showcase the superiority of PhysRepair, especially a comparison of computational costs, since the paper claims PhysRepair can help reduce unnecessary video regeneration compared to existing physics-aware paradigms.

  2. PhysRepair's generalization ability seems limited. It seems that PhysRepair has to be trained using the output of a specific generator and can only fix that generator's errors. The trained PhysRepair does not seem to be able to fix physical errors in a randomly generated video. Also, the extra training/computational cost of PhysRepair should be discussed to justify the performance gain.

  3. The qualitative results are not convincing. Fig. 3 is very confusing, as all the baseline frames have red boxes, which prevent me from focusing on the actual errors. In Fig. 4, example 2, why does the new error (no foot–ball contact) appear compared to the MiniMax H3 baseline (column 2)? Isn't the model supposed to fix only the errors starting from the first frame where they are detected? Why does the model turn a frame without an error into a new frame with a physical error, which seems to contradict the description of the proposed method?

Questions for authors
  1. I would appreciate if the authors can respond to the weaknesses I have identified above.

  2. It would be good to differentiate the proposed method from simulator based methods such as "Physical Simulator In-the-Loop Video Generation" CVPR'26.

8 comments
AuthorsOpenReview ↗

Response to Reviewer RFCK (1/3)

We thank the reviewer for the constructive questions about the positioning of physics-aware methods, cross-generator generalization, computational overhead, and the qualitative examples. We combine the comparison question with the related simulator-based-method question.

W1: The relationship between the physics-aware baselines in Table 1 and the paradigms in Figure 1 is unclear; PhyCo and computational cost deserve a clearer comparison.

Q2: Differentiate PhysRepair from simulator-based methods, such as Physical Simulator In-the-Loop Video Generation (PSIVG, CVPR 2026).

R1: We clarify that VLIPP and PhyCo, as reported in the original Table 1, are not direct implementations of the two iterative paradigms shown in Figure 1. VLIPP predicts coarse physical motion trajectories using visual-language planning and injects those priors before generation, making it closer to pre-generation planning. PhyCo uses physical supervision and VLM feedback to learn controllable physics-aware priors, with feedback used for training rather than an inference-time loop that diagnoses and repairs particular errors. PSIVG reconstructs a scene from a template video, uses simulation to guide motion, and optimizes texture consistency. PhysRepair instead begins with a specific physical failure in an already-generated video, identifies the first invalid state and its causally affected descendants, and constructs a repair specification containing both editable and preserved object–space–time states. This distinguishes post-generation, evidence-grounded causal repair from pre-generation simulator-based guidance and from global prompt/simulator refinement.

To examine those strategy differences under aligned experimental settings, we additionally compared Resampling (new noise and repeated regeneration), Prompt Refinement (revised prompts and global regeneration; representative of Fig. 1(a)), and Simulation Refinement (updated global simulation conditions and regeneration; representative of Fig. 1(b)). All use the same maximum of 20 iterations.

Table R1. Iterative baselines on Physics-IQ Verified, with a 20-round maximum. Lower P-LPIPS indicates better preservation of unaffected content; bold marks the best result.

MethodPhysics-IQ ↑P-LPIPS ↓Mean Calls ↓
Resampling41.10.0929.02
Prompt Refinement42.00.0898.34
Simulation Refinement43.40.0677.52
PhysRepair (Ours)43.60.0524.82

PhysRepair achieves the highest Physics-IQ and best preservation with the fewest mean calls among these controlled strategies. Relative to Simulation Refinement, its mean calls fall by 35.9%, while achieving comparable physical correctness and better content preservation. Unlike broad regeneration, PhysRepair uses a causal specification to restrict which physical states should change and which valid states should remain.

AuthorsOpenReview ↗

Response to Reviewer RFCK (2/3)

W2: PhysRepair may be tied to videos from a specific generator, and additional training and inference costs may offset its gains.

R2: Cross-generator video repair and transfer of long-term experience. PhysRepair is an inference-time physics-repair framework, rather than a model trained exclusively on errors produced by one particular source generator. We apply the same repair pipeline to videos initially generated by Wan2.2-I2V-14B, rather than H3. We also evaluate whether repair experience acquired on H3-generated videos can be reused for Wan2.2 inputs without updating that experience during Wan2.2 evaluation.

Table R2. Cross-generator generalization to Wan2.2-I2V-14B videos. The unrepaired input has zero repair calls by definition; call efficiency is compared between the repaired settings. Bold indicates the best repaired setting.

SettingPhysics-IQ ↑Mean Repair Calls ↓
Original Wan2.2-I2V-14B32.200.00
Wan2.2 + PhysRepair, without long-term memory37.3010.81
Wan2.2 + PhysRepair, frozen H3 experience38.707.24

Even without long-term memory, PhysRepair improves physical correctness on a different source generator. Moreover, reusing frozen repair experience accumulated from H3-generated videos further improves Physics-IQ and reduces mean repair calls. This indicates that previously learned repair strategies can be useful across source video generators; the framework does not require that all input videos originate from H3.

Training and Computational Cost. PhysRepair uses frozen pretrained video-generation and control modules, requiring neither source-generator-specific retraining nor parameter updates to learn new repair strategies. The additional computational overhead primarily comes from physical failure diagnosis, perception tool calls, local physics simulation, and iterative video repair. Our cross-generator experiments further show that reusing accumulated long-term experience reduces redundant repair attempts and lowers the average number of repair calls. Together with the comparisons against Resampling, Prompt Refinement, and Simulation Refinement, these results demonstrate that PhysRepair achieves higher physical correctness with greater repair-call efficiency.

AuthorsOpenReview ↗

Response to Reviewer RFCK (3/3)

W3: Red boxes obscure the qualitative comparison in Figure 3; in Figure 4, a previously valid foot–ball contact seems to disappear after repair.

R3: Figure 3. To improve the clarity of the comparison, we have replaced the two comparison panels with the annotated cue-ball and wrench--box cases and updated the caption in Figure 3 (refined, page 7) of the revised manuscript.

Figure 4. The missing foot–ball contact in Figure 4 is produced by the Without Causal Constraints ablation, not the complete PhysRepair method. In that ablated variant, the repair focuses on the visible soccer-ball/blue-ball collision and can improve the downstream collision while failing to preserve the earlier, physically valid foot–ball contact. The resulting ball trajectory no longer has a valid kick as its cause. This is the failure mode illustrated by the ablation: fixing the immediate error without maintaining the whole interaction's causal structure can disrupt valid preceding states.

The full method explicitly represents the chain foot–ball contact → soccer-ball motion → soccer-ball/blue-ball collision. It treats the valid kick and trustworthy pre-collision motion as states to preserve, while the incorrect collision and its affected descendants constitute the repair target. Thus, the same object can be subject to preservation constraints at one time and repair constraints at another, allowing PhysRepair to correct subsequent physical errors while preserving previously valid interactions as much as possible.

Manuscript revisions. In the revised manuscript, red marks changes to the original content, and blue marks newly added content.

To address W1/Q2, we have integrated PhyCo and PSIVG into the existing discussion of physics-aware video generation in Section 2.1 (page 3), updated the baseline metrics in Table 1 (updated, page 6), and explained how the added iterative controls relate to the prompt- and simulation-refinement strategies in the original Figure 1. Sections 4.1–4.2 define these controls and analyze Table 2 (newly added, page 8).

To address W2, Appendix C.2 (page 14) clarifies the frozen generation/control modules, while Appendix A.4 (page 13) and Table A.4 (newly added, page 14) provide the complete Wan2.2 repair results and discussion of frozen H3 experience transfer without source-specific retraining.

To address W3, Figure 3 (refined, page 7) uses the updated cue-ball and wrench--box comparison panels with per-row failure descriptions, with the corresponding qualitative analysis aligned to those annotations; Figure 4 (refined, page 9) and Section 4.3 (pages 8–9) identify the missing kick as an ablation failure and explain preservation of valid preceding states, with the added cup/basin example showing correction of both physical inconsistencies.

Reviewer_JNBoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes PhysRepair, a post-generation framework that iteratively diagnoses and locally repairs physical inconsistencies in generated videos. Spatiotemporal Causal Constraints identify the affected object–space–time states, while Experience-Augmented In-Context Learning retrieves previous repair trajectories to guide simulation-conditioned regeneration and verification.

Critical strengths

The post-hoc repair formulation is practically relevant. Local correction can preserve already valid appearance and dynamics, unlike methods that globally regenerate a video after each detected failure.

The overall benchmark result improves over the underlying generator. PhysRepair reaches a Physics-IQ score of 43.6, compared with 39.8 for MiniMax H3 FL2VA; ST increases from 22.8 to 34.7, and WS from 40.5 to 44.2.

Critical weaknesses

The two main contributions lack controlled quantitative ablations. Causal constraints are supported only by two qualitative examples, while the memory analysis lacks no-memory, random-retrieval, and fixed-memory controls. Therefore, the reported benchmark gain cannot be attributed reliably to either component.

Important implementation and evaluation details are missing. The paper only refers to “GPT” and a “simulator,” without specifying their versions, causal-graph construction, invalid-state detection, or verification thresholds. Moreover, long-term memory accumulates across videos, but the benchmark contains 198 instances from 66 experiments observed from three viewpoints; it is unclear whether one viewpoint can enter memory before another viewpoint of the same experiment is evaluated.

Questions for authors

Clarification: How are test instances ordered, and is long-term memory cleared before evaluating another viewpoint of the same physical experiment? Please report results with grouping by experiment and no cross-view memory.

Assessment-critical experiment: Can the authors provide quantitative results for variants without causal constraints, without long-term memory, with random retrieval, and with the complete method, including Physics-IQ and average repair rounds?

Clarification: What exact GPT model and simulator are used? How are the causal graph, first invalid state, verification criteria, and stopping thresholds implemented? Please also report the average number of repair rounds and inference cost per sample.

2 comments
AuthorsOpenReview ↗

Response to Reviewer JNBo (1/2)

We thank the reviewer for emphasizing controlled attribution, implementation transparency, and cross-view memory leakage. We combine overlapping weaknesses and questions below.

W1: Neither Spatiotemporal Causal Constraints nor Experience-Augmented In-Context Learning is isolated by controlled quantitative ablations; no-memory, random-retrieval, and fixed-memory controls are missing.

Q2: Provide Physics-IQ and average repair rounds for variants without causal constraints, without long-term memory, with random retrieval, and with the complete method.

Q3: What are mean repair rounds and inference cost per sample?

R1: Spatiotemporal Causal Constraints determine what to repair by tracing affected object–space–time states, while Experience-Augmented In-Context Learning guides how to repair through relevant prior experience. Together, they support targeted correction and more efficient repair decisions. We added ablations of the causal constraint and multiple memory configurations on the same Physics-IQ Verified evaluation set. In addition to disabling long-term memory and using random eligible retrieval, we evaluate Frozen External Memory, which is built by generating videos from batches of independently constructed prompts, repairing those videos, and retaining the resulting experience in a fixed retrieval library for evaluation.

Table R1. Quantitative causal-constraint and memory ablations on Physics-IQ Verified. Bold indicates the best result.

SettingPhysics-IQ ↑Mean Calls ↓
Without causal constraints42.15.96
Without long-term memory42.77.05
Random Eligible Retrieval42.96.37
Frozen External Memory43.15.24
PhysRepair (Ours)43.64.82

Causal constraints contribute directly to repair quality: removing them reduces Physics-IQ and increases mean calls. Memory has its most pronounced effect on efficiency: removing it causes the largest increase in calls, despite a smaller decrease in Physics-IQ. Random retrieval only partly recovers these gains, showing that the relevance of retrieved experience matters, not merely access to historical examples. Frozen external memory performs close to the full method without any updates from the current evaluation sequence. Together, these results provide direct controlled evidence for the complementary roles of causally constrained repair and experience-guided decision-making.

AuthorsOpenReview ↗

Response to Reviewer JNBo (2/2)

W2: Exact GPT/simulator versions, graph construction, first-invalid-state detection, verification criteria, and cross-view evaluation protocol are underspecified.

Q1: How are test instances ordered and long-term memory isolated between viewpoints of the same physical experiment?

Q3: Which GPT model and simulator are used? How are the graph, invalid state, verification criteria, and stopping conditions implemented?

R2: We apologize for any confusion and provide further details on the implementation and evaluation protocol below.

Model and simulator. The Physics Repair Agent uses GPT-6-Astra for event interpretation, evidence gathering, tool orchestration, and repair decisions. Local physical modeling and rendering use PyBullet and Blender.

Causal representation and first-error localization. PhysRepair maintains an object-centric spatiotemporal graph whose nodes represent object states at particular times (including observed identity, position, visibility, and relevant motion information). Temporal edges link consecutive states of the same object; interaction edges represent contacts, collisions, or support, with time windows and supporting evidence. The agent first performs a coarse scan to identify suspicious events and then uses denser frames, segmentation, tracking, depth, and registration cues. Where a violation admits a reliable rule—for instance, a passive object moving before contact and in the absence of an alternative force—the program checks the observation-based temporal conditions. For visual cases that cannot be fully formalized, the agent records an evidence-grounded judgment. Once the first invalid state is identified, supported temporal and interaction edges are traversed programmatically to determine causally affected descendants, editable states, and states that should be preserved.

Verification and stopping. PhysRepair employs a termination mechanism based on repeated physical failure diagnosis rather than a fixed physics-score threshold. Specifically, after each repair iteration, the system invokes the physics diagnostic module to examine the generated video through fine-grained inspection and relevant perception tools, checking whether previously identified failures have been corrected and whether new physical inconsistencies have emerged. If no further physical failures are detected, the system terminates and returns the current video; otherwise, it continues the repair process based on the updated diagnosis. The process also terminates when the maximum iteration budget is reached or a technical failure occurs.

Experiment-group isolation. The 198 Physics-IQ Verified instances arise from 66 underlying physical experiments, each with three viewpoints. We evaluate the three viewpoints of an experiment as one group. All three viewpoints within an experiment group share the same long-term memory snapshot taken before the group begins. Experiences generated within a group are held in a buffer and become available only after all three viewpoints are completed, for use in later, different experiment groups. Thus one viewpoint cannot retrieve repairs from another viewpoint of the same experiment.

Manuscript revisions. In the revised manuscript, red marks changes to the original content, and blue marks newly added content.

To address W1/Q2/Q3, we have added the controlled causal-constraint and memory comparisons in Table 3 (newly added, page 10) and their analysis in Section 4.3, including no-memory, random-eligible-retrieval, frozen-external-memory, and complete-method settings. Appendix A.2 (page 13) and Appendix A.3 (page 13), with Table A.2 (newly added, page 13) and Table A.3 (newly added, page 13), provide the complete budget analysis and native-call distribution. Section 4.1 (pages 6–7) defines Mean Calls as an iteration-efficiency measure.

To address W2/Q1/Q3, Sections 3.1–3.2 (pages 3–5) retain evidence-grounded first-error localization, graph traversal, the counterfactual repair formulation, and diagnosis-based stopping; the additional reconstruction and conditioning details appear in Appendices C.4–C.5 (page 15). Appendix C.2 (page 14) names GPT-6-Astra, PyBullet, Blender, and the frozen repair components; Appendix C.1 (page 14) specifies the shared pre-group memory snapshot and delayed updates across the three viewpoints, and Appendix C.3 (page 14) details dynamic causal-scope updates.

Reviewer_QFiJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes PhysRepair, a post-hoc framework that repairs physical errors in generated videos instead of adding physical guidance before synthesis. A Physics Repair Agent scans the video, gathers perception evidence (tracking, segmentation, depth, contact cues) around suspected failures, and produces a structured diagnosis. Spatiotemporal Causal Constraints organize object states and interactions into a graph, mark the first physically invalid state and its causally affected descendants as editable, and specify which states and content must be preserved. The local scene is reconstructed and simulated to obtain a counterfactual physical state. This state is converted into generation-compatible conditions and, together with a decontaminated source video, used to regenerate the affected content. A short-term working memory and a long-term experience memory guide later repair decisions. On Physics-IQ Verified, PhysRepair scores 43.6, compared with 42.7 for Cosmos3-Super and 39.8 for MiniMax H3 FL2VA.

Critical strengths
  • The repair is specified at the level of interactions rather than appearance. The causal repair specification states what to change (the intervention target and editable object–space–time region) and what must stay fixed, so the downstream consequences of a failure are repaired together.
  • Keeping the erroneous source motion out of the regeneration is a sensible design. The method suppresses conflicting source dynamics within the editable region and, in parallel, supplies simulator-derived trajectory and contact cues, so the old motion does not compete with the corrected one.
Critical weaknesses
  • The core reconstruction and conditioning steps are underspecified. The paper only says the local interaction is "reconstructed and simulated in simulator", without naming the simulator or explaining how the local scene is recovered from a generated video, how conflicting source dynamics are suppressed, or how the H3 control adapter injects the conditions. These steps decide whether a plausible simulation can become a faithful video repair, especially for fluids, optics and thermodynamics.
  • It is unclear how the causal repair boundary handles changed interactions. The graph G^(k) is rebuilt in each round, but within a round the editable region is derived from the incorrect video before the counterfactual is simulated. A corrected trajectory can create contacts that are not in that graph, or require a reaction from an object marked as preserved, and the paper does not say how such conflicts are detected or resolved.
  • Neither claimed contribution is ablated quantitatively. There are no Physics-IQ numbers without the causal constraints, without the long-term memory, without the simulator, or with different round budgets. As a result, the gain cannot be attributed to either component.
  • The repair gain is measured against a different model. The paper generates and repairs with MiniMax H3 Ref2VA plus a control adapter, but presents the improvement from 39.8 to 43.6 over H3 FL2VA as "the benefit of targeted physical repair over the initial generation". The one-pass score of the actual backbone is needed. The experience memory protocol is unclear. The paper states that reference continuations and evaluator masks are never exposed, but each Physics-IQ experiment appears from three viewpoints. If the long-term memory is filled during evaluation, a repair from one view can be retrieved for another, so results with an empty or frozen memory are needed.
Questions for authors
  • Which simulator is used for each Physics-IQ category, and on how many of the 198 instances is simulation actually invoked?
  • What is the one-pass Physics-IQ score of H3 Ref2VA, the backbone used for repair?
  • How is the long-term experience memory populated, and could it contain repairs from other viewpoints of the same experiment?
  • When the counterfactual trajectory creates new contacts or conflicts with preserved states, how is this handled? How is temporal continuity kept at the boundary between edited and preserved frames?
  • How is the first physically invalid state determined: by explicit rules on the perception evidence, or by GPT judgment? Are causally affected descendants traced by following graph edges, or also decided by the agent?
  • What exactly are the "physics criteria" used to stop the loop? On how many Physics-IQ instances does the agent find no failure and return the initial video unchanged? What is the distribution of repair rounds across instances?
7 comments
AuthorsOpenReview ↗

Response to Reviewer QFij (1/4)

We thank the reviewer for the detailed assessment. We group questions with the corresponding weaknesses where they concern the same implementation or evaluation issue.

W1: Scene reconstruction, suppression of conflicting source dynamics, and injection through the H3 control adapter are underspecified.

Q1: Which simulator is used for each Physics-IQ category, and on how many of the 198 instances is simulation actually invoked?

R1: We apologize for any confusion and provide further implementation details below.

Local reconstruction. We use PyBullet and Blender for local physical modeling, simulation, and rendering. The Physics Repair Agent first diagnoses the interaction and determines a causally relevant subset of objects, rather than attempting to reconstruct every object in the video. Segmentation, tracking, relative depth, and background-motion estimates provide approximate geometry, object identities, spatial relationships, and camera alignment. Initial states and physical parameters are estimated from observations judged trustworthy, especially states before the first detected error; erroneous source motion is not treated as a reliable trajectory to fit. Simulation outputs are checked against available evidence through reprojection, contact timing, and state evolution.

Suppressing conflicting source dynamics. Within the causally editable region, we remove motion cues that conflict with the corrected physical trajectory. Object masks and background registration identify contaminated reference regions; usable background is recovered from other frames when available and completed when necessary. Unaffected scene context and trustworthy object appearance remain reference information.

Physics-Guided Condition Injection. We use MiniMax H3 Ref2VA as the repair backbone, together with the pretrained Fun ControlNet Union to inject physics-guided motion conditions. Specifically, the cleaned RGB reference video is fed through the Ref2VA reference-conditioning pathway to preserve scene appearance, object identities, and valid contextual information. In parallel, the simulated counterfactual states are projected and rendered according to the source video's camera parameters and timeline, producing a frame-aligned edge-conditioning video in the original image coordinates. This control video is encoded by the video VAE and passed to the Fun ControlNet branch, whose features are injected into selected Transformer layers of H3 during denoising to guide the target objects along the corrected trajectories. The reference and control pathways thus provide complementary appearance and motion guidance. Finally, the generated video is evaluated by the physics verifier for physical consistency and preservation of non-target content.

Simulator Coverage. Across the five physics categories in Physics-IQ, we use PyBullet and Blender jointly for physical modeling, simulation, and rendering. Guided by video evidence, the agent identifies the dominant physical mechanisms, selects and combines appropriate physical models based on governing physical laws and material priors, and translates the model parameters, boundary conditions, and causal constraints into executable simulation code. PyBullet and Blender are then used to simulate and render the resulting physical interactions, providing counterfactual physical states for subsequent video repair.

AuthorsOpenReview ↗

Response to Reviewer QFij (2/4)

W2: A counterfactual trajectory may create contacts absent from the original causal graph or conflict with preserved states.

Q4: How are new contacts and conflicts handled, and how is temporal continuity maintained between edited and preserved frames?

R2: We apologize for any confusion and provide further clarification on causal repair boundaries and temporal continuity below.

Causal boundaries. The editable support is not treated as an immutable mask. At the start of an iteration, PhysRepair constructs a local simulation using only objects in the currently diagnosed causal chain; objects outside that chain are not automatically simulated. After local simulation, the corrected trajectory is projected back into the original video's space–time coordinates and compared with the observed objects, tracks, relative depth, and spatial relationships across the full source video. This identifies potential new contacts or conflicts with previously preserved states. If a new interaction requires a preserved object to respond, the agent revisits the causal scope, expands the editable object–time region to the newly affected states, reconstructs the enlarged local scene, and resimulates. After generating a repaired video, the system diagnoses it again and updates the next-round graph G(k+1)G^{(k+1)}. Thus, repair boundaries can be adjusted both within a round and between rounds.

Temporal continuity. The term localized repair refers to diagnosis, causal intervention, local simulation, source-motion cleaning, and physical conditioning—not to hard splicing of individually generated frames. H3 jointly regenerates the complete video clip using the corrected motion conditions and retained reference content. Consequently, there is no hard seam between separately stitched edited and unedited clips.

W3: The contributions of causal constraints, long-term memory, the simulator, and the repair-round budget lack controlled quantitative ablations.

R3: Spatiotemporal Causal Constraints determine which states to repair, simulation provides a physically plausible counterfactual, and long-term experience memory guides how the agent performs the correction. We performed controlled ablations of Spatiotemporal Causal Constraints, long-term experience memory, and the simulator, as well as an evaluation at different maximum repair-round budgets.

Table R3(a). Component ablations on Physics-IQ Verified. Bold denotes the best result.

SettingPhysics-IQ ↑Mean Calls ↓
Without causal constraints42.15.96
Without long-term memory42.77.05
Without simulator41.96.27
PhysRepair (Ours)43.604.82

Removing either causal constraints or the simulator produces a larger decline in Physics-IQ, supporting their contribution to accurate physical correction. Removing long-term memory causes a smaller score drop but the largest increase in mean calls, indicating that retrieved experience is especially helpful for avoiding redundant repair attempts.

Table R3(b). Effect of maximum repair rounds. Improvement is measured against the FL2VA initial generation (0 repair rounds).

Maximum Repair RoundsPhysics-IQ ↑Improvement (Δ) ↑
039.80+0.00
542.08+2.28
1042.92+3.12
2043.60+3.80

Even with the maximum number of repair rounds limited to five, PhysRepair achieves a substantial improvement in physical correctness over the initial generation, demonstrating its ability to correct physical inconsistencies within a small number of iterations. As the repair budget increases, Physics-IQ continues to improve, indicating that iterative diagnosis, repair, and verification progressively enhance physical plausibility, although the gains gradually diminish at higher iteration budgets.

AuthorsOpenReview ↗

Response to Reviewer QFij (3/4)

W4: The gain from 39.8 to 43.6 is compared against FL2VA although Ref2VA is used for repair; memory accumulation may allow test-view leakage.

Q2: What is the one-pass Physics-IQ score of H3 Ref2VA?

Q3: How is long-term memory populated, and can it contain repairs from another viewpoint of the same experiment?

R4: H3 FL2VA delivers superior standalone video generation quality compared with H3 Ref2VA, which focuses more on controllability. We therefore use FL2VA-generated videos as the primary baseline to evaluate the improvements achieved by PhysRepair. To further distinguish the benefits of our repair framework from backbone effects, we additionally evaluate H3 Ref2VA under the one-pass generation setting and compare it with H3 FL2VA and the full PhysRepair framework.

Table R4. One-pass generation and complete repair on Physics-IQ Verified.

SettingPhysics-IQ ↑
H3 Ref2VA34.9
H3 FL2VA39.8
PhysRepair43.6

H3 Ref2VA achieves a one-pass Physics-IQ score of only 34.9, notably lower than the 39.8 achieved by H3 FL2VA. Despite the lower standalone performance of the Ref2VA backbone used for repair, PhysRepair improves the Physics-IQ of FL2VA-generated videos from 39.8 to 43.6 through physical failure diagnosis, causal constraints, simulation-guided conditioning, and iterative repair. These results demonstrate that the performance gains stem from our PhysRepair framework rather than the use of the Ref2VA backbone.

Cross-view memory leakage. We clarify that our evaluation adopts experiment-level grouping and delayed memory updates to prevent experience leakage across different viewpoints of the same physical experiment. Physics-IQ Verified contains 198 instances from 66 physical experiments, each observed from three viewpoints. Instances are evaluated in experiment groups. All three viewpoints within an experiment group share the same long-term memory snapshot taken before the group begins. New experiences are buffered until all three viewpoints of that experiment have been evaluated; only then may they become available to subsequent, different experiment groups. Accordingly, no viewpoint can retrieve an experience created by another viewpoint of the same experiment during its evaluation.

Q5: Is the first physically invalid state found by explicit perception-based rules or by GPT? Are causally affected descendants traced along graph edges or decided by the agent?

R5: We apologize for any confusion and provide further clarification on failure localization and causal tracing below.

Evidence-grounded failure localization. PhysRepair combines agent-guided evidence collection with explicit checks when the failure criterion can be formalized. The agent first inspects sparsely sampled frames, proposes potentially invalid events, then zooms into relevant intervals and invokes segmentation, tracking, depth, or motion tools. For rule-checkable events—e.g., a passive object's motion beginning before the interaction that could cause it, after considering alternative support or forces—a program evaluates the relevant temporal evidence. For ambiguous visual phenomena that do not admit a reliable fixed rule, the agent makes an evidence-grounded judgment and retains the supporting observations.

Causal tracing. The causal representation contains object states at different times. Temporal edges link successive states of an object; interaction edges capture evidence-supported contacts, collisions, and support across objects. The agent proposes and checks candidate relations, while a program traverses supported, temporally consistent graph edges from the first invalid state to identify downstream affected states. These become editable; unrelated valid states become preservation constraints. Thus GPT provides physical interpretation and tool orchestration, whereas formalizable checks and causal-scope propagation rely on explicit evidence and graph operations.

Reviewer_QK66OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PhysRepair, which diagnoses physical errors in a generated video and repairs the affected content using simulation-derived conditions. A spatiotemporal causal graph determines the editable states, while retrieved repair experience guides the agent's decisions. Results on Physics-IQ Verified, qualitative examples, and a small user study suggest that this iterative repair process can improve physical plausibility.

Critical strengths
  1. Local repair is a worthwhile direction beyond prompt refinement, as used in PhyT2V. Tracing the consequences of an observed error could correct an interaction while retaining already-valid content. The qualitative examples make the benefit of this design clear.
  2. The improvement from 39.8 to 43.6 over the reported MiniMax H3 baseline on Physics-IQ is encouraging, especially the gain in spatiotemporal agreement. Reusing successful repair experience is also a reasonable way to reduce repeated trial and error.
Critical weaknesses
  1. The experiments do not yet establish the main advantage of localized causal repair over ordinary iterative refinement. The agent can use up to 20 repair rounds, but there is no comparison with regeneration or refinement under a comparable budget. The causal-repair examples are promising; a quantitative assessment of both corrected physics and preservation of unaffected content is still missing.
  2. The benefit of accumulated experience remains preliminary. Fewer repair rounds as memory grows is suggestive, but does not establish that retrieved experience improves repair success or efficiency on the same problems. This matters because learning reusable repair strategies is a central contribution beyond a generate-and-check loop.
  3. Quantitative validation is concentrated on Physics-IQ Verified, and the user study covers only ten scenarios. Although the benchmark spans several physical domains, there is limited evidence about repair reliability in less controlled scenes with multiple simultaneous errors. A broader evaluation of repair success on such scenes would help establish the agent's practical scope.
Questions for authors
  1. Under a comparable generation budget, does localized repair improve physical correctness and preserve unaffected content better than iterative regeneration?
  2. On the same held-out cases, how much does accumulated repair memory improve success rate or reduce the number of attempts compared with an agent without that memory?
3 comments
AuthorsOpenReview ↗

Response to Reviewer QK66 (1/3)

We thank the reviewer for recognizing the motivation and the promising results of localized, causally grounded video repair. We address the concerns and related questions together below.

W1: The advantage of localized causal repair over ordinary iterative refinement is not yet established under a comparable budget; quantitative preservation results are missing.

Q1: Under a comparable generation budget, does localized repair improve physical correctness and preserve unaffected content better than iterative regeneration?

R1: PhysRepair targets the first detected physical error and its causal consequences while constraining unaffected states to remain unchanged. This avoids unnecessary changes to valid interactions during repair. We added three representative iterative baselines under the same evaluation protocol: Resampling, which regenerates candidate videos using different initial noise; Prompt Refinement, which revises the text prompt based on physical feedback and regenerates the video; and Simulation Refinement, which updates global simulation conditions to guide full-video regeneration. All methods use the same maximum of 20 iterations. We evaluate physical correctness (Physics-IQ), preservation of unaffected content (P-LPIPS), and iteration efficiency (Mean Calls).

Table R1. Comparison of iterative refinement strategies on Physics-IQ Verified under a 20-round maximum. Best results are in bold.

MethodPhysics-IQ ↑P-LPIPS ↓Mean Calls ↓
Resampling41.10.0929.02
Prompt Refinement42.00.0898.34
Simulation Refinement43.40.0677.52
PhysRepair (Ours)43.60.0524.82

PhysRepair achieves the best physical correctness and content preservation while requiring the fewest repair calls. Compared with Resampling and Prompt Refinement, it delivers better physical correctness and preservation with fewer calls. It also slightly outperforms Simulation Refinement in both Physics-IQ and P-LPIPS, while reducing mean calls by 35.9%. These results support the benefit of causally localized repair over repeated global regeneration under the same maximum iteration budget.


AuthorsOpenReview ↗

Response to Reviewer QK66 (2/3)

W2: The contribution of accumulated repair experience is preliminary; fewer rounds as memory grows do not establish benefits from retrieved experience on the same problems.

Q2: On the same held-out cases, how much does accumulated repair memory improve correctness or reduce attempts relative to an agent without that memory?

R2: PhysRepair reduces redundant repair attempts by retrieving experiences from similar physical failures, matched to the current failure type and interaction physics. We compare three retrieval strategies on the same evaluation cases: None (no long-term memory), Random Eligible Retrieval (a randomly selected eligible historical repair), and Ours (experience retrieved according to the current failure type and interaction physics). The rest of the repair pipeline is unchanged.

Table R2. Effect of long-term repair experience. Bold indicates the best result.

Memory settingPhysics-IQ ↑Mean Calls ↓
None42.77.05
Random Eligible Retrieval42.96.37
Relevant Retrieval (Ours)43.64.82

Relevant experience improves both physical correctness and repair efficiency. Removing long-term memory substantially increases mean repair calls, whereas random eligible retrieval provides only marginal improvements over the no-memory setting. The advantage over random retrieval indicates that the useful signal is not merely the presence of historical examples; matching prior repair experience to the current physical failure matters. This supports Experience-Augmented In-Context Learning as a way to reuse repair strategies rather than repeatedly exploring them from scratch.

AuthorsOpenReview ↗

Response to Reviewer QK66 (3/3)

W3: Evaluation is concentrated on Physics-IQ Verified and a ten-scenario user study; reliability on less controlled scenes with multiple simultaneous errors remains uncertain.

R3: PhysRepair re-diagnoses each repaired candidate and updates its causal repair specification to address remaining or newly introduced physical errors. We additionally provide a case (Figure 4, bottom example) containing two distinct physical inconsistencies: water leaks implausibly from a cup, and an object in the basin beneath it moves without a plausible physical cause. PhysRepair repeatedly diagnoses the current result, updates the causally affected repair scope, and corrects the two failures across successive repair steps. The example shows that the procedure can process more than one error within the same scene rather than being limited to a single predefined intervention.

Manuscript revisions. In the revised manuscript, red marks changes to the original content, and blue marks newly added content.

To address W1/Q1, we have added the iterative-strategy definitions and preservation/call-efficiency measures in Section 4.1 (pages 6–7), and the controlled comparison in Table 2 (newly added, page 8) with analysis in Section 4.2 (page 8).

To address W2/Q2, Table 3 (newly added, page 10) and Section 4.3 now compare no long-term memory, random eligible retrieval, frozen external memory, and relevant retrieval on the same evaluation cases; Figure 5(a) (refined, page 10) updates the memory-growth analysis, and Appendix C.1 (page 14) explains experiment-group isolation and delayed memory updates.

To address W3, we have added the cup/basin multi-error case as the bottom example in Figure 4 (refined, page 9), with corresponding analysis in Section 4.3 (page 9) explaining repeated diagnosis and causal-scope updates to correct both physical failures.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.