- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces PhysForce, a two-stage, physics-aware force control framework designed to address key challenges in interactive video world models, specifically, the leakage of future motion information caused by trajectory-based force representations, the difficulty in localizing internal representations, and the scarcity of force-annotated data. Stage 1 (SFT) employs a Latent Force Intervention Encoder (LFIE) to encode forces while preventing trajectory leakage, alongside a Physics Attribution Network (PAN) that localizes physical responses through temporal differences. Stage 2 (RL) utilizes Physics-Aware Reinforcement Learning (PARL), which incorporates self-supervised rewards derived from V-JEPA and a "Dynamic Gate" mechanism applied to videos lacking explicit force annotations, thereby enhancing physical consistency and out-of-distribution (OOD) generalization.
Critical strengths
**Comprehensive Evaluation: ** The work demonstrates exceptional empirical rigor through extensive evaluations across 14 metrics on both ID and OOD benchmarks, thorough component-wise ablations, a large-scale user study with 150 participants, and rich qualitative visualizations.
Critical weaknesses
-
The framework lacks fine-grained temporal control over forces. By simply broadcasting force conditions across time, localized interactions are restricted to initial impulses, and the offline generation pipeline precludes real-time or dynamic adjustments.
-
The V-JEPA prediction error used in PARL is a representation-based empirical prior rather than an analytical physics solver. While effective as a statistical proxy for consistency, it remains a heuristic that cannot guarantee strict physical plausibility under complex dynamics.
Questions for authors
-
Since LFIE and PAN operate entirely within 2D image coordinates and video latent spaces without explicit 3D geometry or depth modeling, how does the model handle complex 3D interactions involving severe occlusions, depth-dependent collisions, or out-of-plane forces?
-
Since the V-JEPA prediction error functions as a representation-based empirical proxy rather than an analytical solver, did the authors observe failure modes where the reward favored visually smooth but physically incorrect behaviors (such as violations of momentum conservation in multi-body collisions)?
-
While the paper achieves strong empirical performance through a complex multi-component architecture and two-stage training pipeline, the underlying conceptual takeaways could be highlighted more clearly. Could the authors provide a more high-level abstraction or conceptual insight that would help the community generalize these lessons beyond this specific, complex setup?
I am open to raising my rating once my concerns are addressed.
6 comments
Feedback to Reviewer SRP8 (2/2)
W2 & Q2. Scope and Limitations of the V-JEPA Reward
We thank the reviewer for raising this important concern. We agree that V-JEPA prediction error serves as a representation-level prior rather than an analytical physics solver, and cannot guarantee strict physical plausibility in all complex scenarios.
In PhysForce, Stage 1 first learns force-induced motion and object interactions through supervised fine-tuning on diverse physics-engine-based scenarios from GoalForce and Force Prompting Dataset. In Stage 2, we intentionally introduce a frozen V-JEPA model without force conditioning. This allows RL to leverage V-JEPA's ability to predict future states across diverse real-world scenarios, encouraging the generator to learn more general world dynamics beyond specific force-controlled settings and thus improving generalization. The reward of V-JEPA is also supported by prior work [1], which demonstrates the effectiveness of latent world-model prediction errors in improving physical plausibility, which we have discussed in Appendix A.3.
As shown in Appendix Fig. 17, we observe occasional physical inconsistencies, such as premature energy transfer between objects before contact, which we discuss in the Limitations. Nevertheless, as shown in Fig. 9, V-JEPA guided PARL leverages general future-state prediction priors to improve the learning of world dynamics beyond force-labeled scenarios, reducing unnatural object deformations and jerky motions.
Our qualitative results show reduced abrupt direction changes, interrupted motion, and structural collapse. Quantitatively, PARL improves OOD DCS from 0.625 to 0.782 and ID ST-IoU from 0.275 to 0.332.
[1] Yuan J, Zhang X, Friedrich F, et al. Inference-time physics alignment of video generative models with latent world models[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2026.
Q3. Design Principles and Broader Applicability of PhysForce
-
High-level conceptual takeaway. The central contribution of PhysForce is a unified intervention–response generalization framework that progressively connects physical intervention, response modeling, and dynamics generalization. Specifically, LFIE defines outcome-independent force conditions without leaking future trajectories; PAN learns localized state transitions, providing response supervision for SFT and spatially grounded rewards for RL; and PARL leverages these rewards to explore beyond force-labeled training data while filtering physically inconsistent outcomes. Thus, the three components form a tightly coupled framework rather than an independent combination of modules, enabling the model to learn not only how forces are applied, but also how objects respond and how these dynamics generalize to unseen scenarios. As a result, PhysForce achieves 36.0% and 61.0% improvements in DCS over existing SOTAs on OOD and ID benchmarks, respectively.
-
Broader implications for other tasks. (1) This intervention–response paradigm can extend to robotic manipulation and action-conditioned video generation, where task-specific actions serve as intervention inputs and resulting object-state changes provide response supervision. (2) Furthermore, RL post-training can leverage response-aware self-supervised rewards on unlabeled interaction videos to refine learned dynamics beyond action-labeled training data, improving generalization without requiring additional annotated demonstrations.
Dear Reviewer SRP8,
Thank you again for your thoughtful feedback and constructive suggestions. Your comments motivated our additional explorations of streaming generation and 3D interactions, as well as a deeper discussion of the design of our physics-aware reward. We truly appreciate your engagement with our work and would welcome any further thoughts on our response.
Best regards,
The Authors
Feedback to Reviewer SRP8 (1/2)
Thank you for your constructive feedback and valuable suggestions. We have conducted the requested experiments and incorporated the corresponding results and discussions into the revised manuscript. All revisions are highlighted in blue for ease of reference.
Thank you for suggesting that we extend PhysForce to support interactive force updates, we conduct an experiment to expand our PhysForce on Casual Forcing in Fig.11 in the revised paper. We trained a causal autoregressive variant with casual forcing and applied distribution matching distillation (DMD) to reduce sampling cost. Fig.11(a) shows an example of streaming generation. As shown in Fig. 11(b), the resulting model achieves 16.37 FPS after the first frame, compared with 0.34 FPS for the original model (48.15× speedup), while reducing end-to-end first-frame latency from 236.45 s to 2.17 s (108.96× reduction).
The current model uses 2D image-space force localization and video latent response modeling. Nevertheless, both LFIE and PAN can be extended to incorporate 3D information. During the rebuutal period, we include a preliminary 3D extension in Fig.10:
With these extensions, PhysForce can generate force-aligned responses to both 3D point forces and 3D wind forces in Fig.10, providing preliminary evidence of its potential for 3D force control.