- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the confounding between physical interactions and camera motion in latent action models. The authors propose InterACT, which constructs crossed camera–physics intervention quartets in simulation by varying camera trajectories while holding the physical interaction fixed, and varying physical trajectories while holding the camera fixed, thereby explicitly separating camera-induced variation from genuine physical changes. Methodologically, InterACT builds on a pretrained latent-action encoder and introduces a constrained residual correction module, while using a frozen copy of the pretrained encoder as a reference for preserving physical scale. The model is adapted through camera alignment, physical-scale preservation, action and outcome supervision, and separated gradient paths. Experiments show that InterACT substantially reduces camera-related representation variation while preserving physical interaction information under controlled ID, Camera-OOD, and Physics-OOD settings, and also improves downstream policy learning performance.
Critical strengths
The problem formulation is valuable, and the crossed camera–physics intervention design is clear and effective. The paper correctly identifies the confounding between camera motion and physical interaction in latent action representations. By constructing paired interventions with the same physical interaction under different cameras and different physical interactions under the same camera, the method provides direct supervision for learning representations that are invariant to camera motion while remaining sensitive to physical changes. This design is intuitive, interpretable, and constitutes the main novelty of the paper.
The representation-learning objective and evaluation protocol are carefully designed to balance nuisance suppression and preservation of task-relevant information. The method not only minimizes camera-induced variation, but also uses a frozen reference and a physical-scale constraint to prevent artificially improving alignment by expanding or shrinking physical distances. In addition, action error and outcome gain are used to verify that interaction-relevant information is still retained in the learned representation.
Critical weaknesses
The reliance on synthetic intervention data limits the generality of the approach, and the evidence for transfer to real-world settings is insufficient. InterACT relies on controllable camera–physics interventions generated in simulation, whereas camera motion in real videos is substantially more complex. In addition, existing approaches can suppress camera-related nuisance factors implicitly through large-scale real-video training and view-robust representation learning. The current experiments are mainly built on ALAM, whose representation is relatively camera-sensitive, and the paper does not demonstrate whether InterACT still provides substantial gains when applied to stronger camera-robust latent-action models. More importantly, the improvement in controlled retrieval does not transfer to CALVIN: R@5 remains at 2.73%, while real-video reconstruction performance slightly degrades. This leaves the effectiveness and necessity of the proposed intervention-based approach in real-world settings insufficiently established.
The experiments do not sufficiently disentangle the contribution of the intervention data itself from that of the more complex training framework. The full InterACT model jointly uses crossed quartets, a frozen reference, scale preservation, gradient routing, action/outcome supervision, and several auxiliary losses, but it lacks a direct comparison against a simple contrastive or alignment baseline trained on exactly the same intervention data. Moreover, the key pairing and routing ablations are conducted mainly on the action-bottleneck variant rather than on the full model. As a result, it remains unclear whether the observed gains primarily come from the proposed methodological design or simply from access to additional high-quality controlled supervision, which weakens the evidence for the algorithmic contribution and novelty of the paper.
Questions for authors
Does InterACT remain effective when applied to a stronger camera-robust baseline? The current experiments mainly build on ALAM, whose Camera-OOD performance shows substantial camera sensitivity, with . Could the authors clarify or provide additional evidence on whether InterACT still brings significant and complementary improvements when applied to latent-action models that have already acquired strong camera robustness through large-scale real-video training or explicit view-/camera-invariant objectives?
Does the invariance learned from synthetic interventions genuinely transfer to real videos? Controlled retrieval improves substantially, with R@5 increasing from 3.39% to 29.43%, whereas R@5 on CALVIN remains at 2.73%, and real-video reconstruction performance slightly degrades. How do the authors explain this clear discrepancy between the synthetic controlled setting and the real-video setting?
Do the gains primarily come from the intervention data itself or from the specific InterACT training design? The full InterACT model combines crossed pairing, a frozen reference, scale preservation, gradient routing, action/outcome supervision, and several auxiliary losses. Could the authors compare the full method against a simple contrastive or same-interaction alignment baseline trained on exactly the same intervention quartets?