acceptodds
Withdrawn from ICLR 2027

ActiveContact: Factorized Physical Hypotheses for Active Identification

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Different physical worlds can produce similar motion, so one push is often not enough to know what is happening. We present Active Contact GS, which treats physical understanding as an active identification problem. The model keeps several possible physical worlds, then chooses the next probe that is expected to separate them best. The discrete world contains four directional contact factors and one global friction factor. Local factors take the states *native*, *rounded*, or *compliant*; the friction factor is *isotropic*, *x-grain*, or *y-grain*. This gives combinations, together with continuous parameters for mass, friction, restitution, and inertia. Expected information gain (EIG) scores each unused probe by how much it is expected to reduce uncertainty. With six probes, Active reaches **87.5%** exact-world accuracy on ID tasks, **84.6%** on unseen combinations, and **79.9%** under geometry shift, compared with 37.0%, 29.4%, and 35.7% for Random. On composition-OOD, five Active probes already reach 77.3%, above Random 29.4% and Fixed 32.0% with six probes. We also test two real-data interfaces. Physics101 is an external passive-video benchmark. Separately, we collect 72 tabletop push videos of a cube, sphere, and box; 69 are tracked successfully. In leave-one-repeat-out evaluation, EIG identifies the held-out object with 77.8% accuracy after four probes, compared with 53.6% for Random and 22.2% for Fixed. These results show that factorized physical hypotheses and information-seeking probes are useful when the correct explanation is not known in advance.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Factorization, Gaussian world models, Contact geometry, Simulation, Active identification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_hxTDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ActiveContact, a framework for active physical identification through sequential interaction. It represents the latent physical world using four directional contact factors, one global friction factor, and continuous physical parameters, and maintains a posterior over these hypotheses from observed probe-response histories. At test time, candidate probes are selected using approximate expected information gain (EIG), favoring actions expected to maximally reduce uncertainty over the remaining physical hypotheses. Experiments on synthetic ID, composition-OOD, and geometry-OOD settings show improvements over random and fixed probing, together with limited real-data evaluations.

Critical strengths
  1. The factorized physical representation is well motivated and provides a clear benefit for compositional generalization.

Rather than treating the 243 possible physical worlds as unrelated classes, the proposed representation allows familiar local factors to be recombined into unseen joint configurations.

This is supported by the composition-OOD experiment, where the flat 243-way classifier fails on unseen combinations while the factorized representation retains meaningful accuracy.


  1. The active identification results are strong and consistent within the proposed benchmark.

EIG-based probing substantially outperforms random and fixed action selection across ID, composition-OOD, and geometry-OOD settings, suggesting that information-seeking interaction can identify latent physical properties considerably more efficiently.

Critical weaknesses
  1. The counterfactual response model underlying EIG is not sufficiently specified.

Sec. 3.4 states that counterfactual responses are “simulated,” whereas the appendix refers to a “learned rollout model” / “learned dynamics model.”

It remains unclear whether hypothetical observations are generated using the ground-truth simulator, a separately learned forward model, or simulation from inferred physical parameters.

This distinction is fundamental, since the quality and availability of the forward model directly determine the validity and practical applicability of the EIG policy.


  1. The empirical comparison is largely limited to internal baselines, despite closely related prior work on information-seeking physical exploration.

Random and Fixed baselines demonstrate that EIG is better than uninformed probing, but they do not establish an advantage over existing active physical-property identification methods.

For example, prior work]1] has already used pushing/pulling together with information gain to infer mass, friction, center of mass, shape, and other physical properties, including real-robot evaluations; other work[2] similarly selects exploratory manipulation actions using expected information gain.

The manuscript would benefit from a much clearer comparison against this literature, both conceptually and experimentally.

[1] Predictive Visuo-Tactile Interactive Perception Framework for Object Properties Inference (TRO2025) [2] Interactive Learning of Physical Object Properties Through Robot Manipulation and Database of Object Measurements (IROs2024)

Questions for authors
  1. How exactly are the counterfactual observations used to compute EIG generated?

Are they produced by the ground-truth simulator, a learned rollout/dynamics model, or by running the inferred physical parameters through the simulator? If a learned model is used, please specify its architecture and training objective, and evaluate how rollout prediction errors affect probe selection.


  1. How does ActiveContact differ from and improve upon existing active physical-identification approaches that already use information gain to select exploratory interactions?

In particular, can the authors compare against, or at least provide a detailed discussion of, recent methods using information-seeking pushing/pulling for physical-property inference? The current Random/Fixed comparisons make it difficult to assess how much of the improvement comes specifically from the proposed factorized physical hypothesis representation versus established EIG-based active exploration.

Reviewer_JptQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes ACTIVECONTACT, an active physical-system identification framework combining (i) a factorized representation of physical hypotheses and (ii) expected information gain (EIG) for selecting the next interaction. The discrete hypothesis consists of four directional contact factors plus one global friction factor, each with three states, yielding 3^5=243possible worlds, alongside continuous mass, friction, restitution, and inertia parameters. Rather than treating the 243 configurations as independent classes, the method predicts individual factors and recombines them. EIG then chooses from a bank of candidate pushes to reduce posterior uncertainty. The principal evidence comes from controlled simulation, supplemented by Physics101 transfer, captured-object geometry experiments, and a small real-contact experiment.

Critical strengths
  1. Representation through factorization and identification through EIG is really good idea.
  2. Active probing results are really strong in comparison to random
  3. There is a visibly considerable efforts towards reproducibility.
Critical weaknesses
  1. The 243-world setup is deliberately constructed from four local directional factors and one friction factor, all with three predefined states. How much of the observed compositional generalization comes from learning reusable physical structure, versus being given the correct factorization of the simulator a priori? A harder experiment would scramble or misspecify the factorization, or compare against other structured representations.
  2. Evidence is obtained in a simulator whose hypothesis space is explicitly engineered to match the proposed factorization, while the real-world experiment is small and does not use the complete learned synthetic pipeline.
Questions for authors

1.How does performance change when the assumed physical factorization is imperfect or does not align with the true simulator factors? 2. What happens when the counterfactual rollout model is systematically miscalibrated? Does Active ever become worse than Random under increasing model error? 3.How sensitive is action selection to the use of only eight posterior-world samples per candidate? 4. Can we establish evidence based on significant real-world experiments.

Reviewer_WnL6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ActiveContact investigates active physical interpretation by selecting the next probe action that best distinguishes among multiple physical interpretations when those interpretations can still account for existing observations. The model represents the physical world as four directional contact factors, one global friction factor, as well as mass, friction coefficient, recovery coefficient, and inertial scale. Each of the five discrete factors has three states, resulting in a total of 243 combinations. The Transformer predicts a posteriori based on geometric descriptions and historical action-trajectory pairs, then samples candidate worlds from the posterior to simulate counterfactual responses to each action, selecting the next probe action based on expected entropy reduction and energy penalties.

Critical strengths
  1. Combination variations and geometric variations are evaluated separately, which helps understand the model’s capability boundaries. Combination tests retain only complete factor tuples, ensuring that every local state appears during training; geometric tests, on the other hand, keep the set of factors constant while varying dimensions and aspect ratios. This design allows for the separate evaluation of combinatorial inference and geometric transfer.

  2. The benefits of active probing are clear and manifest in both recognition and prediction. Improvements in accuracy stem not only from discrete labels but also from testing outside the combinatorial distribution.

  3. The paper clearly distinguishes the roles of real-world data experiments. Physics101 is used for passive mass inference, geometric experiments still rely on simulated responses, and real-world contact experiments employ separately fitted Gaussian response models for offline object recognition.

Critical weaknesses
  1. The current comparisons are insufficient to determine the independent advantages of factorized representations over EIG. For active probing, the primary comparisons are with the random and preset fixed sequences. However, the four local contact factors correspond exactly to the four signed directions, and the action library contains different directions, intensities, and offsets. Reasonably covering these directions alone could significantly improve recognition, yet the current experiments do not sufficiently distinguish between them. Furthermore, for factorized representations, the flat classifier in Table 4 did not receive positive-class supervision for the held-out combinations; consequently, its accuracy outside the combination distribution was 0%, primarily indicating that unstructured closed-set classification is unsuitable for this task. This comparison is insufficient to demonstrate the advantages of the specific posterior design in this paper relative to other combinatorial inference methods.

  2. How physical assumptions generate distinguishable responses has not yet been sufficiently clearly defined or analyzed. Although Appendix A.2 provides the qualitative meanings of Native, Rounded, and Compliant, along with some scaling coefficients, it does not fully explain the tangential terms, the softening support rule, or how the four directional factors are triggered. These elements directly determine whether different physical worlds can be distinguished through actions and form the basis for assessing task difficulty and the implications of combinatorial generalization.

Questions for authors
  1. Regarding the improvement in active detection, how much is attributable to adaptive selection, and how much to directional coverage? How exactly is the fixed sequence determined, and does it provide balanced coverage across the four contact directions and different impulsions?

  2. Under nearly isotropic geometries, are different contact modes still distinguishable? For the geometric surrogates of the cube and sphere, do the native and rounded versions produce sufficiently different responses under the existing action library? Can you explain how much of the lower Exact accuracy is attributable to geometric transfer errors, and how much is due to the approximate equivalence of the assumptions themselves?

  3. Which model generates the counterfactual responses in the formal experiments?Are the predicted trajectories shown in Figure 7 for each training stage generated by the simulator using the physical structure and parameters estimated by the posterior network? Explain the correspondence between this visualization and the posterior training described in Section 3.3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.