acceptodds
Withdrawn from ICLR 2027

BUMP: Body-driven Unified Multi-object Physics for HOI synthesis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generating human-object interactions (HOIs) with multiple objects is challenging because the resulting motions arise from different underlying causes. Human motion reflects semantic intent, intentionally manipulated objects must remain coordinated with the human, while many surrounding objects move primarily as physical consequences of contact and collision. We present BUMP, a text-driven framework for multi-object HOI generation based on a simple principle: learn intentional motion and simulate physical consequences. Our method first generates full-body human motion from the text and scene context, then predicts the trajectories of a small number of intentionally manipulated driver objects conditioned on the human motion. Rather than explicitly generating every moving object, we obtain the motions of physically induced responder objects through simulation, allowing the framework to handle varying numbers and configurations of surrounding objects without increasing the learned trajectory output space. To address the scarcity of paired full-body multi-object HOI data, we further combine heterogeneous supervision from human-only motion, full-body HOI, and hand-centric manipulation data augmented with plausible body motion. Finally, we introduce outcome-conditioned interaction editing, which infers a minimal modification of the causal driver trajectory from a desired responder behavior without requiring an explicit target driver trajectory. Extensive experiments demonstrate that BUMP outperforms baseline methods on both single and multiple object interaction generation. We further showcase diverse scenarios where inter-object relationships are crucial (\eg, stacking box), highlighting the advantage of our unified model formulation.

Primary area:
generative models
Keywords:
Human-Object Interaction, Multi-Object Interaction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8wo6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents BUMP, a text-driven multi-object HOI synthesis framework. It sequentially generates human and manipulated-object motions, while using physics simulation for the responses of other objects. The paper also introduces a heterogeneous data integration pipeline and outcome-conditioned interaction editing. Experiments evaluate generation quality and contact consistency on existing HOI benchmarks, with ablations on data supervision and qualitative editing demonstrations.

Critical strengths
  1. Heterogeneous data integration. The integration of human-only, full-body HOI, and hand-centric manipulation datasets is a useful contribution, especially given the scarcity of full-body multi-object interaction data. The fitting and retargeting pipeline provides a practical way to leverage these heterogeneous sources, and the ablations demonstrate some benefits.

  2. Outcome-conditioned interaction editing. I find this component more interesting than the main generation framework. Instead of directly specifying object trajectories, the method allows users to specify a desired physical outcome and adjusts the manipulated object's trajectory accordingly. The local correction and simulation-based verification offer a useful form of interaction controllability.

Critical weaknesses

Limited novelty of the main generation framework. My main concern is that the central design largely combines ideas already explored in HOI generation.

First, decomposing human and object motion generation is not new. For example, OMOMO adopts an object-to-hand-to-human generation pipeline, demonstrating that human–object coordination can be handled through sequential generation. Other methods, such as HOI-Diff, HIMO, and HOI-Dyn, have explored joint or coupled human–object generation. Therefore, BUMP's human-first, object-second formulation does not seem to offer a particularly new modeling perspective.

More importantly, InterDreamer has already explored separating human motion semantics from object dynamics, using a learned world model to predict physical object responses and iteratively refine the interaction. This is conceptually close to BUMP's principle of generating intentional motion while obtaining physical consequences separately. BUMP replaces learned dynamics with a conventional physics simulator, but its main pipeline lacks the iterative physical feedback present in InterDreamer.

The driver–responder decomposition is reasonable, but object roles are determined heuristically during training and provided at inference. Moreover, existing HOI generators could also be used to generate human and manipulated-object trajectories, with subsequent object responses obtained through the same physics simulator.

Overall, while the integration is practical, I find the methodological novelty of the main framework rather limited.

Questions for authors
  1. How does BUMP fundamentally differ from existing approaches such as OMOMO and InterDreamer? In particular, InterDreamer already separates semantic human motion from physical object dynamics, with iterative interaction refinement. Beyond replacing learned dynamics with MuJoCo and introducing the driver–responder distinction, what is the key methodological advance?

  2. Would existing sequential or joint HOI generators combined with the same physics simulator achieve comparable results? A controlled comparison using the same simulation setup would be particularly important for assessing the value of the proposed factorization.

Reviewer_wpUvOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This submission introduces BUMP, a method for multi-object physics-based human-object interaction (HOI) synthesis. The framework categorizes objects into "drivers", those manipulated by a human, and "responders", objects that react passively. Unlike previous approaches that explicitly model the trajectories of all objects, this method employs a diffusion model to generate the motions of the human and the driver objects, while using the MuJoCo physics simulator to compute the motions of the responder objects. Furthermore, the submission proposes "outcome-conditioned interaction editing," which enables the derivation of local adjustments to driver trajectories based on desired responder behaviors. Experiments conducted on the InterAct and HIMO benchmarks demonstrate the effectiveness of the proposed method in ensuring contact consistency.

Critical strengths
  • This submission proposes a new paradigm that differs from previous trajectory generation methods. It categorizes objects into "drivers" and "responders," generating actions only for humans and drivers, while the behavior of responders is determined by the MuJoCo physics engine. This ensures that contacts and chain reactions between objects inherently satisfy rigid-body physical constraints.

  • The author proposes "outcome-conditioned interaction editing", a method that derives local modifications to a "driver" object's trajectory based on desired behaviors from a "responder" object, with these modifications subsequently validated through simulations in MuJoCo. By using physical outcomes as the target for editing, this approach introduces a novel paradigm for interaction editing.

  • The results shown in Table 1 are good, with the Contact F1 metric outperforming the baselines (CHOIS, MDM, and Dex2HOI). Figure 1 illustrates physically plausible multi-object interaction trajectories, providing a compelling qualitative demonstration.

Critical weaknesses
  • The core claim of the submission is not validated by the experiments. In Section 4, the authors employ standard benchmarks, specifically the InterAct dataset (single-object subset), HIMO (two- and three-object subsets), and the HUMOTO ablation study. In these datasets, all objects function as "drivers" (objects being manipulated), with no "responders" present. Consequently, the paper’s stated capability to model multi-object interactions involving a "responder" (computed by Mujoco) remains unverified and has not been evaluated.

  • The "outcome-conditioned interaction editing" proposed by the authors has not been quantitatively evaluated. The method operates in an open-loop fashion, refining the driver's trajectory through iterative adjustments and re-simulations. However, the authors fail to establish a closed-loop feedback mechanism between the generated trajectories and the trajectories computed by the MuJoCo simulator. This raises questions regarding the efficiency and reliability of the proposed approach. The submission also does not discuss the number of iterations or the time required for adjustments, nor does it address the existence of failure cases (for instance, the success rate in ultimately generating a trajectory that meets expectations).

  • The improvements of the proposed method over the strongest baselines in Table 1 are marginal. For instance, on the InterAct dataset, FID is worse than MDM without explanation; the Contact F1 score is 28.11 for BUMP versus 26.09 for CHOIS; on the HIMO two-object task, it is 44.95 for BUMP versus 43.03 for Dex2HOI. All results reported by the authors in Table 1 seem based on single runs, lacking statistical results from multiple experiments.

  • The submission requires proofreading. There are several indications that it may not have been reviewed by the authors. For instance, a LaTeX template phrase "You may include other additional sections here." remains at the beginning of the appendix.

Questions for authors

My questions mainly concern the lack of verification for the submission’s core claims. Furthermore, there is an absence of ablation studies regarding the use of a generative approach for "responder" objects, and the ablation of the framework’s core mechanisms remains unclear.

Reviewer_gYUwOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents BUMP, a text-driven framework for multi-object human-object interaction (HOI) synthesis, which decomposes an interaction according to the causal roles of the involved entities. Specifically, the framework first generates full-body human motion, then predicts the trajectories of intentionally manipulated driver objects conditioned on the generated human motion and hand-object contacts. The trajectories of the remaining responder objects are obtained through rigid-body physics simulation rather than learned generation. To alleviate the scarcity of full-body multi-object interaction data, the authors further construct a heterogeneous training corpus. Finally, the paper introduces an outcome-conditioned interaction editing procedure that modifies local driver trajectories to achieve desired responder behaviors through physics-based replay. Experiments on InterAct and HIMO demonstrate improvements in several generation and interaction metrics, particularly in multi-object settings.

Critical strengths
  • The proposed framework is well motivated and thoughtfully designed, effectively decoupling interactions in multi-object HOI generation. Compared with end-to-end approaches that jointly generate all object trajectories, this framework seems reasonable and I think it offers greater potential for scalability.
  • State-of-the-art results on multi-object HOI benchmarks.
  • The heterogeneous full-body interaction data construction strategy is insightful and helpful for the community.
Critical weaknesses
  • It is better to compare with more SOTA baseline quantitatively [1] in InterAct and also evaluate the method on GRAB dataset, and compare with SOTA methods [2].
  • It is better to include ablations of the outcome-conditioned interaction editing module and the responder physics.

[1]. Wang, Ziyin, et al. "Unleashing guidance without classifiers for human-object interaction animation." International Conference on Learning Representations. Vol. 2026. 2026. [2] Pi, Huaijin, et al. "Coda: Coordinated diffusion noise optimization for whole-body manipulation of articulated objects." Advances in Neural Information Processing Systems 38 (2026): 31758-31780.

Questions for authors
  • What is the method performance on the articulated object setting? I think this method should be promising for articulated object.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.