acceptodds
Withdrawn from ICLR 2027

FlowEgoMotion: Scene-Conditioned Rectified Flow for Egocentric Full-Body Motion Estimation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Estimating full-body motion from a head-mounted egocentric device is severely underconstrained: visual–inertial SLAM gives reliable egomotion estimation, but most of the wearer's body is never directly visible, and prior diffusion-based approaches incur high inference cost while underusing scene context. We present FlowEgoMotion, a scene-conditioned rectified-flow model that combines the SLAM head trajectory with two complementary scene pathways – time-aligned local geometry fused into the egomotion stream, and global tokens encoding room-scale structure – to condition a Transformer that predicts a motion vector field, while global root motion is recovered analytically from the trajectory. On Nymeria, a controlled comparison with the same backbone shows that scene conditioning reduces MPJPE from to  mm, Wrist-PE from to  mm, and floor penetration from to  mm, and the gains carry over to ADT without fine-tuning. FlowEgoMotion also improves on the closest published scene-aware system, HMD, on the same benchmark, reducing MPJPE from to  mm, Wrist-PE from to  mm, and floor penetration from to  mm without using the RGB features that HMD relies on. With only two flow steps, it needs about  ms of GPU compute per second of generated motion and is more accurate than a matched 30-step diffusion model at lower cost.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Egocentric, Flow Matching. Full-Body Motion Estimation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ZxgYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes FlowEgoMotion, an egocentric full-body motion estimation method based on scene-conditioned Rectified Flow. The method integrates SLAM trajectories, local geometry, and global scene information to reconstruct human motion. Experiments on Nymeria and ADT demonstrate improved motion reconstruction accuracy and more plausible ground contact, while enabling efficient inference through a two-step Rectified Flow process.

Critical strengths
  1. Efficient inference. Compared with conventional diffusion-based methods, the proposed rectified-flow framework significantly reduces the number of sampling steps required for motion generation.
  2. Comprehensive experimental evaluation. Extensive quantitative experiments demonstrate the effectiveness of the proposed framework.
Critical weaknesses
  1. Concerns about reconstruction quality. In the demo videos, some reconstructed poses do not appear to match the input observations, raising concerns about reconstruction quality. The visualized scene point clouds lack clear geometric details. For example, in the pillow-hugging sequence, the pillow boundaries remain difficult to distinguish even when zoomed in, making it hard to evaluate human–object contact. Furthermore, the reconstructed motions appear to exhibit noticeable jitter and foot sliding. Although the authors emphasize accurate foot grounding, the visualizations do not clearly show foot–ground contact.
  2. More comprehensive demo videos covering a wider range of scenes are needed. The current 4D visualizations are insufficient to assess reconstruction quality, particularly temporal consistency, human–scene contact, foot penetration, and wrist localization accuracy.
  3. Figure 3 only compares the reconstructed human poses, without showing the baseline reconstructions in the corresponding 3D scenes for evaluating human–scene alignment.
  4. Although the paper emphasizes wrist localization and foot grounding, the qualitative results lack detailed visualizations of human–scene contact.
Questions for authors
  1. Given that HMD² and FlowEgoMotion use different test splits and skeleton conventions, why not provide a more direct comparison on identical test sequences? To what extent can the reported improvements be attributed to the proposed method rather than differences in evaluation protocols or scene representations?
  2. Does scene conditioning actually improve human–scene interactions? The paper highlights scenarios such as sitting on a sofa, descending stairs, bending toward a table, and navigating cluttered environments. However, the quantitative evaluation mainly relies on MPJPE, PA-MPJPE, Wrist-PE, and Floor Pen, while the qualitative results make it difficult to assess actual contact and interactions. Could the authors provide more detailed evaluations of human–scene contact consistency, such as sitting on furniture, avoiding obstacles, and reaching toward object surfaces?
Reviewer_vwJdOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors focus on the estimation of full-body pose from sparsely placed 6DoF trackers in the presence of 3D scene geometry that constrains the human pose. They propose a Rectified-Flow-based framework to solve this problem.

To incorporate scene information, the authors use both a local scene-conditioning latent representation and a global scene-context encoder to inform their rectified flow model via cross-attention.

When evaluated on relevant real-world datasets, the proposed approach compares favorably with several recent benchmarks.

Critical strengths

The authors claim to beat some recent baselines using an iterative generative modeling approach, namely Flow Matching. This approach substantially outperforms Diffusion, which is commonly used in the literature, in both performance and runtime.

Critical weaknesses
  1. The paper doesn't provide much comparison between its proposed scene conditioning mechanism and other works in the literature. One such work is EnvPoser [1], which also uses scene-geometry context conditioning.

  2. Other recent baselines seem to be missing, such as RPM [2]. These baselines also address the problem of tracker dropout, where one or more 6DoF trackers stop sending signals to the model. Robustness to dropouts, especially of the hand controllers, is crucial for real-world deployment of any pose-estimation system.

[1] Songpengcheng Xia et al. EnvPoser: Environment-aware Realistic Human Motion Estimation from Sparse Observations with Uncertainty Modeling. In CVPR 2025.

[2] German Barquero et al. From Sparse Signal to Smooth Motion: Real-Time Motion Generation with Rolling Prediction Models. In CVPR 2025.

Questions for authors
  1. Since the authors claim to use rectified flow, can they substantiate how many flow-rectification iterations were used during training?

  2. Could the authors provide jitter comparisons between multi-step diffusion and their proposed rectified flow method?

  3. See weaknesses.

Reviewer_m4uuOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript proposes FlowEgoMotion for full-body motion estimation from head-mounted SLAM trajectories and reconstructed 3D scene geometry without using RGB images. The method combines local scene features from LitePT and global scene features from SceneScript to condition a rectified-flow Transformer, while recovering global root motion analytically from SLAM. The model is trained on Nymeria and evaluated on Nymeria and ADT.

Critical strengths

S1. The paper addresses an important problem of full-body motion estimation from sparse egocentric observations using SLAM trajectories and scene geometry.

Critical weaknesses

W1. Novelty of the proposed scene conditioning is not clear: The manuscript proposes combining local and global scene features for motion estimation. SceneMI (ICCV 2025) and SceMoS (CVPR 2026) have explored combining local and global scene representations for human motion generation, but neither is discussed in the paper. While these methods address different tasks, the idea of combining local geometry and global scene context is already established, so it would be important to discuss these works and establish what is novel about the proposed approach. Furthermore, Tab. 2 shows the benefit of combining both features but does not establish what is unique about the proposed fusion mechanism.

W2. Improvement over HMD2 might be overclaimed: The manuscript claims to outperform HMD2 on Nymeria. However, the methods use different test sequences (224 vs. 253) and skeleton conventions (Xsens vs. SMPL-H), as acknowledged in Appendix. Additionally, HMD2 evaluates on unseen subjects and environments, while the proposed method specifies only a subject-level split. Although I acknowledge that HMD2's test split is not publicly available, a comparison under matched evaluation settings would be more convincing.

W3. Limited evaluation of scene interactions: The manuscript motivates scene conditioning through interactions such as sitting and reaching. However, the physical-plausibility evaluation (Tab. 6) focuses only on floor penetration, floating feet, and foot skating, without reporting full-body or hand-scene penetration or contact accuracy.

Questions for authors

Q1. Choice of scene representation: SceneMI (ICCV 2025) and SceMoS (CVPR 2026) have explored alternative local and global scene representations for human motion generation (see W1). It would be important to know the motivation of using LitePT and SceneScript over these representations and also to compare against simpler scene representations or fusion mechanisms.

Q2. Evaluation protocol: HMD2 evaluates on unseen subjects and environments, whereas the proposed method specifies only a subject-level split (see W2). It would be interesting to see the performance of the proposed method on unseen environments. Additionally, it would be helpful to clarify how the different skeleton conventions are handled when comparing the reported MPJPE values in Tab. 1.

Q3. Evaluation of scene interactions: The physical-plausibility evaluation (Tab. 6) focuses only on floor penetration, floating feet, and foot skating (see W3). It would be informative to evaluate full-body scene penetration on Nymeria and human-scene contact accuracy on RICH, which provides 3D scene scans and contact annotations.

Q4. Qualitative comparison: Figs. 3 and 6 compare the full model against EgoEgo and EgoAllo, which use different input modalities. Qualitative comparisons between the motion-only, local-only, and full models on the same sequences would help understand the contribution of scene conditioning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.