acceptodds
Withdrawn from ICLR 2027

Efficient 3D Motion Generation from Visual Demonstrations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing 3D motion generators face a trade-off between convenient but coarse text-based control and precise but manually specified pose guidance. Moreover, diffusion-based methods further incur substantial iterative sampling costs. We propose FlowMimic, an efficient framework for generating complete 3D human motion from sparse, timestamped 2D pose observations extracted from videos or image sequences. FlowMimic consists of a Motion Latent Codec (MLC), a Time-Aware Pose Encoder (TAPE) for sparse spatiotemporal conditioning, and a Conditional Motion Flow (CMF) that performs Flow Matching directly in the learned motion latent space. The resulting flow supports single-step motion generation while retaining effective control from sparse pose observations. Experiments on AIST++ show that FlowMimic faithfully follows sparse pose observations while maintaining realistic motion quality, and substantially improves inference efficiency over iterative diffusion-based generation.

Primary area:
generative models
Keywords:
Human Motion Generation, Flow Matching

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fZCQOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes FlowMimic, a framework for generating 3D human motion from sparse 2D pose observations extracted from demonstration videos. The method consists of a motion latent codec for compressing motion sequences, a time-aware pose encoder for incorporating sparse pose observations and timestamps, and a conditional Flow Matching model for motion generation. By combining latent-space generation with Reflow, the method enables single-step sampling. Experiments on AIST++ show good motion fidelity and significantly improved inference efficiency compared with existing methods.

Critical strengths
  1. The motivation is clear. Using visual demonstrations allows more precise motion control than text descriptions.
  2. The experiments are comprehensive, and the method achieves good motion fidelity with faster inference.
Critical weaknesses
  1. The algorithmic novelty is limited. The proposed visual conditioning and time-aware pose encoding mainly rely on existing techniques, while the efficient generation is largely achieved through established latent Flow Matching and Reflow methods.
  2. The experiments are conducted only on AIST++. It is unclear whether the method generalizes to other types of human motion.
  3. The framework figure is rather rough and could be better presented.
Questions for authors

See the weaknesses.

Reviewer_EToNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FlowMimic, a framework for generating 3D human motion from sparse, timestamped 2D visual demonstrations using a motion latent codec, a time-aware pose encoder, and conditional flow matching. Experiments on AIST++ evaluate its motion generation quality and computational efficiency.

Critical strengths

The paper addresses a meaningful problem by enabling efficient 3D motion generation from sparse visual demonstrations. The combination of compact motion representations and flow matching offers a promising approach to reducing generation latency.

Critical weaknesses

Although the paper includes several representative baselines, most use different conditioning modalities, such as text prompts, manually specified poses, or 3D keyframes. The lack of a strong baseline using the same sparse, timestamped 2D pose observations makes it difficult to isolate the improvements attributable to the proposed architecture.

Questions for authors

The paper uses 7 sparse observation frames to guide the generation of a 196-frame 3D motion sequence. For the unobserved frames, to what extent does the model rely on visual conditioning information versus motion priors learned from the training data, and how can the contributions of these two sources be distinguished?

Reviewer_qutaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes FlowMimic, a framework that generates complete human motion from 2D pose extracted from videos or image sequences. The framework combines a Motion Latent Codec, a Time-Aware Pose Encoder, and a Conditional Motion Flow operating in the compressed motion space. The framework uses a reprojection objective to encourage consistency with the observed poses, and an additional reflow stage to enable sampling with one Heun step, corresponding to two solver-level function evaluations. Experiments conducted on AIST++ demonstrate improves condition coherence, reduces foot sliding and substantially lowers generation latency.

Critical strengths
  1. The proposed framework achieves low generation latency through compact motion representations and single-Heun-step sampling.

  2. The paper is well organized, making the overall pipeline easy to understand.

Critical weaknesses
  1. The ablation study has demonstrated the usefulness of pose memory but does not isolate the contribution of timestamp encoding. The paper does not evaluate whether modifying only the observation timestamps produces the intended timing changes.

  2. The compared methods receive different information (text for text-based methods, text + stick figure for StickMotion), making it difficult to distinguish the benefit of the proposed architecture from that of receiving target-specific pose observations.

Questions for authors
  1. How will the performance change if explicit timestamp encoding is removed?

  2. Could the baselines be compared with a same input of sparse 2D poses, timestamps, to distinguish the improvements?

  3. The experiments are mainly conducted on AIST++, a dataset of short fixed-length dance clips. How does the method perform on other motion categories or longer video sequences?

  4. Given the mentioned motivation that text cannot specify “precise spatiotemporal motion”, could the authors add an experiment that fixes the pose observations while changing only the timestamps, to test whether the generated motion can advance or delay specified movements and accelerate or decelerate transitions as intended?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.