- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper tackles the joint human–camera generation task using an asymmetric approach: human motion is generated first, followed by camera motion conditioned on the generated human motion.
To this end, the authors design an asymmetric human–camera latent space on which they train a flow-matching model. The human latent space is learned using an independent autoencoder, whereas for the camera autoencoder, the decoder is additionally conditioned on the human latent representation. Furthermore, the authors introduce data augmentation strategies that modify both the direction of the camera trajectory and its intensity, by changing the trajectory's orientation and scaling its overall magnitude. The resulting flow-matching model supports three tasks: camera generation conditioned on a given human motion, joint human–camera generation, and control over the intensity of camera translation.
Finally, the authors present quantitative evaluations and user studies covering autoencoder reconstruction, camera generation conditioned on given human motion, and joint human–camera generation. They also provide qualitative examples to illustrate the capabilities of their approach.
Critical strengths
-
The quantitative results and qualitative examples provided in the supplementary videos are convincing and support the effectiveness of the proposed approach.
-
The paper introduces a straightforward and intuitive cascaded approach to joint human–camera generation that, despite its naturalness, has been largely overlooked in prior work.
-
The proposed data augmentation strategies for camera direction (DPA) and intensity (IPA) represent valuable contributions to the field.
-
The addition of explicit camera motion intensity control is an interesting feature that enhances the flexibility and practical utility of the proposed method.
Critical weaknesses
-
The analysis of the autoencoder could be further expanded, given its central role in the proposed approach. The quantitative evaluation is limited to a comparison with the PulpMotion autoencoder, while additional ablation studies would help clarify the design choices and better understand its behavior.
- First, the impact of conditioning the camera decoder on the human latent representation remains unclear.
- Second, conditioning the encoder as well would seem to be a natural, and potentially more effective, alternative for learning a conditional latent space.
-
Equation 2 would benefit from a more detailed formulation that explicitly states the assumptions made regarding the independence between modalities.
Questions for authors
My main question concerns the autoencoder design.
-
Could the authors clarify the role and benefits of conditioning the decoder on the human latent representation?
-
Additionally, could they explain why conditioning the encoder was not considered, given that this could be a natural alternative for learning a conditional latent space?
4 comments
Rebuttal (2/2)
(W2) Independence assumptions in Eq. 2
We now state Eq. 2 in latent space, where the factorization is implemented:
p(z_H, z_C \mid T_H, T_C, a) = p_\phi(z_H \mid T_H)\ p_\theta(z_C \mid z_H, T_C, a), \tag{2}
where the human and camera generators and solve the flow ODEs with velocities and (Eq. 1) from Gaussian noise and , and denotes decoding, with the camera anchored to the decoded human root. Intensity enters only the camera generator and is not read by the decoder. The factorization assumes : the human is generated from its own instruction, and the camera text and intensity cannot change it. The camera depends on the human and is conditioned on the complete .
- In representation learning, is learned from human data alone, and the camera decoder reads (Eq. 3).
- Joint generation samples first and keeps it fixed during camera sampling.
- Given-human generation replaces it with the normalized encoding of the supplied motion and uses the same .
We revised Sec. 3.2 accordingly.
Dear Authors,
Thank you for your rebuttal and the additional experiments and clarifications.
The ablation studies on the autoencoder design and the clarification regarding Equation 2 address my concerns. I also appreciate the responses to the other reviewers, particularly the addition of PulpMotion's scores with auxiliary guidance.
Overall, I am satisfied with the rebuttal. I therefore maintain my original score and lean towards acceptance.
Dear Reviewer,
Thank you again for your thoughtful feedback and careful consideration of our rebuttal and additional experiments. We sincerely appreciate your recognition of our efforts to address the concerns and your continued support for our work.
Sincerely,
The Authors
Rebuttal (1/2)
We thank the reviewer for finding the results and videos convincing and the cascaded formulation and DPA/IPA valuable. The questions on the autoencoder target the core of the design; we add a camera-only decoder ablation and the suggested encoder variant.
(W1, Q1) Role and benefit of conditioning the camera decoder on the human latent
The revised paper adds this ablation in App. B.1 (Table S3).
Table R9. Where the human latent enters the camera autoencoder. Camera-only decoder: DC(zC). Conditional encoder: zC=EC(C,sg(zH)). AESOP: zC=EC(C) and DC(zC,sg(zH)).
(W1, Q2) Conditioning the camera encoder
We kept EC camera-only so that zC is a function of the camera trajectory, with human dependence entering through the decoder and the camera flow. To test the alternative, we trained the conditional encoder zC=EC(C,sg(zH)) with the same decoder, followed by its own camera flow; the human pathway, losses and training budget are unchanged.
This supports keeping EC camera-only. The revised paper adds these results (Sec. 4.3, Table S3).