acceptodds
Under review as a conference paper at ICLR 2027

AESOP: Asymmetric Human-Camera Generation with Translation-Intensity Control

Abstract

Human motion defines an action, while a camera trajectory determines how it is presented. Camera generation for a given human motion and joint human–camera generation are usually treated as separate tasks, although both share an asymmetric dependency: human motion can be generated independently, whereas the camera responds to the realized action. We introduce AESOP, a unified framework with an independent human pathway and a shared human-conditioned camera generator. Its asymmetric architecture serves both tasks while preserving the human output during camera generation. Although human context anchors the shot to the action and camera text describes its movement, translation intensity remains underspecified. We therefore construct trajectory pairs that differ in camera translation magnitude while sharing human motion and camera text, then use these pairs to learn an explicit intensity condition. Experiments on the PulpMotion dataset demonstrate strong camera distributional and framing quality in both tasks and effective control over camera translation intensity.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
human-camera generation, asymmetric generative modeling, camera trajectory generation, human motion generation, translation-intensity control, flow matching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_SbYwOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper tackles the joint human–camera generation task using an asymmetric approach: human motion is generated first, followed by camera motion conditioned on the generated human motion.

To this end, the authors design an asymmetric human–camera latent space on which they train a flow-matching model. The human latent space is learned using an independent autoencoder, whereas for the camera autoencoder, the decoder is additionally conditioned on the human latent representation. Furthermore, the authors introduce data augmentation strategies that modify both the direction of the camera trajectory and its intensity, by changing the trajectory's orientation and scaling its overall magnitude. The resulting flow-matching model supports three tasks: camera generation conditioned on a given human motion, joint human–camera generation, and control over the intensity of camera translation.

Finally, the authors present quantitative evaluations and user studies covering autoencoder reconstruction, camera generation conditioned on given human motion, and joint human–camera generation. They also provide qualitative examples to illustrate the capabilities of their approach.

Critical strengths
  • The quantitative results and qualitative examples provided in the supplementary videos are convincing and support the effectiveness of the proposed approach.

  • The paper introduces a straightforward and intuitive cascaded approach to joint human–camera generation that, despite its naturalness, has been largely overlooked in prior work.

  • The proposed data augmentation strategies for camera direction (DPA) and intensity (IPA) represent valuable contributions to the field.

  • The addition of explicit camera motion intensity control is an interesting feature that enhances the flexibility and practical utility of the proposed method.

Critical weaknesses
  • The analysis of the autoencoder could be further expanded, given its central role in the proposed approach. The quantitative evaluation is limited to a comparison with the PulpMotion autoencoder, while additional ablation studies would help clarify the design choices and better understand its behavior.

    • First, the impact of conditioning the camera decoder on the human latent representation remains unclear.
    • Second, conditioning the encoder as well would seem to be a natural, and potentially more effective, alternative for learning a conditional latent space.
  • Equation 2 would benefit from a more detailed formulation that explicitly states the assumptions made regarding the independence between modalities.

Questions for authors

My main question concerns the autoencoder design.

  • Could the authors clarify the role and benefits of conditioning the decoder on the human latent representation?

  • Additionally, could they explain why conditioning the encoder was not considered, given that this could be a natural alternative for learning a conditional latent space?

4 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We thank the reviewer for finding the results and videos convincing and the cascaded formulation and DPA/IPA valuable. The questions on the autoencoder target the core of the design; we add a camera-only decoder ablation and the suggested encoder variant.

(W1, Q1) Role and benefit of conditioning the camera decoder on the human latent

  • Role. The camera is defined relative to the human: its features are human-relative and decoding is anchored to the human root. Reading the frozen human latent lets the decoder resolve camera geometry against the realized motion, so zCz_C only needs to encode the camera information that the human motion does not already explain.
  • Benefit. The decoded camera is always resolved against the human that is actually shown, whether it is supplied (given-human) or generated (joint). Because the decoder reads sg(zH)\mathrm{sg}(z_H), this coupling never changes the human output, so one camera model serves both tasks without affecting the human.
  • Reconstruction (Table R9). Against a camera-only decoder DC(zC)D_C(z_C) with the same features and root anchor, reading zHz_H improves camera ADE from 0.034 to 0.029 m and rotation error from 0.52° to 0.41°.
  • Generation (Table R9). In both tasks, reading zHz_H improves the camera distribution (FDC) and framing (r-FPD, Out), e.g. FDC 8.14 → 6.99 (given-human) and 13.26 → 11.39 (joint).

The revised paper adds this ablation in App. B.1 (Table S3).

Table R9. Where the human latent enters the camera autoencoder. Camera-only decoder: DC(zC)D_C(z_C). Conditional encoder: zC=EC(C,sg(zH))z_C = E_C(C, \mathrm{sg}(z_H)). AESOP: zC=EC(C)z_C = E_C(C) and DC(zC,sg(zH))D_C(z_C, \mathrm{sg}(z_H)).

camera distribution & alignmentcamera framing & geometry
TaskVariantFDC ↓CLaTr ↑F1 ↑RecallC\mathrm{Recall_{\mathit{C}}} ↑r-FPD ↓Out ↓ADE ↓Rot. ↓
ReconstructionCamera-only decoder0.09±0.0470.06±0.18‾\underline{70.06\pm 0.18}0.942±0.0040.999±0.00050.088±0.023.04±0.290.034±0.01‾\underline{0.034\pm 0.01}0.52±0.13‾\underline{0.52\pm 0.13}
Conditional encoder0.15±0.0469.90±0.140.937±0.0010.999±0.00010.091±0.013.01±0.24‾\underline{3.01\pm 0.24}0.035±0.00070.60±0.005
AESOP0.11±0.10‾\underline{0.11\pm 0.10}70.10±0.130.941±0.001‾\underline{0.941\pm 0.001}0.999±0.00040.089±0.01‾\underline{0.089\pm 0.01}2.93±0.280.029±0.00060.41±0.01
Given-humanCamera-only decoder8.14±0.44‾\underline{8.14\pm 0.44}69.24±0.26‾\underline{69.24\pm 0.26}0.905±0.0040.858±0.0011.373±0.1314.14±0.701.712±0.0136.21±0.37
Conditional encoder37.33±4.5359.43±1.340.756±0.020‾\underline{0.756\pm 0.020}0.894±0.0041.163±0.05‾\underline{1.163\pm 0.05}12.45±0.401.404±0.00229.27±0.14
AESOP (no aug.)6.99±0.2069.60±0.330.905±0.0060.866±0.01‾\underline{0.866\pm 0.01}1.160±0.0712.56±0.43‾\underline{12.56\pm 0.43}1.577±0.01‾\underline{1.577\pm 0.01}33.00±0.46‾\underline{33.00\pm 0.46}
JointCamera-only decoder13.26±0.44‾\underline{13.26\pm 0.44}68.92±0.11‾\underline{68.92\pm 0.11}0.889±0.0040.809±0.010.749±0.05‾\underline{0.749\pm 0.05}10.04±0.28‾\underline{10.04\pm 0.28}–72.16±0.87
Conditional encoder30.61±4.9859.06±1.790.741±0.0240.869±0.010.790±0.0610.23±0.53–72.02±0.67‾\underline{72.02\pm 0.67}
AESOP (no aug.)11.39±0.6768.94±0.340.880±0.002‾\underline{0.880\pm 0.002}0.826±0.01‾\underline{0.826\pm 0.01}0.665±0.039.13±0.24–71.56±0.53

(W1, Q2) Conditioning the camera encoder

We kept ECE_C camera-only so that zCz_C is a function of the camera trajectory, with human dependence entering through the decoder and the camera flow. To test the alternative, we trained the conditional encoder zC=EC(C,sg(zH))z_C = E_C(C, \mathrm{sg}(z_H)) with the same decoder, followed by its own camera flow; the human pathway, losses and training budget are unchanged.

  • Reconstruction gets worse (Table R9) on every camera metric except RecallC\mathrm{Recall_{\mathit{C}}}, where both reach 0.999, e.g. FDC 0.11 → 0.15 and ADE 0.029 → 0.035 m.
  • Generation degrades sharply (Table R9). FDC, CLaTr and F1 worsen in both tasks, e.g. FDC 6.99 → 37.33 (given-human) and 11.39 → 30.61 (joint).
  • The cause is the latent space, not decoding. Decoding the conditional encoder's ground-truth latents through the same generation path gives FDC of at most 0.19. When ECE_C reads zHz_H, zCz_C is no longer a function of the camera alone and becomes harder to model.

This supports keeping ECE_C camera-only. The revised paper adds these results (Sec. 4.3, Table S3).

AuthorsOpenReview ↗

Rebuttal (2/2)

(W2) Independence assumptions in Eq. 2

We now state Eq. 2 in latent space, where the factorization is implemented:

p(z_H, z_C \mid T_H, T_C, a) = p_\phi(z_H \mid T_H)\ p_\theta(z_C \mid z_H, T_C, a), \tag{2}

zH=Gϕ(ϵH;TH),zC=Gθ(ϵC;zH,TC,a),z_H = G_\phi(\epsilon_H; T_H), \quad z_C = G_\theta(\epsilon_C; z_H, T_C, a),

H^=DˉH(zH),C^=DˉC(zC,zH).\hat H = \bar D_H(z_H), \quad \hat C = \bar D_C(z_C, z_H).

where the human and camera generators GϕG_\phi and GθG_\theta solve the flow ODEs with velocities vϕv_\phi and vθv_\theta (Eq. 1) from Gaussian noise ϵH\epsilon_H and ϵC\epsilon_C, and Dˉ\bar D denotes decoding, with the camera anchored to the decoded human root. Intensity aa enters only the camera generator and is not read by the decoder. The factorization assumes zH⊥(TC,a)∣THz_H \perp (T_C, a) \mid T_H: the human is generated from its own instruction, and the camera text and intensity cannot change it. The camera depends on the human and is conditioned on the complete zHz_H.

  • In representation learning, zH=EH(H)z_H = E_H(H) is learned from human data alone, and the camera decoder reads sg(zH)\mathrm{sg}(z_H) (Eq. 3).
  • Joint generation samples zHz_H first and keeps it fixed during camera sampling.
  • Given-human generation replaces it with the normalized encoding of the supplied motion and uses the same pθp_\theta.

We revised Sec. 3.2 accordingly.

Dear Authors,

Thank you for your rebuttal and the additional experiments and clarifications.

The ablation studies on the autoencoder design and the clarification regarding Equation 2 address my concerns. I also appreciate the responses to the other reviewers, particularly the addition of PulpMotion's scores with auxiliary guidance.

Overall, I am satisfied with the rebuttal. I therefore maintain my original score and lean towards acceptance.

Dear Reviewer,

Thank you again for your thoughtful feedback and careful consideration of our rebuttal and additional experiments. We sincerely appreciate your recognition of our efforts to address the concerns and your continued support for our work.

Sincerely,
The Authors

Reviewer_LAFZOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AESOP, a unified framework for motion-conditioned camera generation and joint human-camera generation. The authors introduce an asymmetric architecture that independently generates human motion and subsequently generates camera trajectories conditioned on the human motion. They also propose direction-paired augmentation (DPA) and intensity-paired augmentation (IPA) to provide explicit control over camera movement direction and translation intensity. Experiments on the PulpMotion dataset demonstrate improved camera distributional quality, human–camera framing, and translation-intensity controllability compared to existing methods.

Critical strengths
  • I think the asymmetric human–camera generation framework is a simple and practical approach. By independently generating human motion and conditioning camera generation on the resulting motion, the framework supports both given-human and joint human–camera generation while preserving the human motion.

  • The proposed intensity-paired augmentation is an interesting approach to learning camera translation-intensity control. Constructing paired camera trajectories with different translation magnitudes provides a practical way to introduce continuous camera control without requiring additional annotations.

  • The authors provide extensive experimental comparisons and ablation studies. The results demonstrate promising improvements in camera generation and framing quality. The quantitative intensity-control analysis and user study further support the effectiveness of the proposed framework.

Critical weaknesses
  • The comparison with existing joint human–camera generation methods could be more carefully justified. The authors compare AESOP with PulpMotion DiT and MAR, but Appendix Table S5 indicates that the auxiliary framing guidance is disabled for both baselines. Since auxiliary framing guidance is an important component of PulpMotion, this setting may underestimate its framing performance. Therefore, the large improvements reported in Table 1, particularly in framing-related metrics, should be verified against PulpMotion using its recommended guidance settings.

  • The contribution of the asymmetric architecture is not sufficiently isolated. The proposed method factorizes joint generation into independent human motion generation followed by human-conditioned camera generation. While this is a reasonable design, a straightforward two-stage pipeline could also generate human motion first and then apply an existing conditional camera generator. The paper does not compare against such a baseline, making it difficult to determine whether the improvements come from the proposed asymmetric representation or from using stronger human and camera generation components. Although the authors compare symmetric and asymmetric autoencoders, the contribution of the asymmetric representation to the final generation performance is not directly isolated.

Questions for authors
  • Could the authors evaluate PulpMotion DiT and MAR using their original auxiliary framing guidance settings? This would help establish whether the reported improvements remain consistent under a stronger baseline configuration.

  • Could the authors compare AESOP with a two-stage baseline that first generates human motion and then applies an existing human-conditioned camera generator, such as DIRECTOR-C? This would help clarify the contribution of the proposed asymmetric architecture.

4 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We thank the reviewer for recognizing the simplicity and practicality of the asymmetric framework, the intensity-paired augmentation and the extensive evaluation. The two suggested comparisons are valuable for locating where the gains come from, and we added both.

(W1, Q1) PulpMotion with its auxiliary framing guidance

The submitted Table 1 followed PulpMotion's released default sampler, in which the auxiliary projection weight is gz=0g_z = 0. Table R6 turns the guidance on at the weights of PulpMotion's main results (gz=0.25g_z = 0.25 for DiT, 0.50.5 for MAR). The revised paper reports PulpMotion both with and without this guidance in Table 1 (Sec. 4.2).

Table R6. Joint generation with and without PulpMotion's auxiliary framing guidance.

camera distribution & alignmentcamera framing & geometryhuman distribution & alignment
MethodAux. guidanceFDC ↓CLaTr ↑F1 ↑r-FPD ↓Out ↓FD$_{\mathrm{TMR}}$ ↓TMR ↑
PulpMotion DiTgzg_z = 083.80±3.30‾\underline{83.80\pm 3.30}44.92±2.360.592±0.0277.772±0.7335.39±1.28366.21±21.5224.20±0.97‾\underline{24.20\pm 0.97}
gzg_z = 0.2584.24±1.7246.87±1.98‾\underline{46.87\pm 1.98}0.623±0.019‾\underline{0.623\pm 0.019}5.851±0.82‾\underline{5.851\pm 0.82}26.29±2.14‾\underline{26.29\pm 2.14}428.59±15.5524.39±0.71
PulpMotion MARgzg_z = 0131.55±9.1438.64±2.420.514±0.0338.169±0.6539.08±2.14319.98±7.38‾\underline{319.98\pm 7.38}21.57±0.87
gzg_z = 0.5121.86±1.8841.43±0.290.547±0.0096.637±1.0532.96±4.01334.56±10.4823.45±1.31
AESOP–10.18±0.5069.81±0.340.891±0.0060.581±0.028.47±0.04100.98±1.9120.46±0.34
  • Guidance narrows the framing gap, but AESOP still leads by a wide margin overall. Out improves from 35.39% to 26.29% (DiT) and from 39.08% to 32.96% (MAR) at a cost in human fidelity (FDTMR\mathrm{FD_{TMR}} 366.21 → 428.59 and 319.98 → 334.56). Without guidance, AESOP reaches 8.47% Out and 100.98 FDTMR\mathrm{FD_{TMR}}, and its FDC (10.18 vs. 84.24 / 121.86) and CLaTr (69.81 vs. 46.87 / 41.43) remain far better.
AuthorsOpenReview ↗

Rebuttal (2/2)

(W2, Q2) Two-stage baseline and the contribution of the asymmetric representation

AESOP is itself a two-stage cascade: it generates the complete human first and then the camera. We therefore test (i) whether cascading alone explains the gains and (ii) what the representation contributes.

(i) Cascade with DIRECTOR-C. We pass the same AESOP-generated human motions from Table 1 to DIRECTOR-C and measure framing against that same motion. The human motion is therefore identical across rows.

Table R7. Camera generators given the same AESOP-generated human motions.

camera distribution & alignmentcamera framing & geometry
Camera generatorFDC ↓CLaTr ↑F1 ↑r-FPD ↓Out ↓
DIRECTOR-C40.85±4.1752.08±1.390.680±0.0223.243±0.1122.54±0.24
AESOP (no aug.)11.39±0.67‾\underline{11.39\pm 0.67}68.94±0.34‾\underline{68.94\pm 0.34}0.880±0.002‾\underline{0.880\pm 0.002}0.665±0.03‾\underline{0.665\pm 0.03}9.13±0.24‾\underline{9.13\pm 0.24}
AESOP10.18±0.5069.81±0.340.891±0.0060.581±0.028.47±0.04
  • Same human, much better camera. Replacing DIRECTOR-C with AESOP's camera generator improves camera distribution and alignment (FDC 40.85 → 10.18, CLaTr 52.08 → 69.81) and framing (r-FPD 3.243 → 0.581, Out 22.54% → 8.47%). The margin holds without augmentation (FDC 11.39, Out 9.13%).
  • The gain lies in the camera stage, not in cascading. Both pipelines generate the human first and differ only in the camera generator. Part (ii) identifies the design choices of the camera stage that produce it.

(ii) The representation in final generation. Using AESOP's own components, we cross the representation (Symmetric: each encoder reads both human and camera, as in PulpMotion; Asymmetric: the human is encoded alone) with the generation order (Symmetric: both latents are denoised together; Sequential: the camera is generated after the complete human). Everything else is identical, including model size, data and training budget.

Table R8. Representation × generation, joint generation.

camera distribution & alignmentcamera framing & geometryhuman distribution & alignment
ModelRepresentationGenerationFDC ↓CLaTr ↑F1 ↑RecallC\mathrm{Recall_{\mathit{C}}} ↑r-FPD ↓Out ↓Rot. ↓FDTMR\mathrm{FD_{TMR}} ↓TMR ↑
AblationSymmetricSymmetric13.25±0.2966.46±0.220.860±0.0030.852±0.01‾\underline{0.852\pm 0.01}1.305±0.0813.78±0.6775.59±0.41117.22±5.5019.19±0.27
AblationSymmetricSequential17.26±1.1163.77±0.420.813±0.0030.871±0.010.784±0.03‾\underline{0.784\pm 0.03}9.84±0.46‾\underline{9.84\pm 0.46}72.19±0.59‾\underline{72.19\pm 0.59}102.92±7.52‾\underline{102.92\pm 7.52}19.72±0.43
AblationAsymmetricSymmetric8.43±1.2570.19±0.930.907±0.0090.838±0.021.555±0.0315.70±0.3275.61±0.83110.13±2.4720.01±0.52‾\underline{20.01\pm 0.52}
AESOP (no aug.)AsymmetricSequential11.39±0.67‾\underline{11.39\pm 0.67}68.94±0.34‾\underline{68.94\pm 0.34}0.880±0.002‾\underline{0.880\pm 0.002}0.826±0.010.665±0.039.13±0.2471.56±0.53100.98±1.9120.46±0.34
  • The asymmetric representation substantially improves FDC, CLaTr and F1 (Table R8). Under the same symmetric generation, replacing the symmetric representation with the asymmetric one improves FDC from 13.25 to 8.43.
  • AESOP's sequential generation further substantially improves framing (Table R8). Combining the asymmetric representation with sequential generation improves Out from 15.70% to 9.13%.

The revised paper adds these results in Sec. 4.2 (Table S7) and Sec. 4.3 (Table 3).

Thank you for the detailed responses and additional experiments. The guided PulpMotion comparisons and comprehensive ablations address my concerns and strengthen the empirical support for the proposed framework. I maintain my current rating and lean toward acceptance.

Dear Reviewer LAFZ,

We sincerely appreciate your time and effort in reviewing our paper. Thank you again for your valuable feedback and positive recognition of our work.

Sincerely,
The Authors

Reviewer_VP4wOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes AESOP, a unified framework for given-human camera generation and joint human-camera generation. Its asymmetric architecture generates human motion independently and then conditions a shared camera generator on the completed human motion. The paper further introduces direction-paired augmentation (DPA) and intensity-paired augmentation (IPA) for controlling camera direction and translation magnitude. Experiments on PulpMotion show substantial improvements in camera distribution, framing, human-motion quality, and user preference.

Critical strengths
  1. Clear and useful task formulation. The asymmetric dependency preserves the supplied or generated human motion while allowing the camera to respond to it. A single camera generator supports both given-human and joint generation.

  2. Strong and comprehensive empirical results. AESOP improves FDC, r-FPD, text alignment, and human distribution metrics. The user study with 29 participants also strongly favors AESOP over all evaluated baselines, providing perceptual evidence beyond automatic metrics.

Critical weaknesses
  1. The contribution of the asymmetric architecture is not cleanly isolated. AESOP is compared with PulpMotion’s symmetric representation, but the systems also differ in latent dimensions, reconstruction objectives, generator design, training schedule, and augmentation. Consequently, the large performance gap cannot be attributed specifically to asymmetry.

  2. Translation-intensity control remains imperfectly validated. Only about 81% of test clips exhibit monotonic path-length control across all adjacent intensity values, and stronger intensity worsens framing metrics. The paper does not compare against another intensity-control method or evaluate whether users perceive the requested intensity changes as intended.

Questions for authors
  1. Is there a controlled symmetric-versus-asymmetric ablation using the same architecture, capacity, losses, and training schedule?

  2. What are the main failure modes among the approximately 19% of clips without monotonic intensity control? Are failures concentrated in particular camera-motion categories?

  3. Was translation-intensity controllability itself evaluated by human participants, separately from overall generation quality?

7 comments
AuthorsOpenReview ↗

Rebuttal (1/4)

We thank the reviewer for acknowledging AESOP's task formulation, comprehensive evaluation and strong performance. The two questions, on isolating the asymmetry and on validating intensity control, are well taken, and we answer both with new controlled experiments.

(W1, Q1) Controlled ablation of the asymmetric design

We cross the representation (Symmetric: each encoder reads both human and camera, as in PulpMotion; Asymmetric: the human is encoded alone) with the generation order (Symmetric: both latents are denoised together; Sequential: the camera is generated after the complete human). Everything else is identical: features, latent sizes, decoders, losses, flow Transformers of equal size, text encoder, data, no augmentation and training budget.

Table R1. Representation × generation, joint generation.

camera distribution & alignmentcamera framing & geometryhuman distribution & alignment
ModelRepresentationGenerationFDC ↓CLaTr ↑F1 ↑r-FPD ↓Out ↓FDTMR\mathrm{FD_{TMR}} ↓TMR ↑
AblationSymmetricSymmetric13.25±0.2966.46±0.220.860±0.0031.305±0.0813.78±0.67117.22±5.5019.19±0.27
AblationSymmetricSequential17.26±1.1163.77±0.420.813±0.0030.784±0.03‾\underline{0.784\pm 0.03}9.84±0.46‾\underline{9.84\pm 0.46}102.92±7.52‾\underline{102.92\pm 7.52}19.72±0.43
AblationAsymmetricSymmetric8.43±1.2570.19±0.930.907±0.0091.555±0.0315.70±0.32110.13±2.4720.01±0.52‾\underline{20.01\pm 0.52}
AESOP (no aug.)AsymmetricSequential11.39±0.67‾\underline{11.39\pm 0.67}68.94±0.34‾\underline{68.94\pm 0.34}0.880±0.002‾\underline{0.880\pm 0.002}0.665±0.039.13±0.24100.98±1.9120.46±0.34

Table R2. Autoencoder reconstruction on the test split.

human reconstructioncamera reconstructioncamera framing
AutoencoderGlobal MPJPE ↓Aligned MPJPE ↓Root ADE ↓Camera ADE ↓Rot. ↓Trans. jerk ↓r-FPD ↓Out ↓
Matched (sym.)0.1374±0.010.0495±0.0020.1124±0.010.035±0.0010.51±0.0111.66±0.210.108±0.033.16±0.41
AESOP (asym.)0.1304±0.010.0479±0.0030.1063±0.010.029±0.00060.41±0.018.55±0.400.089±0.012.93±0.28

Overall (Table R1), the asymmetric design wins as a whole. Against its matched symmetric counterpart (Symmetric representation, Symmetric generation), AESOP is better overall in all three metric groups, most clearly in framing (r-FPD 1.305 → 0.665, Out 13.78% → 9.13%).

For generation (Table R1), (1) the asymmetric representation drives camera quality. With the same sequential generation, it improves FDC, CLaTr and F1 (FDC 17.26 → 11.39). (2) Sequential generation makes the camera human-aware. With the asymmetric representation but symmetric generation, the camera is denoised against a still-noisy human: camera-only metrics stay high, but framing is much worse (Out 15.70% vs. 9.13%).

For reconstruction (Table R2), the asymmetric autoencoder is more accurate than the matched symmetric one on every metric, e.g. camera jerk 11.66 → 8.55 m/s³.

The revised paper adds this ablation as Table 3 (Sec. 4.3).

AuthorsOpenReview ↗

Rebuttal (2/4)

(W2, Q2) Validation of translation-intensity control

Pairs of numbers are given-human / joint throughout.

(1) Failure modes of the non-monotonic clips (W2, Q2)

What the control is trained to do. The intensity aa scales camera translation; human motion, camera text, rotation and field of view stay fixed. IPA trains it on translation-active shots (reference path ≥ 0.41 m) and trains near-static shots (path ≤ 0.10 m) to stay unchanged for any aa (App. C.2, Table S5). The construction (Eq. 5) scales the smoothed path, so the controlled quantity is path length.

Where the failures are. The 81.50% / 81.37% figure (Sec. 4.4, Fig. 6) counts a clip as monotonic if its camera path length is non-decreasing in all ten adjacent steps of the aa scan (zero tolerance), over all 4,053 test clips. All rates below use this path-length criterion.

  • Failures of the monotonicity check concentrate in near-static clips, which are expected to keep low translation through the null target of IPA training. About 78.1% / 76.4% of the non-monotonic clips have near-static references, which account for 21.7% of the test set (879 of 4,053; Table R3).
  • Translation-active clips respond monotonically. In the IPA active range, the monotonic rate is 98.90% / 98.90% (1,983 of 2,005 clips in each task; Table R3).

(2) Framing at stronger intensity (W2)

  • Larger travel costs some framing, as in real filming, but less than one point of Out in every travel range, while text alignment improves (Fig. 6, Table R3). From aa = 1.0 to 1.5, overall Out worsens from 11.39% to 11.93% / 8.44% to 8.89% and CLaTr improves from 71.09 to 71.65 / 69.51 to 70.72. Near-static clips stay near-static.
  • Smaller aa improves framing. Lowering aa to 0.5 improves Out to 10.61% / 7.86%, so users can choose the balance.
  • Framing stays far ahead of the baselines. Even at a=1.5a = 1.5, joint Out (8.89%) stays far below PulpMotion's 26.29% / 32.96% with its framing guidance (Table 1).

Table R3. Travel, framing and text alignment across intensity, and monotonicity, by reference travel. Path, Out and CLaTr cells give aa = 0.5 / 1.0 / 1.5. A clip is monotonic if its camera path length is non-decreasing in all ten adjacent steps of the aa scan (zero tolerance). Monotonic: share (number) of the stratum's clips that are monotonic. Share of non-monotonic: share of the task's non-monotonic clips that fall in the stratum.

monotonic (non-decreasing, zero tolerance)camera travelcamera framing & geometrycamera distribution & alignment
TaskReference pathNMonotonic, % (n)Share of non-monotonic, %Path (m)Out ↓CLaTr ↑
Given-humanNear-static (≤ 0.10 m)87933.33 (293)78.10.072 / 0.091 / 0.1072.99 / 3.10 / 3.1278.84 / 79.14 / 79.29
Intermediate (0.10–0.41 m)1,16987.85 (1,027)18.90.289 / 0.400 / 0.4855.20 / 5.79 / 6.1568.28 / 70.27 / 71.46
Active (≥ 0.41 m)2,00598.90 (1,983)2.91.107 / 1.489 / 1.73916.25 / 17.39 / 18.2266.76 / 68.04 / 68.40
All4,05381.50 (3,303)–0.647 / 0.872 / 1.02310.61 / 11.39 / 11.9369.82 / 71.09 / 71.65
JointNear-static (≤ 0.10 m)87934.36 (302)76.40.069 / 0.085 / 0.1004.68 / 4.61 / 4.4978.63 / 78.88 / 78.92
Intermediate (0.10–0.41 m)1,16986.66 (1,013)20.70.267 / 0.367 / 0.4454.88 / 5.29 / 5.6766.03 / 68.33 / 69.87
Active (≥ 0.41 m)2,00598.90 (1,983)2.90.556 / 0.776 / 0.93410.53 / 11.47 / 12.1863.11 / 66.10 / 67.63
All4,05381.37 (3,298)–0.367 / 0.508 / 0.6127.86 / 8.44 / 8.8967.32 / 69.51 / 70.72
AuthorsOpenReview ↗

Rebuttal (3/4)

(W2, Q2) Validation of translation-intensity control (continued)

Pairs of numbers are given-human / joint throughout.

(3) Comparison with another intensity control (W2)

The Table 1 baselines have no intensity input, so their only intensity control is the camera text. We insert degree words before the first camera-motion verb of each translation-active test caption, e.g. "The camera slightly moves right", using "very slightly", "slightly", "substantially" and "very substantially". The rule applies to 1,985 of the 2,005 translation-active test clips; the other 20 captions have no motion verb or no valid description. The modified prompts go to DIRECTOR-C and PulpMotion DiT/MAR (with its framing guidance), with the same sampling noise as the unmodified prompt.

  • Degree words give insufficient control and worsen text alignment. Even "very substantially" leaves the median path at 0.833–1.050 times the default, against 1.193 / 1.213 for aa = 1.5, so the five levels are ordered in at most 5.19% of clips. Every degree word also lowers CLaTr against the original caption.
  • AESOP's explicit intensity controls travel accurately and preserves text alignment. From aa = 0.5 to 1.5, the median path ratio moves 0.722 → 1.193 / 0.723 → 1.213 and the five levels are ordered in 99.50% / 99.75% of clips. Over the same range, CLaTr improves from 66.72 to 68.37 / 63.10 to 67.63.

The paper adds this comparison as Table S10.

Table R4. Intensity through text vs. through aa (1,985 translation-active test clips). Path ratio is the median per-clip ratio of camera path length to the default setting (unmodified prompt, or a=1a = 1). The columns very weak, weak, strong and very strong correspond to "very slightly", "slightly", "substantially" and "very substantially" for text, and to aa = 0.5, 0.7, 1.3 and 1.5. Ordering is the share of clips whose path strictly increases across all five levels, including the default. CLaTr (against the original caption) and Out are given for the weakest / default / strongest level.

path ratio to defaultcamera distribution & alignmentcamera framing & geometry
ModelTaskControlOrdering (%) ↑Very weakWeakStrongVery strongCLaTr ↑Out ↓
DIRECTOR-CGiven-humanText0.500.7380.9640.9580.93552.52 / 59.03 / 58.3011.74 / 14.56 / 13.18
AESOPGiven-humanaa99.500.7220.8371.1301.19366.72 / 68.01 / 68.3716.07 / 17.28 / 18.12
PulpMotion DiTJointText0.050.9130.9340.8410.83335.35 / 39.26 / 33.7431.68 / 32.60 / 29.76
PulpMotion MARJointText5.19‾\underline{5.19}0.9540.9630.9611.05040.63 / 44.90 / 38.2348.64 / 48.61 / 51.21
AESOPJointaa99.750.7230.8361.1401.21363.10 / 66.09 / 67.6310.64 / 11.59 / 12.32

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Rebuttal and Revisions​

We sincerely thank all reviewers for their valuable feedback and recognition of the novelty and significance of our formulation, the simplicity and practicality of our unified framework, and the effectiveness of our proposed methods supported by strong experimental results!

We have carefully addressed the reviewers' concerns and revised the paper accordingly, with changes highlighted in blue.

1. Isolating the asymmetric design (VP4w, LAFZ, SbYw)

  • Representation × generation ablation (Sec. 4.3, Table 3). Built from AESOP's own components, the matched symmetric counterpart worsens framing (r-FPD 0.665 → 1.305, Out 9.13 → 13.78%) and human motion (FDTMR\mathrm{FD_{TMR}} 100.98 → 117.22).
  • Two-stage baseline (Sec. 4.2, Table S7). DIRECTOR-C on the same AESOP-generated humans reaches Out 22.54%, against 8.47% for AESOP's camera generator.
  • Camera decoder reading zHz_H (App. B.1, Table S3). Against a camera-only decoder, AESOP's decoder improves FDC from 8.14 to 6.99 (given-human) and from 13.26 to 11.39 (joint).
  • Conditional camera encoder (Sec. 4.3, Table S3). Conditioning the camera encoder on the human latent degrades generation (given-human FDC 6.99 → 37.33), supporting a camera-only encoder.

2. Stronger baseline configuration (LAFZ)

  • PulpMotion with auxiliary framing guidance (Sec. 4.2, Table 1). Table 1 reports PulpMotion both with and without the guidance. With guidance, its Out of 26.29% (DiT) and 32.96% (MAR) remains far from AESOP's 8.47%.

3. Translation-intensity control (VP4w)

  • Failure modes (Sec. 4.4, Table S9). Most non-monotonic clips are near-static shots that IPA trains to stay unchanged; within the IPA active range, 98.90% of clips respond monotonically in both tasks.
  • Framing trade-off (Fig. 6, Table S9). A larger aa costs less than one point of Out in every travel range while improving text alignment, and a smaller aa improves framing.
  • Text-based intensity control (Table S10). Degree words in the camera text order camera travel in at most 5.19% of clips and lower text alignment, whereas AESOP's aa orders travel in 99.50% / 99.75% of clips.
  • Intensity user study (Table R5 in our reply to Reviewer VP4w). In blind pairwise comparisons, participants pick the higher-aa clip in 92.16–97.12% of translation-active pairs and see no difference in 98.85% / 97.70% of near-static pairs.

4. Formulation (SbYw)

  • Eq. 2 (Sec. 3.2) now states the factorization in latent space, where it is implemented, and its core conditional-independence assumption.

We address each reviewer's questions in the separate threads below and are glad to provide further clarifications.

Area_Chair_cSFwOpenReview ↗

Dicussion Period is About to Finish

Dear reviewers,

Thank you for your reviews and constructive feedback.

The author-reviewer discussion period will be finishing soon. Please take a moment to review it and consider whether your concerns have been adequately addressed. It would be helpful if you could briefly indicate:

  • Whether the rebuttal addresses your main concerns;

  • Whether any significant issues remain unresolved;

  • Whether your overall recommendation has changed.

If appropriate, please also update your review and recommendation in the system.

Thank you for your time and valuable contributions.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.