- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies camera positional encodings for camera-controlled video generation, highlighting how homogeneous relative-pose encodings such as GTA, PRoPE, and UCPE can amplify attention logits and feature norms as the physical translation baseline increases. The authors establish a trade-off between full metric relative-pose dependence, per-token factorization, and norm preservation, and propose MeRoPE, which relaxes per-token factorization by combining ray-local relative rotations, multi-frequency rotary encoding of metric displacement, and disparity anchors along spherical epipolar arcs. Experiments on a mixed driving benchmark and PanShot show that MeRoPE achieves the lowest CamMC among the compared encodings.
Critical strengths
- The problem is well motivated. Eq. (3)–(4) make it clear how homogeneous encodings can amplify logits and feature norms as the baseline grows.
- The design is simple. Every block is orthogonal, so raw metric displacement goes in without the positional operator inflating feature norms.
- The paper compares against most recent camera encodings on the same backbone, and the ablations mostly support the role of each of the three components.
Critical weaknesses
- The motivating failure mode is not really shown. In Figure 1, the extra attention UCPE puts on the earliest latent frames likely goes to the conditioning frames, which is not obviously a baseline effect. I would suggest the authors add a direct measurement of attention logits or feature norms as the baseline grows.
- The benefit of raw metric translation could be made clearer. On PanShot, MeRoPE does better with near-depth normalization than without it (CamMC 12.67 vs. 13.92), and the driving benchmark has no normalized or direction-only variant. Since all translation metrics are scale-aligned, they cannot show whether the generated motion matches the commanded magnitude anyway.
- A per-token translation RoPE in the first-camera frame also keeps metric displacement, feature norms, and global-coordinate invariance, and gives up only exact pairwise relativity, which Theorem 1 requires. Since the disparity block needs grouping, I would suggest comparing against a variant that moves only the translation block to the first-camera frame.
Questions for authors
- Are the high-attention frames in Figure 1 the conditioning frames? Does the gap remain when they are excluded, and how do logits and feature norms change with baseline length?
- On the driving benchmark, how does raw metric translation compare with direction-only, normalized, or per-token first-frame variants? It would also be helpful to see translation error with the scale fit only on the conditioning frames.
5 comments
Response to Reviewer LiB8 (2/2)
Reply to W3 & Q2
We trained and evaluated the Direction-only, Clip-max normalized, and First-camera frame translation variants on the driving benchmark, now reported in Appendix C.8 (Table 12). Let denote camera-to-world poses relative to the first frame, with , , and :
| Translation Variant | RoPE Input | Rot () | Tr (%) | AUC@3 | CamMC 49f | 81f | 121f |
|---|---|---|---|---|---|---|---|
| Direction only | 0.97 | 3.26 | 63.77 | 2.14 | 6.86 | 13.52 | |
| Clip-max normalized | 0.93 | 1.85 | 64.77 | 1.51 | 3.44 | 6.77 | |
| First-camera frame | 0.87 | 1.70 | 65.92 | 1.41 | 3.08 | 6.27 | |
| MeRoPE (default) | 0.90 | 1.66 | 66.59 | 1.41 | 3.02 | 6.18 |
- Both
Direction onlyandClip-max normalizedperform worse than raw metric translation.Direction onlydiscards translation magnitude altogether, whileClip-max normalizedmakes the encoded scale of the same physical motion depend on how a clip is segmented: if two clips starting from the same point have maximum displacements of10 mand20 m, an identical2 mmotion is encoded as0.2in one clip and0.1in the other, creating inconsistent supervision during training. First-camera frameperforms close to the default query-frame encoding on driving data (tying on 49f CamMC with slightly lower rotation error, while trailing slightly ontr%,AUC@3, and 81f/121f extrapolation), likely because forward-driving sequences have relatively mild camera rotations. As you noted in W3, it preserves metric displacement, feature norms, and global-coordinate invariance, while relaxing the exact pairwise relativity assumption (H2) of Theorem 1 (where the true pairwise relative translation in Eq. (2) is ). For a concrete example, fix the first camera at the identity pose and place camera at :- Case A: cameras and have no rotation, and is at ;
- Case B: cameras and both rotate by about the -axis, and is at .
In both cases, the pairwise relative pose between and is identical (identity relative rotation, with displaced by
1 malong 's local axis), so default MeRoPE receives in both cases; by contrast, gives in Case A and in Case B, assigning different positional operators to the exact same relative camera motion.
Finally, because our monocular pose estimator VGGT- [3] normalizes each scene to a unit space during training, scale alignment is needed at evaluation. In Appendix C.6 (Table 11), we added an evaluation that fits the scale factor strictly on the first 9 conditioning frames and holds it fixed when scoring all subsequent generated frames:
| Method | 49f Future Pos. Err. (m): Full fit | Cond.-prefix fit | 49f CamMC: Full fit | Cond.-prefix fit | Avg. CamMC: Full fit | Cond.-prefix fit |
|---|---|---|---|---|---|---|
| UCPE (scaled) | 0.752 | 1.263 | 1.457 | 1.974 | 3.706 | 5.995 |
| RayNova | 0.698 | 1.213 | 1.377 | 1.849 | 4.376 | 6.058 |
| MeRoPE | 0.681 | 1.209 | 1.368 | 1.824 | 3.479 | 5.067 |
Fitting the scale on conditioning frames alone leaves the relative ranking unchanged across all metrics, with MeRoPE remaining the best throughout. Because our trajectory-balanced test set includes clips with short displacements over the first 9 frames where prefix-only scale estimation has higher variance, we retain full-trajectory fitting in the main text and include the conditioning-prefix comparison in Appendix C.6.
References
- [1] Yang et al., LongLive: Real-time Interactive Long Video Generation, arXiv:2509.22622, 2025.
- [2] Zhang et al., Unified Camera Positional Encoding for Controlled Video Generation (UCPE), CVPR, 2026 (arXiv:2512.07237).
- [3] Wang et al., VGGT-, CVPR, 2026 (arXiv:2605.15195).
I appreciate the authors' thoughtful response. All of my concerns have been addressed.
Response to Reviewer LiB8 (1/2)
Thank you for your positive assessment of our work. Your suggestions helped us make both our empirical diagnostics and theoretical discussions much more complete.
Reply to W1 & Q1
Yes, the earliest latent frames (indices
0to2) in Figure 1 correspond to the first 9 conditioning RGB frames. We chose a straight-driving scene to reduce viewpoint and appearance changes when inspecting how translation affects attention; however, early frames can act as global context anchors that attract extra attention (for instance, LongLive [1] retains "the first frame chunk of the video as global sink tokens"), so Figure 1 alone cannot separate conditioning-frame effects from baseline length. We have clarified this in the caption of Figure 1 and Section 3.1.Following your suggestion, we added direct measurements in Appendix C.7 (Figure 7): across 12 real driving trajectories, fixing the center query of the final latent frame, we record pre-softmax logit L2 norms and query-aligned value Frobenius norms across all camera-attention blocks and heads, and compute their Pearson correlation r with physical camera displacement both with and without the conditioning latent frames:
Both norms of raw UCPE grow strongly with camera baseline, and this trend persists even after excluding the three conditioning latent frames, whereas MeRoPE stays flat and nearly uncorrelated with distance across the entire range. These observations directly support the baseline-driven amplification analyzed in Section 3.1 (Eq. (3) and Eq. (4)).
Reply to W2
We apologize for this mistake, and thank you for pointing it out. We had mistakenly assumed PanShot carried physical metric scale (as parts of its pipeline use tools capable of metric depth estimation). Re-checking UCPE [2] confirmed that PanShot's base trajectories come from monocular SLAM ("recovers camera trajectories only up to an arbitrary scale"), so disabling near-depth normalization has no physical meaning and we have removed the
normalization-offexperiment from Section 4.3 and Table 5. To evaluate the benefit of raw metric translation and the effect of scale fitting on true metric data, we added the driving-benchmark ablations and conditioning-frame scale-fitting check below.