acceptodds
Under review as a conference paper at ICLR 2027

MeRoPE: Metric Rotary Position Embedding for Camera-Controlled Video Generation

17 views

Abstract

In camera-controlled video generation, geometry-aware positional encodings condition tokens on camera extrinsics and per-token viewing rays. Existing schemes, however, have a scale-dependent failure mode on real-world metric camera trajectories: homogeneous projective encodings can cause attention logits and feature norms to grow unbounded with physical translation baselines. We propose MeRoPE (Metric Rotary Position Embedding), a norm-preserving relative camera encoding for attention. MeRoPE encodes relative orientations between calibrated viewing rays with orthogonal rotation blocks, maps raw metric displacements into multi-frequency rotary phases, and adds a disparity-anchored correspondence prior along the epipolar arc. This design strictly preserves feature norms, bounds pre-softmax attention logits regardless of the physical translation scale, and maintains exact invariance to global rigid coordinate changes. Across a multi-dataset driving benchmark and PanShot, which cover large-baseline trajectories and diverse camera optics, respectively, MeRoPE achieves stronger camera control than prior encodings, with the lowest reported mean joint rotation–translation error between generated camera motion and conditioning poses among the compared encodings. Code will be made publicly available.

Primary area:
generative models
Keywords:
camera-controlled video generation, rotary position embeddings, geometry-aware attention, metric camera trajectories, multi-view geometry, generative models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LiB8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies camera positional encodings for camera-controlled video generation, highlighting how homogeneous relative-pose encodings such as GTA, PRoPE, and UCPE can amplify attention logits and feature norms as the physical translation baseline increases. The authors establish a trade-off between full metric relative-pose dependence, per-token factorization, and norm preservation, and propose MeRoPE, which relaxes per-token factorization by combining ray-local relative rotations, multi-frequency rotary encoding of metric displacement, and disparity anchors along spherical epipolar arcs. Experiments on a mixed driving benchmark and PanShot show that MeRoPE achieves the lowest CamMC among the compared encodings.

Critical strengths
  1. The problem is well motivated. Eq. (3)–(4) make it clear how homogeneous encodings can amplify logits and feature norms as the baseline grows.
  2. The design is simple. Every block is orthogonal, so raw metric displacement goes in without the positional operator inflating feature norms.
  3. The paper compares against most recent camera encodings on the same backbone, and the ablations mostly support the role of each of the three components.
Critical weaknesses
  1. The motivating failure mode is not really shown. In Figure 1, the extra attention UCPE puts on the earliest latent frames likely goes to the conditioning frames, which is not obviously a baseline effect. I would suggest the authors add a direct measurement of attention logits or feature norms as the baseline grows.
  2. The benefit of raw metric translation could be made clearer. On PanShot, MeRoPE does better with near-depth normalization than without it (CamMC 12.67 vs. 13.92), and the driving benchmark has no normalized or direction-only variant. Since all translation metrics are scale-aligned, they cannot show whether the generated motion matches the commanded magnitude anyway.
  3. A per-token translation RoPE in the first-camera frame also keeps metric displacement, feature norms, and global-coordinate invariance, and gives up only exact pairwise relativity, which Theorem 1 requires. Since the disparity block needs grouping, I would suggest comparing against a variant that moves only the translation block to the first-camera frame.
Questions for authors
  1. Are the high-attention frames in Figure 1 the conditioning frames? Does the gap remain when they are excluded, and how do logits and feature norms change with baseline length?
  2. On the driving benchmark, how does raw metric translation compare with direction-only, normalized, or per-token first-frame variants? It would also be helpful to see translation error with the scale fit only on the conditioning frames.
5 comments
AuthorsOpenReview ↗

Response to Reviewer LiB8 (1/2)

Thank you for your positive assessment of our work. Your suggestions helped us make both our empirical diagnostics and theoretical discussions much more complete.

Reply to W1 & Q1

Yes, the earliest latent frames (indices 0 to 2) in Figure 1 correspond to the first 9 conditioning RGB frames. We chose a straight-driving scene to reduce viewpoint and appearance changes when inspecting how translation affects attention; however, early frames can act as global context anchors that attract extra attention (for instance, LongLive [1] retains "the first frame chunk of the video as global sink tokens"), so Figure 1 alone cannot separate conditioning-frame effects from baseline length. We have clarified this in the caption of Figure 1 and Section 3.1.

Following your suggestion, we added direct measurements in Appendix C.7 (Figure 7): across 12 real driving trajectories, fixing the center query of the final latent frame, we record pre-softmax logit L2L_2 norms and query-aligned value Frobenius norms across all camera-attention blocks and heads, and compute their Pearson correlation rr with physical camera displacement both with and without the conditioning latent frames:

MethodLogit L2L_2 Norm (w/ cond.)Logit L2L_2 Norm (w/o cond.)Aligned Value Norm (w/ cond.)Aligned Value Norm (w/o cond.)
UCPE (raw)0.6350.6530.9640.983
MeRoPE-0.030-0.045-0.0060.022

Both norms of raw UCPE grow strongly with camera baseline, and this trend persists even after excluding the three conditioning latent frames, whereas MeRoPE stays flat and nearly uncorrelated with distance across the entire range. These observations directly support the baseline-driven amplification analyzed in Section 3.1 (Eq. (3) and Eq. (4)).

Reply to W2

We apologize for this mistake, and thank you for pointing it out. We had mistakenly assumed PanShot carried physical metric scale (as parts of its pipeline use tools capable of metric depth estimation). Re-checking UCPE [2] confirmed that PanShot's base trajectories come from monocular SLAM ("recovers camera trajectories only up to an arbitrary scale"), so disabling near-depth normalization has no physical meaning and we have removed the normalization-off experiment from Section 4.3 and Table 5. To evaluate the benefit of raw metric translation and the effect of scale fitting on true metric data, we added the driving-benchmark ablations and conditioning-frame scale-fitting check below.

AuthorsOpenReview ↗

Response to Reviewer LiB8 (2/2)

Reply to W3 & Q2

We trained and evaluated the Direction-only, Clip-max normalized, and First-camera frame translation variants on the driving benchmark, now reported in Appendix C.8 (Table 12). Let (Rˉi,tˉi)(\bar R_i, \bar t_i) denote camera-to-world poses relative to the first frame, with dij=Rˉi⊤(tˉj−tˉi)d_{ij} = \bar R_i^\top (\bar t_j - \bar t_i), AT=max⁡0≤s<T∣tˉs−tˉ0∣2A_T = \max_{0 \le s < T} \\|\bar t_s - \bar t_0\\|_2, and ϵ=10−12\epsilon = 10^{-12}:

Translation VariantRoPE Input uiju_{ij}Rot ↓\downarrow (∘^\circ)Tr ↓\downarrow (%)AUC@3 ↑\uparrowCamMC 49f ↓\downarrow81f ↓\downarrow121f ↓\downarrow
Direction onlydij/max⁡(∥dij∥2,ϵ)d_{ij}/\max(\lVert d_{ij}\rVert_2,\epsilon)0.973.2663.772.146.8613.52
Clip-max normalizeddij/max⁡(AT,ϵ)d_{ij}/\max(A_T,\epsilon)0.931.8564.771.513.446.77
First-camera frametˉj−tˉi\bar t_j-\bar t_i0.871.7065.921.413.086.27
MeRoPE (default)dij=Rˉi⊤(tˉj−tˉi)d_{ij}=\bar R_i^\top(\bar t_j-\bar t_i)0.901.6666.591.413.026.18
  1. Both Direction only and Clip-max normalized perform worse than raw metric translation. Direction only discards translation magnitude altogether, while Clip-max normalized makes the encoded scale of the same physical motion depend on how a clip is segmented: if two clips starting from the same point have maximum displacements of 10 m and 20 m, an identical 2 m motion is encoded as 0.2 in one clip and 0.1 in the other, creating inconsistent supervision during training.
  2. First-camera frame performs close to the default query-frame encoding on driving data (tying on 49f CamMC with slightly lower rotation error, while trailing slightly on tr%, AUC@3, and 81f/121f extrapolation), likely because forward-driving sequences have relatively mild camera rotations. As you noted in W3, it preserves metric displacement, feature norms, and global-coordinate invariance, while relaxing the exact pairwise relativity assumption (H2) of Theorem 1 (where the true pairwise relative translation in Eq. (2) is Rˉi⊤(tˉj−tˉi)\bar R_i^\top (\bar t_j - \bar t_i)). For a concrete example, fix the first camera at the identity pose and place camera ii at (2,0,0)(2, 0, 0):
    • Case A: cameras ii and jj have no rotation, and jj is at (3,0,0)(3, 0, 0);
    • Case B: cameras ii and jj both rotate by +90∘+90^\circ about the zz-axis, and jj is at (2,1,0)(2, 1, 0). In both cases, the pairwise relative pose between ii and jj is identical (identity relative rotation, with jj displaced by 1 m along ii's local +x+x axis), so default MeRoPE receives (1,0,0)(1, 0, 0) in both cases; by contrast, tˉj−tˉi\bar t_j - \bar t_i gives (1,0,0)(1, 0, 0) in Case A and (0,1,0)(0, 1, 0) in Case B, assigning different positional operators to the exact same relative camera motion.

Finally, because our monocular pose estimator VGGT-Ω\Omega [3] normalizes each scene to a unit space during training, scale alignment is needed at evaluation. In Appendix C.6 (Table 11), we added an evaluation that fits the scale factor strictly on the first 9 conditioning frames and holds it fixed when scoring all subsequent generated frames:

Method49f Future Pos. Err. (m): Full fit ↓\downarrowCond.-prefix fit ↓\downarrow49f CamMC: Full fit ↓\downarrowCond.-prefix fit ↓\downarrowAvg. CamMC: Full fit ↓\downarrowCond.-prefix fit ↓\downarrow
UCPE (scaled)0.7521.2631.4571.9743.7065.995
RayNova0.6981.2131.3771.8494.3766.058
MeRoPE0.6811.2091.3681.8243.4795.067

Fitting the scale on conditioning frames alone leaves the relative ranking unchanged across all metrics, with MeRoPE remaining the best throughout. Because our trajectory-balanced test set includes clips with short displacements over the first 9 frames where prefix-only scale estimation has higher variance, we retain full-trajectory fitting in the main text and include the conditioning-prefix comparison in Appendix C.6.

References

  • [1] Yang et al., LongLive: Real-time Interactive Long Video Generation, arXiv:2509.22622, 2025.
  • [2] Zhang et al., Unified Camera Positional Encoding for Controlled Video Generation (UCPE), CVPR, 2026 (arXiv:2512.07237).
  • [3] Wang et al., VGGT-Ω\Omega, CVPR, 2026 (arXiv:2605.15195).

I appreciate the authors' thoughtful response. All of my concerns have been addressed.

Reviewer_22ALOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

MeRoPE proposes a norm-preserving relative camera positional encoding for camera-controlled video generation. It identifies that existing homogeneous/projective encodings let attention logits grow unbounded with the physical camera baseline, and formalizes a trilemma (full metric relative pose / strict per-token factorization / norm preservation) with an impossibility result. The method combines ray-local rotation blocks, multi-frequency rotary phases for metric displacement, and disparity-anchored rotations along the epipolar arc. On driving and camera-diverse benchmarks it attains the best joint camera-control accuracy, with component ablations, seed-variation analysis, and a cost study.

Critical strengths
  1. Principled trilemma + impossibility theorem that motivate a norm-preserving, metric-sensitive design rather than being decorative.
  2. Unusually honest evaluation: matched backbone, retrained baselines, seed ranges, cost overhead quantified.
Critical weaknesses
  1. The headline "lowest CamMC" on driving is statistically a tie with RayNova; the solid win is mainly on PanShot.
  2. Non-separable query-camera grouping adds substantial block-level latency and memory overhead (though the full-model overhead is modest at the tested resolution); scalability at higher resolution or longer sequences is untested.
  3. Normalization protocol is not the baselines' original, and the description of UCPE's normalization appears inaccurate. Baselines are retrained under a common fixed global translation scale rather than their original per-trajectory normalization (UCPE's supplementary normalizes each clip by near-plane depth). Stripping baselines of their own normalization to test the non-orthogonal operators the paper critiques may put them outside their designed regime. The paper also describes UCPE as normalizing "by its maximum translation norm," which does not match UCPE's supplementary ("near-plane depth"); this needs correction.
Questions for authors
  1. How do the multi-frequency phases behave at baselines beyond the chosen maximum wavelengths?
  2. What might the grouping overhead look like at higher resolutions or with longer sequences?
  3. The baselines are retrained under a common fixed global translation scale, while UCPE's supplementary normalizes each clip by near-plane depth. Was the per-trajectory normalization intentionally replaced, and would a control experiment with baselines trained and evaluated under their original normalization further strengthen the comparison?
  4. The text describes UCPE as normalizing "by its maximum translation norm," whereas UCPE's supplementary describes "near-plane depth." Could the authors clarify which version is referenced?
7 comments
AuthorsOpenReview ↗

Response to Reviewer 22AL (1/3)

Thank you for recognizing our theoretical formulation and evaluation protocol. Your questions also prompted us to further examine long-horizon extrapolation, scaling overhead, and the distinction between input normalization and evaluation alignment, which deepened our understanding of MeRoPE and improved the revised manuscript.

Reply to W1 & Q1

We agree that the 0.008 CamMC difference between MeRoPE and RayNova at 49f falls within our observed training-run range (0.043), and we now state explicitly in Section 4.2 that the two methods perform at the same level on 49f clips. We were initially curious as well why RayNova performed so strongly at 49f. As its authors note in their follow-up work URoPE [1], RayNova applies RoPE to Plucker rays in a fixed world coordinate frame and "still relies on the global coordinate system". When training and evaluation are both restricted to 49f, this reliance on a global frame shows little disadvantage; however, when we directly evaluate the 49f-trained checkpoints on longer 81f and 121f sequences (Table 2), a clear gap emerges:

MethodCamMC 49f ↓\downarrowCamMC 81f ↓\downarrowCamMC 121f ↓\downarrowAvg. CamMC ↓\downarrow
RayNova1.4233.478.384.42
UCPE (scaled)1.5083.116.653.76
MeRoPE1.4153.026.183.54

Under 81f and 121f extrapolation, RayNova degrades substantially and falls behind scaled UCPE, whereas MeRoPE maintains the lowest error.

Regarding Q1, when displacement δ\delta along an axis exceeds λmax⁡\lambda_{\max}, the orthogonal rotation blocks still preserve feature norms and rotate continuously as θk=2πδ/λk\theta_k = 2\pi\delta / \lambda_k without abrupt breakdown, though the slowest frequency begins to wrap around or enter phases sparsely covered during 49f training. To examine this behavior empirically, we added single-frequency ablations and training-free inference adaptation in Table 4 (excerpted below):

Translation SpectrumCamMC 49f ↓\downarrowCamMC 81f ↓\downarrowCamMC 121f ↓\downarrowAvg. CamMC ↓\downarrow
Default (multi-freq.)1.413.026.183.54
Single 4 m1.443.257.774.15
Single 50 m1.433.207.003.88
Single 100 m1.393.349.064.60
Default (YaRN-inspired)1.413.006.033.48
  1. Even Single 50 m outperforms every baseline in Table 2 except scaled UCPE in Avg. CamMC. By contrast, Single 100 m performs best at 49f yet degrades severely at 121f: a plausible explanation is that 49f training displacements rarely complete its long period, leaving larger phases under-trained during extrapolation, consistent with Scaling Laws [2], while Single 4 m lacks long wavelengths and wraps too rapidly over large baselines. This matches the analysis in Round and Round [3], where high frequencies support local relative resolution ("Positional attention") while low frequencies maintain long-range coherence until phase drift eventually accumulates over extreme distances; combining multiple frequencies balances both regimes.
  2. Inspired by YaRN [4] ("NTK-by-parts" and "Dynamic Scaling"), we keep the three shorter wavelengths fixed at 81f/121f inference and dynamically stretch only each axis's longest wavelength by sa=max⁡(1,qa/Da)s_a = \max(1, q_a / D_a), where qaq_a is the commanded trajectory's P99 displacement and DaD_a is twice the 49f training P99. Without any fine-tuning, this inference-time scaling further lowers 81f and 121f error. For even longer horizons in future work, directly incorporating long videos during training (following "train-long-test-long" in LongLive [5] and LongLive-2.0 [6], "AR-trained model already supports long-video generation") provides a more fundamental solution.
AuthorsOpenReview ↗

Response to Reviewer 22AL (2/3)

Reply to W2 & Q2

Thank you for this suggestion. In Appendix C.4 (Table 10), we added full-model Wan2.2-5B training and inference benchmarks extending from 49f to 121f at 288x512 (4 H20 GPUs for training steps; 1 H20 GPU for 50-step denoising):

Method49f Train Step (s) ↓\downarrow49f Train Mem (GiB) ↓\downarrow49f Denoise (s) ↓\downarrow121f Train Step (s) ↓\downarrow121f Train Mem (GiB) ↓\downarrow121f Denoise (s) ↓\downarrow121f Infer Mem (GiB) ↓\downarrow
Wan2.2 (no camera PE)0.75735.8021.771.74160.3754.5927.51
GTA0.80736.5823.181.82061.8157.7627.87
UCPE0.80636.5823.331.82061.8157.9627.87
MeRoPE (matched HSDP)0.84137.5424.661.92867.9663.2727.94
MeRoPE (packed K/V opt.)0.80946.4724.231.87677.1361.5827.90
  1. Because backbone attention and FFN computation dominate total runtime, extending sequences from 49f to 121f under the same HSDP setup adds only a modest relative step-time and denoising overhead over UCPE, while inference peak memory stays nearly identical. The extra training memory comes from storing query-grouped transformed K/V tensors for backpropagation (O(F2Pd)O(F^2 P d) vs. O(FPd)O(F P d)), which can be avoided in future implementations via query-group chunking or a fused grouped-attention kernel.
  2. Packing transformed K/V frequency bands and caching camera geometry at runtime (with optional unsharded forward-weight retention during FSDP training) further reduces both training step time and 50-step denoising latency while preserving mathematically identical outputs.
AuthorsOpenReview ↗

Response to Reviewer 22AL (3/3)

Reply to W3, Q3 & Q4

We apologize for the lack of clarity in our original text. In UCPE's [7] official codebase, input normalization during training/inference and trajectory normalization during evaluation coexist as two separate operations, which we now distinguish clearly in Section 4.3 and Appendix B.2:

  1. Camera-conditioning input normalization (PanShot retains UCPE's original near-depth protocol). On PanShot, all compared methods use UCPE's [7] per-clip near-depth input normalization so that "median near-plane depth becomes 1" (as implemented in official src/dataset.py). We did not replace PanShot's per-clip normalization with a global constant; the fixed scale 0.02188 applies only to the metric driving benchmark where per-clip near-plane depth is unavailable (Table 2 now reports both raw and fixed-scale driving variants).
  2. Evaluation-time scale handling (origin of "by its maximum translation norm"). The phrase "by its maximum translation norm" in Appendix B.2 referred to UCPE's [7] evaluation script (src/evaluate.py), which divides the ground-truth and predicted relative trajectories independently by their own maximum displacement from the first frame. To avoid distorting scores when predicted motion is near-static or mis-scaled, our shared-scale evaluation (cammc_s in code, written as CamMC in the paper; we will keep naming consistent upon code release) proceeds in two steps:
    • Convert ground-truth and predicted poses into their respective first-frame coordinates, and fit a single nonnegative least-squares scale s⋆s^\star over all sampled frames to align the predicted translation to ground truth;
    • Divide both trajectories by the ground-truth trajectory's maximum displacement from the origin before computing CamMC. (Re-scoring the same recovered PanShot trajectories with UCPE's original independently normalized evaluator preserves the same clear advantage of MeRoPE over UCPE.)

In addition, we fixed two issues in our PanShot experiments:

  • Removed normalization-off. Re-checking PanShot's construction confirmed that its base trajectories come from monocular SLAM (UCPE [7]: "recovers camera trajectories only up to an arbitrary scale"), so disabling near-depth normalization does not test physical metric scale and has been removed.
  • Aligned image-plane grid RoPE channels. We found that our original PanShot MeRoPE (S²Rot36 / Rot36 / Tr120) omitted the X48 / Y48 pixel-grid RoPE bands used by UCPE [7] and CRePE's [8] non-radial layers. Retraining MeRoPE and all three component ablations with the matched S²Rot36 / Rot36 / Tr24 / X48 / Y48 partition further improves performance (Table 5):
PanShot Method / AblationRotErr ↓\downarrow (rad)TransErr ↓\downarrowCamMC ↓\downarrow
UCPE4.2212.3014.78
CRePE3.9811.1013.70
MeRoPE3.469.4211.53
w/o disparity3.6514.1115.80
w/o camera rotation4.289.7212.46
w/o translation3.3311.7513.44

References

  • [1] Xie et al., URoPE: Universal Relative Position Embedding across Geometric Spaces, ECCV, 2026 (arXiv:2604.18747).
  • [2] Liu et al., Scaling Laws of RoPE-based Extrapolation, ICLR, 2024 (arXiv:2310.05209).
  • [3] Barbero et al., Round and Round We Go! What Makes Rotary Positional Encodings Useful?, ICLR, 2025 (arXiv:2410.06205).
  • [4] Peng et al., YaRN: Efficient Context Window Extension of Large Language Models, ICLR, 2024 (arXiv:2309.00071).
  • [5] Yang et al., LongLive: Real-time Interactive Long Video Generation, arXiv:2509.22622, 2025.
  • [6] Chen et al., LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation, arXiv:2605.18739, 2026.
  • [7] Zhang et al., Unified Camera Positional Encoding for Controlled Video Generation (UCPE), CVPR, 2026 (arXiv:2512.07237; code github.com/chengzhag/UCPE).
  • [8] Jin et al., CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation, arXiv:2605.12938, 2026.
Reviewer_jvkeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

MeRoPE is a camera positional encoding for video diffusion transformers. The proposed method replaces the 4×4 relative-pose operators of GTA/PRoPE/UCPE with a pure rotation operator that preserves feature length. It has three main components:

  1. minimum-rotation ray-local frames, for ray orientation;
  2. multi-frequency RoPE angles of the camera displacement, for metric translation;
  3. rotations toward fixed anchor directions along the spherical epipolar arc, for correspondence cues.

Experimental results show that on an author-built driving benchmark, MeRoPE ties RayNova on CamMC while trailing it on rotation and AUC metrics. On PanShot, MeRoPE gives the best translation error and CamMC, but a mid-ranked rotation error.

Critical strengths
  • The paper addresses a relevant and under-explored problem: how translation magnitude affects homogeneous camera encodings such as GTA, PRoPE and UCPE.
  • The operator is clean and well specified. Every component is a rotation, and the construction is explicit.
Critical weaknesses
  • The paper does not analyze the problem it raises. For example, there is no plot of control error against translation magnitude (baseline length).
  • PanShot translation is not metric. It comes from monocular 360° SLAM and is therefore known only up to scale [R1]. Hence the PanShot experiments do not test metric translation; they test unnormalized, arbitrary-unit (SLAM-scale) translation.
  • Among the five pose metrics in Tab. 2, MeRoPE outperforms the other methods only on CamMC, which the authors modified. Its margin over RayNova (0.008) is within MeRoPE's own run-to-run range (0.043, Tab. 5).
  • The translation metrics (CamMC, TransErr) fit a scale factor to each recovered trajectory before measuring error, so they do not penalize errors in translation scale. As a result, the metric scale itself is never evaluated, even though it is the paper's main claim.
  • The proposed encoding has inherent limitations, such as the range of translation it can cover, but the paper does not discuss them. Because the encoding is periodic, "the farther the translation, the smaller the attention" can only be expressed up to about half of the longest wavelength. Because the score is a sum of per-axis terms, joint conditions such as Euclidean distance can only be approximated.
  • The wavelengths are set manually per dataset, but neither the rationale nor a sweep over this setting is given. Only single vs multi-frequency is compared (Tab. 7).
  • CRePE is not compared on PanShot, although it is listed in Tab. 1.

[R1] Unified Camera Positional Encoding for Controlled Video Generation (UCPE; PanShot benchmark). C. Zhang, B. Li, M. Wei, Y.-P. Cao, C. Cruz Gambardella, D. Phung, J. Cai. CVPR 2026. arXiv:2512.07237 (https://arxiv.org/abs/2512.07237); code https://github.com/chengzhag/UCPE

Questions for authors
  • Does the advantage over homogeneous encodings (GTA/UCPE) grow with camera baseline? Could you report control error binned by trajectory length or speed, or under a translation-scale sweep, with the homogeneous baselines trained on raw meters (without the fixed 0.02188 rescale)?
  • Since PanShot poses come from monocular SLAM and are up to scale, in what sense does the normalization-off setting "preserve raw physical displacements" (Sec. 4.3)? What does this experiment show about metric translation?
  • Could you discuss the representational limits of the rotary translation encoding?
  • How were the per-axis wavelength ranges chosen for each dataset? How sensitive are the results to the number of wavelengths and to λ_min/λ_max? Which wavelength does the "Single f" variant in Tab. 7 use?
  • How does MeRoPE compare with CRePE on PanShot under the same setting (Wan2.1-T2V-1.3B adapter), given that CRePE reports gains over UCPE on this benchmark?
5 comments
AuthorsOpenReview ↗

Response to Reviewer jvke (1/4)

Thank you for your constructive review and suggestions. Running the experiments you suggested gave us a much clearer picture of how input scaling, multi-frequency wavelength selection, and long-horizon camera control interact, and we have added these analyses to the revised manuscript to make it a stronger reference on geometric positional encodings.

Reply to W1 & Q1

We previously applied a global scale factor (0.02188) to the homogeneous baselines so that they could perform competitively without raw translation blow-up. Following your suggestion, Table 2 in the revised manuscript now includes GTA and UCPE trained and evaluated directly on raw meter translations, and evaluates all 49f-trained models directly on 81f and 121f sequences to test extrapolation over larger baselines:

MethodCamMC 49f ↓\downarrowCamMC 81f ↓\downarrowCamMC 121f ↓\downarrowAvg. CamMC ↓\downarrow
GTA (raw)1.6734.5113.356.51
GTA (scaled)1.6123.578.464.55
UCPE (raw)1.6174.0411.495.72
UCPE (scaled)1.5083.116.653.76
MeRoPE1.4153.026.183.54

Feeding raw meter translations degrades both GTA and UCPE at 49f and causes much sharper degradation on longer 81f and 121f trajectories.

In Appendix C.5 (Figure 6), we further plot translation error against ground-truth start-to-end net displacement (in 10 m bins):

  1. With raw meter inputs (Figure 6a, c), GTA and UCPE perform close to MeRoPE on short trajectories, while the error gap grows substantially as baseline displacement increases. This directly confirms our motivation in Section 3.1: over large displacements, unscaled translational inner products can dominate attention logits and degrade trajectory control.
  2. With fixed global scaling (Figure 6b, d), shrinking translations prevents large displacements from overwhelming attention and narrows the gap at long baselines. However, this global downscaling trades off local resolution: compressing small motions reduces their discriminability and raises relative translation error (tr%) in short-displacement bins compared with both raw inputs and MeRoPE. By contrast, MeRoPE eliminates baseline-driven norm growth through orthogonal rotations while preserving sensitivity across scales via multi-frequency phases (analyzed further in our reply to W6 & Q4 below).
AuthorsOpenReview ↗

Response to Reviewer jvke (2/4)

Reply to W5, Q3, W6 & Q4

Thank you for suggesting a systematic frequency study. On driving scenes, our default setting uses 4 log-spaced wavelengths per axis over [0.5, 50], [0.5, 6], and [0.5, 200] m for x, y, z, while PanShot uses 4 wavelengths over [0.5, 8] in near-depth-normalized coordinates, chosen empirically from typical per-axis motion ranges. The Single f row in the original appendix table repeated the minimum wavelength 0.5 m. To provide a clear and self-contained sweep at a fixed 24D translation budget, we replaced that row with Table 4 in the main text, comparing single-wavelength baselines, varying λmin⁡\lambda_{\min}, scaling λmax⁡\lambda_{\max}, data-quantile selection, and training-free inference adaptation:

Group / SettingCamMC 49f ↓\downarrowCamMC 81f ↓\downarrowCamMC 121f ↓\downarrowAvg. CamMC ↓\downarrow
Default (multi-freq.)1.413.026.183.54
Default (YaRN-inspired)1.413.006.033.48
Single 4 m1.443.257.774.15
Single 50 m1.433.207.003.88
Single 100 m1.393.349.064.60
Minimum λmin⁡=0.25 m\lambda_{\min}=0.25\text{ m}1.413.066.643.70
Minimum λmin⁡=1.0 m\lambda_{\min}=1.0\text{ m}1.402.996.113.50
Minimum λmin⁡=2.0 m\lambda_{\min}=2.0\text{ m}1.403.046.473.64
Maximum λmax⁡×0.5\lambda_{\max}\times 0.51.443.156.813.80
Maximum λmax⁡×2\lambda_{\max}\times 21.433.046.303.59
Maximum λmax⁡×4\lambda_{\max}\times 41.433.146.943.84
Quantile1.352.926.143.47

Table 4 shows four main findings:

  1. All three single-wavelength variants (4 m, 50 m, and 100 m) underperform the default multi-frequency spectrum in Avg. CamMC, although Single 50 m still beats every baseline in Table 2 except scaled UCPE. A single wavelength λ\lambda cannot distinguish displacements separated by integer periods, whereas combining multiple subspaces with distinct wavelengths avoids single-period aliasing (RoFormer [1]). Notably, Single 100 m gives the lowest 49f error among single-wavelength runs yet suffers the worst degradation on 121f extrapolation: a plausible explanation is that 49f training displacements rarely span its full period, leaving larger phases under-trained during extrapolation, consistent with Scaling Laws [2], while Single 4 m wraps too rapidly over long baselines.
  2. Performance changes little when λmin⁡\lambda_{\min} varies within [0.5, 2.0] m or when λmax⁡\lambda_{\max} is doubled to z = 400 m, and only degrades noticeably under overly narrow or wide bounds (0.25 m, ×0.5\times 0.5, or ×4\times 4).
  3. Motivated by the "field-resolution tradeoff" in How Data Shapes RoPE [3] (where the optimal wavelength scales with dependency width as λ⋆=2W\lambda^\star = 2W), Quantile heuristically sets the four per-axis wavelengths from displacement quantiles (49f train P75, and twice 49f P90, 49f P99, and 121f test P99; Appendix B.1, Table 6), achieving the best Avg. CamMC. Given the mild sensitivity near the default band and the cost of retraining per setting, our empirical default range already provides a practical trade-off.
  4. Inspired by "NTK-by-parts" and "Dynamic Scaling" in YaRN [4], we keep the three shorter wavelengths unchanged at 81f/121f inference and dynamically stretch only each axis's longest wavelength by sa=max⁡(1,qa/Da)s_a = \max(1, q_a / D_a), where qaq_a is the commanded trajectory's P99 displacement and DaD_a is twice the 49f training P99. Without any retraining, Default (YaRN-inspired) achieves the lowest 121f CamMC in Table 4 and nearly matches Quantile overall.
AuthorsOpenReview ↗

Response to Reviewer jvke (3/4)

Regarding W5 & Q3 on the representational limits of rotary translation encoding, we have added an explicit discussion in Section 3.2:

  • Periodicity and effective range. RoPE does not rely on forcing attention to decay monotonically with distance. Because queries QQ and keys KK are learned vectors, the network can align their initial directions so that attention peaks at any target relative offset (Round and Round [5] proves "RoPE can be maximal at arbitrary distance"); likewise, the "long-term decay" in RoFormer [1] is an aggregate multi-frequency envelope rather than a half-wavelength (λmax⁡/2\lambda_{\max}/2) cutoff. The real representational limit arises when displacements grow past the longest wavelength or the motion range seen during training, where even the slowest frequency completes full turns and wraps around. At inference time, stretching the longest wavelength via YaRN [4] mitigates this wrap-around; for much longer trajectories, training directly on longer videos with correspondingly wider wavelengths (as in LongLive [6], "train-long-test-long") is a natural path forward.
  • Per-axis encoding vs. Euclidean distance. Assigning (x,y,z)(x, y, z) to three separate channel groups with 1D RoPE follows the standard design for multi-dimensional positions in vision and video Transformers: image models apply 1D RoPE separately to pixel (u,v)(u, v) coordinates (RoPE-ViT [7], "Axial frequency"), and our Wan [8] backbone likewise applies 1D RoPE separately to (t,h,w)(t, h, w). Two intuitive reasons motivate this choice:
    1. Camera control cannot treat equal Euclidean distances identically because motion direction matters: moving 5 m forward and 5 m sideways have the same Euclidean norm yet induce completely different visual changes. Encoding (Δx,Δy,Δz)(\Delta x, \Delta y, \Delta z) separately preserves both the magnitude and 3D direction of relative motion without losing geometric information.
    2. A single attention dot product does not need to compute Euclidean distance directly. Although one attention logit sums three per-axis terms (unlike "Mixed learnable frequency" in RoPE-ViT [7], which mixes axes inside one phase), just as axial 2D RoPE lets ViTs represent arbitrary 2D spatial relations, multi-head attention, Softmax, and subsequent MLPs combine the (Δx,Δy,Δz)(\Delta x, \Delta y, \Delta z) features nonlinearly across layers to approximate joint cross-axis conditions such as Euclidean distance, complemented by cross-view ray relations from our spherical disparity block.

Reply to W2 & Q2

Thank you for catching this mistake, and we apologize for the oversight. After re-checking PanShot's data construction, we confirmed that UCPE [9] explicitly states: "Monocular SLAM recovers camera trajectories only up to an arbitrary scale". Disabling near-depth normalization therefore retains only an arbitrary SLAM scale, and our original claim that it "preserves raw physical displacements" does not hold. We have removed the normalization-off experiment from Section 4.3 and Table 5, retaining near-depth normalization to evaluate camera control across pinhole, wide-angle, and fisheye optics when physical metric scale is unavailable.

Reply to W3

We agree that the 0.008 CamMC difference at 49f lies well within MeRoPE's training-run range (0.043), and we now state explicitly in Section 4.2 that MeRoPE and RayNova perform similarly at 49f. As noted in the follow-up work URoPE [10], RayNova encodes Plucker rays in a fixed world frame and "still relies on the global coordinate system". When we directly extrapolate the 49f-trained models to 81f and 121f on the driving benchmark (Table 2), RayNova degrades substantially more than both scaled UCPE and MeRoPE:

MethodCamMC 49f ↓\downarrowCamMC 81f ↓\downarrowCamMC 121f ↓\downarrowAvg. CamMC ↓\downarrow
RayNova1.4233.478.384.42
UCPE (scaled)1.5083.116.653.76
MeRoPE1.4153.026.183.54

Comments on the paper

1 comment
AuthorsOpenReview ↗

Discussion summary for the Area Chair

Dear Area Chair,

Thank you and the three reviewers for reading our paper and discussing it with us. We answered each question, and the suggested experiments helped us understand where MeRoPE’s gains are clearest.

We agree with jvke and 22AL that MeRoPE and RayNova are effectively tied on 49-frame driving clips. When we use those same checkpoints at 121 frames, MeRoPE’s camera motion error (CamMC) is 6.18, compared with 8.38 for RayNova, which encodes rays in a fixed world frame. Combining short and long wavelengths gave lower 121-frame error than any single wavelength we tested. Stretching just the longest wavelength at inference reduced it further to 6.03. At jvke’s suggestion, we also trained CRePE (NeurIPS 2026) using its newly released code. MeRoPE had lower CamMC on both the driving benchmark and PanShot.

All three reviewers asked about translation scale. MeRoPE had the lowest average CamMC across 49, 81, and 121 frames in the comparison with raw-meter and globally scaled baselines. Following 22AL’s clarification, we retrained eight methods using UCPE’s per-clip near-depth normalization and applied it at inference too. MeRoPE and UCPE were nearly tied at 49 frames. At 81 and 121 frames, MeRoPE had the lowest CamMC of the eight, 9.0% and 13.8% below UCPE. We also measured high-resolution training and inference costs. Generating 121 frames at 576 × 1024 took 118.0 seconds with MeRoPE and 113.6 with scaled UCPE on the same four GPUs.

LiB8 suggested fitting the evaluation scale only on observed frames and measuring attention on real trajectories. MeRoPE encodes camera motion with rotations that preserve feature norms. On 12 real trajectories, its value norms stayed nearly flat as camera displacement grew, even after we excluded conditioning frames. LiB8 confirmed in the discussion that our response addressed all their concerns.

We believe MeRoPE gives researchers a simple, effective way to encode camera motion for video generation. We will release our code and evaluation protocol so others can test it on their own trajectories and build on it. Thank you again for the time you and the reviewers have given our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.