acceptodds
Withdrawn from ICLR 2027

EpiPE: Epipolar Positional Encoding for Camera-Controlled Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Camera positional encodings provide an efficient way to condition video diffusion transformers on prescribed viewpoints. However, encodings based on homogeneous camera transformations do not directly express the epipolar compatibility of viewing rays, a geometric prior available before the scene content is synthesized. We introduce EpiPE, a relative camera positional encoding that incorporates the epipolar prior into learned attention scores. EpiPE applies the classical Plücker line transformation to learned features, exposing the coupling between translation and viewing direction that underlies epipolar geometry. This places epipolar coupling within a consistent coordinate transformation whose composition structure allows relative camera interactions to factor into per-token operations. The resulting encoding integrates epipolar geometry into standard attention without restricting which tokens may interact. Experiments on PanShot and RealEstate10K show improved camera-pose accuracy with comparable visual quality.

Primary area:
generative models
Keywords:
Camera Control, Positional Encoding, Video Generation, Text to Video

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VCToOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EpiPE, a method for camera-controlled video diffusion. Instead of using 4D homogeneous transformations or epipolar attention masking, the method applies 6D Plücker line transformations to queries and keys to incorporate epipolar coupling into learned attention scores. The method trains a camera adapter on top of a frozen Wan2.1 backbone and evaluates on PanShot and RealEstate10K, alongside a two-view depth estimation task with UniMatch.

Critical strengths
  • Line transformation formulation: Using the Plücker line transformation law to bring epipolar coupling into per-token query and key transformations is sound and a neat idea compared to sampling along epipolar lines.
Critical weaknesses
  • Very weak supplementary video results: The supplementary video presentation is one of the key components of a video generation submission, but here it is very bad. It has only 3 indoor scenes, the camera movement is tiny, the scene is visibly deforming while the camera is moving, and there is clear flickering. There is no supplementary website to compare side by side with previous works. There are also no ablation videos showing the effect of epipolar coupling, no videos for dynamics, and no diverse video results across scenes. This alone is enough to reject the paper.
  • Weak results and marginal gains over UCPE: The quantitative gains over UCPE are very small on PanShot (translation error goes from 17.197 to 15.702), but FID actually gets worse from 62.55 to 65.01, and aesthetic quality drops on RealEstate10K. The results do not show that this formulation actually helps generate visually better videos.
  • Missing comparisons and limited scenes: The video evaluation is only done on PanShot and RealEstate10K, which are just static indoor scenes. There are no results on scenes with dynamic objects, where the epipolar geometry assumption breaks down.
  • Disjointed depth estimation experiment: Adding UniMatch depth estimation feels out of place and does not compensate for the weak video results.
Questions for authors

The method seems sound and I like the idea of using line transformations, but the visual results are not convincing and the evaluation is limited.

I would like authors to address following questions:

  • Why does the supplementary only have 3 indoor scenes with tiny camera movement and clear deforming/flickering artifacts? Can you provide a supplementary website with side-by-side comparisons with previous works across diverse scenes and larger camera trajectories?
  • How does the method handle dynamic objects? Does the approach fail or lead to distortions when dynamic objects violate the epipolar constraint?
  • Why did the visual quality degrade compared to UCPE (worse FID on PanShot and worse aesthetic score on RealEstate10K)?

I am currently negative and it is rather unlikely that I will raise my score to an accept, but I am still happy to consider a rebuttal.

Reviewer_A49xOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EpiPE, a camera positional encoding that applies Plücker line transformations to attention features, explicitly incorporating translation–rotation coupling. The encoding is integrated into a camera adapter on a frozen WAN backbone. Experiments demonstrate improved camera control on PanShot and zero-shot transfer to RealEstate10K, with comparable visual quality and inference cost to UCPE. Additional experiments evaluate its applicability to two-view depth estimation.

Critical strengths
  1. The formulation is geometrically motivated and computationally practical (Sections 3.2–3.3). It incorporates epipolar coupling through independent token transformations while preserving relative-coordinate consistency and compatibility with standard attention.

  2. The comparison with UCPE is well controlled, matching adapter capacity and other conditioning components. EpiPE improves all three aggregate camera-control metrics on both evaluation datasets with nearly unchanged inference time (Table 1). The depth-estimation results provide additional evidence of its usefulness beyond video generation.

Critical weaknesses
  1. The methodological novelty is relatively narrow. The ray frames, adapter architecture, and conditioning framework largely follow UCPE, while the main change is the adoption of the classical Plücker line transformation. This limits the conceptual advance despite the useful empirical gains.

  2. The evidence explaining the gains remains incomplete. The rotation-only ablation removes all relative translation information, so its degradation does not isolate the benefit of the specific epipolar coupling structure (Section 4.1.3). Although the UCPE comparison provides supporting evidence, additional controls and variability estimates would strengthen the conclusions.

Questions for authors
  1. Beyond the UCPE comparison, could a control retaining translation information help distinguish the benefit of the specific epipolar coupling from the general benefit of encoding translation?

  2. How stable are the improvements over UCPE across training or generation seeds, particularly for the smaller translation-error and depth-estimation gains?

Reviewer_jW5xOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Prior approaches for relative camera positional encoding do not embrace epipolar coupling explicitly. To tackle this, the proposed EpiPE replaces the 4x4 rigid transform used by relative camera positional encodings with the 6x6 Plücker line transform. The 6x6 matrix consists of a rotation matrix R and an essential matrix [t]×R[t]_\times R. Since the translation-rotation product term is a world-to-ray transformation by design, computation stays per-token. By plugging into a Wan2.1-1.3B adapter, it reports lower pose errors than reproduced UCPE on PanShot and on zero-shot RealEstate10K, at similar inference time and mixed visual-quality metrics.

Critical strengths
  • The idea of thinking of each feature space as line space rather than points is interesting.
  • It is a direct substitute for UCPE's geometric blocks with the same parameter count and a small amount of extra inference time.
Critical weaknesses
  • In Tab 1-2 (p. 7), UCPE PanShot RotErr 6.088 and FVD 552.19 are different from those of the UCPE paper (RotErr 4.12, FVD 495.14). Since the source of PanShot is cited as the UCPE paper, I believe these numbers should be the same and need clarification.
  • AC3D, CameraCtrl, and Wan CameraCtrl are omitted from the comparison, although UCPE evaluated them under the same RE10K protocol.
  • No baseline is evaluated using officially released weights. Baselines are trained or adapted by the authors, making it hard to anchor where the published methods stand.
  • Tab 3 shows only the importance of the translation, not the importance of the epipolar coupling. I believe additional experiments are needed, such as comparing the proposed method against applying diag(R,R)\text{diag}(R, R) with additional translation.
  • Although the authors claim to "expose the epipolar coupling, allowing the model to exploit this geometric prior" (Sec. 1), it feels overstated. Since the epipolar-coupling term in Eq. 14 is linear, it is zero for keys on the epipolar line and takes opposite signs on either side. So one side of the line is weighted more than those on the line and cannot favor epipolar-consistent tokens.
Questions for authors
  • Please provide visualizations of the attention maps to demonstrate whether the model actually attends to the epipolar lines.
  • Could the authors isolate the effect of epipolar coupling? (e.g., by evaluating a variant that uses 6 channels and 16 blocks with translation, but without coupling)
  • Please evaluate the baselines using the officially released UCPE checkpoints.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.