- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Ray Differential Attention for novel view synthesis without explicit depth estimation. The method uses a central ray and its derivatives along the two image axes to capture the local angular scale, anisotropy, and shear of image patches. These geometric cues are incorporated into both attention scoring and feature aggregation. Specifically, the Q/K transformations model relative geometric relationships, while the V/O transformations enable cross-view feature transfer through invertible mappings. Under a unified LVSM architecture, the authors compare their method against five camera encoding approaches and achieve the best average performance across three metrics on RealEstate10K and DL3DV.
Critical strengths
- The local geometric representation is well motivated. Section 3.2 uses ray differentials to preserve local metric information that is lost through normalization in orthogonal ray frames, and incorporates it into invertible token transformations. The method requires no explicit depth estimation and does not increase the attention channel dimensionality, making it a geometrically well-grounded design.
- The main experiments demonstrate consistent improvements under controlled settings. Table 1 compares RDA against five external baselines across three random seeds. RDA improves PSNR by 0.446 dB over UCPE, while also achieving better SSIM and LPIPS scores.
Critical weaknesses
- Limited novelty. GTA (GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformers) already incorporates geometry-aware feature comparison and value aggregation in §3.1 and Eqs. (4)–(6), and maps the aggregated features back to the query coordinate frame. PRoPE (Cameras as Relative Positional Encoding) aso introduces projective transformations involving both camera intrinsics and extrinsics, combined with 2D RoPE. Therefore, the claim in Section 1 that this work establishes a new principle of using geometry to jointly govern token interactions and feature transformations appears to overstate its actual novelty. UCPE(Unified Camera Positional Encoding for Controlled Video Generation) also constructs token-level local ray transformations. Compared with these prior works, the main contribution of this paper lies in preserving local metric information through ray differentials and adopting distinct transformation designs for attention scoring and feature aggregation. Although the authors cite these works, they do not sufficiently explain why these specific differences constitute a substantial advance. In particular, PRoPE already incorporates camera intrinsics, so the absence of explicit ray derivatives should not be equated with the absence of camera calibration information.
- Insufficient experimental evidence. Table 4 provides the most direct evidence for the benefit of full ray differentials, reports results only for seed 0. Compared with the orthogonal frame baseline, the improvement is limited to 0.115 dB in PSNR and 0.0088 in LPIPS. While these results are encouraging, the lack of repeated experiments makes it difficult to assess the robustness of this key contribution.
Questions for authors
- As mentioned I'm concerned about this paper's novelty. Beyond replacing the geometric representation and adjusting channel allocation, what technical contributions distinguish RDA from GTA , PRoPE, and UCPE’s local ray transformations? Should the claim of establishing a principle that geometry should govern both attention scoring and feature aggregation be narrowed?
- Could you provide a control that retains the content pathway’s 2D RoPE while replacing only its geometric blocks with identity mappings?
- Could you compare the full ray differential against the orthonormal frame across additional random seeds and under the same crop/resize interventions? Please also clarify how the Unit ray configuration in Table 4 is constructed to yield an invertible transform.
Thank the new exp and response. The explicit construction of the unit-ray baseline resolves our invertibility question. The three seed comparison shows consistent gains in every seed, which strengthens the evidence for the core contribution. I also appreciate the more precise positioning relative to GTA, PRoPE, and UCPE.
Some concerns remain. Exact invertible transport and fixed head width are also properties of GTA and PRoPE. For pinhole cameras, the ray differential is determined by K, R, and the pixel coordinate, which PRoPE already uses, so it is unclear what this parameterization adds. The Q2 interventions are run at inference time and do not show that the geometric blocks help beyond 2D RoPE. A model trained with the geometric blocks set to identity is still needed. The zoom result (+0.185 dB) is measured at a single scale without multiple seeds, and it is smaller than the average gain. Given these remaining concerns, I maintain my original score.