- Rating
- 4
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Light Field Primitives (LFP), a novel view synthesis representation that enables explicit differentiable primitive rendering into 4D ray space. Rather than representing scenes with 3D geometry like 3DGS, LFP represents a scene as a collection of ray-space primitives. Each primitive corresponds to a disparity-consistent ray group defined using the classical two-plane parameterization. The primitive stores appearance, opacity, and view-dependent radiance parameters for the ray group.
The key idea is that rays that observe the same scene content satisfy an affine coupling determined by disparity and can therefore be grouped together. Rendering is performed directly in ray space by measuring the deviation between a query ray and primitives and compositing primitive contributions in a way similar to point-based rendering.
Beyond standard NVS, several image formation phenomena are naturally ray-space operations. Consequently, multi-scale anti-aliasing, defocus deblurring with refocusing, fisheye rendering, and transparent-object reconstruction can all be supported using the same renderer with minimal modifications.
Experiments show competitive or state-of-the-art performance on standard NVS benchmarks while maintaining real-time rendering. The representation performs competitively on several tasks that typically require dedicated rendering approach. This validates the paper's central claim.
Critical strengths
- Novel and well-motivated representation paradigm
The paper proposes an explicit primitive representation in 4D ray space, filling a conceptual gap between implicit ray-space methods (e.g., Neural Light Field) and explicit geometry-based methods (e.g., 3DGS). The primitive formulation is technically coherent framework, that combines components such as epipolar geometry, opacity compositing, and view-dependent appearance modeling, etc. This is a good conceptual contribution.
- Strong validation across multiple tasks
The experimental section goes beyond standard NVS benchmarks and evaluates anti-aliasing, defocus deblurring and refocusing, fisheye rendering, and transparent-object reconstruction. These experiments directly validate the central claim that many imaging effects become natural operations in ray space.
The fact that the same renderer can be reused across all settings with minimal modifications is compelling. The transparent-object and fisheye experiments are scenarios that are ray-centric and often require substantial task-specific engineering for geometry-based representations. The ablation study further strengthens the paper. It provides convincing empirical support for the design decisions.
Critical weaknesses
- Insufficient discussion/comparison with respect to Neural Light Field
While the paper extensively compares and contrasts itself with NeRF and 3DGS, its discussion of prior ray-space approaches is surprisingly brief. In particular, the relationship to neural light-field methods such as Learning Neural Light Fields with Ray-Space Embedding is underdeveloped.
Given that both approaches operate directly in ray space, a more thorough comparison is necessary to better position the paper. While I believe LFP is substantially different from prior neural light fields, the paper does not make this distinction as explicitly as it should.
Questions for authors
-
How does primitive count and memory consumption vary across scene categories? Are there systematic differences between predominantly diffuse scenes and strongly specular or refractive scenes?
-
The atlas ablation convincingly justifies the use of multiple atlases. Could the authors comment on scenarios where the training-camera distribution provides poor coverage of scene content? Are failures dominated by insufficient capture coverage or by the atlas parameterization?
-
Have the authors evaluated sparse-view settings? Since the representation organizes primitives according to ray coherence rather than explicit geometry, it would be useful to understand how the method behaves when viewpoint sampling becomes sparser.
2 comments
Thank you for the detailed response. The clarifications are helpful and have addressed my main questions and concerns.
Response to Reviewer 6Z2h
We thank the reviewer for the positive assessment and the insightful questions.
W1. Relation to neural light fields
We added a detailed paragraph to the related work (Sec. A). The differences are:
Q1. Primitive count and memory by category (new Tab. 15)
For both methods, the primitive count varies across categories, with the largest counts for unbounded outdoor scenes and the smallest for synthetic objects. LFP is more compact than 3DGS on every real-scene category and on the refractive scenes, where the gap is largest. On the synthetic Blender scenes, LFP uses more primitives but requires a similar amount of memory.
Regarding diffuse versus specular content, LFP uses the same fixed parameter budget per primitive, including its three directional ASG lobes (Eq. 8). Specular appearance is modelled by the directional lobes of existing primitives rather than by extra material-specific parameters. The total primitive count depends on scene complexity; Tab. 15 gives it per category, including the refractive scenes. The tested refractive scenes have a similar average primitive count to the Tanks and Temples scenes (1.01 M vs. 1.00 M); LFP does not show a substantially higher primitive count on these refractive scenes.
Q2. Poor coverage: capture or charts?
Our experiments indicate that failures under poor coverage stem mainly from insufficient capture rather than from the charts: content not observed by any training ray cannot be inferred from the images. How the charts are constructed has little effect. Placing the 16 chart normals by K-means or on a uniform Fibonacci lattice gives 28.02 / 0.820 / 0.247 vs. 28.02 / 0.818 / 0.245 (PSNR / SSIM / LPIPS) on Mip-NeRF 360 (Tab. 13), and once several charts are used, their number changes little (Tab. 8, PSNR / SSIM / LPIPS). We thus can tell from the experiment results that charts are needed so that every captured ray has well-conditioned coordinates (a single chart is not enough, Fig. 3 (b)), but beyond that their arrangement is not the limiting factor.
Q3. Sparse views (new Tab. 14)
We evaluate sparse-view performance on LLFF with three training views selected as in FSGS. Both methods use the same training views and the same COLMAP points triangulated from these views.
On average over the eight scenes, LFP outperforms 3DGS by a large margin on all three metrics (+3.5 dB PSNR, Tab. 14), without any sparse-view regularizer. We attribute this in part to the explicit epipolar coupling, which ties rays observing the same content across the available views.