- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes StructLFGS for 3DGS reconstruction from sparse light field inputs. The algorithm maps the observed robust disparity range to a pre-specified canonical inverse-depth interval, thereby avoiding the need for SfM, and incorporates disparity-consistency supervision during the optimization phase. Experiments across multiple datasets demonstrate the effectiveness of the algorithm.
Critical strengths
- The algorithm models all cameras as a shared structured camera family, thereby avoiding independent 6-DoF computations; the effectiveness of this approach was validated through ablation studies.
- The algorithm feeds the computed optical flow into the subsequent optimization stage, enhancing the geometric consistency of the overall pipeline.
Critical weaknesses
- The algorithm proposed in the paper relies on the "structured family" assumption; however, actual devices inevitably exhibit hardware errors and calibration errors, yet the paper lacks experiments assessing robustness against such errors.
- The experiments focus primarily on PSNR, SSIM, and LPIPS metrics for RGB images, lacking an evaluation of geometric accuracy.
Questions for authors
Light field images are captured using a fixed camera system that has already been calibrated; why, then, is it necessary to recover camera parameters from the images? How does the accuracy of StructLFGS compare to 3DGS methods that utilize known poses (including sparse-view 3DGS)?
2 comments
Dear Reviewer iQk2,
We are writing to ask whether our rebuttal has addressed your concerns. We would be happy to discuss any remaining issues.
To W1
We agree that no imaging device is perfectly accurate. The two real datasets were captured with Lytro cameras. A fixed microlens array, shared lens, and sensor determine the relative arrangement of their angular views. This integrated design provides a physical basis for our structured camera family [1]. Established LF reconstruction methods also successfully exploit regular angular structure in real captures [2–5]. Their results support structured geometry as an effective approximation for practical LF reconstruction.
Our ablation further supports keeping the canonical family fixed under the evaluated conditions. After constructing the family, we allowed its shared baseline and offset parameters to be optimized. This reduced PSNR by 0.45 dB on the four-scene real-data subset compared with fixed cameras (Table 3). Thus, additional shared geometric refinement did not improve reconstruction. Together, the acquisition design, established LF results, and our controlled comparison support the use of a fixed structured family for these real captures.
[1] Dansereau, D. G., Pizarro, O., and Williams, S. B. Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras. CVPR, pp. 1027–1034, 2013.
[2] Yeung, H. W. F., Hou, J., Chen, J., Chung, Y. Y., and Chen, X. Fast Light Field Reconstruction with Deep Coarse-to-Fine Modeling of Spatial-Angular Clues. ECCV, pp. 138–154, 2018.
[3] Wang, Y., Liu, F., Zhang, K., Wang, Z., Sun, Z., and Tan, T. High-Fidelity View Synthesis for Light Field Imaging With Extended Pseudo 4DCNN. IEEE Transactions on Computational Imaging, 6:830–842, 2020.
[4] Jin, J., Hou, J., Chen, J., Zeng, H., Kwong, S., and Yu, J. Deep Coarse-to-Fine Dense Light Field Reconstruction With Flexible Sampling and Geometry-Aware Fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(4):1819–1836, 2022.
[5] Liang, Z., Wang, Y., Wang, L., Yang, J., Guo, Y., Liu, L., Zhou, S., and An, W. Diving Into Epipolar Transformers for Light Field Super-Resolution and Disparity Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(7):8726–8743, 2026.
To W2
We have added a GT-based disparity evaluation to complement the RGB metrics (§4.5, Fig. 6, Appendix F). On StillLife, directly rendered Gaussian disparity achieves 2.69 MSE×100 and 38.61% BadPix₀.₀₇, compared with 3.10/12.90% for EPINET on the original LF. The Gaussian result has lower average squared error, although more pixels exceed the strict error threshold. Appendix F specifies the checkpoint configuration used for this geometry evaluation.
We also evaluate the geometric information preserved in the reconstructed images. With the same pretrained EPINET, StructLFGS achieves 1.30/10.62%, compared with 5.36/21.73% for FS-GAF and 28.44/23.16% for EPIT-ASR. These results show that StructLFGS supports accurate downstream disparity recovery. Together, the added measurements evaluate both the geometry encoded by the Gaussian scene and the angular information retained in its rendered views.
Table R1. Disparity evaluation on StillLife.
To Q1
LF systems typically obtain regular angular structure through their optical design, as in Lytro cameras, or through calibration and rectification, as in camera arrays. The released images retain this structure. However, the corresponding calibration parameters may be unavailable or omitted from the dataset, as in the real datasets used here. Gaussian reconstruction still requires explicit cameras to project and render the scene. StructLFGS uses the available angular structure to construct compatible cameras and scene coordinates directly from the images. When calibrated parameters are available, our framework can use them directly.
Table 4 directly compares StructLFGS with reconstruction using GT-calibrated cameras in the same 3DGS-MCMC framework. Both use the same sparse input views. Using GT calibration improves PSNR by 0.70/0.16 dB with four/nine inputs on HCInew. Thus, our canonical construction achieves reconstruction quality close to the known-camera reference, especially with nine inputs, without requiring physical calibration parameters. This comparison isolates the camera choice within one reconstruction backend. Specialized known-pose sparse-view 3DGS methods have not been benchmarked separately.