acceptodds
Withdrawn from ICLR 2027

Plenoptic Super-Resolution via Hybrid Light-Field and 3D Gaussian Priors

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Plenoptic super-resolution aims to reconstruct a scene at higher spatial-angular resolution from sparse, low-resolution light-field (LF) observations. Prior methods rely on spatio-angular priors like epipolar-plane image (EPI) continuity, but these can break under wide baselines, large disparities, and occlusions, and most models are tied to fixed upsampling factors, limiting resolution control at inference. To overcome these limitations, we complement LF structural priors with an explicit 3D Gaussian representation, which naturally supports continuous angular rendering and occlusion-aware view synthesis. Building on these complementary LF and 3DGS priors, we propose PlenoGS, a two-stage framework for plenoptic super-resolution. The first stage reconstructs a stable 3D Gaussian representation regularized by LF priors. The second stage enriches this representation with high-resolution spatial details through geometry-consistent refinement and scale-aware local implicit decoding, thereby preserving consistency across viewpoints while supporting flexible spatial magnification. On synthetic and real wide-baseline LF images, PlenoGS reduces occlusion ghosting and grid artifacts and improves spatial detail, angular consistency, and novel-view quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
light field, 3D Gaussian splatting, joint spatial-angular super-resolution, continuous implicit representation, wide-baseline view synthesis, scale-aware rendering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_HyK9OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This article proposes PlenoGS, a two-stage framework that combines prior light field structure with 3D Gaussian speckle for spatial angle super-resolution. Introducing EPI regularization and scale aware implicit decoding to improve geometric consistency and achieve flexible amplification.

Critical strengths

The combination of light field and explicit 3D priors has great motivation.

Critical weaknesses
  1. The novelty of the method is somewhat limited, as several key components are directly built upon existing 3DGS and SuperGS technologies.
  2. This method requires significantly more parameters than SuperGS in the standard settings and achieves lower rendering FPS.
  3. Any scale reconstruction is only qualitatively evaluated and lacks quantitative evidence at invisible magnification.
  4. The evaluation only covers six scenarios, and the ablation study is limited to one scenario.
Questions for authors

Can quantitative results be provided at an invisible magnification? What is the reason for the significant decrease in rendering speed compared to SuperGS?

Reviewer_fV9iOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PlenoGS, a two-stage framework for joint spatial and angular super-resolution of sparse, low-resolution light fields. Stage 1 uses Video Depth Anything (VDA), SfM alignment, a Scaffold-GS representation, and an epipolar-plane-image (EPI) consistency loss. Stage 2 uses pseudo-HR images from a pretrained 2D SR model, multi-view anchor densification, uncertainty-aware optimization, and a Local Implicit Representation (LIR) that predicts new-anchor attribute residuals from neighboring anchors and target scale.

The method is plausible and achieves competitive results, but the experiments do not adequately isolate the effects of external priors or the proposed modules. The necessity of EPI supervision on top of 3DGS is unclear, LIR appears to be an incremental adaptation of 2D local implicit SR, and the quality-efficiency trade-off on challenging real data is weak.

Critical strengths
  • Joint spatial and angular enhancement of sparse, wide-baseline light fields is a useful problem, and an explicit 3D representation is well suited to geometry, occlusion, and novel-view rendering.
  • The two-stage framework is coherent and reports competitive perceptual quality on synthetic and real data, including non-integer output scales.
  • The evaluation includes component ablations and rendering speed.
Critical weaknesses

1. External priors are not adequately controlled

PlenoGS uses a pretrained 2D SR model to generate pseudo-HR supervision. Some principal 3DGS baselines also use 2D SR priors, but the paper does not establish that they receive the same SR model or pseudo-HR images. Consequently, the comparison cannot separate improvements from the reconstruction method from those due to supervision quality.

The same issue applies to geometry: PlenoGS uses VDA and SfM alignment for dense initialization, but the paper neither provides equivalent geometry priors to the main baselines nor isolates the contribution of VDA. These are uncertain but important comparison variables that require controlled experiments.

The real Stanford ×4 results further weaken the empirical case. Compared with SplatSuRe, PlenoGS loses 0.79 dB PSNR, gains only 0.001 SSIM, improves LPIPS, and is substantially slower. The paper does not justify this mixed quality-efficiency trade-off.

2. The motivation and novelty of EPI and LIR are insufficient

Because 3DGS already supports continuous novel-view rendering, the paper must show what EPI loss adds beyond explicit geometry and multi-view RGB/SSIM supervision. The reference EPIs are built from the same observed LR views, making the pixel term partly redundant. The gradient term may help, but the paper lacks multi-scene ablations and targeted evaluation on missing views, occlusions, disparity, or EPI consistency. Moreover, EPI regularization is established in light-field reconstruction.

LIR also appears incremental: it transfers the familiar local-feature, relative-coordinate, scale-conditioned MLP design from 2D implicit SR to 3D anchor attributes. Its benefit is not established against nearest-neighbor copying, K-neighbor interpolation, a scale-agnostic decoder, or simpler scale encoding.

Finally, fixing base anchor positions in Stage 1 and largely preserving geometry in Stage 2 may prevent correction of VDA or SfM errors. This is a plausible explanation—not a demonstrated cause—for the weaker real ×4 result and should be tested with alternative geometry-freezing strategies.

Questions for authors
  1. Which 2D SR model and pseudo-HR images does each baseline use? Can the main methods be compared under identical pseudo-HR supervision?
  2. Which baselines use a comparable pretrained geometry prior? What is the isolated contribution of VDA under common initialization?
  3. What does EPI loss improve beyond standard 3DGS reconstruction losses? Please provide missing-view and EPI-consistency evaluations with multi-scene ablations.
  4. How does LIR compare with simple attribute interpolation and scale-agnostic or simply scale-conditioned decoders? How do alternative geometry-freezing strategies affect real ×4 performance?
Reviewer_GwzrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes PlenoGS, a two-stage framework for reconstructing high-resolution, densely sampled light fields from sparse low-resolution views. Stage I combines depth-guided initialization with EPI regularization to learn an anchor-based 3D Gaussian representation. Stage II uses pseudo-high-resolution supervision and multi-view voting for anchor densification, following SuperGS, and introduces a scale-aware local implicit decoder to refine anchor attributes. Experiments on synthetic and real light-field scenes evaluate spatial-angular super-resolution, rendering efficiency, and non-integer-scale rendering.

Critical strengths
  1. The reported reconstruction improvements are encouraging. PlenoGS achieves competitive results against both light-field and Gaussian-based methods. In the more challenging ×4 setting on FIT, it improves average PSNR over SuperGS from 29.13 to 30.21 dB. Its average LPIPS is also lower than SuperGS across all four dataset-scale settings, suggesting consistent gains in perceptual quality.
  2. The method combines improved reconstruction quality with fast rendering. The comparisons show clear advantages over rendering followed by independent LIIF upsampling, particularly at ×4. Refining shared 3D anchor attributes also provides a reasonable way to incorporate high-resolution details without applying a separate super-resolution network to every rendered view.
Critical weaknesses
  1. The motivation does not clearly separate the benefits of 3DGS from the contribution of this work. Occlusion-aware rendering and continuous viewpoint synthesis, emphasized in the abstract, are largely inherited from existing 3D Gaussian representations. Although the Introduction discusses limitations of Gaussian-based super-resolution, it does not clearly explain how each proposed component addresses a specific limitation. Since much of the two-stage framework follows SuperGS, the paper should focus more directly on the remaining limitations of existing 3DGS-based methods and the corresponding advances made by PlenoGS.
  2. The contribution of the scale-aware LIR design remains unclear. Table 3 shows that removing LIR substantially reduces performance, but this comparison does not separate the benefit of local feature refinement from that of scale conditioning. SuperGS-S1 + LIR achieves 34.72 dB, compared with 35.11 dB for the full method. However, this comparison does not isolate the contribution of scale conditioning. Since the local decoding and cell-conditioning ideas follow LIIF, the authors should compare the same LIR decoder without the cell descriptor and with the descriptor replaced by anchor depth alone, while keeping the decoder capacity comparable. These comparisons would clarify whether the proposed scale conditioning provides benefits beyond local feature refinement and depth conditioning. The current ablation also covers only one scene at ×2.
  3. The training and comparison protocol for arbitrary-scale reconstruction is unclear. Figure 4 presents qualitative results at non-integer scales, but the paper does not specify which scales were used to train the displayed models or whether the same checkpoint is used across all test scales. It is also unclear whether SuperGS is rendered directly from a fixed checkpoint or separately optimized for the target scales. Since Gaussian representations can already render at different resolutions, the claimed advantage should be demonstrated through reconstruction quality at unseen scales under clearly matched training conditions.
Questions for authors
  1. Which specific limitations of SuperGS and other Gaussian-based super-resolution methods are addressed by EPI regularization and scale-aware local decoding, respectively? Please distinguish these contributions from the capabilities already provided by the underlying 3D Gaussian representation.
  2. Can the authors compare the same LIR decoder without the cell descriptor and with the descriptor replaced by anchor depth alone, while keeping the decoder capacity comparable? Do the results hold across additional scenes and scales?
  3. Which spatial scales are used during Stage-II training? For Figure 4, are the non-integer scales excluded from training, and is the same checkpoint used at every scale? Please also clarify how SuperGS is trained and rendered in this comparison and provide quantitative results at unseen scales.
Reviewer_yLfbOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PlenoGS addresses joint spatial-angular super-resolution from sparse, low-resolution light-field observations. The first stage uses depth-guided initialization and EPI regularization to construct an anchor-based Gaussian representation. The second stage builds on SuperGS’s densification and uncertainty-aware supervision, introducing local implicit refinement of anchor attributes conditioned on neighboring anchors and a target-scale descriptor. The evaluation covers three synthetic and three real scenes at ×2 and ×4 spatial magnification, with qualitative demonstrations at non-integer scales.

Critical strengths
  1. The proposed adaptation has a clear purpose in the target setting. The two-stage backbone and local implicit decoding build on established methods, but PlenoGS brings them together through LF-guided geometry estimation and refinement of shared 3D anchor attributes. This is a sensible way to add detail while preserving consistency across viewpoints. The ablation provides evidence for both contributions: removing EPI regularization reduces Classroom PSNR by 0.55 dB, while removing LIR reduces it by 1.17 dB.

  2. The reconstruction results support the usefulness of the combined method. PlenoGS achieves the best reported mean PSNR and LPIPS on FIT at ×2, the best three-metric means on FIT at ×4, and the best three-metric means on Stanford at ×2. The results also show a meaningful tradeoff at ×4 on Stanford: SplatSuRe has higher PSNR, while PlenoGS has better SSIM and LPIPS. The paper reports this outcome accurately. Together with the visual results and reported real-time rendering rates, the evidence supports practical value without requiring the method to lead every metric.

Critical weaknesses
  1. The evidence for scale conditioning is weaker than the evidence for the complete LIR module. Figure 4 demonstrates plausible non-integer-scale rendering, but does not quantify reconstruction fidelity at those scales. Removing the whole LIR module changes considerably more than its scale input, so Table 3 cannot determine how much of the gain comes specifically from scale conditioning. The paper should distinguish flexible resolution control, which is demonstrated, from improved arbitrary-scale fidelity attributable to the scale descriptor, which is less firmly established. An otherwise unchanged decoder ablation would strengthen the latter claim; a more restrained interpretation is also appropriate.

  2. Some consequential evaluation details are unclear in the supplied main manuscript. The pseudo-HR generator is described through examples such as SwinIR or LIIF rather than an exact model configuration. The input and evaluation viewpoints, and the images available to pose/depth initialization, also need to be specified. These details matter because supervision quality and initialization can affect the comparison with SuperGS and SplatSuRe. The manuscript refers to separate supplementary material, which was not included in the material available for this assessment and may already contain these details.

Questions for authors
  1. Which scales supervise the local decoder, and which trained checkpoint produces Figure 4? Do the authors have a comparison with the same decoder but without the target-scale input that would help distinguish scale conditioning from the benefit of local refinement?

  2. Which exact SISR model and checkpoint generate the pseudo-HR labels in each setting? How is this supervision handled for SuperGS and SplatSuRe? Please point to the configuration if it is already documented in the supplementary material.

  3. Please specify the input and evaluation view sets, including whether the novel-view metrics exclude observed viewpoints. Are pose estimation, depth estimation, and initialization restricted to the designated input images?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.