acceptodds
Under review as a conference paper at ICLR 2027

S4GS: Resolution-Asymmetric 4K Feed-Forward Gaussian Reconstruction from Sparse Views

Abstract

Feed-forward 3D Gaussian Splatting (3DGS) provides an efficient way to reconstruct renderable 3D scenes from sparse views, making it attractive for mobile immersive applications. However, immersive rendering demands 4K-level scene reconstruction, while directly processing such high-resolution inputs with large geometry backbones incurs prohibitive GPU memory costs. Existing high-resolution 3DGS methods mainly rely on per-scene optimization and are often enhanced by image or video super-resolution priors, making them difficult to support real-time immersive rendering. We present **S4GS**, a **resolution-asymmetric framework for 4K feed-forward Gaussian reconstruction from sparse views**. The key idea is to perform expensive scene-level reasoning at low resolution, while reserving high-resolution processing for dense geometry construction and local appearance refinement. For global reasoning, S4GS uses a low-resolution VGGT backbone to estimate camera parameters and build a coarse Gaussian scaffold. For high-resolution recovery, we anchor Gaussian positions using accurate dense depth maps, and employ a lightweight RGB branch to extract high-resolution image features and predict residual Gaussian attributes to restore fine details. By separating global 3D reasoning from high-resolution scene recovery, S4GS enables the full training and inference pipeline on a single 24 GB RTX 4090 GPU. Experiments on real-world mobile stereo data show that S4GS produces high-fidelity 4K novel views and supports variable-baseline stereo rendering for practical mobile-to-headset immersive applications.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feed-Forward 3D Gaussian Splatting, Sparse-View Reconstruction, 4K Immersive Rendering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uftyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

What interests me most about this paper is not merely its ability to process 4K inputs, but its focus on the limitations of large geometry models such as VGGT when applied to ultra-high-resolution images. Directly applying such models to UHD inputs causes the number of tokens and activation memory to grow rapidly with resolution. S4GS proposes a resolution-asymmetric architecture that decouples global geometric reasoning from local high-frequency detail recovery. It uses a large pretrained geometry model at low resolution to estimate the camera poses and construct a coarse Gaussian scaffold.

Critical strengths
  1. The paper addresses a practical problem. On StereoGenBench, its full-reference evaluation results are indeed better than those obtained through image upsampling.
  2. Fixing the Gaussian centers with depth is reasonable because it prevents the RGB branch from moving the geometry merely to explain texture.
Critical weaknesses
  1. The motivation is clear, but the construction of the final Gaussians is underspecified. Does every pixel in both views become a Gaussian, and how are duplicate surfaces merged?
  2. The experimental baselines are not sufficiently convincing. The paper compares only against AnySplat-LR with bicubic upsampling, AnySplat-LR with EDSR, and AnySplat-HR, which runs out of memory. These comparisons show that resolution-asymmetric reconstruction outperforms applying bicubic or EDSR upsampling to low-resolution AnySplat results, and that native-resolution AnySplat exceeds 24 GB of memory. However, every baseline that actually runs is still the same low-resolution reconstruction followed by image upsampling.
  3. An inference time of 0.81–0.87 seconds per stereo pair does not support a conventional real-time claim. The paper also does not report rendering FPS, although the conclusion describes the inference process as real-time.
  4. The reported Depth AbsRel is not computed against ground-truth metric depth; instead, it measures cross-view depth consistency.
Questions for authors
  1. Equation (9) lists only the color, rotation, and scale residuals. The output representation should be defined completely.
  2. Additional baseline comparisons are needed, for example, against a simple baseline that directly back-projects the depth predicted by DA3 and assigns colors from the RGB images.
  3. StereoGenBench provides metric depth. Why is geometric accuracy against that ground truth not reported?
Reviewer_r7MMOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors presents S4GS, a framework for feed-forward Gaussian reconstruction from sparse high-resolution images under a 24 GB GPU memory budget. The method separates scene-level reasoning from dense reconstruction. A low-resolution geometry backbone predicts cameras, coarse depth, and an initial Gaussian representation. A separate depth estimator provides dense depth maps, which are scaled to match the coarse reconstruction and back-projected into fixed Gaussian centers. A lightweight RGB-conditioned network then predicts residual updates to Gaussian appearance and covariance-related attributes.

Critical strengths
  • This paper targets on an practical issue with limited resource constraint. The proposed separation between low-resolution global reasoning and high-resolution local reconstruction is well motivated.

  • The method assigns complementary roles to the global model, depth prior, and RGB refinement network. Fixing depth-derived centers reduces the opportunity for appearance refinement to move points arbitrarily, while retaining access to native-resolution RGB observations can help recover details absent from the coarse representation.

  • The paper includes mobile examples and component ablations.

Critical weaknesses
  • The closest missing comparison is LGTM, “Less Gaussians, Texture More: 4K Feed-Forward Textured Splatting”. I think most of the proposed method overlaps with the LGTM paper. In my opinion, this work should be discussed and, where feasible, evaluated under comparable conditions.

  • The baselines and ablations do not isolate the proposed architectural contribution. A particularly informative control would retain the same cameras, aligned DA3 depths, Gaussian centers, and native RGB inputs, while replacing the proposed coarse-attribute-plus-residual construction with a simple direct attribute predictor.

  • the paper should clarify whether the AnySplat baselines receive comparable dataset adaptation and supervision.

  • Eq. 4 applies one scalar depth adjustment independently to each view. Back-projection with common cameras then places the resulting points in a shared coordinate system. However, these operations do not enforce agreement between corresponding surfaces across views.

  • The mobile evaluation uses no-reference image-quality metrics at a single three-times-baseline displacement. Probably a baseline sweep with known target views, including analysis of newly exposed regions, would more directly support the variable-baseline immersive-rendering claim?

  • Table 1 incorrectly emphasizes the MANIQA result.

Questions for authors

See above weakness section.

Reviewer_jAfcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an asymmetric-resolution framework for feed-forward 4K-resolution Gaussian reconstruction. It uses AnySplat on low-resolution input images to generate a coarse Gaussian scaffold, while simultaneously using DA3 to predict high-resolution depth. High-resolution Gaussian centers are then constructed through scale alignment and back-projection. Finally, a specially designed CNN-based network refines the Gaussian attributes, enabling feed-forward 4K-resolution rendering while reducing the GPU memory overhead during inference compared with native 4K reconstruction.

Critical strengths
  1. The motivation is highly practical, focusing on the trade-off between high-quality reconstruction (at native 4K resolution) and the computational resources required for training and inference. The proposed method is interesting. It uses a general low-resolution feed-forward reconstruction model as the backbone, leverages an existing depth estimation model to achieve high-resolution alignment, while employing an additional, lightweight network to refine the results of the low-resolution reconstruction.
  2. Ablation studies demonstrate that the two newly introduced modules are indeed effective.
Critical weaknesses
  1. The method.
  • (Critical) Although this paper achieves feed-forward 4K reconstruction and rendering, the number of Gaussians remains pixel-aligned with the high-resolution input (due to the DA3 alignment). Consequently, there is no significant reduction in either storage requirements or rendering costs. While the GPU memory usage during inference is lower than that of a network processing native 4K input, it remains higher than a common feed-forward reconstruction network reconstructing/rendering at a lower resolution and subsequently upscaling to 4K.
  1. Comparison experiments.
  • (Critical) The paper lacks a direct comparison with existing 4K feed-forward reconstruction methods. While the paper compares the proposed method against other AnySplat variants (-LR+Bicubic, -LR+2DSR, and -HR), it does not include a comparison with existing 4K feed-forward reconstruction methods, such as LGTM [1] and DA3 (which is utilized within the proposed framework and can also output Gaussians). The paper should include comparisons against these relevant methods.
  • It's better to include a comparison with per-scene 4K reconstruction methods. While I understand that this work focuses on feed-forward reconstruction, which differs significantly in paradigm from per-scene optimization and offers substantial gains in efficiency, per-scene 4K reconstruction methods have been well studied. Given the limited number of baselines currently included, adding such per-scene optimization methods (e.g., Vanilla 3DGS [2] or Turbo-GS [3]) would help clarify the upper bound of performance. Note that since SfM may fail under sparse views, initial point clouds generated by methods such as VGGT could be used to facilitate fair comparisons with these per-scene methods.
  • It would be beneficial to include a comparison across a more diverse range of backbones. While the proposed method employs AnySplat as the low-resolution backbone and utilizes DA3 to provide high-resolution depth for alignment, the framework itself imposes no specific constraints, and the proposed method seems to be a plug-in for large 3D foundation models (although it needs additional training). Evaluating the proposed method across different backbones (such as DA3, since DA3 also inherently produces Gaussian outputs like AnySplat) would help demonstrate its effectiveness and generalizability.
  • Lack of multi-view results. The experiments in the paper are limited to two-view inputs. However, both the AnySplat backbone and DA3 support multi-view inputs. If the proposed method restricts the number of input views, the practical application capability will be largely limited.
  • Resolution Choice. As a key baseline, the AnySplat-LR variants are evaluated at an input resolution of 756×756, which differs from the resolution used during training and may degrade their performance. While this choice aligns with the proposed method's input resolution, it would be useful to additionally evaluate AnySplat-LR at its training resolution of 448 while keeping the rendering resolution consistent across methods. I suggest including this setting in the baseline comparisons.
  1. Datasets
  • (Critical) Commonly used datasets and generalizability. The paper mainly reports results from training and evaluation on the Self-collected Mobile Dataset. However, neither the Self-collected Mobile Dataset nor StereoGenBench is a widely adopted benchmark in the 3D reconstruction community. Moreover, the paper does not clarify whether StereoGenBench was used for training. I suggest conducting experiments on commonly used 4K datasets, such as DL3DV 4K, and evaluating the proposed method on additional 4K datasets excluded from training data, such as Mip-NeRF 360 4K, to better demonstrate its generalizability.
  • Dataset statistics. The statistics of the self-collected dataset are not sufficiently comprehensive. Specifically, the total number of scenes and potential scene overlap between the training and test sets are not clearly stated. In addition, scene similarity (both within and between the training and test sets) has not been quantified.

[1] Lao, Y., Xuyang, B. A. I., Wu, X., Yan, N., Luo, Z., Fang, T., ... & Zhao, H. (2026). Less gaussians, texture more: 4k feed-forward textured splatting. In The Fourteenth International Conference on Learning Representations.

[2]Kerbl, B., Kopanas, G., Leimkühler, T., & Drettakis, G. (2023). 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4), 139-1.

[3]Dhiman, A., Lu, T., Srinath, R., Arslan, E., Xing, A., Xiangli, Y., ... & Sridhar, S. (2024). Turbo-gs: Accelerating 3d gaussian fitting for high-quality radiance fields. arXiv preprint arXiv:2412.13547.

Questions for authors
  1. I am curious about the performance when directly compared with other 4K feed-forward reconstruction methods (e.g., LGTM) and DA3 itself in terms of reconstruction quality, inference speed, GPU memory usage, number of Gaussians, and storage requirements. Additionally, it would be valuable to compare with per-scene optimization methods, such as Vanilla 3DGS and Turbo-GS, to see if the proposed method can achieve comparable rendering quality.

  2. How does the proposed method perform on widely used 3D reconstruction datasets, such as DL3DV 4K, and how does the proposed method compare with other baselines when directly evaluated on unseen datasets, such as Mip-NeRF 360 4K?

  3. Does the proposed method support variable resolutions during inference? If so, it would be best to provide a plot showing how rendering metrics vary with different resolutions, and include a comparison with AnySplat.

  4. How about the performance under multi-view setups, for instance, using 4, 8, or 16 views?

  5. Would the AnySplat-LR baselines achieve better performance if their input resolution were set to 448, consistent with the training resolution?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.