acceptodds
Under review as a conference paper at ICLR 2027

GLoD-LVSM: Learning Geometry-Grounded Continuous-LoD Scene Representations for Large View Synthesis Models

Abstract

Large view synthesis models (LVSMs) reconstruct scenes from sparse views and render novel views quickly, but they incur unnecessary storage and rendering costs due to token redundancy. We present GLoD-LVSM, a compact, geometry-grounded scene representation for LVSMs with continuous levels of detail. First, instead of learning a fixed set of scene anchor initializations, GLoD-LVSM learns a continuous field that initializes a flexible number of anchors to match the desired budget. A condenser then aggregates scene information into these anchors, with both the field and condenser trained end-to-end. Second, the condenser infers reference-token and anchor geometry to guide compression through geometry-grounded attention, improving rendering quality and enabling frustum culling during decoding. Compared with existing LVSM scene-compression baselines, GLoD-LVSM achieves the best trade-off between rendering quality and scene token count. Compared with the uncompressed base model, GLoD-LVSM reduces the scene-token count by 33.4% and accelerates rendering by with no quality loss. Under a relaxed quality-loss budget, the reduction increases to 85.5%, with rendering speedup. Further analysis reveals that view-synthesis supervision yields emergent scene depth and self-organized anchors that align with surfaces and adapt their density to local texture complexity. Beyond rendering, we further show that our scene representation also supports scene-semantic understanding, highlighting its potential for broader downstream tasks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Novel View Synthesis, 3D Scene Representation Learning, Scene Token Compression, Geometry-Grounded Representations, Continuous Level of Detail

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZS87OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the redundancy of view-aligned scene tokens in Large View Synthesis Models (LVSMs), where representation size grows with the number of reference views. GLoD-LVSM introduces a compact scene-anchor representation through two main components: (1) a scene-conditioned continuous query field that supports flexible anchor budgets, and (2) a geometry-aware bidirectional condenser that aggregates multi-view information into 3D anchors. The learned anchor positions further enable frustum culling during rendering. Experiments on RealEstate10K and DL3DV demonstrate improved rendering quality–token count trade-offs over existing pruning and condensation baselines.

Critical strengths
  1. The paper addresses a limitation of highway-style LVSMs, in which the size of the scene representation grows with the number of reference views despite substantial cross-view redundancy. By consolidating multi-view observations into a fixed-budget anchor representation, the method allows additional views to improve reconstruction quality without proportionally increasing scene-token storage. This offers a meaningful direction for scaling feed-forward view synthesis beyond view-aligned representations.

  2. The method consistently outperforms pruning and CLiFT-style alternatives under the same backbone and training setup. The improvements across datasets, reference-view counts, and compression budgets demonstrate more effective use of limited scene tokens.

Critical weaknesses

1. Geometry-grounded Representation. The paper's central claim goes beyond improved rendering efficiency: it argues that the learned anchors form a geometry-grounded, scene-centric 3D representation. However, the experiments establish neither the geometric fidelity nor the consistency of this representation. Surface alignment and adaptive anchor allocation are supported mainly by visualizations, while the w/o Geometry ablation removes geometric encoding from both the condenser and decoder.

2. End-to-end Efficiency Gains. Furthermore, the reported speedups establish faster decoding, but not necessarily a more efficient reconstruction-and-rendering pipeline. The method introduces an eight-layer condenser whose computational cost is excluded from the reported rendering comparisons, while the additional benefits of frustum culling are estimated rather than directly measured. Notably, the speedup on DL3DV is only 1.05× at zero PSNR loss, whereas the headline 3.00× result permits a 2 dB quality drop.

3. Novelty of Continuous LoD. The proposed continuous field generates a variable number of anchor queries, but does not directly parameterize the reconstructed 3D scene. Each budget still produces a separately condensed representation, and no evidence establishes meaningful consistency or reuse across LoDs. More importantly, the paper does not show what the continuous parameterization offers beyond simpler variable-length query initialization.

Questions for authors
  1. What evidence establishes that the learned anchors represent coherent 3D scene structure rather than merely providing useful positional cues for attention? Can the authors quantify geometric alignment and consistency, and separately isolate geometry-aware attention in the condenser and decoder?

  2. What does the continuous field enable that simpler variable-length learned queries cannot? A controlled comparison with interpolated learned queries, together with cross-budget consistency analysis, would better establish whether continuous LoD provides a substantive representation-level advantage.

Reviewer_1YvmOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GLoD-LVSM, a compact scene representation for large view synthesis models with continuous levels of detail. The method introduces a scene-conditioned continuous field for flexible anchor initialization and a geometry-grounded condenser that aggregates multi-view information into 3D scene anchors. Experiments on RealEstate10K and DL3DV demonstrate improved rendering quality versus scene-token count trade-offs, along with reduced token counts and faster rendering. The authors also visualize emergent 3D geometry and explore the learned representation for scene classification.

Critical strengths
  1. Effective scene compression with flexible token budgets. The proposed framework addresses the redundancy of view-dependent scene tokens. The continuous anchor initialization and geometry-grounded condensation are well motivated, and the reported quality–token trade-offs are promising.
  2. Solid experimental evaluation. The paper evaluates multiple reference-view counts and compression budgets. In particular, the controlled condenser comparisons in Fig. 5 use the same backbone and training schedule, providing useful evidence for the effectiveness of the proposed design.
Critical weaknesses
  1. The practical efficiency benefits are not fully established. The reported speedups primarily concern rendering, while the additional cost of the continuous field and eight-layer condenser is not clearly accounted for in the main results. Since condensation involves bidirectional attention between reference tokens and anchors, the scene construction overhead could be substantial. Furthermore, scene-token count alone does not fully characterize actual storage and memory costs.
  2. The geometry-grounded representation lacks quantitative validation. The paper claims that meaningful 3D geometry and surface-aligned anchors emerge without explicit geometric supervision. However, the supporting evidence is mainly qualitative. Quantitative depth accuracy, geometric consistency, and culling reliability would strengthen this claim.
  3. The practical scope of continuous LoD needs clarification. The continuous field supports different anchor budgets during scene construction, but it is unclear whether the LoD can be changed efficiently after a scene has been encoded. This distinction matters for the claimed flexibility under varying runtime budgets. In addition, the novelty over simpler budget-conditioned query initialization deserves further investigation.
Questions for authors

Q1. Can the LoD of an already compressed scene be changed without rerunning the condenser on the reference-view tokens? If not, how does the proposed representation support dynamically changing runtime budgets? Q2. Could the authors provide quantitative evaluations of the emergent geometry, such as depth accuracy or multi-view geometric consistency? How sensitive is frustum culling to errors in the predicted anchor positions? Q3. How does the proposed continuous initialization field compare with a simpler budget-conditioned learned-query initialization under the same geometry-grounded condenser? This would help clarify the independent contribution of the continuous-field design. Q4. Section 3.2 distributes the anchor budget uniformly across reference views. How does this design handle unequal view redundancy or scene complexity? Could adaptive per-view budget allocation improve performance, particularly at low token budgets where adding more views can degrade quality (Fig. 8)?

Reviewer_f4JgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GLoD-LVSM, a compact, geometry-grounded scene representation for large view synthesis models that supports flexible scene-token budgets. The method introduces a scene-conditioned continuous field for anchor initialization and a geometry-grounded bidirectional condenser that leverages predicted 3D geometry to guide attention and enable frustum culling. Trained with novel-view synthesis supervision, GLoD-LVSM demonstrates improved rendering-quality versus token-count trade-offs on RealEstate10K and DL3DV compared with existing compression baselines. The authors report a 33.4% token reduction and 1.23× rendering speedup without aggregate PSNR degradation on 10-view RealEstate10K. Additional experiments illustrate emergent geometric structure, adaptive anchor allocation, and the potential of the learned representation for scene-semantic understanding.

Critical strengths
  1. The paper demonstrates consistent improvements in rendering-quality versus token-count trade-offs across RealEstate10K and DL3DV. Importantly, comparisons against random pruning, K-means pruning, learned pruning, and CLiFT-style condensation use the same backbone and training schedule, providing convincing evidence of the proposed condenser's effectiveness. Extensive evaluations across reference-view counts and compression ratios further strengthen the results.
  2. The proposed continuous anchor initialization decouples the scene-token budget from the number of reference views, while geometry-grounded condensation enables efficient multi-view information aggregation and frustum culling. This design addresses a practical scalability limitation of existing LVSMs and offers a useful approach to balancing rendering quality, representation compactness, and computational efficiency.
Critical weaknesses
  1. The claim that meaningful 3D geometry emerges solely from novel-view synthesis supervision is not convincingly established, as the frozen LagerNVS encoder may already contain substantial geometric information from 3D-supervised pretraining. Moreover, the convex aggregation in Eq. (4) does not guarantee surface-aligned anchors, and the geometric evidence is primarily qualitative. Independent geometric evaluations and appropriate encoder-pretraining controls are needed to substantiate this central claim.
  2. The continuous-LoD formulation appears to require re-running condensation when changing anchor budgets, making its advantages over existing variable-budget methods unclear. Additionally, the reported rendering speedups exclude scene-reconstruction costs, while culling-enabled FPS is estimated rather than directly measured on variable-length workloads. The ablations also fail to isolate geometry-aware condensation from geometry-aware decoding. More comprehensive efficiency benchmarks and component-specific ablations are necessary to establish the method's practical and technical advantages.
Questions for authors
  1. How much of the predicted geometry originates from the proposed condenser rather than the 3D-pretrained LagerNVS encoder? Could the authors provide quantitative geometric evaluations and appropriate encoder-pretraining controls to substantiate the claimed emergence of geometry?
  2. Could the authors report the complete scene-reconstruction latency, peak GPU memory, and directly measured rendering FPS with frustum culling? How many target views are required to amortize the additional condensation cost relative to LagerNVS-FT?
  3. Could the authors separately ablate geometry-aware attention in the condenser and decoder, rather than removing both simultaneously? An additional ablation of the anchor geometry aggregation in Eq. (4) would help clarify which components are primarily responsible for the improvements.
  4. Does changing the anchor budget require re-running the condenser, or can the existing scene representation be progressively refined? Could the authors clarify the advantages over existing variable-budget methods and provide comparisons against simpler resizable learned-query baselines?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.