acceptodds
Under review as a conference paper at ICLR 2027

GeoRank: Geometry-Aware Low-Rank Compression for Efficient 3D Reconstruction

Abstract

Feed-forward visual geometry transformers have advanced 3D reconstruction, but their large backbones incur substantial computational and memory costs. Existing efficiency methods primarily reduce redundancy in spatial tokens; we instead investigate a complementary and largely overlooked source: channel-space redundancy. We find substantial low-rank structure in both encoder representations and decoder operators, but its geometric importance is non-uniform across tasks and not fully captured by activation energy alone. Motivated by this, we introduce GeoRank, a geometry-aware low-rank compression framework for feed-forward 3D reconstruction. In the calibration stage, GeoRank first conducts activation-aware factorization to construct low-rank operators that better preserve the model’s effective feature space. Since activation energy does not fully capture geometric importance, we further redistribute a rank budget according to geometric sensitivity. During training, to handle variation beyond static compression, we introduce lightweight adaptive refinement that adjusts the compressed representation to each input and selectively restores information lost during truncation. Geometry-sensitive regularization further stabilizes geometry-critical directions. Furthermore, the factorized representation naturally extends to streaming inference by caching historical features in compact rank space. Benchmarking results demonstrate that GeoRank substantially reduces model cost while retaining strong reconstruction quality. Compared with the full model, it reduces decoder parameter size by 45.7% and GFLOPs by 31.0%, while improving 7-Scenes point-map accuracy from 0.055 to 0.035 and completion from 0.090 to 0.050. The same compact representation also extends effectively to causal reconstruction with strong online performance. Overall, GeoRank establishes channel-space low-rank compression as a complementary direction for building efficient feed-forward and streaming 3D reconstruction models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Reconstruction, Low-Rank Compression, Geometry-Aware Representation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2DdGOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents GeoRank, a geometry-aware low-rank compression framework for feed-forward 3D reconstruction models. The paper observes that geometry transformers exhibit substantial channel-space redundancy, while different geometric tasks have different sensitivity to rank truncation. Based on this observation, GeoRank combines activation-aware low-rank factorization with geometry-aware rank allocation, followed by adaptive refinement and geometry-sensitive regularization. The framework is also extended to streaming reconstruction through rank-space KV caching. Experiments on VGGT and π³X show reduced model size and computation while maintaining competitive or improved reconstruction performance.

Critical strengths
  1. Geometry-specific analysis of channel-space redundancy.

    A key strength is the observation that low-rank redundancy in geometry transformers is not uniform across geometric tasks. The paper shows that depth and camera pose respond differently to rank truncation, and that activation energy, despite being strongly correlated with geometry sensitivity, does not fully determine geometric importance. This provides a meaningful motivation for moving beyond purely weight- or activation-based low-rank compression toward geometry-aware capacity allocation.

  2. Strong practical compression results across offline and streaming settings.

    GeoRank achieves a substantial reduction in model cost while retaining competitive geometric prediction. On π³X, the reported reduction is 45.7% in decoder parameters and 31.0% in GFLOPs, and the final compressed/refined model performs competitively with or better than the full model on several reported metrics. The extension to rank-space KV caching further demonstrates that the same low-rank representation can be applied to bounded-memory streaming reconstruction.

Critical weaknesses
  1. It is unclear how well GeoRank compresses an already well-optimized dense model.

    Table 6 shows that fine-tuning the full π³X model substantially improves performance; for example, the 7-Scenes point-map accuracy error improves from 0.0557 to 0.0387. The final GeoRank model achieves 0.0348, but this result is obtained through GeoRank's own compression-and-refinement pipeline. The paper does not report the natural controlled experiment of first fine-tuning the dense π³X model and then applying GeoRank compression/refinement to the same fine-tuned model.

    This makes it difficult to determine how much of the final advantage should be attributed to the compression mechanism itself versus the subsequent GeoRank-specific refinement/training procedure. More importantly, for a compression method, it is important to establish whether the performance of a strong, already fine-tuned dense model can be retained after compression. A controlled comparison starting from the same fine-tuned dense checkpoint would substantially strengthen the claim that GeoRank is effective as a general model-compression framework.

  2. The central geometry-aware rank allocation is evaluated primarily at a single compression operating point.

    GeoRank explicitly formulates compression as preserving geometric prediction under a fixed capacity budget. However, the main comparison between SVD, ASVD, and geometry-aware allocation is conducted essentially at a single matched compression budget. For example, on π³X, the training-free variants are compared at 213.82M decoder parameters and 21,043.7 GFLOPs.

    Since rank allocation under a resource constraint is a central contribution, demonstrating improvement at only one selected operating point does not establish whether geometry-aware allocation consistently provides a better resource-performance trade-off. I would expect a strong rank-allocation method to provide a favorable Pareto frontier across different compression regimes. Rate-distortion or accuracy-efficiency curves over multiple parameter/FLOP budgets, comparing SVD, ASVD, and geometry-aware allocation under matched resources, would provide substantially stronger evidence for the proposed allocation strategy.

Questions for authors
  1. Compression of a fine-tuned dense model

    Table 6 shows a substantial improvement from full π³X to full π³X with fine-tuning (e.g., 0.0557 to 0.0387 in 7-Scenes point-map accuracy error), while the final GeoRank model reaches 0.0348 through its own compression and refinement pipeline.

    Have the authors evaluated the following controlled pipeline?

    \rightarrow \text{Dense Fine-Tuning} \rightarrow \text{GeoRank Compression} \rightarrow \text{GeoRank Refinement}$$ If so, how much of the performance of the fine-tuned dense model is retained after compression? This comparison would help disentangle the effectiveness of GeoRank as a compression mechanism from gains introduced by its subsequent refinement/training procedure. The answer to this question would materially affect my assessment of GeoRank as a general compression method.
  2. Resource-performance trade-off across rank budgets

    Does geometry-aware rank allocation consistently outperform activation-aware allocation across different compression budgets?

    Since GeoRank explicitly optimizes layer-wise ranks under a fixed resource constraint, I would expect an effective rank-allocation method to demonstrate a consistently favorable resource-performance trade-off across multiple operating points rather than only at the reported compression budget.

    Could the authors provide rate-distortion or accuracy-efficiency curves over several rank/parameter/FLOP budgets, comparing at least SVD, ASVD, and geometry-aware allocation under matched resources? In particular, it would be useful to determine whether

    DGeoRank(R)<DASVD(R)D_{\mathrm{GeoRank}}(R) < D_{\mathrm{ASVD}}(R)

    holds consistently over a meaningful range of resource budgets RR, where DD denotes geometric prediction error and RR denotes rank, parameter count, or computational cost.

    Such results would clarify whether geometry-aware rank allocation establishes a consistently better Pareto frontier rather than improving performance primarily around the selected operating point. This would materially affect my assessment because geometry-aware rank allocation is a central contribution of the paper.

Reviewer_t5kaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies and exploits channel-space redundancy in feed-forward visual geometry transformers for efficient 3D reconstruction. The authors observe that intermediate features in models like π³X exhibit a strongly low-rank structure, and crucially, that this low-rank structure is geometry-anisotropic — different geometric tasks (depth, pose, point-map) and even different directions within a task tolerate different degrees of rank reduction. Based on these observations, GeoRank proposes a two-stage framework: (1) a calibration stage that constructs activation-aware low-rank bases via ASVD and allocates rank budgets across decoder layers according to geometric sensitivity using dynamic programming; and (2) a training stage with adaptive scaling (input-conditioned channel reweighting), a gated tail (rank-s residual branch recovering truncated information), and geometry-sensitive Fisher-weighted regularization to stabilize geometry-critical directions. The factorized representation further extends to streaming inference via rank-space KV caching and online distillation. Experiments on 7-Scenes, NRGBD, Co3Dv2, Sintel, TUM-dynamics, and ScanNet show that GeoRank reduces decoder parameters by 45.7% and GFLOPs by 31.0% while improving sparse-view point-map accuracy and maintaining competitive depth and pose estimation.

Critical strengths
  • S1. The geometry-anisotropic low-rank observation is novel and consequential: Prior work on low-rank compression of neural networks treats all output directions uniformly or uses activation energy as the sole importance signal. This paper demonstrates — with concrete evidence in Fig. 2 and Fig. 3 — that (a) the top-256 PCA directions preserve geometry far better than random or bottom-256 directions, and (b) activation energy correlates with but does not fully determine geometric importance (Spearman ρ ≈ 0.82–0.85 with substantial residual variation). This insight directly motivates the geometry-aware rank allocation and is the conceptual core that distinguishes GeoRank from generic low-rank compression. It is likely to influence future work on efficient geometry models.

  • S2. The method achieves strong practical efficiency gains without sacrificing — and in some cases improving — reconstruction quality: A 45.7% parameter reduction and 31.0% GFLOP reduction while improving 7-Scenes point-map accuracy (0.055→0.035 mean, 0.090→0.050 completion) is a compelling result. The extension to streaming reconstruction at ~12 FPS with bounded memory further demonstrates practical utility. The fact that the same low-rank representation serves both feed-forward and causal streaming modes is an elegant unification.

Critical weaknesses
  • W1. Robustness to calibration data and domain shift is under-explored:
    Both the ASVD bases and the dynamic rank allocations are determined once during the calibration stage using 512 sampled batches. However, in practical 3D reconstruction, input distributions (e.g., camera intrinsics, scene depth scales, dynamic objects, indoor vs. outdoor illumination) vary drastically:

    • The authors do not analyze how sensitive the rank allocation and initial bases are to the size and composition of the calibration set.
    • While Adaptive Scaling is intended to mitigate input variation dynamically, there is no cross-domain evaluation to verify whether this lightweight mechanism is sufficient to prevent degradation under out-of-distribution scenarios.
  • W2. Ambiguity regarding the performance gain and fine-tuning fairness:
    In Table 1 and Table 6, the compressed GeoRank model (786M) remarkably outperforms the full π3X\pi^3\text{X} (992M) on 7-Scenes (Acc: 0.055→0.0350.055 \to 0.035). Even the un-tuned calibration baseline (Calib. Init. in Table 6, 0.0421) outperforms the original full model (0.0557).

    • It is counter-intuitive that lossy rank truncation alone leads to an immediate ~24% accuracy boost without training. Does low-rank truncation inherently act as a regularizer/denoiser against high-frequency feature noise, or is there an artifact in the baseline evaluation?
    • Moreover, when comparing with "Full π3X\pi^3\text{X} w/ ft.", it is unclear whether the full model was fine-tuned under the identical multi-task loss recipe (Eq. 6). Disentangling the gains from low-rank compression versus multi-task joint fine-tuning is crucial for a fair assessment.
Questions for authors
  1. Theoretical mechanism behind Calib. Init. outperforming the full model:
    Could the authors provide deeper insights into why the zero-shot truncated model (Calib. Init. in Table 6) achieves significantly lower reconstruction error (0.0421 vs. 0.0557) than the full uncompressed model? Is there empirical evidence of low-rank truncation acting as a geometric denoising filter?
  2. Fine-tuning setup fairness:
    In Table 6, was the "Full π3X\pi^3\text{X} w/ ft." variant trained using the exact same multi-task loss Lgeo\mathcal{L}_{\text{geo}} (Eq. 6) and schedule as GeoRank?
  3. Hyperparameter sensitivity of η\eta:
    In the rank utility formula uℓ,I=ηEℓ,I+(1−η)Gℓ,Iu_{\ell,I} = \eta E_{\ell,I} + (1 - \eta) G_{\ell,I}, what is the default value of η\eta, and how sensitive is the resulting rank allocation to extremes such as η=0\eta = 0 (pure geometric sensitivity) or η=1\eta = 1 (pure activation energy)?
Reviewer_vwPBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates channel-space redundancy, a relatively underexplored direction in the efficiency optimization of feed-forward 3D reconstruction models. The authors observe substantial low-rank structures in both encoder representations and decoder operators. However, the sensitivity to rank reduction varies across geometric tasks, and activation energy alone is insufficient to fully characterize the geometric importance of different feature directions.

Motivated by these observations, the authors propose GeoRank, a geometry-aware low-rank compression framework consisting of two stages. During calibration, the framework constructs compressed operators through activation-aware factorization and allocates the rank budget according to geometric sensitivity to preserve geometry-critical information. During training, adaptive geometry refinement and geometry-sensitive regularization enable the compressed model to accommodate input variations while maintaining geometric prediction quality. Furthermore, the framework is extended to online streaming reconstruction through rank-space KV caching. Experimental results demonstrate that channel-space low-rank compression offers a promising complementary direction for building efficient feed-forward and streaming 3D reconstruction models.

Critical strengths

1.Novel perspective and well-supported motivation. The paper explores channel-space redundancy from a relatively underexplored perspective. The motivation is clearly established through analyses of feature low-rankness and geometric sensitivity, providing a reasonable foundation for the proposed method. The overall methodology is well-motivated, and the paper is logically organized and clearly written.

2.Favorable efficiency–accuracy trade-off. Experimental results demonstrate that GeoRank substantially reduces model parameters and computational costs while maintaining or even improving reconstruction performance on several benchmarks, highlighting its practical value.

3.Comprehensive experimental evaluation. The method is evaluated across multiple 3D reconstruction tasks and benchmarks. Cross-backbone experiments and component-wise ablation studies provide meaningful evidence supporting the effectiveness of the proposed compression strategy and its individual components.

4.Extensibility to streaming reconstruction. GeoRank is not limited to feed-forward 3D reconstruction but naturally extends to online streaming reconstruction through rank-space KV caching, further broadening its applicability.

Critical weaknesses

I do not identify any major weaknesses in this paper. However, some benchmark evaluation protocols are insufficiently documented. For instance, the frame-sampling intervals for 7-Scenes and NRGBD, as well as the input sequence lengths used for video depth estimation on Bonn and KITTI, are not explicitly specified. I encourage the authors to provide these details to facilitate reproducibility and further strengthen the transparency and credibility of the experimental results.

Questions for authors

1.Could the authors provide more details about how the 512 calibration batches were sampled? For example, what are the sampling proportions across different training datasets? Were the samples randomly selected, manually curated, or selected according to specific criteria? Clarifying these details would help improve the reproducibility of the proposed method.

2.In the 7-Scenes reconstruction examples shown in Figure 5, some input views appear to have limited or even no overlapping regions. To my knowledge, reconstructing such low-overlap views is particularly challenging. If a similarly sparse view-sampling strategy is used in the benchmark evaluation, could the improvements reported in Table 1 be primarily driven by these challenging cases? Additionally, the example input images in Figure 4 appear to be from the ETH3D [1] dataset. Would the authors consider providing additional point-map or point-cloud reconstruction results on ETH3D [1] or DTU [2]? Such experiments could further evaluate the generalization capability of GeoRank across different scene types, particularly object-centric scenarios.

3.Have the authors explored applying GeoRank to VGGT-ω\omega [3]? Is the proposed compression strategy also applicable to architectures incorporating register tokens, particularly the feature interactions and linear operators associated with these tokens? If feasible, including VGGT-ω in the cross-backbone evaluation or baseline comparisons would be valuable. I would be particularly interested in understanding how GeoRank performs on VGGT-ω in terms of compression efficiency and geometric prediction quality.

[1] A multi-view stereo benchmark with highresolution images and multi-camera videos.

[2] Large scale multi-view stereopsis evaluation.

[3] Vggt-omega.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.