acceptodds
Under review as a conference paper at ICLR 2027

GeoLens: Where to Prune Transfers Across 3D Foundation Models, How Much Does Not

Abstract

Pruning is a standard way to make large pretrained models cheaper to run, and 3D foundation models need it, above all in robotics and driving. They have become the default front-end for geometry from images, returning depth, point maps and camera pose in one feedforward pass, and on the multiview, streaming and recurrent models the bypassable blocks take a quarter to half of the latency. Yet in a model that emits dense geometry a badly chosen block warps the scene, and to our knowledge whether training-free block pruning is safe there has not been evaluated. GeoLens is the first cross-architecture evaluation of it: eight architectures across four inference paradigms and 26 indoor RGB-D sequences, 208 cells under one depth acceptance test, plus 304 cells on ETH3D, TartanAir and ScanNet, every number regenerated from released per-cell files. It yields a recipe and two limits. Where to prune transfers in domain: one fixed set of five blocks per architecture, taken from unlabeled scoring of its other sequences, passes 94% of cells and matches per-sequence re-scoring cell for cell. An oracle that spent a labeled pass on every block, measuring its damage on the benchmark itself, gains at most about two points on that set (95% interval): one unlabeled choice per architecture suffices, with no retraining. A prior fit on other architectures picks sets that pass 87% of cells in its seven-model pool and 17/26 outside it. How much is the first limit: capacity, the largest bypass ratio sustaining ≥80% acceptance, spans 3.2× across the roster (2.0× under a wider block count) and shifts with the evaluation distribution; out of domain no K=5 set we tried clears the bar on three of eight architectures on ScanNet. It does not follow from the per-block ranking, because single-block safety does not compose into set safety, and no pre-registered structural scalar of six tracks it under every block count. The second limit is the one that matters in deployment: the acceptance test licensing the recipe does not certify the task. On nine cells the acceptance test accepts, trajectory error grows by up to 2.54×, across five of the seven architectures we checked. On the multiview, streaming and recurrent models, bypass saves 5.8–11.5% of wall-clock at K=5 and up to 22.5% at capacity. We release the benchmark, the per-cell results and the pre-registrations.

Primary area:
datasets and benchmarks
Keywords:
3D foundation models, block pruning, training-free compression, benchmark, depth estimation, camera pose, pre-registration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mSmmOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper applies the philosophy of ShortGPT's block-influence score to eight 3D foundation models and bypasses the K least influential blocks. On 208 indoor cells under a relative depth acceptance test, K=5 passes 94% of cells. A fixed set per architecture matches per-sequence re-scoring and a labeled single-block oracle is not statistically separated from it. The paper also reports four further findings with some experimental support.

Critical strengths
  • As far as I know, this is the first cross-paradigm study of block pruning on 3D foundation models.
  • The authors perform a lot of efforts to evaluate with statistics computation and designed controls.
Critical weaknesses
  • The acceptance test does not certify the model. Of the 18 depth-accepted cells with a measured trajectory, 9 exceed 1.25× ATE. The instrument behind the 94% headline therefore fails on half of the cells that were audited. It also ignores pose, cross-view consistency and metric scale, which are what distinguish a 3D foundation model from a depth network.
  • The title over-claims transfer. What transfers across models is the scoring procedure, not the block choice.
  • The "94%" reflects the chosen K and threshold, not the selector. K=5 is exactly the largest K at which the weakest model still clears the 80% bar. On VGGT and π³, any 5 of the 8 safest blocks pass 100%. "Matching the oracle" is a ceiling effect at 94%, and on continuous metrics labeled greedy search does better.
  • The baselines are weak versions of the defaults being tested. Last-K is evaluated without the healing step that the cited LLM finding relies on. Training-free linear replacement (ReplaceMe) is not evaluated at all, so Table 1's verdicts rest on these weakened versions.
  • The practical payoff is small. K=5 saves 5.8–11.5% of wall-clock time, nothing measurable on the three pair models and no activation memory. In contrast, token merging alone saves 38% at V=64. Also all quality results are at V=4, where the bypassed global blocks matter least.
  • There are no detailed comparisons with the existing efficiency methods for 3D Geometric Foundation Models. It would be interesting to see such evaluation results with some representative works (e.g. FastVGGT/Co-Me/LiteVGGT etc) qualitatively and quantitatively.
  • Also this manuscript mainly tests on relatively old base models (e.g. VGGT) and a lot of more modern backbone networks (e.g. Depth Anything 3 or MapAnything) are not measured. This is important to the model generality capability.
Questions for authors

Please see the weakness part.

Reviewer_BpG3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies whether training-free block pruning can be applied to 3D foundation models. Instead of proposing a new pruning method, the authors systematically evaluate existing pruning strategies across eight models with different architectures. The main finding is that where to prune is relatively stable, but how much to prune varies across models and datasets. By selecting five blocks using a simple block-influence score, the method achieves a 94% pass rate on the depth evaluation, and a fixed block selection for each architecture performs as well as re-selecting blocks for individual sequences. The authors also find that removing multiple blocks can have non-additive effects, meaning that pruning more blocks does not always lead to worse performance. Finally, they evaluate actual inference speedups and downstream trajectory accuracy, showing that preserving depth quality does not necessarily guarantee good trajectory estimation. Overall, the paper provides a broad empirical study of block pruning in 3D foundation models and highlights several important limitations of existing pruning strategies.

Critical strengths

I think the main strength of this paper is its comprehensive experimental evaluation. The authors test block pruning on eight different 3D foundation models, covering several architectures and inference settings, which makes the results more convincing than experiments on a single model. I also like that the paper goes beyond simply showing that pruning works. It looks at how pruning decisions transfer across sequences and models, how different blocks interact when removed together, and how much actual speedup can be achieved. The finding that removing more blocks can sometimes improve performance is quite interesting. Another important observation is that preserving depth accuracy does not always preserve downstream trajectory accuracy. I think this is particularly valuable for practical 3D applications. Overall, the experiments are carefully designed, and the authors provide detailed analyses and additional controls to support their findings.

Critical weaknesses

My main concern is the limited methodological novelty. The paper mainly applies an existing block-influence pruning strategy to 3D foundation models rather than introducing a new pruning method. While the experimental study is extensive, I am not fully convinced that the findings are novel enough for ICLR. Another concern is the cross-architecture transfer claim. The strongest results come from transferring pruning decisions across sequences within the same architecture, while the cross-architecture prior shows noticeably weaker performance, especially on the held-out π³ model. This makes the main claim somewhat stronger than what the experiments actually demonstrate. I also find the comparison with the oracle a bit questionable, since selecting individually safe blocks does not necessarily produce the best pruning combination, as the paper itself shows. Finally, the practical benefits are somewhat limited. The speedups at K=5 are relatively modest, and the scoring overhead is not fully considered in the end-to-end efficiency analysis. The downstream evaluation is also based on a small number of trajectories, so it is still unclear how well these findings generalize to real-world 3D systems.

Questions for authors

I have a few questions about the main findings.

First, the paper emphasizes that pruning decisions can transfer across architectures, but the strongest results seem to come from transferring block selections across sequences within the same architecture. The cross-architecture prior performs noticeably worse on the held-out π³ model. Could the authors clarify what exactly is expected to transfer across different architectures and how generalizable this finding is?

Second, I am curious about the oracle comparison. Since the paper shows that removing multiple blocks can have non-additive effects, is an oracle based on individual block damage really a strong reference for evaluating the quality of the selected block set? Third, the reported speedups are relatively modest at K=5, while computing the block-influence scores also introduces some overhead. Could the authors discuss how this overhead affects the overall efficiency, especially in streaming scenarios?

Finally, the paper shows that preserving depth accuracy does not always guarantee good trajectory estimation, but the downstream evaluation is limited to a few sequences. Do the authors expect this issue to become more significant in longer sequences or more challenging real-world settings?

Reviewer_ej9VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper aims to systematically evaluate LLM block-pruning technique (ShortGPT) on 3D foundation models. It evaluate across archietectures and datasets for windows of 4 frames with an acceptance test that pass with at most 25% relative depth error increase and 5% for 25%-inlier ratio drop. For this depth evaluation, it shows the pruning can generalize across sequence, while the tolerance to pruning is different across architecture.

Critical strengths
  1. Comprehensive experiment. Complete evaluation across models and datasets, with details for the experiments.
  2. Honest about limitation. It mentioned that depth metric doesn't fully evaluate the reconstruction, and poses can degrade a lot.
Critical weaknesses
  1. Limited effectiveness of acceptance test. It mainly evaluate depth result and doesn't represent the full reconstruction quality including poses, as they mentioned. Even monocular model may already provide reasonable relative depth evaluation result, one key benefit of these multi-view model is the multi-view consistency and poses, which is missing in most of their evaluation. Also a 4 frame window may not be able to evaluate the generalization on many frames, where the consistency is more challenging there.
  2. Limited practical usage. The pruning saves only 5.8% - 11.5% of the wall-clock at the K=5.
Questions for authors
  1. Is the depths scaled per image / per window for the evaluation?
  2. Could you directly evaluate the final reconstruction of the unprojected depth from poses, instead of only depth?
  3. What would be the result with more frames in each window?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.