acceptodds
Under review as a conference paper at ICLR 2027

TraDE: Trading Cluster Distortion for Efficient Visual Geometry Transformers

Abstract

Visual geometry foundation models enable powerful feed-forward 3D reconstruction, but their reliance on global attention leads to rapidly growing computational costs for long input sequences. Existing training-free token reduction methods alleviate this cost, yet typically determine token selection and retention through predefined regular structures, overlooking the heterogeneous redundancy of token representations. We observe that token similarity is highly non-uniform across input sequences and spatial regions and dynamically evolves with network depth. Based on this observation, we propose TraDE, a training-free framework that adaptively determines both which tokens to retain and how many are needed according to their underlying representations. TraDE formulates token reduction as iterative clustering and uses cluster distortion to balance representation fidelity against the number of retained tokens, while stage-wise repartitioning accommodates evolving representations across depth. Extensive experiments across multiple visual geometry foundation models, datasets, and geometric tasks demonstrate favorable performance–efficiency trade-offs, achieving up to inference acceleration on long sequences. Code is provided in the Supplementary Material.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Visual Geometry Transformers, Token Reduction, Token Clustering, 3D Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Hke1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces TraDE, a training-free token reduction method designed to accelerate global attention in visual geometry transformers, including VGGT, Pi-3 and VGGT-Omega. TraDE formulates token reduction as an iterative clustering process, merging clusters within local spatio-temporal windows when the resulting increase in cosine-based distortion remains below a predefined threshold. This enables adaptive token reduction based on input characteristics rather than a fixed token budget. During global attention, only one representative token per cluster is processed, and its output is subsequently propagated to the remaining cluster members. Cluster assignments are updated at manually selected stage boundaries. Experiments demonstrate speedups of up to 7.25×, 8.05×, and 4.47× for VGGT, Pi-3 and VGGT-Omega respectively, on 1000-frame ScanNet sequences, with additional evaluations on 7Scenes, NRGBD, ETH3D and TUM-Dynamics.

Critical strengths
  • The developed method is training-free and is shown on three backbones so it is plug-and-play.
  • The target task is important and useful to a lot of down-streamed applications.
  • The authors evaluate widely across different backbone networks and different datasets.
Critical weaknesses
  • Table 4's gains come mainly from repartitioning, which keeps ≈39% of tokens in the middle layers vs. ≈14% elsewhere. A fixed per-stage ratio with the same schedule is the missing control. Without it, the gains cannot be credited to input-adaptivity.
  • TraDE is evaluated at a single operating point against baselines at their defaults. Moreover, "Ret." is a linear average while attention cost is quadratic. Specifically, at the "same" 26.69% retention, TraDE runs at 15.46 FPS vs. 18.4 FPS for the fixed variants. Accuracy-vs-latency Pareto curves are needed.
  • On ScanNet, TraDE has the lowest Pi-3 AUC@3 of all methods at 300-1000 frames (e.g., 17.89 vs. 23.57 dense), which is the setting behind the "8.0×" headline. On NRGBD, AUC@3 falls from 83.60 to 67.80, showing a single \lambda does not transfer across backbones.
  • The conceptual novelty appears somewhat limited. The objective resembles DP-means/facility location, while the merging criterion reduces to a size-weighted cosine threshold. Stage-wise repartitioning and spatio-temporal search are also related to AVGGT and HTTM. Several relevant works are missing, including DP-means, ToMe, ToMeSD, Agglomerative Token Clustering and Run-Length Tokenization. Although input-adaptive token merging may be new to geometric foundation models, the broader token-merging literature is already extensive, making the methodological novelty less compelling.
  • Hand-crafted design choices weaken the central premise. Stage boundaries are manually selected for each backbone and noticeably affect performance (AUC@3: 14.42 vs. 15.28). Hyperparameters also appear to be tuned on 7Scenes, which is used for evaluation. Moreover, repartitioning at every layer performs substantially worse, suggesting that cluster distortion may not be a reliable proxy for geometric error.
  • I think the authors miss some relevant baselines. AVGGT (CVPR 2026) supplies the stage analysis TraDE builds on and is training-free on VGGT/Pi-3, reporting ≈8-10× at 800 frames on 7Scenes, yet it is not compared. LiteVGGT (CVPR 2026) uses the same ScanNet-50/7Scenes/NRGBD protocol with released code, and at least its training-free cached-merging variant should be compared. The same applies to HeSS (CVPR 2026), the head-adaptive successor of the SparseVGGT baseline already used. Spark3R (arXiv 2605.06270, evaluated on VGGT-Ω) and GoToHunt (arXiv 2605.23892) should at least be discussed and hopefully be compared against.
  • Reproduced baselines deviate from published numbers. Under FastVGGT's own ScanNet-50 protocol (Table 8), FastVGGT's published ≈10% CD gain over VGGT* at 1000 frames (0.471→0.425; LiteVGGT reports 0.485→0.436) vanishes (0.467→0.469). ATE and ARE are also systematically off, while RPE matches the FastVGGT paper almost exactly. FastVGGT's measured speedups fall below every published figure (7Scenes 1.87× vs. 2.74–3.65×; ScanNet-50 at 1000 frames 3.39× vs. 4.0–4.9×), and these gaps all favor TraDE.
Questions for authors

Please see the weakness

9 comments
AuthorsOpenReview ↗

Response to Weaknesses

W1. We thank the reviewer for the careful examination of our ablation studies. First, we emphasize that the token-similarity heterogeneity identified in Sec. 3.1 includes not only input-wise non-uniformity but also depth-wise dynamics. Accordingly, the higher retention ratio in intermediate layers (≈39% versus ≈14% in other stages) is itself a manifestation of TraDE's adaptivity, rather than a manually specified allocation. While stage-wise repartitioning determines when token partitions are updated, distortion-guided adaptive stopping determines how many tokens to retain according to the current representations. Moreover, Table 4 demonstrates the benefit of cluster-level distortion, improving AUC@3 from 13.88 to 14.28 over cosine-based pairing at the same 26.69% retention. To further isolate the contribution of adaptive retention, we introduce ΔE-Stage-Fixed in Appendix Table 13, which employs the same distortion-based clustering and stage-wise repartitioning as TraDE, but uses fixed retention ratios matched to TraDE's average retention at each stage. Under the same overall retention ratio of 26.69%, TraDE improves AUC@3/15/30 from 15.04/63.41/78.81 to 15.28/63.81/79.03, with comparable throughput (15.46 vs. 15.85 FPS). These results demonstrate that input-adaptive retention provides additional accuracy benefits beyond stage-wise repartitioning and predefined depth-dependent token allocation.

W2. We thank the reviewer for the valuable suggestion regarding computational efficiency and the fairness of performance comparisons. First, we acknowledge that the average token retention ratio (Ret.) does not directly reflect computational cost, since attention complexity depends quadratically on the number of retained tokens, while repartitioning also introduces additional overhead. Nevertheless, we report Ret. as an intuitive measure of token reduction. Second, TraDE has already been evaluated at multiple operating points in Appendix Table 5, where varying the trade-off coefficient λ\lambda demonstrates its flexibility in balancing geometric accuracy and inference efficiency. Furthermore, we have added an accuracy–efficiency trade-off curve in Appendix Fig. 5, comparing TraDE across different λ\lambda values with existing baselines on 7Scenes. The results demonstrate that TraDE achieves a more favorable accuracy–efficiency trade-off across a wide range of operating points. For instance, at λ=0.04\lambda=0.04, TraDE achieves higher AUC@3 than FastVGGT (15.28 vs. 14.98) while providing substantially greater acceleration (3.95× vs. 1.87×). Similarly, at λ=0.10\lambda=0.10, TraDE achieves comparable AUC@3 to HTTM (12.97 vs. 12.92) with significantly higher acceleration (5.59× vs. 2.92×). These comparisons further substantiate the effectiveness of TraDE beyond a single operating point.

W3. We thank the reviewer for carefully examining the results across different backbones and datasets. As an inference acceleration method, TraDE should be evaluated in terms of both geometric accuracy and computational efficiency. We acknowledge that, without any backbone-specific hyperparameter adjustments, TraDE achieves lower AUC@3 than FastVGGT on π³. However, this accuracy trade-off is accompanied by substantial efficiency gains. On ScanNet with 300–1000 input frames, TraDE exhibits an average relative AUC@3 reduction of 8.48% compared with FastVGGT, while achieving 2.04× higher inference speed and running 1.48× faster than HTTM, the fastest competing method. At 1000 frames, these advantages increase to 2.53× and 1.62×, respectively. Moreover, Table 2 shows that TraDE maintains competitive point-cloud reconstruction quality, with differences from FastVGGT of at most 0.001 in Acc/Comp and 0.003 in NC across these settings.

Similarly, on NRGBD, TraDE reduces AUC@3 from 83.60 to 67.80 relative to dense π³ (18.90%), while achieving 5.96× acceleration, corresponding to 2.11× and 1.56× higher inference speed than FastVGGT and HTTM, respectively. Meanwhile, AUC@30 decreases by only 2.25% relative to dense π³, and the point-cloud reconstruction metrics remain competitive. These results demonstrate a favorable accuracy–efficiency trade-off, although the degradation in AUC@3 also indicates that a unified λ\lambda may favor more aggressive acceleration on certain backbone–dataset combinations. To further investigate this trade-off, we conducted an additional experiment on π³ using a smaller λ=0.02\lambda=0.02 on NRGBD, as reported in Appendix Table 12. Reducing λ\lambda from 0.04 to 0.02 improves AUC@3 from 67.80 to 77.55 and AUC@30 from 95.66 to 97.00, while maintaining a 3.96× inference speedup.

AuthorsOpenReview ↗

Response to Weaknesses

W4. We thank the reviewer for highlighting these connections. We provide a detailed clarification of our methodological contributions below and will incorporate the key distinctions into the revised manuscript. We acknowledge that our cluster-merging objective follows the penalized-clustering principle of DP-means and facility-location formulations, while agglomerative clustering has also been explored in ATC. However, our contribution extends beyond these existing principles to an efficient constrained agglomerative merging algorithm for visual geometry Transformers. Specifically, TraDE evaluates the aggregate distortion change of participating clusters to guide representative selection, parallel merging of disjoint pairs, and adaptive termination, with incremental cluster statistics enabling efficient updates. More importantly, these mechanisms are integrated into a unified training-free framework that adaptively determines both which tokens to retain and how many, according to the evolving token representations across inputs and network depth.

Regarding AVGGT and HTTM, we have explicitly acknowledged their influence in the manuscript (Line 209 and Lines 215-216). Although AVGGT reveals the stage-wise characteristics of geometric Transformers, it still relies on predefined structured K/V subsampling, whereas TraDE determines token retention from representation redundancy through distortion-guided clustering. HTTM performs merging within predefined spatio-temporal blocks, while TraDE dynamically inherits candidate relations during merging, allowing clusters to progressively extend beyond their initial neighborhoods (Lines 215–216). TraDE shares the fundamental principle underlying ToMe, ToMeSD, and Run-Length Tokenization: exploiting redundancy among similar visual tokens to reduce computation. However, TraDE differs from these methods in its specific token-reduction procedure and the target models and tasks it addresses. While Run-Length Tokenization exploits pixel-level temporal redundancy before Transformer inference, TraDE performs adaptive clustering on intermediate token representations. Our contribution is not merely input-adaptive token merging, but a distortion-guided clustering procedure that accommodates heterogeneous redundancy across input sequences, spatial regions, and network depth. This enables clusters with different sizes and representatives to emerge within the same input, while stage-wise repartitioning further updates both cluster assignments and retention levels as representations evolve.

W5. Thank you for these comments. We clarify the rationale behind our stage-wise design, hyperparameter selection, and cluster distortion. First, our stage boundaries follow AVGGT's stage-wise observations, further supported by Appendix Fig. 9, which shows distinct attention patterns across network depth. Uniform repartitioning may reuse token partitions across stages despite changes in token representations, whereas our stage-wise strategy updates partitions at these transitions, explaining its superior performance in Table 7. Second, Appendix Table 5 and Fig. 5 demonstrate favorable accuracy–efficiency trade-offs across different λ\lambda values. Increasing λ\lambda from 0.04 to 0.05/0.06 improves acceleration from 3.95× to 4.44×/4.88×, while maintaining competitive AUC@30 (79.03 vs. 78.88/78.70). We select λ=0.04\lambda=0.04 as a balanced configuration and consistently apply the same hyperparameters across all datasets and backbones. Finally, cluster distortion measures representation discrepancy rather than downstream geometric error, as explicitly acknowledged in Appendix B.3 (Lines 861–871). Therefore, repartitioning at every layer does not necessarily improve geometric accuracy. Our stage-wise strategy instead balances adaptation to evolving representations against computational overhead.

AuthorsOpenReview ↗

Response to Weaknesses

W6. Thank you for highlighting these relevant methods. Our original baseline selection considered both methodological relevance and implementation availability. At the time of the original experiments, we had not obtained official implementations of AVGGT, HeSS, or Spark3R. Moreover, the complete LiteVGGT pipeline incorporates fine-tuning and FP8 quantization, differing from our training-free setting. Given the extensive recent literature and limited space, we initially selected representative baselines covering token merging and sparse attention. Nevertheless, we agree that broader comparisons would strengthen our evaluation.

To address this concern, we have reproduced AVGGT and conducted additional comparisons with AVGGT, LiteVGGT, and GoToHunt, as reported in the Appendix Table 14. Specifically, AVGGT is reproduced with σ=4\sigma=4, while LiteVGGT is evaluated without fine-tuning or FP8 quantization to ensure a fair training-free comparison. On 7Scenes, TraDE achieves the highest inference efficiency (3.95×, 15.46 FPS), compared with 3.30×, 3.41×, and 3.50× for AVGGT, LiteVGGT, and GoToHunt, respectively. TraDE also achieves the highest AUC@3 (15.28) among these accelerated methods while maintaining competitive reconstruction quality. Notably, GoToHunt's reported runtime excludes MegaLoc preprocessing.

We will also expand the Related Work section to discuss all the suggested methods. Since AVGGT, LiteVGGT, and Spark3R have already been discussed in the main text, we briefly clarify the remaining methods below. HeSS allocates head-wise sparse-attention budgets based on offline sensitivity calibration, whereas TraDE performs distortion-guided token clustering to determine retention directly from current representations without additional offline calibration. GoToHunt combines diversity-based reference-frame selection with layer-dependent K/V subsampling, relying on predefined compression budgets. In contrast, TraDE adaptively determines representative tokens and retention ratios through cluster-distortion-guided merging, while stage-wise repartitioning accommodates evolving representations across network depth, without requiring an external place-recognition model or offline profiling.

W7. Thank you for raising this concern. We re-examined the FastVGGT evaluation in Appendix Table 8. All three methods were evaluated on identical scene inputs using the same metric definitions and timing procedure. Our measurements on an NVIDIA RTX PRO 6000 GPU show that FastVGGT and DenseVGGT took 84.86 s and 287.37 s per scene on average, respectively, yielding a 3.39× speedup. Differences in GPU hardware and runtime environments likely contribute to the discrepancy with the originally reported speedup. For geometric evaluation, our CD computation is consistent with FastVGGT's officially released evaluation code, following the same procedure for full-cloud bounding-box alignment, point sampling, voxel downsampling, and clipped bidirectional distances, without any modifications to these evaluation steps. For ATE and ARE, our evaluation logs confirm that Appendix Table 8 consistently uses Sim(3) alignment for all three methods. We further verified that EVO 1.37.0 rejects the simultaneous similarity and origin alignment requested by FastVGGT's released script, which may explain the discrepancies in ATE and ARE relative to its published results. Importantly, these discrepancies are likely attributable to differences in evaluation environments and dependencies, rather than inconsistent evaluation among the compared methods. All methods in our experiments use the same official VGGT checkpoint, identical inputs, and a unified evaluation protocol. Therefore, despite differences from the absolute results reported in FastVGGT, our comparisons remain fair and controlled, providing a reliable assessment of TraDE's accuracy–efficiency trade-off. Furthermore, we have already provided our implementation and evaluation code in the supplementary material, enabling independent verification and reproduction of our reported results.

Reviewer_cL8KOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

TraDE is a training-free framework for accelerating visual geometry Transformers through adaptive token clustering. It balances representative-based cluster distortion against the number of retained tokens, uses local spatio-temporal candidate search to limit clustering overhead, and recomputes token partitions at predefined stages. Experiments across three backbones demonstrate substantial inference acceleration. However, the comparisons do not fully establish the accuracy–efficiency frontier, and the ablations do not isolate the contribution of adaptive retention from stage-wise repartitioning.

Critical strengths
  1. Substantial practical efficiency gains. TraDE integrates with three visual geometry backbones, VGGT, π3\pi^3, and VGGT-Ω\Omega,and achieves the largest speedups in the main comparisons, with gains generally increasing with sequence length, reaching 8.05×8.05\times on π3\pi^3 at 1000 frames. The evaluation re-times the compared methods on the same hardware, describes synchronized forward-pass timing, and reports clustering overhead.

  2. Input-dependent token retention and broad evaluation. The distortion-based criterion, ΔE<2λ\Delta E < 2\lambda, allows the retained-token count to vary with the input representations rather than imposing a fixed retention ratio. Figure 4 illustrates sequence-dependent retention and non-uniform cluster sizes. The evaluation spans five datasets, multiple geometric tasks, different sampling strides, and an alternative full-sequence sampling protocol.

Critical weaknesses
  1. The comparative accuracy–efficiency frontier remains insufficiently established. The main tables compare methods at different operating points. Although TraDE offers substantial acceleration and some favorable accuracy–speed combinations, these results do not establish whether it consistently outperforms competing methods at matched latency or accuracy. FastVGGT often preserves higher fine-grained pose accuracy while running more slowly, leaving the relative trade-off unclear. This matters particularly for π3\pi^3: on ScanNet, TraDE reduces AUC@3 by 4.01, 5.31, and 5.68 points at 300, 500, and 1000 frames, respectively. The average ScanNet drop of 1.42 points reported in Sec. 4.2 masks this backbone-dependent variation. Table 12 additionally shows a reduction from 83.60 to 67.80 on NRGBD. Accuracy-versus-speed curves are needed to establish the comparative trade-off.

  2. The ablations do not isolate the benefit of adaptive retention. In Table 4, ΔE\Delta E-Fixed and the full method differ in both adaptive retention and stage-wise repartitioning. Meanwhile, ΔE\Delta E-Adaptive and the full method differ in both repartitioning and the resulting retention rate. Thus, the improvement cannot be attributed specifically to input-adaptive retention rather than recomputing partitions or allocating different token budgets across depth. A fixed-retention variant with the same repartitioning schedule and matched stage-wise token budgets would provide a cleaner comparison. Moreover, ΔE\Delta E-Fixed is faster than the full method despite identical reported average retention—18.41 versus 15.46 FPS—highlighting the need to compare measured runtime alongside retention.

  3. Novelty and configuration requirements need clearer positioning. Training-free agglomerative token clustering has already been studied in ATC [1], which is not discussed. TraDE’s representative-based distortion objective, token-count penalty, and adaptive stopping rule provide distinctions, but their contribution relative to prior clustering approaches should be explained more clearly. Furthermore, although token retention is input-adaptive, repartitioning boundaries are predefined for each backbone. Table 7 shows that uniform and proposed stage-wise schedules have nearly identical retention and throughput but differ by 0.86 AUC@3. The parameter analyses are also conducted on 7Scenes, which is an evaluation dataset; the paper should clarify whether parameter selection used disjoint validation sequences.

Questions for authors

Questions

  1. Matched operating points. Can you provide accuracy-versus-speed curves comparing TraDE’s λ\lambda sweep with FastVGGT’s merge-ratio sweep and HTTM’s settings, particularly on π3\pi^3? Matching measured forward-pass latency would be more informative than matching retention alone.

  2. Isolating adaptivity. Could you add a ΔE\Delta E-Fixed variant with the same stage-wise repartitioning boundaries as TraDE and stage-specific fixed retention ratios matching TraDE’s average retention at each stage? This would help isolate input-adaptive retention from repartitioning and depth-dependent token allocation.

  3. Dense-layout restoration and attention weighting. In Algorithm 1, when restoring the dense layout through yi←yrGy_i \leftarrow y_{r_G}, does each cluster member receive the representative’s full block output, including its residual stream, or only the attention update added to that member’s original residual? Since cluster sizes are non-uniform and each representative acts as a single key, did you evaluate size-aware attention, such as proportional attention in ToMe [2]?

  4. Stage selection and validation protocol. How should stage boundaries be selected for a new backbone without ground-truth evaluation? How sensitive are the results to boundary placement outside 7Scenes, and were the hyperparameters selected using sequences disjoint from those used for reporting results?

  5. Relationship to prior clustering methods. How does TraDE’s distortion objective and adaptive stopping criterion differ from prior agglomerative token clustering such as ATC [1]? Which gains arise from these choices, as opposed to local spatio-temporal candidate search and stage-wise repartitioning?

References

[1] Haurum et al. Agglomerative Token Clustering. ECCV 2024.

[2] Bolya et al. Token Merging: Your ViT but Faster. ICLR 2023.

4 comments
AuthorsOpenReview ↗

Response to Weaknesses

W1. We thank the reviewer for highlighting the need for more comprehensive accuracy–efficiency comparisons. We acknowledge that the overall average reported in Sec. 4.2 does not fully reflect the backbone-dependent accuracy variations, particularly on π3\pi^3. We will clarify these differences in the revised manuscript. Regarding comparisons at matched computational budgets and accuracy levels, please refer to our detailed response to Q1, which addresses the requested accuracy–speed trade-off analysis.

W2. We thank the reviewer for this careful observation. We agree that the original Table 4 does not fully isolate input-adaptive retention from stage-wise repartitioning. To address this concern, we introduce ΔE-Stage-Fixed in Appendix Table 13, providing a controlled comparison under matched stage-wise retention budgets. The results confirm the additional benefits of input-adaptive retention beyond stage-wise repartitioning. Please refer to our response to Q2 for the detailed experimental settings and results.

Regarding the runtime discrepancy, Ret. is a linear average, whereas attention complexity scales quadratically with the number of retained tokens. We report Ret. as an intuitive measure of token reduction and additionally provide FPS as an equivalent indicator of inference runtime, rather than relying solely on retention ratios.

W3. We thank the reviewer for highlighting these concerns. We acknowledge that the connection to ATC and the rationale behind our configuration choices require clearer explanations. We will expand the Related Work discussion to clarify our methodological contributions relative to ATC. We also acknowledge that stage boundaries are manually determined from attention-pattern analysis and that a separate validation split was not used for hyperparameter selection. Please refer to our responses to Q4 and Q5 for detailed explanations of the stage-selection procedure, parameter settings, and additional controlled ablations isolating the contributions of distortion-guided merging and adaptive stopping.

AuthorsOpenReview ↗

Response to Questions

Q1. We thank the reviewer for this valuable suggestion. To provide a more comprehensive accuracy–efficiency comparison, we have added accuracy–speed trade-off curves on 7Scenes using both VGGT and π³, comparing TraDE's λ\lambda sweep with existing baselines, as shown in Appendix Figs. 5 and 6.. For π³, we additionally evaluate FastVGGT across different merge ratios and HTTM under different configurations. All methods are evaluated under the same protocol, with efficiency measured by actual inference speedup rather than token retention ratios.

The results demonstrate that TraDE achieves more favorable accuracy–efficiency operating points than the competing methods. For example, on π³, TraDE achieves approximately 19.3 AUC@3 at 3.6× acceleration, compared with FastVGGT's 19.17 AUC@3 at 2.70×. More interestingly, further adjusting λ\lambda enables simultaneous improvements in both accuracy and efficiency. At λ=0.05\lambda=0.05, TraDE achieves approximately 19.7 AUC@3 with nearly 5× acceleration, outperforming its default configuration (λ=0.04\lambda=0.04, 19.52 AUC@3 at 4.24×) in both metrics. This operating point also surpasses the evaluated FastVGGT and HTTM configurations in accuracy and inference speed. These results demonstrate that TraDE's advantages extend beyond a single operating point, providing further evidence of its favorable accuracy–efficiency trade-off across different backbones.

Q2. We thank the reviewer for this valuable suggestion. To isolate the contribution of input-adaptive retention, we introduce ΔE-Stage-Fixed in the new Appendix Table 13. This baseline employs the same distortion-based clustering and stage-wise repartitioning boundaries as TraDE, but uses fixed retention ratios matched to TraDE's average retention at each stage. Under the same overall retention ratio of 26.69%, TraDE improves AUC@3/15/30 from 15.04/63.41/78.81 to 15.28/63.81/79.03, with comparable throughput (15.46 vs. 15.85 FPS). These results demonstrate that input-adaptive retention provides additional accuracy benefits beyond stage-wise repartitioning and predefined depth-dependent token allocation.

Q3. We thank the reviewer for these questions. Regarding dense-layout restoration, each cluster member receives only the representative's projected attention update, which is added to its own original residual stream, rather than receiving the representative's full block output. The subsequent FFN and residual connection are then applied independently to each token in the restored dense layout. Thus, tokens within the same cluster share the attention update while preserving their individual representations.

Regarding attention weighting, TraDE uses standard scaled dot-product attention without explicitly accounting for cluster sizes. To investigate this design, we additionally evaluated ToMe-style proportional attention [2] by adding log⁡∣Cj∣\log |C_j| to the attention logits of each representative key jj. Compared with the unweighted baseline under identical settings on 7Scenes, proportional attention decreases AUC@3, AUC@30, and F1@5cm by 0.40, 0.11, and 0.50 percentage points, respectively. These results indicate that explicitly weighting attention by cluster size does not improve geometric performance in our setting, supporting our choice of standard attention.

AuthorsOpenReview ↗

Response to Questions

Q4. We acknowledge that our current stage-boundary selection relies on prior knowledge of attention behavior rather than an automated procedure. In fact, AVGGT similarly identifies stage transitions through empirical analysis of layer-wise attention patterns, without providing an automated boundary-selection criterion. Following this approach, we provide layer-wise global attention visualizations in Appendix Fig. 9 to illustrate the basis for our stage division. For a new backbone, a straightforward approach is to examine the concentration of attention probability distributions across layers and identify stage boundaries where these patterns exhibit substantial changes. Importantly, this process relies only on the model's internal attention distributions and does not require geometric ground-truth annotations.

To further examine the effect of stage-boundary placement beyond 7Scenes, we conducted an additional comparison between stage-wise and uniform repartitioning on NRGBD. The stage-wise strategy achieves higher AUC@3/15/30 (70.72/91.66/95.63 vs. 69.09/91.35/95.47), F1 (0.9366 vs. 0.9319), and NC (0.8820 vs. 0.8803). This is consistent with the trend observed on 7Scenes, where stage-wise repartitioning also achieves superior geometric accuracy. This improvement can be attributed in part to better alignment with stage transitions, where changes in attention patterns suggest evolving token similarity structures. In contrast, uniform repartitioning may reuse cluster assignments across these transitions, failing to reflect the updated token similarities. These results support the effectiveness of stage-wise repartitioning for preserving geometric accuracy across datasets.

We acknowledge that a separate validation split was not used for hyperparameter selection. Nevertheless, the same hyperparameters ((\lambda=0.04), (t=4), and (\rho=2)) were consistently applied across all datasets and backbones without dataset-specific tuning. Furthermore, the sensitivity analyses in Appendix Tables 5–6 demonstrate favorable accuracy–efficiency trade-offs across a range of parameter settings, suggesting that the effectiveness of TraDE is not restricted to a single carefully selected configuration. We will clarify the parameter-selection procedure and this limitation in the revised manuscript.

Q5. We thank the reviewer for highlighting the connection to prior agglomerative token clustering methods. TraDE differs from ATC in its specific merging objective and optimization procedure. ATC employs conventional linkage criteria (single, complete, or average) to merge similar clusters until a prescribed token count is reached, representing merged clusters through weighted averaging. In contrast, TraDE evaluates the aggregate distortion increase of representing all cluster members with an existing representative token, jointly determining merging pairs and representative selection. More importantly, the distortion increase is explicitly balanced against the reduction in retained tokens, yielding an objective-guided stopping criterion that adaptively determines the final retention ratio. Furthermore, incremental cluster statistics and parallel merging of mutually selected, disjoint pairs enable efficient clustering for long multi-view sequences.

To isolate these contributions, we introduce additional controlled ablations in Appendix Table 13, keeping the spatio-temporal candidate search and stage-wise repartitioning boundaries identical. Specifically, ΔE-Global-Fixed applies a uniform fixed retention ratio across all stages, whereas ΔE-Stage-Fixed uses predefined stage-specific retention ratios matched to TraDE-full's average retention at each stage. Cosine-Stage-Fixed adopts the same stage-specific budgets but replaces distortion-based pairing with cosine similarity. In contrast, TraDE-full employs distortion-guided merging with adaptive stopping, allowing retention ratios to vary across both inputs and network depth. All variants maintain the same overall retention ratio of 26.69%. The results demonstrate the distinct contributions of these designs. First, distortion-based pairing improves AUC@3 from 14.83 to 15.04 over cosine-based pairing under identical stage-specific budgets. Second, ΔE-Stage-Fixed improves AUC@3 from 14.08 to 15.04 over ΔE-Global-Fixed, demonstrating the benefit of depth-dependent token allocation. Finally, TraDE-full further improves AUC@3 from 15.04 to 15.28 by adapting retention to individual inputs. Overall, TraDE-full improves AUC@3 by 1.20 over ΔE-Global-Fixed, demonstrating the combined benefits of adaptive retention across inputs and network depth beyond stage-wise repartitioning alone.

Reviewer_cL8KOpenReview ↗

Rebuttal Response

Thank you for the detailed responses and additional experiments. The accuracy–efficiency curves on 7Scenes provide useful evidence of TraDE's performance across different operating points and backbones, addressing my concern about comparisons at different configurations. The additional fixed-retention ablations in Appendix F are particularly helpful in disentangling the contributions of distortion-guided merging, stage-wise repartitioning, and input-adaptive retention.

The clarification of dense-layout restoration and the additional proportional-attention experiment also address my questions regarding residual handling and cluster-size-aware attention. Furthermore, the discussion of ATC and the controlled ablations help clarify the methodological distinctions and contributions of TraDE.

Regarding stage selection, the attention visualizations and additional NRGBD experiments clarify this point. However, stage-boundary selection remains somewhat heuristic, and the authors acknowledge that no separate validation split was used for hyperparameter selection. These remain minor limitations.

Overall, the additional experiments and clarifications have substantially addressed my original concerns, so I maintain my score as is.

Reviewer_HeUAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TraDE, a training-free token reduction framework for efficient visual geometry Transformers. TraDE uses iterative clustering with a distortion-based objective to adaptively determine which tokens to retain and how many are needed. Stage-wise repartitioning further accounts for changes in token representations across network depth. Experiments across three geometry foundation models and multiple datasets demonstrate substantial inference acceleration.

Critical strengths
  1. Practical efficiency. TraDE achieves substantial inference acceleration across multiple geometry foundation models while maintaining competitive geometric performance. Its training-free design also makes it practical to integrate into existing pretrained models.

  2. Comprehensive ablation studies. The paper provides thorough analyses of its key components, hyperparameters, and repartitioning strategies. These experiments offer useful insights into the design choices and provide convincing evidence for the effectiveness of stage-wise repartitioning.

Critical weaknesses
  1. Limited novelty. The proposed objective combines cluster distortion with a penalty on the number of retained clusters, an idea closely related to existing penalized clustering formulations such as DP-means [1]. Clustering-based token reduction has also been explored in Token Pooling [2], while the spatio-temporal candidate search is explicitly motivated by HTTM [3]. The paper could more clearly distinguish its methodological contributions from these existing approaches.

  2. Insufficient isolation of the proposed contributions. Although the paper provides extensive ablation studies, the benefit of adaptive clustering is not sufficiently isolated from stage-wise repartitioning. Moreover, comparisons with existing methods under matched retention ratios or computational budgets are missing. This makes it difficult to determine whether TraDE consistently achieves a better accuracy–efficiency trade-off than existing approaches.

[1] B. Kulis and M. I. Jordan. Revisiting k-means: New Algorithms via Bayesian Nonparametrics. ICML, 2012. https://icml.cc/2012/papers/291.pdf

[2] D. Marin et al. Token Pooling in Vision Transformers for Image Classification. WACV, 2023. https://openaccess.thecvf.com/content/WACV2023/html/Marin_Token_Pooling_in_Vision_Transformers_for_Image_Classification_WACV_2023_paper.html

[3] W. Wang et al. HTTM: Head-wise Temporal Token Merging for Faster VGGT. CVPR, 2026. https://openaccess.thecvf.com/content/CVPR2026/html/Wang_HTTM_Head-wise_Temporal_Token_Merging_for_Faster_VGGT_CVPR_2026_paper.html

Questions for authors
  1. Can the authors provide a controlled comparison between adaptive clustering and fixed-ratio token merging under the same stage-wise repartitioning strategy? How much of the performance improvement can be attributed specifically to the adaptive clustering mechanism?

  2. Can the authors provide comparisons with existing methods under matched computational budgets or accuracy levels?

  3. Could the authors provide more detailed qualitative comparisons of 3D reconstruction, including ground-truth point clouds and close-up views of geometric details? This would help assess the impact of token reduction on reconstruction quality.

2 comments
AuthorsOpenReview ↗

Response to Questions

Q1. We thank the reviewer for this valuable suggestion. To explicitly isolate the contribution of adaptive clustering, we conducted an additional controlled experiment, as reported in Appendix Table 13. Specifically, we introduce ΔE-Stage-Fixed, which employs the same distortion-based clustering procedure and stage-wise repartitioning strategy as TraDE, but replaces adaptive merging termination with predefined stage-specific retention ratios. These ratios are fixed across input sequences and matched to TraDE's average retention ratio at each stage, ensuring a controlled comparison under comparable token budgets. At the same overall token retention ratio of 26.69%, TraDE improves AUC@3/15/30 from 15.04/63.41/78.81 to 15.28/63.81/79.03, corresponding to gains of +0.24/+0.40/+0.22, with comparable inference throughput (15.46 vs. 15.85 FPS). These results demonstrate that adaptive clustering mechanism provides additional accuracy benefits beyond stage-wise repartitioning, validating the effectiveness of our distortion-guided adaptive stopping criterion.

Q2. We thank the reviewer for this valuable suggestion. We would first like to emphasize that our original experimental results already demonstrate a favorable accuracy–efficiency trade-off compared with existing methods. As reported in Table 3, on 7Scenes with VGGT, TraDE achieves both higher AUC@3 (15.28 vs. 14.98) and greater acceleration (3.95× vs. 1.87×) than FastVGGT. Similarly, TraDE outperforms HTTM in both AUC@3 (15.28 vs. 12.92) and acceleration (3.95× vs. 2.92×), demonstrating simultaneous improvements in accuracy and efficiency. Furthermore, the parameter analysis in Appendix Table 5 already enables comparisons at matched accuracy levels. Specifically, TraDE with (\lambda=0.05) achieves nearly identical AUC@3 to FastVGGT (14.94 vs. 14.98), while providing substantially higher acceleration (4.44× vs. 1.87×). Similarly, TraDE with (\lambda=0.10) achieves comparable AUC@3 to HTTM (12.97 vs. 12.92), while increasing the acceleration from 2.92× to 5.59×. To further illustrate these advantages, we have additionally provided an accuracy–speed trade-off curve in Appendix Fig. 5 by varying (\lambda) and comparing the resulting operating points with existing methods. The curve demonstrates that TraDE maintains competitive geometric accuracy across a wide range of acceleration levels and achieves a more favorable accuracy–efficiency trade-off. These results further confirm that TraDE's efficiency improvements are not merely achieved by sacrificing geometric accuracy.

Q3. We thank the reviewer for this constructive suggestion. We would like to clarify that qualitative comparisons between dense VGGT and TraDE across six ScanNet scenes are already provided in Appendix E. These results show that TraDE largely preserves the overall 3D point-cloud reconstruction quality and major geometric structures of the original VGGT, despite substantially reducing global attention computation. To further examine fine-grained geometric details, we have additionally included ground-truth point clouds and enlarged views of two representative regions in Appendix Fig. 8. These comparisons further demonstrate the preservation of overall scene geometry, while also revealing that some fine-grained details are reconstructed less precisely due to token reduction. We acknowledge this limitation as part of the trade-off between reconstruction fidelity and inference efficiency.

Thank you for the response and the additional experiments. The new controls, the accuracy–speed curves, and Fig. 8 largely address my questions on isolation, matched comparisons, and qualitative results. However, they also sharpen my main concern about the limited novelty.

The authors acknowledge that the objective follows DP-means. The other components are also established: clustering-based token reduction (Token Pooling), spatio-temporal candidate search (HTTM), and stage-wise structure (AVGGT). The remaining distinction is the distortion-threshold stopping rule, which yields input-adaptive retention. This rule follows directly from the penalized objective, and Table 13 shows that it adds only 0.24 AUC@3 over fixed stage-wise ratios. The contribution is therefore mainly a well-engineered combination of existing components.

Given this, I am inclined to maintain my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.