acceptodds
Under review as a conference paper at ICLR 2027

OraP: Oracle-Inspired Early Token Pruning for Efficient Visual Place Recognition

12 views

Abstract

Visual place recognition (VPR) has recently achieved remarkable performance with the emergence of powerful foundation models. Nonetheless, their high computational cost remains a major obstacle for efficient deployment. Token pruning offers a promising route to faster inference, yet aggressive early pruning, where the computational benefit is greatest, often causes substantial performance degradation. In this work, we uncover that token rankings induced by the final-block attention provide a highly reliable pruning signal. An oracle study further reveals that these rankings can guide early pruning with near-lossless performance. However, obtaining the oracle rankings requires an additional forward pass and therefore provides no practical acceleration. To bridge this gap, we propose OraP, an oracle-inspired early token pruning framework that anticipates late-emerging token rankings from early representations. Instead of reproducing exact attention values or distributions, OraP preserves only selection-relevant ranking relationships, enabling reliable early pruning in a single forward pass. The proposed OraP can be readily applied to accelerate various representative transformer-based VPR models. Experimental results show that OraP substantially outperforms existing token reduction methods under the same early pruning setting. By removing 50% of the patch tokens within the first transformer block, OraP reduces FLOPs by 48.9% and achieves 2.05× inference throughput with an average absolute R@1 degradation of only 0.8% across twelve benchmark datasets. On an NVIDIA Jetson Xavier NX, OraP achieves a 1.68× speedup in single-query inference, supporting its practicality for edge deployment. Code and models will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Place Recognition, Token Pruning, Efficient Inference

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_jEEhOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates early token pruning for efficient visual place recognition (VPR). The authors first demonstrate through an oracle study that final-block attention provides reliable token rankings for nearly lossless early pruning, although obtaining such rankings requires an additional forward pass. To address this limitation, they propose OraP, a ranking-aware self-distillation framework that predicts late-emerging token rankings from early representations. Specifically, OraP employs a lightweight global-local predictor and a pruning-aware ranking loss to guide token selection within the first transformer block in a single forward pass. Extensive experiments across diverse VPR benchmarks show that OraP substantially improves inference efficiency while largely preserving retrieval accuracy under 50% token pruning. Additional evaluations investigate different pruning strategies, distillation objectives, and practical acceleration on edge devices. The method also demonstrates applicability across different transformer-based VPR models.

Critical strengths
  1. Clear Motivation from Oracle Study. The authors decouple the pruning location from the source of token importance and demonstrate that final-block attention rankings enable nearly lossless pruning within the first transformer block. This observation provides convincing evidence for the feasibility of aggressive early pruning and establishes a clear motivation for the proposed ranking prediction framework.
  2. Strong Experimental Validation and Practical Efficiency. OraP consistently outperforms representative token reduction methods across diverse VPR benchmarks under the same early pruning setting. With 50% token pruning, it reduces FLOPs by 48.9%, achieves 2.05× throughput and 1.68× edge-device speedup, while limiting the average R@1 degradation to 0.8 percentage points. Additional experiments across different VPR aggregators and transformer backbones further support its generalizability and practical relevance.
Critical weaknesses

The comparison with the most closely related VPR method could be strengthened.

Although the paper provides extensive comparisons with representative token reduction methods, WeiToP, a closely related approach for early token pruning in VPR, is discussed but not experimentally evaluated. Since both methods aim to learn reliable token importance for early pruning, a direct comparison would help clarify OraP's advantages over the most relevant prior work. Additionally, the unified early pruning setting may not fully reflect the optimal configurations of some baselines. Further comparisons under consistent computational budgets would strengthen the experimental conclusions.

Questions for authors

Q1. WeiToP is discussed as a closely related VPR token pruning method, but a direct quantitative comparison does not appear to be included. Could the authors kindly clarify the reason for this?

Q2. Does the score predictor need to be retrained for each VPR model or backbone, or can a predictor trained with one aggregation model be directly reused with another model sharing the same backbone?

Q3. How closely do OraP's predicted token rankings align with the final-block oracle rankings? Could the authors provide quantitative evidence, such as Top-K overlap or rank correlation, to further validate the proposed ranking prediction mechanism?

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/2)

Thanks for your positive recommendation and detailed suggestions. The following are Responses to the Questions/Weaknesses.

Q1. Comparison with WeiToP

We thank the reviewer for highlighting the importance of this comparison. WeiToP was discussed in the Related Work section of our original submission, but a direct quantitative comparison was not included for the following reasons.

  • Aggregation-specific supervision. WeiToP derives its supervision from the WeiAD aggregation module. Its original formulation therefore cannot be directly inserted into our default BoQ-based VPR model without adapting the importance estimation mechanism. In contrast, OraP derives its supervision from final-block class-to-patch attention, without requiring aggregator-specific importance estimation.
  • Implementation availability. The official WeiToP implementation was unavailable at the time of submission, making faithful reproduction difficult. The authors subsequently released their training code, which we have now used to conduct the additional comparison below.

Controlled comparison using the same VPR model. To better address your concern, we reproduce WeiToP and apply OraP to the same trained DINOv2-B + WeiAD checkpoint. Both methods are evaluated with identical backbone and aggregation parameters, 322×322 inputs, and the same evaluation protocol. We retain each method's token-selection procedure and pruning location: WeiToP prunes after the first transformer block (original setting in its paper), while OraP prunes between the first MHSA and FFN. Both retain 50% of the patch tokens. Results are reported as R@1/R@5 (%).

MethodFLOPs (G)Pitts30k-testMSLS-valMSLS-testTokyo24/7NordlandSPEDAmsterTime
WeiAD (Full-token)——51.491.6/96.291.3/96.267.1/78.294.3/97.183.1/92.189.6/95.2
+ WeiToP27.187.7/94.489.7/95.663.1/75.279.0/90.575.8/87.385.7/93.949.8/71.2
+ OraP26.390.8/95.990.9/96.065.9/77.693.0/96.581.2/91.389.1/94.755.1/76.8

Despite pruning earlier and requiring fewer FLOPs, OraP achieves higher R@1 and R@5 on all seven benchmarks. The absolute R@1 gains are particularly pronounced on Tokyo24/7 (14.0%), Nordland (5.4%), and AmsterTime (5.3%). Relative to the shared full-token reference, OraP reduces the average R@1 degradation from 6.3% to 1.3%. Since the underlying VPR model parameters are identical, these improvements are not attributable to a stronger full-token model, but to the superiority of OraP.

Q2. Predictor Training and Reuse Across VPR Models

In Table 5 of our original submission, we trained a separate predictor for each VPR model while keeping its backbone and aggregator frozen. We sincerely thank the reviewer for the insightful suggestion to explore whether a predictor trained for one VPR model can be directly reused with other models sharing the same backbone architecture. Following this suggestion, we conducted an additional experiment in which the predictor trained with DINOv2-B + BoQ is directly transferred to NetVLAD-, SALAD-, and ImAge-based VPR models, without retraining or fine-tuning. For each target model, we compare the transferred predictor with its target-trained counterpart using the same VPR checkpoint, pruning location, and 50% pruning ratio. All models use DINOv2-B as the backbone. The table reports R@1/R@5 (%).

Target VPR ModelPredictor SettingPitts30k-testMSLS-valMSLS-testTokyo24/7Nordland
NetVLADFull-token (no pruning)93.8/97.194.7/97.476.3/86.097.1/98.797.0/98.9
NetVLADTarget-trained93.2/97.194.2/97.475.5/85.696.5/98.495.3/98.4
NetVLADTransferred from BoQ92.9/97.094.2/97.375.3/85.595.2/96.595.5/98.7
SALADFull-token (no pruning)93.6/97.094.7/97.676.1/86.097.5/98.196.5/98.7
SALADTarget-trained93.0/96.894.5/97.475.1/85.396.5/97.895.0/98.1
SALADTransferred from BoQ92.5/96.693.6/97.474.7/85.094.9/96.595.3/98.3
ImAgeFull-token (no pruning)93.9/97.294.6/97.476.9/86.797.1/98.797.3/99.1
ImAgeTarget-trained93.6/97.294.3/97.376.1/86.295.9/98.195.2/98.4
ImAgeTransferred from BoQ93.3/97.094.1/97.275.8/86.194.0/97.595.6/98.6

As you can see, the BoQ-trained predictor transfers well to all three target models. Compared with target-trained predictors, its average R@1 decreases by only 0.3%, 0.6%, and 0.5% for NetVLAD, SALAD, and ImAge, respectively. It even achieves slightly higher R@1 on Nordland for all three models. However, transfer is not uniformly lossless: Tokyo24/7 shows larger decreases of 1.3%–1.9%. Thus, a single predictor can be directly reused across these evaluated VPR models, although target-specific training can further improve performance on some datasets. We thank the reviewer again for this valuable suggestion, which has helped us demonstrate the predictor's transferability and further highlight the practical value of OraP.

AuthorsOpenReview ↗

Rebuttal by Authors (2/2)

Q3. Degree of Overlap with Oracle Token Rankings

We sincerely appreciate your attention to this detail. To directly quantify the agreement between OraP's predictions and the oracle, we measure Top-K overlap between the tokens selected using OraP's predicted scores and those selected using final-block attention from a full-token forward pass. For each query, the overlap is calculated as the number of shared retained tokens divided by K. We report the average overlap over all queries in each dataset. The following results report Top-K overlap (%) between OraP and the oracle at a 50% pruning ratio, retaining 264 of the original 529 patch tokens.

Pruning RatioPitts30k-testMSLS-valMSLS-testTokyo24/7Nordland
50%73.481.982.269.380.6

OraP achieves Top-K overlap of 69.3%–82.2% across the five benchmarks, showing that its predictions recover a substantial fraction of the oracle-selected tokens. Although the agreement is not perfect, identical token selection is not necessary for successful retrieval. As illustrated by the qualitative examples in Fig. 8 of our original submission, different retained token subsets can yield correct retrievals when sufficient place-discriminative information is preserved. This is consistent with OraP maintaining strong retrieval performance despite not exactly reproducing the oracle selection.

Additional Response: Comparisons Under Matched Computational Budgets

Thank you for giving this suggestion to improve our paper. In fact, our comparisons in Table 2 fix the token reduction location and ratio to evaluate different methods under the same early pruning setting. We agree that this comparison may not fully characterize the performance of baselines using their original strategies, such as multi-stage pruning or progressive merging. Following your suggestion, we therefore conduct an additional comparison with two representative methods, including EViT and ToMe, at comparable computational budgets, without restricting them to our first-block pruning location. EViT performs multi-stage pruning in the 4th, 7th, and 10th transformer blocks, while ToMe progressively merges tokens across blocks. These pruning strategies are consistent with those in their original paper. We adjust their reduction strengths to approach OraP's computational budget. Specifically, OraP removes 50% of patch tokens within the first block, whereas EViT and ToMe use cumulative token reduction ratios of 80% and 75%, respectively. OraP and ToMe both require 27.3G FLOPs, while EViT uses a slightly larger budget of 28.7G. The table below reports R@1/R@5 and measured throughput.

MethodFLOPs (G)Throughput (img/s)Pitts30k-testMSLS-valMSLS-testTokyo24/7Nordland
EViT28.7474.091.6/96.690.1/97.069.0/81.789.2/96.879.2/90.9
ToMe27.3465.192.6/96.893.6/97.474.2/84.893.0/96.591.7/97.3
OraP27.3507.293.3/97.094.7/97.775.7/85.696.5/98.493.8/97.8

Under these settings, OraP still achieves higher R@1 and R@5 on all five benchmarks, with average absolute R@1 gains of 7.0% and 1.8% over EViT and ToMe, respectively. It also achieves the highest throughput, despite retaining a larger fraction of tokens. In particular, compared with ToMe at identical FLOPs, OraP provides both higher retrieval performance and 9.1% higher throughput. These results further highlight the value of OraP's early-pruning design. By shortening the token sequence within the first transformer block, OraP allows more subsequent computation to benefit from token reduction. It therefore achieves a comparable computational budget with only 50% token reduction, whereas the evaluated EViT and ToMe schedules use approximately 80% and 75% cumulative reduction, respectively. OraP thus achieves substantial computational savings while retaining a larger token set, with higher measured throughput and better retrieval performance.

Thanks again for your valuable comments and suggestions. Please let us know if you have any further concerns. We'd be very glad to address them.

Reviewer_jEEhOpenReview ↗

Concerns Addressed, Score Unchanged

Thank you for the detailed response and additional experiments. My concerns have been addressed, and I will maintain my original score.

AuthorsOpenReview ↗

Official Comment by Authors

We are glad that our response has addressed all your concerns. Thank you very much for your comments and suggestions to improve our paper. We will incorporate them into the final version.

Reviewer_BpY1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OraP accelerates visual place recognition by predicting final-block token importance rankings from early representations. A lightweight predictor, trained using cross-boundary ranking supervision, enables token pruning within the first transformer block. An oracle study motivates the approach, while extensive retrieval benchmarks, architectural evaluations, and hardware measurements demonstrate a favorable accuracy–efficiency trade-off.

Critical strengths

The paper presents a clear motivation, and strong empirical validation. As the result shown, pruning 50% of patch tokens within the first transformer block reduces FLOPs by 48.9%, with only a 0.8 percentage-point average R@1 drop across twelve benchmarks. Results across multiple VPR architectures and measured acceleration on an edge device support its practical value.

Critical weaknesses

Although I believe that the work presented in this paper has practical value, I am unable to distinguish its differences from ToP [1] related methods, which concerns me about its actual contribution. ToP has refined the final layer token ranking into early layers and incentivized ranking supervision for matching attention values. This paper needs to clarify and experimentally establish its additional contributions beyond this closely related method.

[1] Li, J., Zhang, L. L., Xu, J., Wang, Y., Yan, S., Xia, Y., ... & Yang, M. (2023, August). Constraint-aware and ranking-distilled token pruning for efficient transformer inference. In Proceedings of the 29th ACM SIGKDD Conference on knowledge discovery and data mining (pp. 1280-1290).

Questions for authors

See the Critical Weaknesses.

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/2)

Thanks for your comments to help us improve the paper. The following are Responses to the Weaknesses.

W. Additional Contributions Beyond ToP

We truly thank the reviewer for pointing out ToP, an important related work that was inadvertently omitted from our initial discussion. As the reviewer commented, ToP has refined the final layer token ranking into early layers and incentivized ranking supervision for matching attention values. We acknowledge this shared principle. However, OraP's contributions are multifaceted, extending well beyond late-to-early supervision, such as a decoupled, predictor-only self-distillation pruning framework, carefully designed pruning-aware multi-boundary ranking loss, a unique oracle study demonstrating the feasibility of early pruning without VPR model modification, a lightweight global-local predictor, etc. The distinctions from ToP, together with supporting experimental evidence, are detailed below.

A decoupled self-distillation framework for transferable early token pruning. ToP distills the well-trained teacher’s final-layer token rankings into the well-trained student’s early-layer attention scores by optimizing the student Transformer itself (This means the student model requires further training/fine-tuning to adapt to pruning), alongside its pruning masks. Thus, the ranking supervision modifies the underlying model to improve its early attention-based token selection. In OraP, the well-trained VPR model remains entirely frozen and serves as its own supervision source. A lightweight predictor learns to estimate selection-relevant rankings from early patch representations using the rankings derived from the final-block class-to-patch attention as supervision. Only the predictor is optimized. Neither the backbone nor the aggregation module is updated, and gradients do not propagate through the VPR model. That is, the whole VPR model does not require any training or fine-tuning to adapt to pruning (our unique oracle study provides a safeguard for this, as will be explained below). Moreover, this separation makes the learned pruning module independently reusable. Additional experiments (see response to Q2 of Reviewer jEEh) demonstrate that a predictor trained on one VPR model can be directly transferred to other evaluated VPR models sharing the same backbone architecture, without retraining or fine-tuning the predictor or modifying the target VPR weights. These results further support a practical contribution beyond ranking distillation: learning an early token selector that can be reused across existing VPR models to accelerate inference.

A pruning-aware multi-boundary ranking objective aligned with token selection. The distinction lies in which ranking relationships are supervised. ToP employs LambdaLoss to distill token rankings, whereas OraP explicitly structures the supervision around the pruning boundary. For each pruning ratio, the proposed objective requires teacher-retained tokens to receive higher predicted scores than teacher-discarded tokens, while leaving the ordering within either subset unconstrained. Thus, the objective directly targets the ordering relationships necessary for token selection. Moreover, to avoid specializing the predictor to a single pruning ratio, our objective incorporates cross-boundary supervision at multiple training pruning ratios. This allows the same predictor to support different test-time pruning ratios without retraining or fine-tuning, while introducing no additional inference overhead. To isolate the contribution of this objective, a controlled comparison replaces our loss with the LambdaLoss used in ToP’s released implementation. Both settings use the same frozen VPR model, predictor architecture, final-layer class-to-patch supervision, pruning location, and training and evaluation settings. Only the loss formulation differs. As shown in table below, our objective consistently improves both R@1 and R@5 across all five datasets and all three pruning ratios. The average absolute R@1 improvements are 1.3%, 2.6%, and 5.0% at pruning ratios of 50%, 60%, and 70%, respectively. The advantage becomes more pronounced under aggressive pruning, reaching 10.8% on Tokyo24/7 at 70% pruning. These results provide direct evidence for the benefit of our pruning-aware multi-boundary objective beyond adopting ranking distillation alone.

ObjectivePruning RatioPitts30k-testMSLS-valMSLS-testTokyo24/7Nordland
LambdaLoss (ToP)50%92.3/96.593.9/97.374.2/84.994.3/97.592.8/97.5
Ours50%93.3/97.094.7/97.775.7/85.696.5/98.493.8/97.8
LambdaLoss (ToP)60%90.5/96.292.4/97.071.8/83.590.8/96.590.0/96.4
Ours60%92.5/96.793.8/97.675.1/85.295.6/97.891.7/97.2
LambdaLoss (ToP)70%87.4/94.690.8/96.969.5/81.682.9/92.183.6/93.1
Ours70%91.3/96.393.5/97.373.6/84.393.7/96.587.1/95.0
AuthorsOpenReview ↗

Rebuttal by Authors (2/2)

A unique oracle study demonstrating early pruning without VPR model adaptation. Beyond observing that token rankings become more reliable at greater depths, our oracle study explicitly separates the source of the pruning signal from the pruning location. Final-block class-to-patch attention rankings are obtained through a full-token forward pass and then used to prune tokens between the MHSA and FFN sublayers of the first block in a second pass. All VPR model parameters remain unchanged throughout this experiment. As shown in Fig. 2 of the initial submission, removing 50% of patch tokens under this setting causes at most a 0.7% R@1 decrease across four benchmarks. This outcome is not implied by reliable final-layer rankings alone: tokens receiving low final-layer scores may still contribute to earlier representations, and removing them early changes subsequent token interactions. The oracle study therefore provides direct empirical evidence that aggressive early pruning can preserve retrieval performance without adapting/fine-tuning the VPR model, given a suitable selection signal. This finding also motivates our independent predictor design, which learns to anticipate that signal from early representations for single-pass pruning.

A class-to-patch supervision signal for reliable early token pruning. OraP also differs from ToP in its supervision signal. ToP scores each token by averaging the attention it receives across query tokens and attention heads. In contrast, OraP uses only the attention that the class token assigns to each patch, averaged across heads. Fig. 2(a) of the initial submission shows that high final-block class-to-patch attention scores tend to highlight place-discriminative structures, providing qualitative evidence for the relevance of this signal to VPR. Averaging attention across patch tokens additionally incorporates interactions arising from regions with varying relevance to place recognition. These contributions may dilute the emphasis on discriminative structures, since attention useful for updating individual token representations does not necessarily indicate importance for identifying a place. An additional oracle experiment evaluates how this distinction affects early pruning performance. Both signals are extracted from the final block of the same frozen VPR model through a full-token forward pass. The resulting rankings guide pruning at the same early location and pruning ratio in a second pass. The following results report R@1 (%). Our class-to-patch attention achieves higher R@1 across all seven benchmarks, such as absolute R@1 improvements of 2.5% on Tokyo24/7 and 2.6% on AmsterTime. These controlled oracle results support the choice of final-block class-to-patch attention as the supervision signal for OraP.

Supervision SignalPitts30k-testMSLS-valMSLS-testTokyo24/7NordlandSPEDAmsterTime
all-token received attention (ToP)92.593.674.394.092.889.660.9
class-to-patch attention (Ours)93.694.776.196.594.590.663.5

A lightweight global–local predictor for anticipating late-emerging token rankings. OraP incorporates a predictor specifically designed to estimate late-emerging token rankings from early patch representations. As described in Sec. 3.3, a channel-reduction projection is followed by lightweight self-attention and spatial convolutions, capturing global token dependencies and local spatial context, respectively. These complementary interactions provide contextual information beyond the individual token features used by a token-wise MLP. The contribution of this design is evaluated in Appendix A.3, Table 9. Moreover, Table 4 shows that OraP introduces approximately 1.0M additional parameters while achieving 2.05× the full-token inference throughput. These results support the predictor’s effectiveness in recovering useful ranking information from early features with modest computational overhead.

Summary

OraP proposed a decoupled self-distillation pruning framework, a carefully designed pruning-aware multi-boundary ranking objective, and the final-block class-to-patch supervision to enable reliable early pruning of frozen VPR models. The controlled ablations and oracle study support these design choices, while the transfer experiments demonstrate that the learned predictor can be reused across the evaluated VPR models without retraining or fine-tuning. These methodological distinctions and empirical findings clarify OraP’s additional contributions beyond late-to-early ranking supervision. We apologize for this omission and will explicitly cite ToP and discuss its relationship to OraP in the revised paper.

Thanks again for your valuable comments and suggestions. Please let us know if you have any further concerns. We'd be very glad to address them.

Thank you for the author's detailed response and experiment. All my concerns have been resolved, and I will increase my rating to 3 points.

AuthorsOpenReview ↗

Official Comment by Authors

We are glad that our response has addressed all your concerns. Thanks a lot for your comments and suggestions to improve our paper. We will incorporate them into the final version.

Reviewer_os25OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Orap predicts final-block token importance from features in the first transformer block, then uses these predictions to prune tokens before the first FFN. The predictor is trained with the VPR model frozen. Its loss encourages tokens that should be retained to score above those that should be discarded, without requiring a complete ranking. The paper evaluates the method across twelve VPR benchmark settings, several backbones and aggregators, and two hardware platforms. With 50% pruning, the default model loses an average of 0.8 percentage points in Ræ1 while reducing FLOPs by 48.9%. Reported speedups are 2.05x for RTX 4090 throughput and 1.68x for single-query inference on Jetson Xavier NX.

Critical strengths
  1. The oracle experiment in Figure 2 is useful. Using final-block scores to guide first-block pruning largely preserves retrieval performance, suggesting that obtaining a reliable selection signal is a major obstacle to early pruning. This motivates the predictor and gives the method a clearer basis than simply adding another token-scoring module. The location and loss ablations further support the design.
  2. The experiments cover more than the default model. The authors test different aggregators and backbones, report actual hardware measurements, and compare against lowering input resolution at matched FLOPs. The last comparison is particularly relevant because reducing resolution is a straightforward alternative. These results make a reasonable case that OraP offers a useful accuracy-efficiency trade-off.
Critical weaknesses
  1. The most important missing comparison is WeiToP. It already predicts token importance at an early layer through self-distillation and allows the pruning budget to change at inference. OraP changes the supervision source and loss, but the experiments do not establish how much these changes improve on that approach under comparable conditions. Table 7's KL baseline helps assess the loss choice, but does not answer the full comparison.
  2. The evidence for the proposed ranking loss could be stronger. In Table 7, the improvement over full ranking is 0.4–0.6 percentage points. The gains are consistent across the four datasets, but there are no repeated-run results to show their stability. The paper also attributes robustness across pruning budgets to multi-boundary training without comparing it with a predictor trained at a single boundary. These omissions leave some uncertainty about the benefit of the specific loss design.
Questions for authors
  1. How does OraP compare with WeiToP under comparable training and inference budgets? If the different aggregators prevent a direct comparison, could you hold the predictor fixed and compare the two supervision sources?
  2. In Table 7, do full-ranking and cross-boundary training use the same predictor and comparable numbers of sampled pairs? How much do their results vary across training runs?
  3. What happens if the predictor is trained only at the 50% pruning boundary and then evaluated at the other pruning ratios? This would help establish what multi-boundary training contributes.
5 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/3)

Thanks for your positive recommendation and insightful comments. The following are Responses to the Questions/Weaknesses.

Q1/W1. Comparison with WeiToP

We thank the reviewer for suggesting this comparison. We agree that methodological differences alone do not establish their practical benefits under comparable conditions. However, WeiToP's supervision is specifically tied to its WeiAD aggregation module, preventing its direct application to widely used VPR models with different aggregators. Moreover, the official implementation was unavailable at the time of submission. Therefore, in our initial paper, we only discussed the methodological differences and lacked a quantitative comparison. The authors of WeiToP subsequently released their training code on October 19, 2026. Using this newly released implementation, we have undertaken a reproduction to provide a more rigorous comparison under a unified evaluation protocol.

For reference, we first summarize the results (R@1/R@5) reported in the original WeiToP paper under a 50% token retention ratio (ρ=0.5\rho=0.5), where pruning is performed after the first transformer block, as shown in the following table. The degradation is particularly pronounced on Nordland, with absolute R@1 and R@5 drops of 10.8% and 6.7%, respectively, and on AmsterTime, with corresponding drops of 9.7% and 8.9%. These published results are provided for reference rather than as a direct comparison with OraP, since the underlying VPR models differ and the evaluation protocols may not be identical. To ensure a fair comparison, we further evaluate both methods using the same VPR model and evaluation protocol below.

MethodMSLS-valNordlandPitts250k-testAmsterTimeSPED
WeiAD (Full-token)91.7/96.273.1/86.795.1/98.858.4/79.590.6/95.4
+ WeiToP (50% Retained)90.4/95.962.3/80.091.5/97.448.7/70.685.5/92.5
Performance Drop-1.3/-0.3-10.8/-6.7-3.6/-1.4-9.7/-8.9-5.1/-2.9

OraP derives supervision solely from the backbone, independently of the aggregator, allowing it to be applied to WeiAD. We reproduce WeiToP using its official implementation and evaluate OraP on the same WeiAD-based VPR model. Both methods use the same DINOv2-B backbone and WeiAD aggregation checkpoint, with identical VPR model parameters and evaluation protocols. WeiToP follows its original token selection procedure after the first transformer block. OraP is trained with this VPR model frozen and performs pruning between the MHSA and FFN sublayers of the first block. Both settings retain 50% of the patch tokens from 322×322 inputs. Results are reported as R@1/R@5. Our OraP achieves higher R@1/R@5 on all seven benchmarks, with average improvements of 5.0% and 3.0%, respectively. The R@1 gains are particularly pronounced on Pitts30k-test, Tokyo24/7, Nordland, and AmsterTime. These improvements are obtained even while pruning earlier and using fewer FLOPs (26.3G vs. 27.1G).

MethodFLOPs (G)Pitts30k-testMSLS-valMSLS-testTokyo24/7NordlandSPEDAmsterTime
WeiAD (Full-token)51.491.6/96.291.3/96.267.1/78.294.3/97.183.1/92.189.6/95.258.0/78.1
+ WeiToP (50% Retained)27.187.7/94.489.7/95.663.1/75.279.0/90.575.8/87.385.7/93.949.8/71.2
+ OraP (50% Retained)26.390.8/95.990.9/96.065.9/77.693.0/96.581.2/91.389.1/94.755.1/76.8

Moreover, to directly address the reviewer's suggestion to hold the predictor fixed, we conduct an additional ablation in which only the supervision source is changed. Using the same frozen DINOv2-B + WeiAD checkpoint, we train two predictors with identical architectures and training configurations. Both use OraP's global-local predictor and pruning-aware ranking loss, and both prune 50% of the patch tokens between the first MHSA and FFN. Token selection is based directly on the predicted scores in both settings. The only difference is whether the teacher scores are obtained from WeiAD-derived token importance or final-block class-to-patch attention. Results are reported as R@1/R@5. As you can see, the final-block attention supervision achieves higher R@1 on all seven benchmarks, with an average improvement of 2.1 percentage points over WeiAD-derived importance. The absolute gains reach 6.3% on Tokyo24/7, 2.1% on Nordland, and 2.3% on AmsterTime. This supervision choice is grounded in our observations and oracle study (Fig. 2 and Sec. 3.2), which provide direct empirical evidence that reliable token rankings derived from final-block class-to-patch attention can guide early pruning with nearly lossless retrieval performance.

Supervision SourcePitts30k-testMSLS-valMSLS-testTokyo24/7NordlandSPEDAmsterTime
WeiAD-derived Importance89.8/95.590.3/95.864.7/76.886.7/95.279.1/89.788.0/94.552.8/74.2
Final-block Attention (Ours)90.8/95.990.9/96.065.9/77.693.0/96.581.2/91.389.1/94.755.1/76.8
AuthorsOpenReview ↗

Rebuttal by Authors (2/3)

Q2/W2. Ranking Loss Comparison and Stability

We thank the reviewer for raising these important questions regarding the fairness and stability of our ranking loss comparison. To address these concerns, we clarify the experimental settings and provide additional results across multiple random seeds.

Fair Comparison. Both Full Ranking and our pruning-aware ranking objective use the same score predictor architecture and an identical sampling budget of 50,000 token pairs per training image. All other training settings are kept consistent. The only difference lies in the ranking supervision: Full Ranking enforces pairwise ordering across the entire token set, whereas our objective focuses on cross-boundary pairs between retained and discarded tokens. Therefore, the performance differences reflect the effect of the ranking objective rather than differences in predictor capacity or the number of sampled pairs.

Stability Across Random Seeds. Following the reviewer's suggestion, we repeat both experiments with four additional random seeds. Together with the original results in Table 7, this yields five runs per objective. The table below reports the mean and sample standard deviation of R@1/R@5 at a 50% pruning ratio. Our objective achieves higher R@1 on all four benchmarks in every run. The mean R@1 improvements are 0.7%, 0.4%, 0.9%, and 0.4% on Pitts30k-test, MSLS-test, Tokyo24/7, and Nordland, respectively. It also exhibits lower R@1 standard deviations across all four benchmarks. On Tokyo24/7, for example, the standard deviation decreases from 0.44 to 0.16. R@5 also matches or improves upon Full Ranking in each corresponding run. These results show that the gains reported in Table 7 persist across different random seeds under matched predictor capacity and sampling budgets, providing further evidence for the effectiveness and stability of our pruning-aware ranking objective.

ObjectivePitts30k-testMSLS-testTokyo24/7Nordland
Full Ranking92.6±0.19/96.9±0.0575.3±0.11/85.4±0.0995.5±0.44/97.7±0.4393.3±0.13/97.7±0.11
Ours93.3±0.09/97.0±0.0875.7±0.09/85.6±0.0796.4±0.16/98.2±0.1693.7±0.08/97.9±0.05
AuthorsOpenReview ↗

Rebuttal by Authors (3/3)

Q3/W2. Effect of Multi-Boundary Supervision

We thank the reviewer for this insightful suggestion. As discussed in our original paper, multi-boundary supervision is designed to prevent the score predictor from specializing to a single pruning ratio, thereby improving its robustness across different token budgets. To further validate its contribution, we conduct an additional experiment comparing Single-boundary training (50%) with our Multi-boundary training (30%–70%). Both strategies use the same score predictor architecture and training configuration. The only difference lies in the training pruning boundaries: Single-boundary uses only a 50% pruning ratio, whereas Multi-boundary (Ours) uses five ratios, i.e., 30%, 40%, 50%, 60%, and 70%. Importantly, both strategies sample the same total number of 50,000 token pairs per training image, ensuring a comparable supervision budget. We evaluate both predictors at all five pruning ratios on four representative VPR benchmarks. The results are summarized below (R@1/R@5).

Training StrategyPruning RatioPitts30k-testMSLS-testTokyo24/7Nordland
Single-boundary (50%)30%93.5/97.176.0/86.096.5/98.794.5/98.1
Multi-boundary (Ours)30%93.7/97.176.3/86.196.5/98.794.9/98.3
Single-boundary (50%)40%93.4/97.175.8/85.796.5/98.494.2/98.0
Multi-boundary (Ours)40%93.5/97.176.0/85.896.5/98.494.5/98.2
Single-boundary (50%)50%93.4/97.175.7/85.596.5/98.493.7/97.8
Multi-boundary (Ours)50%93.3/97.075.7/85.696.5/98.493.8/97.8
Single-boundary (50%)60%91.5/96.573.4/84.293.0/96.890.3/96.4
Multi-boundary (Ours)60%92.5/96.775.1/85.295.6/97.891.7/97.2
Single-boundary (50%)70%89.1/95.670.6/82.590.2/94.384.2/93.8
Multi-boundary (Ours)70%91.3/96.373.6/84.393.7/96.587.1/95.0

We summarize three key observations: (a) Comparable performance at the training boundary (50%). When evaluated at the same 50% pruning ratio used for Single-boundary training, both strategies achieve nearly identical retrieval performance. This confirms that supervision at a single boundary is sufficient to learn effective token selection for that specific pruning ratio. (b) Small differences at lower pruning ratios (30%–40%). Both strategies maintain strong retrieval performance when fewer tokens are removed, with Multi-boundary achieving modest improvements. Since more tokens are retained, the retrieval process is generally less sensitive to imperfect token rankings. Nevertheless, Single-boundary supervision does not explicitly constrain the ordering among tokens within the teacher-discarded subset, limiting its ability to select additional informative tokens when the retention budget increases. (c) Substantial improvements at higher pruning ratios (60%–70%). The advantage of Multi-boundary becomes increasingly pronounced as the pruning ratio increases. At 60% and 70% pruning, Multi-boundary achieves average absolute R@1 improvements of 1.7% and 2.9%, respectively, across the four benchmarks. At 70% pruning, the improvements reach 3.5% on Tokyo24/7, 3.0% on MSLS-test, and 2.9% on Nordland. This behavior can be explained by the limitation of Single-boundary supervision: training only at 50% enforces ranking relations between the retained and discarded subsets but does not explicitly constrain ordering within the retained subset. At higher pruning ratios, selecting a smaller subset of the most informative tokens requires finer-grained ranking within this previously retained group. Multi-boundary supervision provides additional cross-boundary constraints at these more aggressive pruning ratios, helping the predictor learn more reliable token rankings. Overall, these results demonstrate that multi-boundary supervision improves robustness across different token budgets, particularly under higher pruning ratios, while maintaining comparable performance at the original 50% training boundary. This further supports our design of training a single score predictor with multiple pruning boundaries to enable reliable token selection at different inference budgets without retraining or fine-tuning. Following your suggestion, we will incorporate this experiment into the revised manuscript to further highlight the contribution of multi-boundary training.

Thanks again for your valuable comments and suggestions. Please let us know if you have any further concerns. We'd be very glad to address them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.