- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates spatial aggregation and cross-layer fusion for probing pretrained Vision Transformers. The authors propose SALF-P, which combines per-layer spatial attention with layer fusion, and SALF-A, which averages features across layers before applying a single spatial attention module. The central claim is that effective spatial aggregation reduces the need for sophisticated layer fusion. Experiments across multiple classification benchmarks, pretrained backbones, and image quality assessment tasks demonstrate promising results, particularly on spatially sensitive tasks.
Critical strengths
- The motivation is clear. This paper presents a logical progression from empirical observations to method design, with SALF-A offering a simple and efficient alternative.
- The evaluations cover diverse 19 datasets and backbone families, with detailed ablations and efficiency analyses.
- The investigations into spatial versus layer-wise information, including averaging efficiency and spatial stationarity, provide potentially useful insights.
Critical weaknesses
-
Limited novelty beyond ALF. The closely related ALF paper (Ciernik et al., ICML 2026) already investigates the complementarity of spatial and hierarchical aggregation (Sec. 4.6) and evaluates a hybrid probing strategy attending to spatial tokens across multiple layers (Appendix A.9). SALF primarily extends this direction through different aggregation orders and factorization. The conceptual advancement is therefore somewhat incremental, and a direct comparison with ALF's hybrid variant is missing.
-
Despite the extensive evaluation, the improvements over ALF are overwhelmingly driven by only four datasets (Aircraft, GTSRB, SVHN, and DMLab). Excluding these datasets, the average gains across the remaining 14 benchmarks in Table 1 become nearly negligible: +0.23 pp for SALF-P and +0.02 pp for SALF-A. This substantially weakens the evidence for broad effectiveness. While the authors acknowledge task-dependent gains, the extent of this concentration is insufficiently emphasized.
-
The paper repeatedly highlights evaluations on 19 classification benchmarks, yet the main Table 1 reports only 18. The remaining benchmark, ImageNet-1k, is presented separately in Supplementary Table 12, using only five backbones rather than seven. Notably, SALF-A slightly underperforms ALF on ImageNet-1k on average (81.05 vs. 81.27), while SALF-P achieves only a modest improvement (+0.76 pp). Although the experiments are reported, the rationale for separating this important benchmark and the precise scope of the headline aggregate results should be clarified.
-
Several of the largest improvements occur on tasks exhibiting predominantly static spatial attention patterns. It remains unclear whether the gains genuinely require content-adaptive attention or could largely be explained by simpler learned, input-independent spatial weighting. A controlled static spatial pooling baseline would help establish the necessity of the proposed mechanism.
-
The full fine-tuning baseline receives less extensive hyperparameter tuning and lacks data augmentation, weakening the corresponding comparisons. Evaluation on newer foundation vision backbones would also strengthen the practical relevance.
Questions for authors
Idea is good, but there are major concerns in terms of the experiments, especially that the main gain comes from 4/18 benchmarks. And if we remove these four benchmarks, there will be near zero improvement. Also, it will be much better if the authors can add static learned pooling experiment for comparison.
I still keep open for this paper. If authors can justify the contributions and the mechanisms well, I may raise my rating to 3.
After rebuttal
Although the proposed method is conceptually incremental over ALF, the additional experiments have substantially strengthened the empirical evidence, particularly regarding the effectiveness of dynamic spatial attention and the practical efficiency of SALF-A. The new comparisons and clarifications have adequately addressed my main concerns. I therefore have raised my rating from 2 to 3 (WA).