acceptodds
Under review as a conference paper at ICLR 2027

Token-Level Decisions for Training-Free Few-Shot Segmentation of Overhead Imagery

Abstract

Few-shot segmentation asks a model to delineate a class in a query image from a handful of annotated examples. The need is most acute in remote sensing, where labels are scarce and the imagery departs from the photographs vision backbones are pretrained on. A growing family of methods answers it without training by reading the prediction off a frozen self-supervised encoder, and its design effort has concentrated on the class model that turns support features into a score. We argue that accuracy is governed largely by what surrounds the class model, first by how finely the decision is committed in space and then by how well the feature geometry is refined on the support. FROST is a training-free segmenter built on this position. It projects a positional component out of the frozen DINOv3 tokens, whitens a within-class scatter estimated from the support under heavy shrinkage, scores every token by a density ratio whose threshold the Bayes rule fixes at zero, and commits a label per token rather than per cluster, with nothing tuned per dataset. A component study attributes 13.5 mIoU to the granularity of the decision, more than every other component together, and 5.1 to the feature refinement. An oracle labelling of every decision unit shows the first to be a bound rather than a scoring failure, since coarsening the unit costs almost exactly the accuracy it forecloses, and the effect is not ours alone, since moving the decision of a competing method to the token raises it by 2.3 mIoU. Across seventeen remote-sensing benchmarks FROST surpasses both training-free and learning-based methods, leading by 5.5 mIoU from a single annotated example and by a wider margin at every larger support set, while remaining among the smallest models compared.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
few-shot segmentation, training-free, DINOv3, remote sensing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TRXSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes FROST, a training-free few-shot segmentation pipeline based on frozen DINOv3 features. It combines positional debiasing, support-conditioned shrinkage whitening, foreground/background kernel density-ratio classification, bilateral propagation, and candidate gating. Its central empirical argument is that decision granularity and feature geometry are important for overhead-image segmentation. On 17 remote-sensing benchmarks, FROST achieves reported mean foreground mIoU values of 48.3, 54.1, 55.6, and 56.8 with one, three, five, and ten support images. The authors investigate these results through component ablations, cluster and fixed-block coarsening, oracle labelling, and a token-level INSID3 variant. The appendices additionally examine sampling variability, transfer beyond overhead imagery, support-size scaling, backbone choices, computational costs, and target-absent queries.

Critical strengths

1.Strong empirical results across diverse benchmarks. FROST leads on 16 of 17 remote-sensing benchmarks when averaged over support sizes. Same-backbone comparisons, shared evaluation episodes, and five-seed experiments on five benchmarks strengthen the evidence for its effectiveness. 2. Comprehensive analyses provide insights into the method’s strengths and limitations. Granularity, runtime, and target-absent analyses provide useful insights. The authors acknowledge established components and report FROST’s higher false-positive area than INSID3 on absent targets, strengthening credibility.

Critical weaknesses
  1. Section 5 and Table 3 identify decision granularity and feature refinement as the dominant contributors to accuracy. However, the 13.5-point drop measures the effect of coarsening within FROST, rather than explaining its advantage over competing methods, as the authors acknowledge. The analysis also lacks matched classifier and candidate-gating controls, making their relative contributions difficult to establish. The supplementary experiments strengthen the evidence for a granularity effect, but do not fully isolate the sources of the comparative gains. The central attribution would benefit from targeted controls or more cautious wording.
  2. Section 5 and Table 4 use oracle ceilings to distinguish representational limitations from scoring errors. However, Section 3.4 upsamples continuous scores bilinearly before thresholding, potentially producing boundaries within token patches. It is unclear whether the discrete-unit oracle described in Appendix H captures this same output family. This correspondence matters for interpreting the ceiling and the claim that coarsening primarily reduces attainable accuracy. The authors should clarify the relationship or explicitly limit the analysis to a discrete-unit approximation.
Questions for authors
  1. For the attribution in Section 5, how much of FROST’s advantage remains when classifier and candidate-gating choices are matched across methods? Could the authors provide targeted controls or narrow the attribution accordingly?
  2. Does the ceiling reported in Table 4 account for the exact upsampling-and-thresholding procedure in Section 3.4? If it instead assumes discrete binary units, does using a matched output representation change the conclusions?
  3. Could the authors provide concise pseudocode for the candidate gating in Section 3.4, clarifying the reciprocal matching direction, multi-view aggregation, and dilation order?
Reviewer_ewp4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents FROST, a training-free few-shot segmentation framework for remote-sensing imagery based on frozen DINOv3 features. Its central claim is that decision granularity is the dominant factor affecting accuracy, while support-based feature refinement provides the next largest contribution. FROST combines positional debiasing, shrinkage whitening, token-level density-ratio prediction, and spatial post-processing under a globally fixed configuration. The method is evaluated on 17 remote-sensing benchmarks and several cross-domain datasets, with extensive ablations, oracle upper-bound analysis, support-size studies, and comparisons with both training-free and learning-based baselines.

Critical strengths
  1. The paper provides a substantial experimental evaluation over diverse remote-sensing tasks, including building extraction, land-cover segmentation, drone imagery, and flood scenes. The ablations, oracle analyses, cross-domain experiments, and reproducibility details make the empirical study relatively comprehensive.
  2. The observation that a fine-grained decision unit increases the achievable performance ceiling without necessarily guaranteeing higher real-world accuracy is interesting and potentially useful for future training-free segmentation research.
  3. FROST is fully training-free and uses a globally fixed configuration without per-dataset optimization. This makes the method practically attractive for remote-sensing applications with limited annotations.
Critical weaknesses
  1. The generality of the central granularity claim remains insufficiently established. The main evidence that decision granularity is the dominant factor comes from ablations within the FROST pipeline. The paper performs only a limited transfer of this analysis to INSID3, while comparable token-level modifications are not evaluated for FSSDINO or other compatible baselines. The current results therefore support a strong empirical effect in FROST, but do not yet justify extending the conclusion to training-free few-shot segmentation more broadly.
  2. The algorithmic novelty is somewhat limited because most components are adopted from existing techniques. Token-level prediction is already used by FSSDINO, positional debiasing is inherited from INSID3, shrinkage whitening is a classical covariance-estimation technique, and the von Mises–Fisher kernel, bilateral propagation, and reciprocal matching are established methods. The main contribution appears to be the integration of these components and the accompanying empirical analysis. The paper would benefit from a clearer statement of what is methodologically new and from stronger comparisons with simpler combinations of existing components.
  3. The hyperparameter analysis is incomplete for the paper’s main attribution claim. The paper studies the shrinkage coefficient λ, but the sensitivity of the granularity-versus-feature-refinement conclusion to other important choices, such as the kernel bandwidth, positional projection rank, propagation parameters, and gating parameters, is not fully reported. If different reasonable settings change the relative drops in the component ablation, the claim that granularity is consistently the dominant factor would require qualification.
  4. The interpretation of granularity is tightly coupled to the chosen image resolution. All experiments use a 1024×1024 input and a 64×64 token grid, so the physical size of a decision unit depends directly on the image scale and dataset resolution. The paper does not establish whether granularity remains the main bottleneck at different input resolutions or object-to-patch scales. It is possible that granularity dominates for high-resolution scenes, while feature quality becomes more important when objects are small or the input resolution is reduced.
  5. The one-component-at-a-time ablation does not quantify interactions between granularity and feature refinement. The design implicitly treats the individual performance drops as separately attributable, although the paper also argues that these two stages are strongly complementary. In particular, the “feature refinement” variant removes positional debiasing and whitening together. A joint or factorial analysis is needed to determine whether the combined effect is additive, synergistic, or redundant.
  6. The method uses horizontal flipping to double the support anchors, but the paper does not explain why this augmentation is preferred over vertical flipping, rotations, or other remote-sensing-specific transformations. It is also unclear whether the reported improvement comes from the geometric invariance introduced by flipping or simply from increasing the number of anchors. A matched-budget comparison would clarify this point.
  7. Several key arguments are difficult to follow because important evidence and definitions are deferred to the appendices. The fixed-block granularity ladder, oracle ceilings, INSID3 token-level experiment, and five-method ceiling/shortfall decomposition are all central to the main claim but are mainly reported in Appendices G–J.
Questions for authors
  1. The granularity claim is the key question of the paper. Can the authors apply the same controlled token-level modification to FSSDINO, and to other compatible baselines where possible, while keeping the feature representation, scoring rule, gating, and computational budget fixed?
  2. How robust is the granularity-versus-feature-refinement attribution to hyperparameter choices? Please provide sensitivity experiments for the shrinkage coefficient, kernel bandwidth, positional projection rank, propagation parameters, and gating parameters. More importantly, could the authors repeat the main component attribution under several reasonable configurations and report whether the ranking of the performance drops remains stable?
  3. Does the claimed dominance of granularity persist across image resolutions and object scales? Please evaluate the method at multiple input resolutions or tile sizes, such as 512, 1024, and 2048 pixels, and repeat the granularity and feature-refinement ablations.
  4. Can the authors quantify the interaction between decision granularity and feature refinement? A factorial ablation involving token-level versus coarse decisions and full versus reduced feature refinement would show whether their joint effect is additive or synergistic. In addition, a matched-anchor comparison using horizontal flips, vertical flips, and rotations would clarify whether the gain from support augmentation comes from geometric diversity or simply from increasing the number of anchors.
Reviewer_wrLvOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents FROST, a training-free few-shot segmentation method for overhead imagery built on a frozen DINOv3 encoder. It aims to segment a target class in a query image using a small number of annotated reference images, without task-specific model training. The method applies normalization, positional debiasing, and shrinkage whitening based on support-set within-class scatter to the encoder features. It then uses foreground and background support tokens as anchors to construct log density-ratio scores through kernel density estimation. Bilateral score propagation and forward–backward candidate gating refine the predictions, while the continuous score field is bilinearly upsampled and thresholded at zero under an equal-class-prior assumption to produce the final mask. The paper also investigates the effects of decision granularity and feature refinement through component ablations, fixed-block decisions at different sizes, ground-truth-based oracle labeling analyses, and a token-level adaptation of INSID3. Evaluation covers seventeen remote-sensing benchmarks and six natural-image or cross-domain benchmarks. FROST achieves mean foreground mIoU scores of 48.3 and 56.8 in the one-shot and ten-shot settings, respectively, across the remote-sensing suite. Supplementary experiments further examine sensitivity to support sampling, the number of reference images, backbone choices, inference costs, and prediction behavior on target-absent queries.

Critical strengths

1.The paper investigates few-shot segmentation of overhead imagery without task-specific model training, a problem with practical relevance. By using a frozen backbone and a small set of annotated reference images, the method avoids task-specific parameter optimization. The whitening transform, kernel bandwidth, and gate radius are estimated from the support set, while a single global configuration is maintained across benchmarks and support sizes.

2.The evaluation covers seventeen remote-sensing benchmarks, six additional natural-image or cross-domain benchmarks, four support sizes, and nine baseline methods. Averaged over the four support sizes, FROST achieves the best foreground mIoU on sixteen of the seventeen remote-sensing benchmarks. Appendix C provides complementary evidence through paired statistical tests and support-sampling experiments. The component ablations and fixed-block experiments also provide useful evidence about the effects of feature refinement and decision granularity on segmentation performance.

Critical weaknesses
  1. Appendices G and I attribute performance differences solely to grouping or feature refinement, but partition and unit size change together (Table 13), and Appendix I does not control other pipeline differences. Could you add matched controls, or revise the claims to describe observed differences rather than isolated component contributions?

  2. Appendix H labels discrete units binary, while Section 3.4 upsamples continuous scores before thresholding. These are not equivalent—interpolated boundaries can fall inside token cells. Sharing the candidate gate does not show the oracle bounds the implemented predictor. Could you give the oracle's precise definition or pseudocode and confirm its mask space covers all outputs of the actual prediction procedure? Until then, Tables 4/14/16 should probably not be read as ceilings.

Questions for authors

Please see the questions included in the weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.