acceptodds
Under review as a conference paper at ICLR 2027

Beyond Prototype: Meta-Promptable Mahalanobis Matching for Cross-Domain Few-Shot Learning

Abstract

Cross-domain few-shot learning (CD-FSL) aims to transfer knowledge from a label-rich source domain to novel target domains with only a few labeled examples. Across segmentation, classification, and detection, most existing methods still rely on prototype matching, where each class is represented by a single vector and queries are assigned by Euclidean or cosine distance. This paradigm implicitly treats class-conditional features as isotropic, an assumption that is often violated under severe domain shift: target-domain features commonly exhibit anisotropic covariance structures, large intra-class variation, and class-dependent uncertainty. As a result, nearest-prototype decision rules can produce poorly calibrated boundaries and limited cross-domain generalization. We propose a meta-promptable Mahalanobis matching framework that replaces isotropic prototypes with prompt-conditioned anisotropic Gaussian class models. Given a few support examples and class-specific instructions, large foundation models generate semantic prompts that summarize category concepts, domain context, and variation cues. Conditioned on these prompts and support features, an episodically trained meta-learner estimates class-wise means and covariance matrices, using a low-rank-plus-diagonal factorization to remain stable in the few-shot regime. Query assignment is then performed by Mahalanobis distance, enabling task-adaptive matching that accounts for both feature location and direction-dependent uncertainty. Theoretically, we show that the resulting hypothesis space strictly contains prototype matching as a special case and yields lower Bayes risk when target features are anisotropic. Empirically, our framework achieves consistent state-of-the-art performance on fourteen cross-domain few-shot benchmarks, including four segmentation datasets, four classification datasets, and six detection datasets, demonstrating that distribution-aware matching provides a unified and effective alternative to prototype-based CD-FSL. Code will be made publicly available.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Cross-domain Few-shot Learning, Meta-learning, Mahalanobis Matching

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pVn4OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a meta-promptable Mahalanobis matching framework for cross-domain few-shot learning (CD-FSL). The core idea is to replace isotropic point prototypes with prompt-conditioned anisotropic Gaussian class distributions, and to classify queries by Mahalanobis distance. Furthermore, a meta-learner is designed to estimate class-wise means and covariances through cross-modal fusion and low-rank factorization.

Critical strengths

Strength 1:Replacing point prototypes with Gaussian distributions is a valuable direction for few-shot cross domain learning. Using semantic descriptions to further enrich and characterize the prototypes can also help improve generalization.

Strength 2: Broad and unified evaluation. The paper evaluates the proposed framework on a wide range of cross-domain few-shot benchmarks covering segmentation, classification, and detection.

Critical weaknesses

Weaknesses 1: The paper's core idea "replacing isotropic prototypes with anisotropic Gaussian distributions and using Mahalanobis matching" is very similar to Simple CNAPS [1]. In my view, the only difference is that [1] shows covariances can be estimated from very few samples by feature design, while this paper uses a different mechanism, i.e., LLM CoT prompts and a meta-learner for covariance estimation. However, the paper does not cite [1], and also does not provide any evidence to show the necessity or advantage of using such mechanism for covariances estimation. The lack of novelty is the primary reason that I assigned the current score.

Weaknesses 2: The proposed meta-learner predicts the mean and covariance based on the prompt and image features. My concern is whether such a design can genuinely realize semantic injection for prototypes. As the authors state, "Fig. 4 shows that our method remains stable under noisy or ambiguous captions" and "The small performance variation indicates that our method does not depend on a specific VLM or LLM." This may indicate that the proposed method fails to properly inject semantics into the prototypes.

[1] Bateni P, Goyal R, Masrani V, et al. Improved few-shot visual classification[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020: 14481-14490.

Questions for authors

Question 1: :Can the authors compare with Simple CNAPS [1], or provide evidence that [1] is insufficient for covariance estimation, or explain the advantage of the proposed method over [1]?

Question 2:Can the authors justify the proposed method effectively injects semantics into the prototypes? For example, if the prompt is replaced, would the model output a significantly different prototype, and can this difference be verified? For instance, given two prompts, "wooden boat" and "iron boat," both added to a prototype "boat". Then, can the model generate two substantially different prototypes that genuinely enable it to distinguish between “wooden boats” and “iron boats”?

14 comments
AuthorsOpenReview ↗

Response to Reviewer pVn4 (Part-1)

Q1. Relationship to Simple CNAPS.

Response:

We thank the reviewer for pointing out Simple CNAPS. We agree that it is closely related and will cite and discuss it in the revised paper. Simple CNAPS and our method share the general strategy of representing classes with Gaussian distributions and performing prediction using Mahalanobis distance. We clarify, however, that this decision rule itself is not the main technical contribution of our work. Modeling a class as a Gaussian distribution and using Mahalanobis distance is a natural extension of prototype matching. The central problem addressed in our work is how to estimate a reliable class-specific anisotropic distribution from extremely limited visual observations under domain shift. This presents two difficulties.

First, the source-domain representation does not necessarily preserve the target-domain class geometry. With only one or a few target examples, empirical covariance estimation is highly unstable and cannot adequately capture domain-specific appearance and intra-class variation. We therefore introduce semantic priors that explicitly describe category attributes, target-domain context, and plausible variation patterns. These textual priors complement the limited visual observations and guide target-domain distribution estimation.

Second, estimating a high-dimensional covariance matrix from few support samples is severely underdetermined. Our meta-learner addresses this issue through episodically amortized estimation, low-rank covariance factorization, and residual visual statistics. Rather than independently estimating a covariance matrix from each target support set, the model learns a transferable estimator across source-domain episodes and adapts it using target visual and semantic evidence.

Therefore, our contribution is not Mahalanobis matching itself, but a prompt-conditioned and data-efficient mechanism for estimating anisotropic class distributions under cross-domain few-shot constraints. The same estimator is further applied to segmentation pixels, classification images, and detection proposals, whereas Simple CNAPS focuses on image classification.

The existing ablations directly support the necessity of the proposed estimator.

MethodAverage 1-shot mIoUAverage 5-shot mIoUChange from Ours
Ours76.680.5–
Ours w/o Prompt Generator64.671.0-12.0 / -9.5
Ours w/o Meta-Promptable Distribution Modeling64.970.6-11.7 / -9.9
Ours w/o Covariance71.476.3-5.2 / -4.2
Ours w/o Covariance Factorization74.477.9-2.2 / -2.6
Ours w/o Residual Strategy72.376.7-4.3 / -3.8

The variant without Meta-Promptable Distribution Modeling directly computes means and covariances from visual feature statistics. Its substantial performance drop shows that direct few-shot covariance estimation is insufficient under domain shift. Removing the Prompt Generator causes a similarly large degradation, confirming that semantic priors provide information that cannot be recovered from scarce visual samples alone. The covariance-factorization and residual-strategy ablations further validate the proposed data-efficient estimation design.

Our framework also produces consistent improvements across fourteen benchmarks covering segmentation, classification, and detection. This indicates that the proposed estimator is not tied to a particular prediction unit or task.

Direct comparison with Simple CNAPS.

Following the reviewer’s suggestion, we additionally implement the Simple-CNAPS under the same CD-FSS protocol. All methods use the same backbone, and evaluation settings.

MethodSettingISICChestFSS-1000DeepGlobeAverage
Simple CNAPS1-shot65.480.378.247.367.8
Ours1-shot76.789.385.454.976.6
Simple CNAPS5-shot69.282.181.550.470.8
Ours5-shot79.289.987.665.180.5

As shown, our method consistently and significantly outperforms Simple CNAPS across all datasets and settings. We will add the results, cite Simple CNAPS, and clarify the scope of our contribution in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer pVn4 (Part-2)

Q2. Does the Meta-Learner effectively inject semantic information?

Response:

Yes. We first clarify that robustness to noisy or semantically similar descriptions is different from insensitivity to semantic content. In our framework, prompts condition both the mean and covariance of the estimated class distribution, rather than merely being appended to a fixed prototype.

Fig. 4 evaluates robustness to unreliable descriptions. When a caption contains ambiguous or visually unsupported words, text-image cross-attention assigns lower weights to tokens that are inconsistent with the support features. The residual visual pathway further prevents unreliable text from overriding the observed evidence. Therefore, robustness to caption noise indicates that the model selectively filters unsupported semantics, not that it ignores the prompt.

The comparison across different VLMs and LLMs has a similar interpretation. Although these models produce descriptions with different wording and levels of detail, the underlying category and domain semantics remain largely consistent. The small performance variation therefore demonstrates robustness to the choice of prompt generator rather than insensitivity to semantic content.

Quantitative evidence from the submitted experiments.

If the Meta-Learner ignored semantic information, removing or modifying the prompts should have little effect. Our ablations show the opposite.

VariantAverage 1-shot mIoUAverage 5-shot mIoUDrop from Ours
Ours76.680.5–
Ours w/o Prompt Generator64.671.0-12.0 / -9.5
Ours w/o Domain-aware Prompt72.876.4-3.8 / -4.1
Ours w/o Covariance Prompt72.376.1-4.3 / -4.4
Ours w/o CoT73.677.2-3.0 / -3.3

These results support three conclusions:

  1. Removing all semantic prompts causes substantial drops of 12.0 and 9.5 points, showing that the Meta-Learner does not rely on visual features alone.

  2. Domain-aware and covariance prompts make distinct contributions. Removing either prompt consistently degrades performance, indicating that the model uses both target-domain context and category-specific variation cues.

  3. Prompt quality also matters. Replacing structured CoT descriptions with direct prompting reduces performance by 3.0 and 3.3 points. If the textual information were ignored, changing its semantic completeness would not produce this consistent degradation.

Additional qualitative evidence.

Following the reviewer’s suggestion, we construct four paired few-shot cases from FSS-1000, each placing two semantically related categories in the same comparison setting. FSS-1000 is suitable for this analysis because it contains fine-grained categories that share a high-level concept but differ in language-describable attributes. The selected pairs are black swan versus swan, shakuhachi versus flute, school bus versus double-decker bus, and black stork versus white stork.

Despite their semantic similarity, the categories within each pair differ in color, material, or structural attributes. Our method accurately segments the corresponding targets while preserving these distinctions, rather than collapsing them into a shared coarse representation. This suggests that category-specific descriptions are integrated with visual evidence to produce discriminative distribution estimates.

These cases provide qualitative evidence rather than an isolated prompt-swapping experiment. The causal evidence that the model uses semantic information comes from the submitted ablations: removing the Prompt Generator, domain prompt, covariance prompt, or structured CoT descriptions consistently degrades performance. The paired cases complement these results by showing that fine-grained semantic distinctions are retained during inference. Since OpenReview does not support image insertion in the response box, we include the visualizations in Fig. 5 of the revised manuscript.

Together, the quantitative ablations and qualitative cases show that the model is sensitive to task-relevant semantic content while remaining robust to unsupported or semantically equivalent wording. Therefore, robustness to noisy captions or different foundation models does not imply that the Meta-Learner ignores semantic prompts.

Dear Reviewer, as the discussion deadline approaches, we would appreciate hearing whether you have any remaining questions or concerns about our paper or rebuttal. We would be happy to provide further clarification or additional information. Thank you again for your time and thoughtful feedback!

Reviewer_DQabOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a meta-promptable Mahalanobis matching framework for cross-domain few-shot learning (CD-FSL). Motivated by the limitations of prototype-based matching in capturing anisotropic feature distributions, the proposed method integrates semantic prompts generated by large foundation models with support visual features to estimate class-specific Gaussian distributions. A meta-learner predicts the distribution means and covariance matrices through cross-modal fusion and low-rank factorization, enabling Mahalanobis-based query matching. The framework is evaluated on 14 benchmarks covering cross-domain few-shot segmentation, classification, and detection, reporting consistent improvements over representative baselines.

Critical strengths
  1. Clear and reasonable motivation.

The paper identifies a meaningful limitation of conventional prototype matching, namely its inability to explicitly capture class-dependent covariance and anisotropic feature distributions. The proposed distribution-aware matching framework is well motivated, and the overall methodology follows a clear connection between the identified problem and the proposed solution.

  1. Broad experimental evaluation and strong performance.

The proposed framework is evaluated on 14 benchmarks spanning segmentation, classification, and detection. The consistent performance improvements across different tasks and domains demonstrate promising applicability and provide substantial empirical support for the effectiveness of the proposed approach.

Critical weaknesses
  1. Concerns regarding evaluation fairness and additional semantic and computational resources.

The proposed framework relies on additional large foundation models, including VLMs and LLMs, to generate category-specific, domain-aware, and covariance-related semantic prompts. These components introduce external semantic knowledge and computational resources that are not equally available to conventional prototype-based baselines. Therefore, the performance improvements cannot be attributed entirely to the proposed distribution-aware matching mechanism. Although the authors provide ablation studies and report the offline prompt-generation overhead, the evaluation does not sufficiently isolate the benefits of additional semantic knowledge from those of the proposed matching strategy. A more controlled comparison under comparable semantic resources would strengthen the conclusions. Furthermore, the efficiency comparison primarily emphasizes online computation, while the costs of VLM captioning and LLM reasoning are excluded. A clearer end-to-end computational comparison, including the extent to which generated prompts can be reused across different few-shot episodes, is necessary.

  1. Insufficient justification for the necessity of full covariance modeling.

The paper motivates its method by highlighting anisotropic feature distributions and off-diagonal correlations under domain shifts. However, the experimental analysis does not sufficiently demonstrate the necessity of modeling these cross-dimensional correlations. Although Table 4 evaluates the contribution of covariance modeling, it does not distinguish between diagonal covariance and full covariance matrices. A direct comparison would help clarify whether the performance gains primarily arise from modeling feature-wise uncertainty or capturing inter-dimensional correlations. This distinction is important for validating the central motivation and the necessity of the proposed covariance modeling strategy.

Questions for authors
  1. Limited empirical evidence for the motivation.

The feature distribution visualization and covariance analysis in Figure 1 are primarily based on a sampled DeepGlobe task. Since the proposed motivation is intended to apply broadly across CD-FSL tasks, additional covariance analyses across multiple domains, datasets, and random episodes would strengthen the argument.

  1. Target-domain adaptation protocol.

The method performs meta-finetuning using augmented support samples. More implementation details regarding the adaptation procedure and its consistency with the compared baselines would help assess the fairness and reproducibility of the evaluation.

  1. Covariance matrix invertibility.

The positive definiteness argument relies on the transformation matrix remaining full rank. However, initialization near the identity matrix does not guarantee invertibility throughout optimization. Further clarification of the numerical safeguards would be helpful.

My major concers are important. If the authors solve my major concers, I would like to change the rate.

11 comments
AuthorsOpenReview ↗

Response to Reviewer DQab (Part-1)

Q1. Are the gains due to extra semantic resources?

Response:

No. The gains cannot be attributed entirely to extra semantic resources. The submitted paper provides the following controlled results.

CD-FSS ablation

MethodSemantic promptsCovariance modeling1-shot mIoU5-shot mIoU
Ours w/o Prompt GeneratorNoYes64.671.0
Ours w/o CovarianceYesNo71.476.3
OursYesYes76.680.5
Covariance gain under equal semantic resourcesYesYes vs. No+5.2+4.2

Removing the Prompt Generator reduces performance from 76.6/80.5 to 64.6/71.0, confirming the contribution of semantic guidance. More importantly, when the same semantic prompts and prompt-refined means are retained, adding covariance modeling improves mIoU by 5.2/4.2 points.

CD-FSC comparison under equal semantic resources

MethodSemantic promptsMatching strategy1-shot accuracy5-shot accuracy
SimpleFusionYesPrototype matching50.0361.85
OursYesDistribution-aware matching60.0471.10
ImprovementSameMahalanobis vs. prototype+10.01+9.25

SimpleFusion and our method use the same semantic prompts. Replacing prototype-style fusion with distribution-aware matching improves average accuracy by 10.01/9.25 points.

Therefore, the gains cannot be attributed entirely to additional semantic resources. Semantic prompts improve the representation, while covariance-aware matching provides a separate and substantial contribution under the same semantic information.


Q2. Full computational cost

Response:

The main table reports the online cost, while the offline prompt-generation cost is provided separately in Lines 456–457. For clarity, the complete per-task cost is:

  • VLM captioning: 0.81 s;
  • LLM reasoning: 1.42 s;
  • text encoding, support adaptation, and query inference: 0.45 s.

This results in an end-to-end cost of 2.68 s per task.

MethodOffline generation (s)Online stage (s)End-to-end total (s/task)
Ours2.230.452.68

The domain description is generated only once for each target domain and shared across its tasks. Therefore, 2.68 s represents the uncached cost, while the amortized cost is lower when domain prompts are reused.

AuthorsOpenReview ↗

Response to Reviewer DQab (Part-2)

Q3. Is off-diagonal covariance necessary?

Response:

Yes. The submitted paper already provides direct evidence that cross-dimensional correlations are present in the target-domain features.

First, Fig. 1(c) visualizes the covariance matrices of foreground and background features on DeepGlobe. The pronounced off-diagonal values show that different feature dimensions are correlated and that the class distributions cannot be fully described by independent channel-wise variances.

Second, Appendix Fig. 8 extends the distribution visualization to ISIC, Chest X-ray, FSS-1000, and DeepGlobe. The elongated and direction-dependent confidence ellipses consistently indicate anisotropic feature geometry across the four target domains.

More directly, Appendix Fig. 9 presents the 20-dimensional empirical covariance matrices after PCA projection for all four CD-FSS datasets. Clear off-diagonal responses are observed in every dataset, providing explicit evidence of cross-dimensional correlations rather than merely unequal channel-wise variances.

The existing performance ablation further shows that covariance modeling is important. As reported in Table 4, removing covariance while retaining the same prompt-conditioned mean estimation reduces the average mIoU from 76.6/80.5 to 71.4/76.3, corresponding to drops of 5.2 and 4.2 points in the 1-shot and 5-shot settings. However, this ablation removes the entire covariance term and therefore does not isolate the specific contribution of its off-diagonal components.

Following the reviewer’s suggestion, we add a controlled diagonal-covariance variant. It replaces the full covariance matrix in our method with a diagonal matrix, while keeping the backbone, prompts, mean estimation, training procedure, and all other components unchanged.

Covariance ModelingISIC 1/5-shotChest X-ray 1/5-shotFSS-1000 1/5-shotDeepGlobe 1/5-shotAverage 1/5-shot
No Covariance72.9 / 74.985.1 / 85.978.4 / 84.149.3 / 60.271.4 / 76.3
Ours w/ Diagonal Covariance74.9 / 77.387.2 / 88.182.7 / 85.552.4 / 62.474.3 / 78.3
Full Covariance w/o Factorization74.8 / 77.188.1 / 87.382.6 / 85.352.1 / 61.874.4 / 77.9
Full Covariance, Ours76.7 / 79.289.3 / 89.985.4 / 87.654.9 / 65.176.6 / 80.5

This comparison directly isolates the value of off-diagonal modeling: both variants capture feature-wise uncertainty, but only the full-covariance model captures correlations between feature dimensions. We will include the new results and the above clarification in the revised manuscript.


Q4. Evidence beyond one DeepGlobe example.

Response:

We clarify that DeepGlobe is used only as a representative example in Fig. 1 because of the main-paper space limit. The appendix extends this analysis to all four CD-FSS benchmarks.

Appendix Fig. 8 visualizes foreground and background distributions on ISIC, Chest X-ray, FSS-1000, and DeepGlobe. Across these domains, query features consistently exhibit elongated and direction-dependent structures, while the distributions estimated from support examples align well with the corresponding query features.

Appendix Fig. 9 further presents the 20-dimensional empirical covariance matrices after PCA projection. Clear off-diagonal responses are observed across all four datasets, confirming that correlated and anisotropic feature structures are not unique to the DeepGlobe example.

We will reference this cross-dataset evidence more explicitly in the main paper and clarify that Fig. 1 presents an illustrative case rather than the sole empirical observation.

AuthorsOpenReview ↗

Response to Reviewer DQab (Part-3)

Q5. Target-domain adaptation protocol.

Response:

We follow the common target-domain adaptation protocol of Lei et al. (2022) and Hu et al. (2022). Specifically, only the labeled support set is used during adaptation. We apply data augmentation to the support images and masks to construct pseudo-query samples, and then fine-tune the model using the original support samples and their augmented counterparts. After adaptation, the model estimates the target class distributions from the support set and predicts the query samples. No query image or query annotation is accessed during adaptation.

The same adaptation protocol, including data augmentation, optimization settings, and adaptation iterations, is applied to the compared baseline and our method. We will add these implementation details and the corresponding hyperparameters to the revised manuscript for clarity and reproducibility. We thank the reviewer for this helpful suggestion.


Q6. Covariance invertibility.

Response:

In fact, in our implementation, a small positive value of 1e-6 is added to the diagonal of each covariance matrix before computing the Mahalanobis distance. This regularization ensures that the covariance matrix remains positive definite and invertible, while improving numerical stability during training and inference. We will clarify this implementation detail in the revised manuscript. We thank the reviewer for prompting this clarification.

Reviewer_bC5fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that prototype-based CD-FSL methods implicitly assume isotropic Gaussian feature distributions, which may misclassify queries under domain shift. So the authors propose a meta-promptable Mahalanobis matching framework that replaces isotropic prototypes with prompt-conditioned anisotropic Gaussian class models. Their framework achieves consistent state-of-the-art performance on fourteen cross-domain few-shot benchmarks.

Critical strengths
  1. Broad empirical coverage. The paper evaluates on 14 benchmarks across segmentation, classification, and detection, with 1/5/10-shot settings, CoT and covariance ablations, zero-shot comparisons, and efficiency/robustness analyses.

  2. Robustness to the choice of foundation model is checked. Table 5 swaps BLIP/CoCa/Llama/Qwen-VL and reports only small variation, which partially addresses the concern that gains depend on one specific VLM/LLM.

Critical weaknesses
  1. The SOTA comparison is outdated. Please compare with the latest work, such as HERA (CVPR 2026), DHANet (ECCV 2026), and Jiang et al. (CVPR 2026), under matched protocols, or justify why they are not comparable.

  2. On the empirical support of the core assumption. No source-domain control is provided. If source-domain features are also anisotropic, then anisotropy is a general property of deep features rather than a consequence of domain shift, and the motivation should be reframed accordingly. If it is caused by domain shift, I am looking forward to seeing these properties per dataset because the four domains differ qualitatively.

  3. The claimed problem–solution link is not validated. The experiments show improved average performance and an independent contribution from the covariance term (Table 4), but not that the gain comes from correcting the anisotropic structure identified as the bottleneck. Missing are: (1) per-dataset correlation between anisotropy strength and the prototype-vs-Mahalanobis gap; (2) evidence that Σ̂ recovers the empirical/oracle covariance rather than relying on semantic priors or mean correction; and (3) error-level attribution showing prototype-misclassified queries fall in low-density regions and are fixed by Mahalanobis matching. Without these, “solving the diagnosed problem” is an inference, not a verified conclusion.

Questions for authors

Please see the weakness above.

7 comments
AuthorsOpenReview ↗

Response to Reviewer bC5f (Part-1)

W1. Recent baselines.

Response:

As suggested, we have added comparisons with HERA (CVPR 2026), DHANet (ECCV 2026), and Jiang et al. (CVPR 2026).

Our framework covers three CD-FSL tasks: segmentation, classification, and detection. HERA and DHANet are evaluated on CD-FSS, while Jiang et al. focus on CD-FSD. These works use the same target benchmarks and shot settings as ours, allowing benchmark-level comparison. Since their backbones and training strategies are not always identical to ours, we explicitly report these differences. All baseline numbers below are taken directly from the corresponding papers.

Cross-domain few-shot segmentation

MethodBackboneDeepGlobe 1/5-shotISIC 1/5-shotChest X-ray 1/5-shotFSS-1000 1/5-shotAverage 1/5-shot
HERA (CVPR 2026)DINOv241.2 / 57.855.6 / 68.783.2 / 86.970.2 / 80.362.6 / 73.4
HERA (CVPR 2026)DINOv344.6 / 63.461.2 / 73.685.8 / 87.981.6 / 86.768.3 / 77.9
DHANet (ECCV 2026)ViT-Base49.79 / 56.1364.25 / 67.3886.06 / 86.8783.61 / 86.6770.93 / 74.26
OursResNet-5054.9 / 65.176.7 / 79.289.3 / 89.985.4 / 87.676.6 / 80.5

Our method achieves the best average performance in both settings. Compared with DHANet, it improves the average mIoU by 5.67 points in the 1-shot setting and 6.24 points in the 5-shot setting. It also outperforms the stronger HERA-DINOv3 variant by 8.3 and 2.6 points, respectively. The improvements are consistent across all four target domains, despite our use of a ResNet-50 backbone.

Cross-domain few-shot detection

MethodBackboneShotArTaxOrClipart1kDIORDeepFishNEU-DETUODDAverage
Jiang et al. (CVPR 2026)Swin-T127.9338.7525.6730.4312.887.1823.81
OursViT-L/14126.225.821.324.112.612.520.4
Jiang et al. (CVPR 2026)Swin-T554.9844.8329.4133.8723.6415.6633.73
OursViT-L/14556.148.534.727.518.916.833.8
Jiang et al. (CVPR 2026)Swin-T1068.4047.6334.2535.5824.9624.1839.17
OursViT-L/141068.952.637.133.819.522.839.1

Our method remains competitive with the latest detection baseline. Jiang et al. perform better in the 1-shot setting, with an average mAP of 23.81 versus 20.4. In the 5-shot and 10-shot settings, the two methods achieve nearly identical average performance: 33.80 versus 33.73 and 39.10 versus 39.17, respectively.

These comparisons show that our unified framework remains competitive with task-specific 2026 methods across both segmentation and detection. We will add these results to the revised manuscript and clearly identify them as values reported in the original papers. We thank the reviewer again for pointing out these relevant works.


W2. Is anisotropy caused by domain shift?

Response:

No. Anisotropy is not necessarily caused by domain shift, and our original wording overstated this relationship. Source-domain features may also exhibit anisotropic class distributions.

The key difficulty is instead the combination of domain shift and scarce target observations. In a data-rich source domain, abundant labeled samples allow the feature extractor to learn sufficiently discriminative representations, even when class distributions are anisotropic. In a target domain, however, only a few support examples are available, while the source-domain geometry may not transfer reliably. These limited visual observations can provide a rough class center but are insufficient to recover class-specific variation and covariance.

Our method addresses this estimation problem. We use textual descriptions as semantic priors for category attributes, domain context, and plausible variations, and combine them with the available support features to estimate the latent class distribution. Since our analyses show that target-domain features commonly exhibit anisotropic and correlated structures, we model each class with an anisotropic Gaussian rather than a spherical prototype.

We will therefore revise our claim as follows: domain shift does not create anisotropy; rather, domain shift and scarce target support make the mismatch between isotropic prototype matching and the underlying class geometry substantially more consequential.

AuthorsOpenReview ↗

Response to Reviewer bC5f (Part-2)

W3. Does the gain actually come from correcting anisotropic geometry?

Response:

We thank the reviewer, and provide the following clarifications.

1. Isolating the covariance contribution.

Table 4 already controls for mean refinement. The “w/o Covariance” variant retains the same prompt-conditioned mean estimator and removes only covariance-aware matching.

DatasetFull, 1/5-shotw/o Covariance, 1/5-shotCovariance gain
ISIC76.7 / 79.272.9 / 74.9+3.8 / +4.3
Chest X-ray89.3 / 89.985.1 / 85.9+4.2 / +4.0
FSS-100085.4 / 87.678.4 / 84.1+7.0 / +3.5
DeepGlobe54.9 / 65.149.3 / 60.2+5.6 / +4.9
Average76.6 / 80.571.4 / 76.3+5.2 / +4.2

Covariance modeling improves every dataset and both shot settings with the estimated means unchanged. Mean refinement therefore cannot explain the gain by itself.

2. Does stronger anisotropy predict a larger gain?

Yes. We analyze 600 episodes using identical features and class means for Euclidean and Mahalanobis matching. Off-diagonal covariance energy is positively correlated with the Mahalanobis gain in every domain, with Spearman’s ρ\rho values of 0.42, 0.37, 0.49, and 0.54. The pooled correlation is 0.47 (95% CI: [0.40, 0.54]). Sphericity gives the complementary result, with a pooled correlation of -0.43 (95% CI: [-0.50, -0.36]).

We further sort the episodes by off-diagonal energy and divide them into four equal groups, from the lowest-anisotropy 25% to the highest-anisotropy 25%. The Mahalanobis-over-Euclidean gain increases monotonically across these groups from +1.8 to +3.7, +6.0, and +9.3 mIoU. These results show that Mahalanobis matching becomes more beneficial as class geometry departs from the spherical assumption. We view this as consistent correlation evidence rather than strict causality.

3. Does the estimated covariance recover target geometry?

We compare different estimators with an oracle covariance computed offline from query labels. Query labels are used only for this analysis, never for training or inference. All estimators share the same visual features, class means, and query samples, isolating covariance estimation.

EstimatorFrobenius error ↓SPD distance ↓Subspace similarity ↑Distance correlation ↑mIoU ↑
Simple-CNAPS-style shrinkage0.3612.370.5980.72474.2
Visual-only meta-estimator0.2961.980.6770.78775.3
Prompt-conditioned estimator0.2241.510.7680.85476.6
Oracle covariance0.0000.001.0001.00078.7

Among non-oracle methods, our estimator is consistently closest to the oracle and reduces the performance gap to 2.1 mIoU. Because the class means are fixed, these improvements arise from better covariance recovery rather than mean correction. The appendix visualizations provide complementary qualitative evidence: across all four domains, the estimated ellipses align with query features and the covariance heatmaps retain clear off-diagonal structure.

4. Which prototype errors are corrected?

At the pixel level, 10.9% of query pixels are misclassified by Euclidean matching but corrected by Mahalanobis matching, whereas 3.7% show the reverse behavior. For the corrected pixels, the class margin changes from -0.076 under Euclidean distance to +0.235 under Mahalanobis distance. Euclidean distance incorrectly places them closer to the competing prototype (0.914 vs. 0.990), while Mahalanobis distance favors the true class (1.193 vs. 1.428).

The reviewer’s low-density hypothesis is only partially supported. These pixels are more difficult than those correctly classified by both methods, but they are not arbitrary outliers: their true-class local density is 0.728, compared with 0.405 for pixels missed by both methods, and their oracle NLL is also lower (2.10 vs. 3.45). The failure is therefore more specific: Euclidean center distance can favor the wrong class even when a sample remains plausible along the high-variance directions of its true class. Mahalanobis matching corrects this error by accounting for class-specific covariance.

Together, the results connect the method to the diagnosed problem at three levels: the gain increases with anisotropy, the estimated covariance approaches the oracle geometry under fixed means, and the corrected errors exhibit the expected covariance-dependent margin reversal.

Subject: Regarding the ongoing discussion and any remaining questions

Dear Reviewer,

As the discussion deadline approaches, we would appreciate hearing whether you have any remaining questions or concerns about our paper or rebuttal. We would be happy to provide further clarification or additional information.

Thank you again for your time and thoughtful feedback!

Best regards,

The Authors

Comments on the paper

1 comment

Dear Reviewer, as the discussion deadline approaches, we would appreciate hearing whether you have any remaining questions or concerns about our paper or rebuttal. We would be happy to provide further clarification or additional information. Thank you again for your time and thoughtful feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.