- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper asks whether a single few-shot predictor can be learned once and reused across heterogeneous frozen representation spaces—across datasets, encoders, and even modalities—without test-time updates. RepICL canonicalizes each episode via episodic whitening (episode-wise PCA + coordinate-wise standardization), then applies a shared Transformer-based in-context predictor with prototype classification, in inductive (RepICL-I) and transductive (RepICL-T) variants. On the authors’ RepShiftBench, RepICL-I surpasses per-episode logistic regression in all 12 settings, and RepICL-T gains more under transduction.
Critical strengths
Well written and easy to follow. The motivation—in-context learners break when the representation space changes—is clearly posed.
Critical weaknesses
1.The paper cites cross-domain few-shot learning only in passing (Tseng et al., 2020; Li et al., 2022) and compares against none of it. The central claim—a shared predictor that generalizes across representation spaces without test-time updates—overlaps substantially with what cross-domain few-shot methods on Meta-Dataset (Triantafillou et al., 2020) are designed to test, yet none appears in Table 1. LoRA in Appendix E.2 is a reasonable test-time-adaptation baseline but is not a cross-domain few-shot method. Without one or two recent cross-domain few-shot baselines on the same fixed episodes, the incremental value of RepICL cannot be assessed.
2.Applying an episode-level transformation so a shared predictor can span representation spaces is not new in kind. TAIL (Woerner et al., 2026) already projects encoder outputs into a common latent space and transfers to unseen modalities; the paper distinguishes itself mainly by arguing TAIL’s evaluation is not systematic—a critique of protocol, not mechanism. Representation normalization and alignment have also been studied via episode-statistic normalization (Bronskill et al., 2020), whitening with reference statistics (Su et al., 2021), and relative representations (Moschella et al., 2023). Episodic whitening is a variant of this family, and the paper does not clearly state what it adds beyond per-episode PCA + standardization. The authors should say plainly whether the contribution is the whitening choice, the meta-training on whitened episodes, or the benchmark—right now all three are claimed, but the ablation suggests only the pairing matters.
3.RepICL is PCA, standardization, a linear projection with label embeddings, a Transformer, and a prototype classifier. PCA and standardization are routine preprocessing; the label-embedding-plus-Transformer architecture is inherited from CAML (Fifty et al., 2024) and initialized from its checkpoint; the prototype classifier follows Prototypical Networks (Snell et al., 2017) and FEAT (Ye et al., 2020). Table 3 supports this reading: the same whitening does not help TabPFN or TabICL, so the benefit is the pairing, not a separable new mechanism. The paper needs either a mechanism-level explanation of why the pairing is necessary or a more modest positioning.
4.The method section states two different maximum PCA ranks for the inductive and transductive variants but describes both as matching the maximum ranks under the same five-way five-shot configuration; the numbers fit different estimation subsets (support-only vs. support-plus-query), yet the sentence conflates them, which is confusing in the core method description. Separately, TAIL scores 5.70% on Text-held-out Dataset adaptation in Table 1 while others exceed 56%; the appendix blames feature scale and applies norm calibration, but a near-random collapse still raises the question of how much the cross-encoder evaluation depends on baseline fragility to scale rather than genuine generalization.
Tian, H., Liu, F., Cheung, K. C., Fang, Z., See, S., Liu, T., & Han, B. Cross-domain Few-shot Classification via Invariant-content Feature Reconstruction. Zhuo, W., & Luo, S. (2026). Modality-free Graph In-context Alignment. Zhu, L., Yin, W., Wu, F., Gu, Q., Wang, X., Zhou, C., & Ye, N. Bayes-CAL: Robust Cross-Modal Alignment by Bayesian Approach for Few-Shot OoD Generalization. FAD: Frequency Adaptation and Diversion for Cross-domain Few-shot Learning. (2025).
Questions for authors
See Weaknesses
3 comments
Response to Reviewer giHH (2/2): W2-W4
W2. Contribution and distinction from prior work.
Our primary methodological contribution is RepICL, which couples episodic whitening as an episode-dependent statistical normalization interface with meta-training a shared predictor on whitened episodes to enable reusable few-shot prediction across heterogeneous frozen representation spaces. RepShiftBench provides a complementary contribution by systematically evaluating generalization across unseen datasets, encoders, jointly unseen pairs, and modalities. To clarify this positioning, we revised the contribution framing (Abstract and Introduction, Sec. 1), the comparison with TAIL and prior normalization/alignment methods (Related Work, Sec. 2), and the rationale for episodic whitening as a statistical normalization interface (Sec. 5.1).
The distinction is not only in the evaluation protocol, but also in how the predictor handles representation-space heterogeneity. TAIL uses randomized projections to map heterogeneous features into a common latent space. However, these data-independent projections do not explicitly normalize the statistical structure of each episode. In our setting, a shared predictor must operate across previously unseen encoders whose feature coordinates, scales, and covariance structures may differ substantially. RepICL therefore estimates PCA directions and coordinate scales independently from each episode, reducing representation-specific statistical variation and providing a more consistent interface for learning relationships among examples across representation spaces. This does not require semantic alignment between encoder coordinates.
Beyond the methodological difference, RepShiftBench systematically separates dataset and encoder novelty, while TAIL's evaluation does not examine these axes separately. Our improvements also extend beyond RepShiftBench: using TAIL's released evaluation code, RepICL-T achieves 92.31% mean accuracy despite being meta-trained without image-modality datasets or encoders, compared with 90.47% for our reproduction of TAIL (Appendix D.1).
We have also expanded the Related Work to acknowledge prior uses of normalization, whitening, and representation alignment, while clarifying how their objectives and assumptions differ from reusable few-shot prediction across previously unseen frozen encoder spaces.
W3. Evidence for the whitening–meta-training pairing.
Our ablations provide empirical support for the role of episodic whitening within RepICL's meta-training. Table 2 shows substantial performance degradation when removing coordinate-wise standardization or replacing episodic PCA with fixed or trainable projections. Importantly, these variants still meta-train the shared predictor on their respective transformed representations, yet do not match the performance achieved by the full model meta-trained on whitened episodes.
Table 3 further shows that applying episodic whitening as a drop-in preprocessing step to Logistic Regression, TabICL, or TabPFN does not improve their performance. Together, these findings are consistent with RepICL benefiting from the combination of episode-dependent statistical normalization and a predictor trained on the resulting representations, rather than whitening being a universally beneficial preprocessing step.
The main experiments and component ablations supporting this interpretation were already present in the original submission. We have revised the Ablation Study (Sec. 6.3) to clarify their interpretation of these findings and Appendix D.2 to provide further implementation details for the fixed and trainable projection ablations.
W4. PCA rank and TAIL's sensitivity to feature scale.
(a) PCA rank. Thank you for pointing out this ambiguity. We have clarified the relationship between the PCA estimation sets and their maximum ranks in Episodic Whitening (Sec. 5.1): “We set . RepICL-I estimates PCA from the 25 support examples alone, whereas RepICL-T also includes the 50 unlabeled queries, giving maximum ranks after centering of and , respectively.”
(b) TAIL and feature scale. Table 1 reports 52.79% for TAIL on Text-held-out dataset adaptation; we could not find 5.70%. The reviewer is right that TAIL is sensitive to scale. Without calibration it drops to 24.3–38.2% (Appendix C.4), so every TAIL result in Table 1 uses scale calibration. With calibration it is still below LR in all 12 settings. We therefore do not attribute the gap to feature scale alone. Generalizing to new representation spaces remains difficult after scale is corrected, and this is the research problem RepICL addresses.
Thank you for the revisions. I'm happy to see W4(a) is now fixed, and I appreciate the addition of MOKD for W1—though I'd gently note that a single CDFSL baseline still leaves the cross-domain comparison somewhat thin.
On W3, I hope this comes across as constructive rather than repetitive: the response confirms that the pairing works (whitening alone doesn't), which I fully agree with, but I was hoping for an explanation of why the pairing matters rather than the observation itself. Similarly, W2 describes what the method does, but it would be helpful to say why it must be PCA specifically, and which of the three contributions is the primary one.
I'll keep my score for now, but I'd genuinely encourage the authors to address these "why" questions directly in a future version—I think doing so would strengthen the paper considerably.
Response to Reviewer giHH (1/2): W1
W1. Cross-domain few-shot learning (CDFSL) baselines.
Following the reviewer's suggestion, we evaluated the feature-adaptation component of MOKD [1], a recent cross-domain few-shot learning method (ICML 2024), on RepShiftBench and added the results to Table 1. The method is introduced in Section 6.1, with implementation details in Appendix C. RepICL-I achieves higher accuracy than MOKD in all 12 settings without test-time parameter updates. MOKD exceeds Logistic Regression only in modality adaptation under the image-held-out fold, and falls below it in the other 11 settings. These results suggest that methods developed for cross-domain few-shot image classification do not necessarily provide consistent gains over Logistic Regression across heterogeneous frozen representation spaces, consistent with the observations in Appendix E.
Several of the suggested methods require inputs or model access beyond frozen representations. IFR [2] uses augmented images and spatial features, FAD [3] adapts intermediate backbone features, and Bayes-CAL [4] uses class and domain names for image-text alignment. These methods therefore cannot be applied directly through RepShiftBench's common interface of pooled representations and support labels. MF-GIA [5] studies in-context alignment over pre-vectorized graph features, but its formulation uses graph structure and differs from few-shot prediction across general frozen encoder representations.
The LoRA comparison in Appendix E.2 serves as a reference for per-episode adaptation of the feature encoder at test time, rather than as a CDFSL baseline. Prior work has evaluated LoRA among parameter-efficient methods for few-shot image classification [6] and applied LoRA to few-shot adaptation of vision-language models [7]. LoRA uses the same labeled support examples but additionally accesses the raw inputs and encoder parameters.
References
[1] Hongduan Tian, Feng Liu, Tongliang Liu, Bo Du, Yiu-Ming Cheung, and Bo Han. MOKD: Cross-domain finetuning for few-shot classification via maximizing optimized kernel dependence. Proceedings of the 41st International Conference on Machine Learning, PMLR 235, pp. 48154–48185, 2024.
[2] Hongduan Tian, Feng Liu, Ka Chun Cheung, Zhen Fang, Simon See, Tongliang Liu, and Bo Han. Cross-domain few-shot classification via invariant-content feature reconstruction. International Journal of Computer Vision, 134, article 54, 2026.
[3] Ruixiao Shi, Fu Feng, Yucheng Xie, Jing Wang, and Xin Geng. FAD: Frequency adaptation and diversion for cross-domain few-shot learning. arXiv preprint arXiv:2505.08349, 2025.
[4] Lin Zhu, Weihan Yin, Fan Wu, Qinying Gu, Xinbing Wang, Chenghu Zhou, and Nanyang Ye. Bayes-CAL: Robust cross-modal alignment by Bayesian approach for few-shot OoD generalization. International Journal of Computer Vision, 133(10), pp. 7076–7109, 2025.
[5] Wei Zhuo and Siqiang Luo. Modality-free graph in-context alignment. International Conference on Learning Representations, 2026.
[6] Samyadeep Basu, Shell Xu Hu, Daniela Massiceti, and Soheil Feizi. Strong baselines for parameter-efficient few-shot fine-tuning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(10), pp. 11024–11031, 2024.
[7] Maxime Zanella and Ismail Ben Ayed. Low-rank few-shot adaptation of vision-language models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 1593–1603, 2024.