- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies adaptation when only a fixed tensor of prompt-level class scores is available, without access to images, embeddings, model updates, labels, or additional VLM queries. Paper proposes PCRI which constructs a shared representation from normalized class contrasts, groups prompt-specific deviations, and subtracts their ridge prediction from the shared representation. It uses the combined representation to construct prototypes from confident base pseudo-labels, then fuses prototype scores with the frozen base scores.
Critical strengths
-
This paper concerns a practical and meaningful setup where deployment interface exposes only prompt-level scores, and feature-based adaption is not feasible.
-
The empirical design is deliberate, and clearly isolate effectiveness of different designs. under matched comparisons, PCRI improves on 32 out of 33 settings.
Critical weaknesses
-
The practical gain from distinctive component is modest. For example, standardized score prototypes already improves 1.884 while PCRI only improves 2.378 with dataset-level core normalization contributes only 0.003.
-
Global fusion scale can reintroduce a nuisance removed by the representation. Eq.12 uses the pooled standard deviation of the base scores. With MPE aggregation, adding a sample-specific constant to every prompt score preserves base decisions, but will change this standard deviation. As a result, the correction strength and final predictions can change despite unchanged class contrasts.
Questions for authors
- Why eq.12 includes variation in sample-wise logit means? Can you report an offset stress test and compare the current scale with std(B-row_mean(B))? This would establish whether the identified sensitivity matters on the benchmarks.
Response to Reviewer SzFS
Thank you for the constructive comments. We address the scale sensitivity and component value separately, with matched GPU experiments.
1. Response to Question 1 and Weakness 2 — Fusion-scale sensitivity.
Eq. (12) matches the global dispersion of the structural and base score matrices, allowing alpha to control correction strength across their numerical scales. Pooled Std(B) includes variation in sample-wise logit means, which leaves base class rankings unchanged. To examine this sensitivity, we implemented your suggested Std(B - row_mean(B)) as a row-centered scaling variant and clarified both formulations beside Eq. (12).
Across all 33 settings, we tested six offset levels k = {0, 0.25, 0.5, 1, 2, 4}, including zero, with three fixed seeds. Gaussian vectors were centered, normalized to unit population variance, and multiplied by k Std(B - row_mean(B)). Both rules received identical offsets. Representations, support and prototypes were fixed to isolate fusion-scale sensitivity; base predictions remained unchanged. The new GPU zero-offset gain is +2.382 pp, supplementing the submitted +2.378 pp result.
With injected offsets, at k = 4 the original rule's gain fell from +2.382 to +2.208 pp (-0.174 pp), with 7.078% of predictions changed relative to its own zero-offset output. Row-centered scaling retained +2.311 pp with no prediction changes at any tested level.
Without injected offsets, row-centered scaling gave +2.311 versus +2.382 pp, a difference of -0.071 pp (95% CI [-0.169, +0.011]; 8 wins, 3 ties, 22 losses). Row-centered scaling kept predictions unchanged across the tested offsets; the zero-offset accuracy comparison remained statistically inconclusive. Natural benchmarks may themselves contain offset variation.
We retain Eq. (12) for the submitted primary results and report row-centered scaling as an optional variant. Appendix D.6 provides the protocol and stability/accuracy figure. The invariance result concerns fixed-state fusion, without refitting the upstream aggregator.
2. Response to Weakness 1: Incremental component gains and the normalization control.
Incremental value beyond standardized-score prototypes. The submitted +1.844 pp result already implements PCRI’s shared-Core relational route: standardized-score prototypes with semantically anchored fusion. The component comparisons therefore assess both this shared-Core route and the additional predictive value of conditional prompt information.
To separate the value of the whole prompt branch from conditioning itself, we compare Full against Core-only and same-dimensional Direct-H. Across 33 matched GPU settings, Full exceeds Core-only by +0.531 pp (95% CI [+0.296, +0.787]; 29/33 improvements). This gain measures the additional benefit of the complete prompt branch after Core correction. Full also exceeds same-dimensional Direct-H by +0.316 pp ([+0.169, +0.460]; 31/33 improvements), supporting a contribution from conditioning beyond merely adding prompt information.
These are average increments: on Oxford-IIIT Pets, the Core and prompt-branch gains are +1.117/+0.763 pp with ViT-B/16 and +1.254/+0.818 pp with RN50. The incremental value is therefore not uniformly small. Across the full evaluation, the mean Full–Core gain remains positive on every backbone and ranges from +0.445 to +0.584 pp when any one dataset is omitted. The complete 33-setting breakdown, including negative increments, is provided in Appendix D.5.
The +0.003 pp normalization increment. This increment isolates the auxiliary upstream BlockNorm within the shared-Core pipeline. We test its practical relevance with a simpler normalization control. Removing upstream BlockNorm from the comparator while retaining the prototype head’s normalization yields One-BN Core; Full still exceeds it by +0.540 pp ([+0.298, +0.801]). Full also exceeds raw-score cosine prototypes by +0.725 pp ([+0.381, +1.074]). Thus, its incremental benefit persists against both a normalization-simplified Core and a stronger raw-score alternative.
In three resident-score GPU cases, Full takes 1.25–2.95× One-BN Core latency, while conditioning adds approximately 4–5% over Direct-H, without further VLM queries. These results support the relational correction and conditional prompt information as the substantive contributions, with an explicit accuracy–latency trade-off. Component controls and timing details are provided in Appendices D.5 and G.2, respectively.
Component intervals use 100,000 paired dataset-cluster bootstrap draws over 11 datasets, retaining all three backbones; they are pointwise and unadjusted. GPU results supplement the submitted results.