- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Koo-Fu CLIP adapts frozen visual embeddings using labeled within-class and between-class scatter matrices. A regularized whitening step followed by an ordered discriminative rotation gives a linear transform that can be truncated to different output dimensions without refitting for a fixed regularization setting. Experiments compare nearest-visual-prototype and 15-NN classification across three encoders and seven datasets, including learned linear projection baselines.
Critical strengths
I like the practical simplicity of fitting one transform and then choosing among several embedding sizes without retraining. The evaluation across CLIP, SigLIP2, and DINOv3 shows that the approach can be useful beyond one particular encoder, with especially substantial gains on FungiTastic-Mini. The learned linear comparisons are also helpful, and the paper is clear that those methods often achieve higher accuracy. That makes the proposed method an interesting option when a compact representation and a simple fitting procedure matter.
Critical weaknesses
-
I'm a little unsure how I'd choose a compact configuration before seeing its final evaluation results. Section 4.1 uses validation data to select the regularization and also to report performance, while Tables 1–2 report the best tested reduced dimension. The splits and the source/regularization within each dimension sweep are fixed, which is helpful; my question is whether selection and final evaluation are separate. Could you clarify that and check one existing feature setting with a separate selection split? That would make the reported compression gains easier to interpret as something a user can expect after choosing a configuration.
-
Could we unpack what the between-class ordering adds to the compression gain? At full dimension, the rotation in Eq. (15) leaves cosine similarities unchanged, so those results already capture the corresponding whitening effect. The reduced representations do improve further, but I'm wondering how much depends on that particular ordering of directions. Keeping the whitening matrix fixed and comparing against PCA ordering of the whitened features at one existing nonzero-regularization setting, such as CLIP/ImageNet-1k at lambda = 500 and 192 dimensions, would help separate discriminative ordering from a more general benefit of compression.
-
The learned linear comparisons in Table 5 are useful, and I'd love to see the other half of that tradeoff: how much fitting time does Koo-Fu save? At 192 dimensions on CLIP/ImageNet-1k, for example, it gets 82.18% versus 82.52% for the stronger learned baseline, so the practical case depends on the fitting savings. Could you add timing for one existing setting, showing the cost of producing one embedding size and several sizes, including regularization selection and keeping shared feature extraction separate? Existing timing logs may be enough, and this would make the benefit of reusing one fit much easier to judge.
Questions for authors
Could you clarify the selection/evaluation separation, the added value of between-class ordering under fixed whitening, and the fitting-time tradeoff for one versus several output sizes? I'm referring to the same three points above; these are the pieces that would most change my assessment.