- Rating
- 4
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
To address the semantic leakage problem in conditional representation learning, the authors propose Criterion-Specific Basis Refinement (CSBR) and Null-Space Denoising Projection (NSDP) on top of CRL to refine the conditional representations, thereby improving the semantic purity of conditional representations. Extensive experiments on customized clustering, customized few-shot classification, and customized retrieval tasks demonstrate the superiority and generalization of the proposed method.
Critical strengths
- The paper achieves consistent performance improvements on all three downstream tasks, and the gains persist across three VLM backbones (CLIP, BLIP2, SigLIP).
- The proposed CSBR and NSDP perform basis refinement and null-space denoising of the conditional subspace, and the paper validates the effectiveness of the method through extensive comparative and ablation experiments.
Critical weaknesses
- The self-constructed Flowers dataset reaches 100% on the Species criterion. Does this indicate that the dataset itself is too simple?
- The paper emphasizes being training-free and efficient, but lacks an analysis of the cost of the NSDP greedy search as the number of criteria grows.
Questions for authors
Please see the weaknesses.
4 comments
Response to W2: Computational Cost of NSDP greedy search
Thank you for this insightful question. Although SP-CRL is training-free, we acknowledge that NSDP's greedy search introduces additional computational overhead. We therefore provide a theoretical complexity analysis and additional experiments evaluating its runtime as the number of candidate non-target criteria increases.
Computational Complexity of NSDP. Let denote the number of candidate non-target criteria, the number of greedy search rounds, the number of initial text basis vectors per criterion, the embedding dimension, and the number of image embeddings. At round , NSDP evaluates candidates, each involving at most basis vectors. Accounting for subspace refinement and denoising priority computation, the overall time complexity is upper-bounded by
=O\left(\sum_{j=1}^{s}(m-j+1)jnd(N+d)\right) =O(ms^2nd(N+d)).$$ In the worst case, $s=m$, yielding an $O(m^3nd(N+d))$ upper bound. **Scalability of NSDP.** To evaluate how the computational cost of NSDP scales with the number of candidate non-target criteria, we conduct a stress test on CLEVR4-10k with $m\in\{2,4,6,8,16,32\}$ simulated non-target criteria, each containing 512 initial text basis vectors. All experiments are conducted on an NVIDIA RTX 5070 Ti GPU, with runtimes averaged over 100 runs after 30 warm-up iterations. We disable early stopping (i.e., terminating the search when the denoising priority no longer improves) to evaluate the full greedy search procedure. All results are reported as NSDP greedy search runtime (s). | Target Criterion | 2 | 4 | 6 | 8 | 16 | 32 | |:--|--:|--:|--:|--:|--:|--:| | Shape | 0.227 | 0.538 | 0.873 | 1.721 | 6.415 | 37.509 | | Color | 0.163 | 0.438 | 1.029 | 1.433 | 6.935 | 36.070 | | Count | 0.183 | 0.581 | 1.047 | 1.506 | 5.989 | 37.019 | | Texture | 0.218 | 0.467 | 1.127 | 1.440 | 6.951 | 37.906 | As shown above, NSDP's runtime increases with the number of candidate non-target criteria, consistent with the theoretical analysis. Even with early stopping disabled, the search completes within 1.73 s for up to 8 criteria and within 38 s for 32 criteria. These results demonstrate that NSDP's greedy search incurs manageable one-time computational overhead. Moreover, the number of non-target criteria is typically limited in practical scenarios, and early stopping can further reduce the computational cost.I thank the authors for their response. The Flowers v2 experiment addresses my concern about the saturation of the original dataset, and the complexity analysis together with the scalability test clarifies the cost of the NSDP greedy search as the number of criteria grows. Both of my concerns are now resolved, so I keep my rating at 4.
We are glad that your concerns are fully resolved.
Dear Reviewer odza,
We are glad that your concerns are fully resolved. Thanks again for your time and effort in reviewing this paper. And thanks for your supportive and positive rating.
Regards from the authors.
Response to W1: Difficulty of the Flowers Dataset
Thank you for raising this important concern. We acknowledge that the Species criterion in the original Flowers dataset exhibits performance saturation, as several baseline methods already achieve near-perfect clustering results. To address this concern, we construct an expanded dataset, Flowers v2, containing 1,756 images with 40 species categories, compared with 533 images and 4 species categories in the original dataset.
To ensure a fair comparison, we follow the same experimental protocol as the original Flowers dataset, including the VLM backbone, clustering procedure, evaluation metrics, and repeated evaluations with different random seeds. All methods are evaluated under identical settings. The clustering results on Flowers v2 are summarized below:
Flowers v2 provides a more challenging evaluation under the Species criterion, where SP-CRL achieves 90.17% NMI, 81.37% ACC, and 78.94% ARI, rather than the saturated performance observed on the original dataset. Importantly, SP-CRL consistently outperforms all compared methods across all six evaluation metrics under both Species and Color criteria. These results further demonstrate the effectiveness of SP-CRL on a fine-grained dataset with substantially more species categories. We have uploaded both Flowers v1 and Flowers v2, organized by species, to the supplementary materials.