- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a framework for disentangling pretrained visual embeddings into concept-specific components. The key idea is to model each semantic concept as a cone spanned by a group of dictionary atoms, such that an image embedding can be approximately decomposed into a sparse sum of non-negative concept components. The authors develop both a supervised dictionary-learning formulation and an autoencoder-based implementation, and further extend the framework to self-supervised and unsupervised settings. The learned concept-specific representations are mainly evaluated through concept-filtered image retrieval, including both coarse concept retrieval and finer-grained sub-label retrieval.
Critical strengths
- The paper is built upon an interesting and relatively novel motivation. Rather than treating pretrained visual embeddings as holistic representations, it explores whether their semantic information can be explicitly decomposed into concept-specific components, which provides an appealing perspective on representation disentanglement and controllable retrieval.
- The experimental evaluation is relatively comprehensive. The authors conduct experiments across multiple datasets and pretrained visual backbones, consider both general-label and fine-grained sub-label retrieval, and provide several ablation studies and qualitative analyses to examine different aspects of the proposed framework.
- The proposed method is technically well developed. The paper provides both a constrained dictionary-learning formulation and an autoencoder-based implementation, with carefully designed non-negativity and group-sparsity constraints.
Critical weaknesses
- The experimental evaluation could be strengthened by including additional relevant baselines. In particular, it would be helpful to compare with existing composed retrieval methods or other approaches designed to focus on user-specified semantic content. At present, a considerable portion of the experimental comparisons are among variants of the proposed framework itself.
- The evaluation protocol for the unsupervised variant could also be explained more clearly. Since the learned latent groups do not have predefined semantic labels, the paper associates active concepts with latent groups separately for each query using CLIP text embeddings and greedy exclusive assignment. As a result, the same latent group may correspond to different concepts for different queries.
- I would also appreciate some clarification on the hyperparameter-selection protocol. The paper emphasizes that SLiCC is fitted using only general concept labels, while the dictionary size d_0 is selected according to sub-label retrieval performance on the validation set. Therefore, fine-grained labels are not used in the training objective but are involved in model selection.
- The visual presentation of the figures could be improved. In addition, Figures 3, 4, and 5 contain several presentation details that could be refined, such as the bounding boxes, spacing, and occasional overlap between text and images.
- The related-work discussion could be expanded to include “Conditional Representation Learning for Custom Tasks” (NeurIPS 2025). This work is relevant in spirit, as it studies how to adapt or transform representations to focus on a specified concept or criterion from the perspective of coordinate transformation.
Questions for authors
See the above-mentioned weaknesses.
2 comments
Thanks for the response. My concerns have been resolved, and I will keep my score.
Answers for the questions in **Critical Weaknesses**
C1.
We thank the reviewer for the concern. We have included more methods in the revised Table 1, including one projection-based method (CAV), one composed image retrieval method (CLAY), and one supervised metric learning method (CSN). The results show that SLiCC remains more effective in most of the cases, proving the strength of our proposed methods.
C2.
That is correct. For each query, the goal is to assign one unsupervised latent group to each active concept.
Based on the cosine similarity between the CLIP text embeddings and the latent group components, each concept claims the most similar free latent group. If several concepts claim the same group, the one with the highest similarity score wins; the losers try again on the remaining groups. The process stops when every concept is assigned. Since the same procedure is carried out every time a different query comes in, the same group can map to different concepts for different queries.
C3.
That is correct. Even though the fine-grained labels are not used in the training process, the hyperparameters are selected through the sub-label retrieval performance, which requires the information of the finer-grained labels on the validation set. This is necessary to find the size concept cones necessary to preserve meaningful variation within the concept. This is necessary as using only the coarse-grain labels for validation selects a lower-dimensional cone that fails to encode fine-grained information.
C4.
We apologize for the bad presentation of the figures. The figures have been fixed in the revised paper.
C5.
We thank the reviewer for bringing this work to our attention. It is indeed similar in spirit to our approach. Although its customized conditional retrieval takes into account the object factor and the scene factor at the same time, it inspires us to adopt the prompts to generate lists of expressions and use them for conditional visual similarity modulation (CLAY), which is also included in the revised paper as a baseline.