- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes FASC and investigates whether, after a VLM backbone has already produced composition scores, semantic relationships among attributes can further improve compositional zero-shot recognition. The method organizes attributes into fixed semantic families, aggregates the probability mass of candidate compositions to obtain image-dependent family evidence, and then combines this evidence with frequency priors to rerank the original candidates. A pseudo-unseen ranking objective is used to encourage a more favorable seen/unseen trade-off. Therefore, the contribution of this paper lies in structurally interpreting and adjusting existing scores, rather than proposing a new compositional representation. The closed-world and open-world results and the controlled experiments provide fairly credible evidence that this calibration procedure has practical value.
Critical strengths
-
The use of intermediate semantic structure is well motivated. The paper recognizes that attribute prediction is heterogeneous. The reliability of color, physical state, and material evidence may differ, and uncertainty at the composition level may coexist with more consistent evidence at a coarser semantic level. Treating families as an intermediate structure while retaining predictions at the composition level is a reasonable design. It provides a path to exploit shared evidence without replacing the original task with family classification.
-
The calibration method is practical and complementary. The interface based on logits makes FASC easy to combine with existing CZSL scoring models. The method does not require retraining representations and does not change the candidate vocabulary set, so it has practical value as an additional calibration stage. Experiments on other scoring models support this modular design and show that the method is not limited to the specific implementation of the main backbone.
-
Using baselines from the same source and matched calibrators that do not use family information controls for two key confounders, checkpoint averaging and generic calibration. Taxonomy controls and repeated fitting further demonstrate attention to attribution. For a pipeline with multiple interacting components, such an experimental design is informative.
Critical weaknesses
-
At the evidence level, the core question is whether semantic grouping can further improve recognition after the benefits of ensembling and generic score calibration have already been taken into account. When the source logits and non-family calibration components are held consistent, the remaining gain from family structure is limited. For example, on C-GQA the AUC difference relative to Generic is only 0.04. Therefore, uncertainty in paired comparisons is critical. Repeatedly fitting the calibrator after fixing the source subset and taxonomy can only establish fitting stability. The core claim also requires evidence that the gains relative to matched alternatives are similarly stable. We recommend that the authors add paired repeated experiments for family and non-family variants and use multiple random taxonomies to directly evaluate the semantic effect itself. This would help determine whether family structure provides reproducible useful information or whether the current evidence mainly supports the practical gains from combining existing scores with calibration terms.
-
Family marginalization has a clear mathematical definition, but the resulting quantity mixes visual evidence with the influence of candidate space structure. Even when candidate logits are exactly the same, larger families receive more probability mass. Therefore, the posterior may favor a family because it contains more candidates or inherits object score biases, not because the image provides stronger evidence for that family's semantic cues. Keeping family size in the random control is useful, but it is not sufficient to establish that this aggregation statistic actually reflects the intended visual signal. We suggest that the authors compare with statistics adjusted for candidate count and add family prediction diagnostics as well as an analysis split by whether object recognition is correct. This would make the mechanism more convincing.
-
We suggest that the authors conduct matched open-world comparisons on the same source logits, with particular attention to differences in performance when object recognition is correct versus incorrect. They should also explain how feasibility filtering interacts with family marginalization, since it changes the evidence that the calibrator receives. The additional backbone experiments establish interface compatibility, but they are limited to the closed-world setting on a single dataset and do not yet fully support broader generalization claims. These analyses would identify when semantic aggregation can improve compositional recognition and when it may amplify ambiguity.
-
The writing of the paper needs a comprehensive revision so that readers can better understand the method. For example, the typesetting of Eq. 8 requires the authors to conduct a complete check of the paper.
Questions for authors
Please address the above issues.