- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Cue-to-Context Subsumption Learning for Audio-Visual Generalized Zero-Shot Learning. The method aims to reduce the influence of varying backgrounds and contexts by using localized audio-visual event cues to guide full-sample representations. Specifically, CCSL extracts visual regions and auditory segments as localized box inputs and maps both box and full inputs into Gaussian distributions through a shared distribution mapper. A directional KL-based subsumption objective is introduced to encourage mean alignment and constrain their relative variances. The framework also incorporates category classification, box-to-text alignment, and variance regularization losses. Experiments on VGGSound-GZSL, UCF-GZSL, and ActivityNet-GZSL demonstrate consistent improvements over the compared baselines, with harmonic mean gains of 5.75–7.84 percentage points under CLIP and CLAP features. Ablation studies further examine the contributions of localized boxes and individual learning objectives.
Critical strengths
- Consistent and substantial empirical improvements. The proposed method achieves notable improvements in harmonic mean accuracy across three AV-GZSL benchmarks, particularly for unseen categories. The gains are also observed under different feature configurations, suggesting that the approach is not limited to a specific pretrained backbone. I consider this consistent empirical improvement the strongest reason to support acceptance.
- Intuitive motivation and meaningful experimental validation. The idea of using localized event cues to guide full-sample representations is intuitive and addresses a relevant challenge in AV-GZSL. The authors also provide useful ablation studies showing that simply adding localized boxes as training samples or replacing the proposed objective with MSE alignment does not yield comparable performance. These results provide evidence that explicitly modeling the relationship between localized and full representations is beneficial. Additionally, box extraction is only required during training, which avoids introducing an extra box-extraction step at inference time.
Critical weaknesses
- Limited justification of the proposed subsumption relationship. Although the directional KL-based objective enforces a specific relationship between the means and variances of the box and full distributions, the connection between this constraint and the claimed semantic subsumption remains insufficiently justified. In particular, maintaining a relative variance ratio does not necessarily guarantee that the box distribution captures a meaningful range of contextual variations. A more detailed explanation of the variance behavior would strengthen the methodological contribution.
- Insufficient direct validation of context robustness. The paper is motivated by the challenge of background and context variations, but the experiments mainly focus on standard benchmark performance. A direct evaluation under controlled background changes would provide stronger evidence for the central motivation. Furthermore, comparisons with simpler objectives, such as mean-only alignment or an unscaled KL loss, would help clarify whether the improvements specifically arise from the proposed directional subsumption formulation.These limitations reduce my confidence in the interpretation of the proposed mechanism, although they do not outweigh the consistent empirical improvements reported in the paper.
Questions for authors
- Could the authors provide additional comparisons with simpler alternatives, such as mean-only alignment between box and full representations, an unscaled KL objective, or reverse KL divergence? These comparisons would help clarify whether the performance gains primarily come from the proposed directional variance constraint or from more general localized feature alignment.
- The proposed objective encourages a specific variance ratio between the box and full distributions. However, when the means are aligned and the target variance ratio is satisfied, both variances could potentially shrink together without increasing the subsumption loss. Could the authors clarify how this behavior is avoided in practice and whether the learned variances capture meaningful contextual variations?
- Since the main motivation is to reduce the influence of varying backgrounds and contexts, could the authors provide additional evidence of robustness under background changes, such as controlled context-shift experiments or performance analysis across different background conditions? Such results would help establish a clearer connection between the proposed mechanism and its intended benefit.