acceptodds
Withdrawn from ICLR 2027

Controlling Object-Induced Shortcuts for Compositional Zero-Shot Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Compositional zero-shot learning requires recognizing unfamiliar combinations of familiar attributes and objects. Sparse training combinations make object identity a strong predictor of the composition label, allowing a model to fit observed associations without learning sufficient visual evidence for the attribute. Yet object information also provides the context needed to interpret attribute appearance. We propose , a framework that regulates this dual role across representation, optimization, and inference. A feedback adapter pools attribute evidence under complementary object-related supports and uses their directional residual to guide token updates. Two training objectives evaluate composition scores relative to object support, while an image-dependent score correction balances competition between object groups at inference. These controls address distinct parts of composition recognition: learning discriminative attribute evidence and balancing its competition across objects. Experiments on three standard benchmarks demonstrate strong compositional generalization, with state-of-the-art AUC and harmonic mean on MIT-States and open-world UT-Zappos. All three regulation stages consistently improve recognition across datasets while adding little inference cost.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Compositional Zero-Shot Learning, Shortcut Mitigation, Object-Conditioned Bias, Representation Debiasing, Score Calibration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_rS5SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SPARC, a framework for compositional zero-shot learning (CZSL) that reduces object-induced shortcuts while preserving useful object context. It introduces a complementary-evidence feedback adapter to refine visual representations, along with object-discounted loss, hierarchical margins, and inference-time score correction to mitigate object bias. Experiments on three CZSL benchmarks demonstrate competitive performance in both closed-world and open-world settings, supported by ablation studies.

Critical strengths
  1. The paper identifies a relevant challenge in compositional zero-shot learning, where models may exploit frequently observed attribute-object associations rather than learning sufficient visual evidence for attributes. The distinction between useful object context and harmful object-induced shortcuts provides a reasonable motivation for improving compositional generalization.
  2. The proposed framework addresses object influence at three different stages through complementary-evidence feedback, object-relative training objectives, and image-dependent score rectification.
  3. The experiments demonstrate competitive results on three standard CZSL benchmarks under closed-world and open-world settings.
Critical weaknesses
  1. The methodological novelty of SPARC appears limited, as it mainly combines established techniques such as attention-based feature adaptation, residual projection, auxiliary losses, and score calibration. Similar ideas have been explored in existing CZSL methods, including CPF (ICCV 2025) for object-conditioned attribute learning, ProLT (AAAI 2024) for distribution-prior adjustment, and Mutual Balancing (Pattern Recognition 2024) for balancing attribute-object components.
  2. Although the paper aims to mitigate object-induced shortcuts, the experiments mainly focus on recognition accuracy without directly measuring shortcut reduction. It remains unclear whether the improvements stem from better attribute grounding or generic feature enhancement and score calibration. The authors should include controlled analyses of predictions for the same object with different attributes, as well as evaluations under varying attribute-object co-occurrence biases, to better validate the effectiveness of shortcut mitigation.
  3. The semantic validity of the proposed object-related and complementary supports remains questionable. Low object similarity does not necessarily indicate informative attribute evidence, as it may instead arise from background clutter, occlusion, or noise, particularly in challenging datasets such as MIT-States and C-GQA. It would be better to provide attention map visualizations to verify whether the proposed decomposition effectively distinguishes object-related regions from meaningful attribute cues.
Questions for authors

Please see the Weaknesses.

Reviewer_vsZbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how object information can provide useful context for attribute recognition while also encouraging reliance on training co-occurrences. SPARC addresses this tension at three stages. A visual adapter pools attribute evidence under complementary object-related supports and uses a directional residual to control token updates. Training combines three-branch classification with an object-discounted composition loss and a pair-versus-object specificity margin. Inference combines composition and primitive scores, then subtracts a standardized, image-dependent object-alignment term. Experiments cover MIT-States, UT-Zappos, and C-GQA under closed- and open-world evaluation, with cumulative stage ablations and separate comparisons of the training losses and inference rules.

Critical strengths
  1. The idea of introduing object context for compositional recognition is simple and effective.

  2. The method is clear and easy to follow.

Critical weaknesses
  1. As shown in Table 1, the performance advantage of the proposed SPARC is not significant. Thus, it is not clear that the directional residual feedback is responsible for the representation gain or reduces object-induced shortcuts.

  2. As discussed in the related work section, the linear adapter has been widely used and been available to a CZSL model. The novelty is limited when compared to these works.

  3. In the method section, for image-dependent object correction, it lacks the theoretical proof or experimental verification.

  4. Figure 1 shows the motivation of the proposed method. Green apples are also a relatively common combination compared to red apples. Thus, taking the visual information as a condition should focus on more tailed-class classfication or other cases.

  5. In the experiment section, the top-1 accuracy on seen (S) and unseen (U) compositions should also be reported.

Questions for authors

On one representative benchmark, please also report attribute accuracy conditioned on correct object predictions, ideally with results grouped by training co-occurrence strength. This would help determine whether the gain arises from the proposed feedback mechanism and whether it addresses the specific shortcut behavior motivating the paper.

Reviewer_LVrYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the well-known problem in compositional zero-shot learning (CZSL), where models tend to overlook attribute information and rely on object-biased attribute associations for prediction. The authors aim to balance object and attribute information by reducing object-induced shortcuts while preserving useful object information. They propose a three-step strategy:

  1. Representation learning: Uses attribute information from regions with different levels of object similarity.

  2. Optimization: Trains the model to predict the correct composition even when some object information is reduced.

  3. Inference: Adjusts the composition scores to reduce the influence of strong object information on the final prediction.

The authors evaluate the proposed method on three CZSL benchmarks: MIT-States, UT-Zappos, and C-GQA, under closed-world and open-world settings. The results show competitive performance, with improvements over the adapter baseline and SOTA results on selected settings.

Critical strengths
  1. Overall, the authors have presented an interesting pipeline to address object-induced shortcuts, where the model relies more on object information while overlooking the actual attribute representation.

  2. The proposed method tries to balance object and attribute evidence, which is important for recognizing unseen compositions.

  3. The ablation studies show consistent improvements with the addition of representation learning, object-relative training objectives, and inference correction across the three datasets.

  4. At test time, the proposed correction is computationally inexpensive and preserves the ranking of attributes within each object group.

Critical weaknesses
  1. The authors highlight an interesting problem of object-induced shortcuts. However, the overall performance gains are limited. For example, on MIT-States (closed-world), AUC improves from 24.0 to 24.8 and HM from 41.0 to 41.8. On the other datasets, the results are lower than the SOTA.

  2. In lines 031–032, the authors state that "once an object is recognized, its frequent training attributes become plausible answers." However, no analysis or experimental validation is provided to support this claim. The reported benchmark results in terms of AUC and HM are not sufficient to establish the paper's contribution toward addressing object-induced shortcuts.

  3. The complete approach combines a stronger adapter, additional training objectives, and inference calibration. However, there is no clear evidence that the improvements specifically come from reducing strong object-attribute associations learned during training. The seen and unseen accuracies are provided in the appendix, but they do not show gains over existing methods. A deeper analysis of seen and unseen compositions grouped by object–attribute association strength, particularly when a familiar object appears with an unseen attribute combination, would help validate the proposed motivation.

  4. In the complementary support module, a token with low object similarity may contain useful attribute information, but it may also represent irrelevant background, lighting, or clutter. The paper does not establish that the complementary support consistently captures more discriminative attribute evidence.

  5. The provided ablation studies do not independently evaluate the contributions of semantic object routing, complementary pooling, residual projection, feedback broadcasting, and gating. These experiments would help identify which components are actually responsible for the improvements.

  6. The paper analyzes the specificity margin (m), but does not provide a sensitivity analysis of (\lambda) or how the choice of both parameters affects performance.

  7. The paper is difficult to follow and understand. For example, the architecture figure and in line 034 "without resolving the attribute" where one can assume what authors are trying to say its not clear from the sentence.

  8. The results of prior methods are taken from published tables rather than reproduced under a controlled implementation. This limits the understanding about the effectiveness of the proposed method compared with existing approaches.

Questions for authors
  1. Please address the weaknesses and provide additional experimental/conceptual evidence to support the proposed claims.

  2. The paper claims that SPARC mitigates object-induced shortcuts. However, the reported improvements in classification performance do not necessarily demonstrate that shortcut learning has been reduced. How can the authors establish that these gains arise specifically from mitigating object-induced shortcuts rather than from a more expressive adapter, additional training objectives, or score calibration? A controlled analysis and comparisons with baselines would strengthen this claim.

  3. Since shortcut learning has been well studied in the literature, how is this paper's contribution different from previous works addressing the same problem? How would directly applying existing shortcut mitigation methods compare with the proposed method?

  4. Why should high object alignment be penalized at inference if the object is genuinely present and correctly recognized? Could this penalization suppress useful object information?

  5. The seen and unseen accuracy results are provided in the appendix. Move these results to the main paper, along with a detailed analysis validating the proposed problem. In particular, analyzing seen and unseen compositions grouped by object-attribute association strength, especially when familiar objects appear with unseen attributes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.