- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the compositional generalization task, where a model should recognize a new combination of seen attributes. It focuses on the problem that an unseen combination is often mapped onto one specific seen combination. The authors propose a method to predict such failure before training using the mean image, and then moves the prediction by deleting the captured targets. The method is tested on several datasets to show its effectivenss.
Critical strengths
-
This paper makes the classificaiton failure more predictable with less costs. On dSprites, the nearest seen mean image predicts the captured combination with much higher accuracy than uniform chance. And it only uses the input and training split, without examining the trained model.
-
The experiments provide evidence that deleting the predicted target and retraining moves the prediction more often than re-seeding or a distance matched deletion.
Critical weaknesses
-
The paper is hard to follow. The authors use lots of unnecessarily long sentences separated by commas to express a simple meaning, making it difficult to understand.
-
Missing citation. The method in Atzmon et al., learns separates attribute and object representations by encouraging each representation to be invariant to others. This directly relates to the proposed specialization mechanism.
-
The explanation of the failure remains incomplete. Proposition 1 explains why the prediction heads select a nearby seen combination. However, it does not explain why the encoder places the unseen input near that combination. This gap matters because the deletion experiment retrains both the encoder and the heads, while the theory takes the encoded representation as given. The experiments show that changing the training support changes the wrong prediction, but the encoder’s role in this process remains unclear.
[1] Atzmon, Y., Kreuk, F., Shalit, U., & Chechik, G. (2020). A causal view of compositional zero-shot recognition. Advances in Neural Information Processing Systems, 33, 1462-1473.
Questions for authors
Can the authors explain the connection and differences between their specialisation mechanism and [1]?