- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors propose to represent words as bundles of conceptual features rather than as atomic units. This is hypothesized models' ability to generalize compositionally, presumably because the conceptual features can generalize even when the full word doesn't. The authors show that this method is effective on a number of synthetic generalization tasks.
Critical strengths
None.
Critical weaknesses
-
No insight is provided into what the model learns and why it works. It is unclear what the "concepts" that the model ends up learning are, and how they help generalization. The comparison to tokenization papers implies that the authors think of the model as a sort of morphological decomposition (a more linguistically relevant tokenization than just BPE), but I don't understand why that should improve structural generalization in a dataset like SLOG. There is no error analysis or comparative analysis of models.
-
No attempt is made to compare the model to earlier models (there are empirical experimental comparisons but no discussion of the conceptual differences). Some earlier models are cited but it's not clear how they're related to the current model so it is hard to evaluate novelty. An obvious simple baseline here would be to apply a morphological decomposition and provide the output as the tokens, instead of learning a tokenization algorithm.
Questions for authors
None.