Neural-Symbolic Representation Learning by Structural Abstraction from Noisy Demonstrations

How can neural perception be interfaced with symbolic reasoning when primitive symbols and their structure are not given in advance? Palaeographers infer how oracle-bone characters are composed by analysing their shapes and possible strokes, which are noisy demonstrations composed by a sequence of raw perceptual observations without any symbolic labels or predefined structures. In this case, the learner must abstract both the primitive symbols and infer how they compose, unlike traditional neural-symbolic (NeSy) learning tasks which usually assume a manually specified symbols of primitives and relations as the interface. We study this representation learning problem and introduce structural abstraction, an algorithm for learning the interface from such demonstrations. On few-shot handwritten oracle-bone characters, it recovers expert-annotated radical substructures with 72.28% macro-averaged recall, compared with 9.58–13.09% for feature-based motif-discovery baselines. The learned representations extrapolate to unseen ancient and modern Chinese characters and discovers novel radicals at the same time. Our results show that perceptually grounded symbols and their compositional structure can be learned from demonstrations.