- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper explores structural abstraction as a loss level inductive bias for transferability of learned knowledge. The study is motivated by behavioural and theoretical works from cognitive science. The authors introduce Abstraction-Augmented Training and two benchmark, Relational Cycle Benchmark and Narrative Abstraction Benchmark.
Critical strengths
- The paper is very well written, and the motivation is strong and clear from the cognitive science literature, connecting behavioural and theoretical work on abstraction as a key property for humans in regards to transferring knowledge across contexts. The authors propose a simple loss-level inductive bias, Abstraction-Augmented Training (AAT) in line with this, by optimising over both the concrete instance and its structural abstraction.
- The authors contribute two complimentary benchmarks, which are useful for the investigation into the benefits of AAT.
Critical weaknesses
- Modest Empirical Benefits: (Table 3) AAT achieves the highest unknown accuracy across all three models, but its benefit over alternative methods is modest. Overall cumulative accuracy is similar in comparison to alternative methods, and the benefits in forgetting are mixed. Furthermore, figure 2 shows some benefits in the learning dynamics in comparison to standard SFT, it would also be useful to compare against the other methods in table 3 to establish wether AAT offers similar benefits.
- Benefit of Abstraction: Table 2 compares structured abstractions with random abstractions. It is unclear whether the structured abstraction adds more benefit then simply presenting the same facts in different words.
Questions for authors
- Advantages Over Alternative Methods (W1): Given the emricial improvements are modest, what are the practical benefits of the methods in comparison to the methods comparing against (ie. EWC, A-GEM etc.). A comparison of computational cost and other relevant requirements would help clarify when AAT is suitable to use.
- Paraphrase Control (W2): Would replacing the abstract example with a paraphrase that keeps the original names and facts give similar results with the same training budget?
I agree with the authors that the complexity of the algorithm should not be the judge of whether it is a good contribution or not and does not have to do with novelty. Given the moderate improvements that AAT brings (EWC slightly beats AAT on overall cumulative accuracy and on Qwen 3B there is no method significantly better than SFT) that AAT brings to the table, it is yet unclear whether this generalizes to real datasets (for example real continual learning setups well-studied in LLM literature [1]) beyond the two benchmarks that the authors propose. The authors explain that one could potentially use pre-trained entity detectors, but I recommend that the authors actually showcase that this is feasible and useful on more realistic datasets. This could really strengthen the paper.
[1] Shenfeld, Idan, et al. "Self-distillation enables continual learning." arXiv preprint arXiv:2601.19897 (2026).