- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces a formal, controllable framework for studying knowledge transfer across small discrete worlds defined by forbidden patterns on a grid, instantiated as Sudoku-like completion problems that decompose rules into reusable components. A single depth-recurrent model is trained across many rule combinations and evaluated under four graded novelty regimes. Key findings are that description- and example-based conditioning both excel at recombining trained rules, but when rule components have never co-occurred in training, symbolic descriptions can underperform even the no-information baseline, an effect that is mitigated by increasing co-occurrence exposure or description dropout.
Critical strengths
- The design allows principled tests of compositional generalization, including supported and unsupported new instances.
- This paper proposes a strong formal grounding with an intuitive explanation.
- The paper is well-written and clear for readers to follow.
Critical weaknesses
- The experiments conducted in this paper focus on a single model family and a single grid size (6×6). It is unclear whether the scalability and architectural generality extend to larger Sudoku sizes or non-grid problems.
- The exploration of the example count (M=2) is limited. It is unclear that how many examples are actually needed for identifiability across different worlds is not systematically studied.
- There is a lack of comparison to alternative CSP solvers or LLMs with tool-use methods, which could contextualize the observed effects.
Questions for authors
See Weaknesses.
Followup Question
Thank you for the efforts made by the authors during the rebuttal. However, the applicability of the proposed framework beyond grid-based problems remains unexamined. I understand the deadline of discussion is coming, could the authors provide further discussion on how the framework could be extended to non-grid domains, whether the observed knowledge transfer patterns are expected to generalize to such settings?