- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a neuro-symbolic framework for length generalization. A Transformer predicts local computation results, pointer movements, and control actions based on the local neighborhoods of a fixed number of pointers and the current control state. An external symbolic program then executes these predictions and updates the state. Each task is trained separately using supervised algorithm execution traces, transforming the problem of solving long inputs into the repeated application of the same local policy. Experiments on ten synthetic tasks, including copying, parity checking, and arithmetic operations (addition, subtraction, multiplication, and division), report 100% accuracy on the tested longer inputs after training on shorter ones.
Critical strengths
None.
Critical weaknesses
- The motivation and practical value of this approach are unclear. If the goal is to solve these specific algorithmic tasks, a direct implementation already handles arbitrary input lengths, subject to computational resources. Once the algorithmic decomposition, state representation, pointer interface, and subroutine composition have been specified, why train a neural network to learn the remaining local rules instead of simply writing them in code? If the goal is to address length generalization in general-purpose models, training a separate policy for each task with task-specific algorithmic structure does not demonstrate progress toward that goal. The reported perfect accuracy on longer inputs therefore does not, by itself, establish the significance of the method. What does neural learning contribute beyond the algorithmic structure already supplied by the designer?
- The novelty over prior neural program execution methods is unclear. Neural Programmer-Interpreters already learned pointer-based actions, subprogram calls, and termination from execution traces, while Recursive NPI studied provable generalization to arbitrary input lengths. What fundamentally new capability or principle does the proposed local action predictor and symbolic executor introduce beyond these approaches? The paper should also clarify its contribution relative to structured scratchpad/CoT methods. Without these comparisons, the claim of being the first neuro-symbolic approach to length generalization is insufficiently supported.
Questions for authors
- If the algorithmic structure is already specified, why train a neural model to learn the remaining local rules instead of simply implementing them in code?
- What is fundamentally new about this approach compared with Neural Programmer-Interpreters, Recursive NPI, and structured scratchpad methods?