- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a method/framework to measure how far the effect of a small change inside a model spreads and compares this against a task-defined notion of "nearby" (maze neighbors, Sudoku boxes, ARC objects, 3D object distance). This is then applied to two recursive reasoning models, namely Hierarchical Reasoning Model (HRM; Wang et al. (2025)) and the Tiny Recursive Model (TRM; Jolicoeur-Martineau (2025)) and finds that changes to the low-level state stay more concentrated in the high-level update than in the next low-level step. The paper reports further depth profiles against a non-recursive baseline, a 3D grounding model (MTU3D), and a fine-tuning loss that increases the score.
Critical strengths
I have to say that I really had hard times reading and comprehending the paper, these LLM-polished papers use a language that is very dense and is somehow difficult to understand. But from what I understood is that the strengths of the paper are:
-
The paper seems to make a correct and useful point, which is that a "fraction of effect nearby" score always includes the perturbed site itself, so it can look local just because the effect stays where it started. Separating this out changes the conclusion in two of six main cases.
-
The authors are honest about their own weak spots, reporting calibration failures, off-distribution perturbations, and cases where results reverse.
Critical weaknesses
- It seems that the main finding depends on how strongly the model is perturbed. Apparently, it holds with very large noise but often reverses with milder noise.
In essence, I am a bit unsure about the paper but I am leaning rather negative and would argue for reject because the paper’s main finding only holds when the model is hit with unrealistically large noise, and it largely disappears or reverses under milder or more realistic perturbations.
Questions for authors
- What I did not understand: Why do Table 1 and Table 2 report different values for the same quantity (e.g., TRM/Maze: +0.230 in Table 1 vs. .143 in Table 2)?