- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates the intermediate-state interface between visual operations using closed grid-to-grid programs. The authors construct two parameter-matched executors: latent handoff, which directly passes hidden states between operations, and input-aligned grid-state re-entry, which re-encodes each predicted intermediate grid through the shared color and positional input interface. Grid-state re-entry achieves approximately 44 percentage points of improvement in exact-grid accuracy on held-out operation pairs and longer programs. Experiments involving frozen-model rollout, position-wise probes, boundary-component ablations, MMD controls, transfer to grounded-object operations, and six-step execution chains collectively support the explanation that intermediate states must be represented in a form directly consumable by subsequent operations.
Critical strengths
Rigorous isolation of the key experimental variable. The latent-handoff and grid-state executors share the same 12.9M parameters, primitive-wise computation, training data, final-output supervision, and update budget. All matched seeds consistently favor grid-state re-entry on the Pair and Depth splits, providing substantially stronger evidence than empirical improvements from a single model configuration.
Comprehensive mechanistic evidence. The experiments systematically examine intermediate supervision, spatial alignment, color-simplex representations, MMD-based distribution matching, intermediate-state probes, and frozen-model rollout. These controls help rule out alternative explanations based solely on increased depth, additional parameters, or marginal distribution matching, and provide evidence that the input-aligned execution boundary plays a central role in compositional generalization.
Critical weaknesses
Insufficient explanation of the underlying failure mechanism of latent handoff. Although the experiments convincingly demonstrate the effectiveness of input-aligned grid-state re-entry, they do not fully explain why latent handoff fails in the first place. In particular, given that the latent-handoff executor is trained end-to-end, why does it fail to learn stable and reusable hidden representations across successive operations? A more direct investigation of the underlying representational incompatibility would strengthen the mechanistic understanding of the proposed approach.
Questions for authors
Please refer to Weaknesses
Thanks for the authors' responses, but I do not agree with the response to the core concern about the predictability of this work. The response said: "Why the result is not simply predictable. The latent state already contains the result: the model's own head reads the intermediate grid from it with 98.2% accuracy, so information content alone predicts no gap." I want to clarify that the review never said that the latent state loss needed information to make the combinatorial process degrade. Rather, it is the input gap—meaning the distribution gap—that causes the degradation, because the model is trained on the grid input but not on the contrastive latent state before output head. It is quite predictable that performance with latent state input will degrade a lot, even though this state contains the needed information. Let me give a not-so-perfect but illustrative example: if a language model has just been trained on an English corpus, it cannot understand a Chinese or German sentence, although the information is contained in the sentence. Later, you give a further explanation of the performance degradation of latent state inputs with shortcut; I think this explanation does not conflict with the input domain or distribution gap term—it is just at a more specific level. So until now, I still think this core concern has not been addressed. If you have further clarification, please let me know. Thanks again for the detailed response.