acceptodds
Under review as a conference paper at ICLR 2027

Do the Same Hints Matter Across Input Sizes? Intervening on Semantic Intermediate States in Neural Algorithmic Reasoning

Abstract

Neural algorithmic reasoners often generalize poorly to inputs larger than those seen during training. Although CLRS models are supervised with intermediate algorithmic states (HINTs), it remains unclear whether these states retain the same functional role when input size increases. Unlike opaque latent activations, CLRS HINTs correspond to named algorithmic variables with exact ground-truth trajectories, allowing their role in recurrent execution to be tested directly through intervention. We study this question across 28 CLRS algorithms and two recurrent processors at N=16 and N=64. We separately test the effect of restoring an individual HINT to its ground-truth trajectory, disrupting its input-specific semantic value through valid semantic resampling, and changing the surrounding HINT context in which the same HINT acts. Both restoration and semantic dependence retain cross-size structure, yet at N=64 their HINT-level rankings are nearly unrelated: execution can remain sensitive to a HINT's semantic content even when correcting that HINT in isolation is not beneficial. Native HINT prediction error is insufficient to explain this dissociation. Nevertheless, the explicit HINT interface remains collectively consequential: restoring all HINT trajectories improves N=64 task accuracy by 15.96 percentage points. Surrounding-context interventions further show that the effect of correcting an individual HINT changes more between the two surrounding HINT states at the larger input size. These results indicate that size extrapolation does not simply eliminate dependence on semantic intermediate states; instead, semantic dependence and isolated repairability remain distinct, while individual correction effects vary more across surrounding HINT states at N=64.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.