acceptodds
Under review as a conference paper at ICLR 2027

SUPPORT GEOMETRY BOUNDS DELAYED COPYING IN COMPACT SEQUENCE MIXERS

Abstract

A sequence mixer can have a receptive field longer than a copying task's required delay and still have no access to the token it must recover. We identify this failure through the reachable lag set , obtained by Minkowski addition of the architectural taps in a residual stack. For finite-bandwidth linear time-invariant mixers, the network's Jacobian support is contained in . On delayed copying with independent payload tokens, a required lag outside therefore limits expected accuracy to the optimal payload-marginal baseline, regardless of the learned weights. We test this exclusion while holding support diameter fixed at under a shared, parameter-matched backbone. Schedules that miss the required lag 54 remain at chance; schedules that cover it exceed 99.9% copying accuracy. Replacing one sparse layer with dense taps restores copying without increasing diameter, whereas coprime dilation rates help only when their finite tap sets cover the required lag. These results separate architectural access from soft influence energy and establish a training-free failure screen for compact mixers on tasks with explicit lag requirements.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.