acceptodds
Under review as a conference paper at ICLR 2027

STATIC-SOURCE RECURRENT ATTENTION: TIME, MEMORY, AND PRECISION

Abstract

We study recurrent attention with immutable, position-locally encoded inputs and finite-precision queries. The model captures a class of iterative latent readers, including local-input configurations of Perceiver with a fixed source-kernel library. We prove two communication bounds: a cumulative query bound and a finite response-table bound whose size is independent of recurrent time and mutable memory. For set disjointness on two -bit blocks, these imply and for any fixed worst-case error . Thus additional recurrent computation cannot compensate for sublogarithmic source precision. A matching construction achieves and . Complementing the lower bounds, we construct a fixed rational-weight recurrent-attention program that simulates a -step, -space Turing machine in rounds and mutable bits using precision. A floating-point repair operation restores approximate softmax reads exactly before each state update, preventing error accumulation over time. An end-to-end finite-precision implementation matches independent Turing-machine traces for up to 10,010 transitions.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.