ResiMamba: Budgeted Recurrent State Residency for Efficient MRI Reconstruction

Long-range spatial modeling is essential for accelerated MRI reconstruction, yet its recurrent computation is commonly assigned too coarsely: every spatial token in a state-space scan is executed as a recurrent step. We observe that this is largely unnecessary—sparse recurrent access recovers most of the PSNR gain provided by dense recurrence: one resident token per local window recovers about half of this gain and a 6.25% effective budget over 80%, with strongly diminishing returns beyond. Building on this, we separate a stream’s eligibility to carry recurrent state from the token-level residency that decides which of its positions are executed as scan steps, and propose ResiMamba, a learned, budgeted mechanism that grants direct recurrent access to only a small fraction of tokens. A window-constrained allocator selects resident tokens under a fixed local budget; the remaining tokens are not discarded but routed through a dense evidence pathway that both condi- tions the shortened recurrent scan and contributes to the reconstruction. Across five MRI datasets, highly constrained residency retains quality close to dense re- currence: on fastMRI 4×, the 5% and 20% budgets reach 2.62× and 2.14× end- to-end speedups at only 0.10 and 0.02 dB PSNR cost. ResiMamba turns recurrent state access into a quality–latency design axis for efficient MRI reconstruction, with each budget trained as a separate operating point.