acceptodds
Published as a conference paper at NeurIPS 2026

Spectral Measures of Mamba Conductances Predict and Shape Effective Receptive Fields

Zy Li

Abstract

Mamba's selective state-space model (SSM) achieves long-range dependence through input-dependent gating, yet existing analyses offer limited insight into *why* a particular parameter configuration produces a particular effective receptive field (ERF). We define per-step *conductances* from the zero-order-hold discretization and construct a signed spectral measure whose atoms are the per-dimension conductances weighted by the corresponding input-output residues. The impulse response at lag is a discrete Laplace transform of this measure, and the ERF is governed by the per-channel squared impulse response weighted by output amplitude. Heavy tails arise from the mixture across state dimensions, not from temporal randomness within any single dimension. We extract conductances from trained Mamba models and report three empirical findings. First, replacing the signed measure with its total-variation envelope overestimates the ERF by more than three orders of magnitude. Signed cancellations among residues are what make the prediction quantitative. Second, the framework generalizes beyond the original single-layer validation setting. Applied to held-out -initializations, a two-layer model trained from scratch, and all layers of a pretrained Mamba-370M, the spectral measure tracks the frozen-parameter ERF within - on of pretrained layers, with log-log slopes matching to within on layers and within on all . Third, varying the initialization of causally shifts the ERF tail slope as the spectral prediction anticipates ( for slow init vs. for default), and this shift translates to task performance. Across two long-context tasks evaluated over three random seeds, the slow initialization is the most reliable condition: it solves MQAR in seeds (vs. for both default and fast), and attains the highest mean selective-copy accuracy on every distance bucket ( overall vs. for fast at distances up to tokens, with the ranking preserved in a single-seed extension out to tokens). The spectral measure thus provides a controllable design lever for long-range memory. The closed-form prediction reproduces the in-sample frozen-parameter ERF at ratio by construction, which we use as a pipeline consistency check.

Related papers

See more related submissions when the market opens.