POISE: Post-training Input-aware State Elimination for Deep State Space Models

State space models (SSMs) trade long-context computation for a recurrent state, making state dimension a direct cost of autoregressive inference. Existing post-training reduction methods either prune modes in the trained coordinates or apply classical balanced truncation, whose controllability Gramian reflects generic white-noise excitation rather than the inputs encountered at deployment. We introduce `POISE`, a label-free, post-training method that instead balances each SSM against the state covariance induced by a small set of calibration inputs and allocates a global state budget across the network. For stable linear time-invariant (LTI) SSMs, `POISE` yields stable reduced systems and provides error guarantees for arbitrary inputs and finite horizons. We apply it to the LTI backbones S4D, SaShiMi and S5, and extend the method empirically to selective Mamba-2. On Mamba-2 $1.3$B, `POISE` removes $70\%$ of the recurrent states for only $0.34$ additional WikiText perplexity. At a $2048$-token context, the reduced model comes within $0.6$ perplexity of Llama-$3.2$-$1$B with a four-bit KV cache while using less memory.