How Expressive Are Spike-Driven Transformers? An Image-Capped Finite-Trace Analysis

abstract We study the expressive power of spike-driven Transformers beyond conventional linear-region counting. Once spiking dynamics map continuous inputs to finite states, downstream Transformer layers need not create new input regions to realize different functions; instead, expressivity depends on how these states can be distinguished and recombined. We formalize this effect through region-combination capacity, which counts the finite interaction traces assignable to LIF-induced input regions. Our analysis reveals two fundamental bottlenecks. First, shared branches cannot assign their global state alphabets independently across tokens, imposing a state-exposure constraint. Second, multiplicative interactions can collapse distinct operand pairs into the same score. Together, these effects yield a two-budget attention envelope $D_{\mathrm{real}}\le \min \{L_{QK},L_S\}$, which identifies whether branch sharing or score collisions limit the expressive gain of attention. Composing the surviving distinctions through values, gated FFNs, residual paths, and depth gives $\log {\mathrm{Comb}}^{(L)}(m)=O(msLD\log (T+1))$ for homogeneous count-decoded models, with a tagged construction approaching this exponent. Finite enumeration validates the predicted sharing cap, product distinguishability, phase transition, and envelope tightness, while architecture-level studies on SDT-V3 and Spikformer show how the framework can diagnose the active expressive bottleneck. These results provide a finite-trace theory for understanding when Transformer interactions can, and cannot, overcome the discrete-state bottlenecks of SNNs.