acceptodds
Under review as a conference paper at ICLR 2027

Beyond Firing Less: Activity Objectives Shape Population Organization in a Spike-Driven Transformer

Abstract

An activity objective specifies what training rewards. The statistic measured after training can move opposite to the objective's nominal direction, because training can reorganize which responses remain active. We study this mismatch in a Spike-Driven Transformer using per-input population firing-rate coefficient-of-variation regularization (PopFR-CV), together with mean-rate, lifetime-sparsity, and entropy-style objectives. PopFR-CV explicitly rewards greater firing-rate variation across activation coordinates for each input. Training produces the reverse measured response: evaluation CV falls by more than an order of magnitude, the equal-tensor silent-coordinate fraction increases by over 30 percentage points, mean firing decreases only modestly, and endpoint accuracy remains comparable. Controlled continuations from identical mature states show immediate trajectory separation followed by accuracy recovery. Paired CIFAR-100 endpoints localize the reorganization to strong activity withdrawal from monitored attention pathways; fully retained non-attention responses show little change in mean CV. The low-CV, enlarged-silence endpoint recurs across three static-image datasets within the studied Transformer. Other spiking systems and event data show distinct responses. These findings show that population participation is a distinct outcome of activity regularization that is not reliably predicted by mean firing or by the nominal direction of the optimized statistic.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.