Rotary-state Gated DeltaNet: Enhancing State Tracking through Rotation

We introduce **Ro**tary-**s**tat**e** **G**ated **D**elta**N**et (Rose-GDN), which augments Gated DeltaNet with input-dependent block-diagonal rotations. This adds rotational dynamics to the generalized-Householder transition without requiring multiple delta updates per token, while preserving fixed-size memory, linear-time recurrence, and exact chunkwise-parallel computation. We show that, under exact arithmetic, a one-layer Rose-GDN can track any finite abelian group. Increasing depth extends this result to all finite solvable groups, and a finite-depth Rose-GDN recognizes all regular languages. Empirically, Rose-GDN shows gains on state-tracking and long-context retrieval tasks while remaining competitive with strong recurrent baselines on language modeling and commonsense reasoning benchmarks. Together, these results suggest that combining input-dependent rotations with delta-rule memory provides a practical way to broaden the capabilities of efficient recurrent models.