CQFA: Channel-Wise Query Feedback for Gated DeltaNet-2

Linear attention offers an efficient alternative to softmax attention by enabling sequence modeling with linear-time computation and constant-size recurrent states. Within this framework, compressing sequence history into fixed-capacity memory makes selective state correction essential for retaining useful context while continuously incorporating new information. GDN-2 supports flexible memory updates by decoupling erasure and writing, but its correction remains key-driven and does not explicitly incorporate the query used for memory readout. Q-Delta incorporates query feedback into state correction, yet its per-head scalar coefficient applies the same feedback strength across channels, limiting channel-specific control over state correction. We therefore propose CQFA, which augments GDN-2 with input-dependent channel gates to selectively modulate query-based memory predictions on the erase side, independently of the writing of new values. This design allows feedback to be strengthened in some channels and suppressed in others while preserving the state size and rank-one update structure. We establish a sufficient condition for one-step residual contraction and implement efficient chunkwise parallel training with throughput comparable to that of GDN-2. Systematic analyses and experiments demonstrate that CQFA trains stably and achieves substantial gains over strong baselines on language modeling and information retrieval tasks.