Kalpana: Resonant Interference Memory for O(1) Long-Context Sequence Modeling via Phase-Coded Holographic Dynamics
Desk rejected at ICLR 2027, so left out of the paper lists and the map.
Abstract
The fundamental bottleneck in scaling autoregressive Transformer sequence models is the memory wall imposed by the Key-Value (KV) cache, which scales linearly O(N) with sequence length. In this work, we present Kalpana, a constant-memory neural architecture centered on the Resonant Interference Field (RIF)—a bounded memory substrate that encodes sequence context through continuous Fourier phase-coded superposition in complex Hilbert space. Unlike token-indexed architectures that allocate linear physical memory registers, Kalpana maintains a strictly fixed-size state S in C^B x D, enabling exact O(1) memory scaling invariant to context depth. We formulate and evaluate Kalpana across two complementary paradigms: (1) Paradigm A (Native Attention Replacement), which replaces in-layer KV-cache buffers in RoPE-equipped models with continuous Fourier phase accumulators, flatlining cross-layer persistent memory at 48.00 MB (FP16) across sequences exceeding 3,000,000 tokens while preserving 100% multi-query associative recall (7.2 ms query latency); and (2) Paradigm B (Decoupled Memory Substrate), which equips unmodified pre-trained LLMs with an external holographic memory layer for long-document reasoning up to 100K+ tokens within a constant 6 MiB state. We derive the Spectral Packing Law, proving that retrieval capacity is bounded by spectral phase crowding (Gamma ≈ 12.21) rather than memory exhaustion, and prove that unitary phase conservation avoids the dissipative decay (rho(A) < 1) observed in State Space Models. Hardware profiling demonstrates 892,000 tokens/sec ingestion throughput, a 300x increase in concurrent GPU serving density, and local multi-million token execution via WebGPU on consumer edge devices.
- Primary area:
- foundation or frontier models, including LLMs