CONTINUUMFORMER: PERSISTENT SOFT STRUCTURAL MEMBERSHIP FOR INCREMENTAL SEQUENCE AND DEPTH COMPUTATION

Transformers repeatedly construct input‑dependent relationships through attention, routing, and layer aggregation, yet these structural associations are usually treated as transient. We introduce ContinuumFormer, a Transformer architecture built around persistent soft structural membership: token‑to‑sequence‑structure and layer‑to‑depth‑structure associations are maintained as a persistent state through sparse, overlapping assignments. The state induces a weighted structural topology that parameterizes sparse structural reconstruction, and is updated through candidate retrieval, change detection, hysteresis, local correction, and an explicit update budget. We formalize this mechanism as a state‑transition system and derive conditional properties for topology construction, membership‑to‑topology stability, fixed‑state local sensitivity, and budgeted structural maintenance. We evaluate the architecture on a 150M‑parameter language model using matched variants, controlled ablations, and long‑context validation. The experiments show improved language‑modeling performance over the matched Vanilla baseline across the evaluated standard context lengths, while axis‑wise ablations reveal asymmetric behavior between sequence‑ and depth‑side organization. An additional extended‑context evaluation further probes robustness of the full persistent‑state configuration beyond the standard context range. The study is designed as controlled architectural validation rather than compute‑saturated language‑model pretraining. Overall, the results provide controlled empirical evidence that persistent structural membership can serve as an additional organizational abstraction for Transformer computation.