Toward Self-Referential Continual Learning via a CREB-Inspired Selective Consolidation Mechanism

Continuous parameter updates in streaming Large Language Models (LLMs) can exhibit representational “smearing,” where sequential learning across disjoint formal domains degrades previously learned reasoning structure. We introduce Self-Referential Continual Learning (SRCL), a dual-memory architecture that gates long-term structural consolidation through a discrete, CREB-inspired epistemic threshold. Instead of continuous uncertainty-weighted replay, SRCL processes streaming inputs through a transient Working Memory (WM) adapter and computes a consistency score over successive WM adapter updates, S(x_t) = E[cos(Δφ_t, Δφ_t-1)]; in experiments, this quantity is estimated online with an EMA-based cosine-similarity approximation. When this score exceeds a discrete threshold τ, the WM adapter update is projected into Long-Term Memory (LTM) via orthogonal rank expansion; otherwise, the transient WM weights decay. Principal Component Analysis of parameter trajectories over 50 sequential steps suggests that SRCL more often confines updates to distinct, approximately orthogonal subspaces, reducing the geometric overlap and high-variance interference observed under continuous Prioritized Experience Replay (PER). On the reported sequential math and code streams, SRCL preserves formal reasoning performance more effectively than continuous baselines, with cumulative interference plateauing at 0.244 after 50 steps while the baselines continue to degrade. While this discrete gating mechanism mitigates catastrophic forgetting in our deterministic reasoning tasks, performance degrades sharply under high injected aleatoric noise, suggesting that discrete epistemic thresholds are comparatively effective for the tested formal domain shifts but brittle under irreducible data stochasticity.