Train Short, Reason Long: Compositional Generalization via Frozen Holographic Memory

Trained sequence models compose known facts at the depths they trained on but degrade beyond them. We trace the failure to one architectural choice: trained weights both decide which facts to compose and execute the composition, so an approximation in one corrupts the other. Systems that keep execution symbolic avoid the choice, but where their rules compose along one path over a fixed set of facts, a fact off the path is never read and a fact that changes is never replaced. We present Frozen Holographic Memory (FHM), an architecture in which trained weights decide what enters a composition and a fixed algebra executes it. One algebra holds facts, replaces them exactly, composes them, and induces rules from them. Trained on possession chains up to two facts deep, FHM tracks state exactly on chains 40 facts deep. Trained on CLUTRR stories of two or three facts, it composes ten-fact stories at 100.0% on the net generation count and 98.3% on the full kinship label. On NoRA, FHM induces its own rules and constraints and scores above the Edge Transformer on every split scored, 52.8% against 6.0% on the split whose answers need the most facts off the path. We intend FHM as an architectural primitive for any system that requires state tracking, composition at unseen depths, rule induction, or off-path reasoning.