PCAF: Bounded Exact-Successor Memory for Efficient Language Models

Efficient causal models compress long histories, which can erase token-successor associations. We study PCAF-Exact, a bounded per-key memory that retains recent observed successors and mixes their sparse vote with a causal model’s prediction. Capacity-matched experiments show that exact memory improves PG-19 perplexity across three model scales; at 40M/64K, PPL falls from 129.72 to 114.54. A parameter-free reciprocal-recency reader retains much of this benefit. Under equal record storage, continuous retrieval gives lower perplexity, while exact lookup reads about 2.3× faster; measured teacher-forced pipelines show a smaller speed advantage. A compact hybrid improves on a payload-matched continuous reader across 19 PG-19 test books at both evaluated scales. Same-target tests show gains through 128K context. Together, these results position bounded exact-successor memory as an efficient complement to contextual retrieval for repeated transitions.