Predictive State Has a Scope
Abstract
A temporal model can recognize a relation without remembering its history. We study predictive state scope: where different histories are written and which states future predictions can read. Shared state pools observations but can merge histories with distinct futures, creating predictive aliasing; local states retain separate histories but receive fewer observations. We introduce Factor-Addressed Predictive State (FA-State), a jointly trained neural architecture coupling a shared prediction path with pair-addressed recurrent states. FA-State shares how history is encoded and updated across addresses while preserving where relation-specific history is written and retrieved. Bayes log-risk and random-effects analyses characterize the cost of predictive aliasing and the finite-sample trade-off between shared and local states. Across temporal graph forecasting tasks, FA-State improves prediction with multiple backbones. Holding trained weights and online bank contents fixed, deranging only the retrieval addresses systematically degrades prediction, showing that trained predictors use the correspondence between historical content and its address. Controlled streams and recurrence analyses further reveal how available history shapes the shared–local trade-off. These findings establish state scope as a design choice in temporal representation learning itself: what a model remembers depends on where it writes history and how it reads it back.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.