acceptodds
Under review as a conference paper at ICLR 2027

When Will a Model's Knowledge Expire?

Abstract

A language model may answer a factual query correctly at deployment and later become wrong even though its parameters have not changed, simply because the world has. We study how long such currently correct knowledge can be expected to remain valid. We cast this as a survival problem in calendar time, with left truncation because many facts have already been valid for some time at the evaluation date, interval censoring when the change time is known only within a window, and right censoring when no change is observed. We introduce WIKILIFETIME, a rolling-origin benchmark containing only facts that are both valid in the world and recoverable from a point-in-time language model at the anchor date. We propose EXPIRE (EXpiration Prediction from Internal Representations and Event histories), which combines relation-specific duration patterns, entity-level histories, frozen hidden states, and calendar drift in a low-rank time-varying hazard model. The analysis separates model fitting, tuning, recalibration, retrieval-risk certification, and final testing chronologically. We derive estimation and calibration guarantees, characterize the limits of relation-recency rules, and obtain finite-sample control of missed stale answers for retrieval decisions. Numerical studies show that EXPIRE improves remaining-lifetime prediction, within-relation discrimination, and calibration, while reducing missed stale answers in retrieval decisions.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.