Early Warning of Personalization Model Degradation from User Histories and Hidden States
Abstract
Personalization encoders continually update hidden representations of evolving user interactions to guide recommendations or user-specific generation. Future quality can degrade even while recent output quality appears normal, limiting timely intervention. We ask whether user interaction history and the corresponding encoder states form a joint record that exposes warning evidence of such degradation earlier. We call this relation *Temporal State-History Matching* (TSHM). We establish conditions under which a future-quality-relevant change remains visible in the observed state component of this record while recent utility responds only weakly. We then propose `CoTRAC`, a monitoring framework in which a pluggable sentry learns a broad History-State warning and refines it through Regularity, which measures local organization of projected state motion, and Transition, which compares the latest update with a causal prediction. We evaluate dataset-specific subsets of eight sentry designs on MovieLens-1M, PENS, and Amazon Product Reviews, monitoring 13 frozen personalization host instances. Against matched recent-utility controls, History-State warning improves mean normalized area under the precision-recall curve (nAUPRC) by 0.11 on MovieLens and is higher in all 160 encoder-sentry-horizon settings. The same direction occurs in 67/80 PENS and 21/24 Amazon settings. Regularity and Transition affect event detection, warning lead, and false alarms differently, and their joint use does not systematically dominate the individual views. Source-only warnings retain 86.4% of self-warning discrimination on MovieLens and 83.6% on PENS when applied to held-out encoders. These results establish History-State evidence as a prospective warning source beyond recent utility while showing that finer structured effects are more context-dependent.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.