Learning Predictive Event States for Sequential Modeling and Recommendation

Next-token prediction (NTP) trains causal sequence models for sequential recommendation, but its next-interaction target does not specify which event-level information should be accessible from each causal state. We introduce Event State Prediction (ESP), which augments NTP with two predictive relations over contiguous interaction groups. Intra-event prediction trains token states formed from observed prefixes to predict contextual summaries of their enclosing events; inter-event prediction trains pooled states at event completion to predict the next event before its interactions are observed. Both relations use history-conditioned targets from an exponential-moving-average encoder and single-positive contrastive matching, supervising individual states and event-level aggregates. ESP improves NDCG@10 and NDCG@20 across four datasets while retaining the original NTP inference path. Frozen-state probes on three datasets indicate greater accessibility of future and event-level information on the tested tasks. Geometry analyses find higher effective rank and broader state dispersion, and ablations support the complementary contributions of the two objectives. The code is available at https://anonymous.4open.science/r/ESP-Code/.