ICR: Scaling In-Context Learning for Sequential Recommendation

Sequential recommendation in new domains typically requires extensive per-dataset training data and compute. We present ICR, a zero-shot sequential recommender in the Prior-Fitted Network framework that adapts to new domains at inference time through in-context learning. Given a set of support sequences from the target, ICR produces next-item predictions in a single forward pass without requiring dataset-specific fine-tuning. Our design is motivated by three observations: (1)sequential recommendation dynamics are well-characterized by a hierarchy of aggregation functions, enabling a synthetic prior with provable full support over finite item sequences; (2)dependence on absolute embedding coordinates is not justified for a foundational ICL model—we prove this formally—motivating an ID-based architecture with a pointer mechanism that selects items directly from context; and (3)identity-restricted cross-attention scales sublinearly with context size, enabling performance to improve with more context rather than degrade. On the majority of 11 benchmark datasets, ICR outperforms every zero-shot baseline and, more remarkably, matches or exceeds standard supervised baselines—to the best of our knowledge the first zero-shot recommender to do so, allowing for lower end-to-end cost than supervised training for realistic user populations.