Towards Temporal Interest Modeling in Recommendation via Reinforcement Learning

Reinforcement Learning (RL) methods are promising for recommendation systems due to their ability to optimize long-term user-system interactions. However, current RL recommendation methods treat user interactions as homogeneous decision rounds, ignoring irregular temporal gaps and the resulting interest decay over time. To incorporate temporal interest signals into RL, we study the relationship between temporal gaps and user interest through mutual information, finding power-law decay at the population-level and exponential decay at the individual level. Guided by the empirical observations, we propose a structured decision-process model: Temporal Interest Semi-MDP (TIS-MDP), which incorporates temporal gaps via a survival function to make RL methods temporal-aware. The effectiveness of TIS-MDP is verified across experiments.