EmoForecaster: Advancing Multimodal Emotion Forecasting via State-Transition Modeling and Evidence-Grounded Reasoning

Emotion is fundamental to human experience and social interaction, making the anticipation of future emotional responses important for proactive interactive systems. However, existing multimodal emotion understanding methods largely focus on emotions that have already been expressed. Although several preliminary studies have explored emotion forecasting, they primarily predict emotion categories, providing limited explicit reasoning about how multimodal evidence and interaction dynamics support their predictions. To address these challenges, we propose EmoForecaster, an evidence-grounded multimodal emotion forecasting framework that integrates state-transition modeling with progressive multimodal reasoning. We first construct EmoEvolve with structured interaction reasoning annotations comprising five components that jointly capture conversational context, multimodal affective evidence, and interaction-aware emotional transitions, and establish a clip-disjoint evaluation protocol covering both emotion forecasting and rationale generation. Furthermore, we develop a complementary architecture in which the state-transition path uses historical reference and preceding interlocutor states to construct context-conditioned forecasts, while the evidence-reasoning path progressively interprets multimodal cues to explain how forthcoming emotional responses may develop, with the two paths coupled at the decision level and optimized through supervised fine-tuning followed by path-specific Group Relative Policy Optimization (GRPO). Extensive experiments demonstrate the superiority of EmoForecaster in multimodal emotion forecasting, with strong performance in both predictive accuracy and rationale quality.