AffectAtlas: Structured Neuro-Symbolic Affect Modeling for Emotionally Consistent Talking-Face Generation

Audio-driven talking-face generation has made substantial progress in visual fidelity, identity preservation, lip synchronization. However, existing methods often treat emotion as a weak or isolated conditioning signal, leading to facial behaviour that is visually plausible but affectively inconsistent with speech. We propose AffectAtlas, a neuro-symbolic affect modelling framework for emotion-consistent talking-face generation. We model affect as structured system over 3 complementary representations: expressions, action units, valence-arousal. We first construct a probabilistic affect atlas that captures bidirectional relationships among these affect spaces. We then introduce a teacher-guided atlas adaptation mechanism to adapt the atlas to talking-face data while preserving its structured prior. Finally, we incorporate differentiable neuro-symbolic rules to enforce psychologically meaningful consistency among the 3 affect spaces, and inject the refined representations into a diffusion-based generator via an emotion-aware A/V module. Extensive quantitative, qualitative, cross-dataset, ablation and user-study evaluations show that AffectAtlas improves emotion fidelity, temporal affect alignment, visual quality, and overall perceptual realism compared to SOTA methods.