acceptodds
Under review as a conference paper at ICLR 2027

Beyond End-to-End Black Box Mapping: An Intentional Agent Framework for Cognitive-driven Facial Reaction Generation

Abstract

Automatic human-like facial reaction generation (FRG) is essential for building intelligent systems that can authentically and deeply engage in human-computer interaction (HCI). While diverse and context-appropriate facial reactions can reflect latent appraisal and affective processes in human interaction, most existing FRG methods rely on end-to-end architectures that directly map speaker behaviours to listener expressions without an explicit intermediate internal state. In this paper, we reformulate FRG as generation mediated by a structured internal-state process and propose the Intentional Agent, which shifts FRG from direct stimulus-response mapping to stimulus-grounded generation through explicit intermediate states. To represent temporal internal-state evolution for FRG, we propose an internal dynamics model that integrates emotional drives with an iterative Inner Thought Flow (ITF) within a structured intermediate state used for subsequent generation. This state can continue to update during conversational silences. Furthermore, to bridge abstract internal states with physiological actions, we explicitly formulate FRG as a downstream affective mapping from this latent thought flow to physical facial expressions. Experiments on the REACT 2025 dataset show an FRDist of 72.39 and an FRDiv of 0.5057; perceptual plausibility is evaluated separately through blinded human ratings. A blinded human evaluation of 96 reactions found no significant difference in mean score between Full and ground truth ( vs. , ), while Full significantly outperformed Event-Triggered and Heuristic-Only (both ). The Reaction Quality Scorer (RQS) correlated strongly with human judgements (Pearson ; Spearman , both ), supporting its use as a supplementary automatic metric. These results underscore the immense potential of endogenous dynamics in building highly autonomous, human-like agents.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.