EQ-Agent: From Multimodal Emotion Recognition to Emotionally Intelligent Agents

Recent foundation models have predominantly emphasized “IQ-like” capabilities—reasoning, mathematics, coding—while emotional intelligence (EQ) remains comparatively underexplored in the pursuit of artificial general intelligence (AGI). Multimodal emotion recognition (MER) has advanced affect perception but stays perception-centric: recognizing an emotion alone does not let an agent understand context, plan support, or respond effectively. We propose EQ-Agent, a framework that extends affective AI from the MER paradigm to a complete EQ loop spanning perception, understanding, planning, and response. EQ-Agent first establishes EQ-MER, a sub-1B perception foundation built on a Perception-to-Cognition (P2C) Bridge architecture and a two-stage training recipe. We then build EQ-Dataset, a 30,664-record multitask resource integrating social reasoning, supportive response, and structured emotional-reasoning supervision to bridge emotion labels and emotionally informed decisions. Initialized from EQ-MER, EQ-Agentic couples multitask supervised fine-tuning with single-reward, EQ-Judge-guided agentic policy optimization to learn compact agentic UNDERSTAND–PLAN–RESPONSE decisions, while EQ-Harness supplies the agentic runtime and deployment support for the model. On nine-dataset MER-UniBench, EQ-MER reaches an 82.22% macro score with state-of-the-art parameter efficiency; on the agentic EQ-AppBench, EQ-Agentic scores 47.25 under a unified judging protocol applied identically to all baselines. Together, these components form a unified pathway from multimodal emotion recognition toward emotionally intelligent agents; we will release the full project upon acceptance.