Beyond Fixed Emotion Mappings: Learning Probabilistic Relations Across Emotion Granularities

Emotion recognition is naturally hierarchical: fine-grained affective states such as grief or annoyance relate to broader categories such as sadness or anger. Existing emotion taxonomies specify such categorical relationships, but knowing the hierarchy does not uniquely determine how probability mass in a fine-grained predictive posterior should propagate into a coarse-grained distribution. We therefore formulate cross-granularity emotion prediction as learning a probabilistic relationship between emotion granularities while retaining known hierarchical structure. Our hierarchical consistency framework jointly optimizes distinct fine- and coarse-grained predictors together with a learnable column-stochastic transformation $M_{\theta }$. The transformation maps the complete fine-grained posterior to an induced coarse posterior, which is coupled with a directly predicted coarse posterior through distribution-level consistency. Structural regularization anchors $M_{\theta }$ to a soft empirical mapping $M_{\mathrm{emp}}$ derived exclusively from the multi-label training data. Experiments on GoEmotions show that deterministic, empirical probabilistic, and learned probabilistic mappings yield nearly identical coarse-grained top-1 accuracy ($78.26$–$78.36\%$) while inducing meaningfully different predictive distributions. Relative to the training-derived empirical mapping, a learned transformation achieves a $25.7\%$ relative reduction in expected calibration error, while paired-bootstrap analysis supports reductions in negative log-likelihood for learned mappings. Importantly, the learned mappings preserve the dominant Ekman assignment of every fine-grained emotion. These results demonstrate that a known categorical hierarchy does not uniquely determine the probabilistic relationship across label granularities: mappings with nearly identical categorical behavior can exhibit systematically different predictive distributions.