COSI-Lab: Conference Living Lab for Modeling Multi-Perspective Multimodal Social Intention

COSI-Lab presents a multimodal multi-sensor dataset of a real interdisciplinary scientific workshop with 32 academics at an international conference. It captures ecologically valid social interactions in a weakly scripted setting consisting of two 30-minute mingling sessions with real professional and social consequences. We argue that intelligent systems could better handle subjective perceptions by modeling their multiplicity not as label noise but as an explainable perspective-driven reasoning process. We focus on the Apparent Intent Inference (AII) problem as determined by ex-situ observers and conceptualize intentions to be independent of the future. We contribute a novel annotation process for AII that accounts for a perceiver's own interpretative tendencies, quantitative and qualitative analyses of intent narratives with respect to diversity, plausibility and grounding, benchmark tasks for AII and relevant contextual factors, speech quality audio for all participants as well as privacy-preserving multi-modal environmental and wearable sensor data, enabling lexical and nonverbal behavior analysis; and the coupling of self-reported participant goals (30 minute to 3 hour) with annotated AII (seconds).