acceptodds
Under review as a conference paper at ICLR 2027

LongEmo: Towards Emotion Understanding and Reasoning in Long Videos

Abstract

While recent Multimodal Large Language Models (MLLMs) have shown promise in affective computing, their reasoning capabilities are largely confined to short video clips with limited interactions. However, real-world emotions are not merely isolated instantaneous reactions but dynamic and cumulative processes deeply shaped by past experiences and ongoing events. To bridge this gap, we introduce LongEmoBench, a benchmark dedicated to emotion understanding and reasoning in long videos. It assesses progressive capabilities scaling from continuous scene interactions to complex episodic developments. Furthermore, we propose LongEmo, a novel memory-augmented agentic framework designed to tackle the immense challenges of long-range affective reasoning. LongEmo processes continuous video streams to construct an Event Memory Graph, explicitly modeling long-range dependencies and capturing emotional dynamics across discrete events. Given a question, the agent retrieves a query-relevant event stream from the graph, iteratively integrating multimodal memories and relational dependencies to deduce the final answer. Extensive evaluations of 17 representative methods reveal that they struggle significantly with emotion understanding and reasoning in long videos. In contrast, LongEmo achieves state-of-the-art performance, demonstrating the efficacy of its event-centric memory architecture.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Long Video Understanding, Affective Computing, Emotion Understanding and Reasoning, LLM, Benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_dfznOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LongEmoBench, a benchmark for emotion understanding and reasoning in long videos, along with LongEmo, a method for this task. Existing emotion benchmarks are limited to short clips, while general long-video benchmarks do not focus on emotions. LongEmoBench and LongEmo fill this research gap. Experiments demonstrate the difficulty of LongEmoBench and the efficacy of LongEmo.

Critical strengths

The proposed LongEmoBench could be valuable to the research community. Existing emotion benchmarks are limited to short clips, while general long-video benchmarks do not focus on emotions. LongEmoBench fills this gap.

Critical weaknesses
  1. The main concern is that the evaluation is not sufficiently rigorous. (1) The proposed LongEmo is not evaluated on the G1 tasks in Table 2. (2) The comparison in Table 2 may not be fair, so the conclusion described in L402 - L421 may not be meaningful. For example, the Affective MLLMs use earlier MLLM backbones, so it is natural that their performance is lower. It is important to take backbone models into consideration in a comparison. (3) Similarly, in Table 3, the proposed LongEmo uses the most advanced GPT-6 as the backbone. Hence, its comparison to the Long-Video Architectures is unfair. (4) The evaluation of LongEmo considers only one model configuration described in Appendix E. The generality of LongEmo to different backbone models is not demonstrated.

  2. Several critical details of the LongEmoBench curation are missing. (1) LongEmoBench is annotated by humans. How many annotators are involved? How are they selected or recruited? What is the correlation between the annotations from different human annotators? These questions are important to judge the reliability of the proposed benchmark. (2) The authors should explain more about the video source of this benchmark. Are there any copyright or terms-of-use issues? This is important for the ethics review.

Although the proposed benchmark could be valuable, the above two major concerns make me rate it a clear rejection at this stage.

Questions for authors

See weaknesses.

(Minor) The font sizes in the figures are too small.

Reviewer_C73tOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

LongEmo introduces LongEmoBench, a benchmark for long-video emotion understanding and reasoning, containing 1,975 QA pairs across 1,354 videos totaling 69.8 hours. It evaluates five scene-level tasks and three episode-level tasks. It also proposes LongEmo, a memory-augmented agentic framework that constructs an Event Memory Graph, retrieves relevant event streams, and selectively inspects audiovisual evidence for long-range reasoning. Ablation results support the contributions of event-centric memory and relational retrieval.

Critical strengths
  1. LongEmoBench explicitly evaluates emotional transitions, trajectories, causes, and long-range reasoning rather than focusing only on isolated emotional expressions. Its two-level task taxonomy and substantial video collection provide a useful resource for evaluating long-context multimodal systems.
  2. The Event Memory Graph combines character continuity, historical dependencies, and localized audiovisual evidence. Removing event memory reduces the overall score from 60.24% to 51.64%, while removing event-stream retrieval reduces it to 57.62%. These ablations provide evidence that both components contribute to performance.
Critical weaknesses
  1. Appendix C reports nominal Krippendorff’s α values of only 0.133 for G1 and 0.118 for G2, despite observed reviewer agreement rates of 84.86% and 71.25%, respectively. Moreover, the reported human–LLM judge correlation of r=0.872r=0.872 is not accompanied by sufficient information about the validation sample size or task-specific reliability.

  2. Limited evidence of generalizability and robustness to source familiarity. The dataset's source distribution is heavily skewed toward sitcoms, which account for approximately 65.96% in Figure 2. The appendix also includes questions drawn from recognizable series such as Friends. Without explicit contamination analysis or controlled evaluations against transcript-only and unseen-source settings, it remains unclear how much performance reflects genuine audiovisual emotion reasoning rather than familiarity with known narratives or dialogue.

Questions for authors
  1. How were the reported Krippendorff’s α values calculated, and why are they so low despite the reported raw agreement rates? Please provide agreement statistics for the actual reference annotations after adjudication, separately by task type?
  2. What was the sample size and sampling strategy for the human–LLM judge comparison? Can you report task-specific agreement metrics and uncertainty estimates, particularly for emotional trajectory and causal reasoning?
  3. How do LongEmo and the baselines perform on unseen series or source domains, and under transcript-only versus audiovisual evaluation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.