- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LongEmoBench, a benchmark for emotion understanding and reasoning in long videos, along with LongEmo, a method for this task. Existing emotion benchmarks are limited to short clips, while general long-video benchmarks do not focus on emotions. LongEmoBench and LongEmo fill this research gap. Experiments demonstrate the difficulty of LongEmoBench and the efficacy of LongEmo.
Critical strengths
The proposed LongEmoBench could be valuable to the research community. Existing emotion benchmarks are limited to short clips, while general long-video benchmarks do not focus on emotions. LongEmoBench fills this gap.
Critical weaknesses
-
The main concern is that the evaluation is not sufficiently rigorous. (1) The proposed LongEmo is not evaluated on the G1 tasks in Table 2. (2) The comparison in Table 2 may not be fair, so the conclusion described in L402 - L421 may not be meaningful. For example, the Affective MLLMs use earlier MLLM backbones, so it is natural that their performance is lower. It is important to take backbone models into consideration in a comparison. (3) Similarly, in Table 3, the proposed LongEmo uses the most advanced GPT-6 as the backbone. Hence, its comparison to the Long-Video Architectures is unfair. (4) The evaluation of LongEmo considers only one model configuration described in Appendix E. The generality of LongEmo to different backbone models is not demonstrated.
-
Several critical details of the LongEmoBench curation are missing. (1) LongEmoBench is annotated by humans. How many annotators are involved? How are they selected or recruited? What is the correlation between the annotations from different human annotators? These questions are important to judge the reliability of the proposed benchmark. (2) The authors should explain more about the video source of this benchmark. Are there any copyright or terms-of-use issues? This is important for the ethics review.
Although the proposed benchmark could be valuable, the above two major concerns make me rate it a clear rejection at this stage.
Questions for authors
See weaknesses.
(Minor) The font sizes in the figures are too small.