- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces AirRelComp, a diagnostic benchmark for video-text compositional understanding in airport surveillance videos. The benchmark organizes positive video-text pairs along Temporal, Spatial, and Entity Relations, and constructs four types of negatives (Loc-Replace, Dir-Reverse, Stage-Reorder, Entity-Swap) by altering one relation while preserving the rest.
Critical strengths
The annotation protocol and quality control pipeline of the proposed dataset are described in detail, i.e., LLM checker + human review on 25% of flagged pairs.
Critical weaknesses
Constructing a dedicated dataset for the airport surveillance scenario can only serve as a benchmark for video-text compositional understanding. It should not be claimed that Video-Text Compositional Understanding in Airport Surveillance is itself a novel contribution of this paper.
The main contribution of this paper is the construction of a new dataset, which, in my view, is not well aligned with the research scope and standards of ICLR. Moreover, the paper does not clearly explain why this dataset is necessary or what fundamental limitations of existing datasets it is intended to address.
The proposed AirRelComp is built from only 118 videos (15.19 hours) recorded at a single airport from two fixed viewpoints, yielding 1,544 positive pairs and 3,855 controlled comparisons. The benchmark is too small and too narrow to serve as a general benchmark.
For LMM binary choice (Table 3), the closed-source models (Gemini 3.5 Flash, ChatGPT series) are evaluated zero-shot, but no fine-tuned LMM baseline from prior compositional work is included.
The paper's core pipeline, controlled negative construction from existing positive pairs, a pairwise preference loss that ranks positives above relation-specific negatives, and evaluation of both video-text models and LMMs, is substantially the same as VideoComp [1].
[1] Videocomp: Advancing fine-grained compositional and temporal alignment in video-text models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 29060–29070, 2025.
No comparison to VideoComp's training objective [1] on AirRelComp, despite it being the closest method.
Questions for authors
The paper is not written to a sufficiently rigorous academic standard, and the proposed method does not demonstrate meaningful technical novelty. I strongly recommend that the authors rely less on GPT-generated writing and instead provide clearer technical insights, stronger justification, and more original analysis.