- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces EventTriggerBench to evaluate whether models can identify earlier events that plausibly explain a target event in an ordered video sequence. The target may occur anywhere in the timeline, and the task includes non-adjacent triggers, multi-step explanatory chains, and multiple triggers. The benchmark contains 1,240 videos, 8,936 event segments, and 3,482 target-conditioned samples. Trigger labels are proposed by a language model and automatically verified, and the paper also reports a human audit of 300 test samples. The authors further propose EventGraph-Trigger, which uses an event graph, multimodal features, and target-conditioned message passing to predict triggers and explanatory chains. The experiments compare a range of baselines and include ablation, difficulty, and robustness analyses.
Critical strengths
-
The task allows the target event to appear anywhere in the video timeline, enabling evaluation of non-adjacent triggers, multi-step explanations, and multiple triggers rather than focusing only on the final event. The reported statistics show that many triggers are not immediately adjacent to their targets, and a simple previous-event heuristic performs poorly.
-
The paper describes the data construction and label-quality checks in detail. It reports human acceptance rates before and after automatic verification, inter-annotator agreement, and comparisons between independently collected human trigger labels and automatic labels. The model evaluation also includes a parameter-matched multimodal Transformer, component ablations, and results across different difficulty conditions.
Critical weaknesses
- The trigger events and explanatory chains are proposed and verified by language models that receive only event captions. For videos without existing event captions, the paper uses Video-LLaVA-7B to generate them. If a caption omits or misrepresents a key detail, the automatic label cannot assess the trigger relation based on that visual evidence. In Table 7, removing text features reduces EventGraph-Trigger’s accuracy by 9.0 percentage points, while removing video features reduces it by only 3.3 points. In Table 15, masking captions reduces EventGraph-Trigger’s accuracy from 72.7 to 49.6 and GPT-4o’s from 64.3 to 43.5. These results show that the models rely substantially on captions, but they do not establish whether visual evidence itself is weak or whether captions or automatically generated labels omit important visual information. Although the paper includes a human audit of 300 test samples in which annotators view the videos, labels for the remaining samples rely primarily on the caption-based automatic pipeline.
Questions for authors
- Could the authors report the accuracy of each model on a visually diagnostic subset under three caption conditions, namely complete captions, captions with 50 percent of their content words randomly removed, and fully masked captions? In particular, how much does EventGraph-Trigger improve over text-only baselines on samples where visual evidence is necessary to identify the trigger?
2 comments
Thank you for your response. My concerns are mostly addressed, and I maintain my overall positive rating.
Response to Reviewer VRse
We thank the reviewer for requesting a visual-evidence-specific evaluation.
1. Visual evidence under three caption conditions
On the 60-example human-defined visually diagnostic subset, video improves EventGraph-Trigger (EGT) by 23.3 pp under full captions, 25.0 pp under 50% content-word deletion, and 23.3 pp when captions are fully masked, relative to its matched no-video variant. The much larger gain than on the full test set (+3.3 pp; manuscript Table 7) shows the value of video for these cases.
The modest overall video gain may have two causes: video evidence may add little beyond captions, or caption-derived labels may omit visually important details. To distinguish these, annotators first select triggers using captions alone and then re-evaluate the same 300 audited examples with the original clips. The 60 examples in which visual evidence changes the supported attribution form the visually diagnostic subset, selected independently of model predictions. We evaluate both automatic-label agreement and model accuracy against the video-informed human reference.
n=60), Acc.The original caption-derived labels agree with the video-informed reference on 32/60 (53.3%) visual cases, versus 201/240 (83.8%) remaining cases. This supports the role of both additional visual evidence and caption-label omissions.
The full model's 56.7% accuracy on this difficult subset also shows that visually grounded attribution remains challenging despite the substantial relative gain.