- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents a study of video language models (VLMs), focusing on action-object binding and chronological dependencies in the temporal understanding of three 7B VLM backbones. Towards this aim, the authors first carefully curate a subset of the STAR benchmark (Wu et al., 2021), implementing certain heuristics to make sure that visual reasoning is clearly isolated from language priors, and employing both open-ended and close-ended probes. The intervention experiments, defined based on the existing bounding box annotations of objects, demonstrate that task-critical semantics is highly localized with object-aligned token positions. Next, the paper explores the layer-wise emergence of language-aligned object and action concepts, and shows that they are concentrated to mid-to-late LLM layers. Based on these observations, the authors propose some measures to identify the so-called temporal heads, which route strongly to video tokens and distribute attention across multiple temporal slices. Then, they test the hypothesis that amplifying these temporal heads will carry cross-frame evidence in a more effective way, where they validate this on seven tested benchmarks, providing statistically-significant consistent gains.
Critical strengths
- The story is coherent throughout the paper, and the experiments are consistent with the story. Each analysis is well chosen and creates a building block for the next section of the paper.
- The proposed temporal head amplification is shown to provide consistent gains in seven tested benchmarks, without any finetuning.
Critical weaknesses
- The paper needs to report a more detailed discussion on the conceptual similarities and differences between the findings of the proposed approach and the one in (Kim et al., ICLR 2026) to better assess the contributions of the work.
- The rules to identify temporal heads depend on four different thresholds. It is not clear how the values for these thresholds are selected.
- The name “circuitprobe”, and calling the framework “a circuit-based tracing framework” gives a false impression about the approach. As the authors say in the paper, it is a layer-wise analysis but using the word “circuit” implies a neuron-level detailed computational analysis.
Questions for authors
- Provide more details on the selected thresholds, or a sensitivity analysis would be much better. Do all the experiments in different benchmarks use the same thresholds?
- How does amplifying the temporal heads qualitatively affect the attention maps? What extra evidence can the authors provide to understand the contribution of the amplification?