- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors introduce ChronosKV, a training-free temporal encoding for streaming video LLMs. For that the embedding is decoupled into order and temporal distance between frames, introducing a dual-clock temporal encoding. The modifications are light-weight because of only query-time position index changes.
The authors introduce a new benchmark including the time distance between events to account for the temporal distance called Interval-QA.
Critical strengths
- Introducing a light-weight training-free temporal embedding on already existing video LLMs, might have great impact on already existing systems.
Critical weaknesses
- W1 Insufficient documentation around new Interval-QA Benchmark: While this paper introduces a new temporal encoding it also introduces the benchmark it measures the main uplift in performance as well. The newly introduced one has been annotated with Claude. The protocol for that is insufficiently documented.
- W2 Only limited improvement on public benchmarks: Table 8 shows only marginal improvements on public benchmarks. This indicates that the dual-clock encoder adds only marginal improvement compared to ordered positional encodings. The main improvement is measured on the newly introduced Interval-QA benchmark which makes a transparent protocol highly important.
Questions for authors
In section 4 a frontier multimodal modal has been mentioned without clarifying which one it was. Which exact Anthropic model has been used? How have the Claude annotations been validated? Can you please provide more transparency around the benchmark creation?