- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces a benchmark for joint video-to-product retrieval and temporal grounding. It supports product retrieval, temporal grounding, and their joint evaluation, with the aim of capturing the relationship between product identity and its temporal evidence in a video. The benchmark is challenging, with the evaluated open-source models generally achieving low performance. The paper also introduces UniPro, whose architecture and training approach substantially improve performance on these tasks.
Critical strengths
- The benchmark poses a challenging evaluation setting, as reflected in the generally low performance of the evaluated open-source models.
- UniPro substantially improves performance over the evaluated open-source models, demonstrating the effectiveness of the proposed approach on this benchmark.
Critical weaknesses
- The paper does not convincingly explain why these tasks need to be combined within a single benchmark. Training separate models for retrieval and grounding could still support the downstream applications discussed in the paper. It remains unclear what essential benefit the joint formulation provides over this alternative.
- As I understand the method, temporal information is not incorporated into the video embeddings themselves. Instead, the encoder is frozen, and an additional decoder performs temporal grounding. This does not appear to constitute an end-to-end solution, and it is unclear how the model learns the relationship between retrieval and grounding. This raises the question of whether optimizing the two tasks separately might be more effective.
- The evaluated open-source models were released more than a year ago. Including strong close-source models and more recent open-source models would provide a more informative assessment of the benchmark's difficulty and the proposed method's relative performance.
Questions for authors
- Could you clarify why retrieval and temporal grounding need to be combined, and what benefits this joint formulation offers over training and optimizing separate models for the two tasks?
- Could you provide additional results for more recent open-source models and strong close-source models?