- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses the limitations of sparse and noisy captions in video-text pretraining. It proposes multi-view caption generation with reasoning-based refinement and a granularity-aware text representation using dual CLS tokens. Experiments demonstrate significant improvements in zero-shot and fine-tuned video-text retrieval across multiple benchmarks.
Critical strengths
- This paper proposes an effective multi-view captioning framework with complementary supervision and granularity-aware text representations.
- This paper achieves strong retrieval performance, particularly on fine-grained benchmarks, with fewer pretraining videos.
Critical weaknesses
- Prior works such as HACA[1] have already investigated hallucination correction using LLMs. Could the authors further discuss this work and clarify the novelty of their proposed supervision generation framework?
- In Tables 1–3, the compared methods use different pretraining datasets. For example, UMT uses a mixture of video-text and image-text data, while MVC is trained on InternVid-10M-FLT with only video-text data. The authors should explicitly clarify these differences in data composition and pretraining configurations to ensure a fair interpretation of the performance comparisons.
- The proposed method assigns equal weights to summary and detailed captions during both training and inference. However, Table 5 shows substantial performance differences when using these two types of captions independently. What motivates the equal weighting strategy, and could the authors provide an ablation study comparing different loss weights and similarity fusion weights to justify this design?
[1]Zhao, Lingjun, et al. "Can Hallucination Correction Improve Video-Language Alignment?." Findings of the Association for Computational Linguistics: ACL 2025. 2025.
Questions for authors
refer to weakness
2 comments
Dear Reviewer, thank you again for your review. We have provided additional experiments and revised the manuscript to address your concerns. As the discussion period draws to a close, we would appreciate your feedback on whether these additions address your main concerns and which issues, if any, remain unresolved. We would be happy to clarify further.
Response to Reviewer hGir
We thank the reviewer for the careful reading and constructive suggestions. Below, we clarify our contribution relative to HACA, detail the differences in pretraining data and configurations, and report new experiments on training and inference weights.
W1. Relation to HACA and novelty of the supervision generation framework
We agree that hallucination correction has been studied before, and we will explicitly discuss HACA [1] in the related work. HACA fine-tunes a video-language model to detect and correct hallucinated descriptions, improving that model's own alignment. Our goal is different: we construct pretraining supervision. Our framework:
The resulting corpus then supervises a separate video-text model. Our refinement is training-free: it uses an off-the-shelf reasoning MLLM and is applied once to 10M captions. Refinement is one component of the framework. Our novelty lies in constructing and jointly using complementary supervision views (Tables 4–6), not in hallucination correction itself. The two approaches address different stages of video-language learning and could be combined. Finally, our response to Reviewer YZJy shows on DREAM-1K and CaReBench human annotations that our refinement selectively removes unsupported content.
W2. Differences in pretraining data and configurations
We agree that equal sample counts do not imply matched conditions. For example:
In the revision, we will add a table in the appendix listing, for each compared method:
We will also state in the results that Tables 1–3 compare complete pretrained systems and do not isolate the effect of caption supervision except for comparison with Baseline-B.
W3. Motivation for equal weights, and ablation of loss and fusion weights
Motivation. Table 5 shows that summary and detailed captions differ in quality when used alone, but that they are complementary: combining them outperforms either one. With dual-
CLS, each view is aligned through its own text representation, so a weaker view does not degrade the other. We therefore give both views equal weight as an unbiased default that requires no tuning.Ablation. We trained two additional ViT-B models on 1M videos with summary-to-detailed loss weights of 2/3:1/3 and 1/3:2/3. We also varied the inference fusion weights on the equal-weight checkpoint. We report zero-shot text-to-video R@1.
Table A. Training and inference use the same weights
Equal weighting performs best on eight of eleven benchmarks. The largest gains over both unequal settings are on DiDeMo, CARE-T, and DREAM-E. Giving more weight to the summary view performs best on MSRVTT and LSMDC, while giving more weight to the detailed view performs best on CARE-S. These results compare configurations that use the same weights for training and inference.
Table B. Inference fusion weights on the equal-weight checkpoint
Performance is stable across fusion weights. With the checkpoint fixed, changing the fusion weights shifts R@1 by at most 0.7 points from equal averaging on any benchmark. The best weights vary across benchmarks, so we use equal averaging by default. We discuss HACA in the related works, and W2 and W3 in the Appendix A.8 and A2 respectively.
We hope these clarifications and additional experiments address the reviewer's concerns. We would be grateful if the reviewer would reconsider their score in light of this evidence, and we are happy to answer any further questions.