- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes ChapterNav to evaluate and optimize video chaptering through navigation utility. A model generates a directory without access to the query, and a frozen text encoder selects chapters based on query–title similarity. The proposed SNG metric combines target-interval coverage with excess viewing cost and supplies feedback for GRPO post-training. The main contributions are the navigation evaluation protocol, query–episode annotations, and navigation-based reward design.
Critical strengths
The motivation is reasonable: agreement with reference chapters does not necessarily imply effective access to desired content. Navigation utility provides a meaningful complementary evaluation perspective. Keeping queries out of the generator input preserves the directory’s role as a reusable navigation structure. Combining coverage and excess access cost also avoids rewarding a trivial whole-video chapter.
Critical weaknesses
Temporal annotation reliability limits the claim of verifiable navigation. Table 6 states that KZKT target timestamps are estimated from text length without per-episode verification against the video clock. Since both SNG and the training reward depend on temporal overlap and a coverage threshold of 0.9, annotation errors can directly change success judgments. The paper does not quantify their effect on system rankings or training gains. There are substantive inconsistencies in the experimental setup and main results. The main text and appendix report 105 versus 70 AVLecture test videos, 436 versus 337 KZKT test videos, and 150 versus 300 training steps. More importantly, Table 2 labels the AVLecture navigator as MiniLM, but its RL navigation results match the Qwen results in Table 3. These discrepancies make it difficult to establish whether the main comparisons use a consistent evaluation setup.
Questions for authors
How large are the discrepancies between estimated KZKT timestamps and the actual video clock? On a manually verified subset, do system rankings and the RL gains over SFT remain consistent? Please specify the data splits, training steps, and navigator actually used for the main results, and explain why Table 2’s MiniLM-labeled results match the Qwen values in Table 3. Do the main conclusions hold under a consistent setup? Do improvements in SNG translate into higher success rates or lower viewing costs when humans actually locate content? What are the actual annotator count, valid sample count, inter-annotator agreement, and statistical uncertainty of the existing preference study?