- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies long-term composed video retrieval (CVR), where users iteratively refine their search requests through reference videos and textual feedback. Existing CVR benchmarks mainly focus on single-turn retrieval and do not adequately capture how user intent changes across interactions. To address this gap, the authors introduce LTCVR, a benchmark with 4,000 five-turn dialogues and 3,160 candidate videos. It covers five scenarios: progressive refinement, rollback, cross-turn combination, local correction, and preference change, which test whether models can retain useful historical information, revisit earlier results, combine constraints across turns, and discard outdated preferences. The authors also propose InAHR, a framework that selectively revisits interaction history. It first uses one representative frame from each reference video, together with the corresponding feedback, to infer the current retrieval intent and identify relevant turns. It then selects a compact set of historical frames by combining turn-level and frame-level relevance with a facility-location objective to reduce redundancy and preserve complementary evidence. Finally, a multimodal large language model uses the selected evidence to generate multiple target descriptions, whose text embeddings are averaged for retrieval. On LTCVR, InAHR achieves 41.9% Recall@1 and 66.7% Recall@5, exceeding the strongest baseline, with improvements across all five scenarios. The paper also includes component ablations and evaluations with different model backbones and more histories to further examine the method’s effectiveness.
Critical strengths
The benchmark provides a structured test of history-dependent video retrieval. LTCVR distinguishes between retaining, revisiting, combining, and discarding historical information, enabling a more targeted assessment than single-turn retrieval benchmarks. Its documented human review and quality-control process strengthen its value as a diagnostic resource, although its relevance to real user behavior still needs validation.
Critical weaknesses
- InAHR combines established techniques: MLLM-based intent inference, CLIP-based relevance scoring, facility-location selection, and multi-description aggregation. The pipeline is reasonable, but the paper does not establish a substantial technical advance beyond adapting these components to the proposed task. The main refocusing ablation also changes the available visual evidence, making it difficult to separate the benefit of adaptive selection from that of additional frames. This weakens the method’s contribution to the case for acceptance.
- LTCVR is built from synthetically generated trajectories and scenario-specific rewrites, without real user sessions or a user study demonstrating that these patterns reflect actual video-search behavior. Human verification supports sample correctness and coherence, but does not establish the prevalence of rollback or cross-turn combination. Evaluation also uses fixed histories, leaving the effectiveness of the task in a closed-loop search setting unclear. Furthermore, the paper does not report video-duration statistics or quantify how many queries require temporal reasoning. Given the method’s reliance on static frame features and mean-pooled video representations, a temporally demanding subset or frame-order perturbation study is needed to distinguish video understanding from static appearance matching.
Questions for authors
- What motivates the fixed five-turn setting, and which realistic video-search needs require history-dependent interaction rather than a single, sufficiently detailed query? Please clarify when users would benefit from progressively expressing their requirements, and whether the proposed setup reflects such needs or primarily serves as a controlled task construction.
- What are the average duration and duration distribution of LTCVR videos, and what proportion of queries require understanding action order or temporal relationships? How does performance change when only a single frame is provided or frame order is shuffled?
- In Table 3, InAHR’s Recall@1/Recall@5 on Preference Change drops from 39.8/68.3 with last-turn input to 27.2/51.8 with full history, falling below even the last-turn results of CoTMR and ImageScope. What causes this substantial degradation despite the intent-driven history selection? These results suggest that historical interference is reduced rather than effectively eliminated; could the authors clarify the scope of their claim and whether the method can learn when to discard history entirely?
- Under the same budget of eight evidence frames, two MLLM calls, and three target descriptions, how does InAHR compare with random frame selection, uniform sampling, and simple relevance-based selection? This comparison would help isolate the contribution of adaptive history refocusing from the benefits of additional visual evidence and inference computation.
- LTCVR constructs queries around known target videos, so every evaluated request has a matching answer by design. In real-world retrieval, however, users may specify modifications or combine constraints that no video in the gallery satisfies. How would the proposed framework handle such requests? Can the benchmark include no-match queries or partially satisfiable constraints to evaluate whether the system recognizes the absence of a suitable result rather than simply returning the closest video?
We added equal-budget ablations, clarified the five-turn protocol and temporal limitations, and examined historical interference and behavior on no-match queries.
Dear Reviewer 36Bh,
1. W1 & Q4: Methodological Novelty and Equal-Budget Ablation
We acknowledge that the individual components are established techniques. InAHR's contribution lies in hierarchical, intent-conditioned history refocusing, which integrates soft turn priors, frame-level relevance, and representative selection to recover relevant historical evidence under evolving retrieval intents.
We also recognize the unequal frame budget in the original w/o Refocusing ablation. We therefore conducted equal-budget comparisons using eight frames, two MLLM calls, and three target descriptions, with all other settings unchanged. Relevance-8 corresponds to w/o Representative Selection in Table 3 of the revised manuscript.
InAHR improves Avg. by 9.7, 3.5, and 1.8 points over Random-8, Uniform-8, and Relevance-8, respectively, supporting the benefits of intent-guided, representative evidence selection beyond additional frames or relevance alone.
2. W2 & Q1: Five-Turn Protocol and Realistic Interactions
We adopt a standardized five-turn protocol to systematically evaluate diverse intent transitions under controlled conditions, particularly Rollback and Cross-turn Combination. This setting provides sufficient historical context to construct challenging interaction patterns while ensuring fair comparisons across methods.
Importantly, multi-turn retrieval is not simply splitting a complete query into multiple instructions. For example, when searching for travel videos, a user may prefer the beach scene in an earlier result and the sunset atmosphere in a later one, then request a video combining both. Such preferences emerge through interaction, requiring models to recover and integrate historical visual evidence rather than relying solely on the latest query.
While our benchmark adopts five turns for controlled evaluation, InAHR can also handle longer interactions. Appendix F.4 evaluates 7- and 9-turn extensions (600 sessions each), achieving 33.50/55.33 and 30.33/53.67 R@1/R@5, respectively, with gains of 7.50/10.00 and 6.66/12.17 points over the strongest baselines. We acknowledge that variable-length interactions better reflect real-world retrieval and will explore them in future benchmark extensions.
3. W2 & Q2: Video Duration and Temporal Understanding
We agree that distinguishing temporal understanding from static appearance matching is important. Our multimodal ablation shows that replacing historical videos with captions reduces overall R@1 from 41.9% to 37.4%, highlighting the value of historical visual evidence. We further analyzed video durations and identified temporally dependent queries involving state changes, action order, or temporal relationships.
These results indicate that LTCVR includes substantial temporal dependencies beyond static appearance matching. However, we acknowledge that InAHR primarily leverages multi-frame visual evidence without explicitly modeling fine-grained temporal order, and will clarify this limitation in the revised manuscript.
4. Q3: Historical Interference on Preference Change
We acknowledge that Soft Prior may retain obsolete history. We therefore evaluated a Conditional Hard Evidence Switch that selects all eight frames from the latest turn when Stage 1 identifies it as the only relevant turn; otherwise, Soft Prior is retained. This requires no ground-truth scenario labels and preserves the inference pipeline.
The switch improves R@1 from 27.2 to 28.5, while R@5 remains unchanged. In a separate ablation, Soft Prior outperforms Hard Turn Mask overall (61.7 vs. 60.3 Avg.), suggesting that hard masking may discard useful evidence. We therefore retain Soft Prior for broader applicability across different scenarios.
5. Q5: No-Match Queries