- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Interactive Composed Video Retrieval (ICoVR), extending single-turn composed video retrieval into a multi-turn setting where users progressively refine their search intent through natural-language feedback.
The proposed ReCoVR framework consists of an Intent Pathway that decomposes feedback into text-to-video and composed video retrieval queries, a Reflection Pathway that maintains semantic constraints and demotes rejected or repeatedly retrieved candidates, and Time-Weighted Reciprocal Rank Fusion (TW-RRF), which combines retrieval rankings across turns.
The authors evaluate ReCoVR on three composed video retrieval benchmarks against adapted interactive retrieval baselines, reporting substantial improvements in Recall@K. They additionally provide component ablations, backbone generalization experiments, and a limited human-feedback study.
Critical strengths
1. Consistent empirical improvements across multiple benchmarks.
The full ReCoVR system achieves substantial improvements over the adapted interactive baselines across three CoVR datasets. The component ablations, backbone evaluations, and paired confidence intervals provide evidence that the overall system is effective under the proposed evaluation protocol.
2. Practical training-free integration.
ReCoVR operates with frozen retrieval backbones and introduces no additional retriever training. This makes the framework relatively straightforward to integrate into existing retrieval systems.
Critical weaknesses
1. Limited technical novelty and insufficient differentiation from substantial prior work.
My primary concern is that the paper's central methodological contributions largely consist of straightforward applications and combinations of established techniques, without demonstrating a sufficiently substantive new algorithmic or conceptual insight.
The first contribution, extending single-turn composed video retrieval to a multi-turn setting through interaction history, memory, and feedback, has substantial precedent. Guo et al. (Dialog-based Interactive Image Retrieval, NeurIPS 2018) already investigate iterative visual retrieval using natural-language feedback and simulated users. Yuan and Lam (Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback, SIGIR 2021) explicitly extend composed image retrieval to multiple turns by incorporating the reference image, modification text, and conversation history. Pal et al. (FashionNTM, ICCV 2023) develop persistent memory for multi-turn visual retrieval, while Chen et al. (MAI, ICLR 2025) directly investigate historical semantic aggregation and iterative optimization for multi-turn composed image retrieval.
The distinction that ReCoVR addresses video rather than image retrieval is relevant, but applying an established multi-turn retrieval paradigm to another modality is not, by itself, a compelling methodological contribution. The paper does not sufficiently establish what fundamental new challenge requires its particular formulation beyond the existing history-aware composed retrieval literature.
The second contribution, Time-Weighted Reciprocal Rank Fusion (TW-RRF), also has substantial precedent. Rollings et al. (Multi-Faceted Question Fusion in the TREC 2022 CrisisFACTS Track) already investigate recency-weighted RRF for information retrieval. More directly, Harel et al. (Sentence Retrieval for Open-Ended Dialogue Using Dual Contextual Modeling, ECIR 2023) fuse rankings generated from individual dialogue turns using RRF and explicitly evaluate linear and exponential decay weights. ReCoVR's Eq. (7) follows the same general weighted-RRF construction, with a manually specified linear recency schedule and a sliding temporal window.
Similarly, the Reflection Pathway primarily uses LLM-based constraint accumulation and a simple rejection/repetition heuristic for reranking. Its rank-cap policy (Eq. 5) responds to explicit rejection or persistence of the previous top-ranked result rather than performing a general analysis of retrieval quality or trajectory dynamics.
The individual components are reasonable engineering choices, but I find their algorithmic novelty limited. The paper would need to establish a more distinctive methodological contribution, or demonstrate that the particular combination enables capabilities that established history-aware retrieval and rank-fusion techniques cannot adequately provide.
2. Insufficient evidence for the necessity and incremental significance of the proposed mechanisms.
Although the full system performs well against adapted interactive baselines, the ablations provide considerably weaker evidence that the particular proposed mechanisms are important.
In Appendix A.1, Table 5, TW-RRF achieves 90.22% R@1 at Turn 5 compared with 89.03% for uniform-weight RRF, an improvement of only 1.19 percentage points. This is a relatively small gain for one of the paper's explicitly proposed algorithmic components, particularly given the simplicity of the linear weighting rule and the substantial prior literature on recency-weighted retrieval.
Furthermore, Eq. (6) implies that TW-RRF and uniform-weight RRF assign identical weights when only one feedback turn is available. However, Table 5 reports different Turn-1 R@1 values (74.30% versus 73.63%). This discrepancy cannot be explained by the temporal weighting formula itself under identical channel rankings and suggests either stochastic variation, additional implementation differences, or a reporting inconsistency. Since the final claimed advantage of temporal weighting is only 1.19 points, a properly paired comparison with controlled randomness and confidence intervals is important before attributing the difference to the weighting scheme.
The paper also does not compare against other simple recency functions, such as exponential decay, despite their established use in prior multi-turn retrieval literature. Thus, it is not demonstrated that the particular proposed weighting rule is meaningfully preferable to standard alternatives.
For the Reflection Pathway, Table 3 shows that the Intent Pathway combined with rank-cap already achieves 87.52% Turn-5 R@1 without State Reflection, compared with 90.22% for the complete system. This incremental 2.70-point improvement is positive, but the experiments do not clearly establish that it reflects a distinctive trajectory-level diagnosis mechanism rather than improved semantic query reformulation. The reported recovery statistics are also observational comparisons between different interaction trajectories rather than isolated causal evidence for the reflection mechanism.
There is additionally a potential inconsistency in the claimed prevention of repeated rejected candidates. Demoting the previous top-1 result to rank 11 in both retrieval channels does not guarantee that it cannot remain top-ranked after reciprocal rank fusion, particularly when historical rankings are retained. The claim in Appendix A.4 that repeated rejected results are prevented by construction therefore does not follow directly from Eqs. (5)–(7).
Overall, the experiments convincingly show that combining these components improves the evaluated retrieval pipeline. However, they do not adequately establish the necessity, novelty, or incremental significance of its distinguishing algorithmic choices. Given the extensive prior literature and the modest gains over simpler variants, I do not find the demonstrated contribution sufficiently substantial for acceptance.
Questions for authors
1. What is the substantive methodological novelty relative to existing multi-turn composed retrieval methods?
Prior work including Yuan and Lam (SIGIR 2021), FashionNTM (ICCV 2023), and MAI (ICLR 2025) already investigates maintaining historical visual/textual information and progressively refining retrieval results through natural-language feedback. Beyond transferring this established paradigm to composed videos and combining existing retrieval modules, what specific algorithmic mechanism or previously unsolved technical challenge distinguishes ReCoVR from these methods?
2. How is TW-RRF meaningfully different from established temporally weighted rank fusion?
Rollings et al. (TREC 2022) previously incorporate recency information into RRF, and Harel et al. (ECIR 2023) explicitly fuse per-dialogue-turn rankings using RRF while investigating linear and exponential decay weights. Could the authors clarify the novelty of Eq. (7) relative to this literature and compare the proposed weighting against exponential decay or other simple alternatives under identical retrieval conditions?
3. Why do TW-RRF and uniform-weight RRF produce different results at Turn 1?
With only one feedback turn, Eq. (6) gives a temporal weight of exactly 1, identical to uniform weighting. Nevertheless, Table 5 reports Turn-1 R@1 values of 74.30% and 73.63%, respectively. Could the authors explain this discrepancy and provide a paired evaluation of the two fusion methods with identical channel rankings and controlled randomness? Given that their Turn-5 difference is only 1.19 percentage points, paired confidence intervals would materially affect my assessment of TW-RRF's contribution.
4. What evidence establishes that the Reflection Pathway performs meaningful trajectory-level diagnosis rather than simple feedback-based reranking?
Equation (5) primarily checks explicit rejection or repeated top-1 retrieval. This appears substantially narrower than the claimed diagnosis of retrieval drift and stagnation. Could the authors isolate the improvement in recovery rates attributable specifically to State Reflection, controlling for rank-cap and the other components? A comparison with simpler history-aware retrieval and negative-feedback mechanisms would help establish its independent contribution.
5. How does rank-cap guarantee the prevention of repeated rejected candidates?
Under the reported RRF constant k=60, a candidate appearing at rank 11 in both channels obtains a fused score of 2/(60+11)=0.0282, which can exceed that of a candidate appearing at rank 1 in only one channel, 1/(60+1)=0.0164. Historical rankings can further reinforce previously rejected candidates. How, then, does the stated rank-cap policy guarantee the claim in Appendix A.4 that repeated rejected results are prevented by construction? Is there an additional post-fusion exclusion mechanism not specified in the formulation?
5 comments
[W1b] … Time-Weighted Reciprocal Rank Fusion (TW-RRF), also has substantial precedent. … ReCoVR's Eq. (7) follows the same general weighted-RRF construction …
[W2a] … TW-RRF achieves 90.22% R@1 at Turn 5 compared with 89.03% for uniform-weight RRF, an improvement of only 1.19 percentage points.
[W2c] The paper also does not compare against other simple recency functions, such as exponential decay …
[Q2] How is TW-RRF meaningfully different from established temporally weighted rank fusion? … compare the proposed weighting against exponential decay or other simple alternatives under identical retrieval conditions?
Within the closed loop (W1a), TW-RRF is the step where the corrections take effect. It fuses two heterogeneous channels across turns, and its inputs are the channel lists after rank-cap, so it turns the loop's corrections into the final ranking. TW-RRF builds on RRF and recency weighting, and we have cited both works in Sec. 3.3; its difference from these works lies in what it fuses. The contributions listed in Sec. 1 are the ICoVR task and this closed-loop framework, of which TW-RRF is one component.
Table 5 separates the parts of this fusion step:
| Fusion variant (Table 5) | Turn-5 R@1 | What it removes |
|---|---|---|
| TW-RRF (ours) | 90.22 | — |
| Uniform-RRF | 89.03 | recency weighting |
| Static-RRF (current turn only) | 86.03 | cross-turn aggregation |
| SimSum / SimMax | 83.29 / 82.75 | rank-level fusion |
Each part contributes to the final ranking. The reviewer's question concerns the recency weighting, which we test next.
To test whether the recency effect is reliable, and to compare schedules under identical retrieval conditions as requested, we saved the per-turn channel rankings of the full model and replayed different fusion rules on them. At Turn 5, the interaction up to Turn 4 is kept and only the current-turn fusion rule is replaced, so all rules see exactly the same channel lists (paired bootstrap 95% CI over 2,556 queries):
| Turn 5: linear (ours) minus | ΔR@1 | 95% CI |
|---|---|---|
| Uniform weights | +1.19 | [0.67, 1.76] |
| Exponential decay, | +0.16 | [−0.16, 0.51] |
| Exponential decay, | +0.12 | [−0.16, 0.43] |
(The uniform gap is from Table 5; its CI is from the replay.) Recency weighting gives a consistent gain over uniform weights, and this gain does not depend on the exact schedule: linear and exponential decay perform on par. We have added this paired comparison, including exponential decay, to Appendix A.1. What matters is down-weighting older turns, and we keep the linear form for its simplicity.
[W1c] … Its rank-cap policy (Eq. 5) responds to explicit rejection or persistence of the previous top-ranked result rather than performing a general analysis of retrieval quality or trajectory dynamics.
[W2d] … the Intent Pathway combined with rank-cap already achieves 87.52% Turn-5 R@1 without State Reflection, compared with 90.22% for the complete system. … The reported recovery statistics are also observational comparisons …
[Q4] … Could the authors isolate the improvement in recovery rates attributable specifically to State Reflection, controlling for rank-cap and the other components? A comparison with simpler history-aware retrieval and negative-feedback mechanisms would help …
Within the closed loop (W1a), State Reflection is the step that detects a failed turn, and rank-cap is the step that acts on it. We understand the underlying concern as whether State Reflection adds anything beyond query rewriting and a rejection rule.
First, the requested control is already in Table 3. The row with rank-cap on and State Reflection off is exactly this setting: is fixed to neutral, no constraints are accumulated, and rank-cap reacts only to stagnation. Table 3 isolates State Reflection both with and without rank-cap, on two benchmarks:
| Turn-5 R@1 (Table 3) | WebVid-CoVR | FineCVR |
|---|---|---|
| Intent Pathway only | 77.46 | 63.69 |
| + State Reflection | 83.26 | 69.40 |
| + rank-cap | 87.52 | 76.87 |
| + State Reflection + rank-cap (full) | 90.22 | 78.65 |
State Reflection improves results with and without rank-cap on both benchmarks, and the full model has the best R@1 at every turn. The two mechanisms are complementary: rank-cap corrects the current ranking, while State Reflection shapes later queries through accumulated constraints. Since the system never sees the target, the rule in Eq. (5) acts on the events that are observable in the trajectory: a rejection detected by State Reflection, or a top-1 result that persists across turns.
Second, we traced the variant without State Reflection over the full WebVid-CoVR-Test. Without State Reflection, failed turns become more frequent and recovery drops:
| Active feedback turns | Full ReCoVR (Table 8) | Without State Reflection (new) |
|---|---|---|
| Target rank does not improve | 20.23% | 26.33% |
| After such a turn: next turn improves the rank | 88.15% | 74.67% |
The controlled comparison is the matched ablation in Table 3; these rates, each measured on its own trajectories, are consistent with it. Third, the simpler alternatives in W1a, full-history rewriting (history-aware, no reflection) and UMIVR + rank-cap (negative feedback, no reflection), both stay below ReCoVR. In matched comparisons, State Reflection therefore contributes beyond rank-cap and beyond query rewriting.
We thank the reviewer for the detailed review, and for recognizing the consistent improvements across three benchmarks, the supporting component ablations, backbone evaluations, and paired confidence intervals, as well as the practical value of our training-free integration. We address each point below.
What distinguishes ReCoVR from these methods is the closed loop: the system uses the outcome of its own previous retrieval as a signal to correct the next one. This addresses a concrete challenge in interactive retrieval: recovering from a failed turn.
A turn fails when the user gives feedback but the target does not move up. The adapted interactive baselines update the query from the feedback and search again, and for them such a failure tends to repeat. UMIVR shows the same top-1 result again after 70.77% of feedback turns, and it rarely recovers after a failed turn, whereas ReCoVR usually does (Table 8, Appendix A.4):
ReCoVR handles such a turn in three steps (Sec. 3.2.2). Detect: after each feedback, State Reflection judges whether the user rejected the displayed result, and the system checks whether the same result has stayed at the top across turns. Correct the ranking: in either case, rank-cap moves that result down in both channel lists before fusion. Correct the query: State Reflection also extracts constraints from the feedback, which accumulate in memory and guide later queries.
These rank-level corrections are what separate ReCoVR from the cited works. Those works, also discussed in our Related Work, use a trained model to map the dialogue history to an updated query, which decides what to search for next. To our knowledge, none of the cited methods uses such rank-level signals to correct its ranking across turns, and ReCoVR does so without training, since the CoVR benchmarks contain single-turn triplets only. It also routes the two kinds of composed feedback, edits of the displayed video and direct descriptions of the target, to a CoVR and a T2V channel. We have added a comparison with the cited works (training data, modality, history signal) to Related Work and Appendix H.
Finally, as the reviewer asks, we test whether simpler mechanisms provide the same capability: history-aware rewriting, and negative feedback added to an open-loop baseline, under the same retriever and simulator. Both stay clearly below ReCoVR:
The closed loop therefore provides a capability, recovering from a failed turn, that history aggregation and negative feedback alone do not provide.