acceptodds
Under review as a conference paper at ICLR 2027

ReCoVR: Closing the Loop in Interactive Composed Video Retrieval

Abstract

Composed video retrieval (CoVR) retrieves a target video from a reference video and a modification text, yet a single query often leaves the target underspecified and existing methods offer no way to refine it. We formalize *interactive composed video retrieval* (ICoVR), a multi-turn extension of CoVR in which users refine their intent through natural-language feedback. Adapting interactive text-to-video retrieval methods to ICoVR exposes two structural weaknesses: they route all feedback through a *single retrieval channel*, and they operate *open-loop*, updating the query from feedback without checking whether their own rankings are drifting or stagnating. After a turn that fails to improve the target rank, the strongest adapted baseline improves on the next turn in only 30.12% of cases. We propose **ReCoVR**, a training-free framework that closes this loop by treating the system's own ranking trajectory as evidence alongside user feedback. An Intent Pathway routes each feedback turn to text-to-video and composed retrieval channels, and a Reflection Pathway accumulates constraints from feedback and uses ranking history to demote rejected or stagnant results. On WebVid-CoVR-Test, ReCoVR reaches 74.30% Recall at 1 (R@1) after one feedback turn and 90.22% after five, and its margin over the strongest baseline widens from 7.2 to 15.2 points. The gains carry over to Dense-WebVid-CoVR and FineCVR, and evaluation with real human feedback shows the same improvement trend.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Composed Video Retrieval, Cross-modal Retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zBTgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Interactive Composed Video Retrieval (ICoVR), extending single-turn composed video retrieval into a multi-turn setting where users progressively refine their search intent through natural-language feedback.

The proposed ReCoVR framework consists of an Intent Pathway that decomposes feedback into text-to-video and composed video retrieval queries, a Reflection Pathway that maintains semantic constraints and demotes rejected or repeatedly retrieved candidates, and Time-Weighted Reciprocal Rank Fusion (TW-RRF), which combines retrieval rankings across turns.

The authors evaluate ReCoVR on three composed video retrieval benchmarks against adapted interactive retrieval baselines, reporting substantial improvements in Recall@K. They additionally provide component ablations, backbone generalization experiments, and a limited human-feedback study.

Critical strengths

1. Consistent empirical improvements across multiple benchmarks.

The full ReCoVR system achieves substantial improvements over the adapted interactive baselines across three CoVR datasets. The component ablations, backbone evaluations, and paired confidence intervals provide evidence that the overall system is effective under the proposed evaluation protocol.

2. Practical training-free integration.

ReCoVR operates with frozen retrieval backbones and introduces no additional retriever training. This makes the framework relatively straightforward to integrate into existing retrieval systems.

Critical weaknesses

1. Limited technical novelty and insufficient differentiation from substantial prior work.

My primary concern is that the paper's central methodological contributions largely consist of straightforward applications and combinations of established techniques, without demonstrating a sufficiently substantive new algorithmic or conceptual insight.

The first contribution, extending single-turn composed video retrieval to a multi-turn setting through interaction history, memory, and feedback, has substantial precedent. Guo et al. (Dialog-based Interactive Image Retrieval, NeurIPS 2018) already investigate iterative visual retrieval using natural-language feedback and simulated users. Yuan and Lam (Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback, SIGIR 2021) explicitly extend composed image retrieval to multiple turns by incorporating the reference image, modification text, and conversation history. Pal et al. (FashionNTM, ICCV 2023) develop persistent memory for multi-turn visual retrieval, while Chen et al. (MAI, ICLR 2025) directly investigate historical semantic aggregation and iterative optimization for multi-turn composed image retrieval.

The distinction that ReCoVR addresses video rather than image retrieval is relevant, but applying an established multi-turn retrieval paradigm to another modality is not, by itself, a compelling methodological contribution. The paper does not sufficiently establish what fundamental new challenge requires its particular formulation beyond the existing history-aware composed retrieval literature.

The second contribution, Time-Weighted Reciprocal Rank Fusion (TW-RRF), also has substantial precedent. Rollings et al. (Multi-Faceted Question Fusion in the TREC 2022 CrisisFACTS Track) already investigate recency-weighted RRF for information retrieval. More directly, Harel et al. (Sentence Retrieval for Open-Ended Dialogue Using Dual Contextual Modeling, ECIR 2023) fuse rankings generated from individual dialogue turns using RRF and explicitly evaluate linear and exponential decay weights. ReCoVR's Eq. (7) follows the same general weighted-RRF construction, with a manually specified linear recency schedule and a sliding temporal window.

Similarly, the Reflection Pathway primarily uses LLM-based constraint accumulation and a simple rejection/repetition heuristic for reranking. Its rank-cap policy (Eq. 5) responds to explicit rejection or persistence of the previous top-ranked result rather than performing a general analysis of retrieval quality or trajectory dynamics.

The individual components are reasonable engineering choices, but I find their algorithmic novelty limited. The paper would need to establish a more distinctive methodological contribution, or demonstrate that the particular combination enables capabilities that established history-aware retrieval and rank-fusion techniques cannot adequately provide.

2. Insufficient evidence for the necessity and incremental significance of the proposed mechanisms.

Although the full system performs well against adapted interactive baselines, the ablations provide considerably weaker evidence that the particular proposed mechanisms are important.

In Appendix A.1, Table 5, TW-RRF achieves 90.22% R@1 at Turn 5 compared with 89.03% for uniform-weight RRF, an improvement of only 1.19 percentage points. This is a relatively small gain for one of the paper's explicitly proposed algorithmic components, particularly given the simplicity of the linear weighting rule and the substantial prior literature on recency-weighted retrieval.

Furthermore, Eq. (6) implies that TW-RRF and uniform-weight RRF assign identical weights when only one feedback turn is available. However, Table 5 reports different Turn-1 R@1 values (74.30% versus 73.63%). This discrepancy cannot be explained by the temporal weighting formula itself under identical channel rankings and suggests either stochastic variation, additional implementation differences, or a reporting inconsistency. Since the final claimed advantage of temporal weighting is only 1.19 points, a properly paired comparison with controlled randomness and confidence intervals is important before attributing the difference to the weighting scheme.

The paper also does not compare against other simple recency functions, such as exponential decay, despite their established use in prior multi-turn retrieval literature. Thus, it is not demonstrated that the particular proposed weighting rule is meaningfully preferable to standard alternatives.

For the Reflection Pathway, Table 3 shows that the Intent Pathway combined with rank-cap already achieves 87.52% Turn-5 R@1 without State Reflection, compared with 90.22% for the complete system. This incremental 2.70-point improvement is positive, but the experiments do not clearly establish that it reflects a distinctive trajectory-level diagnosis mechanism rather than improved semantic query reformulation. The reported recovery statistics are also observational comparisons between different interaction trajectories rather than isolated causal evidence for the reflection mechanism.

There is additionally a potential inconsistency in the claimed prevention of repeated rejected candidates. Demoting the previous top-1 result to rank 11 in both retrieval channels does not guarantee that it cannot remain top-ranked after reciprocal rank fusion, particularly when historical rankings are retained. The claim in Appendix A.4 that repeated rejected results are prevented by construction therefore does not follow directly from Eqs. (5)–(7).

Overall, the experiments convincingly show that combining these components improves the evaluated retrieval pipeline. However, they do not adequately establish the necessity, novelty, or incremental significance of its distinguishing algorithmic choices. Given the extensive prior literature and the modest gains over simpler variants, I do not find the demonstrated contribution sufficiently substantial for acceptance.

Questions for authors

1. What is the substantive methodological novelty relative to existing multi-turn composed retrieval methods?

Prior work including Yuan and Lam (SIGIR 2021), FashionNTM (ICCV 2023), and MAI (ICLR 2025) already investigates maintaining historical visual/textual information and progressively refining retrieval results through natural-language feedback. Beyond transferring this established paradigm to composed videos and combining existing retrieval modules, what specific algorithmic mechanism or previously unsolved technical challenge distinguishes ReCoVR from these methods?

2. How is TW-RRF meaningfully different from established temporally weighted rank fusion?

Rollings et al. (TREC 2022) previously incorporate recency information into RRF, and Harel et al. (ECIR 2023) explicitly fuse per-dialogue-turn rankings using RRF while investigating linear and exponential decay weights. Could the authors clarify the novelty of Eq. (7) relative to this literature and compare the proposed weighting against exponential decay or other simple alternatives under identical retrieval conditions?

3. Why do TW-RRF and uniform-weight RRF produce different results at Turn 1?

With only one feedback turn, Eq. (6) gives a temporal weight of exactly 1, identical to uniform weighting. Nevertheless, Table 5 reports Turn-1 R@1 values of 74.30% and 73.63%, respectively. Could the authors explain this discrepancy and provide a paired evaluation of the two fusion methods with identical channel rankings and controlled randomness? Given that their Turn-5 difference is only 1.19 percentage points, paired confidence intervals would materially affect my assessment of TW-RRF's contribution.

4. What evidence establishes that the Reflection Pathway performs meaningful trajectory-level diagnosis rather than simple feedback-based reranking?

Equation (5) primarily checks explicit rejection or repeated top-1 retrieval. This appears substantially narrower than the claimed diagnosis of retrieval drift and stagnation. Could the authors isolate the improvement in recovery rates attributable specifically to State Reflection, controlling for rank-cap and the other components? A comparison with simpler history-aware retrieval and negative-feedback mechanisms would help establish its independent contribution.

5. How does rank-cap guarantee the prevention of repeated rejected candidates?

Under the reported RRF constant k=60, a candidate appearing at rank 11 in both channels obtains a fused score of 2/(60+11)=0.0282, which can exceed that of a candidate appearing at rank 1 in only one channel, 1/(60+1)=0.0164. Historical rankings can further reinforce previously rejected candidates. How, then, does the stated rank-cap policy guarantee the claim in Appendix A.4 that repeated rejected results are prevented by construction? Is there an additional post-fusion exclusion mechanism not specified in the formulation?

5 comments

We thank the reviewer for the detailed review, and for recognizing the consistent improvements across three benchmarks, the supporting component ablations, backbone evaluations, and paired confidence intervals, as well as the practical value of our training-free integration. We address each point below.

[W1a] The first contribution, extending single-turn composed video retrieval to a multi-turn setting … has substantial precedent. … demonstrate that the particular combination enables capabilities that established history-aware retrieval and rank-fusion techniques cannot adequately provide.

[Q1] … what specific algorithmic mechanism or previously unsolved technical challenge distinguishes ReCoVR from these methods?

What distinguishes ReCoVR from these methods is the closed loop: the system uses the outcome of its own previous retrieval as a signal to correct the next one. This addresses a concrete challenge in interactive retrieval: recovering from a failed turn.

A turn fails when the user gives feedback but the target does not move up. The adapted interactive baselines update the query from the feedback and search again, and for them such a failure tends to repeat. UMIVR shows the same top-1 result again after 70.77% of feedback turns, and it rarely recovers after a failed turn, whereas ReCoVR usually does (Table 8, Appendix A.4):

Active feedback turns (Table 8)UMIVRReCoVR
Target rank does not improve54.25%20.23%
After such a turn: next turn improves the rank30.12%88.15%

ReCoVR handles such a turn in three steps (Sec. 3.2.2). Detect: after each feedback, State Reflection judges whether the user rejected the displayed result, and the system checks whether the same result has stayed at the top across turns. Correct the ranking: in either case, rank-cap moves that result down in both channel lists before fusion. Correct the query: State Reflection also extracts constraints from the feedback, which accumulate in memory and guide later queries.

These rank-level corrections are what separate ReCoVR from the cited works. Those works, also discussed in our Related Work, use a trained model to map the dialogue history to an updated query, which decides what to search for next. To our knowledge, none of the cited methods uses such rank-level signals to correct its ranking across turns, and ReCoVR does so without training, since the CoVR benchmarks contain single-turn triplets only. It also routes the two kinds of composed feedback, edits of the displayed video and direct descriptions of the target, to a CoVR and a T2V channel. We have added a comparison with the cited works (training data, modality, history signal) to Related Work and Appendix H.

Finally, as the reviewer asks, we test whether simpler mechanisms provide the same capability: history-aware rewriting, and negative feedback added to an open-loop baseline, under the same retriever and simulator. Both stay clearly below ReCoVR:

MethodTurn-5 R@1
UMIVR (Table 1)75.04
Full-history rewriting, same VLM (Table 12)77.43
UMIVR + rank-cap (Table 6)83.18
ReCoVR90.22

The closed loop therefore provides a capability, recovering from a failed turn, that history aggregation and negative feedback alone do not provide.

[W1b] … Time-Weighted Reciprocal Rank Fusion (TW-RRF), also has substantial precedent. … ReCoVR's Eq. (7) follows the same general weighted-RRF construction …

[W2a] … TW-RRF achieves 90.22% R@1 at Turn 5 compared with 89.03% for uniform-weight RRF, an improvement of only 1.19 percentage points.

[W2c] The paper also does not compare against other simple recency functions, such as exponential decay …

[Q2] How is TW-RRF meaningfully different from established temporally weighted rank fusion? … compare the proposed weighting against exponential decay or other simple alternatives under identical retrieval conditions?

Within the closed loop (W1a), TW-RRF is the step where the corrections take effect. It fuses two heterogeneous channels across turns, and its inputs are the channel lists after rank-cap, so it turns the loop's corrections into the final ranking. TW-RRF builds on RRF and recency weighting, and we have cited both works in Sec. 3.3; its difference from these works lies in what it fuses. The contributions listed in Sec. 1 are the ICoVR task and this closed-loop framework, of which TW-RRF is one component.

Table 5 separates the parts of this fusion step:

Fusion variant (Table 5)Turn-5 R@1What it removes
TW-RRF (ours)90.22—
Uniform-RRF89.03recency weighting
Static-RRF (current turn only)86.03cross-turn aggregation
SimSum / SimMax83.29 / 82.75rank-level fusion

Each part contributes to the final ranking. The reviewer's question concerns the recency weighting, which we test next.

To test whether the recency effect is reliable, and to compare schedules under identical retrieval conditions as requested, we saved the per-turn channel rankings of the full model and replayed different fusion rules on them. At Turn 5, the interaction up to Turn 4 is kept and only the current-turn fusion rule is replaced, so all rules see exactly the same channel lists (paired bootstrap 95% CI over 2,556 queries):

Turn 5: linear (ours) minusΔR@195% CI
Uniform weights+1.19[0.67, 1.76]
Exponential decay, γ=0.5\gamma=0.5+0.16[−0.16, 0.51]
Exponential decay, γ=0.7\gamma=0.7+0.12[−0.16, 0.43]

(The uniform gap is from Table 5; its CI is from the replay.) Recency weighting gives a consistent gain over uniform weights, and this gain does not depend on the exact schedule: linear and exponential decay perform on par. We have added this paired comparison, including exponential decay, to Appendix A.1. What matters is down-weighting older turns, and we keep the linear form for its simplicity.

[W1c] … Its rank-cap policy (Eq. 5) responds to explicit rejection or persistence of the previous top-ranked result rather than performing a general analysis of retrieval quality or trajectory dynamics.

[W2d] … the Intent Pathway combined with rank-cap already achieves 87.52% Turn-5 R@1 without State Reflection, compared with 90.22% for the complete system. … The reported recovery statistics are also observational comparisons …

[Q4] … Could the authors isolate the improvement in recovery rates attributable specifically to State Reflection, controlling for rank-cap and the other components? A comparison with simpler history-aware retrieval and negative-feedback mechanisms would help …

Within the closed loop (W1a), State Reflection is the step that detects a failed turn, and rank-cap is the step that acts on it. We understand the underlying concern as whether State Reflection adds anything beyond query rewriting and a rejection rule.

First, the requested control is already in Table 3. The row with rank-cap on and State Reflection off is exactly this setting: σt\sigma_t is fixed to neutral, no constraints are accumulated, and rank-cap reacts only to stagnation. Table 3 isolates State Reflection both with and without rank-cap, on two benchmarks:

Turn-5 R@1 (Table 3)WebVid-CoVRFineCVR
Intent Pathway only77.4663.69
+ State Reflection83.2669.40
+ rank-cap87.5276.87
+ State Reflection + rank-cap (full)90.2278.65

State Reflection improves results with and without rank-cap on both benchmarks, and the full model has the best R@1 at every turn. The two mechanisms are complementary: rank-cap corrects the current ranking, while State Reflection shapes later queries through accumulated constraints. Since the system never sees the target, the rule in Eq. (5) acts on the events that are observable in the trajectory: a rejection detected by State Reflection, or a top-1 result that persists across turns.

Second, we traced the variant without State Reflection over the full WebVid-CoVR-Test. Without State Reflection, failed turns become more frequent and recovery drops:

Active feedback turnsFull ReCoVR (Table 8)Without State Reflection (new)
Target rank does not improve20.23%26.33%
After such a turn: next turn improves the rank88.15%74.67%

The controlled comparison is the matched ablation in Table 3; these rates, each measured on its own trajectories, are consistent with it. Third, the simpler alternatives in W1a, full-history rewriting (history-aware, no reflection) and UMIVR + rank-cap (negative feedback, no reflection), both stay below ReCoVR. In matched comparisons, State Reflection therefore contributes beyond rank-cap and beyond query rewriting.

Reviewer_4BewOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces a new task, extending single turn composed video retrivel into a multi turn covr. This framework includes a intent pathway for T2V and CoVR, and a reflection pathway for pastretrievaltrajectories. This close-loop framework outperforms other open loop framework in multi turn tasks.

Critical strengths
  1. This paper introduces a close-loop framework for CoVR. It extends single turn CoVR to multi-turn. The reflection pathway extracts semantic constraints and judege user satisfcation. Such design addresses the limitation of previous open-loop methods.
Critical weaknesses
  1. The gap between simulator and human user. Since the simulator can access targets, the gap between simulator and human users may intruduce disturbance into interaction.
Questions for authors
  1. What is the influence of different base model? Will the findings be consistent when transfering to stronger VLM, since the experiments only uses 8B models.
  2. More experiments on human users are recommended. Human users feedback may contain guesses, misunderstandings or intent shifts.
  3. As shown in Table 2, ReCovr is lower than SOTA at turn 1. Could this framework be combined with stronger single turn methods for a higher beginning point?
  4. Does IVR, Merlin, UMIVR get the reference video for retrieval or only get the prompt text?
5 comments

We thank the reviewer for the positive assessment, and for recognizing that our closed-loop framework extends CoVR to multiple turns, and that the Reflection Pathway, by extracting semantic constraints and judging user satisfaction, addresses the limitations of open-loop methods. We address each point below.

[W1] The gap between simulator and human user. Since the simulator can access targets, the gap between simulator and human users may intruduce disturbance into interaction.

We understand the underlying concern as whether results obtained with a target-aware simulator still hold when real users give the feedback.

First, only the simulator sees the target. It plays a user who knows what they are looking for, which is the commonly used simulated-user setting in interactive retrieval (e.g., IVR, UMIVR). ReCoVR and all baselines never access the target; they receive only the feedback text and their own retrieval history (Sec. 4.1). A simulated user could differ from a real one in two ways: how feedback is phrased, and how complete the user's intent is. The human study below tests the first, and Q2 tests the second.

Second, Table 4 tests the phrasing difference directly. Five participants gave free-form feedback on 100 WebVid-CoVR-Test instances, for up to three turns each. Human feedback is about twice as long as simulated feedback and less template-like. Even so, ReCoVR follows almost the same trajectory under both, and stays well above IVR:

MethodFeedback sourceTurn 1 R@1Turn 2Turn 3Avg. words
IVRsimulator58.058.059.06.1
ReCoVRsimulator63.073.079.06.3
ReCoVRreal humans62.070.076.013.7

The difference between simulated and human feedback is much smaller than the difference between ReCoVR and the baseline. Q2 further tests noisy, incomplete, and less informative feedback.

[Q1] What is the influence of different base model? Will the findings be consistent when transfering to stronger VLM, since the experiments only uses 8B models.

To clarify, the default reasoning VLM in all main results is Qwen3-VL-4B (Sec. 4.1), not an 8B model. Table 16 already varies the base model over two families (Qwen3-VL, InternVL3.5) and two sizes (4B, 8B), with the simulator fixed. All four reasoning VLMs give steady gains over turns, so the findings do not depend on one particular model.

To test a clearly stronger model, as suggested, we ran Qwen3-VL-32B (FP8) as ReCoVR's reasoning VLM, with the simulator unchanged, on the full WebVid-CoVR-Test:

Reasoning VLMTurn 1 R@1Turn 5 R@1Gain (Turn 0 → 5)
Qwen3-VL-4B (paper, Table 1)74.3090.22+35.92
Qwen3-VL-32B (new)74.2689.67+35.37

The findings are consistent: with a much larger reasoning VLM, ReCoVR shows a similar trajectory and a similar closed-loop gain. This suggests that most of the improvement comes from the closed-loop design rather than from model scale. Across the 4B, 8B, and 32B models, Turn-5 R@1 stays within about two points (the 32B model runs in FP8). In practice, the 4B default achieves comparable accuracy at about 1/2.7 of the 32B runtime.

[Q2] More experiments on human users are recommended. Human users feedback may contain guesses, misunderstandings or intent shifts.

We thank the reviewer for this suggestion, and we agree that real feedback is often imprecise. Beyond the human study in W1, we test this in two ways.

First, Appendix D.4 (Table 20) degrades the simulated feedback in a controlled way, on the full test set: surface noise (word- and character-level corruption), constraint dropping (randomly removing part of the stated constraints), and a modify-only policy that never describes the target directly. These cover noisy, incomplete, and less informative feedback. ReCoVR keeps most of its interactive gain in every setting, while UMIVR's gain drops sharply under modify-only:

Feedback settingUMIVR Gain@5ReCoVR Gain@5ReCoVR Turn-5 R@1
Clean (default)20.7435.9290.22
Light surface noise20.3534.7189.01
Heavy surface noise19.7634.4388.73
Mild constraint drop (p=0.3p=0.3)20.4734.9889.28
Severe constraint drop (p=0.5p=0.5)19.5334.7889.08
Modify-only9.7531.8586.15

(Gain@5: R@1 gain from Turn 0 to Turn 5.)

Second, Table 8 reports how each method behaves after a turn that fails to improve the target rank, whatever the cause. In this case, ReCoVR improves the rank on the next turn in 88.15% of cases, versus 30.12% for UMIVR. This is consistent with the design of the Reflection Pathway: State Reflection detects a rejection, and rank-cap demotes the rejected result.

Intent shift, where the user changes the target during a session, is a different setting. It changes the success criterion itself and needs a new evaluation protocol. A new human study of this kind also requires ethics approval, so it cannot be completed within the discussion period. We agree that it is an important direction, and we have added it to the discussion of larger user studies in Appendix H.

ReCoVR's advantage therefore does not depend on perfectly informative feedback.

[Q3] As shown in Table 2, ReCovr is lower than SOTA at turn 1. Could this framework be combined with stronger single turn methods for a higher beginning point?

To clarify the turn indexing: at Turn 1, ReCoVR is already above every single-turn method in Table 2. The value below SOTA is Turn 0, the single-turn starting point before any feedback. Interactive turns use additional feedback, so this is a reference rather than a like-for-like comparison, as the Table 2 caption notes:

MethodR@1
Best single-turn method (CADP)63.62
ReCoVR, Turn 0 (BLIP-Large, no feedback)54.30
ReCoVR, Turn 1 (one round of feedback)74.30

On the second part, yes, and the paper already reports this combination. ReCoVR is training-free and backbone-agnostic, so any single-turn retriever can serve as its retrieval channels. Table 14 shows that a stronger retriever gives both a higher starting point and a higher final accuracy:

CoVR / T2V channelTurn 0Turn 1Turn 5
BLIP-Large (default)54.3074.3090.22
Qwen3VL-Embed8B64.3682.4794.33

With Qwen3VL-Embed8B channels, Turn 0 already exceeds all single-turn methods in Table 2, and the closed loop adds about 30 points on top. ReCoVR is complementary to single-turn CoVR: a stronger single-turn method raises the starting point, and ReCoVR builds on it.

[Q4] Does IVR, Merlin, UMIVR get the reference video for retrieval or only get the prompt text?

Yes, all three baselines use the reference video. Each baseline uses the same CoVR retriever as ReCoVR (Sec. 4.1), and at every turn its query fuses the reference video with its updated text. For this reason, all methods start from the identical Turn-0 ranking in Table 1 (R@1 54.30). We have stated the inputs of each baseline explicitly in Sec. 4.1.

Moreover, Appendix A.6 (Table 10) runs ReCoVR with the same fixed original reference as the baselines, instead of its default anchor (the previous top-1 result). It still clearly outperforms UMIVR:

Method (CoVR anchor)Turn 1 R@1Turn 5 R@1
UMIVR (fixed reference)67.1075.04
ReCoVR (fixed reference)72.7886.41
ReCoVR (previous top-1, default)74.3090.22

The baselines therefore receive the reference video, and ReCoVR's advantage holds when it uses the same fixed reference.

Dear Reviewer 4Bew,

Thank you for your review and the positive suggestions. We have submitted our response and updated the paper, including results with a stronger reasoning VLM and the robustness analysis under noisy and incomplete feedback.

We would appreciate it if you could take a moment to review our changes. We look forward to your feedback and hope that our response satisfactorily resolves your concerns.

Kind Regards,

Authors of Paper 37061

Reviewer_sFRJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Interactive Composed Video Retrieval, a multi-turn extension of composed video retrieval in which users refine an initially underspecified target through natural-language feedback over up to five turns. ReCoVR is a training-free, backbone-agnostic closed-loop framework with three components. On WebVid-CoVR-Test, R@1 reaches 74.30% after one feedback turn and 90.22% after five, with a margin over the strongest baseline widening from 7.2 to 15.2 points.

Critical strengths

ICoVR is a natural formalization of progressive refinement in composed retrieval that has not been previously defined.

The training-free, backbone-agnostic design is practically meaningful: gains hold with weak encoders and heterogeneous channel encoders, indicating that closed-loop correction is complementary to advances in single-turn retrieval.

Critical weaknesses

By default, the simulator and the system share the same reasoning-VLM family, feedback is conditioned on the ground-truth target, and the simulator stops only on exact top-1 identity.

All adapted baselines are question-answering I-TVR pipelines, while recent multi-turn composed retrieval with LMM-based reasoning (e.g., CoCo-IR for composed image retrieval.

Could you adapt a recent multi-turn composed-retrieval system (e.g., CoCo-IR) or a conversational text-to-image retrieval method (e.g., DIR-TIR) to the ICoVR protocol as an additional baseline? Relatedly, please clarify whether the single-turn methods in Table 2 use matched retrieval backbones, since some listed R@1 values appear higher than typical published WebVid-CoVR-Test results.

Questions for authors

Beyond the cross-VLM test in Table 16, have you evaluated a cross-family simulator (e.g., an InternVL3.5-based simulator with Qwen3-VL-4B ReCoVR) to rule out family-level alignment between simulator and system on the absolute per-turn trajectory?

What are the gallery sizes in each benchmark, and how do the caption cache (PM^L, ≈1.66 s/entry offline) and top-K dual-channel retrieval scale to full WebVid-CoVR gallery sizes? Is the bank constructed over the full gallery or per test split?

4 comments

We thank the reviewer for the careful reading, and for recognizing ICoVR as a natural formalization of progressive refinement that has not been defined before, as well as the practical value of our training-free, backbone-agnostic design. We address each point below.

[W1] By default, the simulator and the system share the same reasoning-VLM family, feedback is conditioned on the ground-truth target, and the simulator stops only on exact top-1 identity.

We understand the underlying concern as whether the gains could come from an implicit agreement between the simulator and the system, rather than from the method. We address the three settings in turn.

First, target-conditioned feedback and the exact top-1 stop follow the commonly used simulated-user setting in interactive retrieval (e.g., IVR, UMIVR), and they are applied in the same way to every method. All methods share the same simulator, prompts, targets, initial ranking, and stopping rule, and no method ever accesses the target; only the simulator uses it, to play a user who knows what they are looking for (Sec. 4.1). The exact top-1 stop is a strict criterion applied identically to every method, and we report R@5, R@10, and BRI alongside R@1.

Second, on the shared VLM family, Table 16 shows no sign of a family-alignment advantage. There, the simulator is fixed to Qwen3-VL-4B and only ReCoVR's reasoning VLM changes. If same-family pairing gave a clear advantage, the Qwen3-VL system would be expected to perform best; instead, the cross-family InternVL3.5 system performs at least as well at both model sizes.

Reasoning VLM (simulator: Qwen3-VL-4B)FamilyTurn-5 R@1
Qwen3-VL-4Bsame90.22
InternVL3.5-4Bcross92.64
Qwen3-VL-8Bsame91.88
InternVL3.5-8Bcross93.00

Third, Table 4 replaces the simulator with real users (5 participants, 100 instances). With free-form human feedback, ReCoVR closely follows its trajectory under the simulator and stays far above the IVR baseline:

R@1Turn 1Turn 2Turn 3
IVR, simulator58.058.059.0
ReCoVR, simulator63.073.079.0
ReCoVR, real humans62.070.076.0

Together with the cross-family simulator experiment in Q1, these results indicate that ReCoVR's gains do not depend on the simulator setting.

[W2a] Could you adapt a recent multi-turn composed-retrieval system (e.g., CoCo-IR) or a conversational text-to-image retrieval method (e.g., DIR-TIR) to the ICoVR protocol as an additional baseline?

A comparison with LMM-based multi-turn reasoning is important, and the paper already includes its closest training-free counterpart under ICoVR: full-history rewriting (Appendix A.7), where the same VLM reads the complete dialogue history at every turn and rewrites the composed query. It uses the same retriever, reasoning VLM, and simulator as ReCoVR. It improves over turns but stays clearly below ReCoVR, and the gap widens as the interaction gets longer (Table 12):

R@1Turn 1Turn 2Turn 3Turn 4Turn 5
Full-history rewriting65.5470.5873.2475.0477.43
ReCoVR74.3080.3284.3987.7290.22

Giving the LMM more dialogue history is therefore not enough. What ReCoVR adds is a closed loop, in which the system also reads its own ranking trajectory (what was shown, rejected, or stuck) and corrects the next ranking.

We have added both suggested works to Related Work. Their settings differ from ICoVR in essential ways:

  • CoCo-IR (ECCV 2026) fine-tunes an LMM into an image embedding model, using training data built by an LMM-based data engine. Its model, training data, and data engine are not publicly released, so applying it to ICoVR would mean rebuilding the data engine for composed video and training a new model: a re-creation of our own rather than CoCo-IR itself. In contrast, ReCoVR and the baselines in our comparison require no training.
  • DIR-TIR targets text-to-image retrieval and has no reference input, so it cannot take the composed (reference + modification) query that defines ICoVR. Its code is not publicly released either, so an adapted version would likewise be our own re-creation.

Adapting either method would require new training data or removing the reference video, which would change the task itself. We therefore use full-history rewriting as the closest LMM-based comparison under the ICoVR protocol.

[W2b] … please clarify whether the single-turn methods in Table 2 use matched retrieval backbones, since some listed R@1 values appear higher than typical published WebVid-CoVR-Test results.

To clarify, each single-turn method in Table 2 uses its own backbone and training: the numbers are taken from the original papers, and the higher values come from recent methods such as HUD, CADP, and REFINE, as reported in those papers. Table 2 serves as a reference comparison, and its caption notes that interactive turns use extra feedback. The controlled comparison is Table 1, where all interactive methods share the same retriever and the same Turn-0 ranking.

More importantly, ReCoVR is not a competing single-turn retriever but a framework that is orthogonal to the choice of retriever, so the methods in Table 2 could serve as its retrieval channels. As the reviewer noted in the strengths, the gains hold across encoders. Table 14 confirms this over nine channel configurations, from weak to strong and including mixed encoders:

CoVR / T2V channelTurn 0Turn 5
CLIP-Base / CLIP-Base46.3283.88
BLIP-Large / BLIP-Large (default)54.3090.22
BLIP-Large / LanguageBind (mixed)54.3092.33
Qwen3VL-Embed8B / Qwen3VL-Embed8B (strongest)64.3694.33

In every setting, ReCoVR brings a large gain from Turn 0 to Turn 5, and a stronger retriever leads to a higher final accuracy. Progress in single-turn retrieval and ReCoVR's closed-loop correction are complementary.

[Q1] … have you evaluated a cross-family simulator (e.g., an InternVL3.5-based simulator with Qwen3-VL-4B ReCoVR) to rule out family-level alignment between simulator and system …?

This setting tests Table 16 from the other direction, and we ran it as suggested: an InternVL3.5-8B simulator with the default Qwen3-VL-4B ReCoVR, together with UMIVR under the same simulator, on the full WebVid-CoVR-Test.

ReCoVR's trajectory is essentially unchanged when the simulator family changes:

SimulatorTurn 1 R@1Turn 5 R@1
Qwen3-VL-4B (paper, Table 1)74.3090.22
InternVL3.5-8B (new)74.2690.38

Under the InternVL simulator, ReCoVR also outperforms UMIVR at every turn, and the margin grows as the interaction proceeds. All paired bootstrap 95% CIs (2,556 queries) exclude zero:

ReCoVR − UMIVRTurn 1Turn 2Turn 3Turn 4Turn 5
Δ R@1+6.85+7.63+9.00+11.07+11.89
95% CI[5.28, 8.45][6.06, 9.15][7.47, 10.53][9.51, 12.60][10.45, 13.38]

The Turn-5 margin differs from the default setting (11.89 vs. 15.18) because UMIVR scores higher under the InternVL simulator, while ReCoVR's own trajectory is essentially unchanged; the margin remains large and widens at every turn.

Notably, the InternVL simulator gives a clearly different style of feedback: it issues small relative edits much less often than the Qwen simulator (20.8% vs. 37.6% of its non-stop actions) and rewrites the target description more often. ReCoVR's trajectory is similar under both styles, which is consistent with the Intent Pathway routing relative edits and full descriptions to different retrieval channels. Together with Table 16, changing the model family on either side leaves the conclusions unchanged, and we find no evidence that family-level alignment drives the gains.

[Q2] What are the gallery sizes in each benchmark, and how do the caption cache (PML\mathit{PM}^L …) and top-KK dual-channel retrieval scale to full WebVid-CoVR gallery sizes? Is the bank constructed over the full gallery or per test split?

Each gallery consists of all target videos in the test split, and PML\mathit{PM}^L is built over this test gallery, following common practice in CoVR evaluation. We have added the gallery sizes to Sec. 4.1:

BenchmarkQueriesGallery
WebVid-CoVR-Test2,5562,556
Dense-WebVid-CoVR-Test2,556same videos as WebVid-CoVR-Test
FineCVR-Test10,0436,375

FineCVR's gallery is about 2.5× larger than WebVid-CoVR's, and ReCoVR keeps its advantage there as well (Fig. 3).

Regarding scalability, the cost of ReCoVR is dominated by the VLM modules, not by the gallery (Appendix C, Tables 18–19). The caption cache is a one-off offline step that grows linearly with the gallery and is fully parallel. Online, two-channel retrieval takes only a small fraction of each turn, and TW-RRF fuses at most 1,000 candidates regardless of gallery size:

StageCostHow it scales
Caption cache (offline, once)1.658 s / videolinear in gallery size; fully parallel
Two-channel retrieval (online)0.028 s / turngrows with gallery size; small share of each turn
TW-RRF fusion (online)0.001 s / turn≤ 1,000 candidates; independent of gallery size

The per-turn cost is therefore set mainly by the online VLM modules, whose cost does not depend on the gallery size.

Dear Reviewer sFRJ,

Thank you for your review and the constructive suggestions. We have submitted our response and updated the paper, including the cross-family simulator results, gallery sizes, and the discussion of CoCo-IR and DIR-TIR.

We would appreciate it if you could take a moment to review our changes. We look forward to your feedback and hope that our response satisfactorily resolves your concerns.

Kind Regards,

Authors of Paper 37061

Comments on the paper

1 comment

We thank all reviewers for recognizing ICoVR as a natural, previously undefined formalization (sFRJ), the training-free, backbone-agnostic design (sFRJ, zBTg), the closed loop addressing the limitations of open-loop methods (4Bew), and consistent improvements across three benchmarks (zBTg). Revisions in the uploaded PDF are marked in blue.


1. Evaluation protocol (in response to Reviewer sFRJ - W1, Q1; Reviewer 4Bew - W1, Q2)

  • [Add] Cross-family simulator results: Sec. 4.4 (Simulator family) and Appendix B.
  • [Add] Qwen3-VL-32B reasoning VLM: Sec. 4.3 (Generalization) and Appendix B.
  • [Add] Human-feedback and modify-only results: Sec. 4.4.
  • [Add] Intent shift: Appendix H and Sec. 5.

2. Baselines and benchmark details (in response to Reviewer sFRJ - W2a, W2b, Q2; Reviewer 4Bew - Q3, Q4)

  • [Add] CoCo-IR and DIR-TIR: Sec. 2.
  • [Add] Gallery sizes and baseline inputs: Sec. 4.1.
  • [Add] Source of the single-turn results in Table 2: Sec. 4.2.
  • [Add] Fixed-reference comparison (Table 10): Sec. 4.3.
  • [Add] Cost versus gallery size: Sec. 4.4.

3. Relation to prior work (in response to Reviewer zBTg - W1a, W1b, Q1, Q2)

  • [Add] History-aware multi-turn retrieval: Sec. 2 and Appendix H.
  • [Add] Recency-weighted rank fusion: Sec. 3.3.
  • [Add] Full-history rewriting (Table 12): Sec. 4.3.

4. Fusion analysis (in response to Reviewer zBTg - W2a, W2b, W2c, Q2, Q3)

  • [Fix] Lower index above Eq. (6): t0=max⁡(0,t−W+1)t_0=\max(0, t-W+1).
  • [Add] Paired replay with exponential decay: Appendix A.1 and Sec. 4.3.

5. Rank-cap (in response to Reviewer zBTg - W2e, Q5)

  • [Fix] "Prevented by design" replaced with measured repeat rates: Appendix A.4 and Table 8.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.