acceptodds
Under review as a conference paper at ICLR 2027

Modeling Retrieval Intent from Video-Text Context for Composed Video Retrieval

Abstract

Composed Video Retrieval (CVR) retrieves a target video by composing a reference video with a text modification describing desired changes. Effective CVR requires resolving the user's retrieval intent from joint video-text context: what to modify and what to preserve. However, existing methods merely perform task-specific refinement within each modality, underexploring how video-text interaction determines user's retrieval intent. This leads to two critical weaknesses: 1) imprecise modification, where changes are not confined to relevant visual regions, and 2) indiscriminate preservation, where insignificant video content is fully retained. To address this, we propose IC-CoVR, an intent-centric framework that resolves the retrieval intent within the joint video-text context. Specifically, we characterize retrieval intent along two complementary dimensions: explicit intent for targeted modification and implicit intent for selective preservation. Based on this formulation, IC-CoVR first constructs structured video representations to support video-text interaction and intent reasoning, then uses explicit intent to locate and guide targeted modification, and finally leverages implicit intent to selectively preserve unmodified content by suppressing irrelevant details. Extensive experiments on FineCVR-1M and EgoCVR show that IC-CoVR achieves state-of-the-art performance and strong generalization, with a substantial gain of +9.22% R@1 on FineCVR-1M. Comprehensive ablation studies further validate the effectiveness of our intent-centric formulation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
composed video retrieval, multimodal retrieval, video-text interaction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_z56yOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a retrieval intent modeling method for CoVR task. The method achieves more precise modification and selective preservation through component-by-granularity CSP representation, explicit-intent-guided localized modification, and implicit-intent-guided granularity selection, raising R@1 from 25.84 to 35.06 on FineCVR-1M.

Critical strengths
  1. The paper is well-written and easy to understand.
  2. Implicit intent is indeed crucial for CoVR, and the method is well-motivated.
  3. Experimental results validate the effectiveness of the proposed method.
Critical weaknesses
  1. There is a lack of comparison with recent state-of-the-art methods, such as HUD (Chen et al., 2025), REFINE (Hu et al., 2026), and ReTrack (Li et al., 2026).
  2. Why does CSP utilize Qwen 3-VL, whereas EMS employs the clustering method from FDCA to decompose display intent? Would using Qwen 3-VL yield better results?
  3. The paper interprets the granularity attention produced by IGAM as implicit retrieval intent. However, since no explicit supervision or ground-truth annotation of implicit intent is provided, it remains unclear whether these attention weights genuinely capture user intent or merely reflect retrieval-optimized granularity weighting. Could the authors provide stronger quantitative or human-aligned evidence validating this interpretation?
Questions for authors

See the weaknesses above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer z56y: Additional Comparisons and Intent Validation

Thank you for your constructive review. Below, we add comparisons with REFINE and ReTrack, evaluate Qwen3-VL for component selection, and provide human-aligned and query-dependent evidence for IGAM. In the revised PDF, the changes discussed below are marked in blue.

w1:Comparisons with HUD, REFINE, and ReTrack.

HUD is already included in Table 1 of our submitted manuscript. We additionally evaluate REFINE and ReTrack using their official implementations, training on the FineCVR-1M training split and evaluating on the same test split as IC-CoVR.

MethodR@1R@5R@10R@50
HUD (Chen et al., 2025; Table 1)26.8555.9270.2894.28
REFINE (Hu et al., 2026)26.1755.3270.3194.34
ReTrack (Li et al., 2026)27.3256.0171.6494.49
IC-CoVR35.0659.1272.0494.58

We have added the REFINE and ReTrack results to Table 1 (p. 7, lines 331–332) and included their references.

w2:Why use FDCA-based parsing in EMS rather than Qwen3-VL?

In our framework, Qwen3-VL constructs query-independent CSP representations, whereas EMS makes query-specific modification decisions. While Qwen can also select the component to modify, EMS additionally needs to align the operation intent with that component and produce retrieval-relevant features, motivating an integrated framework optimized under retrieval supervision.

We directly compared the two choices by replacing BIP’s component selection with a component index predicted by Qwen3-VL, keeping the remaining pipeline unchanged. Results evaluated after EMS and before IPS are:

Component selectionR@1R@5R@10R@50
BIP (FDCA-based disentanglement)31.5859.4272.0494.58
Qwen3-VL29.0758.9571.5094.81

Qwen-based selection reduces R@1 by 2.51 percentage points. Thus, while Qwen offers a feasible alternative for component selection, BIP achieves higher top-1 retrieval accuracy in this comparison.

w3:Quantitative and human-aligned evidence that IGAM’s weights capture implicit retrieval intent.

IGAM’s granularity weights are optimized for retrieval without intent labels. We test whether this “retrieval-optimized granularity weighting” also reflects human judgments of what to preserve and changes with the query context.

Five annotators rated how much detail each CSP component needed to preserve for retrieval across 100 triplets, using a 1–3 scale in increments of 0.5, where higher scores indicate finer granularity. Of these, 80 triplets were rated by one annotator each, and 20 by all five, using the median per component as the human score. The model score is wc+2wm+3wfw_c+2w_m+3w_f, rounded to the nearest 0.5, where wcw_c, wmw_m, and wfw_f are the coarse, middle, and fine weights. Across all 300 component-level comparisons, agreement with human judgments reaches Spearman’s ρ=0.768\rho=0.768 and quadratic-weighted Cohen’s κ=0.727\kappa=0.727. Inter-annotator Fleiss’ κ\kappa on the 20 jointly rated triplets is 0.652.

Additionally, with the reference video and CSP held fixed, changing the modification text yields a video-averaged variation between granularity-weight distributions of 0.1173, measured across 8,227 text pairs sharing 802 reference videos. This shows that IGAM adjusts its granularity preferences to the query context.

Together, these results support interpreting IGAM as modeling context-dependent, human-aligned preservation granularity, our operational definition of implicit intent. We have added both validations to Appendix D.2 (p. 23, lines 1211–1226).

We welcome any further questions and would be happy to discuss the method and additional results in more detail.

Thank you for your rebuttal, all my concerns have been addressed. So I keep my score.

AuthorsOpenReview ↗

Thank You for the Constructive Review and Future Directions

Thank you for your careful review and constructive feedback. We are glad that our responses have addressed your concerns. Your suggestions strengthened our comparisons and intent validation, while the proposed Qwen3-VL alternative also highlighted a valuable direction for future intent parsing.

Reviewer_jBDvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a method for composed video retrieval. The authors model retrieval intent from video-text context and seperate it into explicit and implicit intent. They use explicit intent to locate and modify relevant video components, while implicit intent controls which visual details to preserve. The method uses MLLM-generated component-structured profiles and a three-stage framework for retrieval. The authors evaluate on FineCVR-1M, EgoCVR, and WebVid-CoVR. The results show improvements on FineCVR-1M and good performance on several other retrieval settings.

Critical strengths
  1. The authors address an interesting problem in composed video retrieval. Deciding what to modify and what to preserve is important for this task. The motivation is clear and the intent-based formulation makes sense.

  2. The method achieves good performance on FineCVR-1M. It improves R@1 by 9.22 points over the CLIP baseline, although the main comparison uses different visual backbones. The authors also show clear improvements when using the same backbone.

  3. The authors provide quite detailed ablation studies and additional analysis. They test different components, loss functions, MLLM backbones, and granularity settings. The results give useful evidence for the effectiveness of the proposed framework.

Critical weaknesses
  1. The authors seperate retrieval intent into explicit and implicit intent, but the two parts mainly rely on different attention and feature fusion operations. The method does not provide direct supervision to distinguish these two types of intent. It is unclear whether the model actually learns meaningful retrieval intents or just better feature representations.

  2. The Bidirectional Intent Parser assumes that the text features more similar to the reference video represent the modification region, and the less similar ones represent the operation. This assumption may fail for changes involving motion or actions that already exist in the video. The authors do not properly explain why global visual similarity can seperate the two intents.

  3. The authors divide every video into only character, object, and background components. This fixed structure may struggle with videos containing multiple people or objects that need different modifications. The method does not explicitly model individual object instances or complex relations between them.

  4. The implicit intent module uses attention weights to suppress less relevant visual details. But details not in the modification text may still be important for preserving the reference video identity. The method does not clearly explain how it avoids removing important but unmentioned information.

  5. The authors first retrieve a small candidate set using modified visual features and then apply the implicit intent module for reranking. If the correct target is missing from the initial candidate set, the second stage cannot recover it. This creates a strong dependence on the first-stage retrieval quality and limits the effectiveness of the preservation mechanism.

Questions for authors

Questions in the weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer jBDv (Part 1): Intent Modeling, BIP Design, and Multi-Instance Limitations

Thank you for your thoughtful review. Below, we clarify intent supervision and BIP’s design, validate the learned localization, modification, and preservation decisions, and discuss limitations in multi-instance editing and candidate recall. In the revised PDF, the changes discussed below are marked in blue.

w1:Without intent labels, does the model learn meaningful intents or merely better representations?

Explicit and implicit intent refer to retrieval requirements: what should change and what detail should be preserved. We translate these requirements into structured decisions: EMS learns where and how to modify, while IPS learns at what granularity to preserve. Retrieval losses supervise these decisions using target videos, without separate intent labels. We have clarified this formulation in Section 3.1 (p. 3, lines 150–152).

We directly evaluate these decisions:

  • Query-dependent localization. We map dataset-annotated query types, grouped as in Figure 9, to CSP groups: background/static-object changes (types 0–1) to background, and action changes (types 2–3), typically involving people or moving/interacting objects, to character/object. For text pairs sharing a reference video, component selection switches between background and character/object 17.91 percentage points more often when the texts belong to different query-type groups than to the same group. This comparison averages equally over 70 reference videos containing both kinds of text pairs. Across the full test set, 71.26% of queries select a component consistent with this mapping (69.98% balanced accuracy).
  • Target-directed modification. We rank gallery videos by comparing only the selected query CSP component with the corresponding component of each gallery video. Modification improves the target’s mean rank from 1407.25 to 1013.27 and its cosine margin over the strongest negative from −0.1769 to −0.1070. This supports that EMS transforms reference content toward the target relative to competing videos, as intended.
  • Meaningful preservation preferences. IPS granularity preferences agree with human judgments, as detailed in our response to W4 in Part 2 below.

These results link localization, modification, and preservation decisions to our intent formulation beyond retrieval gains.

w2:Can visual similarity distinguish region and operation intent when the relevant action already exists?

The situation involving “actions that already exist in the video” is covered by the similar-action category in FineCVR-1M. And our BIP module adopts FDCA’s text disentanglement, whose token features incorporate intra-sentence relationships. Similarity to the reference video accounts for both token semantics and sentence context. Thus, a concept related to an existing action can still contribute to operation intent when it describes the requested change.

Under the similar-action setting, Figure 9 shows an R@1 improvement over FDCA from 23.7% to 33.2%. The resulting performance is comparable to that on most other query types, supporting effectiveness in this setting.

w3:Limitations of fixed components for multiple instances and complex relations.

Modifying multiple people or objects separately remains underexplored in CVR. Current CVR benchmarks lack dedicated evaluation of this setting, and our character–object–background structure offers limited support for it. Our component-based architecture could be extended by separating individual instances and adding relational representations to support finer-grained modification and preservation. We have added this limitation and these future directions to Appendix C.6 (p. 22, lines 1181–1185).

AuthorsOpenReview ↗

Response to Reviewer jBDv (Part 2): Preservation and Candidate Recall

w4:Risk of suppressing important information unmentioned in the modification text.

Preservation is supervised by correspondence with the target video, including information unmentioned in the modification text. The CSP score includes similarities between corresponding query and candidate components (Eq. 4), and the CSP and synergy losses optimize target discrimination using this score (Eqs. 5–6). If suppressing an unmentioned detail weakens the target’s score relative to competing candidates, these losses penalize that decision.

Two results support this mechanism. First, removing LcspL_{\mathrm{csp}}, which directly supervises alignment between text-modulated query and candidate CSP representations, reduces R@1 from 35.06 to 31.94 (−3.12 points, Table 6).

Second, five annotators rated how much detail each CSP component needed to preserve for retrieval across 100 triplets, using a 1–3 scale in increments of 0.5, where higher scores indicate finer granularity. Of these, 80 triplets were rated by one annotator each, and 20 by all five, using the median per component as the human score. The model score is wc+2wm+3wfw_c+2w_m+3w_f, rounded to the nearest 0.5, where wcw_c, wmw_m, and wfw_f are the coarse, middle, and fine weights. Across 300 component-level comparisons, model scores agree with human judgments (Spearman’s ρ=0.768\rho=0.768; quadratic-weighted Cohen’s κ=0.727\kappa=0.727). Inter-annotator Fleiss’ κ\kappa on the 20 jointly rated triplets is 0.652.

Together, these results support preservation learned through target correspondence and consistent with human granularity judgments. They do not guarantee that every important unmentioned detail is retained. We have clarified the supervision in Section 3.5 (p. 6, lines 290–293) and added the human evaluation to Appendix D.2 (p. 23, lines 1211–1218).

w5:Dependence on first-stage recall and inability to recover missed targets.

We agree that IPS depends on first-stage recall. Its role is to improve preservation-aware discrimination among modification-aware candidates, so missed targets remain a limitation. We have stated this limitation in Appendix C.6 (p. 22, lines 1185–1186).

The target appears among EMS’s top-10 candidates for 72.04% of queries. With this candidate set, IPS improves R@1 from 31.58 to 35.06 (+3.48 points, Table 3). Varying KK from 5 to 30 keeps final R@1 around 35.0–35.1 (Table 7), indicating that simply expanding the candidate set does not improve precise retrieval in this range. This supports our coverage–computation trade-off.

We would be happy to discuss any remaining questions or concerns. If our clarifications and additional evidence address your concerns, we would be grateful if you could reconsider your assessment and score.

Thanks for the respnse. I still have several concerns about the core method.

  1. The authors use a predefined mapping between query types and CSP components to validate localization. But this mapping is not real ground truth. The reported consistency doesnt prove that the model selects the correct region. Also, changing component selections across queries doesnt mean these changes are correct. Can the authors compare their predictions with human annotated regions? Without this, the localization claim is still not convincing.

  2. The authors show better retrieval results on similar-action queries. But this doesnt explain why the more similar text feature should represent region intent. Both the region and operation may already exist in the reference video. The authors still dont show that BIP can correctly seperate these two intents. What happens if they swap the two intent assignments? What happens if they randomly assign them? These are important tests for the main assumption.

  3. The human agreement results only show that the granularity weights roughl match human preferences. This doesnt can prove that the method preserves the correct information. Some of the details may not appear in the modification text but still matter for finding the right target. Can the authors test hard negative cases where such details are important? How often does the method wrongly remove these details? I believe that the current results dont really answer this concern.

  4. The authors admit that the second stage cannot recover targets missing from the initial candidate set. But they dont report how often this happens. Can the authors provide the actual target coverage of the first stage? Also, Table 7 shows almost no improvement in top-one recall when the candidate set gets larger. The top-five recall even drops. Why does the preservation module perform worse with more candidates? This makes its actual contribution still not very clear for me at least.

Overall, I appreciate these additional experiments in such a short time period, but they mainly show better retrieval performance. They still dont fully validate the claimed intent modeling mechanism. After considering other reviewers comments so far, I therefore keep my original rating unchanged for now.

Reviewer_qNJgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes IC-CoVR, an intent-centric framework for Composed Video Retrieval (CVR). The method decomposes retrieval intent into explicit intent for targeted modification and implicit intent for selective preservation. It consists of three stages: (1) constructing component-structured video representations at multiple granularities, (2) performing intent-guided modification of relevant visual features, and (3) selectively preserving information through adaptive granularity alignment. Experiments on FineCVR-1M and EgoCVR demonstrate improved retrieval performance over several existing methods, with ablation studies evaluating the contributions of different components.

Critical strengths
  1. The proposed IC-CoVR achieves substantial improvements on FineCVR-1M, particularly in R@1, and demonstrates competitive results on EgoCVR. These results suggest that the proposed framework is effective for composed video retrieval.

  2. The paper provides extensive ablation studies on the main components, granularity alignment, training objectives, and backbone choices, offering useful evidence for the effectiveness of the overall framework.

Critical weaknesses
  1. The proposed framework mainly combines existing techniques, including textual feature disentanglement from FDCA, cross-attention-based feature modification, and adaptive feature fusion. Although the explicit/implicit intent formulation is intuitive, the technical distinction between the proposed intent modeling and conventional attention-based feature selection remains unclear.

  2. The experiments primarily demonstrate retrieval improvements rather than directly validating the proposed intent-modeling mechanisms. In particular, the granularity alignment ablations do not establish that the model actually captures implicit intent rather than simply benefiting from richer MLLM-generated descriptions and adaptive feature fusion. Moreover, the reported +9.22 R@1 improvement over FDCA involves different CLIP backbones (ViT-L/14 vs. RN50x4), and the method underperforms existing approaches on EgoCVR Global R@1, weakening the claimed strong generalization.

Questions for authors
  1. How does the proposed implicit intent modeling fundamentally differ from conventional query-conditioned attention or feature fusion? Can the authors provide more direct evidence that IGAM identifies which unmodified content should be preserved or suppressed?

  2. Can the authors compare IC-CoVR against a simpler, parameter-matched attention-based fusion or reranking baseline using the same CSP representations? This would help isolate the contribution of intent modeling from that of richer semantic descriptions.

  3. Given the lower Global R@1 on EgoCVR compared with BSE-CoVR and TFR-CVR, could the authors explain the limitations of the proposed method in this setting?

3 comments
AuthorsOpenReview ↗

Response to Reviewer qNJg: Intent Modeling and Experimental Validation

Thank you for your review. We clarify intent modeling, provide additional experiments, and address backbone fairness and EgoCVR. PDF revisions are marked in blue.

w1&q1:Distinction from conventional attention/fusion and direct evidence for implicit intent modeling.

Our distinction from conventional query-conditioned attention or feature fusion lies in how information is used to make modification and preservation decisions. FDCA uses disentangled text for cross-modal alignment and fusion; our BIP uses it to derive region intent and operation intent, specifying where and how to modify. EMS then confines modification to selected components.

IPS controls preservation granularity, allowing unmodified components to remain represented while selectively downweighting fine details. We directly evaluate these preservation decisions through two complementary validations (Appendix D.2, p. 23, lines 1211–1226).

First, five annotators rated how much detail each CSP component needed to preserve for retrieval across 100 triplets, using a 1–3 scale in increments of 0.5, where higher scores indicate finer granularity. Of these, 80 triplets were rated by one annotator each, and 20 by all five, using the median per component as the human score. The model score is wc+2wm+3wfw_c+2w_m+3w_f, rounded to the nearest 0.5, where wcw_c, wmw_m, and wfw_f are the coarse, middle, and fine weights. Across 200 comparisons on components left unmodified by EMS (out of 300 components in total), model scores agree with human judgments (Spearman’s ρ=0.779\rho=0.779 vs. 0.768 overall; quadratic-weighted Cohen’s κ=0.730\kappa=0.730 vs. 0.727 overall). Inter-annotator Fleiss’ κ\kappa on the 40 unmodified components in the 20 jointly rated triplets is 0.634 (vs. 0.652 for all jointly rated components).

Second, with the reference video and CSP fixed, IGAM adjusts the relative emphasis on coarse, middle, and fine information as the modification text changes. Across 8,227 text pairs sharing 802 reference videos, the video-averaged variation between granularity-weight distributions is 0.1173, while identical-text repetitions produce no change.

Together, these results support context-dependent granularity preferences, providing direct evidence beyond retrieval improvements alone.

w1&q2:Parameter-matched attention baselines using the same CSP representations.

We replace EMS and IPS separately with standard cross-attention alternatives using the same CSP representations, matching each replaced module’s trainable parameter count. The EMS alternative uses text features as queries and visual/CSP features as memory. The IPS alternative uses text features as queries over all CSP granularity tokens for both query and candidate videos. Both replacement models are fine-tuned end to end using the same training procedure as IC-CoVR.

Evaluation stageMethodR@1R@5R@10R@50
After EMS, before IPSCross-attention fusion26.0755.5070.0894.08
After EMS, before IPSIC-CoVR31.5859.4272.0494.58
After IPSCross-attention reranker32.7757.7572.0494.58
After IPSIC-CoVR35.0659.1272.0494.58

Generic cross-attention yields 5.51-point lower R@1 at EMS and 2.29-point lower R@1 at IPS. These results support the benefits of structured decisions beyond richer CSP descriptions and comparable network capacity. We have added these comparisons to Appendix D.1, Table 15 (p. 23, lines 1192–1208).

w2&q3:Backbone fairness and limitations underlying the lower EgoCVR Global R@1.

Following CoVR (Ventura et al., 2024a), most CVR methods report both CLIP ViT-L/14 and BLIP variants, and we follow this convention. For a backbone-matched comparison with FDCA, our submitted manuscript also includes an RN50x4 variant in Appendix A.3, Table 9. IC-CoVR achieves 33.62 R@1 versus FDCA’s 25.84, retaining a substantial gain of 7.78 percentage points with the same backbone.

We agree that “strong generalization” needs qualification. EgoCVR is an action-centric benchmark with strong temporal dynamics. Egocentric camera motion and viewpoint changes can also hinder consistent localization across frames. Although our cross-frame aggregation and CSP action descriptions improve performance on dynamic queries in FineCVR-1M (Appendix A.5), they may still miss fine-grained temporal dependencies and action evolution. These challenges may hinder precise top-1 discrimination among competing videos in global retrieval, despite improvements in Global R@5/R@10 and all Local metrics. We have qualified this discussion in Section 4.2 (p. 7, lines 366–373).

We welcome further questions and discussion. If this response addresses your concerns, we would appreciate reconsideration of your assessment and score.

Thanks for the additional experiments, which address several of my concerns and make the proposed design more convincing. However, I still see the method mainly as a combination of existing techniques, and the distinction between intent modeling and conventional attention-based weighting remains unclear. I’m therefore still not convinced that the contribution is strong enough for acceptance and will keep my original rating.

AuthorsOpenReview ↗

Clarification of Remaining Concerns on Intent Modeling

Thank you for the continued discussion. We would like to better understand and address your remaining concerns. Could you specify which aspects of the distinction between our intent modeling and conventional attention-based weighting remain insufficiently addressed? In particular, does your concern primarily relate to the intent formulation itself, its architectural implementation, or both? This would help us focus the discussion and provide more targeted clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.