acceptodds
Under review as a conference paper at ICLR 2027

UMVR: A Unified and Efficient MLLM Framework for Short- and Long-Video Retrieval

Abstract

Text-based video retrieval has largely been studied under separate paradigms for short and long videos, despite sharing the same objective of measuring text-video relevance. A key obstacle to unifying these paradigms is that representation granularity is typically predefined for each retrieval setting rather than adapted to the temporal structure of individual videos. To address this limitation, we propose Length-Adaptive Event Representation (LAER), which adaptively presents each video into temporally coherent events whose number and temporal extent adapt to its content. This event-centric representation supports both localized and holistic relevance modeling, providing a common basis for short- and long-video retrieval. Building on LAER, we introduce UMVR, a unified MLLM-based retrieval framework that uses a shared backbone and scoring formulation across both settings. However, encoding videos with an MLLM incurs substantial computational overhead as the number of visual tokens grows, particularly for long videos. To improve encoding efficiency, we further introduce Event-Aware Token Reduction (EATR), which uses the event structure provided by LAER to allocate visual tokens according to event informativeness, retaining more tokens for informative events while compressing redundant ones more aggressively. Extensive experiments across seven short- and long-video retrieval benchmarks demonstrate the effectiveness and efficiency of UMVR, which maintains strong retrieval performance while reducing the visual-token load for MLLM encoding by 90%.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multimodal Large Language Models, Text-based-Video Retrieval, Unified Video Retrieval

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_EzpHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes UMVR, an MLLM-based framework for text-based retrieval of both short and long videos. It first adaptively partitions videos into events using frame-level similarity and local visual feature changes. Event-Aware Token Reduction (EATR) then assigns different token budgets according to event informativeness, using FlashVID for compression. A shared MLLM produces event-level and global video representations, whose matching scores are combined for retrieval. Evaluation spans seven retrieval benchmarks and includes controlled comparisons, component ablations, and efficiency and transfer analyses. Results show that UMVR outperforms the controlled baseline while retaining approximately 10% of visual tokens and reducing post-vision-encoding processing time.

Critical strengths

The method offers a useful balance between retrieval accuracy and encoding cost. It divides videos into events, assigns more tokens to informative events, and combines local and global matching. On ActivityNet, it achieves 23.7% R@1 and 60.1% R@10, compared with 20.1% and 56.8% for VLM2Vec-V2 using the same backbone and training data. It also retains only about 10% of visual tokens and reduces processing time after visual encoding from 296.9 ms to 119.7 ms.

Critical weaknesses
  1. The paper’s novelty is unclear in light of prior work. The motivation presents short- and long-video retrieval as separate paradigms, but earlier methods already accommodate local and global relevance within a common framework. For example, Attentive Snippet Prompting for Video Retrieval combines text-guided snippet representations with global and local video representations and supports both short and long videos. X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval and Unified Coarse-to-Fine Alignment for Video-Text Retrieval also study matching at multiple granularities. Moreover, Uneven Event Modeling for Partially Relevant Video Retrieval already explores non-uniform event representations. These methods are not identical to UMVR, but they make its broad motivation around local–global modeling and adaptive granularity less distinctive. The paper should explain and evaluate what its event partitioning and token allocation add beyond these existing approaches.
  2. The experiments do not clearly show where the gains come from. Table 1 mainly compares general-purpose MLLM embedding models, while closely related local–global and event-based video retrievers are missing. The same-backbone VLM2Vec-V2 comparison is useful, but this baseline only uses a global representation. A stronger comparison would add local–global matching to the same backbone and test whether adaptive event partitioning still helps. The efficiency results also need comparisons at the same actual token count and full encoding times, including visual encoding and event partitioning. Without these controls, it is difficult to judge how much the proposed designs improve accuracy and efficiency.
Questions for authors
  1. How are short and long videos defined? What duration threshold separates them? Is the distinction based on video length or dataset-specific retrieval settings? For a mixed gallery, how would the model decide whether to sample 64 or 128 frames?
  2. Why are closely related video retrieval methods missing from the comparisons? Table 1 mainly includes general-purpose MLLM embedding models. Can the authors compare against local–global and event-based retrievers, and add a same-backbone baseline with uniform temporal segments and local–global matching? This would help isolate the benefit of adaptive event partitioning.
  3. Why is Qwen2-VL-2B the only backbone evaluated? Do the retrieval gains and compression benefits still hold with a more recent, stronger MLLM?
  4. How much of the efficiency benefit comes from EATR rather than FlashVID? Can the authors compare standard FlashVID, uniform budgeting, and event-adaptive budgeting within the same retrieval model at matched actual token counts? Please also report full encoding times, including visual encoding and event partitioning.
  5. Does aggressive compression preserve temporal information? Can the authors compare full-token and compressed versions of the same model on queries requiring action order, motion direction, or brief events? Overall retrieval scores may not reveal the loss of these cues.
  6. What justifies the event-budget allocation? Why should feature norms indicate how many tokens an event needs? How does the method avoid over-compressing visually simple but important events? Also, since events have different lengths, how does the normalization ensure a fixed total token budget?
4 comments
AuthorsOpenReview ↗

Response to Reviewer EzpH (Part 1)

We sincerely thank the reviewer for the valuable comments and constructive suggestions, which have helped us strengthen the experimental analysis and further improve the clarity and completeness of our manuscript.

Q1: How are short- and long-video retrieval defined, and how is frame sampling handled in a mixed video gallery?

Response to Q1: We would like to clarify that short and long videos are distinguished based on dataset-specific setting. Datasets with a mean duration of 60 seconds or less are treated as short-video benchmarks, and those with a mean duration above 60 seconds are treated as long-video benchmarks. Specifically, MSR-VTT (mean duration of $\sim$15s), VATEX ($\sim$10s), MSVD ($\sim$10s), and DiDeMo ($\sim$50s) are evaluated as short-video retrieval benchmarks, while ActivityNet ($\sim$118s), QVHighlights ($\sim$150s), and TVR ($\sim$76s) are evaluated as long-video retrieval benchmarks.

For a mixed gallery, we can adopt a 60-second threshold to separate videos. We sample up to 64 frames for videos of 60 seconds or less and up to 128 frames for longer videos. We evaluate it on a mixed gallery combining VATEX and ActivityNet. UMVR achieves R@1 of 23.0% and R@10 of 57.7%, compared with 20.8% and 55.5% for VLM2Vec-V2, respectively. These results show that UMVR retains its advantage in the mixed gallery. We have incorporated these results in Appendix A.3 (Mixed-Gallery Retrieval with Duration-Adaptive Frame Sampling) of the revised manuscript.

Q2 & W2: How does UMVR compare with existing video retrieval methods, and what is the contribution of adaptive event partitioning?

Response to Q2 & W2: We thank the reviewer for this suggestion. Existing video retrieval methods, such as X-CLIP[1] and UCoFiA[2], are typically trained and evaluated separately on each target dataset, whereas UMVR is trained on a single dataset and evaluated across both short- and long-video retrieval benchmarks. Differences in training settings and backbone architectures make it difficult to conduct a fair comparison based on their originally reported results. To provide a fair comparison, we adapt two representative video retrieval methods using local-global matching, X-CLIP and UCoFiA, to the same backbone as UMVR and evaluate all methods under identical training and testing settings. As shown in the following table, UMVR consistently outperforms both X-CLIP and UCoFiA across all seven datasets, demonstrating its advantages in both short- and long-video retrieval.

MethodMSR-VTT R@1MSR-VTT R@10VATEX R@1VATEX R@10MSVD R@1MSVD R@10DiDeMo R@1DiDeMo R@10ActivityNet R@1ActivityNet R@10QVHighlights R@1QVHighlights R@10TVR R@1TVR R@10
VLM2Vec-V230.765.731.571.449.385.632.166.320.156.820.453.45.718.4
XCLIP28.366.032.473.349.587.028.861.619.255.919.952.25.017.6
UCoFiA31.466.833.174.751.088.433.468.221.256.024.759.56.119.7
UMVR35.669.538.077.853.091.540.675.223.760.130.269.18.925.2

Additionally, to verify the effectiveness of our adaptive event partitioning, we compare it with uniform temporal partitioning and the event partitioning strategy from UEM[3]. As shown in the following table, our method consistently outperforms both counterparts, demonstrating the benefits of our adaptive event partitioning.

We have updated above results to Table 1 and Table 4, with the corresponding analysis to Section 3.2 and Section 3.4 (Length-Adaptive Event Representation) of the revised manuscript.

Temporal PartitionVATEX R@1VATEX R@10ActivityNet R@1ActivityNet R@10
Uniform Parti. (K=KvK=K_v)37.577.022.759.0
UEM37.877.022.559.3
LAER38.077.823.760.1

[1] Ma Y, Xu G, Sun X, et al. X-clip: End-to-end multi-grained contrastive learning for video-text retrieval[C]//Proceedings of the 30th ACM international conference on multimedia. 2022: 638-647.

[2] Wang Z, Sung Y L, Cheng F, et al. Unified coarse-to-fine alignment for video-text retrieval[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023: 2804-2815.

[3] Zhu S, Chen H, Zhang W, et al. Uneven event modeling for partially relevant video retrieval[C]//2025 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2025: 1-6.

AuthorsOpenReview ↗

Response to Reviewer EzpH (Part 2)

Q3: Why is Qwen2-VL-2B the only backbone evaluated? Do the retrieval gains and compression benefits still hold with a more recent, stronger MLLM?

Response to Q3: We thank the reviewer for this suggestion. Besides Qwen2-VL-2B, we extend UMVR to the stronger Qwen3-VL-2B-Instruct backbone and compare it with VLM2Vec-V2 under the same backbone and training protocol. For retrieval performance, UMVR consistently outperforms VLM2Vec-V2 across all seven benchmarks with clear margins. For compression benefits, since EATR reduces visual tokens before LLM encoding and does not depend on a specific backbone, its compression benefits still hold with Qwen3-VL, retaining only about 10% of visual tokens and reducing the subsequent LLM processing cost. We have incorporated these results in Appendix A.4 of the revised manuscript.

MethodMSR-VTT R@1MSR-VTT R@10VATEX R@1VATEX R@10MSVD R@1MSVD R@10DiDeMo R@1DiDeMo R@10ActivityNet R@1ActivityNet R@10QVHighlights R@1QVHighlights R@10TVR R@1TVR R@10
VLM2Vec-V233.570.638.577.853.791.641.275.123.059.424.056.87.420.8
UMVR36.172.441.982.159.692.848.780.128.064.634.070.39.927.3

Q4: How does EATR contribute to efficiency and retrieval performance and what is the complete video encoding cost?

Response to Q4: We would like to clarify that EATR achieves comparable encoding efficiency to standard FlashVID. The main difference is that EATR adaptively allocates tokens across events according to their informativeness, rather than using a uniform budget. This allows EATR to preserve more relevant visual information and achieve better retrieval performance without increasing the encoding cost.

Following the reviewer's suggestion, we compare standard FlashVID, uniform budgeting, and event-adaptive budgeting under matched actual token counts on VATEX and ActivityNet. As shown in the following table, UMVR outperforms standard FlashVID on both datasets, demonstrating the combined effectiveness of LAER and EATR. Furthermore, under the same LAER event partitioning, event-adaptive budgeting consistently outperforms uniform budgeting, confirming the benefit of adaptive token allocation.

We have updated Table 3 to include the comparison with standard FlashVID (denoted as FTP+UB) and uniform budgeting (denoted as LAER+UB) in the revised manuscript.

VariantVATEX R@1VATEX R@10ActivityNet R@1ActivityNet R@10
FlashVID36.476.121.757.3
Uniform Budgeting37.277.022.158.6
UMVR (ours)38.077.823.760.1

Q5: Does aggressive visual-token compression degrade retrieval performance on queries requiring fine-grained temporal information?

Response to Q5: We thank the reviewer for this valuable suggestion. To investigate whether token compression affects the preservation of temporal information, we group queries from ActivityNet into different subsets based on their temporal characteristics, including action order, motion direction, and brief events via Qwen3-VL-7B-Instruct. Queries identified as not involving any of them are grouped into the Others subset.

As shown in the following table, compression leads to some performance degradation on ActivityNet, but the drops across these temporal subsets are comparable to those for other queries. We attribute this to EATR's event-adaptive token allocation built upon FlashVID's spatiotemporal token reduction, which helps preserve relevant temporal information during compression.

Overall, these results suggest that aggressive token compression does not consistently cause greater performance degradation on queries involving temporal information.

Detailed experimental results and query categorization criteria are provided in Appendix A.5 of the revised manuscript.

Query Subset# QueriesFull Tokens R@1Compressed R@1
Overall1575325.323.7
Action Order116328.026.7
Motion Direction637323.322.1
Brief Events99920.519.2
Others889226.824.5
AuthorsOpenReview ↗

Response to Reviewer EzpH (Part 3)

Q6: Why does EATR use feature norms for token allocation, how does it avoid over-compressing visually simple but important events, and does its normalization guarantee a fixed total token budget?

Response to Q6: Our use of visual feature norms is motivated by the empirical findings of [1], who observed a positive correlation between visual-token norms and semantic informativeness through qualitative and statistical analyses. Accordingly, we allocate more tokens to events with higher estimated informativeness, allowing them to retain more visual information during compression. We acknowledge that feature norms may not fully capture the semantic importance of visually simple events. Nevertheless, EATR adjusts token budgets rather than discarding entire events, preserving their compressed representations for subsequent matching and reducing the risk of losing important information. We have further discussed this limitation and potential improvements in Section 4 (Limitation and future work) of the revised manuscript.

Additionally, our event-wise normalization indeed does not strictly guarantee that the total number of retained tokens matches the specified token budget when event lengths differ, but the resulting token counts remain close to the target budget in practice. When comparing our EATR with other budgeting strategies, we adjust their token budgets to match the actual number of tokens retained by EATR for each video, ensuring a fair comparison under identical token counts.

[1] Luo B, Wang T, Chen H, et al. Enhancing Visual Token Representations for Video Large Language Models via Training-free Spatial-Temporal Pooling and Gridding[C]//Proceedings of the International Conference on Learning Representations (ICLR). 2026.

W1: The paper's novelty is unclear in light of prior work.

Response to W1: We thank the reviewer for the valuable comment. We would like to clarify that the main contribution of UMVR lies in establishing a unified and efficient retrieval framework for both short and long videos. Although ASP, X-CLIP, and UCoFiA have explored local-global or multi-grained matching, these methods primarily focus on improving retrieval accuracy on conventional short-video benchmarks rather than explicitly addressing the challenges of long-video retrieval. Specifically, short-video retrieval typically emphasizes holistic relevance, whereas long-video retrieval requires more localized matching, making it challenging to accommodate both within a unified framework. Moreover, increasing video duration introduces more visual tokens and substantially higher computational costs.

To address these challenges, UMVR introduces two complementary designs. First, LAER adaptively adaptively determines event boundaries and representation granularity based on both global scene transitions and local visual changes, enabling a single trained model to handle both short- and long-video retrieval without setting-specific architectures or scoring functions. Second, EATR leverages these adaptive events to allocate visual-token budgets according to their informativeness, substantially reducing MLLM encoding costs while preserving retrieval performance.

Experimental results further validate these contributions. Under the same backbone and training settings, our UMVR consistently outperforms existing retrieval methods, including VLM2Vec-V2, XCLIP and UCoFiA, across both short- and long-video benchmarks. Moreover, UMVR retains only 10% of visual tokens while achieving better retrieval performance and lower encoding latency than the full-token baseline.

We also acknowledge that UEM explores non-uniform event partitioning. Unlike UEM, our LAER employs a coarse-to-fine partitioning strategy and further integrates event-aware token allocation for efficient MLLM encoding.

We thank the reviewer again for the valuable feedback and hope that the reviewer could reconsider the assessment of our work.

Dear Reviewer EzpH,

Thank you for your valuable feedback and thoughtful comments. As the discussion period is approaching its end, we would greatly appreciate it if you could let us know whether our clarifications and additional analyses have adequately addressed your concerns. We have also revised the manuscript accordingly to incorporate these improvements. We would be grateful for any further feedback and hope our efforts can be taken into consideration in your final assessment.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_mAMDOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes UMVR, a unified and efficient Multimodal Large Language Model (MLLM) framework for both short and long video retrieval. To address the differing granularity requirements in representation between long and short videos, the authors propose Length-Adaptive Event Representation (LAER), which adaptively segments videos into temporally coherent event sequences based on their temporal structure. To improve the efficiency of MLLM encoding, the authors further introduce Event-Aware Token Reduction (EATR), which dynamically allocates the visual token budget based on the information content of each event. Finally, relevance scores are computed via a unified local-global matching mechanism. Experiments on three long-video and four short-video retrieval benchmarks demonstrate that UMVR reduces the visual token load for MLLM encoding by 90% while maintaining strong retrieval performance, and that the EATR module is transferable to other MLLM-based retrieval models.

Critical strengths

1.By adaptively segmenting events via LAER, this approach elegantly resolves the contradiction in traditional methods where short videos require global representations while long videos require local ones. This enables a single model and a single scoring formula to handle videos with diverse temporal structures, demonstrating strong theoretical elegance and practical value. 2.EATR discards the traditional fixed token budget, instead dynamically allocating resources based on the visual information content of each event. Experiments not only demonstrate its effectiveness within UMVR but also successfully transfer it to other models such as VLM2Vec-V2 and UME-R1, proving the versatility of this mechanism.

Critical weaknesses

1.The coarse-to-fine decomposition of LAER requires computing the global similarity and local spatial token variations across all sampled frames. Although Table 2 reports the compression time on ActivityNet (averaging a few minutes), for longer videos (e.g., >10 minutes), will this frame-by-frame computation and subsequent token reduction become a new preprocessing bottleneck? The paper lacks a theoretical or empirical analysis of the computational complexity for ultra-long videos. 2.Although UMVR outperforms baselines on long-video datasets (e.g., TVR, QVHighlights), its absolute performance remains relatively low (e.g., R@1 on TVR is only 8.9% - 9.0%). This indicates that the current unified local-global matching or event representation still has limitations when handling highly complex, fine-grained queries in long videos. However, the paper does not provide a specific failure analysis to explain whether the performance bottleneck occurs during the event segmentation phase or the matching phase. 3.LAER relies on thresholds such as α and β for event boundary detection, while EATR relies on heuristic choices like the top-20% and quadratic scaling. Do these parameters require different tuning for short videos (e.g., MSR-VTT, with compact content) versus long videos (e.g., ActivityNet, with divergent content)? The paper primarily conducts ablations on ActivityNet and does not sufficiently discuss the generalization capability of these parameters across datasets with different length distributions.

Questions for authors

1.For videos exceeding 10 minutes in length, what is the proportion of preprocessing time consumed by LAER's coarse-to-fine decomposition and EATR? Does the model support hierarchical or chunking processing strategies to handle ultra-long videos? 2.On long-video datasets such as TVR, the absolute R@1 value remains relatively low. Could you provide specific failure cases to illustrate whether the model fails to capture fine-grained actions during LAER's event segmentation phase, or fails to correctly align the query with the events during the Unified Local-Global Matching phase? 3.Are the boundary sensitivity parameters (α, β) in LAER and the top-p% setting in EATR kept consistent across short-video datasets (e.g., MSR-VTT) and long-video datasets (e.g., ActivityNet)? If separate tuning is required, does this to some extent weaken the plug-and-play advantage of the "unified" framework?

1 comment
AuthorsOpenReview ↗

Response to Reviewer mAMD

We greatly appreciate the reviewer's positive assessment of UMVR's theoretical elegance and practical value in unifying short- and long-video retrieval. We also thank the reviewer for the constructive suggestions that have helped us further improve the manuscript.

Q1 & W1: What is the computational overhead of LAER and EATR on ultra-long videos, and can the framework support chunk-based processing?

Response to Q1& W1: First, we would like to clarify that we sample at most 128 frames per video for long-video retrieval. Therefore, for videos exceeding 10 minutes, only 128 sampled frames are processed. Under this sampling setting, the full-token model requires approximately 370 ms for LLM encoding, while our proposed UMVR reduces the overall processing time to 139 ms, representing a 62.4% reduction. Specifically, LAER, EATR, and LLM encoding in UMVR require approximately 4 ms, 91 ms, and 44 ms, respectively, with LAER and EATR accounting for 2.9% and 65.5% of the total processing time. Although EATR introduces additional computational overhead, the resulting visual-token reduction substantially decreases the subsequent LLM encoding cost, leading to a considerable improvement in overall efficiency. We have updated Table 2 of the revised manuscript to report the detailed processing times of LAER and EATR, and add the above discussion to Section 3.3.

Additionally, our current implementation does not incorporate hierarchical or chunk-based processing. UMVR can also be extended to hierarchical or chunk-based processing by applying its event modeling mechanisms to individual video chunks, allowing it to better handle extremely long videos. We have discussed this potential extension in Section 4 (Limitations and Future Work) of the revised manuscript.

Q2 & W2: Why is retrieval performance on TVR relatively low, and can the authors provide a detailed failure analysis?

Response to Q2& W2: The relatively low retrieval performance on TVR can be partly attributed to its challenging nature. Specifically, TVR contains videos from TV series, where many queries involve character names that are difficult for the model to recognize, while some queries rely on subtitles that may not directly align with the visual content.

To further investigate these challenges, we provide two representative failure cases from TVR. The first case illustrates a limitation in Unified Local-Global Matching, where LAER correctly identifies the query-relevant event, but the model assigns a higher similarity to another visually similar yet semantically different event. The second case reveals a limitation in LAER's event segmentation, where a short action is not sufficiently isolated due to subtle visual changes and the minimum boundary spacing constraint. These cases highlight the remaining challenges in fine-grained semantic matching and event decomposition. We have included the corresponding visualizations in Figure. 8 and detailed analysis in Appendix A.6 of the revised manuscript.

Q3 & W3: Are the boundary sensitivity parameters in LAER and the top-pp% setting in EATR kept consistent across short-video datasets and long-video datasets?

Response to Q3 & W3: We would like to clarify that all hyperparameters, including the boundary sensitivity coefficients α\alpha and β\beta, and the top-pp% event scoring ratio, are kept identical across all seven short- and long-video retrieval benchmarks without dataset-specific tuning.

Thanks again for your time and valuable feedback.

Reviewer_cotfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes UMVR, a unified retrieval framework, to address the difficulty of capturing both local events and overall semantics in short- and long-video retrieval, as well as the high cost of processing visual tokens with multimodal large language models. Its main contributions are to construct event representations through coarse-to-fine adaptive event partitioning, allocate token compression budgets according to event visual informativeness, and obtain both event-level and global representations in a single video encoding pass, combining the two types of similarity for retrieval. The authors evaluate the method on four short-video and three long-video benchmarks, validating it through baseline comparisons using the same backbone and training data, module ablations, parameter analyses, and efficiency tests. UMVR outperforms VLM2Vec-V2 in R@1 and R@10 on multiple datasets; with a base retention budget of 10%, it demonstrates a favorable trade-off between retrieval performance and computational cost.

Critical strengths
  1. Unified modeling of local events and overall semantics, supported by experiments. UMVR uses the same event representations, global aggregation, and scoring function for short- and long-video retrieval. With the same backbone and ActivityNet training data, it outperforms VLM2Vec-V2 in both R@1 and R@10 across seven benchmarks, and the ablation studies also support the complementarity of local and global information. Importance to my assessment:This provides direct evidence of the effectiveness of the core design and its applicability across datasets, and is the main basis for a positive assessment.

  2. Competitive retrieval performance under a low token budget. With a base budget of 10%, the average number of input tokens is reduced by approximately 90.8% relative to the baseline, while retrieval performance still exceeds that of the full-token baseline. Importance to my assessment:This indicates that the method can reduce the computational cost of MLLM-based video representations, although the efficiency advantage supported by the current evidence is limited to the processing stages after vision encoding.

Critical weaknesses

1.EATR's budget normalization does not ensure a consistent total token budget, making it unclear whether the gains come from adaptive allocation.

Equation 7 only ensures that the budget factors have a mean of 1 across events, yet the event budget Bk=Bw^kB_k = B\hat{w}_k is applied to every frame within each event. Let event kk contain nkn_k frames, with T=∑knkT = \sum_k n_k. The total budget before rounding and clipping is then

Nalloc=BK∑knkwk2∑jwj2,N_{\mathrm{alloc}} = BK \frac{\sum_k n_k w_k^2}{\sum_j w_j^2},

which may be higher or lower than the uniform budget BTBT when event lengths differ. Thus, the same base retention ratio does not automatically ensure the same actual total number of tokens. Table 3 and Figure 4 do not provide sufficient data to verify that adaptive and uniform allocation use strictly equal budgets.

This gap makes it difficult to attribute the performance gains entirely to EATR's allocation strategy, weakening the evidence for it as a core contribution.

2.The independent contribution of the core event modeling design and its incremental value over existing methods have not been sufficiently established. The comparison with VLM2Vec-V2 using the same backbone and training data supports the effectiveness of the overall UMVR framework. However, this baseline uses a single global representation, and therefore cannot fully distinguish whether the gains come from multiple local representations and local matching or from LAER's content-adaptive partitioning. The uniform partition in Table 4 is fixed at ten segments and does not match LAER's number of events for each video. In addition, Appendix A.1 explicitly identifies UEM, which uses uneven event modeling, as closely related work, yet the experiments do not provide direct comparisons with methods of this kind. This limitation constrains the assessment of LAER's independent contribution and its substantive advance over existing work.

Questions for authors
  1. When event lengths differ, how does the budget allocation in Equation 7 ensure that the actual total number of tokens conforms to the specified retention ratio? Does the implementation include a correction weighted by the number of frames in each event?

  2. In the implementation, are the two maxima in Equation 3 taken over the entire similarity sequences before and after the current position? If a local window is used, what is its size?

  3. What exactly is the minimum temporal interval between consecutive boundaries in the fine-grained partitioning?

  4. Is there a comparison that applies FlashVID directly to the entire video under the same total token budget?

2 comments
AuthorsOpenReview ↗

Response to Reviewer cotf

We sincerely thank the reviewer for recognizing the effectiveness of our unified retrieval framework, as well as the constructive suggestions for improving our work.

Q1 & W1: How does EATR handle token-budget normalization for events of different lengths, and are its performance gains still valid under matched token budgets?

Response to Q1 & W1: Indeed, our event-wise normalization does not guarantee an identical video-level token budget conforms to the specified retention ratio when event lengths differ, but it can give a similar budget. To examine the actual number of visual tokens retained, we calculate the average token counts of EATR and uniform budgeting on ActivityNet. On ActivityNet, our EATR retains an average of 1,085 visual tokens per video, a comparable and slightly lower count than the 1,178 retained by uniform budgeting.

To ensure a fair comparison, we use the actual number of tokens retained by EATR for each video as the target budget for uniform budgeting and distribute it evenly across the sampled frames. Under the same token budget, EATR achieves an R@1 of 23.7% and an R@10 of 60.1% on ActivityNet, still outperforming the 22.1% R@1 and 58.6% R@10 achieved by uniform budgeting. These results confirm the effectiveness of event-adaptive token allocation. We have accordingly updated Table 3 and Figure 4 with the uniform budgeting results obtained under strictly matched token budgets in the revised manuscript.

Q2: In the implementation, are the two maxima in Equation 3 taken over the entire similarity sequences before and after the current position? If a local window is used, what is its size?

Response to Q2: The two maxima are computed over the entire similarity sequence before and after the current position, respectively, without using any additional window. We have clarified this detail in Section 2.2 (Coarse Scene Decomposition) of the revised manuscript.

Q3: What exactly is the minimum temporal interval between consecutive boundaries in the fine-grained partitioning?

Response to Q3: We empirically set the minimum temporal interval between consecutive event boundaries to ten sampled frames to avoid generating excessively short events. We have clarified this setting in Section 2.2 (Fine-grained event mining) of the revised manuscript.

Q4: Is there a comparison that applies FlashVID directly to the entire video under the same total token budget?

Response to Q4: Following the reviewer's suggestion, we compare UMVR with standard FlashVID under the same token budget. The results show that UMVR consistently outperforms FlashVID on both VATEX and ActivityNet, further demonstrating the effectiveness of our model. We have updated this result into Table 3 (the performance of FlashVID is denoted as FTP+UB) and the corresponding analysis in Section 3.4 (Main Components) of the revised manuscript.

MethodVATEX R@1VATEX R@10ActivityNet R@1ActivityNet R@10
FlashVID36.476.121.757.3
UMVR38.077.823.760.1

W2: The independent contribution of LAER and its advantages over existing event-based methods are not sufficiently demonstrated.

Response to W2: We would like to clarify that our existing ablation studies in Table 3 have evaluated the individual contributions of LAER, EATR, and the local-global matching branches, beyond the overall comparison with VLM2Vec-V2. To further distinguish the benefits of LAER's content-adaptive partitioning, we conduct two additional comparisons.

To further distinguish the benefits of LAER’s content-adaptive partitioning, we compare uniform partitioning with K=KvK=K_v, where KvK_v denotes the number of events generated by LAER for video vv. LAER achieves better performance on both datasets, demonstrating the effectiveness of content-adaptive event partitioning beyond simply varying the number of events. Furthermore, to assess LAER against existing non-uniform event modeling approaches, we replace LAER with the event partitioning strategy used in UEM[1]. LAER also achieves better retrieval performance on both datasets, further supporting the effectiveness of its content-adaptive partitioning.

We have updated these results into Table 4 and the corresponding analysis in Section 3.4 (Length-Adaptive Event Representation) of the revised manuscript.

Temporal PartitionVATEX R@1VATEX R@10ActivityNet R@1ActivityNet R@10
Uniform Parti. (K=KvK=K_v)37.577.022.759.0
UEM37.877.022.559.3
LAER38.077.823.760.1

[1] Zhu S, Chen H, Zhang W, et al. Uneven event modeling for partially relevant video retrieval[C]//2025 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2025: 1-6.

Thanks again for your time and valuable feedback.

Thanks for your response and the additional experiments. They address my concerns and support my initial positive assessment, so I will maintain my rating of 3

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.