acceptodds
Under review as a conference paper at ICLR 2027

VEDJE: Video-Efficient Discriminative Joint Encoder for Scalable Video-Text Retrieval

Abstract

Finding the right video often requires distinguishing similar scenes in which different events occur. Joint matching improves retrieval, but processing rich video representations for each query is costly. VEDJE compresses features within sampled frames while keeping their representations separate in a reusable cache. Feature-change prediction supplies an auxiliary training signal that improves retrieval from the compressed cache without adding work at query time. On MSR-VTT, MSVD, DiDeMo, and ActivityNet, VEDJE improves R@1 over matched first-stage retrievers in both retrieval directions. On MSR-VTT, it reaches 59.8 text-to-video R@1 with a fine-tuned VideoCLIP-XL first stage. In the VideoPrism configuration, shrinking the per-video cache fourfold to 12 KiB preserves text-to-video recall within 0.2 points. These results show that accurate video search can operate on compact evidence, encoded once and reused as new queries arrive. The accompanying code is provided in the supplementary material.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video-text retrieval, multimodal retrieval, vision-language models, multimodal reranking, efficient inference, visual token compression

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_bPtrOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce VEDJE, a video reranker that compresses each sampled frame into a few cached tokens offline. They evaluate on MSR-VTT, MSVD, DiDeMo, and ActivityNet across several first-stage retrievers.

Critical strengths
  1. This paper is a well-executed extension of EDJE to video, taking EDJE’s cached joint-reranking recipe from images to video and shows that it transfers.
Critical weaknesses

W1. Missing comparisons and related work undermine the paper’s novelty and central claims.

The paper’s contributions are a compact cache read by a lightweight joint reranker, feature-change supervision, and “scalable” retrieval. Each is evaluated without comparison to the work that most directly addresses it. That leaves both the novelty and the main claims unsubstantiated.

Novelty

The paper itself notes that EDJE already outlines the video extension via temporal compression and positional encodings, and VEDJE largely implements that outline. Beyond EDJE, each remaining component has close precedent that goes uncited or compared against:

  1. Caching document representations offline and running a small interaction model online, with a compressed or quantized cache. This is the core of PreTTR [4], MORES [5], Poly-encoders [6], and SDR [7].
  2. Query-agnostic, fixed-budget compression of video for retrieval. This is studied by AGC [1], H-Pool [2], and TempMe [3]. AGC includes memory-token compressors closely resembling VEDJE’s learned queries.
  3. The residual first-stage prior. This mirrors score interpolation [9] and score injection into a MiniLM cross-encoder [8]. The prior carries much of the result: the joint stream alone reaches 43.8 T2V R@1, below the 50.1 first stage (Table 11).

Comparisons.

No existing method is run on the paper’s candidates. Video-native rerankers such as Qwen3-VL-Reranker [12] and RankVideo [13] are absent, even though the motivation rests on their query-time cost, which is asserted but never measured. Also absent are compressed-representation alternatives: AGC [1] (56.9 MSR-VTT R@1 at 32 tokens per video), training-free H-Pool [2] on the same VideoPrism features, and embedding-based rerankers such as PE-Rank [14] and E2Rank [15]. The pooling baselines in Table 4 are constructed by the authors rather than drawn from this literature.

Scale.

Despite “scalable” in the title, all evaluations use corpora of at most 5K videos. Large corpora produce denser near-duplicate candidate lists, which is exactly where a query-blind compressed cache could lose discriminative evidence. MultiVENT 2.0 [16] (110K-video test collection, already used by AGC [1]) would directly test this.

Without these comparisons, a reader cannot determine three things: whether the gains come from the video-specific design or from known ingredients (e.g., the first-stage prior), whether VEDJE improves on existing points of the accuracy–cost frontier, and whether the results hold at the scale the title claims. Addressing this would require same-candidate comparisons against AGC/H-Pool and at least one video-native reranker with measured latency. It would also need an evaluation on a large-scale benchmark such as MultiVENT 2.0. Without them, the paper is better framed as a storage- and latency-efficient empirical study than as introducing new methods for scalable retrieval.

[1] https://dl.acm.org/doi/10.1145/3805712.3809589 [SIGIR 2026] [2] https://arxiv.org/abs/2409.14683 [3] https://arxiv.org/abs/2409.01156 [ICLR 2025] [4] https://dl.acm.org/doi/10.1145/3397271.3401093 [SIGIR 2020] [5] https://aclanthology.org/2020.emnlp-main.342/ [EMNLP 2020] [6] https://arxiv.org/abs/1905.01969 [ICLR 2020] [7] https://aclanthology.org/2022.acl-long.457/ [ACL 2022] [8] https://dl.acm.org/doi/10.1007/978-3-031-28244-7_5 [ECIR 2023] [9] https://dl.acm.org/doi/10.1145/3471158.3472233 [ICTIR 2021] [12] https://arxiv.org/abs/2601.04720 [13] https://arxiv.org/abs/2602.02444 [14] https://dl.acm.org/doi/10.1145/3696410.3714554 [WWW 2025] [15] https://arxiv.org/abs/2510.22733 [16] https://arxiv.org/abs/2410.11619 [CVPR 2025]

W2. The efficiency comparison doesn’t reflect query cost.

The paper’s central selling point is efficiency, but the evidence it uses (online parameter counts in Figure 1) doesn’t measure what it’s used to claim. It has the following issues:

  1. Single-stage retrievers (e.g., Video-ColBERT) run their text encoder once per query against precomputed video representations. VEDJE runs its cross-encoder once per candidate (20 forward passes per query, over up to 64 text and 64 cache tokens each). Its per-query compute therefore grows with the number of candidates, which a parameter count hides.
  2. Single-stage retrievers are plotted alongside two-stage systems, implying an apples-to-apples comparison that doesn’t exist.
  3. In the default configuration, the cache is built with a different backbone than the first stage (VideoPrism-Base vs. VideoPrism-LvT-B, Table 7). This requires a second visual encoding pass over the corpus and a second per-video store alongside the first-stage index. That cost matters most at the scale the paper claims to operate at.

Efficiency is better measured with per-query latency or FLOPs on the same hardware and candidate count. For a figure like Figure 1, it would be better to plot latency, or at least to separate first-stage retrievers from rerankers. Indexing cost and storage are also missing, including both the first-stage index and the cache. This is consequential because VEDJE trails CaRe-DPO and only ties LamRA in accuracy (Table 1a), so its case rests on cost, and the evidence presented can’t support that case.

Questions for authors

none, see above.

5 comments

Thank you for identifying the comparisons needed to assess the contribution. We have added same-candidate AGC and H-Pool comparisons, measured Qwen3-VL-Reranker on the same hardware, and evaluated the full 109,800-video MultiVENT 2.0 gallery.

W1. Contribution and relation to prior work. Our goal is accurate joint video–text matching with a small cache and low query cost. VEDJE compresses each sampled frame’s features into a separate token group, trains the cache with feature-change supervision, and uses a small joint reader to combine those groups with the query.

EDJE is the closest architectural precedent: it already combines a compressed visual cache with a lightweight joint reader, and its Appendix I outlines video extensions using temporal aggregation. VEDJE builds on this design and studies how to organize and train the video cache.

The text-retrieval precedents are also relevant. PreTTR [4] and MORES [5] precompute document-token representations and defer query–document interaction. PreTTR and SDR [7] additionally compress the stored token representations. Poly-encoders [6] cache one vector per candidate and use it to attend over multiple context summaries. These works establish precomputation and subsequent interaction; our methodological contribution concerns the structure and training of a compact video cache.

Preserving frame groups. We compare against an EDJE-inspired global resampler that aggregates features across all 16 frames, with frame-position encodings and the same compressor architecture and approximately 5M parameters as VEDJE. These controls isolate the value of frame-group organization and feature-change training while keeping the first-stage prior and joint reader fixed. All rows below use the same VideoPrism features, top-20 candidates, first-stage score, joint reader and 64-token/48-KiB budget. Results are MSR-VTT R@1.

Cache constructionT2VV2T
Global learned resampler52.051.6
H-Pool adapted to our features and joint reader53.253.4
VEDJE frame groups, without feature-change supervision54.053.6
Full VEDJE54.654.6

Preserving frame groups adds 2.0 points in both directions over the capacity-matched global resampler before feature-change supervision is introduced. Full VEDJE exceeds H-Pool by 1.4/1.2 points, with 95% paired confidence intervals [+0.8, +2.0]/[+0.5, +1.9]. These comparisons hold the first-stage prior fixed.

AGC, H-Pool and TempMe. AGC [1] uses attention-guided centroid selection and weighted clustering; its paper evaluates learned memory-token compression as a separate baseline. H-Pool [2] uses hierarchical clustering and pooling without additional training. Both were developed for compact representations used in MaxSim matching. Our H-Pool adaptation applies the published pooling algorithm to our frozen features and trains a projection and joint reader. We separately evaluate the released AGC checkpoint below.

On the same VEDJE cache, joint reading also improves over trained MaxSim: 54.6 versus 51.9 T2V R@1 (Table 5). TempMe [3] reduces video-encoding computation by progressively merging spatial and temporal tokens during encoding; VEDJE compresses extracted features for reusable query-time matching.

Feature-change supervision. At 64 tokens, predicting feature changes reaches 54.6/54.6, versus 54.2/53.9 for current-frame reconstruction and 52.8/53.8 for future-feature prediction (Table 18). For T2V, the benefit is greater under tighter compression: reducing 64 to 16 tokens costs 1.5 points without this supervision, but only 0.2 with it. The prediction head is discarded after training, adding no query-time computation.

W1. Role of the first-stage score. The prior is an important ingredient: Table 11’s independently trained joint-only model reaches 43.8 T2V R@1, compared with 50.1 for the first stage. Our new controls measure what the complete training procedure adds beyond combining independently learned scores.

Taking the stronger control in each direction (ordinary interpolation for T2V and a learned fusion head for V2T) gives 52.4/51.5, compared with 54.6/54.6 for VEDJE. The respective gains are +2.2 (95% CI [+1.0, +3.4]) and +3.1 ([+2.2, +4.0]).

We also freeze the learned caches and train fresh readers using the same protocol, with access to the first-stage score:

How the frozen cache was trainedT2VV2T
Without the first-stage prior52.552.1
Jointly with VEDJE’s prior-aware scoring head54.554.5

The 2.0/2.4-point advantage remains after retraining the reader, showing that cache training contributes beyond adding the prior afterward. Architecturally, the embedded prior enters the scoring head after joint text–cache encoding; cache construction remains query-independent (Eq. 5).

Relevant precedents include score interpolation [9], supplying the BM25 score as text to a trained cross-encoder [8], and CLEAR (Gao et al., 2021), which trains dense text representations to complement BM25. Our frozen-cache experiment tests the value of learning the video cache jointly with the prior-aware scorer.

W1. Same-candidate accuracy–cost comparison. AGC-32 (released MSR-VTT checkpoint) and Qwen3-VL-Reranker-2B [12] rerank the same fine-tuned VideoCLIP-XL top-20 candidates as VEDJE. T2V timings use one NVIDIA L40S, with one query at a time.

MethodT2V R@1Query total (ms)Rerank only (ms)GPU memory (GB)
First stage56.25.71—2.39
AGC-3256.514.989.2710.01
AGC-32 + score58.114.979.2610.02
Qwen3-VL-Reranker-2B53.1747.39741.686.64
Qwen3-VL-Reranker-2B + score57.3753.84748.136.65
VEDJE59.89.663.952.48

Against the validation-selected fusion variants, VEDJE gains 1.7 R@1 points over AGC (95% CI [+0.2, +3.2]) and 2.5 over Qwen ([+0.5, +4.5]). The query totals give 1.55× lower latency than fused AGC and approximately 78× lower latency than fused Qwen. VEDJE uses approximately one quarter of fused AGC’s measured GPU memory. As a fusion sensitivity check, fixed 50/50 weights give AGC and Qwen 58.7 and 58.0 R@1, respectively, also below VEDJE’s 59.8.

On VideoPrism candidates, fused Qwen reaches higher accuracy (57.9 versus 54.6) at higher latency (731 versus 5.79 ms); that fusion weight is transferred from the VideoCLIP-XL validation experiment. These results characterize the accuracy–cost tradeoff under both candidate generators.

PE-Rank [14] replaces passage text with projected embeddings for LLM-based listwise ranking. E2Rank [15] encodes the query together with candidate passage text, then scores cached document embeddings by cosine similarity. Both provide related text-retrieval approaches to efficient reranking with reusable representations.

All reported intervals use paired query bootstrapping, averaging each query’s metric contribution across training seeds where three seeds were evaluated.

Additional reference: Gao et al., Complement Lexical Retrieval Model with Semantic Residual Embeddings, ECIR 2021.

W1. Evaluation at scale. We evaluate T2V retrieval on the full 109,800-video MultiVENT 2.0 test gallery, using a frozen PE-Core-B first stage. All rerankers, including the reader in our H-Pool adaptation, are trained on MultiVENT training queries and rerank identical top-100 candidate lists. Video inputs are the same 16 sampled frames, with no audio or transcripts at inference. Timings use one NVIDIA L40S.

MethodnDCG@10R@10Complete query (ms)
First stage31.033.05.0
H-Pool-64 with our joint reader32.534.28.9
VEDJE-1636.036.66.8
VEDJE-6437.538.08.5

VEDJE-64 improves nDCG@10 by 6.5 points over the first stage and 5.0 over H-Pool at the same cache budget. VEDJE-16 retains a 5.0-point improvement over the first stage with one quarter of the cache payload. On the 20% of queries with the smallest first-stage top-1/top-2 score margins, VEDJE-64 gains 10.5 points (95% CI [+8.8, +12.2]).

Published MultiVENT results provide additional context: AGC reports 46.3 nDCG@10 with an audiovisual backbone; RankVideo reports 47.8 with a CLIP first stage; and Video-ColBERT reports 42.2 (Table 2 of [13]). These use different configurations. Our controlled comparison measures VEDJE’s added value with the first stage, visual inputs and candidate lists fixed; our direct latency comparisons use the locally profiled systems.

W2. Query cost and single-stage comparisons. VEDJE scores candidates in batches, and its computation grows with the candidate count. With the default 64-token VideoPrism cache, complete T2V query latency is 5.79/6.21/7.83 ms for 20/50/100 candidates (Table 6).

On the same NVIDIA L40S and MSR-VTT queries and gallery, native single-stage AGC-32 achieves 56.4 T2V R@1 in 8.79 ms using 7.77 GB of GPU memory. VEDJE with fine-tuned VideoCLIP-XL achieves 59.8 R@1 using 2.48 GB, with its measured stages summing to 9.66 ms: 3.4 additional R@1 points for 0.87 ms, with approximately one third of AGC’s GPU memory use.

We will replace Figure 1’s parameter-count comparison with T2V R@1 versus query latency, using the same NVIDIA L40S and MSR-VTT queries and gallery. Separate panels will show complete retrieval pipelines, with native single-stage and two-stage systems clearly marked, and rerankers evaluated on identical VideoCLIP-XL top-20 candidates. Each point will identify its first stage and candidate count; the caption will specify which totals are calculated from measured stage costs. LamRA and CaRe-DPO will remain published accuracy references in Table 1a; measured efficiency claims will be limited to the systems we profiled.

W2. Offline construction and storage. The default configuration uses VideoPrism-LvT-B for first-stage retrieval and VideoPrism-Base for cache features. Complete construction takes 88 ms/video, versus 62 ms for the first-stage representation alone: 26 ms additional offline cost.

The measurement includes decoding and preprocessing 16 frames at 288×288, both visual encoders, compression and writing, on one NVIDIA L40S at batch size 1. The visual encoders use FP32 and the compressor uses BF16.

For this default configuration, packed representations occupy 15,370 bytes/video at 16 tokens or 52,234 bytes/video at 64 tokens, including the first-stage vector and identifiers. ANN infrastructure and filesystem overhead are excluded. The 64-token cache payload for the full MultiVENT gallery is 5.4 GB.

Our scalability claim concerns fixed cache storage per video and bounded neural reranking work at a fixed candidate count. First-stage search and total index storage remain dependent on gallery size.

We will incorporate all additional results, protocols and explanations into the revised manuscript and appendix, including expanded Related Work and the revised Figure 1. Thank you for prompting these additions; we hope the new evidence addresses your concerns and would be grateful for your reassessment. We would be happy to respond to any remaining concerns.

Reviewer_UzdHOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes VEDJE for efficient video-text retrieval. It stores a few compressed tokens for each sampled frame. A small model uses these tokens, the text, and the first-stage score to rerank candidates. During training, predicting feature changes helps improve the compressed tokens without extra cost at search time. Tests on four datasets show gains over the first-stage retrievers. On MSR-VTT, text-to-video R@1 improves from 56.2 to 59.8. The paper also studies cache size, model components, and query speed。

Critical strengths

1, A practical balance between accuracy and cost. VEDJE stores compact video tokens and uses a small reranker, avoiding video encoding during search. With VideoPrism, reducing the cache from 48 to 12 KiB per video lowers text-to-video R@1 by only 0.2 points. This is a useful result for video search systems where storage and query cost matter.

2, Clear experimental support. The method improves retrieval on four datasets and works with several first-stage models. It also improves a fine-tuned VideoCLIP-XL model from 56.2 to 59.8 R@1 on MSR-VTT. Controlled experiments support the value of separate frame groups and feature-change training. These results give me confidence that the main design choices are useful.

Critical weaknesses

A minor concern is that the time needed to build the video cache is not clearly reported. A brief estimate of this offline cost would help readers better understand the practical efficiency of the method.

Questions for authors

See Weaknesses.

Reviewer_sYjAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a hybrid offline–online approach. It first compresses and caches features from sampled video frames offline, then uses a lightweight joint encoder online to match each query against the cached features while incorporating the first-stage retrieval score. The authors also introduce a video feature-change prediction module to guide the offline feature-compression process during training.

Critical strengths

1、Each video is encoded offline, making the approach suit to repeated-query scenarios. 2、Rather than jointly encoding spatial and temporal information, the method compresses frame features separately and then arranges them in temporal order. 3、Although using feature changes is a fairly standard technique, it helps improve the cached feature training. 4、The experimental analysis is thorough and detailed.

Critical weaknesses

1、The first-stage candidate set limits the reranker’s retrieval ceiling. The paper reports little change when increasing the candidate pool from 20 to 100, but it does not clarify whether this is due to the test-set size or another factor. 2、The gains vary substantially across datasets, yet the paper provides little analysis of what drives these differences. 3、Extracting frame features offline is a standard approach in early video understanding systems. Is the paper’s main contribution therefore the improved token-compression method?

Questions for authors

Given the relative simplicity of the method, the authors’ motivation is particularly important. What specific problem does the approach primarily address: improving the token encoder, or enabling efficient retrieval through an offline–online design?

3 comments

Thank you for your positive assessment and constructive questions. Our new analyses explain the candidate-depth results and variation across datasets.

W1. Why does increasing the candidate pool help little? Most available correction opportunities are already within the top 20. On MSR-VTT T2V, among the 470 first-stage errors with a relevant item in the top 100, 402 (over 85%) already have that item in the top 20. Expanding the pool makes a relevant item available for another 68 queries.

The original 20 candidates retain their relative ordering. The added candidates correct one query and introduce four new errors, giving three fewer correct answers. This explains the small change in same-checkpoint T2V R@1: 54.7 at K = 20 and 54.4 at K = 100, while complete T2V query latency increases from 5.79 to 7.83 ms. The paper’s 54.6 result at K = 20 is the three-seed mean.

We also test a short list on the full 109,800-video MultiVENT 2.0 gallery. With PE-Core-B as the first stage, nDCG@10 increases from 31.0 to 36.4 with 20 candidates, versus 37.5 with 100. Thus, 20 candidates retain 83% of the improvement obtained with 100. A short candidate list remains an effective operating point at this scale, with additional accuracy available from a larger pool on MultiVENT.

W2. What drives the variation across datasets? Our new analysis identifies two measurable sources of variation: the first-stage errors available for correction and the fraction VEDJE successfully corrects. Newly introduced errors remain similar across the configurations below.

An available correction opportunity is an initially incorrect query whose relevant item appears at ranks 2–20.

Setting, T2VAvailable opportunitiesCorrectedNewly incorrectNet gain (pp)
MSR-VTT, VideoPrism40%5.5%1.0%+4.5
DiDeMo, VideoPrism38%2.0%0.9%+1.1
DiDeMo, VideoCLIP-XL38%9.9%1.0%+8.9
MSR-VTT, fine-tuned VideoCLIP-XL32%4.6%1.0%+3.6

On DiDeMo, both configurations provide approximately the same proportion of correction opportunities, but VEDJE corrects about 5% of them with VideoPrism and 26% with VideoCLIP-XL. Newly introduced errors remain nearly unchanged. The larger gain therefore comes predominantly from more successful corrections.

On MSR-VTT, both configurations correct approximately 14% of the available errors, but the fine-tuned VideoCLIP-XL configuration leaves fewer opportunities, 32% versus 40%. This helps explain its smaller remaining gain. Changing the backbone changes both the retrieved candidates and cached visual features; these comparisons describe their combined effect.

We also examined whether gains follow a common pattern across queries. Within each setting, we define the first-stage margin as the difference between the top-1 and top-2 scores, and compare the lowest-margin 20% of queries with the remaining 80%. Small margins mean that the first stage assigns similar scores to its two leading candidates.

VEDJE’s net gain is larger in the low-margin group in all 16 settings. For example, on MSR-VTT with VideoPrism in V2T, the gains are +13.3 versus +2.7 points, a difference of +10.6 points (95% CI [+3.3, +18.1]).

Thus, the size of the overall gain varies with the available errors and correction success, while larger gains consistently occur among queries whose leading candidates have similar first-stage scores.

W3/Q1. Is the contribution token compression or the offline-online design? The problem is efficient video retrieval; our methodological contribution is the compact video cache and its training for joint matching. Offline precomputation is established, including in EDJE. VEDJE preserves separate token groups for sampled frames, trains them with feature-change supervision, and lets a small joint reader combine them with the query.

The controlled comparisons below quantify these choices on MSR-VTT. The frame-group comparisons use a 48-KiB cache, matched compressor capacity, and no feature-change supervision. The compression comparisons use VEDJE’s frame groups.

Controlled comparisonR@1 before → afterChange
Global resampler → frame groups, T2V52.0 → 54.0+2.0
Global resampler → frame groups, V2T51.6 → 53.6+2.0
48 → 12 KiB without feature-change supervision, T2V54.0 → 52.5−1.5
48 → 12 KiB with feature-change supervision, T2V54.6 → 54.4−0.2

Separate frame groups improve accuracy at a fixed budget, and feature-change supervision preserves T2V accuracy under 4x cache compression. The resulting compact cache supports joint matching without re-encoding indexed videos at query time.

We will incorporate all new results, analyses and explanations into the revised manuscript. Thank you again for your feedback. We would be happy to respond to any remaining concerns.

Dear Reviewer sYjA,

We sincerely thank you for your constructive feedback, which has helped us strengthen the paper and clarify its contributions.

We hope that our detailed responses and additional results have adequately addressed your concerns. Should any questions or concerns remain, we would be more than happy to provide further clarification.

If you find that our responses have satisfactorily resolved your concerns, we would greatly appreciate your reconsideration of the score and stronger support for the paper.

Thank you again for your time and thoughtful evaluation.

Best regards,
The Authors

Dear Reviewer sYjA,

Thank you again for your time and effort.

We hope that our rebuttal has addressed your concerns. With the discussion period nearing its end, we would appreciate hearing your thoughts.

We would be happy to address any remaining concerns or provide further clarification. If our responses have addressed your remaining concerns, we would greatly appreciate it if you could consider strengthening your support for the paper.

Best regards,

The Authors

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

Thank you for your thoughtful reviews. We have added controlled AGC, H-Pool and Qwen comparisons, full-gallery MultiVENT evaluation, cache-training controls, and measurements of query and indexing costs. The main text presents the key results; Appendix C provides protocols and supporting analyses. Revised passages are highlighted in orange.

  1. Contribution and positioning (bPtr W1; sYjA W3/Q1).

Concern: What does VEDJE contribute beyond established offline–online retrieval?

Response: We clarify that the contribution is the structure and training of a compact video cache for joint matching. Expanded Related Work explains its relationship to EDJE, cached interaction models, video compression, and first-stage score integration.

Where: Section 2, lines 091–130.

  1. Sources of the gains (bPtr W1; sYjA W3/Q1).

Concern: Which design choices account for the improvements?

Response: At matched capacity and storage, frame groups add 2.0 R@1 points in both directions over the global resampler before feature-change supervision. That supervision reduces the T2V loss from fourfold compression from 1.5 to 0.2 points. With fresh readers, frozen caches trained with the prior retain a 2.0/2.4-point T2V/V2T advantage over those trained without it.

Where: Sections 4.3–4.4, lines 372–420, and Section 4.6, lines 432–471 (Table 6). Protocols: Appendix C.1.

  1. Accuracy and query cost (bPtr W1/W2).

Concern: How does VEDJE compare with relevant alternatives?

Response: On identical fine-tuned VideoCLIP-XL top-20 candidates, VEDJE reaches 59.8 T2V R@1 at 9.68 ms per complete query, versus 58.1 at 14.99 ms for fused AGC-32 and 57.3 at 753.91 ms for fused Qwen3-VL-Reranker-2B. Fusion weights are validation-selected; fixed 50/50 fusion gives 58.7/58.0. With VideoPrism candidates, VEDJE exceeds the H-Pool adaptation (54.6 versus 53.2 R@1) at similar latency (5.79 versus 5.96 ms), while fused Qwen reaches higher accuracy (57.9 R@1) at 730.91 ms. Figure 1 plots measured complete-query latency, including first-stage retrieval, for complete pipelines (a) and rerankers on identical candidates (b).

Where: Figure 1 and Section 4.7, lines 484–508. H-Pool timing: Appendix C.1, lines 1482–1483. Other measurements: Appendix C.2.

  1. Candidate depth and scale (sYjA W1; bPtr W1).

Concern: Why does increasing the candidate pool help little, and do gains persist at scale?

Response: On MSR-VTT, 402 of the 470 first-stage errors recoverable within 100 candidates already have a relevant item in the top 20. For one fixed checkpoint, expanding from 20 to 100 candidates corrects one query and introduces four new errors, accounting for the change from 54.7 to 54.4 R@1. On the full 109,800-video MultiVENT 2.0 gallery, nDCG@10 rises from 31.0 to 37.5 with 100 candidates, exceeding the matched H-Pool adaptation by 5.0 points at the same cache budget. With 20 candidates, VEDJE retains 83% of the gain over the first stage.

Where: Section 4.2, Table 3 and lines 366–370; Section 4.7, lines 510–514. Supporting analyses: Appendices C.3–C.4. Scalability scope: Section 5.

  1. Variation across datasets (sYjA W2).

Concern: What drives the different gain sizes?

Response: The new analysis separates available correction opportunities, successful corrections and newly introduced errors. In the analyzed configurations, gains vary with the first two quantities while new errors remain similar. Gains are also larger among the 20% of queries with the smallest first-stage score margins in all 16 dataset/backbone/direction settings.

Where: Section 4.2, lines 359–364. Full error-transition and margin analyses: Appendix C.5.

  1. Offline cost and storage (UzdH W1 and question; bPtr W2).

Concern: What do the additional encoder and cache cost?

Response: Complete default construction takes approximately 88 ms/video on one L40S, versus 62 ms for the first stage alone. Packed representations require 15,370/52,234 bytes/video at 16/64 tokens, including the first-stage vector and identifiers. We state the measurement scope and excluded overheads.

Where: Section 4.7, lines 520–523. Construction protocol and storage accounting: Appendix C.6.

We hope these revisions clarify the contribution and address your concerns. We welcome any further questions.

Kind regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.