- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies the cost of rebuilding the index when a visual document retriever (e.g., a ColPali/ColQwen-style multi-vector retriever) is upgraded. The authors observe that many retriever upgrades change only the language-side adapter or the retrieval head while leaving the visual computation unchanged, and propose ReprForge:
- Discovering the state to retain. From one traced forward pass, ReprForge identifies every visual output read by later stages (the "visual interface", e.g., the three DeepStack features that Qwen3-VL injects into its language layers), together with the parameters and buffers that produce them (the "state dependencies"). This visual state is saved during the initial index build.
- Checking whether the state can be reused at upgrade time. The check has two levels. At the model level, the state dependencies of the source and target models are compared after loading. At the page level, the processed visual input is compared, either by rerunning the target processor on every page or, when the deployment guarantees unchanged inputs, through a "fixed input contract".
- Resuming with the target model. If both checks pass, encoding resumes from the retained state using the target model's own text embeddings, positions, language layers, and retrieval head; otherwise, the page is encoded in full from the raw image.
Experimental results:
- State discovery takes less than 1.5 seconds on each of five retriever configurations.
- On 76 released upgrade pairs, the reuse decisions agree with independent bitwise comparisons.
- On a 20,946-page collection, using a common processing configuration (602,112 pixels) and the fixed input contract, four upgrades achieve 4.35–4.48× rebuild speedups with indexes bitwise identical to full encoding. Total time for one initial build plus four upgrades falls by 61%, at the cost of 63.9 GB of additional storage.
Critical strengths
- Thorough verification of exact reconstruction.
- Across all 2,840 ViDoRe v1 pages (about 1.54 billion representation elements), the maximum absolute error is zero, and all 4 × 20,946 page representations in the main experiment are bitwise identical to full encoding.
- The authors show that this is not because the source and target indexes are already similar: before replay, they agree on the top-1 result for only 80.5% of queries, and no query keeps the same ordered top-10. After replay, all 1,033 queries recover the target's ordered top-10 exactly.
- Identifying page input as a separate validity condition. Ten upgrade pairs keep the visual weights unchanged but differ in processing configuration (e.g., pixel budget) and therefore cannot be reused directly (Table 2). This is a useful reminder for practitioners that comparing weights alone does not guarantee reusability. The ablation showing that exact reconstruction fails without the DeepStack features (Appendix C.3) is also a concrete and useful observation.
Critical weaknesses
W1. The central claim that "reusing visual computation requires more than saving the final output of the vision encoder" is not well supported: in the paper's own experiments, saving the final output is exactly what most cases require
The introduction presents identifying the complete visual state as the core difficulty the method addresses (using Qwen3-VL's DeepStack as the example) and motivates trace-based automatic discovery on this basis. However, the paper's own data indicate that cases requiring multiple visual interfaces are the exception:
- The main results do not exercise this claim. Among the five configurations in Table 1, Qwen2.5-VL, PaliGemma, and Idefics3/SmolVLM each have a single visual interface, i.e., only the final output of the visual branch (including the merger) needs to be cached; only Qwen3-VL requires four. Of the 76 upgrade pairs in Table 2, 74 belong to model families without DeepStack (Qwen2/2.5-VL, PaliGemma, Idefics3), and the main experiment in Table 3 (20,946 pages, four upgrades) uses only the single-interface Qwen2.5-VL.
- Automatic discovery matches the manual specification exactly. In Appendix C.1, the automatically discovered interface and dependencies are identical to the manual specification (391 dependencies for Qwen2.5-VL and 316 for Qwen3-VL-4B), and Appendix C.4 shows that a replay boundary later than post-vision saves only about 1% more time per page. For existing architectures, then, choosing the cache location by hand is not difficult, and the practical benefit of automatic discovery is not demonstrated.
W2. The statistic that "107 of 151 resolvable adapter releases contain no vision-side updates" does not support the motivation that visual computation is usually reusable across upgrades
The 151 adapters were collected by searching for model-family names (Appendix I.1). Most of them are independent fine-tunes of the same Qwen or PaliGemma base by different publishers, not successive versions along an upgrade chain that a single deployment would go through. Because the sample is drawn from derivatives of the same base that share a frozen vision tower, finding that the vision side is mostly unchanged is largely a consequence of how the sample was selected, and it does not show that visual computation usually stays unchanged in real upgrades.
The main experiment has the same issue. Of the four "real upgrade" targets, only ColQwen2.5 v0.2 is a successor of the source model v0.1; Metric-AI, T-Systems, and ColNomic are models independently trained by other publishers on the same base, which is closer to switching models than to upgrading.
In addition, some of the paper's more detailed analyses seem to point to a more conservative conclusion. Among the 18 historical version pairs in Appendix I.1, 5 match in both weights and processing fields. In Appendix H.3, under each release's native processing configuration, the visual input matches the source on only 15/200, 15/200, and 30/200 pages for v0.2, T-Systems, and ColNomic, respectively.
W3. The contribution is mainly an engineering implementation, with limited methodological novelty
In implementation terms, the fixed-input-contract path used in the main experiment can be summarized as follows: save the output of the visual branch during the initial build; at upgrade time, compare once the parameters and buffers that produce this output in the source and target models; if they match, load the cached state and run only the language side and the retrieval head. This is essentially memoization with dependency checking. The tracing implementation and bitwise verification are solid, but the paper offers no insight beyond the engineering, such as an analysis or characterization of which kinds of upgrades preserve the validity of intermediate representations. The main contributions lie in the system implementation and careful timing measurements.
In the related work, the paragraph on reuse of intermediate computation cites only HELIX (2018) and materialized-view maintenance (1995). Similar mechanisms already exist in recent LLM/VLM inference systems and are not discussed:
- Activated LoRA (arXiv 2504.12397) modifies how the adapter is applied so that an adapter, once switched in, can directly reuse the base model's KV cache without recomputation;
- VLCache (arXiv 2512.12977) reuses the encoder cache and KV cache of recurring multimodal inputs across requests and has been integrated into SGLang;
- LRAgent (arXiv 2602.01053) splits the cache in multi-LoRA settings into a component shared from the base model and an adapter-specific component, and shares the former.
ReprForge's increment lies mainly in applying the mechanism of "reuse shared upstream computation and recompute only the changed downstream part" to offline index rebuilding, with weight comparison used to decide reuse across versions.
Questions for authors
- Necessity of automatic discovery. On which existing or foreseeable architectures is manually identifying the visual interface genuinely difficult or error-prone, so that automatic discovery becomes necessary? Appendix K states that the method requires the visual interface to appear as module outputs. Does this mean the method covers precisely those architectures where the boundary is also easy to identify by hand?
- Real upgrade chains. Could you report, for successive versions from the same publisher (e.g., colqwen2 v0.1 → v1.0, colpali v1.2 → v1.3), what fraction of pages can be reused under each release's native processing configuration, and the corresponding end-to-end speedups?
- Gains under native configurations. The 4.35–4.48× speedups in the main experiment rely on forcing a common 602,112-pixel configuration and the fixed input contract. What are the end-to-end speedups if each of the four targets uses its own released processing configuration with the generic validation path? Since the common configuration changes the target's retrieval quality (ColNomic nDCG@10 drops by 1.83 points), how do the authors view the fact that what is exactly reproduced is a target model with a modified configuration?
- Relation to inference-side cache reuse. Compared with inference-side work that reuses shared computation, such as Activated LoRA, VLCache, and LRAgent, what is the essential methodological difference of ReprForge beyond the application setting?