- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors introduce RandSlot, a parameter-free readout strategy for learning compact multi-vector representations in visual document retrieval. Instead of using learnable latent tokens (e.g., MetaEmbed) or feeding model-generated representations back autoregressively (e.g., CausalEmbed), RandSlot appends independently sampled, L2-normalized Gaussian random vectors to the query and document sequences during every encoding call. A shared causal vision–language encoder (adapted via LoRA) extracts L2-normalized hidden states, one from the terminal token of the original content and from the appended random positions, which are jointly optimized using a directional MaxSim InfoNCE objective.
Critical strengths
The central finding that continually resampled random inputs outperform persistent learnable tokens, autoregressive feedback, and fixed/zero tokens, and still transfer to deterministic zero-input inference is both practical and conceptually illuminating.
Critical weaknesses
- Unlike Direct-Feedback (where input depends on output ), RandSlot’s auxiliary inputs are sampled independently of the model's outputs. In a causal transformer, feeding the entire extended sequence in a single parallel forward pass produces the exact same hidden states at the final positions as feeding prefixes sequentially. However, Appendix A ("Readout execution") states that training recomputes the full prefix times sequentially and inference decodes step-by-step with KV caching. This sequential implementation inflates training cost and causes an artificial 47.4% jump in query encoding latency (from 109.92 ms to 162.08 ms in Table 5) when scaling from to . Simultaneously, Table 13 reveals that on the query side, using a single readout alone (: 45.42%) outperforms the full 4-vector query set, and removing or improves retrieval. Thus, RandSlot fails to induce complementarity on the query side, meaning 75% of the query budget and MaxSim dot products at test time are redundant.
- In the controlled ablations, Learned-Feedback (adapted from MetaEmbed) and Direct-Feedback (adapted from CausalEmbed) are implemented without their original method-specific training objectives (i.e., omitting MetaEmbed's Matryoshka nested-budget loss and CausalEmbed's auxiliary losses). Because those auxiliary objectives were specifically designed to regularize multi-vector generation, omitting them likely exacerbates the severe slot collapse observed for L and DF in Figure 4, making the controlled comparison less charitable to those paradigms. Moreover, all models are trained exclusively at , where inference scaling saturates at (46.0% on 3B) and leaves a notable 4.2-point gap to the full-token Base model (50.2% on ViDoRe V3).
Questions for authors
- Since are known prior to the forward pass in RandSlot (as well as in Z, F, and L), why did you recompute the full prefix sequentially during training and run sequential KV-cached steps at inference instead of passing all soft tokens in a single parallel forward pass? Is there a specific architectural constraint in Qwen2.5-VL / Qwen3-VL (such as M-RoPE position ID assignment) that prevents single-pass extraction of , or is the 47.4% latency increase from to in Table 5 purely an implementation artifact?
- Table 13 demonstrates that a single query vector () matched against 4 document vectors () achieves 45.42% on ViDoRe V3—matching or slightly exceeding the default configuration while reducing MaxSim dot products from 16 to 4 and eliminating auxiliary query steps. Why does directional MaxSim fail to produce complementary query readouts under RandSlot, and does this single-query-vector sufficiency also hold on Qwen3-VL-4B?
- Given that Learned-Feedback and Direct-Feedback exhibit extreme slot collapse in Figure 4 ( and ), to what extent is their underperformance in Table 3 caused by stripping away MetaEmbed’s nested-budget loss and CausalEmbed’s auxiliary losses? Did you experiment with adding nested-budget (Matryoshka) supervision to Learned-Feedback or RandSlot, which might also resolve the performance drop observed at and ?
6 comments
Q2: Table 13 demonstrates that a single query vector () matched against 4 document vectors () achieves 45.42% on ViDoRe V3—matching or slightly exceeding the default configuration while reducing MaxSim dot products from 16 to 4 and eliminating auxiliary query steps. Why does directional MaxSim fail to produce complementary query readouts under RandSlot, and does this single-query-vector sufficiency also hold on Qwen3-VL-4B?
Turning query redundancy into a matching tradeoff. In the original 3B diagnostic run, retaining query readout with four document vectors scores 45.42, versus 45.26 with all four query vectors. The score for one query vector and four document vectors is
requiring four -dimensional dot products instead of 16. This identifies an opportunity to cut matching cost fourfold while retaining the diagnostic ranking quality (Fig. 4 and Table 15, Appendix D).
Encoding and matching savings have different sources: one-pass inference removes serial model calls, while selecting reduces matching vectors. Obtaining still requires preceding causal positions. The original 3B selection is post hoc and needs independent deployment validation.
As a cross-backbone check, the newly reported 4B query-readout scores 46.117 versus 46.201 for the full 4/4 representation; the post-hoc best query position, , scores 46.297 (Table 6; Appendix F.3, Table 23). These within-run matching restrictions do not establish identical rankings or statistical equivalence.
Directional MaxSim allows multiple query readouts to select the same document position, so query redundancy is compatible with the objective; our complementarity claim concerns document readouts.
Q3: Given that Learned-Feedback and Direct-Feedback exhibit extreme slot collapse in Figure 4 ( and ), to what extent is their underperformance in Table 3 caused by stripping away MetaEmbed’s nested-budget loss and CausalEmbed’s auxiliary losses? Did you experiment with adding nested-budget (Matryoshka) supervision to Learned-Feedback or RandSlot, which might also resolve the performance drop observed at and ?
What restored objectives change. We added DF with the document/query auxiliary terms and Learned with the average of the 1/1, 2/2, and 4/4 retrieval losses. These controls retain our original 3B adaptation recipe and four-readout training budget.
Retrieval performance. Under this recipe, DF scores 42.1 without the auxiliary terms and 42.23 with them, while Learned scores 42.9 without nested supervision and 41.71 with it (Table 3(A/C)). At the reported precision, these differences are approximately +0.13 pp and -1.19 pp, respectively. RandSlot scores 45.4 under the common recipe. Thus, restoring these objectives does not close the observed retrieval gap in these single-seed runs. These results do not establish a general effect of either objective across training seeds or recipes.
Readout utilization. Across the same 74,016 positive pairs, H is normalized marginal winner entropy and U is the mean number of distinct document winners per pair (Fig. 6 and Table 22, Appendix F.2):
| Method | H | U |
|---|---|---|
| Original DF | 0.150 | 1.066 |
| DF + auxiliary | 0.395 | 1.358 |
| Original Learned | 0.410 | 1.361 |
| Learned + nested | 0.601 | 1.927 |
| RandSlot | 0.909 | 1.376 |
Both added objectives alleviate concentration. RandSlot retains the highest global winner entropy, while Learned+nested uses the most distinct winners within a pair. These utilization improvements do not necessarily translate into higher retrieval scores, so we distinguish readout utilization from retrieval quality.
Low-budget behavior. Table 17 (Appendix E.2) reports 37.6264/39.5986/41.7106 for the same Learned+nested checkpoint at 1/1, 2/2, and 4/4. Historical RandSlot/Zero-Feedback scores are reference points, not paired reruns. Without matched, verified low-budget evaluations of Learned without nested supervision, these results do not isolate the effect of nested supervision at 1/1 or 2/2.
Separate released-code comparison. We also retained the released CausalEmbed objective and auxiliary losses in a matched, single-seed 4/4 input-replacement experiment. Replacing the original inputs with random inputs improves the score from 29.25 to 30.15 (+0.90 pp), with gains on seven of eight tasks (Appendix E.3, Table 18). This uses a separate recipe from Table 3 and provides additional retrieval evidence for the input-strategy benefit when auxiliary losses are retained.
One-pass inference resolves the serial-execution issue. All auxiliary inputs are available in advance. Our Qwen2.5-VL-3B and Qwen3-VL-4B implementations now process them in a single full-sequence forward pass, preserving the evaluator’s masks and position/M-RoPE convention. We compare the same checkpoints and pre-sampled inputs against sequential inference with KV caching.
Full-V3 quality and A100-80GB query encoding latency (Table 7; Appendix F.4, Table 25). S/O denotes sequential KV / one-pass:
All macro-average changes are below 0.044 percentage points. For 3B, query encoding is 4.17×/6.22× faster at K=4/6, while page throughput increases from 4.07/3.35 to 6.05/6.05 pages/s. Thus, M-RoPE does not prevent this optimization, and the substantial K-dependent serial overhead is avoidable.
These BF16 measurements use batch size 1 with 32 query or 1536 visual tokens, excluding preprocessing, sampling, and retrieval. They establish nearly unchanged macro retrieval quality, not identical vectors or rankings.
For training, we retained sequential prefix recomputation primarily to keep the training procedure consistent with Direct Feedback for a controlled comparison of auxiliary-input schemes. This was an experimental-design choice, not an architectural constraint; one-pass training with dropout remains unvalidated. We will update the cost discussion accordingly.