acceptodds
Under review as a conference paper at ICLR 2027

Receiver-Conditioned Selection and Reconstruction for Latent Communication Between LLM Agents

Abstract

Latent communication through key-value (KV) caches can avoid natural-language decoding and repeated prefilling by transferring internal states between large language model (LLM) agents. Existing approaches fix transmission choices before the current query is known or reuse latent states formed outside the receiver context. We find that the content a sender ranks first can hurt the receiver and that different receivers need different content. We therefore condition what to retrieve and how to read it on the receiver. Each document is written once, before any receiver or query exists, as an INT8 snapshot of its early-layer state. At read time, the receiver uses its state to select a document and reruns the upper layers under its private context, so document states are formed for that receiver rather than transplanted. Across Qwen3-4B, Qwen3-8B, and Llama-3.1-8B-Instruct, receiver-aware selection with BGE-M3 improves F1 by 15.5–17.2 points over query-only selection for both text and checkpoint delivery. With selection fixed, read-time reconstruction improves F1 by 5.1–13.5 points over full static KV reuse with payloads to smaller. Under controlled TCP networking, the complete system reduces time to first token (TTFT) by 6.91%, 5.81%, and 4.23% on Qwen3-4B, Qwen3-8B, and Llama-3.1-8B-Instruct, respectively. In the end-to-end evaluation, answer quality remains comparable to native text, with F1 differences ranging from to points. Code is available at anonymous.4open.science/r/rcsr-latent-comm-B184.

Primary area:
foundation or frontier models, including LLMs
Keywords:
KV-Cache Communication, Multi-Agent LLM Systems, KV-Cache Reuse, Receiver-Aware Selection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vfgtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies a write-once, query-later store for same-backbone LLM agents. Documents are stored as INT8 residual checkpoints at an early layer boundary. A receiver selects a document using its question and locally held evidence, inserts the checkpoint, and reconstructs upper-layer document states under its private prefix. No parameters are trained. Experiments separate BGE-M3 receiver-aware retrieval, matched read paths, and an end-to-end lexical-address system with overlapped transfer/preparation, using Qwen3-4B, Qwen3-8B, and Llama-3.1-8B-Instruct.

Critical strengths
  1. Reconstruction addresses the mismatch between stored states and the receiver's context. With query-free writes, the stored upper-layer states have never seen the eventual receiver's prefix. Holding the selected document fixed, reconstruction improves over static reuse by 13.54, 6.53, and 5.13 F1 points across the three backbones. Wrong-example checkpoints and random retrieval provide additional evidence that the selected content matters. The description of ragged-cache positions, masks, and execution paths also makes the approximation reasonably clear.

  2. The timing comparison is fairly constructed and the limitations are clearly reported. The native-text baseline also overlaps transfer and preparation. Paired timing, process restarts, stage breakdowns, and transferred bytes give useful support for the modest TTFT benefit in this workload. I appreciate that the paper reports the much larger checkpoint traffic alongside the speedup, as well as the small completion-latency changes, shared-GPU limitations, and unresolved Llama quality margins.

Critical weaknesses
  1. The practical operating envelope remains narrow and uncharacterized. TTFT improves by 4.23–6.91%, while checkpoint transactions carry approximately 413–661 times as many bytes as native text. The larger transfer is hidden behind preparation under the tested network and short windows; there is no demonstrated break-even range. The 8B runs also share GPUs with uncontrolled processes, and question-bootstrap intervals do not quantify independent load/network variation. Supporting-fact-centered windows, one gold document already held by the receiver, and a guaranteed target in a small directory further restrict workload representativeness. These are legitimate controls, but broader efficiency claims need a limited sensitivity check.

  2. Component results do not fully establish the deployed system's contribution. The 15.5–17.2-point retrieval gains use BGE-M3 over document text, whereas the deployed selector uses 96-byte rare-term hash addresses. There is no matched query-only-address ablation to establish receiver conditioning at that budget. Adaptive-cache comparisons also exchange substantially different costs: reconstruction reruns 83–84% of document layer–token computations versus approximately 19–34% for CacheBlend, while transferring fewer bytes. Higher F1 alone does not establish the better systems tradeoff. The existing boundary sweep measures quality without alternative-boundary timing.

Questions for authors
  1. Based on the recorded transfer and preparation times, at approximately what bandwidth would the method cease to outperform native text? Could a slower-bandwidth or longer-window condition be used to check this estimate?
  2. For the existing 400-question sample, what happens to target retrieval and answer quality if receiver-document features are removed from the 96-byte-address scorer while keeping the rest of the selector unchanged?
  3. Could latency, transferred bytes, and recomputation be reported together for one representative adaptive-cache comparison? Alternatively, timing one more boundary from the existing sweep would help show how the reconstruction cost changes with the layer choice.
3 comments
AuthorsOpenReview ↗

Response to Reviewer vfgt 1/3

We thank the reviewer for the precise questions, and for recognizing that reconstruction addresses the stored-state mismatch and that our timing comparison is fair. We ran the experiments the review asks for.

Q1 & W1: break-even bandwidth and longer windows

We ran the end-to-end system on Qwen3-8B at six link rates, each with the same 100 questions in three independent batches with process restarts and five timings per request.

Link rateText TTFT (ms)Ckpt TTFT (ms)
1 Gbps84.590.0584.59_{0.05}79.190.05\mathbf{79.19}_{0.05}
224 Mbps86.340.0786.34_{0.07}81.370.06\mathbf{81.37}_{0.06}
75 Mbps85.210.0685.21_{0.06}80.820.05\mathbf{80.82}_{0.05}
50 Mbps85.040.05\mathbf{85.04}_{0.05}93.400.2093.40_{0.20}
25 Mbps85.870.06\mathbf{85.87}_{0.06}154.200.09154.20_{0.09}
10 Mbps85.290.06\mathbf{85.29}_{0.06}288.800.11288.80_{0.11}

Table 1: TTFT versus link rate (Qwen3-8B, 2Wiki), per-question medians of 15 timings averaged over questions. The 50 Mbps point is a monitored rerun. Bold marks the faster path, and subscripts are standard errors.

Table 1 shows that text TTFT does not depend on the link, because text sends only 397 bytes, while checkpoint TTFT falls from 288.80 ms at 10 Mbps to 79.19 ms at 1 Gbps. The sign reversal lies between 50 and 75 Mbps, and all three batches agree in direction at every rate. Both paths overlap transfer with the receiver's prefix preparation, so the checkpoint's extra transfer stays off the TTFT critical path until it outlasts that preparation. Unlike the paper's 8B timings, which shared GPUs with unrelated processes as its appendix states, this sweep and all other rebuttal timings ran on otherwise idle GPUs. Current GPU servers connect at 100–400 Gb/s per network adapter, so even a small fraction of that rate is far above this range.

Longer windows. Table 2 measures receiver-side time after the payload arrives on Qwen3-8B (10 documents, transfer excluded).

LengthText (ms)Checkpoint (ms)Checkpoint payloadText payload
1K tokens122.520.18122.52_{0.18}103.590.13\mathbf{103.59}_{0.13}4.0 MiB4 KiB
4K tokens508.930.70508.93_{0.70}429.120.60\mathbf{429.12}_{0.60}16.0 MiB16 KiB
8K tokens1099.801.651099.80_{1.65}925.351.72\mathbf{925.35}_{1.72}32.0 MiB32 KiB

Table 2: Receiver-side time after the payload arrives, and payload size, versus document length. Bold marks the better path, and subscripts are standard errors over the 10 documents.

Table 2 shows that the checkpoint saves about 15% of receiver time at every length, while its payload grows by about 4 KB per token. At 8K tokens, serializing 32 MiB takes 268 ms at 1 Gbps, more than the 174 ms saved, but under 3 ms at 100 Gb/s, so by this component-level calculation the 8K break-even is about 1.5 Gb/s. Quality declines with length for both text and the checkpoint, and at 8K checkpoint F1 is 4.05, 5.77 and 12.20 points below text (Table 5, post 3/3).

AuthorsOpenReview ↗

Response to Reviewer vfgt 2/3

Q2 & W2: removing receiver features from the 96-byte scorer

We removed the receiver-document features from the deployed address scorer, turning receiver-aware (RA) selection into query-only (QO) selection, with addresses, directories and everything else unchanged.

400 questionsQwen3-4BQwen3-8BLlama-3.1-8B
Target hits, RA / QO310 / 179310 / 179310 / 179
Text F1, RA60.322.40\mathbf{60.32}_{2.40}64.282.25\mathbf{64.28}_{2.25}63.202.29\mathbf{63.20}_{2.29}
Text F1, QO43.662.4043.66_{2.40}45.582.3745.58_{2.37}45.392.3445.39_{2.34}
Checkpoint F1, RA61.602.32\mathbf{61.60}_{2.32}65.642.24\mathbf{65.64}_{2.24}62.522.30\mathbf{62.52}_{2.30}
Checkpoint F1, QO44.022.4044.02_{2.40}45.442.3745.44_{2.37}43.762.3343.76_{2.33}
Checkpoint EM, RA56.002.43\mathbf{56.00}_{2.43}59.252.42\mathbf{59.25}_{2.42}56.252.42\mathbf{56.25}_{2.42}
Checkpoint EM, QO39.252.4239.25_{2.42}39.502.4239.50_{2.42}37.752.4237.75_{2.42}

Table 3: Receiver conditioning under the 96-byte address budget. The selected document changes on 179 of 400 questions: 134 wrong-to-correct, 3 correct-to-wrong and 42 wrong-to-wrong. Every gain is significant. Bold marks the better selector, and subscripts are standard errors.

Table 3 shows that receiver conditioning raises target hits from 44.75% to 77.50% and F1 by 16.7–20.2 points under the deployed budget, not only with BGE-M3 on full text, with nearly equal gains for text and checkpoint delivery. Post 3/3 tests this advantage in harder directories.

Q3 & W2: latency, bytes and recomputation for adaptive caching

We evaluated five reconstruction boundaries, four CacheBlend ratios, native text and static KV on Qwen3-8B, with quality from 800 target-fixed questions and receiver-side timing from 100 questions with five runs each. Table 4 shows representative points of this sweep.

Read pathRecomputeF1After arrival (ms)Total receiver work (ms)Payload
Native text100%71.501.4871.50_{1.48}49.920.1249.92_{0.12}98.010.2498.01_{0.24}265 B
Reconstruction, b = 392%73.011.46\mathbf{73.01}_{1.46}45.400.0245.40_{0.02}93.110.0693.11_{0.06}262 KB
Reconstruction, b = 683%71.711.4771.71_{1.47}42.440.1042.44_{0.10}91.280.2391.28_{0.23}262 KB
Reconstruction, b = 1850%68.131.4968.13_{1.49}26.020.02\mathbf{26.02}_{0.02}73.760.04\mathbf{73.76}_{0.04}262 KB
CacheBlend 30%34%68.781.5268.78_{1.52}112.410.31112.41_{0.31}160.650.41160.65_{0.41}9.44 MB
CacheBlend 84%84%70.841.4970.84_{1.49}112.570.29112.57_{0.29}160.570.40160.57_{0.40}9.44 MB
Static KV, INT80%64.861.5764.86_{1.57}60.260.1660.26_{0.16}108.480.27108.48_{0.27}4.79 MB

Table 4: Representative operating points (Qwen3-8B). "After arrival" assumes the receiver prepared its state in advance, and "total" adds that preparation. Bold marks the best value in each metric column, and subscripts are standard errors.

Table 4 shows three findings. The boundary trades quality for time, with 4.9 more F1 and 26 to 45 ms after arrival from b = 18 to b = 3. All five boundaries take 26–45 ms after arrival, versus 109–113 ms for the four CacheBlend ratios, and transfer 36× fewer bytes. At low budgets CacheBlend scores slightly higher with less recomputation (68.78 F1 at 34% vs. 68.13 at b = 18), though it stays slower and larger. End to end over TCP on 100 questions, reconstruction reaches 77.57 F1 at 82.33 ms TTFT, against 75.32 F1 at 269.08 ms for CacheBlend 84% and 76.39 F1 at 87.31 ms for native text. At ~84% recompute on 800 questions, the two reach similar F1, with reconstruction ahead by 1.44 and 0.87 points on the Qwen models and behind by 0.61 on Llama.

AuthorsOpenReview ↗

Response to Reviewer vfgt 3/3

W1: workload representativeness

The original workload used favorable controls: annotation-centered 64-token windows, a held gold first-hop document and a small directory that always contains the target. We relaxed each control separately, keeping the others fixed.

Windows and document length. Table 5 replaces the 64-token target window with the whole natural paragraph and with 1K–8K documents that embed the complete evidence paragraph among real paragraphs from other questions.

F1Paragraph1K2K4K8K
Qwen3-4B, text72.522.17\mathbf{72.52}_{2.17}66.892.26\mathbf{66.89}_{2.26}68.272.21\mathbf{68.27}_{2.21}65.352.30\mathbf{65.35}_{2.30}63.142.37\mathbf{63.14}_{2.37}
Qwen3-4B, ckpt71.962.1771.96_{2.17}64.602.2764.60_{2.27}64.352.3064.35_{2.30}58.642.3758.64_{2.37}59.092.3659.09_{2.36}
Qwen3-4B, static KV62.802.3662.80_{2.36}25.392.1025.39_{2.10}19.781.8919.78_{1.89}18.091.8818.09_{1.88}19.991.9319.99_{1.93}
Qwen3-8B, text74.822.0274.82_{2.02}73.332.11\mathbf{73.33}_{2.11}68.002.16\mathbf{68.00}_{2.16}68.542.14\mathbf{68.54}_{2.14}72.242.08\mathbf{72.24}_{2.08}
Qwen3-8B, ckpt75.991.98\mathbf{75.99}_{1.98}67.942.2067.94_{2.20}65.402.2365.40_{2.23}64.412.2164.41_{2.21}66.472.1866.47_{2.18}
Qwen3-8B, static KV69.632.1269.63_{2.12}42.132.3042.13_{2.30}37.332.2837.33_{2.28}34.302.2134.30_{2.21}32.002.2132.00_{2.21}
Llama-3.1-8B, text73.822.11\mathbf{73.82}_{2.11}67.792.28\mathbf{67.79}_{2.28}62.612.36\mathbf{62.61}_{2.36}60.952.37\mathbf{60.95}_{2.37}49.192.43\mathbf{49.19}_{2.43}
Llama-3.1-8B, ckpt71.762.1571.76_{2.15}59.492.4159.49_{2.41}56.022.4156.02_{2.41}49.272.4249.27_{2.42}36.992.3536.99_{2.35}
Llama-3.1-8B, static KV67.102.2967.10_{2.29}34.252.3734.25_{2.37}29.012.2229.01_{2.22}22.252.0722.25_{2.07}16.091.7816.09_{1.78}

Table 5: Quality versus document length (same 380 questions and held window). Bold marks the best read path, and subscripts are standard errors.

Table 5 shows that the checkpoint stays within about 2 F1 of text on whole paragraphs, and within 0.9 F1 in a separate 400-question test with both documents as complete paragraphs (64–1,125 tokens). From 1K tokens both text and the checkpoint decline, the checkpoint more, while static KV drops much further, to 16–42 F1.

Held first hop. Table 6 replaces the held gold first-hop document with the top BM25 result for the question.

Checkpoint pipeline F1Qwen3-4BQwen3-8BLlama-3.1-8B
Gold first hop held (paper), RA61.602.3261.60_{2.32}65.642.2465.64_{2.24}62.522.3062.52_{2.30}
BM25 first hop, RA23.612.00\mathbf{23.61}_{2.00}27.942.14\mathbf{27.94}_{2.14}28.382.15\mathbf{28.38}_{2.15}
BM25 first hop, QO13.221.5813.22_{1.58}15.681.6915.68_{1.69}18.061.7918.06_{1.79}

Table 6: Checkpoint pipeline F1 with the first hop held or retrieved (400 questions). Bold marks the better selector under BM25, every RA gain is significant, and subscripts are standard errors.

Table 6 shows that pipeline F1 drops by 34–38 points, because BM25 returns the gold first hop for only 136 of 400 questions, while receiver conditioning still adds 10–12 F1.

Small directory with the target. Table 7 enlarges the directory to the full store of 815 windows, or replaces 30 or all 60 distractors with the BM25 windows that best match the question and the held window among 397,053 2Wiki training-context paragraphs.

Checkpoint F1, 400 questionsTarget hitsQwen3-4BQwen3-8BLlama-3.1-8B
Full store (815 windows), RA23951.892.41\mathbf{51.89}_{2.41}55.192.33\mathbf{55.19}_{2.33}52.562.34\mathbf{52.56}_{2.34}
Full store (815 windows), QO13637.712.3637.71_{2.36}41.592.3141.59_{2.31}39.412.2839.41_{2.28}
30 BM25 hard negatives, RA12639.022.31\mathbf{39.02}_{2.31}43.362.31\mathbf{43.36}_{2.31}42.132.32\mathbf{42.13}_{2.32}
30 BM25 hard negatives, QO12037.812.3437.81_{2.34}41.302.3241.30_{2.32}40.792.3240.79_{2.32}
60 BM25 hard negatives, RA12338.862.32\mathbf{38.86}_{2.32}43.152.31\mathbf{43.15}_{2.31}42.072.31\mathbf{42.07}_{2.31}
60 BM25 hard negatives, QO11036.492.3036.49_{2.30}40.722.3140.72_{2.31}41.222.3141.22_{2.31}

Table 7: Checkpoint F1 in harder directories (400 questions, original directory in Table 3). Bold marks the better selector, and subscripts are standard errors.

Table 7 shows that RA keeps a 13–14 F1 lead in the full store, where its median target rank is 1 against 46 for QO. Hard negatives retrieved with the same question and held window that RA uses make its signal less discriminative, and the lead narrows to 0.85–2.43 F1.

Reviewer_y8QwOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the selection and consumption of latent representations in a write-once, query-later communication setting, where agents share the same frozen language model. The approach separates receiver-conditioned document selection from state reconstruction. At write time, each document is encoded into an INT8 residual checkpoint at an early layer. At read time, the receiver selects a document using its question and an already-held document, inserts the checkpoint at the corresponding layer, and recomputes the upper-layer states under its private prefix. No additional parameters are trained.

The experiments cover document selection, read-path comparisons with the selected document fixed, and end-to-end evaluation over controlled TCP networking on three backbones. In the BGE-M3 selection study, incorporating the receiver's existing document improves F1 by approximately 15.5–17.2 percentage points over query-only selection for both text and checkpoint delivery. With selection fixed, reconstruction improves F1 by approximately 5.1–13.5 points over full static KV reuse while reducing the checkpoint payload by a factor of approximately 32–57. Relative to native-text delivery, end-to-end time to first token (TTFT) decreases by 4.23%–6.91%, while completion latency changes by less than 2% in magnitude.

Critical strengths
  1. The experimental decomposition distinguishes selection gains from reconstruction gains.

    The authors separately evaluate receiver-conditioned selection, read paths with selection held fixed, and the complete system. The selection study includes both text and checkpoint delivery, helping distinguish retrieval improvements from benefits specific to latent communication. Comparisons with static KV reuse, native text, and deranged checkpoints provide reasonably clear evidence that reconstruction recovers useful information.

  2. The paper measures actual transfer with overlapped computation and reports limitations transparently.

    TTFT includes selection, receiver preparation, TCP exchange, payload decoding, and first-token computation. The native-text baseline also overlaps transfer with prefix preparation. TTFT decreases on all three backbones, and repeated timings and stage-level measurements provide stronger evidence than computation or payload estimates alone. The authors also disclose the larger communication volume relative to text, limited completion-latency gains, and Llama's failure to meet some prespecified criteria.

Critical weaknesses
  1. The comparisons do not sufficiently establish the overall trade-off among quality, recomputation, and communication cost.

    Reconstruction still performs approximately 83%–84% of document layer–token computation at read time, compared with approximately 19%–34% for the evaluated CacheBlend configurations. Its higher F1 therefore needs to be interpreted alongside its higher computation budget. The 32–57-fold payload reduction relative to full static KV reuse is valuable, but the main comparison uses the full static cache. The appendix provides only storage estimates for INT8 static KV, without evaluating its quality or latency. Quality, TTFT, and communication-volume curves across reconstruction depths, recomputation ratios, and cache-quantization settings would help establish when this design offers a better overall trade-off.

  2. Cross-dataset generalization is insufficiently validated, and formal evaluation on additional multi-hop QA datasets is needed.

    The main selection, reconstruction, and end-to-end evaluations concentrate on 2WikiMultihopQA. Although HotpotQA and MuSiQue are used in diagnostic or development-stage studies, those results do not replace independent cross-dataset evaluation with settings fixed in advance. It remains unclear whether the gains from receiver-conditioned selection and reconstruction transfer consistently across different question constructions and evidence-dependency patterns.

Questions for authors
  1. Could you jointly report F1, TTFT, and actual communication bytes across several reconstruction depths, CacheBlend recomputation ratios, and quantized-KV transmission settings? Could these comparisons identify the operating conditions under which the proposed design offers advantages over alternative read paths, beyond comparing answer quality at different computation budgets?

  2. With the current hyperparameters, reconstruction boundaries, and retrieval rules fixed, could you conduct formal evaluations on HotpotQA and MuSiQue held-out samples disjoint from the existing diagnostic and development data, reporting selection, reconstruction, and end-to-end results separately? For the complete pipeline, please report answer quality, TTFT, and communication overhead to assess whether the gains are consistent across datasets.

4 comments
AuthorsOpenReview ↗

Response to Reviewer y8Qw 1/2

We thank the reviewer for the constructive review and for recognizing that our design separates selection gains from reconstruction gains. We ran the experiments the review asks for.

W1 & Q1: quality, recomputation and communication together

Table 1 reports nominal recomputation, F1, receiver-side time and payload for each read path on Qwen3-8B, with F1 from 800 target-fixed questions and timing from 100 questions with five runs each.

Read pathRecomputeF1After arrival (ms)Total receiver work (ms)Payload
Native text100%71.501.4871.50_{1.48}49.920.1249.92_{0.12}98.010.2498.01_{0.24}265 B
Reconstruction, b = 392%73.011.46\mathbf{73.01}_{1.46}45.400.0245.40_{0.02}93.110.0693.11_{0.06}262 KB
Reconstruction, b = 683%71.711.4771.71_{1.47}42.440.1042.44_{0.10}91.280.2391.28_{0.23}262 KB
Reconstruction, b = 975%71.411.4871.41_{1.48}37.670.0237.67_{0.02}85.370.0485.37_{0.04}262 KB
Reconstruction, b = 1267%70.111.4970.11_{1.49}34.460.0834.46_{0.08}83.340.2183.34_{0.21}262 KB
Reconstruction, b = 1850%68.131.4968.13_{1.49}26.020.02\mathbf{26.02}_{0.02}73.760.04\mathbf{73.76}_{0.04}262 KB
CacheBlend 15%19%66.741.5366.74_{1.53}108.880.07108.88_{0.07}156.160.08156.16_{0.08}9.44 MB
CacheBlend 30%34%68.781.5268.78_{1.52}112.410.31112.41_{0.31}160.650.41160.65_{0.41}9.44 MB
CacheBlend 50%53%70.751.5070.75_{1.50}109.180.08109.18_{0.08}156.210.10156.21_{0.10}9.44 MB
CacheBlend 84%84%70.841.4970.84_{1.49}112.570.29112.57_{0.29}160.570.40160.57_{0.40}9.44 MB
Static KV, BF160%65.181.5765.18_{1.57}61.710.1561.71_{0.15}109.810.28109.81_{0.28}9.44 MB
Static KV, INT80%64.861.5764.86_{1.57}60.260.1660.26_{0.16}108.480.27108.48_{0.27}4.79 MB

Table 1: Read-path trade-offs on Qwen3-8B. "After arrival" assumes the receiver prepared its state in advance, and "total" adds that preparation. Timing comes from a separate receiver microbenchmark, so these are not end-to-end TTFTs, which Table 2 reports. Bold marks the best value in each metric column, and subscripts are standard errors.

Table 1 locates reconstruction among the alternatives. Moving its boundary from b = 18 to b = 3 buys 4.9 F1 for 19 ms more time after arrival at a fixed 262 KB payload. Every reconstruction depth is faster than every CacheBlend ratio (26–45 vs. 109–113 ms after arrival) and sends 36× fewer bytes, and at ~84% recompute the two reach similar F1, within 1.5 points on all three backbones. At b = 6, reconstruction beats BF16 static KV by 6.53 F1 with less receiver work, and it sends about 18× fewer bytes than INT8 static KV. Against text, it saves about 7% of receiver work with 0.21 more F1, but its payload is about 1,000× larger, so end-to-end TTFT depends on the link.

End-to-end TTFT with real transfer. Table 2 runs five read paths over TCP on the same 100 questions (Qwen3-8B, 1 Gbps).

Read pathF1TTFT (ms)Completion (ms)Bytes per request
Native text76.393.9676.39_{3.96}87.310.0987.31_{0.09}311.7813.28\mathbf{311.78}_{13.28}397
Reconstruction (b = 6)77.573.98\mathbf{77.57}_{3.98}82.330.08\mathbf{82.33}_{0.08}318.2113.74318.21_{13.74}262,427
INT8 static KV67.974.3467.97_{4.34}168.020.74168.02_{0.74}398.4313.78398.43_{13.78}4,792,481
CacheBlend 30%69.424.3469.42_{4.34}268.820.71268.82_{0.71}490.5713.43490.57_{13.43}9,437,616
CacheBlend 84%75.324.0975.32_{4.09}269.080.67269.08_{0.67}497.5413.98497.54_{13.98}9,437,616

Table 2: End-to-end F1, TTFT and bytes per request (100 questions). Bold marks the best value in each column, and subscripts are standard errors.

Table 2 shows that reconstruction has the highest F1 and the lowest TTFT of the five paths. The full-KV paths spend much of their time in transfer: their 4.8–9.4 MB take roughly 110–220 ms, because the application-level throughput we measured on the 1 Gbps link is only about 0.34 Gbps.

Quantized KV transfer. INT8 halves the static payload but does not close the quality gap (Table 3).

800 questions, document fixedQwen3-4BQwen3-8BLlama-3.1-8B
Static KV F1, BF1653.721.6953.72_{1.69}65.181.5765.18_{1.57}64.351.6064.35_{1.60}
Static KV F1, INT853.741.6953.74_{1.69}64.861.5764.86_{1.57}64.121.6064.12_{1.60}
Reconstruction F167.261.58\mathbf{67.26}_{1.58}71.711.47\mathbf{71.71}_{1.47}69.471.55\mathbf{69.47}_{1.55}
Payload, INT8 static / checkpoint4.79 MB / 164 KB4.79 MB / 262 KB3.97 MB / 245 KB

Table 3: INT8 static KV stays within 0.33 F1 of BF16 static KV but below reconstruction, while sending 16–29× the checkpoint's bytes. Bold marks the highest F1, and subscripts are standard errors.

AuthorsOpenReview ↗

Response to Reviewer y8Qw 2/2

W1 & Q1 (continued): when reconstruction has the advantage

Against static KV and CacheBlend, reconstruction has the stronger operating points: higher F1 than static KV on every target-fixed test (+4 to +18 on short documents, +21 to +45 at 1K–8K tokens), and less time and fewer bytes than CacheBlend at every ratio. Against text, the deciding condition is the link, because only the checkpoint's transfer grows as the link slows. Table 4 measures end-to-end TTFT on Qwen3-8B at six link rates, with three independent batches of the same 100 questions per rate.

Link rateText TTFT (ms)Ckpt TTFT (ms)
1 Gbps84.590.0584.59_{0.05}79.190.05\mathbf{79.19}_{0.05}
224 Mbps86.340.0786.34_{0.07}81.370.06\mathbf{81.37}_{0.06}
75 Mbps85.210.0685.21_{0.06}80.820.05\mathbf{80.82}_{0.05}
50 Mbps85.040.05\mathbf{85.04}_{0.05}93.400.2093.40_{0.20}
25 Mbps85.870.06\mathbf{85.87}_{0.06}154.200.09154.20_{0.09}
10 Mbps85.290.06\mathbf{85.29}_{0.06}288.800.11288.80_{0.11}

Table 4: TTFT versus link rate (Qwen3-8B, 2Wiki), per-question medians of 15 timings averaged over questions. The 50 Mbps point is a monitored rerun. Bold marks the faster path, and subscripts are standard errors.

Table 4 shows that the checkpoint has lower TTFT from 75 Mbps up and higher TTFT at 50 Mbps and below, consistently in all three batches. Current GPU servers connect at 100–400 Gb/s per network adapter, so even a small fraction of that rate is far above this crossover. Text keeps higher F1 for long documents (4–12 points at 8K), and at 8K INT8 static KV needs less receiver time (280 vs. 925 ms), but static KV scores 34.5 F1 lower there and the INT8 payload is 18× larger.

W2 & Q2: held-out HotpotQA and MuSiQue

We froze every setting (hyperparameters, boundary, window rule, 96-byte address, directory recipe and selectors) and evaluated 400 questions per dataset from the official training splits, none used by any earlier experiment, on all three backbones. RA and QO denote receiver-aware and query-only selection.

F1, 400 questions eachHotpotQA 4B8BLlamaMuSiQue 4B8BLlama
Text, target fixed71.832.04\mathbf{71.83}_{2.04}77.741.85\mathbf{77.74}_{1.85}70.682.05\mathbf{70.68}_{2.05}60.422.24\mathbf{60.42}_{2.24}71.882.04\mathbf{71.88}_{2.04}59.682.2559.68_{2.25}
Checkpoint, target fixed70.752.0970.75_{2.09}77.611.8777.61_{1.87}68.682.0968.68_{2.09}59.432.2959.43_{2.29}70.022.0870.02_{2.08}60.962.22\mathbf{60.96}_{2.22}
Static KV, target fixed64.882.1664.88_{2.16}72.192.0272.19_{2.02}64.222.1964.22_{2.19}41.172.2941.17_{2.29}53.062.2753.06_{2.27}48.862.3248.86_{2.32}
RA checkpoint pipeline67.352.14\mathbf{67.35}_{2.14}74.071.98\mathbf{74.07}_{1.98}66.142.14\mathbf{66.14}_{2.14}42.452.33\mathbf{42.45}_{2.33}51.782.32\mathbf{51.78}_{2.32}45.592.31\mathbf{45.59}_{2.31}
QO checkpoint pipeline66.512.1666.51_{2.16}71.932.0471.93_{2.04}64.392.1864.39_{2.18}34.722.2334.72_{2.23}42.492.3142.49_{2.31}38.662.2538.66_{2.25}
Target hits, RA / QO358 / 337274 / 210

Table 5: Held-out evaluation. Bold marks the best target-fixed read path and the better selector, and subscripts are standard errors. All three MuSiQue RA gains and the HotpotQA gains on Qwen3-8B and Llama are significant.

Table 5 shows that the reconstruction result transfers to both datasets: the checkpoint beats static KV everywhere and stays within 2 F1 of text. The selection result transfers in direction, with RA adding 6.9–9.3 F1 on two-hop MuSiQue questions and 0.8–2.1 on HotpotQA, where QO already finds the target in 337 of 400 questions and the selectors differ on only 39.

100 questions eachHotpotQA textHotpotQA ckptMuSiQue textMuSiQue ckpt
F181.553.5081.55_{3.50}82.353.42\mathbf{82.35}_{3.42}54.494.7054.49_{4.70}55.394.71\mathbf{55.39}_{4.71}
TTFT (ms)90.700.0990.70_{0.09}85.110.09\mathbf{85.11}_{0.09}86.860.0586.86_{0.05}81.320.05\mathbf{81.32}_{0.05}
Completion (ms)261.9210.02261.92_{10.02}255.389.73\mathbf{255.38}_{9.73}256.008.63256.00_{8.63}251.028.45\mathbf{251.02}_{8.45}
Bytes per request389262,419383262,413

Table 6: End-to-end results on the first 100 held-out questions of each dataset (Qwen3-8B, 1 Gbps). Bold marks the better path within each dataset, and subscripts are standard errors.

In Table 6, the TTFT gain (5.6 and 5.5 ms) matches the 2Wiki result at the same link rate (5.0 ms, Table 2), and checkpoint F1 is 0.8–0.9 points above text. These absolute TTFTs, like those in Tables 2 and 4, are lower than the paper's 8B timings because those shared GPUs with unrelated processes, as its appendix states, while all rebuttal timings ran on otherwise idle GPUs.

I thank the authors for their responses, which have addressed my concerns. I have therefore decided to raise my score, and I defer to the AC for the final decision.

Thank you for reading our responses and for raising your score. Your two questions shaped the revision. We appreciate your careful and constructive review.

Reviewer_QcHaOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks how to preprocess a document store so that a receiver, given a question and a document it already holds, can (1) pick which stored document to fetch (selection), and (2) place that document's representation into its own KV cache. Each document is written once as an INT8 residual checkpoint at layer b (6 of 36 for Qwen3), plus a 96-byte lexical address. At read time the receiver scores addresses with its question and its own document, transfers the selected checkpoint while prefilling its own prompt, inserts the checkpoint at layer b, and recomputes layers b..L−1 under its own context. In effect, this replaces text RAG with RAG over a cache of partial prefills.

Selection is query expansion, so the main ideas are pipelining the KV transfer with the receiver's prefill, and the upper-layer recompute. Experiments use 2WikiMultihopQA with Qwen3-4B/8B and Llama-3.1-8B.

Critical strengths

Preregistered criteria, most conditions reported in appendix The appendix is honest about systems costs (byte overhead, recompute fraction) released code matches the method.

Critical weaknesses

The benefit over text is minor. End to end, TTFT drops 4.2–6.9% (6–12 ms; Table 3) and F1 is within −0.82 to +1.88 of text. For Llama, the paper misses its own preregistered F1, EM, and TTFT targets (App. H, Table 14), which the abstract doesn't mention. Meanwhile the checkpoint moves 413–661× the bytes of text (Table 13). It doesn't seem worth the complexity.

The paper claims to avoid repeated prefill. It doesn't. With b = 6 of 36, reconstruction reruns 83–84% of the document's layer computation (App. J, Table 16). The only saving is the lower b layers: the first-token forward pass shrinks by 8–20 ms (Table 13).

Selection is query expansion. Querying with question + own document is standard multi-hop retrieval (MDR, Baleen, IRCoT). The gains are the same for text and checkpoint delivery (Table 1), so they are not specific to latent communication.

The benchmark is too simple. The hard parts of multi-hop are deciding what to fetch (the query for the next hop), focusing on the relevant subset of the retrieved document, and long context. None of these are exercised: two-hop questions, the receiver already holds the first-hop document, one document is fetched, and each document is a 64-token window centered on the supporting sentences (§4.1, App. C). The experiment is really testing fact transfer and detection, not reasoning.

The retrieval setup is too easy. 56 of the 60 distractors per directory are random; only 4 are lexically close to the question (App. C). A random distractor is effectively a document with zero overlap with the receiver's document, so conditioning on it trivially separates the target.

Tiny documents hide the true costs. At 64 tokens prefill is cheap, so the benefit is capped. The costs that grow with length don't show up: checkpoint storage is ~2.5–4 KB per token versus a few bytes for token IDs, and transfer volume (413–661× text) is only tested on a single 1 Gbps / 5 ms link.

Questions for authors

How do TTFT and F1 change with longer documents (1K–8K tokens), several fetched documents, and slower links? At what point does the byte overhead erase the gain? How does reconstruction compare with a compute-matched baseline, e.g., CacheBlend at ~84% recompute, or text prefill with prefix caching? Does selection still help with distractors chosen to be close to question + own document, or with more than two hops where the next query has to be generated? Why does the abstract call quality "comparable to native text" when Llama misses the preregistered margins?

6 comments
AuthorsOpenReview ↗

Response to Reviewer QcHa 1/4

We thank the reviewer for the detailed review. We number the questions in order: Q1 longer documents, several documents and slower links, Q2 compute-matched baselines, Q3 hard distractors and more than two hops, and Q4 the abstract's quality claim.

W1: the benefit over text

Reconstruction shortens the first-token forward by 16–19%, and the receiver's own preparation dilutes this to 4–7% of TTFT. The receiver first prepares its own prompt (instructions, question and held document), which every delivery method must do, and then runs the first-token forward over the delivered document. Table 1 shows that the saving comes from the second stage.

400 questions, 1 Gbps, msQwen3-4BQwen3-8BLlama-3.1-8B
Prompt preparation, text49.80.01\mathbf{49.8}_{0.01}97.90.40\mathbf{97.9}_{0.40}69.90.05\mathbf{69.9}_{0.05}
Prompt preparation, ckpt50.60.0150.6_{0.01}102.90.41102.9_{0.41}74.70.0674.7_{0.06}
First-token forward, text52.10.00352.1_{0.003}101.90.42101.9_{0.42}71.30.0571.3_{0.05}
First-token forward, ckpt43.80.003\mathbf{43.8}_{0.003}82.40.36\mathbf{82.4}_{0.36}57.90.05\mathbf{57.9}_{0.05}
TTFT, text104.30.01104.3_{0.01}207.20.83207.2_{0.83}148.90.08148.9_{0.08}
TTFT, ckpt97.10.01\mathbf{97.1}_{0.01}195.20.79\mathbf{195.2}_{0.79}142.60.09\mathbf{142.6}_{0.09}
Questions where ckpt is faster400 / 400399 / 400400 / 400

Table 1: Mean stage durations from the paper's end-to-end appendix. Bold marks the faster path, and subscripts are standard errors. TTFT also includes selection and transfer waits, so the stages need not sum to it.

The forward pass saves 8.3–19.5 ms, while preparation, about half of TTFT, is 0.8–5.0 ms longer with checkpoints.

Reconstruction recovers most or all of the quality that static KV reuse loses. Static KV reuse loses 6.3–12.9 F1 to text on 800 questions (Section 4.3). Reconstruction closes 105%, 103% and 74% of this gap, and end to end its F1 is within −0.82 to +1.88 points of text.

The method adds no trained component, unlike Cache-to-Cache, XKV, KV Packet and SemPIC. Each document is written once as an INT8 checkpoint with a 96-byte address, and the receiver reruns the upper layers with the same frozen backbone. It sends 32–57× fewer bytes than static KV and 36× fewer than CacheBlend (post 2/4). The 413–661× ratio to text reflects how small text is (under 400 bytes), and the checkpoint's transfer overlaps with the receiver's preparation (post 4/4).

Q4: what "comparable" in the abstract states

The abstract reports Llama's 4.23% TTFT reduction and its −0.82 F1 difference, the lower end of the reported range. "Comparable" describes these measured differences: on all three backbones, every F1 and EM difference from text is below 2 points and none is significant. The preregistered criteria ask the stricter question of whether a loss above 1 F1 or 2 EM points can be ruled out. For Llama it cannot, so non-inferiority is not established, and its TTFT meets the 1 ms criterion but not the 5% target. Table 2 lists all eight criteria, which Section 4.4 and Appendix H already report.

400 questions, 1 GbpsQwen3-4BQwen3-8BLlama-3.1-8B
TTFT reduction, ms (≥ 1)7.210.017.21_{0.01}12.050.1612.05_{0.16}6.300.066.30_{0.06}
TTFT reduction, % (target ≥ 5)6.910.016.91_{0.01}5.810.075.81_{0.07}4.230.044.23_{0.04} ✗
Completion latency change, % (< 5)−0.131.10-0.13_{1.10}+0.440.77+0.44_{0.77}−1.720.75-1.72_{0.75}
P95 completion ratio (≤ 1.05)0.9970.0010.997_{0.001}0.9950.0220.995_{0.022}1.0090.0041.009_{0.004}
Ckpt − text F1 (margin −1)+1.881.16+1.88_{1.16}+1.150.77+1.15_{0.77}−0.820.66-0.82_{0.66} ✗
Ckpt − text EM (margin −2)+0.751.21+0.75_{1.21}+1.751.15+1.75_{1.15}−0.750.89-0.75_{0.89} ✗
F1 gain over no communication+22.562.49+22.56_{2.49}+21.182.51+21.18_{2.51}+19.172.42+19.17_{2.42}
F1 gain over a deranged checkpoint+22.342.53+22.34_{2.53}+25.362.52+25.36_{2.52}+25.812.39+25.81_{2.39}
Preregistered verdictpass (8/8)pass (8/8)fail (5/8)

Table 2: Preregistered end-to-end criteria. Subscripts are standard errors, and ✗ marks a missed criterion (Appendix H).

AuthorsOpenReview ↗

Response to Reviewer QcHa 2/4

W2 & Q2: repeated prefill and compute-matched baselines

Reconstruction reuses the lower b layers, about 16–17% of document layer–token work, and recomputes the other 83–84% under the receiver's context. It reduces prefill but does not remove it. With its check layer, CacheBlend at ~84% recompute redoes 83.8–84.3% of this work, close to reconstruction's 83.3–84.4%.

F1, 800 questions, document fixedQwen3-4BQwen3-8BLlama-3.1-8B
CacheBlend, ~84% recompute65.821.6065.82_{1.60}70.841.4970.84_{1.49}70.081.54\mathbf{70.08}_{1.54}
Reconstruction67.261.58\mathbf{67.26}_{1.58}71.711.47\mathbf{71.71}_{1.47}69.471.5569.47_{1.55}

Table 3: F1 at matched recomputation (800 questions, document fixed). Bold marks the higher F1, and subscripts are standard errors.

Table 3 shows that the two reach similar F1, within 1.5 points on every backbone. Since their costs differ, Table 4 measures five read paths end to end (100 questions, Qwen3-8B, TCP at 1 Gbps).

Read pathF1TTFT (ms)Completion (ms)Bytes per request
Native text76.393.9676.39_{3.96}87.310.0987.31_{0.09}311.7813.28\mathbf{311.78}_{13.28}397
Reconstruction (b = 6)77.573.98\mathbf{77.57}_{3.98}82.330.08\mathbf{82.33}_{0.08}318.2113.74318.21_{13.74}262,427
INT8 static KV67.974.3467.97_{4.34}168.020.74168.02_{0.74}398.4313.78398.43_{13.78}4,792,481
CacheBlend 30%69.424.3469.42_{4.34}268.820.71268.82_{0.71}490.5713.43490.57_{13.43}9,437,616
CacheBlend 84%75.324.0975.32_{4.09}269.080.67269.08_{0.67}497.5413.98497.54_{13.98}9,437,616

Table 4: End-to-end F1, TTFT and bytes per request. Bold marks the best value in each column, and subscripts are standard errors.

Table 4 shows that reconstruction has the highest F1 and the lowest TTFT, with 36× fewer bytes than CacheBlend. Absolute TTFTs differ from Table 1. The paper's 8B runs shared their GPUs with unrelated processes, as its appendix states, and their compute stages were 2.2–3.3× slower than in our runs on otherwise idle GPUs, with network stages unchanged. On the same 100 questions the relative reduction is similar (6.0% shared, 5.7% here, 6.4% in Table 11 of post 4/4).

Prefix-cached text. A prefix cache reuses only the shared instructions, since the question precedes the held document, while our text baseline already prefills the whole receiver prefix during transfer.

W3: selection is query expansion

Receiver-conditioned retrieval builds on MDR, Baleen and IRCoT, and we do not claim a new retriever. We test it where documents are written before any question exists and the receiver sees only a 96-byte address, by removing the receiver features from the deployed scorer, which turns receiver-aware (RA) selection into query-only (QO) selection.

F1, 400 questionsQwen3-4BQwen3-8BLlama-3.1-8B
Text delivery, RA60.322.40\mathbf{60.32}_{2.40}64.282.25\mathbf{64.28}_{2.25}63.202.29\mathbf{63.20}_{2.29}
Text delivery, QO43.662.4043.66_{2.40}45.582.3745.58_{2.37}45.392.3445.39_{2.34}
Checkpoint delivery, RA61.602.32\mathbf{61.60}_{2.32}65.642.24\mathbf{65.64}_{2.24}62.522.30\mathbf{62.52}_{2.30}
Checkpoint delivery, QO44.022.4044.02_{2.40}45.442.3745.44_{2.37}43.762.3343.76_{2.33}

Table 5: The deployed scorer with (RA) and without (QO) receiver features. Target hits are 310 vs. 179 of 400 on every backbone. Bold marks the higher F1, subscripts are standard errors, and every RA gain is significant.

Table 5 shows that receiver conditioning finds 131 more targets and adds 16.7–20.2 F1, by similar amounts for text and checkpoint delivery. The gain is therefore an effect of receiver conditioning under a fixed address budget, independent of how the document is delivered.

W4 & Q1: harder benchmark settings

We relaxed each simplification separately: the held first hop here, length and several documents in post 3/4, and more hops in post 4/4.

Retrieved first hop. Replacing the held first-hop document with the top BM25 result lowers pipeline F1 by 34–38 points, because BM25 returns the gold first hop for only 136 of 400 questions (Table 6). Receiver conditioning still adds 10–12 F1.

Checkpoint pipeline F1Qwen3-4BQwen3-8BLlama-3.1-8B
Gold first hop held (paper), RA61.602.3261.60_{2.32}65.642.2465.64_{2.24}62.522.3062.52_{2.30}
BM25 first hop, RA23.612.00\mathbf{23.61}_{2.00}27.942.14\mathbf{27.94}_{2.14}28.382.15\mathbf{28.38}_{2.15}
BM25 first hop, QO13.221.5813.22_{1.58}15.681.6915.68_{1.69}18.061.7918.06_{1.79}

Table 6: Checkpoint pipeline F1 with the first hop held or retrieved (400 questions). Bold marks the better selector under BM25, every RA gain is significant, and subscripts are standard errors.

AuthorsOpenReview ↗

Response to Reviewer QcHa 3/4

W4 & Q1 (continued): longer documents and several documents

Longer documents. With the target fixed, the paragraph setting stores whole natural paragraphs, and the 1K–8K setting embeds the complete evidence paragraph among real paragraphs from other questions.

F1Paragraph1K2K4K8K
Qwen3-4B, text72.522.17\mathbf{72.52}_{2.17}66.892.26\mathbf{66.89}_{2.26}68.272.21\mathbf{68.27}_{2.21}65.352.30\mathbf{65.35}_{2.30}63.142.37\mathbf{63.14}_{2.37}
Qwen3-4B, ckpt71.962.1771.96_{2.17}64.602.2764.60_{2.27}64.352.3064.35_{2.30}58.642.3758.64_{2.37}59.092.3659.09_{2.36}
Qwen3-4B, static KV62.802.3662.80_{2.36}25.392.1025.39_{2.10}19.781.8919.78_{1.89}18.091.8818.09_{1.88}19.991.9319.99_{1.93}
Qwen3-8B, text74.822.0274.82_{2.02}73.332.11\mathbf{73.33}_{2.11}68.002.16\mathbf{68.00}_{2.16}68.542.14\mathbf{68.54}_{2.14}72.242.08\mathbf{72.24}_{2.08}
Qwen3-8B, ckpt75.991.98\mathbf{75.99}_{1.98}67.942.2067.94_{2.20}65.402.2365.40_{2.23}64.412.2164.41_{2.21}66.472.1866.47_{2.18}
Qwen3-8B, static KV69.632.1269.63_{2.12}42.132.3042.13_{2.30}37.332.2837.33_{2.28}34.302.2134.30_{2.21}32.002.2132.00_{2.21}
Llama-3.1-8B, text73.822.11\mathbf{73.82}_{2.11}67.792.28\mathbf{67.79}_{2.28}62.612.36\mathbf{62.61}_{2.36}60.952.37\mathbf{60.95}_{2.37}49.192.43\mathbf{49.19}_{2.43}
Llama-3.1-8B, ckpt71.762.1571.76_{2.15}59.492.4159.49_{2.41}56.022.4156.02_{2.41}49.272.4249.27_{2.42}36.992.3536.99_{2.35}
Llama-3.1-8B, static KV67.102.2967.10_{2.29}34.252.3734.25_{2.37}29.012.2229.01_{2.22}22.252.0722.25_{2.07}16.091.7816.09_{1.78}

Table 7: Quality versus document length on the same 380 questions and held window. Bold marks the best read path, and subscripts are standard errors.

Table 7 shows two regimes. On short paragraphs the checkpoint stays within about 2 F1 of text, and within 0.9 F1 in a separate 400-question test with both documents as complete paragraphs (64–1,125 tokens). From 1K tokens both text and the checkpoint decline, the checkpoint more, while static KV drops much further, to 16–42 F1. On the latency side, the checkpoint still saves about 15% of receiver time from 1K to 8K tokens, and with its 32 MiB payload at 8K the break-even is about 1.5 Gb/s by a component-level calculation.

Several documents. Checkpoints of different documents are concatenated along the token axis and reconstructed in one pass. On 152 MuSiQue 3-hop questions with the second- and third-hop windows (D2 and D3) fixed rather than retrieved, two independently written checkpoints exceed static KV by 9.6–18.8 F1 and stay within 0.5–4.2 points of text. Table 8 lets each selector fetch its top two documents.

Text F12Wiki top-12Wiki top-2MuSiQue top-1MuSiQue top-2
Qwen3-4B, RA60.322.40\mathbf{60.32}_{2.40}62.292.32\mathbf{62.29}_{2.32}20.762.9520.76_{2.95}30.133.4230.13_{3.42}
Qwen3-4B, QO43.662.4043.66_{2.40}43.692.4043.69_{2.40}26.623.29\mathbf{26.62}_{3.29}33.383.51\mathbf{33.38}_{3.51}
Qwen3-8B, RA64.282.25\mathbf{64.28}_{2.25}64.792.26\mathbf{64.79}_{2.26}25.633.2125.63_{3.21}38.223.6538.22_{3.65}
Qwen3-8B, QO45.582.3745.58_{2.37}44.982.3844.98_{2.38}32.403.50\mathbf{32.40}_{3.50}40.083.61\mathbf{40.08}_{3.61}
Llama-3.1-8B, RA63.202.29\mathbf{63.20}_{2.29}63.412.29\mathbf{63.41}_{2.29}26.863.2526.86_{3.25}36.983.58\mathbf{36.98}_{3.58}
Llama-3.1-8B, QO45.392.3445.39_{2.34}45.452.3345.45_{2.33}30.923.39\mathbf{30.92}_{3.39}36.843.5236.84_{3.52}
Gold hits, RA / QO310 / 179342 / 192D2 72 / 26, D3 45 / 62both: 47 / 24

Table 8: Text F1 with one and two fetched documents. 2Wiki has one gold document to fetch, MuSiQue two. Bold marks the better selector, and subscripts are standard errors.

RA adds the held document's lexical features to the question, so it favors documents that share entities with the receiver's evidence. On 2Wiki the missing document is such a document, and fetching more does not reproduce RA's choice: RA with one checkpoint beats QO with two by 18.0–21.2 checkpoint F1 (61.6–65.6 vs. 43.6–44.5) at half the bytes, and by 16.6–19.3 F1 with text. On MuSiQue the held document links to D2, while the answer is one hop further, in D3. RA's first fetch is gold more often than QO's (117 vs. 88) but more often D2 than D3, and its top-1 F1 is lower. At top-2 it retrieves the complete chain twice as often, and its D3 coverage is similar (80 vs. 84). Each fetched document adds one checkpoint, so the payload doubles.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.