acceptodds
Under review as a conference paper at ICLR 2027

Attuner: Recomputation-Free KV Cache Reuse via Query-Side Adaptation

15 views

Abstract

Large language model (LLM) agents repeatedly load reusable content, such as skills, documents, and memory entries, into the current context. Re-encoding this content for every request wastes computation. Position-independent caching (PIC) alleviates this by encoding each artifact independently and reusing its key-value (KV) states at arbitrary positions, but it incurs a quality loss relative to full-context prefill. Existing methods repair this loss by restoring global position IDs or recomputing selected tokens. In this work, we isolate the source of the loss, finding that the positional mismatch has minor effect, and independently cached artifacts retain faithful representations: reading a provided artifact stays largely accurate, and performance degrades only when the model must select among multiple artifacts. Moreover, replacing PIC's attention scores with full-prefill scores recovers performance with the cached KV unchanged, localizing the failure to the attention rather than KV recomputation. Motivated by this, we propose Attuner, a query-side adaptation method that learns to read a frozen artifact cache. Attuner inserts low-rank adapters into the query projections and is trained by distilling full-prefill distribution into the student. It trains fewer than 0.05% of the model parameters and, at inference, requires neither cache recomputation nor a full-context reference. On Qwen3-4B and Qwen3-8B across seven benchmarks covering skills, documents, memory, and code, Attuner substantially outperforms prior PIC baselines in both in-domain and out-of-domain settings, matches full-context prefill quality while providing up to speedup.

Primary area:
foundation or frontier models, including LLMs
Keywords:
KV Reuse, LLM inference, Position-Independent Caching, Agent

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_Uz9uOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies how to reuse independently computed KV caches without losing too much answer quality. Recomputing the full context can recover quality, but also reduces the benefit of caching. Attuner keeps the cached keys and values unchanged and trains small adapters on the query projections. A full-prefill model provides the training targets, but is not needed at inference time.

Critical strengths

1. The problem is interesting, and the method directly addresses it.
I find the main question useful: does recovering quality require recomputing the cached representations, or can the model learn to read the existing caches better? Attuner tests the second option without changing the caches. This makes the contribution more than another way to speed up a particular task. The oracle experiment also shows that quality can be recovered while keeping the artifact KV fixed. It motivates the method, although it does not establish a unique cause of the quality loss.

2. The main results support a useful quality and latency trade-off.
The evaluation covers two model sizes and different types of cached content. Relative to PIC, the average in-domain scores improve by 5.13 and 7.77 points, while time to first token remains 1.79x and 2.45x faster than full prefill. A shared adapter also improves performance on held-out tasks. The Full-adaptation control is useful: the much smaller gain under Full suggests that the adapter is mainly helping with independent-cache reading rather than simply improving the base model. Attuner is not best on every metric, but the trade-off it offers is meaningful.

Critical weaknesses

The robustness of the trained reader to irrelevant artifacts is still unclear.
The main evaluation directly supplies the annotated relevant artifacts, without additional distractors. The paper has a distractor experiment, but it compares Full and PIC rather than the trained Attuner. The current results therefore show that Attuner reads a relevant set of materials better; they do not yet show whether it can maintain this advantage when irrelevant materials are mixed in.

This matters because identifying which cached content to use is part of a reusable cache-reading method. Adding the trained adapter to the existing distractor experiment would clarify the scope of the result.

Questions for authors

Could the authors evaluate the same trained adapter in the existing distractor setting? In particular, does its gain over PIC remain as more irrelevant artifacts are added, and how does the gap to Full change?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Uz9u

We sincerely thank the reviewer for recognizing the value of our research question and ATTUNER’s quality–latency trade-off. We appreciate your thoughtful feedback and hope the additional evaluation below addresses your concern about robustness to distractors.

Q1: Could the authors evaluate the same trained adapter in the existing distractor setting? In particular, does its gain over PIC remain as more irrelevant artifacts are added, and how does the gap to Full change?

A1: As discussed in Section 3.1, we distinguish artifact discovery from evidence utilization. We view evidence utilization as a starting point for robust cache reuse. At the same time, artifact discovery is important in practical retrieval settings, where retrieved materials are not accompanied by ground-truth relevance labels. Our focus here is on repairing the reading of cached evidence as a foundation on which broader discovery mechanisms can build.

To address the reviewer's question, we evaluate the agent-relevant skill-use setting built on BigCodeBench, where the model receives both relevant skills and distractors. We use the same Qwen3-4B adapter as in the paper, without additional training. Each condition retains all relevant skills and adds distractors to reach a target total of 4, 8, 12, or 16 skills. The paper adapter was trained with relevant artifacts only. To assess whether training with distractors improves recovery, we additionally train a separate adapter from the base model for each condition, using 4,000 synthetic examples built from training-side skills disjoint from evaluation.

As shown in Table 1, ATTUNER consistently improves over PIC when relevant skills are mixed with distractors, although its gap to Full widens as distraction increases. Training under the corresponding conditions further improves performance at all four skill counts, supporting effective cache reading when retrieved materials include irrelevant content.

Table 1. BigCodeBench pass@1 (%).

Skills numberFullPICPIC + paper adapterPIC + new 4K adapter
443.6337.7842.5845.13
842.8832.0839.7343.18
1242.5828.7938.2342.13
1642.5824.4434.9338.83

We also evaluate inference efficiency. Since PIC primarily accelerates prompt processing, we measure wall-clock TTFT, including cache lookup, CPU-to-GPU transfer, KV assembly, online prefill, and first-token selection. The paper adapter retains essentially the same TTFT as PIC. As more skills are added, a larger portion of the prompt can be reused from cache, increasing the speedup over Full.

Table 2. BigCodeBench warm-cache efficiency. Token counts are per-request means (rounded); TTFT is the median in ms.

Skills numberPrompt tokensCached artifact tokensFull TTFTPIC TTFTAdapter TTFTSpeedup over Full
46,4236,252586.15106.11107.125.52×
810,27810,1081063.90160.89160.966.61×
1214,16913,9991626.86215.69216.297.54×
1618,05517,8842316.32268.43268.468.63×

From the experiments above, we can see that ATTUNER improves cache reading in this mixed-context setting while retaining near-PIC warm-cache latency. Our contribution is to establish evidence-utilization repair through query-side adaptation as a foundation for effective reuse of independently cached artifacts. Robust artifact discovery may additionally benefit from document-level supervision or selection mechanisms, particularly when evidence is interdependent across artifacts. Please see our response to Reviewer GkER, A2, for the attention diagnostics and discussion of this direction.

We thank the reviewer for highlighting this important point, and we have clarified the paper's scope and the directions for more robust artifact discovery in the revised manuscript (Line 1386, page 26).

Thank you for testing the original adapter with distractors. It still beats PIC, but its gap to full prefill grows from about 1 to nearly 8 pass@1 points as more skills are added. I’m keeping my rating at 3 because identifying which cached material to use is an important part of reading the cache.

Thank you for reviewing the additional results. We would like to clarify that the original adapter was trained only on relevant artifacts, without distractors, so some degradation under this distribution shift is expected, although it continues to outperform PIC at every tested level.

To examine adaptation to this setting, we additionally trained adapters with distractors. As shown in Table 1 PIC + new 4K adapter in Response to Reviewer Uz9u, these improve performance at all four skill counts, achieving results close to or above Full with 4–12 skills and narrowing the remaining gap at 16 skills.

We sincerely thank you for your thoughtful feedback and for helping us strengthen the evaluation and clarify the scope of our work.

Reviewer_GkEROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper is addressing the problem of re-using documents within agentic loops. These documents need to be re-processed for every pre-fill as their previous context changes the token representations. Processing them once and re-using would bring many efficiency gains however, there are significant accuracy drops. The paper performs a sequence of quantitative analysis to find the source of accuracy drop and proposes a method to fix it. The issue turns out to be the attention routing and the KV cache can be re-used. They adapt the query matrices using LoRA and it works well.

Critical strengths

I believe the quantitate study of finding the source of accuracy drop is conclusive and impressive. I personally liked separating the impact of attention and the value vectors of tokens. This results in a simple and effective solution. Overall, a clean, conclusive and important finding.

Resulting method of LoRA adaptation of the model for cached KV over only the query matrices is simple, effective and efficient.

The empirical study conclusively shows strong and promising results. The paper would have an impact in the community.

Critical weaknesses

Within the original study of showing that position encoding is not a problem, why is PIC-Pos only for 2.b included. I believe to completely conclude that it is not a problem, we should see PIC-Pos as part of 2.a with the distractors in the play. I prefer to see this added to the Figure 2.a.

Although the empirical study follows the baseline. I would appreciate a more realistic agentic setting where authors evaluate the method in its entirety and simply compare wall-clock time and accuracy with and without the ATTUNER. I at least expect to see MuSiQue in a more realsitics setting with all candidate documents including the distractors.

Questions for authors

See strengths and weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer GkER (1/2)

We sincerely thank the reviewer for recognizing the value of our analysis and the simplicity, effectiveness, and efficiency of ATTUNER. We appreciate your constructive suggestions and hope the responses below address your concerns.

Q1: Please include PIC-Pos in the distractor setting of Fig. 2(a).

A1: Position alignment remains insufficient under distractors.

We initially evaluated PIC-Pos with relevant artifacts only to isolate whether position alignment can resolve PIC's loss even without distractors. Following your suggestion, we have also evaluated it under the distractor settings of Fig. 2(a) and included the results in the updated manuscript (Line 204, page 4). These results support our conclusion that position alignment alone remains insufficient to close the gap to Full under distractors.

Table 1. MuSiQue Answer F1 (%).

DistractorsFullPICPIC-Pos
061.0148.6449.30
254.7334.6734.90
451.3726.4927.60
847.0214.6514.40
1642.156.186.10

Table 2. BigCodeBench pass@1 (%). Target counts include relevant and distracting skills.

Target total skillsFullPICPIC-Pos
443.4838.2336.88
843.3332.3827.29
1243.1828.9424.59
1640.9323.5423.09

Q2: Could you evaluate quality and wall-clock time with and without ATTUNER in a more realistic setting, including all MuSiQue candidate documents?

We consider the BigCodeBench skill-use setting particularly suitable for PIC, because its self-contained skills can be selected and used independently, matching the premise of preparing reusable artifacts without conditioning on one another. Please refer to our response to Reviewer Uz9u for the quality, retraining, and efficiency results under increasing numbers of distractors in this setting.

Comparing to BigcodeBench, MuSiQue presents a more demanding case: answering requires combining evidence across documents, so attention allocation can depend on intermediate reasoning. We report the requested MuSiQue evaluation below.

A2-1: Quality, training, and efficiency with distractors.

We evaluate MuSiQue with a controlled sweep that retains all supporting documents and adds nested distractor sets, comparing Full, PIC, and PIC + ATTUNER on identical inputs within each condition. Since the paper adapter was trained with relevant artifacts only, we train one fresh adapter per distractor level on the same 8,192 MuSiQue questions, holding the architecture, distillation objective, and training budget fixed.

PIC primarily accelerates prompt processing, so we also report warm-cache TTFT. Timing includes cache lookup, CPU-to-GPU transfer, KV assembly, online prefill, and first-token selection on a single H20 with batch size one. Query updates are merged into the projection weights; offline cache construction and subsequent decoding are outside this interval.

Table 3. MuSiQue Answer F1 (%) / median warm-cache TTFT (ms), 2,417 requests per condition. ATTUNER F1 uses the adapter trained for each row; TTFT uses the paper adapter. Speedup refers to TTFT.

Added distractorsPrompt tokensCached artifact tokensFull: F1 / TTFTPIC: F1 / TTFTPIC + ATTUNER: F1 / TTFTSpeedup over Full
0455.5377.861.01 / 39.0948.70 / 37.7858.60 / 37.421.04×
2715.8638.154.54 / 57.4834.69 / 44.5048.52 / 44.081.30×
4978.7901.151.28 / 77.1826.58 / 50.8940.06 / 50.411.53×
81,505.61,427.946.96 / 116.6014.65 / 62.4234.42 / 61.851.89×
162,549.92,472.242.09 / 199.586.24 / 84.8526.33 / 84.332.37×

The adapters trained for these conditions improve over PIC at every distractor level, while a gap to Full remains. The timing measurements show that ATTUNER retains near-PIC warm-cache latency, with increasing speedup over Full as more documents are reused. We next examine which aspects of cache reading can explain the remaining quality gap.

AuthorsOpenReview ↗

Response to Reviewer GkER (2/2)

A2-2: The attention-routing motivation remains supported under distractors.

We first intervene on PIC's reading process directly. At each layer and head, the routing oracle uses Full's complete attention rows to read the intervention branch's values, leaving the independently constructed artifact KV unchanged.

Table 4. Routing oracle, MuSiQue Answer F1 (%), 2,417 questions per condition; matched Full/PIC baselines for this diagnostic.

Added distractorsFullPICRouting oracleOracle − PIC
0 (GT only)61.0249.0160.52+11.51
255.0334.0552.49+18.44
452.5225.9450.34+24.40
846.6915.0644.34+29.28
1641.716.1138.34+32.23
All original candidate documents41.865.5837.88+32.30

The oracle approaches Full even with all candidate documents. This supports our original motivation under increasing distraction: correcting attention routing can recover much of PIC's lost performance without rebuilding the artifact cache. The question is therefore how to learn this effective reading behavior when relevant and irrelevant documents are mixed together.

A2-3: Restoring attention allocation between documents recovers substantial performance.

Table 5 (left). Between-document attention correction. Table 6 (right). Mass and decoding controls. MuSiQue Answer F1 (%).

Method (Table 5)F1 (%)│Method (Table 6)F1 (%)
Full40.42│ATTUNER control29.84
PIC5.34│+ Between-document correction42.14
ATTUNER control29.29│+ Total mass correction only28.86
+ Between-document correction41.09│+ Between-document correction (decoding only)39.25

Correcting the relative attention allocated between documents brings ATTUNER close to Full in this diagnostic, while preserving within-document attention, total artifact attention mass, and cached KV. Correcting only total mass provides no observed improvement, whereas document redistribution during decoding alone recovers much of the benefit. These results identify which documents receive attention during generation as an important part of the recovery.

A2-4: Implications for training and the scope of the paper.

These findings suggest three implications:

  1. More direct supervision for document-level allocation. Response-distribution distillation supervises reading behavior indirectly. Additional supervision or constraints on between-document attention proportions could provide a more direct signal for recovering effective allocation when distractors are present.

  2. Artifact structure may affect the difficulty of discovery. For self-contained artifacts, such as skills that can be selected and used independently, identifying the relevant artifact may be sufficient to use its content effectively. MuSiQue requires combining evidence across documents, so selecting useful evidence can depend on intermediate reasoning. Such dependencies may call for mechanisms that guide document allocation throughout generation.

  3. Evidence utilization is a starting point for robust cache reuse. Restoring the ability to read and integrate relevant cached evidence provides a foundation for handling broader candidate sets. Building on this foundation requires addressing artifact discovery together with evidence utilization, particularly when documents are interdependent.

The contribution of this paper is to establish evidence-utilization repair through query-side adaptation as a foundation for effective reuse of independently cached artifacts. Building on the distinction introduced in Section 3.1, we have clarified this scope in the revised manuscript (Line 1386, page 26).

Reviewer_GkEROpenReview ↗

Thanks for the update

Thanks for the update, this answers most of my questions. I suggested acceptance and still believe the paper should be accepted but the impact is somewhat limited so I'll keep weak acceptance.

Thank you for taking the time to review our responses and for your continued support for acceptance. We are glad that the additional results addressed most of your questions, and we sincerely appreciate your constructive feedback. We believe this work provides a useful starting point for more robust and efficient reuse of cached artifacts in broader agentic settings.

Reviewer_AnWsOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors investigate quality degradation from independent KV caching and show that restoring full-context attention weights can recover performance while preserving cached artifact values. Accordingly, they propose ATTUNER, which trains low-rank query adapters through full-context distillation to improve how the model reads the fixed cache. They evaluate the method on Qwen3-4B and Qwen3-8B across skill, document, and memory tasks, demonstrating substantial quality improvements while retaining the latency benefits of cache reuse.

Critical strengths
  • I find the paper well motivated and its problem timely, since agentic workflows frequently reuse artifacts. The authors therefore address a practical source of inference overhead by reducing the need to encode this content repeatedly.
  • I also appreciate the simplicity and low overhead of ATTUNER. By adapting only the query projections, the authors substantially reduce time to first token while keeping task performance close to full prefill, offering a compelling balance between quality and efficiency.
Critical weaknesses
  • My main concern is that the evaluation remains a bit limited relative to the agentic motivation, since the authors mainly study selected artifacts followed by a question. In other words, independent encoding in these experiments mainly removes conditioning on earlier artifacts, and not on a broader interaction history. Consequently, it remains unclear whether the method works equally well when artifacts appear within ongoing interactions with the user, where preceding exchanges may affect their interpretation.
Questions for authors
  • How do the authors expect ATTUNER to behave when cached artifacts appear within ongoing interactions with the user and environment, particularly when preceding exchanges affect how those artifacts should be interpreted?
4 comments
AuthorsOpenReview ↗

Response to Reviewer AnWs (1/2)

We sincerely thank you for your strong support and are deeply encouraged by receiving the acknowledgements from the expert in the community, especially for our novelty and efficiency. We have revised the paper following your suggestion, and hopefully could address your concerns through the rebuttal hereunder.

Q1: How does ATTUNER behave when cached artifacts appear within ongoing interactions, where preceding exchanges affect their interpretation?

A1-1: Thank you for highlighting this important setting. Below, we explain this choice and present an additional experiment on artifact reuse within ongoing interactions.

The evaluation protocol of prior PIC methods. Prior PIC methods are evaluated on requests that place independently cached chunks before an online question [1–5]. AgentKVShift [6], which targets agentic memory systems, is the closest to the agent setting: memory entries are retrieved and their cached KV is reused for the current request, again placed before the query. This matches our setting of skills, documents and memory messages followed by a question. To our knowledge, none of these works evaluates the case where earlier turns of an ongoing interaction change how a cached artifact should be read. Keeping the same request format lets ATTUNER and all baselines be compared on identical inputs.

We agree that multi-turn ongoing interactions reuse is an important future application of PIC, we have highlighted this setting and its systematic evaluation as important future directions in Appendix D.2 (Line 1380, page 26). To test whether ATTUNER generalizes to it, we conducted the experiment described below.

A1-2: Experimental setup.

  • Request structure. We build multi-turn requests on BigCodeBench with the same skill-disjoint evaluation skills as the paper: [history] → [cached skills] → [short final instruction]. History turns are online text, which is equivalent to prefix caching in deployment. Only the skills are independently cached artifacts. The adapter acts on all online tokens and never modifies the artifact KV.
  • Four conditions for the same task. To separate the effect of history itself from the effect of history that determines how the skills should be used, each task appears in four conditions:
    • No History (NH): the original prompt (skills, then the function specification), as in the paper;
    • Unrelated History (UH): two unrelated question–answer turns before the original prompt;
    • Spec in History (SH): the user gives the function specification in an earlier turn; the skills are loaded afterwards, followed only by "Implement the function from the specification earlier in this conversation." The cached skills therefore never see the specification, and only about 30 online tokens follow them;
    • No Spec (NS): the skills and instruction without any specification. This is a control confirming that the task cannot be solved without the history.
  • Data. Evaluation tasks are paired within identical skill sets from the 667 evaluation instances. Training uses multi-turn views of the 167 real BigCodeBench training-side questions, whose skills are disjoint from evaluation.
  • Adapters. We evaluate the paper's adapter unchanged, and an adapter trained for this setting. For the latter, we continue training the paper adapter with the same objective and hyperparameters on the training data above; teacher responses come from Full prefill on each view. Training turns are drawn from paraphrases that differ from the evaluation templates, and unrelated turns are drawn from MuSiQue training questions. The checkpoint is selected by validation KL only, without looking at the evaluation split.

[1] Yao et al. CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion. EuroSys 2025.

[2] Hu et al. EPIC: Efficient Position-Independent Caching for Serving Large Language Models. ICML 2025.

[3] Yang et al. APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding. ICLR 2025.

[4] Yang et al. KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse. NeurIPS 2025.

[5] Zhao et al. You Need an Encoder for Native Position-Independent Caching. 2026.

[6] Pandey et al. AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems. 2026.

AuthorsOpenReview ↗

Response to Reviewer AnWs (2/2)

A1-3: Results.

ConditionFullPICATTUNER (paper)ATTUNER + multi-turn data
NH44.539.843.244.9
UH42.742.742.443.2
SH44.130.735.939.0
NS0.00.00.00.0

From the experimental results, we can observe that PIC is affected only when the history determines how the skills should be read: it performs on par with Full under unrelated history, but falls behind when the task specification is moved into preceding turns. ATTUNER recovers part of this gap without multi-turn training, and further training on history-dependent requests improves recovery while preserving performance in the no-history setting.

A1-4: A representative example (SH).

[User]       I need a Python function. Here is the specification.
             Given a dictionary 'animal_dict' with people's names as keys and their favorite
             animals as values, reverse it so that the original values become keys and the
             original keys become lists of values; also count each animal in ANIMALS with a
             random count. def task_func(animal_dict, max_count=10, seed=0): ...
[Assistant]  Thank you, I have read the specification. Do you have any reference material for it?
[User]       <cached skill 1> --- <cached skill 2> --- <cached skill 3>   (≈18K characters)
             Implement the function from the specification earlier in this conversation.
MethodReversal step (excerpt)Result
Fullfor person, animal in animal_dict.items(): reversed_dict[animal].append(person)✓
PICfor key, value in animal_dict.items(): reversed_dict[key].append(value) (not reversed)✗
ATTUNER (paper)reversed_dict[value].append(key)✓
ATTUNER + multi-turn datareversed_dict[value].append(key)✓

Under PIC, the model even writes the comment "values become keys" but then keeps the original direction: the cached skills never saw the specification, and the detail is lost when it is only available in the history. The adapted queries read the same cached skills while bringing the earlier specification back into play.

A1-5: Behavior of ATTUNER in multi-turn interaction. When the history does not change how artifacts should be read, ATTUNER behaves as in the paper. When it does, the adapter needs such requests in training, and because it adapts every online token while keeping the artifact KV fixed, it can then learn to connect the cached content with earlier context.

Preceding context can affect PIC in many ways: it may be irrelevant, may specify the task, or may select among several tasks, and our experiment covers only a few of these cases. A promising direction is therefore to build a long-horizon benchmark that controls whether and how earlier interaction affects artifact reuse, and to evaluate PIC methods, including ATTUNER, on it.

I would like to thank the authors for their response! I also find the addiitonal experiment clean. I also find it promising how Attuner beats PIC in the spec in history (SH) case, even without using multi-turn data.

I will maintain my positive assessment.

We sincerely appreciate your thoughtful engagement with our work and your continued support. Your question motivated the additional experiment, and we are encouraged that you found the results promising. Thank you again for your time and constructive feedback, which helped us better understand ATTUNER’s potential in ongoing interactions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their time and constructive comments. We are encouraged by their recognition of our paper’s contributions, including the practical importance and motivation of cache reuse (AnWs, Uz9u), the analysis separating attention routing from cached value representations (GkER, Uz9u), and the simplicity, effectiveness, and favorable quality–latency trade-off of ATTUNER (all reviewers).

We have provided individual responses to each reviewer. To summarize:

  • On cache reuse within ongoing interactions (AnWs): We appreciate the suggestion to examine how preceding exchanges affect artifact interpretation. We conducted a controlled BigCodeBench experiment that distinguishes unrelated history from history containing the task specification. When earlier turns determine how cached skills should be used, the original ATTUNER partially recovers PIC’s performance loss even without multi-turn training. Further training on history-dependent requests improves recovery while preserving performance without history. We have also highlighted broader, long-horizon evaluation as an important future direction in Appendix D.2 (see Response to AnWs, A1-2–A1-5).

  • On robustness and efficiency with distractors (Uz9u, GkER): We evaluated the unchanged paper adapter on BigCodeBench with increasing numbers of distracting skills. Its advantage over PIC persists at every tested level, although the gap to Full widens. Training with distractors further improves performance. Additional MuSiQue experiments likewise show consistent improvements over PIC, with a larger remaining gap when evidence must be combined across distracting documents. ATTUNER retains near-PIC latency as the candidate sets grow (see Response to Uz9u and Response to GkER, A2-1).

  • On position alignment and the remaining recovery gap (GkER, Uz9u): We added PIC-Pos results under distractors to the revised manuscript; position alignment alone remains insufficient to close the gap to Full on both benchmarks. Further attention-oracle experiments recover much of PIC’s lost performance even with all original MuSiQue candidate documents, supporting our central motivation under distraction. Document-level interventions identify between-document attention allocation as an important bottleneck and motivate more direct allocation supervision. These findings reinforce evidence-utilization repair as a foundation for effective cache reuse, while identifying additional challenges in artifact discovery. We have clarified this scope in Appendix D.2 (see Response to GkER, A1 and A2-2–A2-4).

We hope these additional experiments and clarifications address the reviewers’ concerns. We are happy to discuss any further questions and sincerely thank the reviewers for helping us improve the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.