- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
While lots of prior work focuses on model behavior on knowledge conflict between retrieved text and models' (LLM) internal knoweldge, this paper answers to the question whether which knowledge source does the model refer to, by altering (plugging in and out certain parts) the KV-cache. They provide three hypotheses for this aspect (document re-reading, parametric knowledge comparison, and cached trace), where they verify the hypotheses across 5 experiments (document drop/replace, Q+A KV transplant, qa value steer, projection intervention, and answer source steering). The author shows that LLMs often relies on KV cache made for QA and that these KV cache can affect the model's answer whether which source information the model has referred to.
Critical strengths
- The idea of using KV cache for the verification of the source report is interesting, where a better analyses with justifications and writing is needed.
Critical weaknesses
- I think the biggest weakness of the paper is readability as a AI research paper. I believe that re-writing the paper with this in mind is a must for the paper to be published. (1) Detailed related work section in the appendix is doable, however, a bit more in detail how this paper's findings differ from other relevant work should be more addressed in the paper writing, at least as a detailed paragraph; the paper consists of an extremely small number of citations in the main writing. (2) The paper writing seems more of a story telling, which in my opinion is less-favored for a research paper. The writing itself might be easy to follow, but in detail hard to track which the authors are trying to show with rigorousness. (3) Instead of having three similar figures to explain each experiment (fig. 1,2,3), I believe that a figure that explains the whole framework.
- Most findings are too incremental to the prior work, for instance it is well known that steering vectors exist to guide the model in a certain direction. Plus, besides the quantitative numbers, some qualitative analyses are needed to see how the KV cache are actually behaving.
Questions for authors
- Are there any qualitative analyses for the KV-cache representation itself that can support the findings?
- Instead of asking the LLM to answer where they had refer to (which itself can be hallucinated), are there any other ways to check the source reports?
- Is there a reason the KV cache is the best option for this analysis? Why not other model internal representations?
4 comments
Part 2/2: Questions
Q1. Qualitative analyses of the KV cache. The paper reports content checks after each edit: the model still repeats its answer and quotes the document (Appendix C.7). It also reports one qualitative observation: after a question-cache transplant from an irrelevant-document run, Qwen3-8B often states that the document does not mention the subject (Appendix C.2).
Q2. Checking source reports without asking the model. The report is our object of study; we do not use it as a measurement of provenance. Where provenance must be known, we read it from the answer. In counterfactual runs the answer value shows which source was used, and Section 7 measures source choice from the answer alone, with no source question. In our setting, reports match the expected label in 92–100% of runs in all but one model–condition cell. How the cached state behaves when the report is wrong is an open question that we will state in Section 8.
Q3. Why the KV cache. At report time, later tokens can access earlier positions only through attention, that is, through the keys and values cached at those positions. The cache is therefore the complete interface between the earlier computation and the report. Editing it changes exactly what the report can read. It also lets us intervene after the answer is complete while every token stays fixed. Appendix C.9 compares value steering with residual-stream steering for source selection. We will state this motivation in Section 2.
Thank you for your clarifications and changes. I have checked the revised draft, but no signs of the figure update yet. Can the authors update the figures as stated and add a holistic framework figure?
I believe that the revision makes the paper not as a clear reject, so I raised the scores accordingly, but as the other reviewer mentions, I believe that the work needs some re-write.
Thank you for raising your score. The revision adds the related-work paragraph to the introduction and the motivation for KV-cache interventions to Section 2. We have not yet finalized an overview figure that works well; we will add a single framework figure covering all interventions and shorten Figures 2–3 in the camera-ready version.
Part 1/2: Weaknesses
We thank the reviewer for the comments on presentation. We accept most of them and list the changes below. We also clarify the main contribution, which our writing did not make prominent enough. We answer the reviewer's questions in Part 2.
W1. Presentation.
(1) Related work. We agree. The revision adds a paragraph to the main text that states the difference from prior work. Attribution work (Rashkin et al., 2023; Cohen-Wang et al., 2024) evaluates whether the context supports an output. Knowledge-conflict work (Longpre et al., 2021; Ortu et al., 2024; Jin et al., 2024; Zhao et al., 2025) studies which source the model selects. We study a different object: what the model's later report about its own source depends on.
(2) Structure. The paper contains several similar interventions, and we ordered them so that each one tests a single hypothesis and motivates the next. Document edits test whether the report depends on the document (Section 3). Transplants test whether it depends on the Q+A state (Section 4). Steering and projection edits locate the components of that state that carry the effect (Sections 5–6). Section 7 links them to source selection. Table 1 maps each hypothesis to its evidence. In the revision, every section will open with the claim under test and the result.
(3) Figures. Figures 2 and 3 show different interventions: one edits the document cache, the other the Q+A cache. We will add a single overview figure of all interventions and shorten these two.
W2. Novelty. We agree that steering vectors are well known. We use steering as a tool. The main finding is in Sections 3–4: after the model has answered, changing or removing the document never changes the source report, while transplanting the Q+A cache always does, with every token fixed. The report depends on state computed before the source question is asked. To our knowledge, no prior work tests this.
Two properties separate our directions from a standard steering result. First, they are fitted on states recorded before any source question, yet they change the later report. Second, without refitting, the same directions change which source supplies the answer during generation (Section 7).