acceptodds
Under review as a conference paper at ICLR 2027

Retrieved but Not Used: Access–Use Dissociations in Long-Context Language Models

Abstract

Long-context evaluations ask whether a language model can find information far back in its input. We ask a narrower question: once a value can be found, can the model use it? We construct partially evaluated programs in which a probe value is defined once and consumed once after a controlled number of intervening lines, with total length and live state held fixed, and we compare three queries at the same position: copy the value (retrieve), add a one-digit literal to it (compute), or add a one-digit local variable to it (combine). Across four Qwen3 models (4B to 32B) at 28K tokens, retrieve stays at 88.5 to 99% after 1,024 lines of separation while compute falls to 37.5 to 76% and combine to 0.5 to 43.5%. On identical programs the use failures sit inside the retrieval successes: Qwen3-8B copies the value on all 200 programs on which it then fails compute (100) or combine (198), and the paired excess of combine over retrieve is 47.5 to 86.5 points on the three models tested. Context structure matters in a scale-dependent way. At a fixed distance, growing the context from 3.7K to 28K tokens shows no detectable load effect on retrieve or compute at 8B, 14B or 32B, while combine falls from 95 to 66% at 32B; the far-distance collapse occurs at constant length. Across families the dissociation is a boundary rather than a universal: Llama-3.1-8B shows the one-operation gap (19 points) but not the two-variable collapse, and Gemma-3-12B shows both (37 and 61 points) where its position-dependent retrieval holds. A 2,048-token generation budget returns compute to 86 to 99% on the same items with or without an instruction to reason, while the question format at short budgets answers nothing; what the budget buys is not identified. A pre-specified decision rule based on a log-odds interaction returned no-go; under ceiling-level retrieval that estimator tests a different scale from the verbal hypothesis, so we make no confirmatory claim from it and label the probability-scale analysis post hoc. All evidence comes from one synthetic task.

Primary area:
datasets and benchmarks
Keywords:
long-context language models, retrieval, reasoning, dependency distance, compositionality, context length, language model evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eHj1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the relationship between retrieving and using information in long contexts. Using controlled experiments on a synthetic arithmetic task, it compares copying a value with computing from that value while varying dependency distance at fixed context length. It finds a performance gap between these tasks and examines how that gap changes with factors like context structure, model size, and generation budget.

Critical strengths

The fixed-length dependency-distance experiments and paired retrieval/computation comparisons provide useful controlled experiments. They establish a systematic behavioral difference on the same underlying programs, giving the paper empirical value beyond an aggregate accuracy comparison.

Critical weaknesses
  1. The additional insight from the task construction is insufficiently established. GSM-Infinite(https://arxiv.org/abs/2502.05252v1) builds on the controlled synthetic reasoning constructions in Physics of Language Models(https://physics.allen-zhu.com/part-2-grade-school-math/) and extends them to long contexts, including symbolic assignments and accuracy comparisons at fixed operation counts. The present paper’s fixed-length distance manipulation and matched retrieval/use queries provide useful additional controls. However, it remains unclear which substantive conclusion these controls establish beyond prior findings. A larger performance drop, or its appearance in a simpler task, needs a clearer explanation of what new understanding it provides.

  2. The evaluation supports a behavioral contrast more clearly than the stronger access/use interpretation. Successful copying in a separate query does not establish successful retrieval during a failed computation. Although intermediate values are supplied, computation still requires locating and binding the relevant operands; query dependent retrieval, interference, and composition therefore remain plausible contributors. The response protocol adds uncertainty: the main evaluation uses eight-token raw completions, while Table 5 shows substantial recovery under longer question-based responses. Those comparisons also change prompting and answer extraction, leaving their contributions to the main contrast incompletely separated. The paper acknowledges several of these limitations, but wording about using information the model has “already located” exceeds what the comparisons can establish. This limits both the interpretation of the gap and the explanatory contribution of the study.

Questions for authors

Which specific conclusion depends on the fixed-total-length distance manipulation and matched retrieval/use controls, and goes beyond the findings of Physics of Language Models and GSM-Infinite? How do these controls distinguish the proposed phenomenon from previously observed context-related degradation?

Is “access–use dissociation” intended as an operational difference between query types, or as evidence that information was successfully retrieved during computation but subsequently not used? For the latter interpretation, what evidence distinguishes failed use from query dependent retrieval or operand binding difficulties?

Could you provide a small matched near/far comparison of retrieval, compute, and combine using a supported chat template and budgets sufficient for answers to complete, with prompting and answer extraction held fixed within each budget comparison? Reporting accuracy alongside missing-answer and formatting-error rates would help determine how much of the gap persists beyond the restricted completion tokens.

Reviewer_dyHNOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether a long-context LM that can locate a value can also use it. It builds synthetic, partially evaluated programs: a chain of modular-arithmetic assignments in which every line shows its result. The authors argue, with a simulation, that this estimator is wrong-signed when retrieval is at ceiling. They report it unchanged, make no confirmatory claim, and present a probability-scale analysis explicitly labelled post hoc.

Critical strengths
  • Several variables usually move together in long-context benchmarks: total length, live state, operand difficulty, and the distance from value to use. This design holds the first three fixed while varying the last, and the paired arm cross-tabulates all three queries on identical programs. This rules out the reading that the excess loss mixes retrieval failures on some items with computation failures on others.
Critical weaknesses
  • “Retrieved but not used” is not shown within the use query itself. Success on ms5 = ts1 =\texttt{ms5 = ts1 =} shows that vv is retrievable under that cue, not that it was retrieved while the model processed \texttt{ms5 = (ts1 + 7) % 9973 =}, whose cue position and final tokens differ. The paper does not analyze the wrong compute/combine answers, and the possible error types point to different conclusions.
  • Although I do understand, but for long-context research, the model choices (Qwen, 4B-32B) and context length (~32K) are really not sufficient.
  • The submission doesn't use the official ICLR 2027 template formatting, the fonts of title, main text are not looking correct.
Questions for authors

See weakness above.

Reviewer_TKCYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether long-context language models can use information they have already retrieved by constructing a controlled synthetic task based on partially evaluated modular-arithmetic programs, where a probe value is defined once and consumed once after a variable number of intervening lines; at each distance, three queries are issued at the same position—copy the value (retrieve), add a one-digit literal to it (compute), or add a one-digit local variable to it (combine)—so that any performance gap beyond retrieval reflects the cost of "using" the value rather than merely finding it. Across Qwen3 models (4B–32B) at 28K tokens, retrieval accuracy remains near ceiling (88.5–99%) even at 1,024 lines of separation, whereas compute drops to 37.5–76% and combine collapses to 0.5–43.5%, and crucially, on identical programs the use failures nest almost entirely within the retrieval successes—indicating an access-use dissociation. The effect persists across model scales, is robust to scratchpad prompting and cross-stack replication, but shows nuanced cross-family variation (Llama reproduces only the compute gap, Gemma's retrieval itself deteriorates at intermediate distances), and the authors provide both descriptive and logistic-statistical analyses while acknowledging the synthetic task's limited generalizability to natural-language settings.

Critical strengths

The strength is the methodological rigor: the task design cleanly isolates retrieval from use by holding total length and live state constant, and the paired-arm design (all three queries on identical programs) enables a nested-failure analysis that directly demonstrates the dissociation. The authors also release all code and 74K+ scored outputs with thorough ablations (scratchpad, cross-stack replication, load variations), making the work highly reproducible and the statistical claims well-supported.

Critical weaknesses
  1. The novelty claim feels overstated: the retrieval-use gap is already well-documented in RAG literature (where it's common knowledge that retrieval ≠ correct downstream use) and in LLM reasoning benchmarks (e.g., MATH, program execution) that independently show models struggle with multi-step ops even when info is present. The paper presents this as a new discovery rather than a domain-specific instantiation of a known limitation; its real contribution is methodological (clean isolation), not conceptual.

  2. The task is built on modular-arithmetic programs with opaque variable names and fully evaluated intermediates — it's a symbolic puzzle, not natural language reasoning. The access-use dissociation may be an artifact of this narrow domain and may not generalize to semantic retrieval, multi-hop QA, or any setting where retrieval and computation are naturally intertwined.

  3. Token budget and tokenizer differences confound results. Scoring only the first integer on the first generated line penalizes models that output reasoning text — they may know the answer but run out of output tokens. Meanwhile, Llama's programs compress to ~22.9K tokens while Qwen3's stretch to ~28K, so cross-family gaps may reflect tokenizer efficiency rather than actual model capability.

  4. Limited backbone coverage. The paper evaluates only Qwen3 models (4B–32B) plus Llama-3.1-8B, Gemma-3-12B, DeepSeek-R1 distill-7B, and Mistral-7B. It omits newer backbones such as Qwen3.5-9B or Qwen3.8-27B, leaving open whether the access-use dissociation persists or diminishes on more recent architectures that may have improved long-context reasoning through better pretraining or architectural updates.

  5. Lack of relevant citations: [1] Chen, T., Tan, Z., Bo, X., Wu, Y., Gong, T., Chu, Q., & Ye, J. (2026, March). Flora: Effortless Context Construction to Arbitrary Length and Scale. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 40, No. 36, pp. 30288-30296). [2] Lin, T., Zhou, Z., Zhao, X., Wang, C., Li, X., Chen, Y., ... & Deng, Y. (2026). Beyond the Needle's Illusion: Decoupled Evaluation of Evidence Access and Use under Semantic Interference at 326M-Token Scale. arXiv preprint arXiv:2601.20276. [3] Moon, H., & Lim, H. S. (2026, July). NEEDLECHAIN: Measuring Intact Context Comprehension Capability of Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2026 (pp. 32718-32730).

Questions for authors

See weaknesses.

Comments on the paper

1 comment
Area_Chair_XK43OpenReview ↗

Discussion Period

Dear Reviewers,

The authors have posted their response. Please take a moment to review it and engage in discussion with both the authors and fellow reviewers, especially on any remaining concerns or points of disagreement. Brief updates to your assessment would be very helpful.

Thank you.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.