acceptodds
Under review as a conference paper at ICLR 2027

Beyond Lost in the Middle: Causal Locality in Tool-Using Agent Contexts

Abstract

Long-context evaluations usually move a relevant document while treating the remaining input as exchangeable. Agent histories are different: a tool call and its result form an atomic transaction, and later transactions can depend on values or decisions produced earlier. Reordering an agent context may therefore gain favorable absolute position while destroying causal locality. We introduce a paired intervention testbed that varies the producer position and the amount of unrelated evidence inserted inside a critical dependency chain, while holding the evidence multiset fixed and preserving a valid topological order. Across 24,144 greedy-decoded calls to two open-weight model artifacts, the separation contrast is -4.40 percentage points for GPT-OSS 20B, whereas Gemma 3 27B has no changed binary outcomes in the paired cells. Their equal-weight arithmetic mean is -2.20 points (95% trajectory-bootstrap CI: [-4.05, -0.23]), but is not a cross-model population estimate. Because the original separation treatment also moves the consumer endpoint, this pilot does not identify a position-independent locality coefficient. We find no reliable front-versus-middle benefit or pre-registered interaction. An exploratory end-pinning heuristic improves GPT-OSS by 4.63 points over chronological order, but has no effect on Gemma. These results expose capability, endpoint-position, and run-reliability checks that are necessary before asserting a universal agentic “lost-in-the-middle” rule or deploying an optimized placement policy.

Primary area:
generative models
Keywords:
Language-model agents, Long-context, Position bias, Exact next-action success

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_gDAKOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper evaluates the effect of ordering of context across a set of somewhat limited 6 procedural task families, and evaluates various orderings across two models (Gemma 3 27B and GPT-OSS 20B).

Critical strengths

None noted.

Critical weaknesses
  • The abstract (and much of the paper) was extremely difficult to read, with seemingly unintroduced terms and concepts. It seems highly AI generated without much human review or editing.

  • Procedural tasks should be more formally defined somewhere (are these prompts, what code/data are they working on, etc).

  • Section 4.2 introduces a lot of variables and content which was not described or introduced. Some figures would be useful to better present what is going on here.

  • The discussion (and many other sections) reads as rough AI generated text.

Questions for authors

None noted.

Reviewer_rsPLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies context placement in tool-using agents under fixed evidence and dependency-preserving reorderings. It introduces a synthetic testbed that varies the position and separation of critical transactions, evaluates two model artifacts, and compares several placement policies, including DAGPlace. The main separation contrast reduces GPT-OSS performance by 4.40 percentage points but produces no changed paired outcomes for Gemma. The authors appropriately acknowledge that the intervention also moves the consumer endpoint and does not identify a position-independent locality effect.

Critical strengths
  1. The paper studies a relevant topic in agent context management.
  2. The authors explicitly acknowledge the limitations of their findings.
Critical weaknesses
  1. The central effect remains confounded. Increasing dependency separation also changes the consumer’s absolute position. As acknowledged by the authors, the observed difference cannot be attributed specifically to locality, leaving the main research question unresolved.
  2. The empirical findings are difficult to interpret. The directional separation effect appears only in GPT-OSS. Gemma exhibits substantial ceiling and floor effects, making its unchanged outcomes weak evidence of placement robustness. The study therefore lacks a sufficiently clear characterization of the observed model differences.
  3. Measurement reliability is insufficiently established. Each layout is decoded only once, and approximately 15% of GPT-OSS controlled outputs are invalid JSON. The primary contrasts do not clearly separate formatting failures from semantic errors or establish repeatability across inference runs.
  4. The remaining contribution is preliminary. Evaluation is limited to synthetic dependency chains and single next-action prediction. DAGPlace does not outperform the strongest observed heuristic, while the proposed endpoint controls and capability calibration remain future work.
Questions for authors

see above

Reviewer_6eqFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper tries to answer which of two factors influences agent performance more: the position of the critical evidence, or the continuity of the evidence chain. To this end, the paper introduces legal agent-context placement to craft the dataset and tests it on two open-weight models.

Critical strengths
  1. This paper tries to solve an interesting problem: whether the position of the critical tool-call units or the continuity of the critical tool-call chain is more important for agent performance.
Critical weaknesses
  1. Based on the limited evidence in the paper, the experimental design cannot evaluate the research problem that the authors want to tackle. When the distance between the critical units changes, their relative position in the trajectory also changes. The authors could keep the relative position by adding more filler units, but then the trajectory length is no longer controlled. It is impossible to control these variables for the experiments.

  2. Lack of critical design details. First, the paper never details what the logs look like. There are no examples of the critical units or filler units, so it is impossible to judge whether the design is plausible. Second, there are no statistics of the logs used in the experiments, such as the typical critical chain length.

  3. The dataset is unrealistic. The topic and task details of the dataset are only briefly introduced, with one sentence per task family. Based on the limited evidence, the authors only run experiments on a very narrow scenario, and the logs are generated by template filling. The dataset is therefore far from realistic scenarios and agent use.

  4. The experiments only use two models of similar size, which cannot ensure that the conclusion is generalizable.

  5. Lack of discussion of important related work on the influence of the position of tool results in agent histories [1], on the separation of dependent facts [2], and on context reordering [3].

[1] Kwak, Beong-woo, et al. "ToolHaystack: Stress-Testing Tool-Augmented Language Models in Realistic Long-Term Interactions." EMNLP (Findings). 2025.

[2] Wang, Meiyun, et al. "Lost in the distance: Large language models struggle to capture long-distance relational knowledge." NAACL (Findings). 2025.

[3] Chen, Xinyun, et al. "Premise order matters in reasoning with large language models." ICML. 2024.

Questions for authors

None. Please see the Weaknesses section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.