acceptodds
Under review as a conference paper at ICLR 2027

ContextRender: From Execution Dependencies to Agent Context

Abstract

LLM agents performing long-horizon tasks accumulate tool results that later steps may need. Passing the full history to every invocation is costly even when it fits within the context window, while reducing it risks omitting needed information. Existing context management methods can overlook how earlier tool results are used in subsequent execution, leaving needed information out of context. We introduce ContextRender, which manages context through a persistent graph of execution dependencies. We develop Tool-Flow Analysis to track how later operations reuse information from earlier tool results, providing a signal called observed reuse. A renderer combines this signal with recency and semantic relevance to select results within a fixed history budget, retaining omitted results for later use. Across AppWorld and 8-objective QA with three execution models, ContextRender outperforms the evaluated context management baselines using a 6K history budget, well below the models’ maximum context windows. Within this budget, it achieves task performance close to or above that of passing the full history while reducing mean inference cost by 10.2%–32.2% relative to Full history. Ablations show that observed reuse improves task performance and retention of results reused later.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, context management, long-horizon agents, tool use, agent memory, context retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_t7DgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CONTEXTRENDER, a context manager for tool-using LLM agents. It stores tool outputs in a persistent graph and selects which results to include under a fixed history budget. Selection combines recency and semantic relevance with observed reuse, which tracks when later execution steps reuse identifiers from earlier tool results. The evaluation covers AppWorld and an eight-objective QA benchmark with three execution models. At a 6K history budget, the reported results improve over the evaluated context-management baselines and approach or exceed full-history performance at lower estimated cost. The reuse ablation suggests that this signal is particularly helpful for retaining useful results from much earlier in the trajectory. CONTEXTRENDER FROM EXECUTION DE…

Critical strengths

I like the use of observed reuse as a signal for context selection. An older result can still matter even when it does not look especially relevant to the current step. Tracking its later use gives a concrete reason to retain it. The matching rules and selection policy are explicit, so the approach is relatively easy to inspect and implement. This makes it a useful contribution to a practical problem in long-running agents.

The reuse ablation is the most convincing part of the evaluation for me. It connects the proposed signal to both task performance and Later-Used Result Recall, particularly at long lookback distances. The experiments across two benchmarks and three models also gives some evidence that the benefit is not specific to one agent setup. I appreciate that the evaluation includes a measure of what the context manager actually retains, alongside final task scores.

Critical weaknesses

My main concern is how reliably the experiments establish the reported gains. The main results use one run per task without confidence intervals, which makes the smaller differences difficult to interpret. The Semantic Retrieval baseline also reserves 60% of its budget for recent turns, and the cost estimates apply the Full-history cache rate to all methods. Neither choice is necessarily unreasonable, but the conclusions may depend on them. I would be more convinced by uncertainty estimates and a sensitivity analysis showing whether the performance and cost advantages survive changes to these assumptions.

The coverage of the reuse signal is also unclear. Exact, case-sensitive matching can miss paraphrases and transformed values, while the first tool result containing a value may not be its actual source. A single reused value also increases the priority of the whole result, potentially bringing back irrelevant content. These seem like reasonable compromises for a lightweight heuristic, but I would like to know how often they matter in the experiments. Without coverage statistics or a breakdown of failure cases, it is hard to judge how well the method would work when tool outputs contain mostly free-form text rather than stable identifiers.

Questions for authors

What fraction of tool results receives a non-zero reuse score? Could you report this seperately for each benchmark and include a few examples where exact matching misses useful context or attributes reuse to the wrong result? This would help clarify how much of the observed benefit depends on identifier-heavy environments.

Could you report confidence intervals for the main performance differences and repeat the evaluation on a representative subset of tasks? I am particularly interested in whether the smaller gains remain consistent across repeated agent runs.

How sensitive are the comparisons to the Semantic Retrieval baseline’s fixed 60% allocation? Also, do the estimated cost savings remain similar when each method’s own cache behavior is used instead of the Full-history cache rate? These checks would help assess the practical advantage claimed for the method.

3 comments
AuthorsOpenReview ↗

Q1. Reuse coverage and attribution.

Thank you for the thoughtful feedback.

Tool-Flow Analysis (TFA) assigns reuse credit when qualifying values from an earlier tool result reappear in later model outputs or tool calls. In the recorded GPT-4.1 runs, 39.0% of distinct AppWorld tool results and 4.1% of QA results receive a non-zero score, showing that the signal is more common in identifier-heavy AppWorld.

One audited AppWorld task obtains a Spotify playlist URL ending in /654 from a Todoist tool result, then uses playlist_id=654 in a later call. The identifier falls below TFA’s six-character threshold and receives no reuse credit. The agent had already copied the URL into an assistant message, so the ID remained available and the task succeeded.

Another audited AppWorld task uses [email protected]. Its bradley_ball token appears only in Bradley’s contact result and contributes credit 1. The shared gmail.com token appears in seven results and contributes 1/7 to the earliest, Paul’s contact result. Frequency weighting reduces this ambiguous credit. This specific pattern, where part of a value credits a different result from the full value’s source, accounts for at most 1.3% of AppWorld reuse credits in the audit.

The replay audit checks whether values missed by TFA remain available through complete source results or earlier assistant messages. They do in 95.8% of 1,614 AppWorld cases and 99.4% of 681 QA cases. Table 2 separately tests task performance by adding reuse to the same recency-and-relevance selector under the same 6K budget; scores improve on both benchmarks, including QA where fewer results receive reuse credit.

AuthorsOpenReview ↗

Q2. Uncertainty across tasks and repeated executions.

The paired bootstrap intervals support positive QA F1 gains against Recency + Summary, Semantic Retrieval, and ACON. The intervals against Full history, Prune, and Compact include zero, leaving those smaller differences uncertain.

Table R1 uses GPT-4.1 results on 100 8-objective QA tasks. The paired bootstrap resamples the same task IDs for both methods 10,000 times to estimate the uncertainty in their mean F1 difference. Each method has one recorded execution per task, so the intervals measure uncertainty across tasks, not variation across repeated executions of the same task.

Table R1. ContextRender’s F1 is 53.1. Gains are its F1 minus each comparison method’s F1, in percentage points; each gain has a separate 95% confidence interval.

Comparison methodMethod F1Gain (F1 points)95% CI for gain
Full history50.8+2.3[−0.03, +4.8]
Prune51.2+1.9[−0.7, +4.7]
Compact (LLM)52.0+1.1[−1.7, +4.0]
Recency + Summary48.5+4.6[+2.3, +7.2]
Semantic Retrieval46.4+6.7[+2.8, +10.8]
ACON48.1+5.0[+1.0, +9.5]

Q3a. Sensitivity to Semantic Retrieval’s recent-history cap.

The 60% setting is a ceiling on recent results, not a fixed reservation of context. At the main 6K history budget, it binds in only 5 of 257 managed steps (1.9%) in the audited 30-task AppWorld GPT-4.1 OpenRouter sweep. Moreover, 87% of the tokens in tool results shown in full were selected by semantic similarity. This addresses the concern that the baseline allocates most of its context to recency: in this audited setting, most retained full-result content is selected for relevance.

We checked the QA runs by asking whether the recorded recent results would exceed the same 60% allowance. At 6K, this occurs in only 8 of 5,439 steps for DeepSeek V4 Pro and none of 1,710 for Muse Glimmer 30B. These counts describe how often the cap would constrain the recent results, based on their recorded sizes.

The cap measurements above cover the retriever configurations just described. The local GPT-4.1 QA runs identified in the audit use a different retriever without this 60% cap, so they are not included in these measurements.

Q3b. Cache behavior underlying the cost comparison.

ContextRender’s observed cache-hit rate on AppWorld Test-Normal is 85.3%, compared with 80.3% for Full history, a 5.0-percentage-point advantage. The table reports all methods, showing that ContextRender preserves cache reuse while selecting history.

AppWorld Test-Normal. ρ is the observed fraction of input tokens served from cache. Differences are in percentage points relative to the Full history row.

MethodObserved ρDifference (pp)
Full history80.3%-
Prune69.0%−11.3
Compact73.2%−7.1
Recency + Summary72.2%−8.1
Semantic Retrieval85.8%+5.5
ACON74.4%−5.9
ContextRender85.3%+5.0

The paper’s cost estimates apply a shared cache rate across methods. The observed rates above show that ContextRender preserves cache reuse on Test-Normal. They do not by themselves quantify revised cost savings, which also depend on the corresponding token usage.

Reviewer_aR7WOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ContextRender, a system for managing the interaction history of LLM agents under a token budget. It stores the complete history in a persistent graph and uses Tool-Flow Analysis (TFA) to detect when values from earlier tool results appear in subsequent assistant outputs or tool calls. Experiments show improvements over the evaluated context-management baselines, with lower standardized inference cost than passing the full history.

Critical strengths
  1. The paper tacles a practical problem. Long-running agents need to retain information from earlier tool interactions without repeatedly passing the entire history to the model. Using observed reuse to complement recency and semantic relevance is a reasonable idea, particularly for identifiers and other exact values whose importance may not be captured well by embedding similarity.

  2. The proposed approach is lightweight and preserves access to the original information. TFA uses lexical matching and counting, without requiring a learned policy or an LLM call to infer each reuse relationship. Keeping complete tool results outside the visible context also allows previously omitted information to be restored.

  3. The evaluation provides good results.

Critical weaknesses
  1. The paper's technical contribution needs stronger justification. TFA detects exact lexical repetition and attributes each match to the first earlier tool result containing the value. As acknowledged in Appendix A.2, this does not establish an execution dependency: the value may already have appeared in the user request or an assistant message, or a more recent result may have supplied the relevant information. Furthermore, the renderer uses the recorded relationships through a scalar reuse score; it does not exploit paths or other graph structure. As such, it is unclear how much is gained from the proposed attribution mechanism over a history store with simple lexical-use counters. It would be better to include comparisons against plain reuse counts, time since last use, and time-decayed reuse. Ablations of first-source attribution and inverse-result-frequency weighting would also be useful.

  2. The recall metric does not really validate the proposed mechanism. Later-Used Result Recall measures whether the complete original result is visible when a qualifying string reappears in a Full-history execution. It does not establish that the original result is needed or that its relevant information is unavailable elsewhere. This is particularly important because earlier assistant messages are retained verbatim and may already contain the required value.

  3. The experimental comparisons do not cover the closest existing approaches. The paper discusses ACE and PACE, which provide adaptive context selection, but neither is included in the experiments. A comparison with at least one of these methods would help establish the advantage over an approach that also preserves historical information for later use.

Questions for authors

Please see the weaknesses

3 comments
AuthorsOpenReview ↗

W1. Reuse weighting, source attribution, and the role of the graph.

Thank you for the thoughtful feedback.

ContextRender uses observed reuse, detected by Tool-Flow Analysis (TFA), alongside recency and semantic relevance to select complete tool results within a history budget. The graph stores results and reuse relationships; the renderer uses their accumulated score without traversing graph paths. Table 2 tests the value of adding reuse to an otherwise unchanged recency-and-relevance selector.

First-result attribution credits the first earlier tool result containing a matched value, so intermediate copies do not shift the credited result. Frequency weighting gives each match weight 1/n, where n is the number of historical tool results containing that value (§3.2).

In an audited AppWorld contact task, the agent uses [email protected]. The token bradley_ball occurs only in Bradley’s contact result and contributes 1 to it. The shared gmail.com token occurs in seven results and contributes 1/7 to the earliest, Paul’s contact result. This illustrates reduced credit for a shared token attributed to a different contact. That specific partial-value attribution pattern accounts for at most 1.3% of AppWorld reuse credits in the audit.

Attribution determines ranking credit; it does not establish where the agent obtained a value (Appendix A.2). Table 2 evaluates the combined reuse signal with attribution and weighting fixed; it does not individually compare those choices with plain counts, time since last use, or time-decayed reuse.

AuthorsOpenReview ↗

W2. What the recall metric and task ablation establish.

Later-Used Result Recall checks whether the complete earlier tool result is visible immediately before a later turn reuses one of its values. All selectors replay the same recordings made with the Full history baseline, using only preceding messages (Appendix B.4). A value may already be available in an assistant message after its source result is omitted. The audit observed this in 511 of 1,614 AppWorld cases and 96 of 681 QA cases missed by TFA. Recall therefore measures full-result availability, not necessity.

Retaining the source can still preserve other useful fields. In Figure 2, a single contact result contains Troy’s and Nancy’s email addresses. Reuse of Troy’s identifier raises that result’s priority, keeping Nancy’s email available before her address has been copied forward or reused.

Table 2 tests the practical benefit through independently scored task outcomes. With GPT-4.1 and the same 6K history budget, adding reuse to a selector using only recency and semantic relevance raises AppWorld Test-Normal task goal completion (TGC) from 72.6% to 75.6% and QA answer F1 from 50.8 to 53.1, with other components held fixed.

AuthorsOpenReview ↗

W3. Comparison with adaptive context-management methods.

ContextRender uses observed value reuse to prioritize earlier results. ACE uses an LLM to choose full, summarized, or hidden historical steps; PACE selects detail levels by semantic relevance and offers a glimpse tool for recovery. Their described selection rules do not explicitly track observed value reuse.

We attempted to evaluate PACE using the authors’ implementation, but the code link was inaccessible. Our performance comparisons therefore cover the evaluated baselines; the ACE/PACE discussion identifies the difference in selection signals.

Reviewer_GbMuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ContextRender, a context-management system for long-horizon tool-using LLM agents that stores all messages, tool calls, and complete tool results in a persistent graph. A component called Tool-Flow Analysis detects when identifier-like values from earlier tool results reappear in later model outputs or tool calls, using exact string matching, and turns these matches into an “observed reuse” score for each result. When the history exceeds a budget B, a renderer selects complete tool results by combining recency, semantic relevance, and observed reuse. Unselected results remain stored and can be restored later. On AppWorld and 8-objective QA, with three execution models at B = 6K, the authors report that ContextRender outperforms five context-management baselines in all settings, matches or exceeds Full history in seven of nine, and reduces inference cost by 10.2–32.2%. An ablation shows that the reuse signal improves both task performance and retention of results reused many turns later.

Critical strengths

1. A real, clearly demonstrated failure mode, addressed with a cheap and practical signal.

The paper shows clearly that recency- and relevance-based selection can drop early tool results that later steps still need. The proposed reuse signal is training-free, adds no LLM calls, and is easy to plug into existing runtimes. It noticeably improves retention of results reused many turns later. This makes the work immediately useful to practitioners.

2. Lossless storage with re-selection, evaluated consistently across models.

Omitted results are kept in full and can be restored later. The method outperforms all five baselines in all nine model–benchmark settings under the same budget, and the implementation is documented in enough detail to reproduce. This consistency suggests the gains are not an artifact of a single setting.

Critical weaknesses

1. Limited novelty and lack of comparison with an intuitive baseline.

The paper introduces a reuse graph and Tool-Flow Analysis, but neither is fully utilized. The former only uses graph edges to compute a reuse-frequency-based score, while the latter simply detects whether values are reused. Although both are simple and potentially effective methods, their actual implementations seem less sophisticated than their names suggest.

The paper mentions that context compression may omit important details, potentially requiring additional tool calls. Such additional tool calls provide a natural baseline for comparison, but the paper does not evaluate this alternative.

2. Insufficient experimental evidence.

Equation 3 uses the cache-hit rate observed under Full History, but the paper does not report the actual cache-hit rates measured for each method.

Questions for authors
  1. What are the actual cache-hit rates ρ\rho observed for each method, rather than assuming the rate observed under Full History? How do they compare with Full History?

  2. How were the hyperparameters in the paper selected?

  3. Does adding the historical reuse signal cause recent or semantically relevant tool results to be omitted? I suggest adding an ablation study to investigate this.

  4. Please compare CONTEXTRENDER with a baseline that uses Prune for context compression and additional tool calls to recover omitted earlier details.

4 comments
AuthorsOpenReview ↗

Q1. Observed cache-hit rates by method.

Thank you for the thoughtful feedback.

On AppWorld Test-Normal, ContextRender’s observed cache-hit rate is 85.3%, compared with 80.3% for Full history, a difference of 5.0 percentage points. The table reports the measured rate for each method and its difference from Full history.

AppWorld Test-Normal. ρ is the observed fraction of input tokens served from cache. Differences are in percentage points relative to the Full history row.

MethodObserved ρDifference (pp)
Full history80.3%-
Prune69.0%−11.3
Compact73.2%−7.1
Recency + Summary72.2%−8.1
Semantic Retrieval85.8%+5.5
ACON74.4%−5.9
ContextRender85.3%+5.0
AuthorsOpenReview ↗

Q2. Parameter selection and sensitivity.

We selected the parameters on separate development tasks and used them unchanged for AppWorld and QA. In a sensitivity replay of those development traces, ContextRender’s highest miss rate across 15 tested settings remains 4.5–8.6 percentage points below selecting the newest results first, across the four task-group and budget combinations in the table.

Development covered API building, large tool catalogs, and incident response. We selected the recency-decay parameter λ = 0.3 and reuse-saturation parameter γ = 0.2 by replaying saved runs and measuring whether the selector omitted results whose values were reused at the recorded turn. Recency, relevance, and reuse each have weight 1.0; all three score terms lie in [0, 1]. The relevance query combines the original request and latest visible user and assistant messages with fixed weights of 0.4 and 0.6 (§3.3; Appendix A.3).

The sensitivity replay uses lexical relevance and holds the recorded conversations and history budgets fixed while varying λ, γ, and the signal weights. Miss rate is the percentage of recorded reuse events whose source result would be omitted. The table reports the paper’s configuration and the highest miss rate across the tested settings.

Task group History budgetNewest results first Miss rate (%)ContextRender Paper parameters Miss rate (%)ContextRender Changed parameters Highest miss rate (%)
API building 6K tokens53.9%42.7%45.3%
Large tool catalog 6K tokens37.1%27.5%32.6%
Large tool catalog 2.5K tokens71.3%61.7%65.4%
Incident response 6K tokens36.7%26.9%30.9%
AuthorsOpenReview ↗

Q3. Regressions caused by competing selection signals.

To test whether reuse crowds out information favored by recency or relevance, we used the controlled ablation in Table 2. One configuration selects results using recency and semantic relevance; the other adds observed reuse to that same selector. Both use GPT-4.1, the same 6K history budget, and otherwise identical selection components.

We then examined tasks that changed from success without reuse to failure with reuse. There was only one such AppWorld task and none on QA under this analysis. In the AppWorld trace, adding reuse changed which results fit within the budget: a value from an API documentation page was omitted, whereas it remained available without reuse. This shows concretely how the additional signal can displace useful information.

Across the full evaluation, however, adding reuse improves AppWorld Test-Normal task goal completion by 3.0 percentage points and QA answer F1 by 2.3 points (Table 2). The comparison therefore shows an aggregate performance benefit under the same budget, while the trace identifies an individual task where the change in selection was harmful.

AuthorsOpenReview ↗

Q4. Comparison with Prune plus recovery calls.

Prune + recall achieves 66.7% TGC on all 168 AppWorld Test-Normal tasks, compared with ContextRender’s 75.6%, an observed gap of 8.9 percentage points. The comparison uses GPT-4.1 and the same 6K history budget.

Prune keeps the newest tool results that fit the history budget. Prune + recall also stores omitted outputs in full and leaves a placeholder identifying each output and its recall_output(K) recovery call. The agent receives an instruction explaining recovery and can also reissue the original tool calls.

AppWorld Test-Normal, GPT-4.1, B = 6K. TGC is the percentage of tasks completed successfully.

MethodTGC (%)Result source
Prune66.1Paper, Table 6
Prune + recall66.7New run
ContextRender75.6Paper, Table 6
Reviewer_pyN5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ContextRender, a context management approach for long-horizon LLM agents. Instead of passing the full interaction history to the model, ContextRender maintains a persistent graph of historical messages, tool calls, and tool results, and selects relevant information under a fixed context budget.

The main component is Tool-Flow Analysis (TFA), which detects the reuse of identifiers from earlier tool results in subsequent model outputs and tool calls. This observed reuse signal is combined with recency and semantic relevance to determine which historical tool results should be included in the current context.

Experiments on AppWorld and 8-objective QA with three LLMs show that ContextRender generally outperforms the evaluated context management baselines while reducing standardized inference costs compared with using the full history. The ablation study also shows improvements from incorporating the observed reuse signal.

Critical strengths
  1. The paper addresses a practical problem in long-horizon agent systems. Simply removing older tool results or relying on semantic relevance may cause important information to be omitted from the context. Considering how historical information has been used during execution is a reasonable direction for improving context selection.

  2. The experiments cover multiple models and two agent benchmarks, with comparisons against several context management baselines. The ablation study also shows that adding the observed reuse signal improves task performance under the same context budget. These results provide some evidence for the practical usefulness of the proposed approach.

Critical weaknesses
  1. Limited novelty and a gap between the motivation and the actual method. The paper emphasizes modeling execution dependencies to identify historical information that may be important but is not obviously relevant to the current step. However, the proposed TFA mainly relies on exact, case-sensitive identifier matching. This is a relatively simple heuristic rather than a general analysis of execution dependencies. Important information may be used implicitly without repeating the original identifiers, and information that has not been reused before may still be essential at a later step. It is therefore unclear how effectively the method addresses the broader problem described in the motivation.

  2. Coarse-grained context selection and insufficient validation of dependency detection. The reuse of a single identifier increases the priority of the entire tool result, even when most of its content may no longer be useful. This could waste a substantial portion of the limited context budget. Moreover, the Later-Used Result Recall metric is itself defined using lexical reuse events, which are closely related to the signal used by TFA. Although the task performance improvements are encouraging, this metric does not directly establish whether the detected dependencies correspond to genuinely necessary information.

  3. Limitations in experimental evaluation. The evaluation compares several context management methods, but some recent adaptive approaches discussed in the related work, such as ACE, PACE, and Sculptor, are not included in the main comparison. Also, each method is executed only once per task, without reporting run-to-run variation. Finally, the reported cost reductions are based on standardized token costs and exclude local graph maintenance and context selection overhead.

Questions for authors
  1. Capturing implicit dependencies: Since TFA relies on exact identifier matching, how does the method handle cases where earlier information is needed without explicit lexical reuse? Could the authors provide an analysis of such cases and clarify what types of execution dependencies the current method can and cannot capture?

  2. Granularity and reliability of context selection: Why is reuse of an identifier used to increase the priority of the entire tool result rather than the relevant field or a smaller portion of the result? Would a more fine-grained selection strategy improve context efficiency? Additionally, could the authors evaluate dependency detection against independently identified necessary historical information, rather than relying mainly on lexical reuse events?

  3. Experimental comparisons and efficiency: How does ContextRender compare with more recent adaptive context management approaches such as ACE or PACE under the same context budget? Could the authors also provide more information about run-to-run variation and the runtime overhead of graph maintenance and context rendering, beyond the reported token-based cost estimates?

3 comments
AuthorsOpenReview ↗

Q1: Retaining useful information without an exact match.

Thank you for the thoughtful feedback.

Tool-Flow Analysis (TFA) detects exact, case-sensitive reuse of qualifying values from earlier tool results. A match increases the priority of the complete result. ContextRender also scores recency and semantic relevance, so information can remain available without an exact match to the particular value needed next (§3.3; Appendix A.2).

In Figure 2, one contact result returned at Turn 8 contains both Troy’s and Nancy’s email addresses. Reusing Troy’s identifier, tr_solo, at Turn 16 raises that result’s priority, preserving Nancy’s email for Turn 18 even though her address has not yet been copied forward or reused.

Exact matching can miss paraphrased or transformed values. To examine the practical effect of missed matches, we audited whether the affected values were still available in the selected source result or an earlier assistant message. They remained accessible in 95.8% of 1,614 AppWorld cases and 99.4% of 681 QA cases. Thus, within the audited cases, a missed TFA match usually did not remove access to the value. Recency and relevance continue to guide selection, and earlier assistant messages provide another route to retaining information.

AuthorsOpenReview ↗

Q2: Whole-result selection and what the evaluations establish.

We retain complete results because evidence that one field is useful can help preserve related fields before their first use. In Figure 2, Troy’s and Nancy’s emails occur in the same contact result. Reusing Troy’s identifier raises the result’s priority and keeps Nancy’s address available before she is referenced. Selecting only fields already matched would lose that benefit.

To test whether the reuse signal helps in practice, Table 2 compares the same whole-result selector with and without reuse, holding GPT-4.1, the 6K history budget, and the other selection components fixed. Adding reuse improves AppWorld Test-Normal task goal completion by 3.0 percentage points and 8-objective QA answer F1 by 2.3 points. Because the benchmarks score these outcomes independently of TFA’s matching rules, this comparison provides task-level evidence for the added signal’s usefulness. It tests reuse at the chosen granularity, rather than comparing whole-result and field-level selection.

Later-Used Result Recall measures whether the complete earlier tool result is visible immediately before a later turn reuses one of its values. Each selector replays the same recordings made with the Full history baseline, using only messages preceding the evaluated turn (Appendix B.4). Because the value may already appear in an assistant message, this metric measures source-result availability rather than whether that result was necessary.

Q3a. Comparison with adaptive context-management methods.

ContextRender uses observed value reuse to prioritize earlier results. ACE uses an LLM to choose full, summarized, or hidden historical steps; PACE selects detail levels by semantic relevance and offers a glimpse tool for recovery. Their described selection rules do not explicitly track observed value reuse.

We attempted to evaluate PACE using the authors’ implementation, but the code link was inaccessible. Our performance comparisons therefore cover the evaluated baselines; the ACE/PACE discussion identifies the difference in selection signals.

Q3b. Graph maintenance and context-rendering overhead.

Measured local graph-maintenance and rendering overhead is small relative to model execution: approximately 0.4 s per task, equivalent to 1.5% of the 27.5 s of model time measured in the live harness. To isolate this local processing, we replayed recorded AppWorld trajectories through a freshly started server on an Apple M4 laptop, without model API calls.

Reuse tracking and selection require no generative LLM calls, so adding the signal does not introduce another model-generation step. Each tool result is embedded once and its embedding reused; query embeddings update as the latest interaction changes. All embedding-token charges are included in the paper’s inference cost. The overhead comparison above measures the local graph and rendering computation.

Q3c. Uncertainty in the reported performance differences.

To assess which reported gains are supported by the recorded tasks, we computed paired bootstrap intervals for GPT-4.1 on 100 8-objective QA tasks. Each of 10,000 resamples uses the same task IDs for both methods, so the comparison remains paired by task. With one execution per method and task, this assesses uncertainty across tasks rather than variation across repeated executions.

The intervals in Table R1 lie above zero for ContextRender’s gains over Recency + Summary, Semantic Retrieval, and ACON. They include zero for the smaller differences against Full history, Prune, and Compact. The analysis therefore supports the first three improvements while distinguishing them from differences the recorded results do not resolve clearly.

Table R1. ContextRender’s F1 is 53.1. Gains are its F1 minus each comparison method’s F1, in percentage points; each gain has a separate 95% confidence interval.

Comparison methodMethod F1Gain (F1 points)95% CI for gain
Full history50.8+2.3[−0.03, +4.8]
Prune51.2+1.9[−0.7, +4.7]
Compact (LLM)52.0+1.1[−1.7, +4.0]
Recency + Summary48.5+4.6[+2.3, +7.2]
Semantic Retrieval46.4+6.7[+2.8, +10.8]
ACON48.1+5.0[+1.0, +9.5]

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.