acceptodds
Under review as a conference paper at ICLR 2027

Stable Geometry with Divergent Task Evidence for Efficient Long-Horizon Agent Compression

Abstract

Long horizon agents accumulate growing interaction histories that increase context and inference costs. We find that geometric redundancy alone is an insufficient criterion for safe compression. Although agent histories exhibit strong low dimensional structure, similar global geometry can preserve very different amounts of task evidence. At identical retained block counts, evidence aware selection raises next action Top 3 retention from 0.31 to 0.69, while centroid similarity remains 0.98. Controlled replacement further shows that action related information can be substantially altered while global geometric measures remain nearly unchanged. Motivated by this gap between geometry and evidence, we introduce Geometry Guided Evidence Preserving Memory (GEM), a training free compressor that protects task and execution evidence before using geometric residuals to complete coverage. GEM reduces mean combined token usage from 2.69M to 2.11M per task, a 21.4% reduction, while maintaining comparable task reward. Our results show that efficient agent history compression should optimize for preserved task evidence rather than geometric coverage alone.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Long-Horizon Agents, Agent Memory, Context Compression, Evidence Preservation, Geometric Selection, Efficient LLM Agents

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WHDbOpenReview ↗
Rating
2(+1)
Confidence
4(+1)
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies context compression for long-horizon LLM agents. It proposes that redundant messages are close in embedding space, but there still exist some important evidence that can not be deleted. The paper thus designs a training-free compressor GEM, which designs some heuristic metrics and rules to select and protect these evidence. Experiments show that GEM can save ~20% token usage without losing performance.

Critical strengths
  1. On the 260-task WorkBuddyBench, GEM reports a 21.4% token usage reduction, with performance keeping at ~70%, demonstrating that the proposed method may be effective in specific tasks.
Critical weaknesses
  1. The geometry property of redundant messages of LLM agents is not clearly verified. The refered related work (word embedding, bert, simcse) is much different with modern LLM agents and cannot serve as evidence. Fig.1 shows the low-rank structure of the aggregated trajectory embeddings, however, the details of these trajectories and the inference configurations are not disclosed.
  2. GEM relies on heuristic rules like recency, similarity and tool-calling patterns. However, the justification and necessity of these rules are not verified. The construction of state query, method of error detection, ranking of candidates are also not clearly demonstrated.
  3. The main experiments are conducted on a single WorkBuddyBench with only 260 tasks, and the other ablations use only 20 or 40 tasks. Such small evaluation scale cannot support the soundness and generality of the proposed method.
Questions for authors
  1. Is there any clear evidence proving that the low-rank structure of the aggregated trajectory embedding indicates the redundancy of the history messages? How are the trajectories in fig.1 sampled, are these trajectories representative?
  2. Please provide the implementation details of the evidence-protection rules, and what is their contribution to the proposed method?
  3. What's the performance of GEM on other common agent benchmarks (e.g. tau-bench, swe-bench)?
  4. What's the total cost of GEM? Though the token usage is reduced, the reconstructed context cannot hit prefix cache and enjoy discounts for cached tokens. Hence, the final cost may not be cheaper than other baselines.
14 comments
AuthorsOpenReview ↗

Response to Weakness 1: Geometric evidence and trajectory documentation

Dear Reviewer WHDb,

We sincerely thank you for the time and care you have devoted to reviewing our work. We also appreciate your recognition of the token reduction and comparable overall performance on WorkBuddyBench. Your detailed questions about geometric evidence, implementation, evaluation coverage, and system costs have helped us identify where the paper needs clearer explanations and stronger empirical support. We respond below to your three weaknesses and four questions in order.

Weakness 1: Evidence for the geometric properties of modern agent histories.

Thank you for emphasizing the need for direct evidence from modern agent trajectories and reproducible descriptions of the data and inference settings.

We agree that studies of word embeddings, BERT, and SimCSE provide background rather than direct evidence about modern agent histories. Our empirical claim is grounded in the agent trajectories analyzed in this work.

We distinguish two properties:

  • Representational redundancy: variation among historical block representations concentrates in a limited number of directions.
  • Execution redundancy: a historical block is unnecessary for subsequent task execution.

Low-rank structure characterizes the first property; assessing the second requires evidence about subsequent actions and task outcomes. This distinction motivates GEM's evidence protection before geometric completion.

To strengthen the empirical analysis, we provide:

  1. Spectral statistics from actual agent histories in two representation spaces, with feature-shift and Gaussian controls.
  2. The trajectory selection procedure, encoder settings, representation layers, and input limits.
  3. A follow-up analysis covering all 260 tasks, with Code/Office/Security/Web counts of 80/50/60/70.
  4. Controlled continuation experiments that measure final task Reward after replacing action-related or matched control blocks.

The numerical results and sampling details are provided in our response to Question 1. We will revise the presentation to make the distinction between representational structure and execution relevance explicit.

AuthorsOpenReview ↗

Response to Weakness 2: Justification and contribution of evidence-protection rules

Thank you for your careful examination of the evidence-protection rules. Your request for both implementation details and component-level evidence helps us make the method more transparent.

The rules preserve complementary execution cues: immediate working context, task relevance, artifact changes, recent failures, and information sources. We supplement the implementation description with an online component ablation on 40 fixed tasks, with 10 tasks from each domain.

Each variant starts from the same initial environment for each task. The acting model, tools, task time limits, evaluator, and stopping rules are held fixed.

Online component ablation. Combined includes model input, model output, and embedding tokens. Values are per task.

MethodNReward, mean ± SDReward varianceCombined, mean ± SD
Full GEM400.7073 ± 0.22460.05042.50M ± 2.10M
Geometry-only400.6468 ± 0.21910.04802.85M ± 2.73M
w/o Recent400.6299 ± 0.20810.04332.80M ± 2.21M
w/o Goal400.6513 ± 0.19380.03762.70M ± 1.94M
w/o State400.6732 ± 0.22810.05202.75M ± 2.56M
w/o Error400.6587 ± 0.20970.04402.62M ± 2.44M
w/o Source400.6724 ± 0.22560.05092.58M ± 2.44M

Paired comparisons. Reward differences are Full GEM minus the corresponding variant; positive token savings indicate lower Combined for Full GEM.

VariantReward differencePaired-difference SD95% CIToken savings95% CI
Geometry-only+0.06050.2909[−0.0248, 0.1470]12.28%[−18.72%, 33.64%]
w/o Recent+0.07750.2939[−0.0080, 0.1588]10.71%[−9.62%, 28.11%]
w/o Goal+0.05610.2831[−0.0295, 0.1407]7.41%[−18.20%, 28.20%]
w/o State+0.03410.2853[−0.0460, 0.1134]9.09%[−8.65%, 23.23%]
w/o Error+0.04860.2876[−0.0385, 0.1342]4.58%[−23.74%, 25.70%]
w/o Source+0.03490.3124[−0.0544, 0.1186]3.10%[−24.92%, 23.57%]

Full GEM has higher Reward point estimates than all six variants, with differences of approximately 0.034–0.077. The paired intervals span zero, reflecting substantial task-level variability in this 40-task experiment. The token-saving intervals also span zero, so we report the estimates together with their uncertainty.

Our response to Question 2 provides the block definition, state-query schema, error detection, candidate ordering, protection-membership audit, offline component analysis, and a comparison at identical retained block counts. These analyses separately examine implementation, representation-level evidence retention, and downstream execution.

AuthorsOpenReview ↗

Response to Weakness 3: Evaluation scale and coverage across benchmarks

Thank you for highlighting the evaluation scope and the evidence needed to assess generality. Your comment encouraged us to extend the experiments beyond a single benchmark and agent harness.

We added evaluations on three benchmarks:

  • τ-bench: all 165 tasks, comprising 50 airline and 115 retail tasks.
  • Terminal-Bench 2.0: 89 tasks.
  • SWE-bench Verified: a fixed subset of 100 tasks.

These experiments add 354 tasks covering interactive tool use, terminal operations, and software repair. Within each benchmark, Base and GEM use the same acting model and inference parameters, task set, tools, environment, and stopping rules; their difference is the context-management mechanism.

We also supplement the Full260 evaluation with task-paired uncertainty analysis. The overall mean Reward difference, GEM minus Base, is +0.003069, with a paired-bootstrap 95% CI of [−0.01971, +0.02568]. This describes uncertainty in the overall result alongside the domain-level differences.

The 20- and 40-task experiments address specific mechanisms through controlled comparisons. We report their task counts and uncertainty separately from the full benchmark evaluations. Our response to Question 3 provides the configurations and complete results for the three added benchmarks.

We will revise the conclusions to reflect the observed quality–resource trade-offs in these evaluated settings and the variation across task types.

Reviewer_kJgqOpenReview ↗
Rating
4
Confidence
1
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

In long-horizon language model agents, geometric redundancy in embedding space does not imply execution redundancy. While agent interaction histories concentrate strongly along a few low-dimensional principal directions, critical task evidence (e.g., whether a file was actually modified versus merely planned or read) often resides in low-energy orthogonal directions. Compressing history purely to preserve global representational geometry discards crucial next-action evidence.

Critical strengths
  1. authors introduce spectral subspace diagnostics and controlled block-replacement interventions that prove next-action evidence heavily populates the trailing 10% low-energy spectral tail of agent trajectories.

  2. the authors formalize context compression as constrained subspace coverage—initializing the basis with deterministic execution-state rules before running Modified Gram-Schmidt greedy residual completion.

3.they design an interception layer with atomic tool-block parsing, a δ=0.05\delta=0.05 per-step deletion damper, and incremental set union updates between global reselections.

Critical weaknesses

Code tasks represent the clearest boundary where compression yields minimal benefit. In Code, token reduction is tiny (decreasing from 1.37M to 1.35M tokens per task), and mean task reward experiences a slight decrease (from 76.99 to 75.25). This occurs because coding trajectories feature dense dependencies where most historical blocks contain critical syntax and logic, leaving very little redundant bloat to safely discard.

Questions for authors
  1. The evidence protection set (Pt\mathcal{P}_t) relies on specific hardcoded heuristics (e.g., Bash/Python write patterns, an error window of 8 blocks, and a source activation fraction of ≥0.50\ge 0.50). How fragile are these rules to agents that use custom, non-standard tool schemas or non-terminal environments? If an agent implements custom file-handling APIs instead of standard shell commands, does GEM fail to protect state?

  2. Code tasks exhibited a slight regression in task reward (76.99 →\rightarrow 75.25) with almost no token savings (1.37M →\rightarrow 1.35M). Given this negative result, why didn't the authors implement a domain-adaptive gating mechanism that automatically disables or relaxes compression when a task is classified as dense software engineering?

  3. GEM’s net token savings are partly offset by a spike in uncached input tokens (from 48.70K to 201.48K) due to periodic prompt rewrites invalidating prefix caching. Maybe a missing control is a system-level baseline that measures performance and latency when using modern prefix-caching infrastructure without compression, isolating whether GEM's advantages stem purely from token reduction or from attention de-noising ("Lost in the Middle" mitigation)?

  4. he deletion damper is fixed at δ=0.05\delta = 0.05 (retaining ≥95%\ge 95\% of request length per rewrite step). Perhpas testing tighter (δ=0.01\delta = 0.01) or looser (δ=0.15\delta = 0.15) constraints would show how aggressively the request length guard shapes end-to-end token efficiency versus agent stability?

  5. The online experiments are evaluated exclusively on a single acting model (DeepSeek-V4-Flash). maybe could test GEM with acting models of varying scales or architectures to see if different LLMs react differently to compressed history formats?

6 comments
AuthorsOpenReview ↗

Response to Weakness: Code-Task Quality and Token Trade-off

Dear Reviewer,

We sincerely thank you for the valuable time and effort you have devoted to reviewing our work so carefully and constructively. We are especially grateful for your positive assessment of the core findings and technical design. Your accurate understanding of the distinction between geometric and execution redundancy, and your specific recognition of the spectral diagnostics, controlled block replacement, constrained subspace coverage, atomic tool-block parsing, and incremental updates, are an important encouragement to us.

Your suggestions on custom-tool adaptation, dependency-dense tasks, prefix-caching infrastructure, deletion strength, and cross-model evaluation connect the algorithm closely to practical deployment. They reflect careful professional judgment and a comprehensive systems perspective. We greatly value their technical specificity and experimental guidance, and have added analyses and experiments to clarify applicability and quality-resource trade-offs.

Thank you again for recognizing this work and helping us improve it. Below, we address the main weakness you identified and your five questions in order.

Weakness: Code tasks show little token reduction and a slight quality regression.

Thank you very much for examining domain differences carefully. Your attention to the relationship between quality and resource savings in Code helped us present the variation behind the aggregate results more precisely.

MethodCode tasksScore (100 × mean Reward)Combined/task
Base8076.991.37M
GEM8075.251.35M

Mean Score decreases by 1.74 points while Combined decreases by 0.02M. In the added task-paired analysis, the Code mean Reward difference is -0.0174 on the 0-1 scale, with a Paired Bootstrap 95% CI of [-0.0523, +0.0178].

Your analysis of dependency-dense coding trajectories provides an important direction for understanding this result. Cross-file interfaces, exact syntax, file-state changes, and error-recovery records can remain relevant to later execution even when their representations are similar.

The revision will highlight this domain-specific trade-off. Dependency-aware adjustment of compression strength is a concrete direction for further work, discussed in our response to Question 2 below. We sincerely appreciate your help in clarifying GEM's applicability across task types.

AuthorsOpenReview ↗

Response to Question 1: Custom Tool Schemas and Non-Terminal Environments

Question 1: How sensitive are the protection rules to custom tool schemas and non-terminal environments?

Thank you very much for examining the tool interface and practical transfer of the protection mechanism. Your attention to the recognition rules and adaptation layer helped us specify how GEM connects to different agent environments.

Write recognition and source protection use tool names, argument fields, and recognizable operation semantics. Custom tools require adapter mappings to common read/write entities, success or failure states, and source semantics.

1. Current protection rules

ComponentImplementationAdaptation requirement
RecentRetain the latest 4 complete blocksIndependent of write-tool names
GoalFirst nonempty user task; prioritize a unique user_query; protect the most relevant blockMultiple query tags trigger an integrity error
State3 slots excluding Recent4; up to 2 prioritize latest writes to distinct files, then fill using the fixed State SchemaMap write entities and state fields
Write recognitionWrite/Edit/MultiEdit/NotebookEdit path fields and supported Bash/Python write patternsAdd mappings for custom file APIs
ErrorProtect up to 2 recognized failures within the latest 8 blocksMap failure status and error fields
SourceDeduplicate Read/NotebookRead/Glob/Grep by tool, path, and pattern; protect recent sourcesMap source fields; apply the fixed activation threshold

An atomic block comprises an assistant message containing tool calls and all complete tool results matched by call ID. Calls and results are retained or removed together. Non-tool messages and messages that cannot safely form complete blocks remain unchanged.

2. Schema-renaming audit

We performed a read-only audit on 66 eligible requests from one trajectory, changing tool and argument names and checking recognition outputs without executing custom tools.

Audit itemRequests with changed rules or outputs
Mutation/write-state recognition66/66
Source protection13/66
Candidate set66/66

Explicit inverse mappings restore rule consistency in this audit. Tool names and field mappings are therefore concrete integration requirements for custom schemas. The statistical unit here is a request; all 66 requests come from one trajectory.

3. Error audit in a non-terminal environment

Across 165 valid τ-bench tasks and 2,204 Actor requests, we audited nonempty top-level error fields in tool-result JSON. There were zero such structured errors, averaging 0.00/task. Another 22 requests had valid error-anchor audit records, with zero actual anchor activations.

This describes the observed behavior under that error definition; evaluating error-window recall requires data containing corresponding positive error cases.

We sincerely appreciate your professional guidance on adaptation. The revision will distinguish generic atomic-block handling from protection rules requiring semantic mappings, and specify integration requirements for terminal and non-terminal environments.

AuthorsOpenReview ↗

Response to Question 2: Dependency-Aware Compression Gating

Question 2: Why not automatically disable or relax compression for dependency-dense software-engineering tasks?

Thank you very much for proposing dependency-aware gating. Connecting the Code results to online policy design provides a concrete and valuable direction for improving the method.

This evaluation uses a uniform, frozen history-management policy so that behavior across tasks can be compared under the same configuration. The current implementation does not include an independently validated dependency-state classifier or dynamic gating threshold, and therefore retains the domain differences observed under the fixed policy.

We agree that compression decisions can further account for the dependency structure of the current execution state. We identify three directions:

  1. Dependency-state identification. Use cross-file dependencies, write-read chains, unfinished state changes, error-recovery chains, and relationships among historical evidence to identify stages requiring continued access to precise execution records.
  2. Dynamic compression strength. Depending on the current state, apply ordinary compression, reduce deletion strength, or temporarily retain the complete history. Prioritize execution evidence during dependency-dense stages, while using evidence protection and geometric completion when redundancy is higher.
  3. Independent policy validation. Determine dependency features, classification thresholds, and misclassification costs on development data, then evaluate quality and resource savings on independent tasks.

Your suggestion helped translate a domain-level observation into a state-level policy-design question. The revision will clearly distinguish this research direction from the current fixed policy and retain the actual Code results.

Reviewer_bSJCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether geometric redundancy is a reliable basis for compressing long-horizon agent histories. It finds that independently encoded interaction blocks have concentrated representation geometry, but subsets with similar global coverage can retain different amounts of information associated with the recorded next action. It proposes Geometry Guided Evidence Preserving Memory (GEM), a training-free external compressor. GEM first protects records identified through recency, goal relevance, state changes, errors, and sources, then adds blocks whose embedding directions are poorly covered by the selected set. This selected core is distinguished from the history actually sent to the agent: each global rewrite can remove at most 5% of the request's serialized characters, so additional blocks may remain. Retained call–result blocks are forwarded unchanged and in chronological order. On WorkBuddyBench Full260 with DeepSeek-V4-Flash, combined token usage, including embedding usage, decreases by 21.4%, while the mean score (reward multiplied by 100) changes from 69.87 to 70.18. Further evaluation includes a 40-task reference panel, a 20-task online ablation, and offline selection and replacement analyses.

Critical strengths
  1. The paper provides a concrete example of global geometric coverage missing action-related information. At matched block counts, Top-3 retention increases from 0.31 to 0.69, while centroid similarity remains 0.98 at the reported precision (Table 1). Here, Top-3 measures retention of the three history blocks closest to the recorded next action in embedding space. The result usefully shows that preserving the overall representation does not necessarily preserve those action-related records.

  2. GEM shows a promising token reduction on a substantial task set. Across 260 tasks in four domains, combined token usage falls by 21.4%, including embedding usage, while the mean score remains similar (69.87 to 70.18). The method preserves original call–result blocks without retraining the acting model. In the smaller online ablation, GEM also achieves a higher overall score and lower token usage than both random completion and geometry-only selection.

Critical weaknesses
  1. The added value of geometric completion over simpler selection remains unclear. GEM protects records using plausible cues such as recency, goal similarity, write patterns, and recent errors, then fills the remaining core slots by geometric novelty. The 20-task online ablation supports this completion rule over random selection, but does not test whether recency or goal relevance would work as well with the same protected set. The conservative deletion limit also retains history beyond the selected core, making it important to compare what the agent actually receives. Finally, the replacement experiment measures changes in embeddings rather than execution. It supports the paper's representation-level finding, but leaves the connection to execution-critical evidence unresolved.

  2. The comparison with existing methods leaves the quality–token advantage uncertain. Table 4 explicitly combines GEM results drawn from Full260 with baseline results from separate fixed-40 runs. The paper does not clarify whether relevant conditions, including prompts, tools, environments, and stopping rules, match across these runs. In that panel, GEM scores higher than the base agent (73.12 versus 67.54), but also uses more tokens (2.26M versus 1.89M). This does not negate the Full260 reduction; it motivates comparisons at similar token usage to determine the advantage over other compressors. Full260's similar aggregate score is encouraging, but task-paired uncertainty is needed to assess the domain-dependent losses already acknowledged by the authors.

Questions for authors
  1. What does geometric completion add? Please compare it with filling the remaining core slots by recency or similarity to the current goal, keeping the protected set, eligible candidates, and online retention rules unchanged. Report task scores and token usage at comparable retained-context budgets. Because the 5% per-rewrite character limit can retain blocks outside the core, please also report the sizes of the selected core and the history actually forwarded. This would show whether the geometric rule improves outcomes when the alternatives retain similar amounts of context.

  2. What explains the different reward–token tradeoffs in Tables 3 and 4? Please report the base-agent results from Full260 restricted to the same 40 task IDs used in Table 4, alongside the corresponding GEM results. This would help distinguish the effect of the task subset from differences between evaluation runs. Please also clarify whether the Table 4 runs shared the same model version, prompts, tools, environment, and stopping rules, and explain the adaptation of ACON-Core. If relevant conditions differed, a matched comparison with representative baselines would help. Comparisons at similar token usage would further clarify GEM's advantage over lower-token methods such as PACE and ACON-Core.

  3. How consistent is quality preservation across tasks? For each Full260 task, please compute the score difference between GEM and the base agent, and report confidence intervals for the mean difference overall and within each domain, for example by bootstrapping task pairs. Please also report the proportion of tasks with a practically meaningful decrease, stating the threshold used. These results would help determine whether the similar aggregate score reflects broadly preserved quality or gains on some tasks offsetting losses on others.

  4. What does the replacement experiment establish about execution? Section 3.3 identifies the matching variables; please additionally explain where the replacement blocks come from, how close the matched blocks must be on those variables, and whether Table 2's deltas are relative to the original selection or the matched-control condition. On a representative subset of checkpoints, can the agent be resumed from the same environment state after each intervention to compare valid next-step execution or final task reward? This would connect the reported embedding changes to the execution relevance of the affected records.

9 comments
AuthorsOpenReview ↗

Response to Weakness 1: Geometric Completion and Execution Relevance

Dear Reviewer,

We sincerely thank you for the valuable time and effort you have devoted to reviewing our work with such care and rigor. We are especially grateful for your recognition of the finding that global geometric coverage and action-related evidence retention can diverge, and for your positive assessment of GEM's Full260 token reduction, preservation of complete call-result blocks, and online ablation. Your encouragement means a great deal to us.

Your specific suggestions on the contribution of geometric completion, evaluation comparability, task-level quality changes, and the execution meaning of replacement experiments helped us identify the evidence to strengthen. We greatly appreciate your efforts to improve this work. Below, we address the two weaknesses and four questions raised in your review.

Weakness 1: The contribution of geometric completion over simpler selection rules, and its connection to execution, remain unclear.

Thank you very much for your careful analysis of geometric completion. Comparing reasonable alternative completion rules under the same protection policy, while measuring the history actually received by the agent, provides a more direct online evaluation.

We added Recent-fill and Goal-fill, which fill the remaining core slots by recency and similarity to the current goal, respectively. All three methods share the protection rules, candidate-pool construction, reference-K calculation, coverage threshold, online append policy, and 5% character-deletion cap. They change candidate ordering and use the same coverage, capacity, and candidate-exhaustion stopping rules.

The new experiment includes 40 independent tasks, 10 per domain, producing 120 execution records. We report Reward, cumulative tokens, selected-core size, and actual forwarded history in Response 3.

We also added same-state execution interventions covering 20 tasks and 40 checkpoints. These compare intact context, targeted replacement, and matched-control replacement using final Reward, first-step tool validity, and resource consumption, as detailed in Response 6. This connects the original representation-level intervention to actual task execution.

AuthorsOpenReview ↗

Response to Weakness 2: Evaluation Comparability and Quality Preservation

Weakness 2: Evaluation snapshots differ, and the quality-token trade-off and consistency of Full260 quality preservation need further analysis.

Thank you very much for your specific suggestions on evaluation comparability and task-level variation. They helped us distinguish task-subset effects, execution variability, and history-management effects.

The evaluations use the same task prompts, acting model, tool protocol, benchmark environment, and stopping rules. Original Table 4 combines distinct snapshots: GEM is extracted from Full260, while the reference methods come from separate fixed-40 evaluations.

In Table 4, GEM scores 73.12 versus 67.54 for Base, with Combined usage of 2.26M versus 1.89M. This snapshot therefore exhibits higher task quality with higher token usage.

1. Five repeated fixed-40 evaluations

We added five independent repetitions using the same 40 task IDs, 10 per domain. Each repetition reinitializes the environments while retaining the model, prompts, tools, environment version, and stopping rules. Score = 100 × mean Reward.

RunBase ScoreGEM ScoreBase Combined/taskGEM Combined/task
169.5271.091.986M1.863M
263.6167.172.899M1.720M
365.4868.062.174M2.620M
468.5772.181.897M2.146M
567.5270.102.882M1.873M
Mean ± SD66.94 ± 2.3969.72 ± 2.082.368 ± 0.488M2.044 ± 0.357M

GEM's mean Score is 2.78 points higher, and mean Combined usage is 13.65% lower. GEM uses fewer tokens in Runs 1, 2, and 5, and more in Runs 3 and 4. Task-level logs show that a small number of long trajectories can substantially affect cumulative usage, motivating reporting each repetition and its dispersion.

2. Repeated information acquisition under a sliding window

In four jointly valid tasks under the K=5 sliding-window baseline, the Repeated Action Rate (RAR) is 10.35%. There are 8 repeated successful actions per task on average, including 7.5 that return exactly the same content as before. All repeated successful actions are reads.

Separately, we report a long-tail case outside this four-task aggregate: canvas-webgl-scene-L4-026. It repeatedly reads the same six files:

FileReads
mountain-overlook-scene-brief.md43
camera-and-lighting-constraints.yaml43
scene-graph.json43
scene-module-cues.json43
implementation-build-notes.md42
dependency-task-chain.md42

The same six-file sequence appears in log-line ranges 37-47, 49-59, and 61-71, with identical returned content.

There are 256 reads: 6 first reads and 250 repeats. Repeats constitute 97.66% of reads; relative to all 259 successful tool actions, RAR is 96.53%. Input usage reaches 6.6717M tokens, versus 21.6K output tokens. The task reaches the 256-model-call limit and receives Reward 0.

These records distinguish the size of currently visible history from cumulative task-level consumption: reacquiring and forwarding existing information also increases cumulative input. We report the two diagnostic cohorts separately as complementary observations of trajectory-dependent resource use.

Response 5 further reports Full260 task-paired uncertainty and quality-decrease proportions.

AuthorsOpenReview ↗

Response to Question 1: Controlled Completion-Rule Comparisons

Question 1: What does geometric completion add relative to recency or goal-similarity completion at comparable retained-context budgets, and how large are the selected core and actual forwarded history?

Thank you very much for this targeted comparison suggestion. We added both online alternatives and measured selection separately from forwarding.

1. Online comparison

The experiment includes 40 independent tasks, 10 each from Code, Office, Security, and Web. Tasks satisfy a common eligibility condition ensuring unprotected candidates are available for all three methods at the initial global-selection checkpoint.

Each task starts from the same initial environment state in GEM, Recent-fill, and Goal-fill, yielding 120 execution records. Protection rules, candidate construction, reference K, coverage threshold, and the 5% deletion cap are shared. Each ordering stops at the coverage target, core capacity, or candidate exhaustion.

MethodMean Reward [SD]Combined/task [SD]
GEM0.7275 [0.1892]2.305M [1.206M]
Recent-fill0.7168 [0.1801]2.423M [1.249M]
Goal-fill0.7238 [0.1848]2.230M [1.162M]

Against Recent-fill, GEM has 1.07 percentage points higher mean Reward and approximately 4.85% lower Combined usage. Against Goal-fill, Reward is 0.37 points higher and Combined usage is 3.36% higher. Displayed values are rounded.

2. Domain results

DomainGEM RewardRecent RewardGoal RewardGEM CombinedRecent CombinedGoal Combined
Code0.7350.7420.7402.14M2.23M2.05M
Office0.8000.7800.7951.02M1.05M0.98M
Security0.5150.5250.5204.01M4.25M3.97M
Web0.8600.8200.8402.05M2.16M1.92M

GEM has higher Reward in Office and Web; Recent-fill and Goal-fill have higher Reward in Code and Security. Goal-fill uses fewer tokens in all four domains.

3. Selected core versus forwarded history

The 5% cap applies to the complete serialized request at each global rewrite, so the forwarded set can exceed the selected core.

MethodNCore blocksFill blocksForwarded blocksCore charsForwarded-history charsPrompt tokens/Actor callChar retention
GEM409.42.214.383.4K143.4K43.98K95.80%
Recent-fill409.72.614.888.2K154.9K43.45K96.30%
Goal-fill409.22.014.180.5K146.1K43.50K96.10%

Core, fill, and forwarded-block means use actual global-selection events; prompt-token statistics use Actor requests. The three methods have similar per-request input sizes, with the specific forwarding differences shown above.

MethodGlobal selections/taskCumulative history-input retention
GEM2.888.7%
Recent-fill2.691.2%
Goal-fill2.789.8%

Cumulative retention is the sum of forwarded-history lengths divided by the sum of corresponding full-history lengths across tasks and requests. History carried in multiple requests is counted repeatedly. This measures cumulative history input, rather than the final retained fraction.

4. Execution trajectories and token composition

MetricGEMRecent-fillGoal-fill
Actor calls/task49.853.448.9
Tool calls/task37.540.836.8
Repeated reads/task1.93.12.0
Observation tokens/task0.58M0.66M0.56M
Long-context tasks6/409/406/40

A long-context task has at least one Actor request reaching 90% of the 1M-token context window.

Token category, M/taskGEMRecent-fillGoal-fill
Model Input2.1902.3202.127
Model Output0.0700.0650.065
Embedding0.0450.0380.038
Combined2.3052.4232.230

Model Input is partitioned into cached and uncached input using actual usage logs and aggregated by request. Observation tokens are included in Model Input. Combined sums Model Input, Model Output, and Embedding usage.

Cumulative usage reflects both forwarded content and subsequent execution. On this cohort, GEM achieves higher Reward with lower Combined usage than Recent-fill, and slightly higher Reward with higher Combined usage than Goal-fill. The revision will report these results together with the forwarding audit.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.