acceptodds
Under review as a conference paper at ICLR 2027

Memory Has an Interface: From Facts to Accepted Actions

Abstract

The same remembered facts can lead to different task outcomes depending on how they enter an agent's context and how its responses reach execution. We study this interface by removing reference-action facts from user requests and restoring byte-identical marked JSON either inline or through a historical assistant–tool exchange. Across 146 selected tasks and three models at one primary seed, historical delivery achieves 29.68% success versus 23.52% for inline delivery, a gain of 6.16 percentage points. A separate 59-task retail study with quantized Qwen3.6-27B reveals a distinction between proposing an action and having it accepted: all 25 first-response pairs with different native acceptance outcomes propose identical tool names and arguments. We then cross the delivery interface with the execution rule in 236 complete interactions. Native rules yield 1/59 inline and 6/59 historical successes; admitting schema-valid text-plus-tool responses to execution raises both totals to 56/59. Content controls further establish the value of correct facts through both interfaces, with model-dependent delivery contrasts. Together, these measurements identify message construction and response acceptance as consequential stages between selected facts and completed tasks, and provide a concrete framework for evaluating how agent memory becomes action.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agent memory, large language models, context construction, tool-use agents, agent evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yYgAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether the way remembered facts are delivered to an agent, and the way the agent’s reply is admitted to execution, change task outcomes when the facts themselves are held fixed. A deterministic selector extracts one or two scalar argument values from each task’s first reference tool call, redacts them from the user request, and restores the identical canonical JSON either inline in the user message or as the result of a fabricated historical memory.read exchange. On 146 selected tasks from AgentDojo, τ-bench (airline, retail) and ToolSandbox, with three sub-30B open models at one seed, historical delivery reaches 29.68% success against 23.52% for inline delivery (+6.16 pp). A follow-up on the 59 retail tasks with a quantized Qwen3.6-27B shows that the two interfaces propose identical tool calls in 58/59 cases but differ in whether the native communication check accepts the first reply (1/59 vs 24/59); admitting schema-valid mixed text-plus-tool replies raises both interfaces to 56/59. Context-state and content controls on AgentDojo find that placing the fact in a trailing assistant message is the worst construction and that correct content is worth roughly 40 pp in either interface.

Critical strengths
  • This paper explores a practical question: whether the way remembered facts are delivered to an agent, and the way the agent’s reply is admitted to execution, change task outcomes when the facts themselves are held fixed.
  • The paper isolates a stage of memory use that end-to-end benchmarks cannot see; the chain fact → message → proposed call → accepted reply → execution → score is a useful vocabulary.
  • The response and protocol studies are the most informative part: 58/59 identical proposals with divergent acceptance (1/59 vs 24/59), and parity once mixed replies are admitted (56/59 each; Table 4, Fig. 3), cleanly show that the acceptance rule, not the decision, carries the difference.
Critical weaknesses

W1. The experiment does not measure what the paper says it measures. The stated object is the interface through which remembered facts enter an agent’s context. The design, however, takes the facts from the conversation itself: a value that already appears in the user request is cut out of the request and put back by one of two constructions (Section 3.1). What is being measured is therefore how well a model recovers a literal string that was removed from, and then reattached to, a damaged request. This is parameter recovery, as the authors call it, not memory use. Nothing is retrieved, nothing competes with the fact, and the request no longer resembles anything a user would write. The authors acknowledge in §8 that facts fixed from prior interactions would be the appropriate source; that design was not pursued.

W2. Both delivery arms are artificial, so the contrast between them is hard to interpret. The inline arm appends to the user message a data-handling sentence, a machine-readable version marker and a compact JSON object, a construction with no obvious counterpart in training data or deployment. The history arm fabricates an assistant memory.read call and its tool result and places the model after a trajectory it did not produce. The comparison is thus between two off-policy contexts.

W3. The method is training-free, yet it is evaluated only on small models, and the paper’s own mechanism predicts that the result depends on that choice. All experiments use sub-30B open models; the protocol studies use a single quantized checkpoint that also serves as the user simulator. What the paper finds is that under inline delivery these models more often produce replies that mix text with a tool call, and that τ-bench’s half-duplex rule rejects such replies; the two interfaces propose the same tool calls and differ only in this respect, and once schema-valid mixed replies are admitted they are indistinguishable (§6, Table 4). Format compliance of exactly this kind is what larger models do most reliably. By the paper’s own account, then, the delivery gap under the native rule should shrink or vanish on a capable model, as it already does under the relaxed rule. Because the manipulation requires only inference, a frontier-class model, or a harness that tolerates mixed replies, could have been included at little cost; without either, the direction of the headline result on the models that matter in practice is unverified.

Questions for authors

Q1. Does the History–Inline contrast survive on a frontier-class model, and does it survive on a harness that accepts replies mixing text with a tool call?

Q2. Would the authors consider the design described in §8, with facts fixed from prior interactions rather than taken from the current request, as the primary experiment, keeping the present redaction design as a secondary control?

Q3. If the consistency gate used for the context-state study (a minimum effect with the same direction across models and seeds) is applied to the main History–Inline contrast using the three-seed data, is it met?

1 comment

Thank you for proposing prior-interaction records and an explicit consistency test. The new continuation experiment gives a direct memory-utility result: supplying earlier interaction records eliminates 29 repeated read queries and reduces model requests by 34.44% on 12 tasks. The original-weight GLM repeat separately tests the admission mechanism.

Q1 — Larger model and mixed-response-compatible execution. With the original GLM-4.7-Flash agent and Qwen3.6-27B simulator in BF16, we completed the 59-task delivery-by-admission experiment at seed 5201. Native admission gives 0/59 Inline and 10/59 History DB successes. Admitting schema-valid mixed text-and-tool responses raises these to 42/59 and 45/59; first responses are accepted on all 59 tasks in each arm. All ten native History-only successes propose the same first tool action as Inline, whose response is rejected.

The paired History–Inline difference is +16.95 percentage points under native admission (95% task-bootstrap interval [8.47, 27.12]) and +5.08 under permissive admission ([−6.78, 18.64]). Their interaction is −11.86 points ([−28.81, 5.08]). A positive residual delivery effect under permissive execution is therefore unestablished.

For the larger API agent, DeepSeek-V4-Pro-0813, we tested prior-interaction records under permissive execution: retrieved context succeeds on 12/12 tasks and History on 11/12, a paired difference of −8.33 points ([−25.00, 0.00]). This experiment uses temperature 0 and non-thinking mode. It tests natural continuation with records; the original 59-task redaction-based History–Inline contrast on this larger model remains unmeasured.

Q2 / W1–W2 — Prior interactions as the basis for the memory-use argument. Yes. We propose to lead the memory-use argument with prior-interaction records and use redaction as a controlled analysis of delivery and admission. The new 12-task panel is exploratory, alongside the existing 146-task controlled panel.

Each new current request is an intact user reply from a public GPT-4.1-generated benchmark conversation. The record contains preceding dialogue and read-only tool results, selected by a fixed task-ID hash order and prefix rule. No value is removed from the current request. A persistent store reads the session record and supplies identical serialized bytes through retrieved context or History. The assistant–tool exchange in History is constructed by the application; this tests a context-reset continuation with stored records, including that explicit delivery choice.

ConditionDB successAgent-plus-simulator requests
No supplied record11/12180
Retrieved context12/12118
History11/12118

Both record deliveries use fewer requests on all 12 tasks. Mean paired reductions are 5.17 requests per task, with 95% task-bootstrap intervals [4.00, 6.17] for retrieved context and [4.25, 6.08] for History. Users can supply information again in every condition, accounting for the high no-record completion rate. A post-hoc trace check finds 29 exact repeats of earlier read queries without records and zero with either delivery, before the first non-read business call. On the ten tasks completed by both History and no-record conditions, History saves 5.50 requests on average. The measured benefit is reduced reacquisition effort, with no established History advantage over retrieved context. Retrieval competition and updating are untested.

Q3 — Does the full consistency gate pass? No. On the matched 40-task, three-model, three-seed panel, History succeeds in 75/360 cells versus 65/360 for Inline: +2.78 points, with a task-stratified 95% interval [−1.39, 7.22]. The exact two-sided whole-task sign-flip test gives p = 0.291, a single post-hoc contrast outside the original four-contrast Holm family. The minimum-effect (at least 2 points) and two-of-three pooled-model-direction components pass. The within-every-seed direction and statistical components fail: at seed 5202, Qwen, GLM, and Nemotron differ by 0.0, +10.0, and −5.0 points. Consequently, the three-seed data support no claim of a stable History–Inline advantage.

Reviewer_oNnTOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper compares two ways of supplying the same factual JSON to an agent: inline in a request or through a historical tool exchange. Historical delivery improves success in the primary multi-model experiment. In a separate retail study, accompanying text changes whether an otherwise identical tool call is accepted. Allowing schema-valid mixed responses greatly improves both delivery conditions. Further experiments test context layouts and correct versus scrambled content.

Critical strengths
  1. The two delivery conditions receive identical factual bytes. Recording the proposed action separately from execution acceptance makes it possible to identify a protocol failure even when the agent selected the right action. Task outcomes alone would miss this distinction.

  2. All 25 pairs with different acceptance outcomes propose identical tool names and arguments. That is persuasive evidence that response acceptance can change the apparent success of factual use even when the proposed action stays the same.

  3. The protocol intervention follows execution through to task outcomes. Under permissive routing, both interfaces succeed on 56 of 59 tasks. The result shows that the acceptance rule has consequences beyond how a parser classifies the first response.

  4. The reporting includes negative results and uncertainty, rather than only the aggregate gain. Per-model results and task-cluster analyses reveal where the evidence comes from. The limits of the historical records and content controls are also acknowledged.

Critical weaknesses
  1. The original requests, first responses, and parts of the adapter and dispatcher code are missing. That prevents direct verification of the mechanism behind the headline result. Reconstructing current templates does not recover the original inputs. This matters particularly for the assistant-final condition, where the generation boundary is unknown.

  2. The follow-up uses a different quantized Qwen configuration and one seed per study. The conditions differ from the primary experiment. GLM is not included, despite contributing the largest net tau gain in the main panel. The mechanism is demonstrated in the follow-up setting. Whether it explains the original result remains unresolved.

  3. The experiment tests a narrow form of factual recovery: short parameters selected from reference actions, with a median payload of one key and 43 bytes. Moreover, 22 of the 27 net additional successes come from tau tasks, while ToolSandbox has no successes in any arm. This supports a focused interface diagnostic, but gives little evidence about retrieval or less constrained memory use.

  4. A robust advantage for historical delivery remains unestablished. The delivery-by-protocol interaction is -8.47 points with a 95% interval of [-18.64, 1.69]; the content-by-interface interaction is also inconclusive. Both largest adjacent context-state effects involve assistant-final E2, whose historical generation boundary is unknown. The controls therefore do not isolate a stable delivery effect.

Questions for authors
  1. Can you repeat the focal comparison under the original model configurations, especially GLM? Please retain the rendered requests and parsed responses, including proposed calls and acceptance events. Those records would allow the claimed mechanism to be checked against the success gain.

  2. Does the crossed delivery/protocol result hold across seeds and another model contributing to the primary gain? Please report paired effects and uncertainty, including whether a delivery difference remains when both interfaces reliably reach execution.

  3. Can you control new-reply versus assistant-continuation boundaries explicitly in the E2 comparisons? Does the diagnostic also hold for memory records obtained independently of the target reference actions?

1 comment

Thank you for identifying the original-configuration repeat and prior-interaction records as decisive tests. We have completed both. In the new GLM repeat, all ten History-only native successes propose the same first tool action as Inline; their responses diverge at admission.

Q1 — Original GLM configuration and inspectable execution. We repeated all 59 retail tasks with the original GLM-4.7-Flash agent and Qwen3.6-27B simulator weights in BF16, seed 5201, on a new GPU placement. We crossed delivery with native admission (P0) and admission of schema-valid mixed text-and-tool responses (P1), retaining native tool execution and DB scoring.

Delivery / ruleFirst response acceptedFirst response executes a toolDB success
Inline / P02/590/590/59
History / P032/5929/5910/59
Inline / P159/5957/5942/59
History / P159/5957/5945/59

The relevant component is the native half-duplex rule: accompanying text causes an otherwise valid tool response to be rejected. Under P0, first responses mix text and tool calls in 56/59 Inline and 27/59 History cases. Across 50 pairs with identical nonempty proposed tool sequences and parsed arguments, 25 have different admission outcomes. These include all ten History-only successes. This locates an admission difference in every task carrying the new native gain.

All 236 trajectories are complete. We retained actual request bodies, rendered prompt tokens, parsed responses, admission events, and tool executions; rendered token counts match provider counts for all 1,832 requests. Native success is 0/59 versus 10/59 in this repeat, compared with 0/59 versus 11/59 historically. These are prospective records; the missing original requests remain unavailable.

Q2 — Another contributing model, paired uncertainty, and seeds. The original-weight GLM result extends the crossed test beyond quantized Qwen. The paired History–Inline DB-success difference is +16.95 percentage points under P0 (95% interval [8.47, 27.12]) and +5.08 under P1 ([−6.78, 18.64]). The interaction, defined as the P1 difference minus the P0 difference, is −11.86 points ([−28.81, 5.08]). Intervals use 10,000 task-bootstrap draws retaining all four conditions together. Adding the communication condition to DB success gives 42/59 versus 44/59 under P1.

Thus both interfaces reliably reach first-response execution under P1, while the residual task-success difference and its reduction remain statistically uncertain. The earlier Qwen panel gives 1/59 versus 6/59 under P0 and 56/59 for both under P1. Each crossed panel uses one seed; the requested cross-seed crossed replication is still missing. The content-by-interface interaction also remains inconclusive.

Q3 — Prior-interaction records and the E2 boundary. We completed 12 exploratory natural continuations with a DeepSeek-V4-Pro-0813 API agent and a separate Qwen3.6-27B simulator, temperature 0, non-thinking mode. Complete current user replies and preceding dialogue/read-only results come from public GPT-4.1-generated benchmark logs, independently of target reference-action arguments. A persistent store retrieves each session's record. Retrieved-context and History delivery receive identical record bytes; History uses an application-constructed assistant–tool exchange after a context reset.

Under P1, DB success is 11/12 without a record, 12/12 with retrieved context, and 11/12 with History. Both record deliveries reduce agent-plus-simulator requests from 180 to 118 (34.44%), with fewer requests on every task. In a post-hoc trace count, exact repeats of prior read queries before the first non-read business call fall from 29 to zero in both arms. This demonstrates a concrete use of prior records: avoiding information reacquisition during continued service.

All new continuation inputs end with a complete user turn and request a fresh assistant reply. The original E2 new-reply/assistant-continuation comparison remains untested; its historical boundary is unknown, so those E2 contrasts cannot establish a context-state mechanism.

Reviewer_oMTWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how identical selected facts affect agent execution when delivered inline or through a historical assistant–tool exchange. Across a constructed 146-task, three-model panel, historical delivery improves success by 6.16 percentage points. A separate retail follow-up distinguishes proposed tool actions from message acceptance and shows that allowing schema-valid mixed text-and-tool responses raises both interfaces to 56/59 successes. Additional experiments vary context structure and content correctness.

Critical strengths

The paper studies an interesting question about how different ways of presenting the same information can affect agent performance. I also appreciate that the authors test different models and benchmarks, and further investigate how the execution process affects the results.

Critical weaknesses
  1. My main concern is the limited novelty of this work. The main finding is that putting the same information in different message formats can lead to different results. However, this problem has already been studied in previous work on prompt formatting and context construction. The paper does not propose a new memory method or provide much theoretical explanation. I am not sure what new knowledge this paper provides beyond these existing studies.
  2. I am also not convinced that this is really an agent memory problem. The experiments mainly extract some values from the correct tool arguments, remove them from the original request, and then provide them back as JSON in different message positions. There is no actual memory retrieval, storage, or updating involved regarding the previously long context. This seems more like a prompt formatting experiment rather than a memory study. I think the authors should better explain how this setting represents real agent memory systems and why the findings are useful for memory research in modern background.
  3. Another major concern is that the main improvement may come from the execution framework rather than the memory interface. The paper reports that historical delivery performs better than inline delivery. However, in the follow-up experiments, once the framework allows valid responses containing both text and tool calls, both methods achieve exactly the same success count (56/59). This makes me question whether the improvement really comes from better memory usage? It seems possible that the results are mainly caused by how the framework handles model outputs, rather than how the information is presented...
  4. I also think the experimental results are not strong enough to support a general conclusion. Most of the improvement comes from the τ-bench tasks, while the results on other benchmarks are much weaker. The effects also vary across models. Although the paper compares several message formats, it does not clearly explain why some formats work better or which part of the interface is responsible. Without a clear explanation or consistent results, it is difficult to know what practical lessons we can learn from this study.
  5. Finally, the paper does not compare with real memory systems. Existing methods often use retrieval, conversation history, or structured memory to help agents complete tasks. However, this paper only compares two ways of presenting a small amount of JSON information. I think experiments with actual memory systems would make the motivation and practical value much stronger
Questions for authors

n/a

1 comment

The new experiments identify where a delivery effect enters execution and demonstrate useful memory use with prior-interaction records. In the original-weight GLM repeat, all ten History-only native successes propose the same first tool action as Inline, but admission differs. In a separate DeepSeek continuation study, stored records eliminate 29 repeated read queries and reduce model requests by 34.44%.

What new knowledge does the study add? The contribution is an empirical account of how a fact becomes an executed action. Sclar et al. study few-shot prompt-format sensitivity; Chen et al. study role-dependent explicit error flagging; Peleg Pelc et al. describe agentic contexts and compare context layouts. Our crossed delivery/admission experiment traces identical proposed actions through response acceptance, actual execution, and native DB outcomes. It identifies a system-level mechanism that task scores alone cannot distinguish from improved factual use.

Which component is responsible, and is the framework carrying the gain? The implicated component is the native half-duplex admission rule, which rejects accompanying text even when a tool call is valid. With the original GLM-4.7-Flash agent and Qwen3.6-27B simulator weights in BF16, seed 5201, first responses mix text and tool calls in 56/59 Inline and 27/59 History cases under native admission (P0). Among 32 pairs with different admission outcomes, 25 propose identical tool names and parsed arguments, including all ten History-only successes.

RuleInline first response acceptedHistory first response acceptedInline DB successHistory DB success
Native (P0)2/5932/590/5910/59
Admit schema-valid mixed replies (P1)59/5959/5942/5945/59

Under P1, first-response tool execution occurs in 57/59 tasks for both deliveries. The paired History–Inline success difference falls from +16.95 percentage points (95% task-bootstrap interval [8.47, 27.12]) to +5.08 ([−6.78, 18.64]); the interaction is −11.86 ([−28.81, 5.08]). The framework therefore contributes directly to the observed delivery effect. The intervals leave the residual task-success effect and the size of its reduction uncertain. The practical lesson is to evaluate memory delivery jointly with response admission and report proposed actions separately from executed calls.

Does this extend to actual stored interaction records? We tested a persistent session-record lookup on 12 natural retail continuations using a DeepSeek-V4-Pro-0813 API agent and Qwen3.6-27B API simulator, temperature 0, non-thinking mode. Current user replies are intact. Records contain preceding dialogue and read-only tool results from public GPT-4.1-generated benchmark logs, independently of target reference-action arguments. Retrieved context and History receive identical record bytes; History's assistant–tool exchange is application-constructed. This implements continuation after a context reset within the same session.

Under P1, success is 11/12 without a supplied record, 12/12 with retrieved context, and 11/12 with History. Total agent-plus-simulator requests are 180, 118, and 118. Both deliveries reduce requests on every task, averaging 5.17 fewer per task (95% paired intervals [4.00, 6.17] and [4.25, 6.08]). A post-hoc trace count locates the saving in information reacquisition: exact repeats of prior read queries before the first non-read business call drop from 29 to zero. History also saves 5.50 requests on the ten tasks both it and the no-record condition complete. This supplies a concrete memory-utility endpoint on a larger API agent.

Generalization and memory-system comparisons. The original gain is concentrated in tau tasks, with zero ToolSandbox successes. The new GLM crossed panel uses one seed, and the 12-task continuation panel is exploratory. Together they support the measured admission mechanism and reduced reacquisition cost in retail. The stored-record experiment evaluates exact session lookup and two delivery constructions. Comparisons with named memory systems, retrieval ranking, updating, and cross-session transfer remain unperformed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.