acceptodds
Under review as a conference paper at ICLR 2027

Fresh Memory, Stale Plans: Derivation Currency for Distributed LLM-Agent Memory

Abstract

A large language model (LLM) agent that inherits a plan through shared memory can hold the latest requirement yet act on a plan derived from an older one: fresh memory, stale plan. Freshness checks miss this failure because they compare local copies with current state (observation currency) rather than the inputs the plan was derived from (derivation currency). PlanFence makes derivation currency checkable after a handoff. Stored plans carry exact links to their recorded inputs; before a protected action, PlanFence follows those links to an action-specific dependency frontier, asks each input's owner for its current head, refreshes what changed, and allows one replan before blocking. Application code supplies the links and declares the scope; no shared memory service is required. Holding the native S-Bus validator fixed, supplying inherited input versions raises detected handoff conflicts from 0/30 to 30/30: retained evidence is the missing ingredient. In 30 live five-agent workflows with a revision inserted after planning, a freshness-only executor acts on the stale plan every time, whereas PlanFence, like a centralized-lineage baseline that requires a shared store, completes all 30 correctly. In matched replay under emulated LTE traces, PlanFence's stall stays within 143-237 ms per action across a 64 range of update rates while per-update synchronization grows from 52 to 1,196 ms; synchronization is cheaper only at the lowest tested rates. Scoping queries to the declared dependencies holds traffic at 8.1 KiB per action, a tenth of all-key validation at 128 keys; at full scope the two tie.

Primary area:
foundation or frontier models, including LLMs
Keywords:
distributed systems, multi-agent systems

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Sy5KOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

PLANFENCE addresses a failure mode in distributed LLM-agent systems where an executor has current state but inherits a plan derived from older inputs. It stores exact parent links with derived artifacts, reconstructs an action-specific dependency frontier, validates recorded versions against owner heads, and replans once on mismatch. The paper evaluates this through evidence ablations, live five-agent workflows, and controlled replay under different update rates and network conditions.

Critical strengths

1. The distinction between observation currency and derivation currency is crisp and practically important: current local state does not guarantee that an inherited plan was derived from current inputs.

2. The evidence ablation is particularly strong. Holding the validator fixed while supplying inherited versions changes handoff-conflict detection from 0/30 to 30/30, cleanly isolating the missing-evidence problem. The paper is also unusually transparent about negative and regime-dependent results.

Critical weaknesses

1. The strongest baseline is missing.

Table C.2 shows that refreshing the executor's inputs before it generates yields the correct primary action in 28/30 tasks. The workflow still fails (0/30) because the downstream auditor reads stale state. The authors note in Appendix C.5 that this control does not test refreshing every role before generation. That refresh-all baseline is the natural competitor and should be run, with both task success and coordination cost reported. I recognize that two of the 30 executors kept the stale choice despite fresh inputs, so refresh alone depends on the model while PLANFENCE's check is deterministic. The open question is how much the deterministic check adds over refresh-all in practice. Without this comparison, it is unclear whether PLANFENCE provides a distinct correctness benefit or mainly a more selective way to enforce current inputs.

2. The contribution relative to supplied read sets is narrower than the framing suggests.

In Table 1(a), the S-Bus validator given inherited versions detects all 30 handoff conflicts and completes 30/30; PLANFENCE reaches 29/30. The paper itself says the two are equally safe in this setting and differ in what must cross the handoff and where validation runs. In both designs the application must record the inputs: S-Bus needs a declared read set, and PLANFENCE needs parent links and a declared scope D(a). Appendix C.4 shows that model-generated plans did not carry such parent links on their own. Proposition 1 follows from the immutability of records. The contribution is therefore artifact-carried version validation without a shared registry, which is useful but should be stated at that scope.

Questions for authors

Q1 (experiment, limited scope): Please add the refresh-all-roles baseline to the live study and report task success, invalid primary actions, and coordination cost next to PLANFENCE and centralized lineage.

Q2 (clarification): Compared with S-Bus using declared read sets, what must the application supply in each design, and what does PLANFENCE enable beyond removing the shared registry?

Suggestion (does not affect my rating): The paper could state explicitly that owner heads are taken as authoritative and that their currency relative to the external world is out of scope. Related arXiv work on evidence freshness at commitment time, e.g., TRW (arXiv:2607.21910), may be a useful pointer.

2 comments

We thank the reviewer for the careful reading, for finding the observation/derivation distinction practically important, and for recognizing the evidence ablation and our reporting of negative results.

weakness 1 / question 1

The strongest baseline is missing. [...] That refresh-all baseline is the natural competitor and should be run, with both task success and coordination cost reported. [...] The open question is how much the deterministic check adds over refresh-all in practice.

We ran the refresh-all-roles baseline. Before each role generates, it fetches the owner heads of the role's declared inputs and installs any changed record. It makes no derivation check. Its tool schema allows both the inherited action and the fresh alternatives, as in Table C.2. We compare it with PlanFence, centralized lineage, owner-head freshness and the executor-only refresh of Table C.2, using the same workflows, model, decoding and seeds. Each arm runs 30 tasks per condition, after a smoke stage on a separate seed. We test two revision timings. In the first, the revision lands between planning and execution, as in the paper; we run it over loopback and over the emulated AT&T link. In the second, it lands while the executor is generating, after any refresh before generation and before the action.

MethodBefore execution, loopbackBefore execution, AT&TDuring generation, loopback
PlanFence29 / 029 / 027 / 0
Centralized lineage29 / 028 / 029 / 30†
Refresh-all-roles28 / 430 / 226 / 27
Executor-only refresh0 / 300 / 290 / 30
Owner-head freshness0 / 300 / 290 / 28

Each cell gives successful tasks / stale actions executed, out of 30 tasks. Success is graded after the workflow's repair step; a stale action counts even when it is repaired later. In 17 of the 450 tasks (0 to 4 per cell, under every method), a call between agent processes timed out, and these tasks count as failures.

†Our live centralized-lineage executor compares the plan with the requirement version it read before generation, so it misses an update during generation. A registry check at the action would reject these actions, as in replay. This gap lies in our harness, not in centralized lineage.

Cost per task (loopback, before execution)Messages over the linkKiBCoordination timeModel callsEnd-to-end
PlanFence1318.20.61 s9.06.9 s
Centralized lineage1833.30.25 s9.16.4 s
Refresh-all-roles1721.10.65 s8.06.6 s
Executor-only refresh1317.40.65 s8.06.5 s
Owner-head freshness1013.40.59 s8.05.8 s

Medians over the tasks that finished; model calls are means. Messages and KiB count what crosses the link: owner checks, record fetches and replication for the replicated designs, and every read and write for centralized lineage.

The deterministic check adds what refresh-all-roles cannot guarantee. When the revision lands before execution, refresh-all-roles depends on the model adapting to the refreshed input. It did so in 53 of the 59 tasks that finished, but in the other 6 the executor acted against the revised requirement it had been given. When the revision lands during generation, no refresh before generation can see it, and refresh-all-roles executed the stale action in all 27 tasks that reached the action. PlanFence executed no stale action in either timing, and it replanned in all 27 tasks that finished during generation. Final task success is close (58 of 60 each before execution, 26 against 27 during generation) because the workflow's repair step fixes stale actions afterwards. The difference is whether a stale action reaches the outside world. Refresh-all-roles executed 33 across the three conditions, and PlanFence none. A shipment or a deployment is not undone by a later repair, so this is a distinct correctness benefit, not a more selective refresh.

PlanFence is also lighter on the link. Refresh-all-roles queries the owners before every role call (7 head checks and 2 record fetches per task). PlanFence makes 2 head checks, one at the action and one after the replan, and 3 record fetches. Per task, it sends 13 messages and 18.2 KiB against 17 messages and 21.1 KiB, with a tie on coordination time (0.61 s against 0.65 s). Despite one extra model call, the replan that produces the correct plan, its median task time ties refresh-all-roles by the paper's 10% rule (6.9 s against 6.6 s).

weakness 2 / question 2

The contribution relative to supplied read sets is narrower than the framing suggests. [...] Compared with S-Bus using declared read sets, what must the application supply in each design, and what does PLANFENCE enable beyond removing the shared registry?

Beyond removing the registry, PlanFence lets any later holder validate a plan derived through several hops. The plan writer's read set alone does not. We measured this in a new experiment on the real systems, with plans derived in one to three hops on one or two branches (30 templates per cell). When S-Bus receives the read set of the plan's writer, it catches every stale plan that reads the requirement directly, and 0 of 30 once one intermediate record lies between them. Forwarding the full read set restores detection. However, every intermediate writer must then forward it, and the stored evidence grows from 43 to 199 bytes with hops and branches. PlanFence catches every stale plan in these cells by following parent links, and it blocks all 60 plans that mix two versions of one input. In a second experiment, we read the scope from the plan's recorded parents, with no separate declaration, and none of 240 actions was stale. Both experiments are detailed in our response to Reviewer tJ8m.

The two designs ask the application for the following.

S-Bus with a declared read setPlanFence
At each writenothingthe IDs of the records the writer was given (known locally)
At each handoffthe creator's read set, and for a multi-hop plan the union of all upstream read setsthe plan record
At the protected actionthe read setthe scope D(a)D(a), declared or read from the plan's parents
To follow the evidencenothing beyond the read setthe records behind the plan, fetched if not held (3.1 to 13.2 KB per handoff in our experiment)
Validation targetone logical registrythe owners of the declared keys, queried concurrently

Neither design relies on the model for this evidence. The examined model-generated plans did not attach the memory record as a parent on their own (Appendix C.4), so our adapter records the exact record supplied to each role call. The designs differ after the write. A PlanFence writer stores only IDs it already knows, while a read set must be assembled and carried across every handoff. We will state the contribution at the reviewer's scope, together with the multi-hop result that shows what it adds (Section 2 and the first contribution), and present Proposition 1 as a lemma.

Our contribution is artifact-carried version validation. Each derived record keeps exact parent links. Any later
holder of a plan can follow them back to the plan's recorded inputs and check those versions against their owners,
without the creator's session or a shared registry. A session-based validator matches this safety only when the full
read set behind the plan reaches it. For a multi-hop plan, every intermediate writer must then forward its read set
(Appendix G.2).

suggestion

State explicitly that owner heads are taken as authoritative and that their currency relative to the external world is out of scope. [...]

Thank you. We will add to Section 3 and Appendix E.3: "Owner heads are authoritative by assumption. Whether an owner's head reflects the external world is out of scope." We will also cite TRW (Chang, arXiv:2607.21910) as related work on keeping evidence current at commitment time.

Please let us know if this response sufficiently addresses your concerns or if further clarification would be helpful.

Reviewer_tJ8mOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces derivation currency: an inherited plan can be stale even when the executor's local memory is current. PlanFence records plan-input provenance and checks those versions against current owner states before execution.

Critical strengths

The distinction between observation currency and derivation currency is clearly presented, and the system evaluation is carefully organized.

Critical weaknesses

The main technical idea appears incremental. PlanFence is essentially provenance tracking plus version validation at execution time, and the S-Bus control detects the same handoff conflicts once inherited input versions are supplied. This makes the core contribution feel more like a systems design choice than a substantial new agent-memory mechanism.

More importantly, the guarantee depends on exact parent links and a correctly declared dependency scope supplied by the application. If a relevant dependency is missing, PlanFence cannot detect the stale plan. This assumption is difficult to justify for general LLM-agent workflows and substantially limits the claimed applicability.

The live evaluation is also narrow: it uses one model family and mostly a single inserted revision, while harder cases with ongoing updates are mainly studied through replay or synthetic delays.

Questions for authors
  1. What capability does PlanFence provide beyond carrying an explicit read set across handoffs and using an existing validator?

  2. How can the method remain reliable when dependencies are incomplete or not explicitly known?

3 comments

We thank the reviewer for the time, effort, and valuable feedback, and for finding the distinction between observation and derivation currency clearly presented.

weakness 1 / question 1

The main technical idea appears incremental. PlanFence is essentially provenance tracking plus version validation at execution time, and the S-Bus control detects the same handoff conflicts once inherited input versions are supplied. [...] What capability does PlanFence provide beyond carrying an explicit read set across handoffs and using an existing validator?

A read set carried by the plan's writer is not enough, and this is the gap PlanFence closes. In multi-agent work, a plan often reads records that other agents derived from the changed input, not the input itself. To test the reviewer's alternative on this case, we ran a new experiment. We derive a plan from the requirement in one to three hops, on one or two branches, and a different agent writes each record. After the requirement changes, the executor holds the new requirement and validates the inherited plan. We run the real PlanFence replicas and the native S-Bus service, with 30 templates per cell.

Evidence the executor validates1 hop2 hops3 hops2 branches, 2-3 hops2 branches, mixed versionsEvidence stored in the plan
S-Bus, session observations0/300/300/300/600/60none
S-Bus, read set of the plan's writer30/300/300/300/600/6040-79 B
S-Bus, read set forwarded through every hop30/3030/3030/3060/6060/6043-199 B
PlanFence, parent links30/3030/3030/3060/6060/60, blocked68 B per branch

Stale plans caught before the action. On the 600 unchanged plans, no design raised a false alarm.

With the writer's read set, S-Bus misses every stale plan once one intermediate record lies between the plan and the requirement. The read set names that record. That record is still current, so the change goes unseen in all 30 cases. Forwarding the full read set restores detection. However, every intermediate writer must then merge the read sets of its inputs and pass them on, and the carried set grows with depth, from 43 to 199 bytes here. With PlanFence, each writer records only the IDs of the records it was given, 68 bytes per branch at any depth. It knows these IDs at write time. Any later holder follows these links and catches every stale plan in these cells. It fetches the records behind the plan that it does not hold, from 3.1 KB per handoff for a direct plan to 13.2 KB at three hops on two branches. When two branches reach different versions of the requirement, PlanFence blocks the action, as Algorithm 1 specifies.

Table 1(a) holds the validator fixed on purpose. It isolates what a later executor lacks, the evidence of which input versions the plan used. PlanFence keeps and recovers this evidence across agents without a shared service, and its guarantee does not depend on depth (Proposition 1, measured here to three hops).

Where validation runs also changes the costs. Because PlanFence checks the plan at the action instead of announcing every update, its stall at ρ=4\rho=4 is 43% below metadata sync. Because it asks the owners directly instead of a shared service, its stall is also 55% below centralized lineage (Table 2). With one declared dependency per action, it uses a tenth of all-key validation's traffic at 128 keys (Fig. 3).

We will state the contribution at this scope and add the experiment as Appendix G.2.

Our contribution is artifact-carried version validation. Each derived record keeps exact parent links. Any later
holder of a plan can follow them back to the plan's recorded inputs and check those versions against their owners,
without the creator's session or a shared registry. A session-based validator matches this safety only when the full
read set behind the plan reaches it. For a multi-hop plan, every intermediate writer must then forward its read set
(Appendix G.2).

weakness 2 / question 2

More importantly, the guarantee depends on exact parent links and a correctly declared dependency scope supplied by the application. If a relevant dependency is missing, PlanFence cannot detect the stale plan. [...] How can the method remain reliable when dependencies are incomplete or not explicitly known?

PlanFence does not need a hand-declared scope. The scope can be read from the plan record's own parents. Our adapter records these parents at each role call, without asking the model. Because they name every record supplied to the role call, the scope covers every supplied input by construction. In a new experiment, this admitted no stale action in 240. The same experiment measures what an incomplete declared scope costs.

Eight shared keys are owned by five agents. A plan's recorded parents are its two or four true inputs plus two context records that the planner was given but does not use. One true input changes after planning, and the executor holds the new value. We obtain the dependency scope D(a)D(a) in three ways and run 120 actions per cell (30 templates and four seeds for the changed input) on the real replicas.

Dependency scopeStale actions issued, 2 inputsStale actions issued, 4 inputsValidation traffic per action, 2 / 4 inputs
Declared exactly0/1200/1205.0 / 6.7 KiB
Declared, each input left out with probability 0.2526/12033/1204.0 / 5.6 KiB
Declared, each input left out with probability 0.563/12064/1203.6 / 4.9 KiB
Read from the plan record's parents0/1200/1206.7 / 8.4 KiB

When an input is left out of a declared scope, a stale action follows whenever that input is the one that changes. The failures therefore follow the omitted share, close to the expected 30 and 60 of 120. This holds for any read-set validator, since a version check covers only the inputs that were recorded (Assumption 1). When we read the scope from the plan's own parents, we need no separate declaration, and none of the 240 actions is stale. The price is 26% to 35% more validation traffic than the exact scope, because the context records are checked too.

The parents themselves do not come from the model. In our adapter, a stored role decision names the exact record that was supplied to that role call, and the plan record carries the same link (Appendix C). Neither the live nor the replay path asks the model what it used. When the scope is read from these parents, an input escapes the check only if it reached the plan outside the supplied records, for example from the model's own knowledge. No version-based check can see such an input. Two further properties keep failures safe. First, missing ancestry, an unknown owner or an ambiguous version blocks the action (Algorithm 1). Second, declaring more keys than needed never lets a stale action through. It can add traffic, replans or blocks. At full scope, PlanFence and batched all-key validation tie within 6% in stall and 8% in traffic (Table D.6).

We will add the following paragraph to Section 4, and the experiment to Appendix E:

Dependency scope in practice. Each derived record names the exact records supplied to the role call that wrote it,
so the scope D(a) can be read from the plan's parents without a separate declaration. A narrower declared scope
reduces traffic. In our omission experiment, a left-out input admitted a stale action whenever that input changed
(Appendix E.4).

weakness 3

The live evaluation is also narrow: it uses one model family and mostly a single inserted revision, while harder cases with ongoing updates are mainly studied through replay or synthetic delays.

PlanFence's check compares record identities, so it does not depend on the model. The model decides only whether the executor adapts to a changed input and whether one replan repairs the plan. To test the harder timing the reviewer describes, we added a live condition in which the revision lands while the executor is generating, after any refresh before generation. No check made before generation can see this update.

MethodBefore execution, loopbackBefore execution, AT&TDuring generation, loopback
PlanFence29 / 029 / 027 / 0
Centralized lineage29 / 028 / 029 / 30†
Refresh-all-roles28 / 430 / 226 / 27
Executor-only refresh0 / 300 / 290 / 30
Owner-head freshness0 / 300 / 290 / 28

Each cell gives successful tasks / stale actions executed, out of 30 tasks (three workflow families, ten seeds). Success is graded after the workflow's repair step; a stale action counts even when it is repaired later. In 17 of the 450 tasks (0 to 4 per cell, under every method), a call between agent processes timed out, and these tasks count as failures.

When the revision lands during generation, every freshness-only design executed the stale action each time it acted, refresh-all-roles included (27 of 27). PlanFence checks the plan's recorded inputs at the action. It executed no stale action and replanned in all 27 tasks that finished. The submitted Appendix F also has a live case in which the update lands during the replacement model call, and PlanFence blocks all nine such attempts. Ongoing updates at controlled rates run in replay, with the real replica processes and recorded model decisions, so that cost differences come from the policy and not from generation variance. A second model family did not fit in the rebuttal window, and we will add it to the revision.

Please let us know if this response sufficiently addresses your concerns or if further clarification would be helpful.

Reviewer_nKd1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper addresses a problem in multi-agent systems with shared memory. An agent writes a plan using some input data (requirements, decisions, etc.) and then a different agent later uses the plan and the input data to execute it. If the input data changes after planning, the executor’s local data can be updated (fresh memory) to the new version, but the plan was made using an older version of the input, thus making the plan stale. The executor can then take the wrong action even though the local memory is updated.

The paper introduces an algorithm called PlanFence, a validator that checks a plan before a protected or risky action. Different data artifacts are assigned to different owners, who hold the current version. The plan record will keep a copy of the IDs that reference the data used to create the plan, and the individual executor will have a local copy of the IDs of the data used as input to execute the action. The data is referred to using keys, and they are identified by version IDs, as each data record is immutable. This is important for risky actions, as the local data as well as the plan both must be fresh and match the owner’s version IDs before an action is executed. The PlanFence algorithm assumes that the owner of a key is honest and that the responses are always authenticated when returning the latest data ID. Before executing a plan, the local agent’s copy of the data’s key ID must match both the plan’s version of the key’s ID and the owner’s version of the ID. Once this is ensured for all keys that the action is dependent on, the action is allowed. If this does not match, the local data is updated with the owner’s ID first (if the local is behind the owner) and allows one replan and then checks the same condition on the new plan. If that also fails, then the action is blocked. PlanFence always does this at the time of taking the protected action, and it assumes that all the dependencies of the action are given and recorded successfully.

There are 4 validation experiments to show the effectiveness of PlanFence:

  • A comparison for validity with an existing validator called S-BUS in which 30 recorded agent decisions of an agent workflow are replayed. When S-Bus is given session observations without the plan’s inherited versions of the input data, then it misses that the plan is stale but is able to capture all 30 staleness conditions when the plan’s input versions of the data are given along with the session observations. PlanFence, in comparison, is able to capture 29 of those staleness conditions.
  • A comparison for validity with a validator named centralized lineage in a live 5-agent workflow where one revision is inserted after planning. It is found that when only the data’s freshness is checked, the executor acts on stale plans in all 30 tasks. Centralized lineage can detect and replan for all 30 tasks, just like PlanFence.
  • A comparison for stall (coordination time without the model inference time) with a validator called metadatasync, where every data change is announced for all agents using recorded emulated LTE traces (AT&T, Verizon, and T-Mobile) with 0.25 to 16 updates per action. PlanFence, in comparison, makes one round of queries for every protected action, and hence the stall does not increase with a growing number of updates, while metadatasync does. Metadatasync is cheaper when the rate of updates of data per action is 1, while PlanFence is cheaper for 4 updates per action.
  • A comparison for traffic size with a validator that queries the current version of all shared keys per action, called batched all-key validation. On testing with 4 updates per action and 1 dependency, PlanFence's traffic stays at 8 KiB as shared (unrelated) keys grow from 8 to 128, while batched all-key validation traffic grows from 16 KiB to 81.7 KiB. In a separate test where the action declares all 8 keys as its dependencies, PlanFence and batched all-key validation use roughly the same amount of traffic.
Critical strengths

S1. The paper explains the problem statement clearly and, through multiple experiments, showcases where the problem can be solved by PlanFence and even includes where other methods can be used instead, with trade-offs. The claims are mostly consistent with the results.

S2. The experiments show various ablations other than the stale plan problem. They show that PlanFence can help with the Stale Plan problem without increasing stall time when the rate of change of input data is greater than some value (4 or more updates per action). They also show that when the number of keys (unrelated to the action) grows in the system, then querying only a declared set of dependencies keeps the data size transmitted small rather than querying every key.

Critical weaknesses

W1. The biggest weakness of the paper is that the motivation for this problem statement is not explained at all with any real workflow. The shipping example ( r3, r4, p3) is used as an analogy rather than any real use case. Even the live workflow that was mentioned in the experiments (reservation, fulfilment, deployment) was not explained at all. The appendix gives prompts and the tool format but not what the tasks actually are. If inputs for the plan should change, it means there is a major redesign of the plan, and the entire system should stop and start from scratch during replanning. The paper would benefit from clearer real-life use cases as well as complete workflow descriptions and diagrams in the appendix.

W2. The experimental design section 6.1 is not clear at all. What exactly are the three workflow families, and why do they need three roles that need 5 Qwen3.5 35B A3B agents? It is not explained anywhere in the appendix either. The paper mentions that it creates simulations with AT&T, Verizon, and T-Mobile traces that add message delay, but the tasks themselves are not mobile network tasks, so it is unclear why LTE is the right setting. And why do they even need LLMs when most experimental results use replay with recorded decisions? What do these recorded results and update schedules look like, and why do they need replanning? Please provide a clear explanation of the data in each experiment and a detailed explanation of the environment and harness in the appendices.

W3. The savings in stall time are around 10 to a few hundred ms. That is very small in comparison to LLM inferences themselves, which the paper’s own role call states at about 0.6s. The cost experiments are replayed, so the end-to-end costs with LLM inferences are not shown. In the live experiment, PlanFence is not shown to improve the system (5.7 s vs 5.4 s for centralized lineage in Appendix C.4). The largest gain comes at 16 updates per action, which I find very unrealistic for a real live agent system. The traffic difference between batched all-key validation (16 to 84 KiB/action across 8 to 128 keys) is small unless there are thousands of actions, which again is very impractical for an LLM system. Please provide a stronger motivation and explanation of why the time and space savings matter and when PlanFence can save major time and space.

W4. The paper’s writing is also very unclear and uses confusing concepts. It defines Observation and Derivation currency, but the experiments never use them. Also, it mentions agents a1, a2, …, but never uses them. Instead, we are made to understand ‘a’ as the actions that the functions D_a(x), C_a(x), F_a(x), etc., use. Please include a much clearer explanation of the terms and use simpler language.

Questions for authors

Q1. Can you provide a clear, real, deployed use case where a shared input changes after planning is done? Why is one replan correct instead of replanning when any input to the plan is changed?

Q2. In the live workflow, what exactly are the descriptions of those workflows, and why do they need reservation, fulfillment, and deployment? Also, why do they need 5 LLM inferences? And why do they need replanning after an input is changed?

Q3. The cost results exclude model inference, and in the live study, PlanFence’s task time is 5.7s against 5.4s for centralized lineage. Can you provide an end-to-end time breakdown with the model in the loop, and can you identify where the stall savings and traffic savings are high in comparison to the total cost?

Q4. What are the sources of the AT&T, Verizon, and T-Mobile traces, and what do you mean by exchange delays ? Are agent processes run on separate machines or one machine with emulated delays ?

4 comments

We thank the reviewer for the time, effort, and the detailed summary, which is accurate. Below we answer each point, with new experiments and the revisions we will make.

weakness 1 / question 1

The biggest weakness of the paper is that the motivation for this problem statement is not explained at all with any real workflow. [...] Can you provide a clear, real, deployed use case where a shared input changes after planning is done? Why is one replan correct instead of replanning when any input to the plan is changed?

Use cases. The pattern arises whenever one agent plans, another agent later acts, and an input changes in between. Our three live workflow families are three such cases.

  1. Order fulfillment. The plan picks the lowest-cost warehouse with stock for a three-unit order. The customer then reduces the quantity or cancels. The stale plan ships the wrong quantity, or ships a cancelled order.
  2. Release deployment. The plan selects the build that matches the version and configuration and passes its tests. A hotfix then supersedes the requirement. The stale plan deploys the superseded build.
  3. Travel reservation. The plan books the lowest eligible fare for a destination and a fare cap. The traveler then changes both. The stale plan books the wrong trip.

In each case, the protected action has an external effect, so acting on the stale plan is the costly error. We observe this failure with Qwen3.5 in these workflows. In 30 live runs without validation, 15 executors held the revised requirement and still acted on the stale plan (Appendix C.4). Current memory alone does not prevent it. We will move these cases into Sections 1 and 6.1, so that the shipping example is not the only illustration.

Why not stop and restart on a change. A changed input does not say which plans used it. Finding them would need a descendant index or an invalidation fan-out across agents. PlanFence needs neither. Each plan is checked at its own protected action, so a change stops only the actions whose recorded inputs it touched, and only those replan. Restarting the whole team on every change would also discard work that the change does not affect.

Why one replan. Correctness does not depend on the replan budget. When a declared input has changed, PlanFence asks the workflow for a new plan, and the new plan must pass the same check before it can act. The budget therefore limits only how many new plans one action may try before it blocks and hands control back to the caller. It does not limit which changes are detected. One replan is the paper's default because it is the fewest that repairs a single revision. Our live workflows insert one revision per task. We agree that the paper should present it as a parameter. To measure what the budget trades, we swept it from 0 to 3 in the Appendix F setting, where relevant updates keep arriving while the new plan is generated. Each cell has 30 templates and two replan delays (1 s and 2 s), so 60 actions.

Replan budgetCompleted, 0.25 updates/sCompleted, 1 update/sCompleted, 2 updates/sMedian time to act or block, 2 updates/s
0 (check and block)0/600/600/600.05 s
1 (paper)39/6013/604/601.6 s
253/6024/609/603.2 s
357/6030/6012/604.8 s

More replans complete more actions when inputs change slowly. They help little when inputs change fast, because each new plan is then often stale again before it can act, and the wait before blocking triples. Centralized lineage shows the same pattern. At every budget, no issued action failed the check, and every issued action's inputs were still current when it took effect. In the revision, Algorithm 1 will take the replan budget as a parameter, Section 6 will state that the live workflows use one, and Appendix F will report this sweep.

weakness 2 / question 2 / question 4

The experimental design section 6.1 is not clear at all. What exactly are the three workflow families, and why do they need three roles that need 5 Qwen3.5 35B A3B agents? [...] why LTE [...] why do they even need LLMs when most experimental results use replay [...] What are the sources of the AT&T, Verizon, and T-Mobile traces [...] Are agent processes run on separate machines or one machine with emulated delays?

We agree that Section 6.1 and Appendix C describe the harness too briefly. We will add an appendix, "Workflows and harness", with a diagram, one complete example per family and an example update schedule. In summary:

  • Roles and agents. Each workflow has eight role units on five agent processes: a change owner, two evidence scouts, a constraint reviewer, a policy reviewer, a plan integrator, an action executor and an auditor. Each role unit makes one model call, plus one more when the executor must replan (Table C.1). The units run on five processes because the failure we study happens when a plan crosses from one agent's memory to another's. Replay varies the team from three to eight agents (Appendix D).
  • Why replanning. The correct action depends on the requirement. After a revision, the grader marks the old plan's action as wrong, so the workflow needs a new plan for the revised requirement.
  • Why LLMs, and why replay. The live studies use real model calls to show that generated plans go stale (Appendix C.4) and that PlanFence repairs them with a real new plan (Table 1(b)). Replay answers a different question: what each policy costs. It fixes the decisions and the update schedule, so that stall and traffic differences come from the policy and not from generation variance. In replay, the replica, storage and protocol code are real. The replicas run as separate processes and exchange the serialized protocol over authenticated TCP, and only the model call is replaced by a recorded decision.
  • Why LTE. Our setting is agents spread over edge devices and servers (Polese et al., 2025), where every coordination round pays a network round trip. The tasks are not network tasks, but the network sets the price of coordination. We also report loopback, for agents on one machine (Section 6.3).
  • Traces (Q4). We use the public Mahimahi LTE driving traces of AT&T, T-Mobile and Verizon (ATT-LTE-driving-2016, TMobile-LTE-driving and Verizon-LTE-driving, from Netravali et al., USENIX ATC 2015), pinned by SHA-256. "Per-exchange delay" means that every TCP request and response passes through a user-space link emulator. The emulator releases bytes at the packet-delivery times recorded in the trace, after a 20 ms base delay. In replay, the agents of one episode run as separate OS processes on one CPU node, and each pair of agents talks through its own emulator process. Episodes are spread over three nodes. The submitted live studies ran on one GPU node without link emulation. The new live runs in our next comment add the emulated AT&T link.

weakness 3 / question 3

The savings in stall time are around 10 to a few hundred ms. That is very small in comparison to LLM inferences [...] PlanFence is not shown to improve the system (5.7 s vs 5.4 s for centralized lineage) [...] 16 updates per action [...] very unrealistic [...] Can you provide an end-to-end time breakdown with the model in the loop?

With the model in the loop, PlanFence's check is a small cost, and it buys what no cheaper design without a shared registry delivered. We ran the live workflows over the emulated AT&T link and split each task's time into its parts.

Method (AT&T link)Task success / stale actionsModel timeCoordination (of which replication)Data over the linkEnd-to-end
PlanFence29 / 05.70 s1.88 s (0.62 s)18.2 KiB8.97 s
Centralized lineage28 / 05.55 s1.37 s (none)33.3 KiB7.86 s
Refresh-all-roles30 / 25.03 s1.91 s (0.63 s)21.1 KiB7.68 s
Executor-only refresh0 / 295.13 s1.38 s (0.58 s)16.7 KiB7.11 s
Owner-head freshness0 / 294.93 s1.05 s (0.57 s)13.4 KiB6.45 s

Revision before execution, 30 tasks per method. Times are medians per task over the tasks that finished. Coordination is the time the roles wait on their memory layer, 17% to 26% of the task. Replication copies each written record to the other agents. Every design without a shared registry does this. PlanFence's own check at the action takes 0.33 s.

This workload has one revision per protected action, so ρ≈1\rho\approx 1. The paper claims its stall gains from ρ=4\rho=4 upward (Fig. 2), and the live study was built to test correctness and repair. Here, PlanFence's check costs 0.33 s against 5.7 s of model time, and every faster design executed stale actions or validates against a shared registry. Against centralized lineage, PlanFence spends 1.88 s on coordination against 1.37 s. Replication is larger than this gap, and without it PlanFence waits 1.21 s. Over AT&T, it also sends 45% less data, because its reads and writes stay on the agent. In the slowest tenth of tasks, single calls between local processes stalled for tens of seconds, on loopback as well (90th percentile 19 s for centralized lineage, 57 to 66 s for the other designs). We are tracing these stalls, report medians, and keep every task in the outcome counts.

The gains appear from ρ=4\rho=4 and grow with the link delay and the size of the shared memory. Here ρ\rho counts owner changes across the whole team per protected action, not changes to one input. At ρ=4\rho=4 over AT&T, PlanFence's stall is 43% below metadata sync and 55% below centralized lineage (Table 2). At ρ=16\rho=16, per-update synchronization stalls 1.2 s per action, about two model calls, while PlanFence stays near 0.23 s (Table D.2). In our keyspace study, all-key traffic grows by about 0.54 KiB per action for each added key, while PlanFence's traffic stays flat (Fig. 3). These costs recur on every protected action.

weakness 4

It defines Observation and Derivation currency, but the experiments never use them. Also, it mentions agents a1, a2, …, but never uses them [...]

Both notions are what the experiments measure, and we will make this explicit. An action is invalid exactly when it fails derivation currency (Eq. 1). The freshness-only controls provide observation currency only, and in our new live runs every one of them executed stale actions while PlanFence executed none. We will label each row of Tables 1 and 2 with the property its policy enforces. We will also rename the agents g1,…,gNg_1,\dots,g_N, keep aa for actions, add a notation table, and explain each formal term in plain words at first use.

Please let us know if this response sufficiently addresses your concerns or if further clarification would be helpful.

Reviewer_nKd1OpenReview ↗

Response to the authors

Thank you for the detailed response and the substantial additional work during the discussion period. I appreciate your efforts! The following is my point-by-point response.

Q1. Regarding the three explanations given as motivations for the system:

A1.

  • Order fulfillment: If a user changes plans and cancels or changes quantity, any sufficiently strong e-commerce provider will rightly cancel the plan. No e-commerce provider will allow the option for the user to change requirements.
  • Release deployment: A hot fix that changes the requirement made for a deployment will be immediately stopped. It is much simpler than the complicated PlanFence method.
  • Travel Reservation: Same as the e-commerce order. The reservation should be cancelled with any change by the customer.

I asked for the examples, hoping to find sufficient reasoning for your own experiments. What kind of system needs such an elaborate tracking of plans and their inputs to be fixed in sub-second speeds when LLM systems themselves are very slow generative systems?

Q2. Why not stop and restart at every change?

A2. In my opinion, keeping an index is very useful. It helps both coding agents and us humans to figure out the whole system. Why not go for such a design? And that is what I asked above. What kind of real-world system needs tracking of exact plans and their input version IDs? What kind of complexity is needed?

Q3. Why one replan?

A3. Accepted. But the way 2 updates per second makes the number of completed actions fall drastically really raises the question of the usefulness of planfence.

Q4. Roles of the different agents.

A4. Thank you for actually providing this input description, but without the diagram it is hard to judge what exactly the workflow is trying to do. Also, what exactly are the traces used for? You have cited the papers, but they need descriptions of what exactly they are replaying. That will also provide us with an actual use case and motivation for this paper’s algorithm and why fixing stale plans matters.

Q5. End-to-end co-ordination time

A5. Thank you for providing the end-to-end co-ordination time. And this is exactly what my concern highlights. That the savings in time are small compared to LLM inference latency itself. So then why must saving time matter in replanning from scratch? Also, 0.54 kB per action sounds really small. Why would it matter that memory stays flat with PlanFence? I don’t see any real use case for an LLM agent system with millions of actions.

Q6. Observation Currency and Derivation Currency.

A6. Thank you for rewriting the paper and making these terms explicit.

Overall, I feel that though some of my concerns are clarified, the majority of my concerns are not sufficiently addressed, and I feel this paper needs to be substantially rewritten. Hence, I will keep the same score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of new experiments and revisions

We thank all reviewers for their time and their detailed feedback. The reviews asked what PlanFence adds beyond carrying a read set, how it behaves when dependencies are incomplete, and how much it matters with the model in the loop. We ran five new experiments on the real replica processes, with live model calls in the live study, and every scheduled attempt stays in the denominator.

Across the new experiments, every design that only refreshes memory executed stale actions, including the refresh-all baseline that Reviewer Sy5K proposed. PlanFence executed none in the live runs and caught every stale plan in the handoff experiment. The designs that match its safety pay for it elsewhere. Centralized lineage and S-Bus validate against a shared registry, and a forwarded read set must pass through every intermediate writer. Metadata sync announces every update (43% more stall than PlanFence at ρ=4\rho=4, Table 2), and all-key validation queries every shared key (ten times PlanFence's traffic at 128 keys).

ExperimentQuestionResult
Multi-hop handoffs (tJ8m, Sy5K)Is a read set carried across handoffs enough?No. S-Bus with the plan writer's read set catches 0 of 30 stale plans once one intermediate record lies between the plan and its input. It catches them only if every intermediate writer merges and forwards its read set. PlanFence catches all of them from parent links that each writer records locally.
Dependency scope (tJ8m)What if a dependency is missing or not declared?No declaration is needed. Reading the scope from the plan's recorded parents admits none of 240 stale actions, for 26% to 35% more validation traffic. A hand-declared scope that leaves inputs out admits stale actions in proportion (26 to 64 of 120), as for any read-set validator.
Replan budget (nKd1)Why one replan?Safety does not depend on the budget. Every new plan must pass the same check, and no issued action failed it at any budget from 0 to 3. The budget trades completed actions (39 to 57 of 60 when inputs change slowly) against waiting time, and the revision makes it a parameter of Algorithm 1.
Refresh-all-roles (Sy5K, tJ8m)Does refreshing every role before generation suffice?No. It executed 6 stale actions when the revision landed before execution, and 27 of 27 when it landed during generation. PlanFence executed none, with fewer messages and less data per task (13 messages and 18.2 KiB, against 17 messages and 21.1 KiB on loopback).
Live runs over the AT&T link (nKd1)What does validation cost with the model in the loop?PlanFence's check takes 0.33 s per task at the median, against 5.7 s of model calls. On loopback, its task time ties centralized lineage and refresh-all-roles by the paper's 10% rule. Over AT&T, every faster design executed stale actions or validates against a shared registry.

The main revisions are:

  1. The contribution stated as artifact-carried version validation, with the multi-hop experiment that shows what a carried read set misses (tJ8m, Sy5K).
  2. The refresh-all-roles baseline and a revision during generation in the live study (Sy5K, tJ8m).
  3. Dependency scope read from the plan's parents, with the omission experiment (tJ8m).
  4. A new appendix on the workflows and the harness, with use cases, roles, traces, process layout and an end-to-end time breakdown (nKd1).
  5. The replan budget made a parameter of Algorithm 1, with the sweep in Appendix F (nKd1).
  6. Plain-language terms and a notation table (nKd1).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.