- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Language agent model (LAM) is an abstract model of an agent's harness that treats the model as a block box and is designed to study the harness controls: the model calls, the contents of context and external memory (including movements between the two), and tool calls. The paper aims to provide a resource theory of LLM agent harnesses, showing that by keeping the model capability fixed, the harness costs can be bound, compared, and optimized. The authors show an equivalence of harness execution with red-blue pebbling, the type of memory access changes asymptomatic cost, and a characterization of how storage is traded for model calls.
Critical strengths
(S1) Studying the harness in agent systems is an important and timely endeavor.
(S2) The proofs seem correct and careful (although several are fairly obvious — see W2 below).
Critical weaknesses
(W1) The language agent model seems to be heavily inspired by Andrej Karpathy's LLM OS from late 2023 and MemGPT. The paper does not add new architectural insight outside of the counters and theorems over those counters. However, the LAM seems to me like it counts the wrong things: its cost units are not the same units as that which agent systems are typically bill or timed in. There is no relationship with prefix caching, input/output/reasoning tokens, or latency. This leads me to wonder what concrete design decisions the LAM actually changes compared to actual deployed systems. What is an example of a result the theoretical model gives us that could not have been obtained through measurement alone?
(W2) I am not a theoretician, but several theorem appear either true by construction or fairly obvious. I do not mean the fancy theorems such as, e.g., 3 and 6 (which seem to be restatements of classical results in the LAM context), but even the first two theorems hardly feel like they prove anything. For example, theorem 1 argues that harness == red-blue pebbling, which basically derives trivially from the definition, while theorem 2 just leaves the algorithm zero freedom, which means the bound is necessarily the expected distance between consecutive cells on a random permutation. Please let me know if I am misreading the intent, but this seems fairly expected and unsurprising. I think the paper could avoid leaving readers with this impression by explaining what surprising results would interest a practitioner or change a design decision.
(W3) How does the model account for code that is model-generated that itself issues model calls (for example, CodeACT or recursive language model harnesses)? This design is more and more common in real systems, yet it does not seem to map cleanly to the proposed model. If I understand correctly, modeling this case as a tool would mean that nested calls would loop lose access to memory, whereas modeling it as controller logic would not work due to finiteness. What happens if the control flow is generated by the LLM itself?
(W4) Several pieces of related work in the space of context window and its relationship to memory/IO pages are uncited and should have been included in the related work survey. Mainen 2026, “The Library Theorem” and Rafique 2026, “ClawVM” come to mind. A quick search also turned up this paper Meyerson et al. 2025, “MAKER” which seems highly relevant. It seems a proper survey of the related work would likely show that the claims of presenting a “new resource theory” is too strong and should be revised.
(W5) The paper is fairly hard to approach. Terminology is consistent but sentences are overly verbose and technically-loaded. Here is an example among many, appearing on page 2: "Using value provenance, we quotient block occurrences by computational origin and prove an instancewise, resource- reserving correspondence: for every L-realisable DAG, memory- nitialized vertex-granular LAM evaluations and red–blue pebblings have exactly the same feasible call–transfer budgets.” You have managed to pack 8 or 9 technical terms to say what you could have said in plain English: if a model call computes one step of the task, the harness run is a red-blue pebble game.
(Minor) The appendix is EXCESSIVELY long for such a paper. I am honestly not sure most of it adds much value.
Questions for authors
Please see my review for the details.
(Q1) What is an example of a result LAM gives us that could not have been obtained through measurement alone?
(Q2) How do you deal with code that the model generates which itself calls the model?
(Q3) Please compare LAM with missing related work, in particular Mainen and Meyerson et al.
6 comments
Response to Reviewer kn2r (part 2/6)
W2. Classical foundations and the content of the results.
The reviewer writes "I am not a theoretician". We appreciate your perspective and thoughtful questions. As researchers in the theory of LLMs and agents, we aim to make our results clear and useful to practitioners.
Our core novelty is introducing a unified resource theory for LLM agent harnesses, covering four aspects: (i) communication between context and memory, (ii) memory-access interfaces, (iii) persistence–recomputation tradeoffs, and (iv) reliability through sampling, verification, and recovery.
Our contributions go beyond these classical foundations. We use classical pebbling, time--space tradeoffs, sequential-testing tools, and checkpointing laws as starting points, but derive new harness-level results that account for model calls, active context, persistent memory, correctness feedback, and rollback. Concretely, we characterize joint call--communication costs, separate persistence from active context in recomputation, prove a tight no-feedback sampling law for multi-step execution, and derive joint verification--rollback policies that can outperform either mechanism alone.
The revised Table 1 makes these distinctions explicit. We further provide a comparison table to classical works, and we also list how we have revised the paper to better reflect our novelty.
Response to Reviewer kn2r (part 3/6)
Table A. Classical foundations, our novelties and revisions
| Classical foundation | What we inherit | What is novel in our paper | How we have revised the paper |
|---|---|---|---|
| Hong & Kung (1981) | The red–blue pebble model and classical input/output lower bounds. | We derive optimal call and transfer costs on a grid, showing that minimum model-call complexity and optimal-order memory communication can be achieved simultaneously. See Section 4. | We rewrote Section 4 to separate classical results from our contributions: (a) shortened the pebbling background, (b) simplified the correspondence theorem, (c) replaced the abstract Pareto-frontier emphasis with a concrete joint call–communication result, and (d) moved the sequential input-traffic result into the main text with a clearer explanation of its harness-level significance. |
| Lengauer & Tarjan (1982) | The classical bit-reversal time–space tradeoff. | In Section 6, we separate active context from persistent memory and show how persistence can eliminate repeated model calls without enlarging the context window. On the bit-reversal DAG, the optimal call cost is , and a matching harness achieves this bound with only memory transfers. | We rewrote Section 6 to make the distinction from the classical result explicit: (a) motivated recomputation as a harness-level problem, (b) explained the bit-reversal construction and why it creates delayed reuse, (c) separated active context from persistent memory, and (d) added a concrete interpretation showing how storing intermediate results avoids repeated model calls while still requiring retrieval into context. |
| Wald (1947); Bretagnolle & Huber (1979) | Classical sequential-testing tools for lower-bounding the samples needed to distinguish two distributions. | In the Reliability section (Section 7), we prove that without correctness feedback, reliable multi-step execution necessarily incurs a logarithmic sampling overhead. On a hidden-tree task where a wrong transition produces no later warning, even adaptive sampling requires model calls, and plurality matches this order. | We rewrote the sampling analysis to make the contribution explicit: (a) motivated the no-feedback setting as a harness-level problem, (b) explained why hidden-tree errors cannot be detected later, (c) emphasized that the lower bound allows adaptive stopping, and (d) connected the lower and upper bounds into a tight end-to-end model-call law. |
| Young (1974); Daly (2006); Aupy et al. (2013) | Classical checkpointing tradeoffs between checking overhead and repeated work, including square-root checkpoint scaling and silent-error verification. | Beyond the classical checkpointing law, we jointly optimize local verification and segment rollback. We derive the exact combined cost and show that an interior hybrid can be strictly cheaper than either local verification or checkpointing alone; our extensions also incorporate transfer cost and non-independent error hazards. | We revised Section 7's discussion to explicitly credit the classical square-root law and distinguish it from our new joint verification–rollback optimization, transfer-priced policy, and conditional-hazard extension. |
Response to Reviewer kn2r (part 1/6)
We thank you for recognizing the importance of studying harness costs and for the careful comments on our proofs, scope, and presentation. We have revised the paper to address these concerns.
W1. Architectural precedents, deployment costs, and design choices.
About architectural precedents. LAM is a theoretical framework that establishes resource bounds and achievable tradeoffs for agent-harness execution. Its contribution lies in lower bounds, matching constructions, and explicit conditions for optimal execution. Karpathy’s LLM OS discussion and MemGPT provide architectural context for managed memory and context. LAM supplies a mathematical basis for analyzing such designs: how many model calls and memory transfers are necessary, when persistence avoids recomputation, and how verification changes reliability cost. This complements work on model capability by characterizing the resources required to realize that capability through a harness. The revised Section 2 further discusses the relationship to MemGPT and related inference abstractions.
About token costs, prefix caching, and latency. We clarified both the theoretical counting convention and the treatment of caching in Section 4. The revised text states: “Prefix caching reuses computation but leaves logical input unchanged.” It then specifies the scope: “Our main theoretical results count logical traffic; the communication experiment in Section 8 additionally evaluates the effect of exact-prefix caching.” These statements distinguish the full input supplied to the model from the prefix computation that caching can reuse. LAM provides a theoretical basis for predicting and explaining harness costs across deployments. A measured bill describes one execution under one serving configuration. Our resource laws establish how costs scale with task size, context, persistence, and verification, and provide lower bounds across the execution classes covered by the theorems. Deployment-specific prices and calibrated error rates then turn these laws into cost predictions and policy choices. For example, our checkpoint-policy experiment fixes its prediction before evaluation: under the implemented segment-check interface, the selected four-step rollback policy has a forecast weighted cost of 25.2 and a measured cost of 25.2. This tests a prospective prediction, beyond reporting the cost of a completed run.
Prefix caching refines this prediction by changing the cost of repeated input processing. Our communication evaluation (see revised Section 8, “Communication”) quantifies its effect: the replay-to-dependency-only ratio changes from 7.14× in logical input to 1.72× in estimated prefill on the synthetic chain, and from 11× to 3.05× on development MATH. These results explain why a large logical-traffic reduction can produce a smaller deployment benefit. Combining the resource laws with cache behavior, serving prices, and measured reliability supports concrete harness choices: whether to replay or route state, retain or recompute results, and verify frequently or recover over longer segments. The theoretical contribution is therefore both predictive and explanatory: it identifies unavoidable costs and guides the search for better cost–reliability tradeoffs across harness designs.
About guidance for practical harness design. LAM supports a concrete selection procedure: specify the application's budget and reliability requirement, compare feasible policies, and select the lowest-cost policy that meets the requirement. On development tasks, measure model and verifier error rates, token costs, and cache behavior. Use these measurements with the resource laws to compare routing, storage, and verification/recovery policies. Fix the selected policy before held-out evaluation. Our checkpoint experiment follows this procedure: the calibration-derived rule selects a four-transition rollback interval, which is also the cheapest certified policy on the held-out chains under the implemented segment-check interface. Its measured weighted cost of 25.2 matches the frozen forecast. The execution-granularity experiment provides a second concrete design lesson: select grouping and verification together. The added Figure 3 shows that verified four-step calls achieve 96% success at 3,200 tokens per chain, compared with 92% at 3,600 tokens for unverified one-step calls. Verified eight-step calls retain 92% success at 2,700 tokens, a 25% cost reduction. Thus, a designer can reduce cost by grouping steps while using verification to recover reliability. The resource laws organize the candidate choices, and task-specific measurements determine which configuration meets the application objective.