- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors propose a new context management strategy (STM) that incorporates 4 structured modules, observation, reasoning, action, and outcome, together with pre-defined multi-level rules to further abstract and compress each module's information density. Experiments across inter-session and intra-session agentic benchmarks show improvements compared with other context management baselines.
Critical strengths
- The compared baselines and ablation studies are extensive.
- The considered agentic scenarios incorporate both inter-session and intra-session settings, and for the intra-session ones, incorporating difficult SWE and CLI benchmarks (SWE-Bench Pro, Terminal-Bench 2.0) is meaningful and timely.
Critical weaknesses
W1 The main weakness is the actual contribution of this paper. To me, it feels more like an incremental engineering improvement than a principled solution to a core research problem:
- The four slots directly mirror the standard ReAct workflow, which already separates observations, reasoning, and actions. As the authors also note, these slots and work-item updates are mainly parsed from raw execution logs using hand-crafted prompts.
- The rate-distortion analysis in Section 3.3 (Proposition 1) does not really support the design: once the definition of is plugged into Eq. (4), the terms simply cancel out, leaving a general mutual-information inequality that holds for any arbitrary partition of a turn rather than justifying this specific four-slot structure.
W2 The experimental comparisons could be organized more clearly, and it is hard to tell where the gains actually come from:
- In Table 1, direct inference without external memory compression (i.e., standard ReAct with full context) is missing. Only when looking at Table 2 (
Full context) do we see that all six memory baselines in Table 1 actually perform worse than directFull contextinference across all benchmarks. - Given the heavy engineering pipeline, the actual improvement of STM over strong references is relatively marginal (especially when trajectories fit within the context window, where STM is only slightly better than
Full context), and it remains unclear what truly drives the gain.
Questions for authors
N/A
3 comments
Response to Reviewer W1fV (Part 2)
(Following above Part 1)
On hand-crafted prompts. Fair, and we have tried to bound how much rests on them. Appendix B.7 inventories every hand-set quantity with its sensitivity; the retrieval weights can be learned, with learned weights at 44.6% against 44.2% hand-set; a single configuration transfers across benchmarks at a cost of at most 0.9 points (Table 34); the same configuration transfers across four backbones and to an unseen benchmark (Table 35). On extraction specifically, Appendix D.15 reports that constraining the extractor's decoding to the output schema and validating the parsed record against the harness log reduces its error rate on unusual turns from roughly 15% to 6% and raises success rate to 45.1%, which is the cheaper half of the fine-tuned-extractor direction we had only speculated about. We report the unconstrained setting in the main results so that all numbers come from one configuration.
W2 The experimental comparisons could be organized more clearly, and it is hard to tell where the gains actually come from:
- In Table 1, direct inference without external memory compression ...
- Given the heavy engineering pipeline, the actual improvement of STM ...
A2: Both criticisms are about presentation hiding a result that should be stated up front, and we have changed the presentation.
The missing reference row. Table 1 now carries a top block with two references that use no memory system: recency truncation at the shared per-step memory budget, and full context at the 400K cap. The accompanying text states the comparison plainly instead of leaving it to be discovered two tables later: every memory baseline trails the full-context reference on every benchmark, compressive memory buys a far smaller prompt and these six baselines pay accuracy for it, and among the memory systems STM alone is both cheaper and more accurate than that reference. We also added a history-tokens-per-step column to Table 2 so the cost side of every row is visible in the same place as its accuracy, which makes the trade the reader is being asked to evaluate explicit: 8.2K for recency truncation, 3.4K for retrieval over raw turns, 9.8K for the task-aware summary, 38K for the playbook, 31K to 400K for full context, 12.7K for STM.
Where the gain comes from. This was the most useful part of the review for us, and we have added the analysis rather than restating the aggregate. The new Table 7 decomposes the 5.6-point difference to full context on SWE-Bench Pro along two axes at once. The nested path from the reference to STM is: full context 38.6; plus slot-ordered prompt assembly with no change to what is stored, 39.3 (+0.7); plus slot-scoped retrieval over uncompressed slots in place of the full trace, 41.5 (+2.2); plus slot-aware compression with the action floor and the verifier, 43.1 (+1.6); plus action-chain reconstruction, 44.2 (+1.1). Reading the prompt in slot order accounts for little; retrieval over typed units is the largest single step; compression keeps the surviving records usable once the trace overflows; action-chain reconstruction supplies the rest. The same table splits every step by whether the trajectory fits in the window, which answers the second half of the criticism directly: 4.9 of the 5.6 points come from the 61% of episodes that exceed the window, and only 0.7 from the 39% that fit.
On the gain being marginal where trajectories fit. We agree, and we now say so as a claim rather than treating it as a caveat. Inside the window STM is a cost-quality improvement, roughly +1.8 points at 8% to 30% of the prompt tokens, and we no longer describe it as an accuracy gain there. Outside the window it is an accuracy improvement, +8.0 points, because the reference degrades to recency truncation. The conclusion and Appendix E both state the result as two claims rather than one average, and Appendix E's "histories that fit in context" limitation now carries the decomposition explicitly. We think this is the honest reading of our own numbers and it is also the reading that makes the scope of the contribution clear: the pipeline earns its complexity on long-horizon episodes, which is where we claim it applies.
Follow-Up on Response and Revision
Dear Reviewer W1fV,
Thank you again for pushing us to clarify STM’s contribution and the sources of its gains. We wanted to follow up on whether our revised theoretical claims, comparison across alternative partitions, and new breakdown of gains by component and episode length have addressed your concerns about the design justification and empirical contribution. We would welcome any remaining questions or concerns. If the revisions and additional evidence have improved your assessment of the paper, we would be grateful if you would consider raising your rating.
Best regards,
The Authors
Response to Reviewer W1fV (Part 1)
W1: The main weakness is the actual contribution of this paper. To me, it feels more like an incremental engineering improvement than a principled solution to a core research problem:
A1: The second bullet is a correct derivation and we accept it without reservation. Substituting the definition of
G_alloc(w)into Eq. (4) does cancelR_slot(D)and leavesR_mono(D) >= sum_s R_s(D_s^mono) - TC(X), which holds for any partition of a turn. Proposition 1 therefore does not derive the four-slot schema, and the submission was wrong to present it as though it did. We have changed what we claim and added the piece that was actually missing.What the proposition is now said to do. Appendix A states the general version explicitly, with blocks
Bin place of slots, and notes that substituting the definition makes the cancellation visible. Section 3.3 says in the main text that neither the statement nor the proof uses the four-way split and that the proposition identifies a trade-off a schema faces rather than deriving one.What we added instead. The missing content is that the two right-hand terms are partition-dependent and move together. Appendix A adds Proposition 2, a refinement identity: for
P'refiningP,TC^{P'}(X) = TC^{P}(X) + sum_{B in P} TC(X_B) >= TC^{P}(X). The proof is two lines of entropy bookkeeping. Refining a partition can therefore only raise the redundancy it pays, while the allocation gain is bounded by the spread of task weights and rate-distortion slopes across blocks and stops growing once blocks are homogeneous in those respects. The marginG_alloc^P - TC^Pconsequently has an interior optimum rather than favoring ever finer schemas, and a partition is useful to the extent that it separates content differing in task weight or compressibility without separating content that shares information.And we made the criterion empirical. Appendix D.5 and Table 24 estimate both terms for seven candidate partitions using the protocol of A1 to Reviewer ZaK7: no decomposition, a random four-way token split, action versus rest, {O, R+A, U}, {O+U, R, A}, our four slots, and an eight-way sub-slot refinement. The random split is the informative control, since it separates content without separating roles: its allocation gain is 0.19, its redundancy upper bound 1.92, its margin -1.73, and it buys 1.6 points over monolithic storage. The eight-way refinement has the largest allocation gain of all, 1.84, but pays 1.21 in redundancy exactly as Proposition 2 requires, and does not improve success rate. Our four slots have the largest margin, 0.85. Across the seven settings the estimated margin and the success rate are rank-correlated at Spearman rho = 0.85 (p = 0.016). This is the sense in which the analysis now bears on the design: it does not derive the four slots, but it supplies a quantity that distinguishes a useful partition from an arbitrary one and that can be computed for a candidate schema before running the pipeline. We also carry this into Appendix E as a limitation, since it is a selection criterion evaluated over a handful of hand-written candidates and not a search over schemas.
On the slots mirroring ReAct. We take the point and have sharpened the claim rather than disputing the resemblance. The decomposition is indeed the standard agent step, and we now say so directly in Section 3.1. The relevant fact is that every method we compare runs on the same ReAct backbone, so the decomposition is available to all of them at generation time; what differs is whether it survives into storage, where a slot becomes the unit that receives its own budget and that retrieval can address. The contribution we claim is the storage-side consequence, and it is the part the experiments isolate: per-slot rate allocation with a fidelity floor, invariants that are verified rather than instructed, addressability by slot by level by work item, and a reconstructable action chain. Three results speak to whether that is more than relabelling. Transplanting slot typing into Mem0, A-Mem, and AWM without touching their consolidation or retrieval gains 3.5 to 5.5 points (Table 6), which would not follow if the backbone already supplied the benefit. The new Table 7 shows that the mechanisms that depend on storage structure, slot-scoped retrieval and action-chain reconstruction, account for 3.3 of the 5.6 points over full context, while reading the prompt in slot order accounts for 0.7. And Table 24 shows that an arbitrary partition of the same turn buys almost nothing, so it is the typing and not the splitting that matters.