Decision-Sufficient State Representations: Measuring and Reducing Write-Time Regret

Long tasks produce more history than an LLM agent can hold in its context, and more than it uses reliably even when the history fits. A growing line of work therefore has agents carry a short written state instead: at every step a writer rewrites the state, and a reader acts from the state alone. Steps stay cheap, but anything the writer drops is lost before later decisions reveal that they need it. We quantify this loss and ask whether training can reduce it. Comparing the written state with the best state of the same size written in hindsight, we split the reader's loss into a budget loss, which any state of that size must incur, and a write-time regret, which comes from the writer's choices. In TextWorld cooking games where we control how long a fact must be carried before it is needed, a 128-token state holding the facts wins nearly every game, while prompted language-model writers win at most 17%. Almost all of the loss is write-time regret, and it grows with the delay. Creating a real delay takes care: steps inserted between a fact and its use delay it only if they do not show the fact again. We then train the writer from the reader's own loss. DSSR (decision-sufficient state representations) scores candidate states by how well the reader acts after the writer carries them forward, and teaches the writer to prefer the better ones. This forward-rolled score predicts game outcomes (ρ = 0.48; 0.26 under the delayed protocol), whereas scoring a candidate as a fixed context, as hindsight methods usually do, does not (ρ ≤ 0.07). On a pre-registered test split opened once, training adds +7.0 [+1.9, +12.2] points of success when facts are needed soon, bringing a plain summary writer to the level of belief- and slot-based memory prompts. The gain shrinks as the delay grows: on held-out games it is only at the shortest delay, and our pre-registered tests for a gain under long delay fail. A hand-written rule that names what to keep, using task knowledge the writer is not given, does better at every delay. We trace this limit to credit assignment: keeping a fact now pays off only if every later rewrite keeps it too, which a per-step score cannot see. Anonymous codebase: https://anonymous.4open.science/r/AgentDSSR