HiDe: Hindsight Deletion of Environment-Substitutable Memory for Efficient LLM Agents

Long-horizon LLM agents repeatedly carry historical reasoning and observations into subsequent decisions, even as environmental feedback refreshes task information and makes some records replaceable. Learning when to delete this history is difficult: task rewards provide little direct feedback on individual memory choices, while token savings alone cannot distinguish useful deletion from information loss. We introduce HiDe (Hindsight Deletion), a framework that lets the agent's realized future teach its past memory decisions through dual-level hindsight self-teaching. Its dual-view counterfactual scoring evaluates the same next realized response under the reduced history and a matched context restoring only the newly deleted blocks. With the new observation shared across both views, score differences provide an approximate measure of the response's remaining dependence on removed history. HiDe combines this dependence signal with cumulative input savings into a shared hindsight utility, discounting savings when the response remains sensitive to the deleted content. At the decision level, the utility supplies dense credit to earlier DELETE/WAIT choices; at the trajectory level, it shapes task-gated rewards to favor strong task performance and useful memory reduction. Hindsight supervision requires only forward rescoring, without additional rollouts or a separate teacher model; the trained policy makes memory decisions from its current context alone. Across WebShop, BabyAI, SciWorld, TextCraft, and SearchQA with two model scales, HiDe achieves the best or tied-best task scores among compared methods in nine of ten settings. WebShop ablations support the contribution of hindsight credit to task performance and input efficiency, while independent Keep/Delete continuations and information-support analyses examine deletion consequences and replacement evidence.