- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the trade-off between retrieval accuracy and inference overhead in long-horizon LLM agent memory systems. Existing memory approaches typically rely either on lightweight retrieval or pre-compression, which risks losing fine-grained dependencies, or on expensive query-time memory reconstruction and deep analysis, which incurs substantial compute and latency overhead when initial retrieval is already sufficient. To tackle this issue, the authors introduce the Router-Mem framework. Instead of relying solely on static query complexity, this framework models early termination as an evidence-conditioned progressive execution process grounded in the actual sufficiency of the retrieved evidence.
Critical strengths
The paper presents a clear problem motivation and a well-founded paradigm, breaking down long-horizon agent memory access into an evidence-sufficiency-conditioned progressive pipeline.
Critical weaknesses
-
This paper forces a 1.7B student without reasoning tokens to mimic a teacher's hidden state derived from explicit CoT rationales creates a severe information bottleneck. The student likely learns surface statistical correlations rather than true causal sufficiency reasoning.
-
On the million-token BEAM benchmark, Router-Mem drops 4.5% compared to Full Completion. This shows a noticeable false-positive exit rate where reasoning terminates prematurely despite missing key dependencies.
Questions for authors
Please see weaknesses
2 comments
Thank you for engaging with our responses — your follow-up directly prompted the paired deletion experiment in our most recent reply, which we believe is the most direct test of exactly your concern: reliability when key dependencies are missing.
A two-line summary, in case a compact form helps:
Removing the dependency-carrying entries flips 39/47 (83%) of the router's early exits to continuation; removing the same number of unrelated entries flips none (47/47 retained; paired Δ = +0.440, p = 1e-16; AUC 0.861). The two surface signals — the retrieval-score mean (AUC 0.548) and the untrained Qwen3-1.7B yes/no confidence (0.555) — both fail this test, so the router tracks whether the required evidence is present, not how much the context changed. On the 4.49-point concern: τ = 0.9 narrows the gap to 1.17 (42.09 vs. 43.26) at an 8.1% latency reduction — τ = 0.5 is one operating point, not a universal recommendation. The window closes within hours, so if any part of this remains unconvincing, we would be glad to clarify before then. We fully respect your assessment either way and a one-line reaction would mean a lot to us.
Response by Reviewer xzPD
Thank you for the clarifications and additional ablations.
However, I remain concerned about the reliability of its sufficiency judgments when key dependencies are missing. I understand that the drop in accuracy does not directly tell us how often the router stops too early. Still, I am not convinced that the efficiency gains justify the 4.49-point drop on BEAM. Overall, the response does not sufficiently change my assessment, and I maintain my original score.