- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies "context drift", meaning divergence from still-active user commitments in long-horizon dialogue under a token budget. It proposes MACC, a two-layer MLP controller trained with PPO. For each memory segment, the controller chooses RETAIN, SUMMARIZE, or PRUNE, using H=4 fixed linear-projection alignments between segments and commitments. Its reward combines a heuristic task score, a drift proxy computed by a separate encoder (response dissimilarity plus prediction error), and token cost. With a frozen Qwen2.5-1.5B-Instruct generator, MACC reaches 62.9% judge acceptability on LoCoMo at 0.092 normalized cost. Full-History reaches 59.8%, and a No-History control reaches 60.4% at 0.087 (Table 1). On a human-labelled diagnostic of 60 probes (ACCST), three heuristics fail on 55–67% of probes and a post-hoc MACC checkpoint fails on 78.3% while retaining no dialogue history (Table 2, §6.3). An H=4 vs. H=1 pilot and a partial adaptive-search transfer are reported as exploratory.
Critical strengths
Careful, self-critical evaluation design. The paper uses a method-blind judge, an explicit No-History control, two independent human annotators with Krippendorff's α and threshold checks, confidence intervals clustered by scenario, and full release of artifacts (§5, §6.3, A.5). This design is what exposes the gap between task–cost scores and commitment fidelity, and it is the most reusable part of the work.
Critical weaknesses
The controller learns to discard memory, and the headline metric cannot detect this. Compared with No-History, MACC has a mean prompt of 79 vs. 72 tokens, cost of 0.092 vs. 0.087, and acceptability of 62.9±5.1 vs. 60.4±7.5, with overlapping intervals (Table 6). On the diagnostic, context hashes confirm that no history was retained in any of the 60 cases, and the failure rate is 78.3%, worse than Full-History at 56.7% (§6.3, Table 2). The reported "91% cost reduction at competitive acceptability" therefore comes from an evaluation in which history does not help: No-History scores at least as well as Full-History, and LLMLingua-2 keeping only 0.6 history tokens scores 62.1%. The same judge model failed to detect any of the 13 human-labelled stale cases (§6.3), which undermines the claim that q_eval "scores commitment fidelity". The reward is a plausible cause of the collapse: it combines a cost penalty, a heuristic q_train, and a drift term that the authors acknowledge measures response dissimilarity rather than commitment error (§4.2). This may make pruning everything optimal, but the paper does not analyse it.
The claimed components are not isolated, and the evidence base is thin. The multi-aspect heads are untrained in the main runs. The only H ablation uses one seed, two conversations, 60 updates, and a different "strict coherence" judge, and there H=4 costs 44.8% more and has a worse diagnostic score (Table 3). There is no λ_drift=0 ablation. MemAct-matched differs from MACC in both representation and reward (§6.2). All LoCoMo results come from two test conversations (A.1), and the three-seed t-intervals (±4–7.6) overlap for most methods. BST conversations are too short for memory management to matter: four heuristics have cost of exactly 1.000 (Table 5). BST heuristics also used different decoding settings, which confounds MACC's token-F1 advantage (10.2 vs. 6.8). Beyond the evaluation, the components are recombinations of multi-view retrieval (Zhang et al., 2022) and RL memory-as-action (Zhang et al., 2026), and none of the claimed differences is shown to matter.
Questions for authors
1:For each seed, what share of segments are retained, summarized, or pruned at test time? Does the policy prune everything in all three seeds? Which reward term drives this? Please report results with λ_cost=0 and with λ_drift=0.
2:In the standard evaluation, do the MACC and No-History prompts include C_t through M_core? If commitments already reach the generator that way, what is left for the controller to preserve, and how does MACC differ functionally from Goal-Reinjection?
3:How well does q_eval agree with the human labels on the 240 diagnostic responses?