- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper treats long-term user memory as a routing policy on a frozen MoE: an additive bias on router logits, applied before top-k, either as a static vector estimated from native top-k selection frequencies or as a prompt-conditioned readout from a token–expert graph. The authors give results on route containment and top-k stability, and report lower off-topic intrusion than prompt, LoRA and memory-expert baselines on Qwen3-30B, disposition shifts on GLM-4.5-Air and Qwen3-Next-80B, and a serving cost no higher than the base model for the static variant.
Critical strengths
- Paired installation protocol. The paper installs a known memory and measures, against the no-memory model, both its intended effect and its appearance on unrelated prompts. Over-personalization has been studied before, but this pairing lets routing and content memory be compared on one criterion, benefit at low intrusion.
- Small training-free state. The static variant needs no training and a small state (24 KiB on 30B), and does not lengthen the context. If the benefit-with-low-intrusion claim holds, this is a practical advantage over prompt memory in context length and latency.
Critical weaknesses
- The evidence does not separate user preference from history topic. The static state contrasts expert frequencies on the user's history with those on a neutral corpus (Eq. 3), so what it stores is which experts the history's content activates. Each disposition effect is consistent with that reading: the selective-disposition lift (Sec. 5.2, Fig. 4) measures how often the model names a person from the memory's domain, the crossed design shows only that this depends on the query, and the style transfer on Qwen3-Next-80B (Table 2) is what a French-style history would induce. The paper itself calls the metric a topic-disposition lift, yet Sec. 6 describes the router as storing user-specific preferences, and no experiment installs histories with similar topics and opposite preferences, the one comparison that would tell the two apart. Either that comparison or a narrower description of the state, as a topic disposition, is needed to support this framing.
- The advantages over injection and over the static variant are not shown under matched conditions. Against injection: intrusion is compared on Qwen3-30B (Fig. 2) at each method's installed strength, which Sec. 5.1 notes does not match personalization benefit, while the positive effects in Table 2 come from GLM-4.5-Air and Qwen3-Next-80B, so benefit and intrusion are never reported for one backbone side by side. Because the detector is a lexicon and no uncertainty estimates are given, the gap of 0.007 against 0.449–0.624 is as consistent with weaker memory action as with better selectivity. Against the static variant: the co-loading curves (Fig. 5) compare one static vector with a graph that stores the history, and the graph's query dependence is only in direction, since any query sharing a literal token with it receives a bias of full strength β (Sec. 3.3, Eq. 4); the curves therefore do not show that query-dependent readout, rather than more stored state, is what retains the lift. Low intrusion at useful strength is the paper's central claim, so this is the main reason for my rating; a strength sweep on one backbone should be doable within the rebuttal period.
Questions for authors
- Can histories that share topic vocabulary but state opposite preferences be installed, and does the bias move behavior in the stated direction on unseen queries, compared with a shuffled-history bias of equal norm?
- On GLM-4.5-Air, how do target lift and off-topic intrusion compare when the strength of routing, prompt memory and a relevance-gated retrieval arm is swept, and which β underlies the reported results?
- Can the co-loading experiment include a baseline that stores one static vector per memory and selects it by query match, at the same storage as the graph?
- How many runs underlie Table 2 and Figs. 2, 4 and 5, what are their intervals, and do the intrusion scores in Fig. 2 hold under a non-lexical judge?
- Table 3 shows the static state faster than the no-memory model (first token 352.6 vs. 414.2 ms) although it adds a bias operation; what explains this, and was warm-up controlled across arms?
- Eq. 4 yields one vector over experts while the bias is layer-specific (Sec. 3.1). How is the readout assigned to each layer, and is the dynamic bias computed once from the prompt or updated during decoding?
2 comments
Q4. How many runs support the added tables, and do the intrusion results hold with a non-lexical judge?
Appendix D.1–D.4 gives strengths, decoding settings, intervention layers and interval definitions. Each added point is linked to its generated answers. Sampling is summarized below.
| Added experiment | Histories/questions and repetitions | Interval unit |
|---|---|---|
| GLM joint benefit/intrusion | Eight test histories, 144 target + 384 unrelated cases per setting, five seeds | Domains, histories and shared query families after seed averaging |
| Same-topic preferences | 24 paired cases per attribute, three seeds, positive/negative and neutral conditions | Six synthetic topic/history variants after seed averaging |
| GLM co-loading | Five loads; six generic + eighteen constrained cases per load, three seeds | Histories and queries after seed averaging |
| 80B response attributes | 12 held-out history pairs per attribute, three generations per query, seed 0 | History pairs |
The added GLM records cover 214 main jobs and 15 capability jobs, totaling 61,248 response rows. Steering supplies 12 separate runs. Original figures retain their own protocols and seed counts. Appendix D.3 also reports the three existing 80B capped-sum seed curves individually.
The semantic evaluator gives zero mean intrusion for static/PPR beta=3, compared with 0.888 for three-record prompting and 0.493 for active retrieval. Its ordering agrees with the lexical detector on the new GLM test.
Prompting has per-response intrusion 0.242 and at-least-one-in-five intrusion 0.646. Active retrieval gives 0.169 and 0.404. The latter counts cases with any detection across five answers. The evaluator sees no method label or installed preference pole. Unique question/answer/rubric triples are judged once and scores mapped to originating responses. All frontier scores parse. The 25 invalid unique preference judgments are excluded with valid counts reported. The evaluator has no human calibration.
Q5. Why was static faster than base, and was warm-up controlled?
We repeated the 30B timing test with six discarded warm-ups per arm and three independent process starts. Each process uses four prompts and five repeats, randomizing arm order within blocks and synchronizing CUDA.
| Process | Static minus base: first token (ms) | Static minus base: turn (ms), 95% CI |
|---|---|---|
| First | −7.07 | +24.32 [20.82, 27.85] |
| Second | −6.95 | +27.23 [24.28, 30.08] |
| Third | −7.30 | +24.88 [20.47, 29.11] |
Static retains a smaller first-token advantage, whose cause remains unresolved. Full-turn time increases by about 1.4%. Table 5 uses these repeated measurements, with intervals in Appendix D.5.
Table 5 reports medians of three process-level medians. No-memory first-token/turn times are 109.3/1799.1 ms, versus 101.8/1825.1 ms for static at 32 tokens. The paired estimates above match prompts and repetitions across arms within each process. Their intervals bootstrap repetitions.
The measurement uses an exclusive RTX 6000D, PyTorch 2.13.0+cu130 and CUDA 13.0. Each process records 120 turn and 18 session observations across six arms after warm-up. Earlier measurements remain in Appendix B.5. Context length is unchanged. Installation and PPR propagation are excluded.
Q6. How is one graph readout assigned to layers, and is it updated while decoding?
The graph pools expert evidence across intervened layers. PPR computes one normalized vector per prompt and broadcasts it to those layers, holding it fixed during decoding. Static retains a separate vector per layer. The GLM runs and Qwen graph construction use this implementation.
Max-normalization fixes the largest component at beta for supported prompts. Prompts change the direction, with no separate semantic-relevance gain. Section 3 states the layer assignment and update schedule.
The graph connects tokens by local co-occurrence, tokens to routed experts, and experts by co-activation. Overlapping prompt token IDs form the seed. After 30 PageRank iterations with restart alpha=0.15, we normalize the expert coordinates by their maximum and multiply by beta. Empty seeds stay zero. A nonzero readout receives the same maximum gain even for semantically distant questions. GLM broadcasts the 128-dimensional vector across all 45 MoE layers.
The bias precedes native scoring and selection. GLM retains sigmoid scoring, correction-bias and grouped selection. Neither the graph nor the bias is updated during generation.
Q1. Do opposite preferences on the same topic move behavior in the stated direction?
Not consistently. We tested opposing preferences about the same topic against neutral-history and equal-norm wrong-user controls. Results at beta=3 are below.
Histories about statistics and computing state opposing preferences for length, formula-first explanations, bullets or answer-first order. Three variants per topic and attribute are tested on unseen query families. The 288 cases comprise 96 positive, 96 negative and 96 neutral histories, giving 24 positive/negative pairs per attribute with three seeds. Wrong-user histories are reassigned within each topic/attribute group at the same routing norm, beta=3.
Correct histories have a larger effect on bullets but smaller effects on the other attributes. Every answer reaches the 192-token budget, so these scores describe generated prefixes. Appendix D.2 gives intervals and scoring rules.
The bullet differences are +0.069 [0.000, 0.139] for correct histories and +0.028 [−0.028, 0.097] for wrong-user assignments. This ordering alone does not establish a significant advantage. Answer-first moves in the opposite direction at beta=3. At beta=6, differences are −0.012 for length, 0.000 for formula order, +0.028 for bullets and +0.014 for answer order, all with intervals spanning zero. Base and neutral pole differences are zero because their inputs and states are shared.
The control permutes history-to-user assignments while preserving each history's words and order. We did not run word shuffling. Preference direction is not reliably recovered on these four GLM attributes, and the revision describes the observed behavior as history-conditioned dispositions.
The separate 80B tests find changes in language, register, formatting and reading level, with user-turn and assistant-turn histories compared directly. They use different history construction and scoring from this GLM control. Both experiments are reported. Our intended application is to retain a way of answering that the frozen model already knows.
Q2. What happens at matched benefit, with a gated retrieval arm?
We ran the GLM grid with base and relevance-gated retrieval. Routing produces much smaller target effects than prompting or active retrieval. No routing setting qualifies for a matched-positive-benefit comparison.
Eight held-out histories cover football, basketball, tennis and motorsport. Each setting has 144 target-domain and 384 unrelated cases, with separate validation histories and question families. We sweep four routing strengths, prompt record counts and retrieval thresholds. At beta=3, static gives +0.026 [−0.017, 0.083] and PPR +0.040 [−0.094, 0.218]. Three-record prompting gives +0.815 [0.681, 0.919], and active retrieval +0.797 [0.665, 0.910].
Retrieval inserts positively matched records at threshold zero and abstains at the higher tested thresholds, returning zero benefit and intrusion. PPR applies nonzero bias even on unrelated queries. Its low intrusion occurs with memory active. Section 5.1 reports both outcomes, with matching rules in Appendix D.1.
Q3. Can the co-loading test separate query dependence from stored state?
We added a query-selected static vector bank and a fixed readout of the same PPR graph. The fixed readout changes only the seed. On generic queries its lift is 0.000 throughout, while PPR ranges from −0.056 to 0.000. At eleven distractors on country-constrained queries, PPR gives +0.037 versus −0.037 for fixed readout. PPR's advantage depends on the query type.
The bank stores one static state per history and selects by IDF-cosine query matching. It returns zero bias without overlap and breaks ties by history hash. Fixed-graph readout retains the PPR transition matrix and seeds all retained history tokens uniformly, holding graph storage fixed.
Capped sum adds independently normalized signatures with a per-layer cap of 2 beta, without query selection. Its lift is +0.333 at four and seven distractors and +0.222 at eleven on generic queries. Concatenated-history static stays near zero. Appendix D.3 reports all five loads, six generic and eighteen country-constrained cases per load, three seeds and history/query-resampling intervals.
At eleven distractors, the stored numeric arrays are:
These sizes exclude the bank's text retrieval index. Appendix D.3 gives serialized sizes. Only the fixed-graph/PPR pair holds complete graph storage exactly equal.
The revised cost description includes graph storage and query-time propagation. Static state has fixed size once installed.