acceptodds
Under review as a conference paper at ICLR 2027

How Do Language Models Use Memory? From Internal Strategy Readout to Adaptive Control

Abstract

Retrieval determines which memories a language model can access, but does not ensure their appropriate use. We investigate whether memory-use requirements can be read from a frozen language model's internal representations and used to control how memories influence its responses. Activation differences induced by individual memories encode information about required memory use, while the effect of isolating memories on readout accuracy varies across backbones. Oracle-guided interventions using these differences improve memory use, but struggle to jointly satisfy mixed requirements to enhance some memories and suppress others; stronger interventions can also induce repetitive generation. Motivated by these findings, we propose DUET (Dual-Adaptive Memory Control), which uses internal strategy readouts to specify control targets, updates memory-specific intervention directions at each generation step, and adapts intervention strengths to the resulting output-distribution changes, without updating model parameters. We evaluate DUET across six backbones and three benchmarks. On RPEval, it achieves 68.3% Multi Macro with Qwen3-8B and jointly satisfies all memory-use requirements in 50.0% of mixed queries. Ablations further show that dynamic directions and adaptive strengths jointly improve multi-memory control.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Memory Utilization, Memory-Augmented Language Models, Personalized Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_PF37OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper studies whether requirements for memory use can be read from the activations of a frozen language model and translated into better control of generation. It proposes DUET, which predicts a control coefficient for each memory from a labeled reference bank, updates directions derived from activation differences during decoding, and adapts their strengths using changes in the output distribution. Experiments across six backbones and three benchmarks report improvements in selective memory use and joint satisfaction of multiple requirements.

Critical strengths
  1. The paper addresses a meaningful problem and distinguishes reading strategies for memory use from enforcing them during generation. The limitations of fixed interventions motivate DUET, which reports substantial improvements on key metrics of memory use across multiple models.

  2. The ablations and supporting analyses are relatively thorough. Ablations across component combinations and steering strengths, experiments varying calibration data size, and tests with references from an independent source help explain the contributions of the components and reference data. The results reveal both complementary effects on controlling multiple memories and trade-offs on individual metrics.

Critical weaknesses
  1. The main comparisons use different test sets without explaining why. Appendix B.3 reports 252 baseline versus 285 DUET queries on RPEval, and 351 versus 370 query groups on BenchPreS, before invalid judgments are excluded. The reported gaps therefore cannot be cleanly attributed to the methods rather than differences in sample composition. All methods should be compared on the same test set, separate from calibration data.

  2. The replacement judge lacks reliability validation. DUET uses a single 27B judge, whereas RPEval and SteeM use judges such as GPT-4.1 and Gemini-2.5-Pro, supported by comparisons with human judgments or other judges. This reliability evidence does not automatically transfer to the replacement, particularly for Support versus Dominate and graded memory dependence. Evaluation with the original benchmark judges, or capable independent judges larger than the 27B model used here, is needed to check whether the gains and rankings are robust.

Questions for authors
  1. Why do the baseline and DUET test sets differ, and how do they overlap? Evaluating DUET and all baselines on the same test set for each benchmark would help confirm whether the reported gains hold.

  2. Could the authors validate the key findings using the original benchmark judges or comparably capable larger models? This would help determine whether the conclusions remain robust to the choice of judge, given the smaller judge used here and the absence of human validation for the replacement.

5 comments
AuthorsOpenReview ↗

Response to Reviewer PF37 (1/2)

We thank the reviewer for recognizing the importance of the problem, the distinction between strategy readout and execution, and our supporting analyses.


W1/Q1. Common evaluation sets and their overlap

Due to time constraints before submission, we were unable to complete all methods on a common evaluation list. The earlier baseline and later DUET experiments used different calibration splits, resulting in different evaluation sets. We sincerely apologize for this inconsistency.

We have now completed all six methods across six backbones on common evaluation sets. The original overlaps and corrected evaluation populations are:

BenchmarkOriginal baseline / DUET setsOriginal overlapCommon evaluation set
RPEval252 / 285 queries237 queries285 queries
BenchPreS351 / 370 groups334 groups370 groups
SteeM (Qwen3-4B)190 / 190 groups180 groups190 groups, 950 instances

Current calibration data are excluded from the evaluation lists; few-shot examples come only from the corresponding calibration sets. Each metric is compared with its strongest baseline. Arrows denote baseline→DUET; differences are DUET minus baseline, and brackets show 95% paired intervals.

RPEval: joint satisfaction of multiple memory requirements. Multi measures joint satisfaction of all memory requirements; Mixed is the subset with mixed usage requirements. Higher is better.

BackboneMulti: best baseline → DUET (%)Difference [95% CI]Mixed: best baseline → DUET (%)Difference [95% CI]
Qwen3-4BFew-shot 36.36 → 55.94+19.58 [10.49, 28.67]Few-shot 20.00 → 32.94+12.94 [2.35, 24.71]
Qwen3-8BFew-shot 40.56 → 67.83+27.27 [17.48, 37.08]Few-shot 23.53 → 49.41+25.88 [11.76, 38.82]
Qwen3.5-4BFew-shot 23.78 → 53.15+29.37 [19.58, 39.16]Few-shot 21.18 → 41.18+20.00 [8.24, 31.76]
Qwen3.5-9BFew-shot 25.17 → 58.74+33.57 [24.48, 42.66]Few-shot 17.65 → 34.12+16.47 [4.71, 28.24]
Llama-3.1-8BFew-shot 38.46 → 58.04+19.58 [11.19, 27.97]RP-R 17.65 → 30.59+12.94 [0.00, 25.88]
Llama-3.2-3BFew-shot 44.06 → 55.94+11.89 [4.20, 20.28]Few-shot 16.47 → 27.06+10.59 [-1.18, 22.35]

BenchPreS: appropriate adoption, misapplication, and task completeness. AAR measures appropriate adoption, MR misapplication, and TC task completeness. Higher AAR/TC and lower MR are better. Differences are in percentage points for AAR/MR and on the original scale for TC.

BackboneAAR ↑ (%)MR ↓ (%)TC ↑ (1–5)
Qwen3-4BCoT 67.91 → 76.50; Δ+8.59 [4.94, 12.23]Reminder 29.44 → 2.51; Δ-26.93 [-29.42, -24.60]Reminder 3.186 → 3.030; Δ-0.157 [-0.281, -0.032]
Qwen3-8BRP-R 72.45 → 76.50; Δ+4.05 [0.00, 8.18]CoT 36.42 → 1.22; Δ-35.20 [-38.06, -32.28]Reminder 3.332 → 3.505; Δ+0.173 [0.057, 0.289]
Qwen3.5-4BRP-R 70.66 → 74.39; Δ+3.73 [0.00, 7.47]CoT 44.61 → 1.05; Δ-43.55 [-46.81, -40.29]Few-shot 2.365 → 3.324; Δ+0.959 [0.824, 1.089]
Qwen3.5-9BReminder 68.40 → 74.88; Δ+6.48 [3.41, 9.56]CoT 57.50 → 1.95; Δ-55.56 [-58.58, -52.49]Reminder 2.608 → 3.405; Δ+0.797 [0.676, 0.916]
Llama-3.1-8BRP-R 69.69 → 77.47; Δ+7.78 [4.74, 10.95]CoT 10.54 → 1.78; Δ-8.76 [-10.97, -6.69]Reminder 3.111 → 3.411; Δ+0.300 [0.184, 0.414]
Llama-3.2-3BRP-R 65.80 → 78.44; Δ+12.64 [9.23, 16.12]CoT 10.71 → 2.68; Δ-8.03 [-9.86, -6.27]Reminder 2.697 → 3.192; Δ+0.495 [0.378, 0.611]

SteeM: error in controlling memory dependence. Lower MAE is better.

BackboneBest baselineMAE: baseline → DUET ↓Difference [95% CI]
Qwen3-4BRP-R1.205 → 1.121-0.084 [-0.148, -0.019]
Qwen3-8BCoT1.201 → 1.122-0.079 [-0.147, -0.011]
Qwen3.5-4BRP-R1.213 → 1.031-0.182 [-0.256, -0.105]
Qwen3.5-9BRP-R1.198 → 0.979-0.219 [-0.294, -0.140]
Llama-3.1-8BFew-shot1.244 → 0.968-0.276 [-0.335, -0.217]
Llama-3.2-3BVanilla1.328 → 1.080-0.248 [-0.307, -0.189]

On the common evaluation sets, DUET retains higher RPEval Multi accuracy, lower BenchPreS misapplication, and lower SteeM MAE than the strongest respective baselines on all six backbones; the paired 95% intervals exclude zero. Thus, DUET's core advantages in multi-memory control, reducing memory misuse, and regulating reliance remain.

We will update the main results tables in the final manuscript with these common-set results and revise the splits and experimental settings in Appendix B.3. We again apologize for the original inconsistency and thank you for bringing this issue to our attention.

AuthorsOpenReview ↗

Response to Reviewer PF37 (2/2)

W2/Q2. Judge reliability and independent re-evaluation

Thank you for raising judge reliability. We selected Qwen3.8-27B for its overall capability, controlled local deployment, and evaluation reproducibility. Previously published Artificial Analysis Intelligence Index (version 4.3.2) results are as follows (higher is better):

ModelAA Intelligence Index
Qwen3.8-27B (xhigh)34
GPT-5.6 Sol (low)33
Gemini 2.5 Pro16
GPT-4.113 (AA estimate)

These results place the reasoning variant of Qwen3.8-27B at a capability level comparable to some newer proprietary models and clearly above the earlier models listed above.

As described in Appendix B.3, we deploy the model locally with SGLang, enable deterministic inference, and set temperature to zero to improve evaluation stability and reproducibility. We also manually inspect a sample of evaluation results; 90% of the inspected judgments are considered reasonable.

We further re-evaluate the same responses with the same rubrics using the original benchmark judges, Sol, and Terra.

For RPEval, we re-evaluate all six methods on the same 285 Qwen3-4B queries. The table reports DUET's advantage over the strongest baseline; Few-shot is the strongest baseline for both metrics under all four judges.

JudgeMulti gap (percentage points)Mixed gap (percentage points)
Paper judge (27B)+19.58+12.94
GPT-4.1 (benchmark judge)+10.49+0.00
Sol+28.67+25.88
Terra+23.08+15.29

Manual inspection identifies errors in GPT-4.1's interpretation of per-memory target labels. For example, a Few-shot response to query rpval_impl_multi_0022 has target AABC: Ignore the first two memories, Support the third, and let the fourth Dominate. The response recommends gentle bedtime stories according to the third memory while avoiding meditation, which the fourth explicitly rejects. GPT-4.1 instead interprets the first three memories as Ignore and penalizes the response, assigning 2/4. Sol and Terra correctly interpret the third requirement and assign 4/4.

We further randomly sample 10 Mixed queries and inspect one Few-shot and one DUET rationale per query, yielding 20 rationales per judge. Explicit label-reading errors are:

JudgeRationales with label-reading errorsSample error rate
GPT-4.110/2050.0%
Sol1/205.0%
Terra2/2010.0%

Mixed measures joint satisfaction of all memory-use requirements, so a local label-reading error can affect the entire query's judgment. Sol and Terra exhibit fewer such errors while retaining clear Multi and Mixed advantages, supporting DUET's multi-memory control effectiveness.

For SteeM, we add Gemini 2.5 Pro ratings on the previous random 30 groups, using the same 150 responses per method:

MethodGemini 2.5 Pro MAE ↓
Few-shot1.293
CoT1.207
DUET1.200

Gemini's mean ratings of the same DUET responses by requested dependence level are:

JudgeTarget 1Target 2Target 3Target 4Target 5
Gemini 2.5 Pro1.901.533.203.133.43

Gemini assigns mean ratings of 3.20, 3.13, and 3.43 to responses requested at levels 3, 4, and 5, without clear separation across these adjacent levels. Manual inspection also confirms fine-grained dependence-level confusions in some Gemini judgments and supports the evaluations of the paper's 27B judge, Sol, and Terra on the reviewed examples.

For example, a feedback-revision response about work-life boundaries cites historical details about boundary enforcement, communication scripts, and recovery activities. Gemini consequently assigns the highest level, 5. However, the response retains a self-contained explanatory structure and does not meet the highest-level requirement of directly continuing the history in a way whose full understanding depends on that specific history. Sol and Terra both assign 3, a judgment supported by manual inspection.

We then sample 30 of the 77 groups where DUET outperforms both Few-shot and CoT under the original 27B ratings, retaining all five target levels per group, and re-evaluate the same responses with the same rubric using Sol and Terra. The table reports baseline MAE minus DUET MAE; positive values favor DUET.

JudgeMAE reduction vs Few-shotMAE reduction vs CoT
Paper judge (27B)+0.407+0.493
Sol+0.360+0.493
Terra+0.447+0.413

Both Sol and Terra reproduce the performance advantages on these 30 groups. We will include the cross-judge results, manually checked examples, and audit statistics in the final manuscript to clarify the basis of our evaluation.

We hope these results address your concerns and would be grateful if you could reconsider your rating.

Dear Reviewer PF37,

Thank you again for your thoughtful comments and constructive suggestions throughout the discussion. As the author–reviewer discussion period is coming to an end, we would like to kindly ask whether our responses have addressed your main concerns. If there are still aspects that remain unclear or require further clarification, we would be very happy to discuss them further.

We sincerely appreciate your time and valuable feedback.

Reviewer_vEGEOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether the appropriate use of retrieved memories can be inferred from the internal representations of frozen language models and controlled through activation interventions. The authors first demonstrate that memory-specific activation differences can predict usage strategies, including Ignore, Support, and Dominate. They then propose DUET, which combines strategy readouts, dynamically updated memory-specific steering directions, and KL-based adaptive intervention strengths. Experiments across six language models and three memory-use benchmarks demonstrate substantial improvements over several prompting-based baselines.

Critical strengths
  1. This paper studies the memory use of LLMs from the angle of activations. This mechnistical approach seems to have good empirical performance.
  2. The paper is generally well-written and easy to follow.
Critical weaknesses
  1. The evaluation might be a little flawed and insufficiently controlled. Appendix B.3 explicitly states that the results are computed on different valid evaluation sets and are not paired estimates. For example, the baselines and DUET are evaluated on 252 and up to 285 RPEval queries, respectively. BenchPreS also uses different numbers of evaluation groups.
  2. The novelty is limited. There's a wide array of literature on using activation differences to control model behaviors. I am not an expert in memory-specific settings, but I feel like the proposed method is too straightforward and brings no additional novel insights to the community.
  3. It appears to me that DUET requires additional memory-specific and memory-free decoding branches at every generation step. The computational and memory requirements therefore scale with the number of retrieved memories. However, the paper provides no quantitative measurements of inference latency, throughput, FLOPs, or peak GPU memory. Without such results, it is difficult to assess whether the performance improvements justify the substantial additional inference cost, especially relative to simpler memory-selection approaches.
Questions for authors

Aforementioned in weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer vEGE (1/3)

We thank the reviewer for the careful assessment and for recognizing the empirical performance and clear presentation of our work. We address the three concerns below.


W1. Evaluation on different test sets.

Due to time constraints before submission, we were unable to complete experiments for all methods on common evaluation sets, so the original comparison used different evaluation sets. We sincerely apologize for this inconsistency.

We have now completed experiments for all six methods across six backbones on common evaluation sets:

BenchmarkCommon evaluation set
RPEval285 queries
BenchPreS370 groups
SteeM190 groups, 950 instances

The corrected results follow. Each metric is compared with its strongest baseline. Arrows denote baseline→DUET; differences are DUET minus baseline, and brackets show 95% paired intervals.

RPEval: joint satisfaction of multiple memory requirements. Multi measures the proportion of responses that satisfy all memory requirements simultaneously; Mixed is the subset with mixed usage requirements. Higher is better for both.

BackboneMulti: best baseline → DUET (%)Difference [95% CI]Mixed: best baseline → DUET (%)Difference [95% CI]
Qwen3-4BFew-shot 36.36 → 55.94+19.58 [10.49, 28.67]Few-shot 20.00 → 32.94+12.94 [2.35, 24.71]
Qwen3-8BFew-shot 40.56 → 67.83+27.27 [17.48, 37.08]Few-shot 23.53 → 49.41+25.88 [11.76, 38.82]
Qwen3.5-4BFew-shot 23.78 → 53.15+29.37 [19.58, 39.16]Few-shot 21.18 → 41.18+20.00 [8.24, 31.76]
Qwen3.5-9BFew-shot 25.17 → 58.74+33.57 [24.48, 42.66]Few-shot 17.65 → 34.12+16.47 [4.71, 28.24]
Llama-3.1-8BFew-shot 38.46 → 58.04+19.58 [11.19, 27.97]RP-R 17.65 → 30.59+12.94 [0.00, 25.88]
Llama-3.2-3BFew-shot 44.06 → 55.94+11.89 [4.20, 20.28]Few-shot 16.47 → 27.06+10.59 [-1.18, 22.35]

BenchPreS: appropriate adoption, misapplication, and task completeness. AAR measures appropriate adoption, MR measures misapplication, and TC measures task completeness. Higher AAR/TC and lower MR are better. Differences are in percentage points for AAR/MR and on the original scale for TC.

BackboneAAR ↑ (%)MR ↓ (%)TC ↑ (1–5)
Qwen3-4BCoT 67.91 → 76.50; Δ+8.59 [4.94, 12.23]Reminder 29.44 → 2.51; Δ-26.93 [-29.42, -24.60]Reminder 3.186 → 3.030; Δ-0.157 [-0.281, -0.032]
Qwen3-8BRP-R 72.45 → 76.50; Δ+4.05 [0.00, 8.18]CoT 36.42 → 1.22; Δ-35.20 [-38.06, -32.28]Reminder 3.332 → 3.505; Δ+0.173 [0.057, 0.289]
Qwen3.5-4BRP-R 70.66 → 74.39; Δ+3.73 [0.00, 7.47]CoT 44.61 → 1.05; Δ-43.55 [-46.81, -40.29]Few-shot 2.365 → 3.324; Δ+0.959 [0.824, 1.089]
Qwen3.5-9BReminder 68.40 → 74.88; Δ+6.48 [3.41, 9.56]CoT 57.50 → 1.95; Δ-55.56 [-58.58, -52.49]Reminder 2.608 → 3.405; Δ+0.797 [0.676, 0.916]
Llama-3.1-8BRP-R 69.69 → 77.47; Δ+7.78 [4.74, 10.95]CoT 10.54 → 1.78; Δ-8.76 [-10.97, -6.69]Reminder 3.111 → 3.411; Δ+0.300 [0.184, 0.414]
Llama-3.2-3BRP-R 65.80 → 78.44; Δ+12.64 [9.23, 16.12]CoT 10.71 → 2.68; Δ-8.03 [-9.86, -6.27]Reminder 2.697 → 3.192; Δ+0.495 [0.378, 0.611]

SteeM: error in controlling memory dependence. MAE measures the distance between observed reliance and the requested level; lower is better.

BackboneBest baselineMAE: baseline → DUET ↓Difference [95% CI]
Qwen3-4BRP-R1.205 → 1.121-0.084 [-0.148, -0.019]
Qwen3-8BCoT1.201 → 1.122-0.079 [-0.147, -0.011]
Qwen3.5-4BRP-R1.213 → 1.031-0.182 [-0.256, -0.105]
Qwen3.5-9BRP-R1.198 → 0.979-0.219 [-0.294, -0.140]
Llama-3.1-8BFew-shot1.244 → 0.968-0.276 [-0.335, -0.217]
Llama-3.2-3BVanilla1.328 → 1.080-0.248 [-0.307, -0.189]

Across all six backbones, DUET retains higher RPEval Multi accuracy, lower BenchPreS misapplication, and lower SteeM MAE than the strongest respective baselines; the paired 95% intervals exclude zero. Part 2 compares these results with our original scores and addresses novelty.

AuthorsOpenReview ↗

Response to Reviewer vEGE (2/3)

W1 (continued). Comparison with our original results.

We also compare DUET's original results with the re-evaluated scores. The RPEval table reports original experiment → re-evaluation (change), with changes in percentage points.

BackboneSingle: old → new (change)Multi Macro: old → new (change)Multi Micro: old → new (change)Mixed Macro: old → new (change)Mixed Micro: old → new (change)
Qwen3-4B78.9 → 78.87 (-0.03)55.4 → 55.94 (+0.54)81.7 → 81.62 (-0.08)30.9 → 32.94 (+2.04)73.4 → 73.68 (+0.28)
Qwen3-8B83.1 → 83.10 (+0.00)68.3 → 67.83 (-0.47)87.5 → 87.35 (-0.15)50.0 → 49.41 (-0.59)81.9 → 81.68 (-0.22)
Qwen3.5-4B78.2 → 78.17 (-0.03)53.1 → 53.15 (+0.05)84.6 → 84.62 (+0.02)41.18 → 41.18 (+0.00)83.82 → 83.82 (+0.00)
Qwen3.5-9B76.1 → 76.06 (-0.04)58.7 → 58.74 (+0.04)88.5 → 88.53 (+0.03)34.12 → 34.12 (+0.00)83.82 → 83.82 (+0.00)
Llama-3.1-8B73.2 → 73.24 (+0.04)58.5 → 58.04 (-0.46)85.6 → 85.53 (-0.07)31.0 → 30.59 (-0.41)79.4 → 79.34 (-0.06)
Llama-3.2-3B69.7 → 69.72 (+0.02)56.7 → 55.94 (-0.76)83.0 → 82.14 (-0.86)27.7 → 27.06 (-0.64)74.9 → 73.68 (-1.22)

Single and all BenchPreS metrics remain unchanged at the original reporting precision. The maximum absolute change in Multi Macro is 0.76 percentage points, and the maximum absolute change in SteeM MAE is 0.006, as detailed below. The originally reported scores retain their original precision; these differences describe numerical changes after re-evaluation.

BackboneOriginal MAERe-evaluated MAENew minus original
Qwen3-4B1.1211.121+0.000
Qwen3-8B1.1251.122-0.003
Qwen3.5-4B1.0311.031+0.000
Qwen3.5-9B0.9730.979+0.006
Llama-3.1-8B0.9680.968+0.000
Llama-3.2-3B1.0801.080+0.000

These results preserve DUET's core advantages in multi-memory control, reducing memory misuse, and regulating reliance. We will update the main results tables in the final manuscript with the common-set results and revise Appendix B.3 accordingly. We again apologize for the original inconsistency and thank you for identifying it.


W2. Novelty and research contributions

Thank you for the suggestion. We agree that controlling model behavior using activation differences has been extensively studied. Our contribution is to address memory use in multi-memory settings, rather than to introduce a new activation-difference algorithm.

The difficulty is that a single response may need to ignore some memories, draw on others, and let a particular memory dominate the answer. The model must identify how each memory should be used and jointly fulfill these different requirements during generation. Our paper studies both stages and finds that, even with correct strategy labels, fixed interventions struggle to satisfy mixed memory-use requirements. This reveals a gap between identifying strategies and executing them.

DUET addresses this difficulty by updating each memory's intervention direction using the current generated prefix and adjusting intervention strength as generation proceeds. To test these designs, we add a direction-by-strength comparison on Qwen3-8B under the main 5% reference-bank setting, holding strategy coefficients, evaluation queries, and decoding settings fixed. The table reports Multi Macro, the proportion of responses satisfying all memory-use requirements together.

Intervention directionIntervention strengthMulti Macro (%) ↑
FixedFixed42.66
FixedAdaptive37.76
Updated with the generated prefixFixed56.64
Updated with the generated prefixAdaptive (DUET)67.83

With strength held fixed, updating directions improves Multi Macro from 42.66% to 56.64%; adding adaptive strength further improves it to 67.83%. Adapting strength alone with fixed directions does not improve performance. These results support updating directions with the generation state and combining this with strength adaptation to improve joint execution of multiple memory-use requirements.

We will clarify our contributions to memory-use strategy identification and joint execution across multiple memories in the final manuscript and include these additional controlled experiments.

AuthorsOpenReview ↗

Response to Reviewer vEGE (3/3)

W3. Inference overhead and efficiency improvements

We agree that inference overhead is an important limitation of DUET. As acknowledged under “Inference overhead” in Appendix A (Limitations), the additional memory-specific branches and per-step updates increase computation and memory costs. To quantify this limitation, we add cost and performance comparisons on the same hardware, together with measurements for memory counts N=1–10.

Our primary goal is to explore whether internal signals in a frozen model reveal how each memory should be used, and whether these signals can be translated into effective control over multiple memories. DUET demonstrates the feasibility of this direction while also revealing room to improve inference efficiency.

We measure Qwen3-8B costs on the same 285 RPEval evaluation queries as the main experiment. Methods are compared on the same physical A100 80GB per query under identical decoding settings; the performance column reports joint satisfaction on all 143 Multi queries. Filtering and strategy prompting use the same predicted strategies as DUET: the former removes Ignore memories, while the latter additionally states Support/Dominate requirements.

MethodOutput tokens/sPeak GiBTime to first token (ms)Multi Macro (%)
Vanilla30.1917.8039.692.10
Filtering30.2517.7836.8851.75
Filtering + strategy prompt30.2417.7738.0058.04
DUET25.0721.14173.9467.83
DUET (compact branches)27.2819.46105.6063.64

Time to first token measures the generation stage; a separate strategy-reader replay averages 151.48 ms. The following N=1–10 breakdown reports only throughput, peak memory, and time to first token for original and compact DUET:

NDUET tokens/sDUET peak GiBDUET TTFT msCompact tokens/sCompact peak GiBCompact TTFT ms
127.2818.1152.1728.5817.9439.42
226.4518.2677.9226.5618.0466.36
325.5518.85163.7426.5018.3589.89
425.3319.28174.5526.8218.66113.64
523.5819.69235.8626.1718.78143.03
622.6120.24315.6126.0319.22202.54
719.8820.62410.6424.7719.38233.47
819.2620.61605.3623.7319.46280.37
917.6421.14616.8024.2619.37319.31
1017.7319.98725.3623.4618.95399.75

The compact implementation shares identical memory-free branches and skips zero-coefficient auxiliary branches, while retaining all memories in the main context. On the full evaluation set, throughput improves by 8.8% and peak memory decreases by 7.9%; Multi Macro is 67.83% for original DUET and 63.64% for compact DUET. The compact variant passes 6 fewer Multi queries. We present this as a trade-off between speed and control quality and report both.

Future work will explore more efficient implementations, such as combining steering with OPSD and using steering-guided memory-use behavior as a training signal to internalize this control capability in the model parameters, with the aim of reducing reliance on additional decoding branches at inference time. We will include the efficiency results in the final manuscript and explicitly discuss this limitation and future directions.

Reviewer_GWGEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether memory-use requirements can be predicted from a frozen language model’s activations and used to control generation. It first studies strategy readout from activation differences induced by individual memories, then evaluates oracle-guided fixed interventions and their limitations under mixed requirements. Based on these observations, it proposes DUET, which obtains memory-specific control coefficients from a labeled reference bank, updates intervention directions with the generated prefix, and adapts intervention strengths using changes in the output distribution. Results across six backbones and three benchmarks are promising, particularly for jointly satisfying multiple memory-use requirements. However, unmatched evaluation subsets and missing controls make the magnitude and source of the improvements difficult to establish conclusively.

Critical strengths
  1. The problem is relevant and well motivated. Retrieving a memory does not determine whether it should be ignored, used as supporting information, or treated as a governing constraint. The emphasis on mixed requirements and joint satisfaction captures a meaningful challenge beyond memory recall.

  2. The experimental organization separates several important questions. The paper distinguishes readable information, oracle-guided controllability, and an end-to-end system using predicted coefficients. Reporting the limitations of fixed interventions, including severe repetition and weak mixed-requirement performance, provides useful evidence rather than only favorable results.

  3. The component analysis supports the importance of dynamic control. With the same predicted coefficients and evaluation queries, Table 10 shows that updating directions increases Qwen3-4B Multi Macro from 44.0% to 60.67% at α=1\alpha=1, while adding adaptive strength increases it further to 66.0%. These controlled comparisons provide more direct support for the design than the headline baseline comparisons alone.

  4. The evaluation has useful breadth. Results span multiple model families and benchmark formulations. The calibration-size analysis and independent-source reference experiment address dependence on the reference bank. The paper also acknowledges that isolation does not consistently improve readout across all backbones.

Critical weaknesses
  1. The main comparisons use different evaluation populations. Appendix B.3 reports 252 RPEval queries for the baselines versus 285 for DUET, and 351 BenchPreS query groups versus 370. Remaining invalid judgments are also excluded from each method’s scores. The manuscript explicitly notes that these are not paired estimates. Although the large and consistent gains are encouraging, the reported differences cannot be attributed cleanly to the method without controlling query composition. Results on a common held-out subset are needed to establish the effect sizes.

  2. The baselines do not isolate the contribution of the controller from that of strategy identification. DUET obtains explicit control decisions from a labeled reference bank, but the main comparisons do not reuse those same decisions in a simpler generation method. A particularly informative baseline would remove predicted Ignore memories and explicitly describe the predicted Support/Dominate requirements in the prompt. Without this control, it remains unclear how much improvement requires the proposed intervention mechanism. Since the intervention is applied after the final normalization and before the linear vocabulary projection, its control component can also be expressed as a weighted combination of memory-conditioned logit differences. This makes a matched fixed-strength contrastive control particularly relevant.

  3. The readout experiment does not directly evaluate the reader deployed by DUET. RQ1 uses a calibration-selected, single-layer nearest-centroid reader, whereas DUET uses nearest-neighbor matching over a fixed multilayer window (Section 3.1 versus Section 5.1 and Appendix C.2). Consequently, the reported RQ1 accuracy does not directly establish the reliability of DUET’s control coefficients. The deployed reader’s classification results and the gap between predicted- and oracle-coefficient DUET would help connect the representation analysis to the final system. More generally, successful readout and intervention support controllability, but do not by themselves identify the model’s native decision mechanism.

  4. The computational trade-off is insufficiently quantified. DUET maintains full-context, no-memory, and memory-specific decoding branches and computes per-memory distributional probes at each step. The limitations section acknowledges this overhead, but does not report latency, throughput, or peak memory as the number of memories increases. These measurements are important for assessing whether the additional control is worthwhile relative to simpler methods using the same strategy information.

Questions for authors
  1. Why do the main baseline and DUET evaluations use different query sets? Can the authors report the main and mixed-requirement metrics on a common held-out subset, with paired uncertainty estimates and a consistent treatment of invalid judgments?

  2. Using exactly DUET’s predicted coefficients, how does its controller compare with removing Ignore memories and explicitly prompting the remaining requirements? A comparison on one representative existing backbone would directly address the controller’s incremental contribution.

  3. What is the classification performance of the actual multilayer nearest-neighbor reader, and how much end-to-end performance is lost relative to DUET with oracle coefficients? Does this differ substantially between All Ignore and Mixed queries?

  4. What are DUET’s latency, throughput, and peak-memory costs under the existing benchmark settings? How do these costs vary with the number of memories, and how do they compare with a simpler method supplied with the same predicted strategy information?

5 comments
AuthorsOpenReview ↗

Response to Reviewer GWGE (1/3)

We thank the reviewer for recognizing the motivation, the distinction between strategy readout and execution, and the evidence for dynamic control. We address the four concerns below.


W1. Main and mixed-requirement results on common evaluation sets

Due to time constraints before submission, we were unable to complete all methods on a common evaluation set, resulting in comparisons across different evaluation sets. We sincerely apologize for this insufficiently rigorous comparison.

We have now completed all six methods across six backbones on common evaluation sets:

BenchmarkCommon evaluation set
RPEval285 queries
BenchPreS370 groups
SteeM190 groups, 950 instances

All main-table judgments are now complete and valid. Scores use fixed evaluation lists, with 95% paired bootstrap intervals calculated by resampling queries or complete query groups. Each metric is compared with its strongest baseline; arrows denote baseline→DUET, and differences are DUET minus baseline.

RPEval: joint satisfaction of multiple memory requirements. Multi measures joint satisfaction of all memory requirements; Mixed is the subset with mixed usage requirements. Higher is better.

BackboneMulti: best baseline → DUET (%)Difference [95% CI]Mixed: best baseline → DUET (%)Difference [95% CI]
Qwen3-4BFew-shot 36.36 → 55.94+19.58 [10.49, 28.67]Few-shot 20.00 → 32.94+12.94 [2.35, 24.71]
Qwen3-8BFew-shot 40.56 → 67.83+27.27 [17.48, 37.08]Few-shot 23.53 → 49.41+25.88 [11.76, 38.82]
Qwen3.5-4BFew-shot 23.78 → 53.15+29.37 [19.58, 39.16]Few-shot 21.18 → 41.18+20.00 [8.24, 31.76]
Qwen3.5-9BFew-shot 25.17 → 58.74+33.57 [24.48, 42.66]Few-shot 17.65 → 34.12+16.47 [4.71, 28.24]
Llama-3.1-8BFew-shot 38.46 → 58.04+19.58 [11.19, 27.97]RP-R 17.65 → 30.59+12.94 [0.00, 25.88]
Llama-3.2-3BFew-shot 44.06 → 55.94+11.89 [4.20, 20.28]Few-shot 16.47 → 27.06+10.59 [-1.18, 22.35]

BenchPreS: appropriate adoption, misapplication, and task completeness. AAR measures appropriate adoption, MR misapplication, and TC task completeness. Higher AAR/TC and lower MR are better. Differences are in percentage points for AAR/MR and on the original scale for TC.

BackboneAAR ↑ (%)MR ↓ (%)TC ↑ (1–5)
Qwen3-4BCoT 67.91 → 76.50; Δ+8.59 [4.94, 12.23]Reminder 29.44 → 2.51; Δ-26.93 [-29.42, -24.60]Reminder 3.186 → 3.030; Δ-0.157 [-0.281, -0.032]
Qwen3-8BRP-R 72.45 → 76.50; Δ+4.05 [0.00, 8.18]CoT 36.42 → 1.22; Δ-35.20 [-38.06, -32.28]Reminder 3.332 → 3.505; Δ+0.173 [0.057, 0.289]
Qwen3.5-4BRP-R 70.66 → 74.39; Δ+3.73 [0.00, 7.47]CoT 44.61 → 1.05; Δ-43.55 [-46.81, -40.29]Few-shot 2.365 → 3.324; Δ+0.959 [0.824, 1.089]
Qwen3.5-9BReminder 68.40 → 74.88; Δ+6.48 [3.41, 9.56]CoT 57.50 → 1.95; Δ-55.56 [-58.58, -52.49]Reminder 2.608 → 3.405; Δ+0.797 [0.676, 0.916]
Llama-3.1-8BRP-R 69.69 → 77.47; Δ+7.78 [4.74, 10.95]CoT 10.54 → 1.78; Δ-8.76 [-10.97, -6.69]Reminder 3.111 → 3.411; Δ+0.300 [0.184, 0.414]
Llama-3.2-3BRP-R 65.80 → 78.44; Δ+12.64 [9.23, 16.12]CoT 10.71 → 2.68; Δ-8.03 [-9.86, -6.27]Reminder 2.697 → 3.192; Δ+0.495 [0.378, 0.611]

SteeM: error in controlling memory dependence. Lower MAE is better.

BackboneBest baselineMAE: baseline → DUET ↓Difference [95% CI]
Qwen3-4BRP-R1.205 → 1.121-0.084 [-0.148, -0.019]
Qwen3-8BCoT1.201 → 1.122-0.079 [-0.147, -0.011]
Qwen3.5-4BRP-R1.213 → 1.031-0.182 [-0.256, -0.105]
Qwen3.5-9BRP-R1.198 → 0.979-0.219 [-0.294, -0.140]
Llama-3.1-8BFew-shot1.244 → 0.968-0.276 [-0.335, -0.217]
Llama-3.2-3BVanilla1.328 → 1.080-0.248 [-0.307, -0.189]

On the common evaluation sets, DUET retains higher RPEval Multi accuracy, lower BenchPreS misapplication, and lower SteeM MAE than the strongest respective baselines on all six backbones; the paired 95% intervals exclude zero. DUET's core advantages in multi-memory control, reducing memory misuse, and regulating reliance remain.

We will update the main results tables and the experimental settings in Appendix B.3 in the final manuscript. We again apologize for the original comparison and thank you for bringing this issue to our attention.

AuthorsOpenReview ↗

Response to Reviewer GWGE (2/3)

W2. Contributions of strategy identification and the controller

Thank you for suggesting this important control. We agree that the contributions of strategy identification and the controller should be distinguished more directly. On Qwen3-8B, we therefore added controls using the same strategies predicted by DUET: removing memories predicted as Ignore, and additionally specifying the Support/Dominate requirements for the remaining memories in the prompt.

We also agree that our intervention can be expressed equivalently as a weighted combination of memory-conditioned logit differences. We therefore added a matched fixed-strength control that retains the same predicted coefficients, generation branches, and directions updated with the generated prefix, while fixing the intervention strength. All methods are evaluated on the same 285 RPEval queries.

MethodMulti Macro ↑All Ignore Macro ↑Mixed Macro ↑
Filter predicted Ignore memories51.7574.1436.47
Filtering + explicit strategy prompting58.0477.5944.71
Fixed-strength control56.6493.1031.76
DUET67.8394.8349.41

With the same strategy information, DUET improves Multi Macro by 9.79 percentage points over filtering plus strategy prompting and by 11.19 points over the fixed-strength control. It also achieves higher scores on All Ignore and Mixed. These results show that the controller contributes beyond strategy identification by helping the model execute the requirements of multiple memories. We will add these controls to the revised paper and distinguish the contributions of strategy identification and control execution more clearly.


W3. Reliability of the deployed reader and its impact on end-to-end performance

Thank you for highlighting the need for a more direct connection between the representation analysis and the deployed system. We agree that RQ1's single-layer nearest-centroid experiment cannot replace an evaluation of DUET's deployed multilayer nearest-neighbor reader. We therefore directly examined the predicted strategies used during generation. The Qwen3-8B results are as follows:

BenchmarkNumber of strategy classesClassification accuracy (%) ↑Macro-F1 (%) ↑
RPEval391.3186.87
BenchPreS299.7399.70
SteeM550.7450.57

The deployed reader identifies memory-use strategies accurately on RPEval and BenchPreS, while substantial room remains for improving the five-level dependency readout on SteeM.

We then kept the DUET controller and decoding settings unchanged and replaced only the predicted coefficients with coefficients corresponding to the ground-truth labels, to examine how strategy-identification errors affect end-to-end performance:

RPEval subsetExact strategy-vector accuracy (%)DUET with predicted coefficients (%)DUET with ground-truth coefficients (%)Gain (percentage points)
All Ignore86.2194.8398.28+3.45
Mixed58.8249.4154.12+4.71

Exact strategy-vector accuracy requires every memory's strategy within a query to be predicted correctly; end-to-end performance uses the Macro metric. Identifying the complete strategy vector is more difficult on Mixed queries. In this 8B setting, replacing the coefficients produces similar-sized gains on the two subsets, both below 5 percentage points.

These results directly connect the deployed reader to the final control outcomes. We will add these evaluations to the revised paper and explicitly limit our mechanistic interpretation to the finding that model activations contain readable information about memory-use strategies, and this information can be used to steer generation behavior.

AuthorsOpenReview ↗

Response to Reviewer GWGE (3/3)

W4. Computational overhead and efficiency improvements

We agree that inference overhead is an important limitation of DUET. As acknowledged under “Inference overhead” in Appendix A (Limitations), the additional memory-specific branches and per-step updates increase computation and memory costs. We have added cost and performance comparisons on the same hardware, together with measurements as the number of memories increases.

We evaluate Qwen3-8B on the same 285 RPEval queries as the main experiment. Methods are compared on the same physical A100 80GB per query under identical decoding settings. Filtering and strategy prompting use the same predicted strategies as DUET; the performance column reports joint satisfaction on all 143 Multi queries.

MethodOutput tokens/sPeak GiBTime to first token (ms)Multi Macro (%)
Vanilla30.1917.8039.692.10
Filtering30.2517.7836.8851.75
Filtering + strategy prompt30.2417.7738.0058.04
DUET25.0721.14173.9467.83
DUET (compact branches)27.2819.46105.6063.64

Time to first token measures the generation stage; a separate strategy-reader measurement averages 151.48 ms. Grouping the queries by their number of memories gives the following costs for original and compact DUET:

NDUET tokens/sDUET peak GiBDUET TTFT msCompact tokens/sCompact peak GiBCompact TTFT ms
127.2818.1152.1728.5817.9439.42
226.4518.2677.9226.5618.0466.36
325.5518.85163.7426.5018.3589.89
425.3319.28174.5526.8218.66113.64
523.5819.69235.8626.1718.78143.03
622.6120.24315.6126.0319.22202.54
719.8820.62410.6424.7719.38233.47
819.2620.61605.3623.7319.46280.37
917.6421.14616.8024.2619.37319.31
1017.7319.98725.3623.4618.95399.75

Throughput, peak memory, and time to first token are measured in tokens/s, GiB, and ms, respectively.

Queries with more memories generally have lower throughput and longer waits for the first token. To improve efficiency, we implemented branch compression: sharing identical memory-free branches and skipping zero-coefficient auxiliary branches, while retaining all memories in the main generation context. On the full evaluation set, throughput improves by 8.8% and peak memory decreases by 7.9%, while Multi Macro changes from 67.83% to 63.64%, illustrating the trade-off between control quality and efficiency.

Our primary goal is to explore whether internal signals in a frozen model reveal how each memory should be used and whether these signals enable effective control over multiple memories. Future work will explore more efficient implementations, such as combining steering with OPSD to internalize steering-guided memory-use capabilities in the model parameters and reduce reliance on additional branches at inference time. We will add these efficiency results to the final manuscript and further discuss this limitation and directions for improvement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.