acceptodds
Under review as a conference paper at ICLR 2027

ProTTT: Learning to Learn Semantic User Memory with Test-Time Training

Abstract

Personalization requires language models to capture user-specific knowledge from a growing user history. Existing context-based approaches incur increasing inference costs as user history accumulates and rely on separate retrieval or summarization stages, while parametric-based approaches often require reconstructing user representations when new user data is added. We introduce **ProTTT**, a profile-supervised meta-learning framework for learning semantic user memory. The memory construction starts from a shared initialization and is updated for each user through test-time training on user history, allowing it to evolve continuously as the history grows. However, since test-time training alone does not explicitly encourage the memory to capture semantic user knowledge necessary for personalization, we learn this shared initialization using textual user profiles as supervision, so that test-time training on user history captures semantic knowledge more effectively. **ProTTT** consistently outperforms both full history ICL and all parametric baselines across diverse benchmarks, while substantially reducing inference cost by compressing user history into a lightweight parameterized memory. Our analysis also shows that profile supervision is a reliable objective for learning semantic user knowledge and that the resulting memory can track and retain evolving user preferences, while remaining robust across different history sizes. Overall, we demonstrate the effectiveness of test-time training for personalization and establish **ProTTT** as a baseline for continuously evolving user memory.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
language model personalization, semantic user memory, test-time training, meta-learning, parametric memory, long-term user modeling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wm3uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a profile-supervised meta-learning framework, namely PROTTT, for constructing semantic user memory in lightweight adapters. It learns a shared initialization that enables user-specific memory to be constructed and incrementally updated through next-token prediction on user history. Experiments show improvements over full-history ICL and the evaluated parametric baselines on several personalization benchmarks, while reducing per-query inference costs.

Critical strengths
  1. The paper studies a meaningful and practical problem in personalized language models. Its parametric memory compresses extensive user histories into lightweight adapters, reducing the need to repeatedly include the history in the context.

  2. The proposed profile-supervised meta-learning provides a useful way to guide history-based updates toward capturing semantic user information.

  3. The experiments demonstrate the overall effectiveness of the method across multiple personalization benchmarks. PROTTT outperforms full-history ICL and the evaluated parametric baselines on the main benchmarks.

Critical weaknesses

This paper could be strengthened by the following additional experiments:

  1. Comparison with stronger textual-profile baselines. Guided Profile Generation (Zhang, 2024) and ReCAP (Park et al., 2026) should be considered for inclusion as baselines. Given the limitations of textual-profile approaches discussed in the Introduction, evaluating this category solely through generic PAG and PAG + Full History leaves the comparison somewhat incomplete, particularly since the paper already uses the CMV data and profile-generation prompts from ReCAP.

  2. Comparison with standard supervised fine-tuning. It is suggested to conduct standard supervised training using the same history–profile data to train the shared adapter to predict profiles from histories, followed by the same TTT process, and then compare the results against PROTTT. The current "TTT (naive)" baseline removes both profile supervision and meta-training, making it difficult to isolate the incremental contribution of meta-learning.

  3. A history-summary prediction objective. Using the same teacher model, it could be useful to compare the effectiveness of using a "history summary" versus a "high-level user profile" as the meta-learning supervision signal, while matching output lengths, query conditions, and training budgets as closely as possible. This would help determine whether the performance gains stem from semantic abstraction, rather than simply from the addition of extra textual supervision or differences in the richness of the supervision.

If these experiments cannot be included, the authors could explain why and discuss the implications for the scope of the conclusions.

Questions for authors

Could the authors provide a specific case showing a user's history and how the model's personalized responses change before and after memory construction? An example involving newly added history would also help readers understand how the memory incorporates new user information.

5 comments
AuthorsOpenReview ↗

Official Response to Reviewer (Part 1 of 5)

We sincerely thank you for the constructive feedback and for recognizing that our parametric memory addresses a meaningful and practical problem by compressing extensive user histories into lightweight adapters, that profile-supervised meta-learning provides a useful way to guide history-based updates toward semantic user information, and that ProTTT outperforms the baselines across multiple personalization benchmarks. We apologize for our late response, as running all the suggested experiments took longer than expected. Below, we address your key points on stronger textual-profile baselines (GPG and ReCAP), comparison with standard supervised fine-tuning, and history-summary supervision, as well as your question on the case study.


W1. Comparison with stronger textual-profile baselines. Guided Profile Generation (Zhang, 2024) and ReCAP (Park et al., 2026) should be considered for inclusion as baselines. Given the limitations of textual-profile approaches discussed in the Introduction, evaluating this category solely through generic PAG and PAG + Full History leaves the comparison somewhat incomplete, particularly since the paper already uses the CMV data and profile-generation prompts from ReCAP.

A1.

Comparison with stronger textual-profile baselines

We thank the reviewer for this suggestion. We have added both methods as baselines.

  • GPG (Zhang, 2024). No code has been released, and the paper gives the first-stage questions only for its own tasks, so we reimplemented its two-stage protocol, writing the questions for our tasks in the paper's format. First, the LLM lists the key information in each history record (e.g., the stance expressed in each CMV message). Then it writes a descriptive profile from the history and this list. We use the same profiler (gpt-5-mini) and the same 100 most recent records as for PAG.
  • ReCAP (Park et al., 2026). We evaluated ReCAP on CMV. Following the official recipe, we trained a Llama-3.1-8B-Instruct profiler with DPO on the ReCAP CMV training split, with rewards computed by Qwen3-0.6B, the same task model used throughout our experiments.

All rows use Qwen3-0.6B as the task model and the same test splits as the main results. "+ full history" adds the same 100 records used by ICL. Bold marks the best value in each column.

MethodCMV AUC ↑PMem T6 ↑PMem T7 ↑LaMP-3 MAE ↓LaMP-3 RMSE ↓
Base0.4660.2730.0882.102.60
ICL (full history)0.4750.2730.1400.891.57
PAG profile0.4110.3090.1232.983.28
PAG profile + full history0.4940.2730.1231.302.03
GPG profile0.4290.4730.2112.893.22
GPG profile + full history0.5200.2910.1231.011.72
ReCAP profile0.434––––
ReCAP profile + full history0.476––––
ProTTT (w/ user)0.6240.3270.2980.771.11

ProTTT outperforms all textual-profile baselines on CMV, PersonaMem T7, and LaMP-3. GPG with full history is the strongest textual-profile baseline. The one exception is PersonaMem T6, where GPG's profile alone is best.

We have added these baselines and results to Appendix A.6 (Table 11) of the revision.

References

Jiarui Zhang. Guided Profile Generation Improves Personalization with Large Language Models. In Findings of EMNLP, 2024.

Sejun Park, Yoonah Park, Jongwon Lim, and Yohan Jo. Learning to Retrieve User History and Generate User Profiles for Personalized Persuasiveness Prediction. In Findings of ACL, 2026.

AuthorsOpenReview ↗

Official Response to Reviewer (Part 2 of 5)

W2. Comparison with standard supervised fine-tuning. It is suggested to conduct standard supervised training using the same history–profile data to train the shared adapter to predict profiles from histories, followed by the same TTT process, and then compare the results against PROTTT. The current "TTT (naive)" baseline removes both profile supervision and meta-training, making it difficult to isolate the incremental contribution of meta-learning.

A2.

Comparison with standard supervised fine-tuning (profile supervision without meta-learning)

We thank the reviewer for this suggestion. Before reporting the results, we would like to confirm that we understood the request correctly.

Our understanding of the request. We read it as training the shared adapter on the same history–profile data with ordinary supervised learning, with no meta-learning, and then applying the same TTT process at test time. For each training user, we run TTT on the user's history from the current adapter weights, compute the profile-prediction loss at the resulting weights, and apply its gradient directly to those weights as a standard supervised update. The gradient is not propagated back through the TTT steps, and the weights are not reset, so the next user starts from the updated weights. ProTTT differs in exactly these two points: it propagates the profile loss back through the TTT steps to one shared initialization, to which the model returns for every user. Could the reviewer confirm that this is the comparison intended? If a different setup was meant, we would appreciate a short clarification.

Setup. Everything other than the training rule is identical to ProTTT. Evaluation is also identical: we run the same TTT on the test user's history from the final weights and then score the model. Each score is shown as before → after TTT on the test user's history.

MethodProfile supervisionMeta-learningCMV AUC ↑PMem T6 ↑PMem T7 ↑LaMP-3 MAE ↓LaMP-3 RMSE ↓
Base––0.4660.2730.0882.102.60
Supervised + TTT (suggested control)✓✗0.460 → 0.4680.309 → 0.2910.105 → 0.1052.04 → 2.032.65 → 2.64
ProTTT (ours)✓✓0.518 → 0.6240.218 → 0.3270.246 → 0.2981.35 → 0.771.55 → 1.11

Without meta-learning, TTT on the user's history has no effect: the model learns nothing about the user, so no memory is formed. On every benchmark, the control's scores barely change with TTT and stay close to the base model, whereas the same TTT process improves ProTTT on every benchmark.

Profile supervision alone is therefore not sufficient. In the control, the profile loss only improves the current weights on the profile task, and nothing in the objective requires the TTT step itself to produce a useful change. In ProTTT, the profile loss is propagated through the TTT steps to the shared initialization, so the initialization is trained specifically so that TTT on a new user's history yields a model of that user. This is the incremental contribution of meta-learning that the reviewer asked us to isolate.

AuthorsOpenReview ↗

Official Response to Reviewer (Part 3 of 5)

W3. A history-summary prediction objective. Using the same teacher model, it could be useful to compare the effectiveness of using a "history summary" versus a "high-level user profile" as the meta-learning supervision signal, while matching output lengths, query conditions, and training budgets as closely as possible. This would help determine whether the performance gains stem from semantic abstraction, rather than simply from the addition of extra textual supervision or differences in the richness of the supervision. (If these experiments cannot be included, the authors could explain why and discuss the implications for the scope of the conclusions.)

A3.

History summary vs. high-level profile as supervision

Thank you for this suggestion. We have now run the suggested comparison. We replaced each profile used as the meta-learning supervision signal with a summary of the same history that records what the user said and asked, without generalizing into traits, preferences, or background. Output lengths are matched for every target (the same number of items and a similar number of tokens), and the query conditions, training users, and training budget are identical to the main model. For CMV, the summaries and the profiles are written by the same teacher (gpt-5-mini) from the same input. For AlpsBench, whose profiles are the dataset's annotated memories, the summaries are written by gpt-5-mini from the same conversations. Cells show before → after writing the user's history.

SupervisionCMV AUC ↑PMem T6 ↑PMem T7 ↑LaMP-3 MAE ↓LaMP-3 RMSE ↓
High-level profile (main)0.518 → 0.6240.218 → 0.3270.246 → 0.2981.35 → 0.771.55 → 1.11
History summary0.474 → 0.5510.218 → 0.2360.421 → 0.0702.57 → 2.342.90 → 2.77

A detailed history summary is a much weaker supervision signal than a high-level profile, so the gains come from semantic abstraction rather than from extra or richer text supervision. After writing, the summary-supervised model is below the main model on every benchmark, and on PersonaMem T7, writing even lowers accuracy. The summary has the same length as the profile and records the same history in more detail, so neither the amount nor the richness of the supervision explains the gap. Abstracting the history into a high-level description of the user is therefore a more effective supervision signal for meta-learning than recording the history in detail.

We have added this comparison to Section 5.2 (Table 4, “History summary”) of the revision.

Reviewer_gcanOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces test-time training into personalized language modeling and proposes a new method, PROTTT. The proposed approach aims to balance personalization performance and computational efficiency by addressing the accuracy limitations of retrieval-based methods and the high adaptation costs of parameter-update-based approaches. It offers a new perspective on personalization.

Critical strengths
  1. The motivation is clear, and the paper is generally well-written, with a clear presentation of the proposed method.

  2. The proposed method has practical value. By applying test-time training to personalization tasks, it enables the incorporation of user-specific knowledge while reducing the computational cost of parameter updates, offering a promising approach to efficient personalization.

  3. The experimental evaluation is reasonably comprehensive, and the key experimental designs are appropriate and provide empirical support for the main conclusions.

Critical weaknesses
  1. The methodological novelty appears limited. The proposed test-time training approach essentially performs LoRA-based adaptation for each user, with an additional optimization of the LoRA initialization. This design is highly similar to OPPU.

  2. Reproducibility remains a concern. Although the paper includes a Reproducibility Statement, the corresponding code has not been made available. Moreover, the user profiles in the training dataset are additionally constructed, and their quality may substantially affect model training and performance. Releasing the implementation and the constructed profile data would be important for ensuring reproducibility.

  3. Experimental validation on larger models is missing. The experiments are limited to models with 0.6B and 1B parameters, whereas practical personalization applications often involve larger language models.

  4. The analysis of the initialization parameter Φs\Phi_s is insufficient. Its initialization relies on user profiles, and factors such as the number of users involved in initialization training and the quality of their profiles may significantly influence the resulting performance. However, the paper lacks systematic experiments investigating these factors.

Questions for authors

Why is the proposed method not evaluated on personalized text generation tasks, such as LaMP-4, LaMP-5, and LaMP-7?

5 comments
AuthorsOpenReview ↗

Official Response to Reviewer (Part 1 of 4)

We sincerely thank you for the constructive feedback and for recognizing the practical value of our method for efficient personalization and our comprehensive experiments supporting the main conclusions. We apologize for our late response, as we took the time to complete the requested experiments. Below, we address your key points on novelty relative to OPPU, reproducibility, larger models, and the analysis of the initialization (the number of users and profile quality), as well as your question on personalized text generation (LaMP-4, LaMP-5, and LaMP-7).


W1. The methodological novelty appears limited. The proposed test-time training approach essentially performs LoRA-based adaptation for each user, with an additional optimization of the LoRA initialization. This design is highly similar to OPPU.

A1.

On novelty relative to OPPU and other per-user LoRA methods

We thank the reviewer for raising this comparison. OPPU and related methods that train a LoRA adapter for each user, such as the per-user semantic memory of PRIME (Zhang et al., 2025), end up with a LoRA module for each user, and so does our method. They differ in where the knowledge of what to store comes from, and this difference is the core of our contribution.

Per-user LoRA methods get this knowledge from task labels; our method gets it from a meta-learned initialization. OPPU trains each personal module by supervised fine-tuning on the user's history, formatted as input–label pairs of the target task, on top of a base model adapted to the same task with other users' labels; PRIME likewise uses task-oriented fine-tuning. Because the labels decide what is stored, a standard initialization suffices. At write time, we only have the user's raw, unlabeled history, so this knowledge can only reside in the starting point of the update. We therefore meta-learn one initialization, shared across users, such that gradient steps on a new user's raw history alone turn the model into a model of that user. The initialization is thus not an extra optimization added to per-user LoRA; it is the write rule itself.

Removing it shows this directly (Table 2 of the revision; Qwen3-0.6B, same raw history). A per-user LoRA trained on the raw history with the next-token loss, which is also OPPU's own fallback for histories without task labels, stays well below our model on every task and leaves PersonaMem Type 7 at the base level. Naive TTT, our model with a standard instead of the meta-learned initialization, changes CMV and Type 7 only slightly and makes LaMP-3 worse. With the same memory block and write steps, our model is clearly better on all three. This gap isolates the effect of the learned rule.

What the initialization stores and how to form it is a core design variable. The outer objective determines what goes into the initialization, and changing it changes what transfers. With the next-token objective of TTT-E2E in place of ours, on the same architecture, writing changes CMV only slightly and lowers PersonaMem Type 7. With CMV judge labels as the objective, the same architecture learns the task, but writing the history no longer helps. Per-user LoRA methods have no such axis, because their labels fix what is stored. This is where we see the methodological novelty.

References

  • Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning. In Proceedings of EMNLP, 2024.
  • Xinliang Frederick Zhang, Nicholas Beauchamp, and Lu Wang. PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought Process. In Proceedings of EMNLP, 2025.

W2. Reproducibility remains a concern. Although the paper includes a Reproducibility Statement, the corresponding code has not been made available. Moreover, the user profiles in the training dataset are additionally constructed, and their quality may substantially affect model training and performance. Releasing the implementation and the constructed profile data would be important for ensuring reproducibility.

A2.

Reproducibility

We thank the reviewer for raising this point.

Code and data release. We will publicly release the full implementation, including the training and evaluation code and all experiment configurations. We will also release all constructed user profiles used for training, along with the prompts and scripts used to construct them, so that the profile data can be reused directly or regenerated.

Profile quality. We agree that the quality of the constructed profiles may affect training and downstream performance. The reviewer also raises this in W4, so we address it there with additional experiments on profile quality. Please see our response to W4.

AuthorsOpenReview ↗

Official Response to Reviewer (Part 2 of 4)

W3. Experimental validation on larger models is missing. The experiments are limited to models with 0.6B and 1B parameters, whereas practical personalization applications often involve larger language models.

A3.

Experiments on larger models

We thank the reviewer for raising this point. In addition to Qwen3-0.6B and Llama-3.2-1B in the paper, we trained ProTTT on Qwen3-4B and Qwen3-8B with the same training data and objective; only the memory rank (16 for 4B and 8 for 8B, vs. 18 for 0.6B) and the outer learning-rate schedule were adjusted for stable training at larger scale. We compare against the prompting baselines from the main results, run on the same backbone: the base model without user information, ICL with the user's history (the same up-to-100 most recent records that ProTTT trains on at test time), and the PAG profile. "ProTTT (w/o user)" and "ProTTT (w/ user)" are the same model before and after test-time training (TTT) on the user's history. Bold marks the best value in each column.

CMV (AUC ↑)

MethodQwen3-0.6BQwen3-4BQwen3-8B
Base0.4660.6170.591
ICL (full history)0.4750.5600.601
Profile (PAG)0.4110.5710.602
ProTTT (w/o user)0.5180.5170.578
ProTTT (w/ user)0.6240.6290.658

LaMP-3 (MAE ↓ / RMSE ↓)

Method0.6B MAE0.6B RMSE4B MAE4B RMSE8B MAE8B RMSE
Base2.102.600.861.100.620.96
ICL (full history)0.891.570.390.690.290.62
Profile (PAG)2.983.281.111.390.901.24
ProTTT (w/o user)1.351.550.911.151.401.58
ProTTT (w/ user)0.771.110.760.990.550.85

PersonaMem (accuracy ↑; T6 = preference-aligned recommendation, T7 = generalizing preferences to new scenarios)

Method0.6B T60.6B T74B T64B T78B T68B T7
Base0.2730.0880.2910.2110.3090.228
ICL (full history)0.2730.1400.3820.1400.5640.421
Profile (PAG)0.3090.1230.4180.1930.4550.351
ProTTT (w/o user)0.2180.2460.2000.1230.2910.281
ProTTT (w/ user)0.3270.2980.4000.2980.3820.368

TTT on the user's history improves every metric at every scale (w/ user vs. w/o user), and the comparison with the prompting baselines shows two regimes.

Smaller backbones: ProTTT is both more effective and more efficient than ICL. Small models struggle to exploit a long user history in context. Prompting with the history or a profile barely helps or even hurts on CMV, where the user's disposition has to be inferred from many records rather than read off any single one, and gives inconsistent gains on PersonaMem. ProTTT moves this burden from in-context reasoning to test-time training: it consolidates the history into the memory once and then answers each query from the query alone. At 0.6B it outperforms ICL on every benchmark, and at 4B it still does on CMV and PersonaMem. These results suggest that ProTTT can be a particularly useful personalization memory mechanism for on-device assistants and other resource-constrained applications that require small language models, especially when a growing user history must support repeated queries under tight inference budgets.

AuthorsOpenReview ↗

Official Response to Reviewer (Part 3 of 4)

A3 (continued).

Larger backbones: ProTTT stays competitive at a much lower per-query cost. As the backbone grows, ICL becomes strong on tasks whose history directly contains the answer. In LaMP-3, the history consists of the user's past reviews together with their ratings, which serve as labeled examples of the query task. In PersonaMem, the user states their preferences explicitly in the conversation, and the 8B model is able to find and apply these statements in context. ICL with the full history therefore performs best on these two benchmarks, although ProTTT also keeps improving with model size and stays ahead of the base model and the profile baseline on PersonaMem T7. CMV presents a more challenging form of personalization: whether a comment would change the user's mind is never stated explicitly in the history, but must instead be inferred from patterns in the user's past responses. This makes CMV particularly dependent on capturing semantic user knowledge rather than retrieving explicit user information. ProTTT remains the best-performing method at every scale, showing that its gradient-based memory can capture the user-level knowledge required for this more demanding setting. Finally, ICL has to process the entire history again for every query, so its cost grows with the length of the history. ProTTT instead reads the history once during TTT and stores it in a fixed-size memory, which keeps the cost of each query constant regardless of how long the history is.

We have added these results to Section 4.3 (Table 3) of the revision.


W4. The analysis of the initialization parameter is insufficient. Its initialization relies on user profiles, and factors such as the number of users involved in initialization training and the quality of their profiles may significantly influence the resulting performance. However, the paper lacks systematic experiments investigating these factors.

A4.

Analysis of the initialization: number of users and profile quality

Thank you for raising this. We have run the requested experiments now. Each run retrains the initialization with the main recipe. Cells show before → after writing the user's history.

FactorSettingCMV AUC ↑PMem T6 ↑PMem T7 ↑LaMP-3 MAE ↓LaMP-3 RMSE ↓
—Main (100% = 3,309 users, original profiles).518 → .624.218 → .327.246 → .2981.35 → 0.771.55 → 1.11
# users75% (2,493).541 → .576.236 → .345.228 → .2461.28 → 0.881.93 → 1.30
50% (1,650).562 → .487.327 → .255.246 → .2111.32 → 1.692.05 → 2.38
25% (834).439 → .495.309 → .273.193 → .0700.75 → 2.941.38 → 3.26
10% (341).542 → .487.236 → .182.088 → .3863.18 → 2.873.41 → 3.21
Profile qualityIncorrect: profile of another user.463 → .522.255 → .255.088 → .0880.94 → 0.981.21 → 1.65
Summary profile: detail summary of the history.474 → .551.218 → .236.421 → .0702.57 → 2.342.90 → 2.77

Number of users. We subsample the meta-training users at the user level. The learned write becomes generalizable only with enough users. Up to 50% of our pool, the write does not generalize: in every run, writing a new user's history hurts at least one benchmark, and no run improves on all of them. From 75%, the write becomes generalizable and helps on every benchmark as in the main model, but it still falls short of the model trained on the full pool on CMV, PersonaMem Type 7, and LaMP-3.

Profile quality. We degrade the profiles used as supervision in two ways, keeping the user's history, the query prompts, and the length of every target unchanged.

  • Incorrect profiles. Each training user's profile is replaced with the profile of another user from the same source. The text has the same style and amount but no longer describes the user. CMV AUC is lower both before and after writing, writing leaves PersonaMem unchanged (Type 7 stays at the base level), and writing no longer reduces LaMP-3 error.
  • Summary profiles. Each profile is replaced with a length-matched summary of the same history that records what the user said and asked, instead of describing the user's traits, preferences, and background as a semantic profile does. Writing still raises CMV AUC, but LaMP-3 is much worse already before writing, and PersonaMem Type 7 drops sharply after writing.

Both degradations hurt the initialization while the amount of text supervision stays the same: after writing, both fall below the main model on every benchmark. The initialization therefore needs semantic profiles that are correct for the user; profile-like text alone is not sufficient.

We have added these results to Section 5.2 (Table 4) of the revision.

Reviewer_RoCaOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ProTTT, a profile-supervised meta-learning framework that uses test-time training to construct and incrementally update semantic user memory. Experiments show improved personalization performance and inference efficiency across multiple benchmarks, with additional analyses of memory retention and general capability trade-offs.

Critical strengths
  1. The work presents a framework that combines profile-supervised meta-learning with test-time training to construct incrementally updatable user memory.

  2. The experiments cover multiple personalization benchmarks, providing empirical evidence of the method's effectiveness across different tasks.

  3. The paper provides useful analyses of memory acquisition, retention, inference efficiency, and the trade-off between personalization and general capabilities.

Critical weaknesses
  1. The authors simulate evolving user histories by combining compatible information from different users, rather than examining genuine preference changes within the same user. Consequently, the experiments do not adequately demonstrate whether ProTTT can handle conflicting preferences, preference reversals, or catastrophic forgetting in realistic long-term interactions.

  2. The proposed meta-learning objective explicitly supervises user profile prediction, while the continual-memory analysis evaluates knowledge acquisition and retention primarily through the likelihood of those profiles. This alignment between training and evaluation makes it difficult to determine whether the learned memory captures generalizable user semantics beyond profile reconstruction.

  3. The experimental comparisons do not sufficiently establish the novelty and effectiveness of the proposed approach relative to closely related TTT-based methods. It lacks direct comparisons with TTT-E2E, PERK, and GradMem, which also investigate gradient-based memory construction.

  4. The proposed memory mechanism can substantially degrade general model capabilities and generation quality. For example, MMLU accuracy drops from 0.455 to 0.259 on Llama-3.2-1B when using the full memory readout.

  5. Experiments are limited to relatively small language models (0.6B and 1B), and maintaining approximately 1.06 million memory parameters per user raises concerns about deployment at scale.

Questions for authors

Pls see the weakness

6 comments
AuthorsOpenReview ↗

Official Response to Reviewer (Part 1 of 6)

We sincerely thank you for the constructive feedback and for recognizing our framework that combines profile-supervised meta-learning with test-time training for incrementally updatable user memory, its effectiveness across multiple personalization benchmarks, and our useful analyses. We apologize for our late response, as we took the time to complete all the additional experiments. Below, we address your key points on within-user preference changes, the alignment between profile supervision and evaluation, comparisons with TTT-based methods (TTT-E2E, PERK, and GradMem), general capabilities, and larger models and per-user memory size.


W1. The authors simulate evolving user histories by combining compatible information from different users, rather than examining genuine preference changes within the same user. Consequently, the experiments do not adequately demonstrate whether ProTTT can handle conflicting preferences, preference reversals, or catastrophic forgetting in realistic long-term interactions.

A1.

Preference changes within the same user (conflicting preferences, reversals, forgetting)

To address the reviewer's concerns, we have run a new experiment that covers the three situations the reviewer raises: a preference replaced by another, a preference reversed, and an earlier preference that is never mentioned again (forgetting).

Why synthetic data. For the same user, we need posts written before a preference changed, posts written after it changed, and a way to check which preference the model holds at each point. We did not find a public dataset with all three, so we carefully generated one as follows.

Data. We generated 100 users. Each user has 8 everyday preferences on different topics (food, sports, music, work habits, etc.):

  • 3 stay the same throughout (unchanged);
  • 3 flip direction, e.g., "Dislikes eating Thai food" → "Enjoys eating Thai food" (reversed);
  • 2 move to another value in the same category, e.g., "Favorite sport to watch is baseball" → "… is basketball" (replaced).

GPT-5-mini then wrote 47 first-person posts per user, about 200 words each, in the same format as the user posts we write into memory in the CMV evaluation. Each post in blocks 1–6 is about exactly one of the user’s 8 preferences and does not mention the other seven. In it, the user states their stance in their own words and explains it through an everyday situation, e.g., “I still avoid eating Thai food” while describing turning down a group order for Thai takeout. The posts in blocks 7–8 are about none of the 8 preferences. The posts come in 8 blocks, written in this order:

BlocksNumber of posts per blockWhat the posts show
1–38each of the 8 preferences once, at its original value
4–65each of the 5 changed preferences once, at its new value
7–84unrelated topics

Each changed preference therefore gets 3 posts before the change and 3 after it. Unchanged preferences do not appear after block 3. The first post after a change says that the user's view changed and why (e.g., a coworker took the user to a good Thai restaurant). The later ones just show the new preference.

An LLM judge that was not told the intended preferences checked that each post shows its own preference and none of the others, that the original values can be recovered from blocks 1–3 alone, and that the final values can be recovered from all 47 posts. 720 of the 800 preferences passed every check (182 replaced, 260 reversed, 278 unchanged). We restricted our evaluation to these 720 validated preferences.

Procedure. We use the main ProTTT checkpoint (Qwen3-0.6B) as is, with no further training. For each user, we write block 1 into an empty memory with test-time training, then write block 2 into the same memory, and so on up to block 8. The memory is never reset.

Measurement. Before any writing and after each block, we give the model the prompt "What do you remember about this user from your conversations with them? List each fact you remember about the user, one per line." and compute how likely it is to answer with a one-line sentence stating one of the user’s preferences (a preference statement), e.g., "- Enjoys eating Thai food" (average log-probability per token, the same readout as in Sec. 5). For a changed preference we subtract the old statement's score from the new one's, e.g., log p("Enjoys eating Thai food") − log p("Dislikes eating Thai food"). Above 0 means the model now finds the new preference more likely than the old one. For an unchanged preference we report the score of its statement.

AuthorsOpenReview ↗

Official Response to Reviewer (Part 2 of 6)

A1 (continued).

Averages over preferences; block 0 is before any writing.

After block012345678
Posts in the block–originaloriginaloriginalnewnewnewunrelatedunrelated
Replaced: new − old−0.045−0.037−0.030−0.048−0.038−0.016+0.007−0.014−0.013
Reversed: new − old−0.148−0.083−0.090−0.094−0.109−0.104−0.099−0.099−0.092
Unchanged: statement−3.761−3.207−3.212−3.227−3.283−3.326−3.363−3.359−3.357

Unchanged preferences are mostly kept. Writing block 1 raises the score of these statements from −3.761 to −3.207. After block 3 they are never mentioned again, and after the last block the score is −3.357, so about three quarters of the increase remains.

Replaced preferences move toward the new value. While the posts about new preferences are written (blocks 4–6), new − old rises from −0.048 to +0.007, so by block 6 the new statement is slightly more likely than the old one.

Reversed preferences are not tracked. After the change, new − old stays where it was (−0.094 after block 3, −0.099 after block 6). Writing the posts raises the scores of both "Enjoys X" and "Dislikes X" by about the same amount. This already holds in blocks 1–3: posts showing the original stance do not make that stance more likely than its opposite. The memory picks up that the user cares about X, but not whether the user likes or dislikes it.

Interpretation and scope. We hypothesize that these patterns reflect how ProTTT writes information into memory. Each write updates the memory through next-token prediction on the user's new text, which can reinforce unchanged preferences and help track replacements. For example, “basketball” and “baseball” compete as alternative continuations of “Likes to play", so updates on posts expressing the new preference can shift the model toward the new target. Reversals may be harder because learning to predict the new posts does not necessarily suppress the previously stored, opposing preference. Without an explicit mechanism for identifying and revising conflicting information in memory, the updates may leave both preferences accessible rather than reliably favoring the current one. This explanation is consistent with our observation that writing posts expressing the original preference raises the scores of both the original and opposite statements similarly in early blocks. We have added this experiment to Appendix A.10.1 of the revision and clarified that ProTTT is effective for unchanged and replaced preferences but limited in preference reversals, which would be interesting future work.


W2. The proposed meta-learning objective explicitly supervises user profile prediction, while the continual-memory analysis evaluates knowledge acquisition and retention primarily through the likelihood of those profiles. This alignment between training and evaluation makes it difficult to determine whether the learned memory captures generalizable user semantics beyond profile reconstruction.

A2.

Alignment between profile supervision and the continual-memory analysis

We thank the reviewer for this point. To address this, we conducted the continual-memory analysis on a different task, rather than in the profile format used during training.

Setup. We use HiCUPID (Mok et al., 2025), which is not used in training. We evaluate on 250 test users, none of whom appear in training. Each user has 25 dialogues, and each dialogue reveals one of the user's traits (e.g., dislikes Disney movies). Each dialogue comes with one question whose ideal answer depends on that trait (e.g., "What are some animated movies suitable for adults?"). Each question has two reference answers: a personalized answer that reflects the trait ("...avoiding Disney's offerings") and a general answer with similar content but no personalization.

For each user, we split the 25 dialogues into five blocks of five (A–E). We write blocks A → B → C → D → E into the memory in sequence, without resetting it in between. Only user turns are written (~1K tokens per block). After each write we ask every question with the question alone as the prompt, and we measure the margin: the mean token log-probability of the personalized answer minus that of the general answer. A larger margin means the model prefers the answer that reflects the user's trait over a generic answer with the same content. All numbers are averages over the 1,250 questions in each block (250 users × 5).

AuthorsOpenReview ↗

Official Response to Reviewer (Part 3 of 6)

A2 (continued).

Acquisition. At each write step, the margin of the block just written increases more than the average margin of the other four blocks, which removes any shift that a write causes on every question:

Write stepΔ margin, block just writtenΔ margin, other four blocks (mean)Difference
+B+0.050+0.030+0.020
+C+0.007−0.012+0.019
+D+0.028+0.004+0.024
+E+0.024+0.003+0.022
(Δ is the change from the previous state.)

Retention. As later blocks are written into the same memory, the margin on block A's questions under ProTTT stays at or above its level right after A was written. Under full-history ICL, where the later dialogues are appended to the prompt, block A's margin drops:

Change in block A's margin (vs. right after A)+B+C+D+E
ProTTT+0.031+0.015+0.019+0.025
Full-history ICL−0.074−0.084−0.081−0.098

These results show that newly written dialogues raise the model's preference for trait-consistent answers to new questions. They also show that this preference does not decay as more history is written. We have added this analysis to Appendix A.10.2 of the revision.

Reference

Jisoo Mok, Ik-hwan Kim, Sangkwon Park, and Sungroh Yoon. Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis. In Proceedings of ACL, 2025. arXiv:2506.01262.


W3. The experimental comparisons do not sufficiently establish the novelty and effectiveness of the proposed approach relative to closely related TTT-based methods. It lacks direct comparisons with TTT-E2E, PERK, and GradMem, which also investigate gradient-based memory construction.

A3.

Comparison with TTT-based methods (TTT-E2E, GradMem, PERK)

We thank the reviewer for this suggestion. We have added all three methods.

These methods share the idea of writing context into weights or memory through gradient steps. They differ in what the write is trained for.

  • GradMem and PERK meta-learn the write for one specific task using that task's answer loss, requiring a separate model trained on labeled data for each evaluation task. This design is fundamentally different from ProTTT, which is meta-trained once and applied zero-shot to all evaluation benchmarks without training on their task labels.
  • TTT-E2E meta-learns the write for long-context language modeling and pretrains the model from scratch.
  • Our model is meta-trained once with a task-agnostic objective and is evaluated zero-shot on every benchmark, without using the labels of any evaluation task.

The relevant question is therefore not only whether a method writes by gradient steps, but whether the written memory stores information specific to that user and carries over to tasks it was not trained on.

Setup. For a controlled comparison, all methods use the same frozen Qwen3-0.6B backbone, the same user histories, and the same evaluation protocol.

  • TTT-E2E: Pretraining from scratch would change the backbone, so we keep our entire pipeline and replace only the meta-objective with theirs (the next-token loss on the history along the inner trajectory). This isolates the effect of our objective.
  • GradMem and PERK: We use their official code and configurations. Neither method has a task-agnostic recipe, so we train both on CMV, one of our evaluation tasks, using its labeled training split (users disjoint from the evaluation users). CMV is thus in-domain for them, while it is zero-shot for our model.

Each score below is reported before → after writing the user's history, so that the effect of the memory itself is visible.

MethodTrained onCMV AUC ↑PMem T6 ↑PMem T7 ↑LaMP-3 MAE ↓LaMP-3 RMSE ↓
Base–.466.273.0882.102.60
TTT-E2Enext-token loss.539 → .568.309 → .218.246 → .0882.28 → 1.692.80 → 2.28
GradMemCMV labels.702 → .707 †.273 → .273.105 → .0881.86 → 1.122.31 → 1.44
PERKCMV labels.781 → .771 †.364 → .273.123 → .0881.61 → 0.692.21 → 1.28
Ourstask-agnostic.518 → .624.218 → .327.246 → .2981.35 → 0.771.55 → 1.11
† in-domain (trained on CMV labels)

Comments on the paper

3 comments
AuthorsOpenReview ↗

Review Summary (1/3): Summary of recognized strengths

We thank all the reviewers for their time and valuable feedback.

1. A meaningful and practical problem, and a promising approach to efficient personalization

  • wm3u noted that the paper "studies a meaningful and practical problem in personalized language models" and that its "parametric memory compresses extensive user histories into lightweight adapters, reducing the need to repeatedly include the history in the context."
  • gcan stated that the method "has practical value" and "enables the incorporation of user-specific knowledge while reducing the computational cost of parameter updates, offering a promising approach to efficient personalization."

2. Profile-supervised meta-learning for incrementally updatable, semantic user memory

  • wm3u found that our profile-supervised meta-learning "provides a useful way to guide history-based updates toward capturing semantic user information."
  • RoCa described the work as "a framework that combines profile-supervised meta-learning with test-time training to construct incrementally updatable user memory."

3. Effectiveness across multiple personalization benchmarks

  • wm3u noted that the experiments "demonstrate the overall effectiveness of the method across multiple personalization benchmarks" and that "PROTTT outperforms full-history ICL and the evaluated parametric baselines on the main benchmarks."
  • RoCa stated that the experiments "cover multiple personalization benchmarks, providing empirical evidence of the method's effectiveness across different tasks."
  • gcan found the experimental evaluation "reasonably comprehensive" and noted that "the key experimental designs are appropriate and provide empirical support for the main conclusions."

4. Useful analyses and clear presentation

  • RoCa noted that the paper "provides useful analyses of memory acquisition, retention, inference efficiency, and the trade-off between personalization and general capabilities."
  • gcan found that the "motivation is clear, and the paper is generally well-written, with a clear presentation of the proposed method."
AuthorsOpenReview ↗

Review Summary (2/3): Addressed concerns and additional experiments

We addressed all 14 weaknesses and questions raised by the reviewers. New experiments include 2 larger backbones, 5 baselines, 7 retrained ablation models, and 6 new analyses.

1. Comparison with TTT-based methods

  • Concern: RoCa (W3) noted that the paper lacked direct comparisons with closely related TTT-based methods (TTT-E2E, PERK, and GradMem).
  • What we did: We added all three on the same backbone and evaluation protocol. None of them forms a memory that helps across benchmarks. PERK and GradMem are trained on CMV labels, and their CMV scores do not change when the user's history is written. TTT-E2E, which differs from ProTTT only in the meta-objective, lowers PersonaMem accuracy after writing. ProTTT is trained without labels from any evaluation task and improves on all three benchmarks after writing. Gradient-based writing alone is therefore not sufficient, and our profile-supervised meta-learning is what makes the write effective.

2. Novelty relative to OPPU

  • Concern: gcan (W1) questioned the methodological novelty relative to OPPU, which also adapts a LoRA module for each user.
  • What we did: We clarified that LoRA is only where the memory is stored; our contribution is how the memory is written. While OPPU fits each user’s adapter with a specific target task, ProTTT is task-agnostic: what to store is meta-learned once from semantic user profiles, and a single model then writes any new user’s raw, unlabeled history and transfers zero-shot to every benchmark. We reorganized the Related Work around this difference (Table 1). The gains also do not come from LoRA itself. A per-user LoRA trained on the same raw history stays below ProTTT on all three benchmarks (Table 2). So does ProTTT’s own LoRA block with the same write steps when our meta-learning is removed (naive TTT, Table 2; supervised training, item 4) or its objective is replaced (TTT-E2E, item 1).

3. Stronger textual-profile baselines

  • Concern: wm3u (W1) asked us to include Guided Profile Generation (GPG) and ReCAP as stronger textual-profile baselines.
  • What we did: We added both. ProTTT outperforms them on CMV, PersonaMem T7, and LaMP-3, so its advantage holds against stronger textual-profile methods.

4. Contribution of meta-learning

  • Concern: wm3u (W2) noted that the "TTT (naive)" baseline removes both profile supervision and meta-training, and suggested a control that keeps profile supervision but removes meta-learning.
  • What we did: We ran this control: standard supervised training on the same history–profile data, followed by the same TTT. Without meta-learning, TTT on the user's history barely changes the model, whereas the same TTT improves ProTTT on every benchmark. Profile supervision alone is therefore not sufficient, which isolates the contribution of meta-learning.

5. History summary vs. high-level profile as supervision

  • Concern: wm3u (W3) asked whether the gains come from semantic abstraction or simply from additional text supervision.
  • What we did: We retrained ProTTT with history summaries in place of profiles, matching output length, query conditions, and training budget. The summary-supervised model falls below the main model on every benchmark, so the gains come from semantic abstraction.

6. Number of training users and profile quality

  • Concern: gcan (W4) requested systematic experiments on how the number of training users and the quality of their profiles affect the initialization.
  • What we did: We retrained ProTTT with 10%, 25%, 50%, and 75% of the training users, and with incorrect profiles in which each user is paired with another user's profile. The learned write generalizes only once enough users are available, and it requires profiles that are correct for the user; profile-like text alone is not sufficient.
AuthorsOpenReview ↗

Review Summary (3/3): Addressed concerns and additional experiments (continued)

7. Larger models

  • Concerns: RoCa (W5) and gcan (W3) noted that the experiments were limited to 0.6B and 1B models.
  • What we did: We trained ProTTT on Qwen3-4B and Qwen3-8B with the same data and objective. Writing the user’s history improves every metric at every scale. On smaller backbones, ProTTT is both more effective and more efficient than ICL. On larger backbones, it stays competitive with ICL at a much lower per-query cost, since ICL re-reads the entire history for every query. On CMV, where the user’s semantic knowledge must be inferred from the history rather than read from it, ProTTT remains the best-performing method at every scale. ProTTT is therefore not limited to small models.

8. Per-user memory cost

  • Concern: RoCa (W5) was concerned that keeping about 1.06M memory parameters per user hinders deployment at scale.
  • What we did: We compared the per-user state with other methods and measured GPU memory at query time. The per-user state of ProTTT is no larger than those of other per-user adapter methods (P2P, OPPU, per-user LoRA). Text-based methods must hold the history in GPU memory at every query, so full-history ICL needs several times more GPU memory per query than ProTTT. The memory is therefore compact in both storage and serving.

9. Preference changes within the same user

  • Concern: RoCa (W1) noted that our continual analysis combined different users and did not examine conflicting preferences, reversals, or forgetting within the same user.
  • What we did: We built a controlled stream of 100 users whose preferences stay unchanged, are replaced, or are reversed, and wrote it sequentially into one memory without resets. ProTTT retains unchanged preferences that are no longer mentioned and moves toward replaced preferences. It does not yet track reversals, and the revision states this as a limitation.

10. Continual acquisition and retention, evaluated on other task

  • Concern: RoCa (W2) noted that the meta-learning objective and the continual analysis both rely on profile likelihood, so it was unclear whether the memory captures user semantics beyond profile reconstruction.
  • What we did: We repeated the analysis on HiCUPID, a QA dataset that is not used in training, with unseen users and only the question in the prompt. Each newly written set of dialogues raises the model's preference for personalized answers to the corresponding questions. This preference is retained as more history is written, whereas it decays under full-history ICL. The memory therefore captures user information beyond profile reconstruction.

11. General capabilities

  • Concern: RoCa (W4) pointed to the MMLU drop on Llama-3.2-1B when the full memory readout is used.
  • What we did: We clarified that the readout scale α is a core component of ProTTT. It is built into the method to balance personalization against general capability, it is applied only at inference time, and it can be set per deployment without retraining, so ProTTT should not be assessed at full readout alone. We also extended the readout-scale sweep to Qwen3-4B and 8B. The drop on Llama-3.2-1B at full readout is an outlier: it is much smaller on every Qwen3 model and shrinks further as the model grows.

12. Personalized text generation

  • Concern: gcan (Q) asked why LaMP-4, LaMP-5, and LaMP-7 were not evaluated.
  • What we did: We ran all three tasks on the official test sets. ProTTT gives the lowest NLL of the user's actual output on all three tasks and the best ROUGE on LaMP-7, so the memory also captures what the user writes.

13. Case study

  • Concern: wm3u (Q) asked for a concrete case showing responses before and after memory construction, and a case with newly added history.
  • What we did: We added both case studies. After memory construction, the model applies a trait expressed in the user's history to new situations. When new history is added, the information in it becomes more likely under the model.

14. Reproducibility

  • Concern: gcan (W2) asked us to release the implementation and the constructed profile data.
  • What we did: We will release the full implementation, all experiment configurations, and the constructed profiles together with the prompts and scripts used to build them, upon acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.