- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a profile-supervised meta-learning framework, namely PROTTT, for constructing semantic user memory in lightweight adapters. It learns a shared initialization that enables user-specific memory to be constructed and incrementally updated through next-token prediction on user history. Experiments show improvements over full-history ICL and the evaluated parametric baselines on several personalization benchmarks, while reducing per-query inference costs.
Critical strengths
-
The paper studies a meaningful and practical problem in personalized language models. Its parametric memory compresses extensive user histories into lightweight adapters, reducing the need to repeatedly include the history in the context.
-
The proposed profile-supervised meta-learning provides a useful way to guide history-based updates toward capturing semantic user information.
-
The experiments demonstrate the overall effectiveness of the method across multiple personalization benchmarks. PROTTT outperforms full-history ICL and the evaluated parametric baselines on the main benchmarks.
Critical weaknesses
This paper could be strengthened by the following additional experiments:
-
Comparison with stronger textual-profile baselines. Guided Profile Generation (Zhang, 2024) and ReCAP (Park et al., 2026) should be considered for inclusion as baselines. Given the limitations of textual-profile approaches discussed in the Introduction, evaluating this category solely through generic PAG and PAG + Full History leaves the comparison somewhat incomplete, particularly since the paper already uses the CMV data and profile-generation prompts from ReCAP.
-
Comparison with standard supervised fine-tuning. It is suggested to conduct standard supervised training using the same history–profile data to train the shared adapter to predict profiles from histories, followed by the same TTT process, and then compare the results against PROTTT. The current "TTT (naive)" baseline removes both profile supervision and meta-training, making it difficult to isolate the incremental contribution of meta-learning.
-
A history-summary prediction objective. Using the same teacher model, it could be useful to compare the effectiveness of using a "history summary" versus a "high-level user profile" as the meta-learning supervision signal, while matching output lengths, query conditions, and training budgets as closely as possible. This would help determine whether the performance gains stem from semantic abstraction, rather than simply from the addition of extra textual supervision or differences in the richness of the supervision.
If these experiments cannot be included, the authors could explain why and discuss the implications for the scope of the conclusions.
Questions for authors
Could the authors provide a specific case showing a user's history and how the model's personalized responses change before and after memory construction? An example involving newly added history would also help readers understand how the memory incorporates new user information.
5 comments
Official Response to Reviewer (Part 2 of 5)
W2. Comparison with standard supervised fine-tuning. It is suggested to conduct standard supervised training using the same history–profile data to train the shared adapter to predict profiles from histories, followed by the same TTT process, and then compare the results against PROTTT. The current "TTT (naive)" baseline removes both profile supervision and meta-training, making it difficult to isolate the incremental contribution of meta-learning.
A2.
Comparison with standard supervised fine-tuning (profile supervision without meta-learning)
We thank the reviewer for this suggestion. Before reporting the results, we would like to confirm that we understood the request correctly.
Our understanding of the request. We read it as training the shared adapter on the same history–profile data with ordinary supervised learning, with no meta-learning, and then applying the same TTT process at test time. For each training user, we run TTT on the user's history from the current adapter weights, compute the profile-prediction loss at the resulting weights, and apply its gradient directly to those weights as a standard supervised update. The gradient is not propagated back through the TTT steps, and the weights are not reset, so the next user starts from the updated weights. ProTTT differs in exactly these two points: it propagates the profile loss back through the TTT steps to one shared initialization, to which the model returns for every user. Could the reviewer confirm that this is the comparison intended? If a different setup was meant, we would appreciate a short clarification.
Setup. Everything other than the training rule is identical to ProTTT. Evaluation is also identical: we run the same TTT on the test user's history from the final weights and then score the model. Each score is shown as before → after TTT on the test user's history.
| Method | Profile supervision | Meta-learning | CMV AUC ↑ | PMem T6 ↑ | PMem T7 ↑ | LaMP-3 MAE ↓ | LaMP-3 RMSE ↓ |
|---|---|---|---|---|---|---|---|
| Base | – | – | 0.466 | 0.273 | 0.088 | 2.10 | 2.60 |
| Supervised + TTT (suggested control) | ✓ | ✗ | 0.460 → 0.468 | 0.309 → 0.291 | 0.105 → 0.105 | 2.04 → 2.03 | 2.65 → 2.64 |
| ProTTT (ours) | ✓ | ✓ | 0.518 → 0.624 | 0.218 → 0.327 | 0.246 → 0.298 | 1.35 → 0.77 | 1.55 → 1.11 |
Without meta-learning, TTT on the user's history has no effect: the model learns nothing about the user, so no memory is formed. On every benchmark, the control's scores barely change with TTT and stay close to the base model, whereas the same TTT process improves ProTTT on every benchmark.
Profile supervision alone is therefore not sufficient. In the control, the profile loss only improves the current weights on the profile task, and nothing in the objective requires the TTT step itself to produce a useful change. In ProTTT, the profile loss is propagated through the TTT steps to the shared initialization, so the initialization is trained specifically so that TTT on a new user's history yields a model of that user. This is the incremental contribution of meta-learning that the reviewer asked us to isolate.
Official Response to Reviewer (Part 3 of 5)
W3. A history-summary prediction objective. Using the same teacher model, it could be useful to compare the effectiveness of using a "history summary" versus a "high-level user profile" as the meta-learning supervision signal, while matching output lengths, query conditions, and training budgets as closely as possible. This would help determine whether the performance gains stem from semantic abstraction, rather than simply from the addition of extra textual supervision or differences in the richness of the supervision. (If these experiments cannot be included, the authors could explain why and discuss the implications for the scope of the conclusions.)
A3.
History summary vs. high-level profile as supervision
Thank you for this suggestion. We have now run the suggested comparison. We replaced each profile used as the meta-learning supervision signal with a summary of the same history that records what the user said and asked, without generalizing into traits, preferences, or background. Output lengths are matched for every target (the same number of items and a similar number of tokens), and the query conditions, training users, and training budget are identical to the main model. For CMV, the summaries and the profiles are written by the same teacher (gpt-5-mini) from the same input. For AlpsBench, whose profiles are the dataset's annotated memories, the summaries are written by gpt-5-mini from the same conversations. Cells show before → after writing the user's history.
| Supervision | CMV AUC ↑ | PMem T6 ↑ | PMem T7 ↑ | LaMP-3 MAE ↓ | LaMP-3 RMSE ↓ |
|---|---|---|---|---|---|
| High-level profile (main) | 0.518 → 0.624 | 0.218 → 0.327 | 0.246 → 0.298 | 1.35 → 0.77 | 1.55 → 1.11 |
| History summary | 0.474 → 0.551 | 0.218 → 0.236 | 0.421 → 0.070 | 2.57 → 2.34 | 2.90 → 2.77 |
A detailed history summary is a much weaker supervision signal than a high-level profile, so the gains come from semantic abstraction rather than from extra or richer text supervision. After writing, the summary-supervised model is below the main model on every benchmark, and on PersonaMem T7, writing even lowers accuracy. The summary has the same length as the profile and records the same history in more detail, so neither the amount nor the richness of the supervision explains the gap. Abstracting the history into a high-level description of the user is therefore a more effective supervision signal for meta-learning than recording the history in detail.
We have added this comparison to Section 5.2 (Table 4, “History summary”) of the revision.
Official Response to Reviewer (Part 1 of 5)
We sincerely thank you for the constructive feedback and for recognizing that our parametric memory addresses a meaningful and practical problem by compressing extensive user histories into lightweight adapters, that profile-supervised meta-learning provides a useful way to guide history-based updates toward semantic user information, and that ProTTT outperforms the baselines across multiple personalization benchmarks. We apologize for our late response, as running all the suggested experiments took longer than expected. Below, we address your key points on stronger textual-profile baselines (GPG and ReCAP), comparison with standard supervised fine-tuning, and history-summary supervision, as well as your question on the case study.
A1.
Comparison with stronger textual-profile baselines
We thank the reviewer for this suggestion. We have added both methods as baselines.
All rows use Qwen3-0.6B as the task model and the same test splits as the main results. "+ full history" adds the same 100 records used by ICL. Bold marks the best value in each column.
ProTTT outperforms all textual-profile baselines on CMV, PersonaMem T7, and LaMP-3. GPG with full history is the strongest textual-profile baseline. The one exception is PersonaMem T6, where GPG's profile alone is best.
We have added these baselines and results to Appendix A.6 (Table 11) of the revision.
References
Jiarui Zhang. Guided Profile Generation Improves Personalization with Large Language Models. In Findings of EMNLP, 2024.
Sejun Park, Yoonah Park, Jongwon Lim, and Yohan Jo. Learning to Retrieve User History and Generate User Profiles for Personalized Persuasiveness Prediction. In Findings of ACL, 2026.