acceptodds
Under review as a conference paper at ICLR 2027

PersonaFactor: Preference Factors with Dynamic Selection for LLM Personalization

Abstract

Although users may express different personalization needs across tasks, these task-specific behaviors can reflect shared underlying preference mechanisms. Existing LLM personalization methods often focus on surface-level interactions or undifferentiated histories, overlooking the shared mechanisms underlying these behaviors. We introduce PersonaFactors, a framework that distills reusable and interpretable personalization factors from heterogeneous user interactions. We construct a library of thirteen PersonaFactors by extracting evidence-grounded rationales and clustering recurring patterns across tasks, and develop a dynamic factor-guided generation framework that selects factors relevant to each user's history and current request. Across four personalization benchmarks and four backbone models, Dynamic PersonaFactor achieves the best score on every reported metric. Representation analyses further show that PersonaFactors and user history induce partially overlapping low-dimensional representations, and that their joint use yields complementary shifts. Taken together, these findings suggest that factor-based and history-based personalization can mutually reinforce each other. Our code is available at https://anonymous.4open.science/r/PersonaFactor-2118.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large language model, LLM personalization, user preference modeling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PhXkOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper first mines 13 factors from large-scale data to represent fine-grained explanations of user preferences, and then dynamically injects a subset of these factors as additional context for personalized question answering. Experiments across both open-source and proprietary models show improvements compared with direct RAG methods.

Critical strengths

The proposed framework is evaluated across a range of both open-source and proprietary LLMs, as well as multiple personalization benchmarks.

Critical weaknesses
  1. Limited methodological contribution and heuristic mining process. The main contribution of this paper is injecting 13 static factor definitions into the prompt context for personalized question answering. However, the offline factor mining pipeline is heavily heuristic-driven and restricted to specific training datasets, without sufficient discussion on out-of-distribution generalization or why these specific 13 generic factors are representative enough to cover diverse user preferences.

  2. Insufficient comparison with prior personalization methods. While the introduction categorizes existing literature into model adaptation and context conditioning, the experimental evaluation completely omits model adaptation methods and even narrows context conditioning down to simple RAG variants, leaving out stronger published profile-generation or memory-construction baselines.

  3. Unjustified inference overhead, method inconsistency, and questionable necessity for strong models. The modest gains of the proposed dynamic selection mechanism hardly justify its complexity, specifically in three aspects:

(i) Unreported overhead on open-source models: The score-based selector requires multiple preliminary generations followed by leave-one-factor-out likelihood evaluations across all factors for every single query, introducing substantial inference latency and compute overhead that are not analyzed in the main paper.

(ii) Degradation on proprietary models: Because sequence log-likelihoods are inaccessible in black-box APIs, the method degenerates into a simple prompt-based selector for proprietary models, an inconsistency where the mathematical formulation in Section 3.2 is not actually used, and which the appendix ablation shows can even underperform vanilla RAG on open-source models.

(iii) Questionable necessity as the base policy scales: When the underlying LLM is sufficiently capable, it should already be able to attend to relevant preference cues directly from the retrieved user history, making it unclear whether injecting and selecting from a fixed checklist of 13 generic definitions remains necessary or meaningful at scale.

Questions for authors

Please refer to my weaknesses

8 comments
AuthorsOpenReview ↗

Q1: Methodological contribution and generality of PersonaFactor(Response to Weakness 1)

Dear Reviewer,

We thank you for your careful and critical assessment of our work. Your comments raise important questions about the methodological contribution, experimental comparisons, and practical implications of our approach. We respond to each concern point by point below.

Q1: Methodological contribution and generality of PersonaFactor(Response to Weakness 1)

A1: Thank you for raising this concern. PersonaFactor is built from a broad collection of training data covering heterogeneous personalization scenarios. Specifically, we use the LaMP and LongLaMP training splits together with CoPA and HiCUPID, totaling 66,018 instances across 14 task-specific subsets. These data cover diverse tasks and domains, including news headline generation, scholarly title generation, tweet paraphrasing, abstract generation, product review generation, topic writing, and personalized question answering. This broad coverage exposes the factor-mining process to different task objectives and user preference patterns, reducing its dependence on any single benchmark or domain. We then construct a reusable factor library through clustering and semantic aggregation, rather than manually defining a task-specific checklist. The resulting library is effective across LaMP, LongLaMP, LaMP-QA, and PersonalLLM. Notably, LaMP-QA and PersonalLLM are not used to construct the factor library, yet PersonaFactor remains effective on both benchmarks, supporting the transferability of the mined factors.

AuthorsOpenReview ↗

Q2: Clarification of baseline selection and comparisons(Response to Weakness 2)

Q2: Clarification of baseline selection and comparisons(Response to Weakness 2)

A2: Thank you for suggesting that we include stronger personalization baselines. Existing context-conditioning methods, such as MemoryBank [1] and A-MEM [2], primarily focus on accurately storing, organizing, and retrieving user-specific memories. PersonaFactor instead targets fine-grained aspects of user personality and preferences, including self-concept, cognitive consistency, and schema structure. The two lines of work therefore address complementary objectives. To provide a direct empirical comparison, we additionally evaluate MemoryBank and A-MEM under the same backbone models, user histories, prompts, and evaluation metrics. As reported in the table below, Dynamic PersonaFactor achieves higher scores than both memory-based baselines across all evaluated backbones and metrics. These results support the effectiveness of explicitly modeling fine-grained preference factors beyond factual user-memory retrieval.

ModelMethodLaMP R-1LaMP R-LLongLaMP R-1LongLaMP R-LLaMP-QAPersonalLLM
DeepSeek-V4-ProDirect0.29320.25800.21250.12420.4915-0.9013
DeepSeek-V4-ProMemoryBank0.33750.29960.29550.16550.54100.1433
DeepSeek-V4-ProA-MEM0.33570.30230.30390.17310.4865-0.3117
DeepSeek-V4-ProStatic PersonaFactor0.34950.30760.31410.15810.74180.4795
DeepSeek-V4-ProDynamic PersonaFactor0.35600.31400.36450.19350.75750.4915
Gemini-2.5-FlashDirect0.32860.29180.23050.13090.4354-0.4993
Gemini-2.5-FlashMemoryBank0.34390.32370.30110.15970.5516-0.4573
Gemini-2.5-FlashA-MEM0.34210.32770.31860.16010.4976-0.5388
Gemini-2.5-FlashStatic PersonaFactor0.41000.36960.34610.17890.59140.2822
Gemini-2.5-FlashDynamic PersonaFactor0.42250.37610.38880.20510.61410.3022
Qwen3-8BDirect0.31450.27850.21510.12870.3271-1.2800
Qwen3-8BMemoryBank0.31090.27640.25750.15360.2218-0.9754
Qwen3-8BA-MEM0.32650.29160.26440.15900.2116-0.9499
Qwen3-8BStatic PersonaFactor0.32760.28920.27190.14760.4328-0.1876
Qwen3-8BDynamic PersonaFactor0.33360.29560.33770.18590.4417-0.1722
Gemma-3-4BDirect0.25220.22110.20050.11940.3762-0.2998
Gemma-3-4BMemoryBank0.26200.22980.22860.13260.3922-1.1280
Gemma-3-4BA-MEM0.27920.24820.23270.13870.4194-0.7723
Gemma-3-4BStatic PersonaFactor0.27100.23530.23460.13390.4944-0.1977
Gemma-3-4BDynamic PersonaFactor0.28180.24900.29430.16510.5107-0.1836

References

[1] Zhong, W., Guo, L., Gao, Q., et al. MemoryBank: Enhancing Large Language Models with Long-Term Memory. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(17): 19724–19731.

[2] Xu, W., Liang, Z., Mei, K., et al. A-MEM: Agentic Memory for LLM Agents. Advances in Neural Information Processing Systems, 2026, 38: 17577–17604.

AuthorsOpenReview ↗

Q3: Response to concerns about the inference overhead of dynamic factor selection(Response to Weakness 3(i))

Q3: Response to concerns about the inference overhead of dynamic factor selection(Response to Weakness 3(i))

A3: Thank you for raising this important concern. We additionally measured the inference overhead of dynamic factor selection on DeepSeek-V4-Pro and Qwen3-8B, with the results reported below; we will include these measurements in the camera-ready version. As the reviewer correctly points out, dynamic selection introduces additional computational cost. However, in a session-based personalization setting, factor selection typically needs to be performed only once per session, allowing this cost to be amortized over subsequent interactions. We also provide Static PersonaFactor, which avoids the dynamic selection overhead while still delivering effective personalization. In future work, we will investigate more efficient approaches for constructing and selecting personalization factors. We appreciate the reviewer’s valuable suggestion.

Qwen3-8B

DatasetMethodInput tokensGenerated tokensAverage latency (s)
LaMPRAG603.1724.820.950
LaMPStatic1,278.1725.331.026
LaMPDynamic3,235.73154.486.417
LongLaMPRAG1,168.00160.875.789
LongLaMPStatic1,843.00175.736.369
LongLaMPDynamic5,003.33443.6318.609
LaMP-QARAG656.80108.123.167
LaMP-QAStatic1,282.80124.843.692
LaMP-QADynamic3,284.52352.4411.869
PersonalLLMRAG2,737.00487.6517.031
PersonalLLMStatic3,330.00355.7512.584
PersonalLLMDynamic9,499.05731.0526.350

DeepSeek-V4-Pro

DatasetMethodInput tokensGenerated tokensAverage latency (s)
LaMPRAG547.2825.522.880
LaMPStatic1,217.2825.003.025
LaMPDynamic1,566.93228.008.579
LongLaMPRAG1,316.42214.777.086
LongLaMPStatic1,986.42200.676.632
LongLaMPDynamic3,207.83504.5813.348
LaMP-QARAG753.10190.727.021
LaMP-QAStatic1,374.10207.057.260
LaMP-QADynamic2,017.57490.3714.184
PersonalLLMRAG2,698.35519.4012.409
PersonalLLMStatic3,286.35560.4514.130
PersonalLLMDynamic5,961.40738.8519.507
Reviewer_jtAqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PersonaFactor builds a library of thirteen personalization factors and selects a subset for each request. For models that expose likelihoods, selection measures how removing each factor changes a relevance judgment and a draft response. For API models, a prompt selects the factors directly. The paper evaluates four models on four benchmarks, with selector ablations and analyses of history size and model representations.

Critical strengths
  1. The approach is useful across models and tasks without task-specific model training. Dynamic improves over Static for all four backbones on LaMP, LaMP-QA, and PersonalLLM, where the released scripts use matching default inputs. These results support the method beyond LongLaMP, whose evaluation setup needs clarification.

  2. The selector has relevant experimental support. Table 3 compares against a factor-count-matched random selector and ablates the two selection signals while holding histories, prompts, and decoding fixed. These controls support the value of choosing factors for each request. Together with the broader results, they justify a modest positive recommendation.

Critical weaknesses
  1. The evaluation needs to separate factor benefits from input and prompt differences. On LongLaMP, the released baseline and Dynamic pipelines select histories differently. The baseline inputs may already be ranked, so this does not establish an unfair comparison; the actual histories and generation settings need confirmation. Factor-based prompts also add guidance beyond plain RAG, so that comparison alone does not isolate the factor library. Static and CoPA share much of this guidance and partly address the concern.

  2. The selector’s reliability and contribution need clearer evidence. Main tables omit uncertainty, and the selection rule and thresholds differ between the paper and code. The actual settings and how they were chosen independently of final evaluation need clarification; the release does not prove evaluation-set tuning. Prompted selection also usually underperforms Static on open models (Table A.13), while API results favor Dynamic. This may reflect backbone differences, but leaves its API contribution less clear. A matched random selector would strengthen that attribution.

Questions for authors
  1. Clarification: For the LongLaMP results in Table 2, did Static and Dynamic receive the same history items and use compatible prompt and decoding settings? Please provide the selected history IDs and effective configurations. If they differed, a matched comparison is needed to interpret the gains.

  2. Clarification: Which selection rule and thresholds were used for each result, and on which data were they chosen? How was that data kept separate from final evaluation? What varied across the three reported runs?

  3. Validation of the main results: Can you provide paired uncertainty for Dynamic versus Static and for the full versus relevance-only selector, using the existing predictions?

Confirmation of matched inputs, independently chosen parameters, and stable improvements would strengthen my acceptance recommendation.

13 comments
AuthorsOpenReview ↗

Q1: Clarification of the user-profile ordering and system-prompt design used in the experimental setup.(Response to Weakness 1 and Question 1)

Dear Reviewer,

We sincerely thank you for recognizing the value and broad applicability of our approach and for providing thoughtful and constructive feedback. We address each question and concern point by point below.

Q1: Clarification of the user-profile ordering and system-prompt design used in the experimental setup.(Response to Weakness 1 and Question 1)

A1: Thank you for your careful review. We use Qwen3-Embedding-0.6B to rank user-profile entries by relevance to the current query. All history-based methods receive the same selected history items in the same order; dynamic factor selection does not alter this history input. Furthermore, these methods share the same system-prompt instructions, differing only in the factor content provided. Therefore, the comparisons are controlled for history ordering and general prompt guidance.

AuthorsOpenReview ↗

Q2: Response to concerns about the selector’s reliability.(Response to Weakness 2 and Question 2)

Q2: Response to concerns about the selector’s reliability.(Response to Weakness 2 and Question 2)

A2:

Thank you for your careful review. We acknowledge that the original manuscript did not include an analysis of the selector parameters. To address this omission, we conduct a sensitivity analysis by varying the relative weights assigned to the relevance signal (α\alpha) and the generation-influence signal (β\beta), while keeping all other settings fixed. The results are shown below.

ModelMethodLaMP R-1LaMP R-LLongLaMP R-1LongLaMP R-LLaMP-QAPersonalLLM
Qwen3-8Bα=0.4,  β=0.6\alpha=0.4,\;\beta=0.60.32980.29210.33750.18790.4507-0.1781
Qwen3-8Bα=0.5,  β=0.5\alpha=0.5,\;\beta=0.50.33010.29290.33750.18780.4681-0.1939
Qwen3-8Bα=0.6,  β=0.4\alpha=0.6,\;\beta=0.40.33160.29390.33810.18850.4353-0.1673
Qwen3-8Bα=0.7,  β=0.3\alpha=0.7,\;\beta=0.30.33360.29560.33770.18590.4417-0.1722
Qwen3-8Bα=0.8,  β=0.2\alpha=0.8,\;\beta=0.20.33290.29500.33820.18810.4446-0.1809

The results remain stable across the tested parameter settings, with no single weighting consistently dominating all benchmarks. Our main experiments use the selection settings specified in Appendix B.10. These parameters apply to the score-based selector, whereas the prompt-based selector uses the prompt shown in Figure A9 without these thresholds. The general generation settings are provided in Appendix C.1. After the paper is formally accepted, we will publicly release the complete code and relevant implementation details.

Additionally, we construct the factor library using the training splits of LaMP and LongLaMP, CoPA, and HiCUPID. Final evaluation uses the validation splits of LaMP and LongLaMP, LaMP-QA, and PersonalLLM.

We also report the mean ± standard deviation across three runs below to make run-to-run variability explicit.

ModelMethodLaMP R-1LaMP R-LLongLaMP R-1LongLaMP R-LLaMP-QAPersonalLLM
DeepSeek-V4-ProDirect0.2932 ± 0.00420.2580 ± 0.00240.2125 ± 0.00250.1242 ± 0.00100.4915 ± 0.0144-0.9013 ± 0.0291
DeepSeek-V4-ProRAG0.3400 ± 0.00380.3004 ± 0.00360.3003 ± 0.00440.1538 ± 0.00220.6503 ± 0.01810.4049 ± 0.0168
DeepSeek-V4-ProRAG-SelfFactor0.3325 ± 0.00220.2904 ± 0.00300.2891 ± 0.00270.1561 ± 0.00160.5940 ± 0.0171-0.3861 ± 0.0133
DeepSeek-V4-ProRAG-CoPA0.3456 ± 0.00220.3062 ± 0.00370.3082 ± 0.00430.1565 ± 0.00250.7196 ± 0.02050.4642 ± 0.0140
DeepSeek-V4-ProStatic PersonaFactor0.3495 ± 0.00460.3076 ± 0.00340.3141 ± 0.00400.1581 ± 0.00170.7418 ± 0.02270.4795 ± 0.0149
DeepSeek-V4-ProDynamic PersonaFactor0.3560 ± 0.00400.3140 ± 0.00420.3645 ± 0.00240.1935 ± 0.00250.7575 ± 0.03190.4915 ± 0.0234
Gemini-2.5-FlashDirect0.3286 ± 0.00290.2918 ± 0.00300.2305 ± 0.00350.1309 ± 0.00200.4354 ± 0.0122-0.4993 ± 0.0142
Gemini-2.5-FlashRAG0.4010 ± 0.00330.3614 ± 0.00460.3121 ± 0.00420.1615 ± 0.00230.5748 ± 0.01980.2426 ± 0.0106
Gemini-2.5-FlashRAG-SelfFactor0.3809 ± 0.00370.3419 ± 0.00330.2689 ± 0.00340.1394 ± 0.00240.5178 ± 0.0154-0.6971 ± 0.0190
Gemini-2.5-FlashRAG-CoPA0.4027 ± 0.00370.3637 ± 0.00400.3395 ± 0.00420.1746 ± 0.00230.5810 ± 0.01900.2689 ± 0.0151
Gemini-2.5-FlashStatic PersonaFactor0.4100 ± 0.00450.3696 ± 0.00420.3461 ± 0.00370.1789 ± 0.00190.5914 ± 0.01490.2822 ± 0.0108
Gemini-2.5-FlashDynamic PersonaFactor0.4225 ± 0.00530.3761 ± 0.00360.3888 ± 0.00610.2051 ± 0.00160.6141 ± 0.01820.3022 ± 0.0117
Qwen3-8BDirect0.3145 ± 0.00450.2785 ± 0.00330.2151 ± 0.00340.1287 ± 0.00160.3271 ± 0.0076-1.2800 ± 0.0254
Qwen3-8BRAG0.3249 ± 0.00360.2877 ± 0.00260.2581 ± 0.00310.1437 ± 0.00220.4159 ± 0.0150-0.2778 ± 0.0124
Qwen3-8BRAG-SelfFactor0.3065 ± 0.00340.2674 ± 0.00300.3017 ± 0.00350.1594 ± 0.00250.3684 ± 0.0102-0.3638 ± 0.0144
Qwen3-8BRAG-CoPA0.3273 ± 0.00380.2899 ± 0.00320.2657 ± 0.00310.1457 ± 0.00200.4303 ± 0.0104-0.1901 ± 0.0103
Qwen3-8BStatic PersonaFactor0.3276 ± 0.00310.2892 ± 0.00250.2719 ± 0.00260.1476 ± 0.00150.4328 ± 0.0187-0.1876 ± 0.0093
Qwen3-8BDynamic PersonaFactor0.3336 ± 0.00270.2956 ± 0.00370.3377 ± 0.00370.1859 ± 0.00230.4417 ± 0.0142-0.1722 ± 0.0096
Gemma-3-4BDirect0.2522 ± 0.00240.2211 ± 0.00280.2005 ± 0.00330.1194 ± 0.00170.3762 ± 0.0098-0.2998 ± 0.0116
Gemma-3-4BRAG0.2706 ± 0.00350.2364 ± 0.00200.2319 ± 0.00210.1328 ± 0.00170.4450 ± 0.0127-0.2414 ± 0.0102
Gemma-3-4BRAG-SelfFactor0.2645 ± 0.00280.2333 ± 0.00200.2029 ± 0.00210.1156 ± 0.00150.3995 ± 0.0074-0.2562 ± 0.0112
Gemma-3-4BRAG-CoPA0.2695 ± 0.00360.2354 ± 0.00190.2280 ± 0.00270.1312 ± 0.00190.4981 ± 0.0148-0.2012 ± 0.0094
Gemma-3-4BStatic PersonaFactor0.2710 ± 0.00210.2353 ± 0.00180.2346 ± 0.00320.1339 ± 0.00140.4944 ± 0.0103-0.1977 ± 0.0088
Gemma-3-4BDynamic PersonaFactor0.2818 ± 0.00350.2490 ± 0.00270.2943 ± 0.00480.1651 ± 0.00200.5107 ± 0.0170-0.1836 ± 0.0092
Reviewer_V2FCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PersonaFactor, a framework that extracts a taxonomy of personalization factors from user interactions across task scenarios, and uses it to define a library of 13 standardized and interpretable factors. These factors describe the reasons why a response fits a particular user. The library is constructed by using an LLM to extract rationales from user histories and responses across datasets like LaMP, LongLaMP, and CoPA, clustering the rationales, and then naming and iteratively merging the clusters into canonical factors. The paper then introduces a method to dynamically select the factors relevant to each request, which are added to the context as natural-language guidance alongside the retrieved user history. For open-weight models, factors are selected with a leave-one-factor-out test that measures how much removing each factor lowers the likelihood of the model’s own relevance judgment and draft response; for black-box API models, a prompt-based selector is used instead. Across four benchmarks (LaMP, LongLaMP, LaMP-QA, and PersonalLLM) and four generation models, Dynamic PersonaFactor achieves the best score on every reported metric (ROUGE-1/L, LLM judge, or reward model depending on the benchmark), outperforming RAG, a CoPA-factor baseline, and static inclusion of the full factor set, with the largest gains on LongLaMP. An ablation shows that the combined selection signal outperforms single-signal and random selection. Lastly, the paper analyzes how user history and personalization factors shift the model’s internal representations, finding that they move along partially shared directions and that their combination produces a larger, complementary shift. The primary finding of the paper is that adding a structured set of natural-language personalization factors as context, in addition to RAG, can further improve personalization performance, with dynamic factor selection providing a smaller additional gain.

Critical strengths

Overall, the method presented in the paper is simple, works through prompting alone, and is evaluated broadly. The finding that structured personalization factors can improve personalization on top of RAG is a useful, practical result. Key strengths include:

  1. The evaluation is broad and the findings are consistent. Across evaluation paradigms (ROUGE, LLM-as-a-judge, and reward models) and backbones ranging from 4B open-weight models to frontier black-box APIs, Dynamic PersonaFactor consistently achieves the best scores. The comparison against RAG-CoPA is useful for understanding how the proposed factors compare against an existing factor set and against RAG-SelfFactor is useful for understanding how on-the-fly generation from context (as opposed to a structured set) compares.
  2. The structured dimensions are particularly useful when there is less history context. On LaMP-QA, the advantage of Dynamic PersonaFactor over RAG is largest when only a few history items are available (Figure 3 and Figure A11). This suggests the factors are particularly useful in cold-start settings, which often pose real challenges in practical applications.
  3. The rationale extraction, clustering, and factor naming/merging procedures are documented in detail (Appendix B) and an anonymized code link is provided, so the pipeline could be reapplied to new data or domains when a different factor set is needed.
Critical weaknesses
  1. For the 13 factor library that the paper contributes, there is limited evidence that this outperforms previous factor sets, like CoPA. For example, Static PersonaFactor and RAG-CoPA performance are similar (sometimes one is better and sometimes the other is better). This makes it hard to tell how much the large-scale extraction and clustering pipeline contributes. Maybe some additional baselines (e.g. a generic factor set written by an LLM) or analysis of how often each factor is selected can help understand this better.
  2. The paper is missing a cost-benefit analysis of dynamic selection. For some benchmarks, the gains are relatively small so this would be helpful in understanding the real usefulness in online settings. From my understanding, the score-based selector generates a relevance judgment and a full draft response, runs 26 additional likelihood passes, and then generates the final response. This means that there are multiple processing passes and twice the generated tokens, which likely leads to significant additional cost (both latency and compute) over RAG. Since Dynamic improves over Static by only about 0.01-0.02, and the draft response is generated with the full factor library (effectively the Static output), it would be useful to report latency and token cost per request against quality, and to compare the final response against the draft.
  3. The evaluation benchmarks selected may not fully cover all relevant dimensions. Specifically, irrelevant or conflicting preferences, which motivate dynamic selection, are not tested, and real human preference judgments are not used. Potentially consider including other relevant personalization benchmarks (like PrefEval or PersonaMem) that include things like distractor or outdated preferences. Also, why include CoPA in the factor construction instead of holding out a split, so that one could also evaluate on their benchmark for better comparison?
  4. The generation prompt for the API models (Figure A8) includes the instruction to “reuse that concrete pattern directly” when the history closely matches the request, which could by itself increase ROUGE on LaMP and LongLaMP. Since the baseline prompts are not shown, it is unclear whether all methods use the same prompt; if they do not, this could be a confounder in the comparison.
Questions for authors
  1. Suggestion: It would be interesting to explore what most influences the set of selected factors for the dynamic process, e.g. is it mostly related to the task (so consistent across multiple users and their history for the same query) or to the user (so it changes when the user history is swapped out for the same task).
  2. Clarification: Do all baselines use the same generation prompt as Dynamic PersonaFactor, including the "reuse that concrete pattern directly" instruction?
  3. Clarification: Could you also report the standard deviations across the three runs? This would help understand the significance of the relative performance differences.
  4. Clarification: When using LaMP for the factor construction pipeline, was the training split time-based or user-based?
  5. Clarification: In the main paper content, different methods are used for selection across open-weight and closed-weight models. In the Appendix Table A.13, it shows that the score-based selection outperforms the prompt-based selection for open-weight models, but more interestingly, the prompt-based selection is sometimes worse than Static PersonaFactor or even RAG. This is something that probably should go into the main paper content and have some additional analysis to better understand what is going on, especially since the closed-weight models use the prompt-based selector and still show large gains over Static.
8 comments
AuthorsOpenReview ↗

Q1: Experiment with an LLM-generated generic factor set(Response to Weakness 1)

Dear Reviewer,

We sincerely thank you for your careful and constructive evaluation of our work. Your detailed comments have helped us identify several aspects that require further clarification and analysis. We address each concern point by point below.

Q1: Experiment with an LLM-generated generic factor set(Response to Weakness 1)

A1:

Thank you for this valuable suggestion. Among the baselines in our original submission, RAG-SelfFactor dynamically generates factors from the current query and user context. It therefore provides a direct test of whether query-conditioned factor prompting alone is sufficient to explain the observed performance improvements.

Following your suggestion, we additionally construct an LLM-generated generic factor set by prompting an LLM to write a general-purpose set of personalization factors.

We compare this baseline with Static PersonaFactor and Dynamic PersonaFactor in the table below. The results show that, across the evaluated models and benchmarks, both PersonaFactor variants consistently outperform the LLM-generated generic factor baseline.

ModelMethodLaMP R-1LaMP R-LLongLaMP R-1LongLaMP R-LLaMP-QAPersonalLLM
DeepSeek-V4-ProLLM-generated generic factor0.33740.29030.29110.15230.61220.3165
DeepSeek-V4-ProRAG-SelfFactor0.33250.29040.28910.15610.5940-0.3861
DeepSeek-V4-ProStatic PersonaFactor0.34950.30760.31410.15810.74180.4795
DeepSeek-V4-ProDynamic PersonaFactor0.35600.31400.36450.19350.75750.4915
Gemini-2.5-FlashLLM-generated generic factor0.39020.35270.27720.14010.5003-0.5822
Gemini-2.5-FlashRAG-SelfFactor0.38090.34190.26890.13940.5178-0.6971
Gemini-2.5-FlashStatic PersonaFactor0.41000.36960.34610.17890.59140.2822
Gemini-2.5-FlashDynamic PersonaFactor0.42250.37610.38880.20510.61410.3022
Qwen3-8BLLM-generated generic factor0.31590.27820.26420.13660.2486-0.3291
Qwen3-8BRAG-SelfFactor0.30650.26740.30170.15940.3684-0.3638
Qwen3-8BStatic PersonaFactor0.32760.28920.27190.14760.4328-0.1876
Qwen3-8BDynamic PersonaFactor0.33360.29560.33770.18590.4417-0.1722
Gemma-3-4BLLM-generated generic factor0.25070.21560.19450.11410.2271-0.6013
Gemma-3-4BRAG-SelfFactor0.26450.23330.20290.11560.3995-0.2562
Gemma-3-4BStatic PersonaFactor0.27100.23530.23460.13390.4944-0.1977
Gemma-3-4BDynamic PersonaFactor0.28180.24900.29430.16510.5107-0.1836
AuthorsOpenReview ↗

Q2: Contribution and selection frequency of the proposed 13-factor library(Response to Weakness 1 and Question 1)

Q2: Contribution and selection frequency of the proposed 13-factor library(Response to Weakness 1 and Question 1)

A2:

Thank you for raising this important question. The CoPA factors are covered by our work, while PersonaFactor further introduces finer-grained factors for more comprehensive coverage of heterogeneous personalization scenarios. This explains why RAG-CoPA is competitive on LaMP-QA but less effective on LaMP and LongLaMP, where the broader library better captures task-specific preference patterns.

The following analysis reports the six most frequently selected factors for each model and benchmark. The variation across settings shows that the selector adapts the selected factors to the benchmark and model rather than applying a fixed subset universally.

ModelDatasetSix most frequently selected factors (frequency)
Qwen3-8BLaMPSituational Anchoring (57.4%); Schema Congruence (41.4%); Goal Congruence (39.9%); Cognitive Trust (32.5%); Expressive Style Consistency (20.4%); Behavioral Pattern Consistency (16.5%)
Qwen3-8BLongLaMPSituational Anchoring (60.4%); Goal Congruence (29.0%); Cognitive Trust (28.4%); Schema Congruence (28.1%); Self-Concept Congruence (18.7%); Expressive Style Consistency (17.8%)
Qwen3-8BLaMP-QASituational Anchoring (77.1%); Goal Congruence (71.6%); Cognitive Trust (69.8%); Schema Congruence (65.9%); Behavioral Pattern Consistency (62.9%); Self-Concept Congruence (59.5%)
Qwen3-8BPersonalLLMSituational Anchoring (45.6%); Perceived Autonomy Control (29.2%); Schema Congruence (25.2%); Cognitive Trust (22.2%); Goal Congruence (21.2%); Constraint Consistency (19.7%)
Gemma-3-4BLaMPCognitive Trust (69.4%); Cognitive Consistency (36.2%); Goal Congruence (35.1%); Self-Concept Congruence (27.0%); Behavioral Pattern Consistency (24.5%); Situational Anchoring (21.7%)
Gemma-3-4BLongLaMPCognitive Trust (66.7%); Behavioral Pattern Consistency (42.1%); Goal Congruence (29.5%); Schema Congruence (22.8%); Self-Concept Congruence (21.5%); Cognitive Consistency (17.0%)
Gemma-3-4BLaMP-QACognitive Trust (76.4%); Schema Congruence (64.6%); Cognitive Consistency (62.9%); Behavioral Pattern Consistency (58.9%); Constraint Consistency (57.3%); Self-Concept Congruence (56.5%)
Gemma-3-4BPersonalLLMCognitive Trust (63.2%); Schema Congruence (33.3%); Self-Concept Congruence (32.5%); Behavioral Pattern Consistency (28.5%); Cognitive Load (25.9%); Situational Anchoring (18.9%)
AuthorsOpenReview ↗

Q3: Cost–benefit analysis and inference overhead of dynamic factor selection(Response to Weakness 2)

Q3: Cost–benefit analysis and inference overhead of dynamic factor selection(Response to Weakness 2)

A3: Thank you for suggesting a cost–benefit analysis of dynamic selection. We agree that quality scores alone are insufficient to assess its practical value in online settings, particularly when the gains are modest. We therefore additionally report input tokens, generated tokens, and average inference latency for RAG, Static PersonaFactor, and Dynamic PersonaFactor on DeepSeek-V4-Pro and Qwen3-8B below. We will include these measurements in the camera-ready version.

The results confirm that dynamic selection incurs additional overhead, while its benefits vary across tasks. Dynamic selection yields more substantial quality improvements on LongLaMP. For settings with smaller gains or stricter latency requirements, Static PersonaFactor offers a lower-cost alternative that still provides effective personalization. We therefore present Static and Dynamic as two options suited to different quality and efficiency requirements.

In addition, when the task context and relevant preferences remain stable within a session, reusing previously selected factors could amortize the selection cost over multiple interactions. In future work, we will further investigate factor reuse and more efficient selection mechanisms.

Regarding your suggested direct comparison between the final response and the intermediate draft, the existing Static-versus-Dynamic comparison can be viewed as capturing the difference between the two.

Qwen3-8B

DatasetMethodInput tokensGenerated tokensAverage latency (s)
LaMPRAG603.1724.820.950
LaMPStatic1,278.1725.331.026
LaMPDynamic3,235.73154.486.417
LongLaMPRAG1,168.00160.875.789
LongLaMPStatic1,843.00175.736.369
LongLaMPDynamic5,003.33443.6318.609
LaMP-QARAG656.80108.123.167
LaMP-QAStatic1,282.80124.843.692
LaMP-QADynamic3,284.52352.4411.869
PersonalLLMRAG2,737.00487.6517.031
PersonalLLMStatic3,330.00355.7512.584
PersonalLLMDynamic9,499.05731.0526.350

DeepSeek-V4-Pro

DatasetMethodInput tokensGenerated tokensAverage latency (s)
LaMPRAG547.2825.522.880
LaMPStatic1,217.2825.003.025
LaMPDynamic1,566.93228.008.579
LongLaMPRAG1,316.42214.777.086
LongLaMPStatic1,986.42200.676.632
LongLaMPDynamic3,207.83504.5813.348
LaMP-QARAG753.10190.727.021
LaMP-QAStatic1,374.10207.057.260
LaMP-QADynamic2,017.57490.3714.184
PersonalLLMRAG2,698.35519.4012.409
PersonalLLMStatic3,286.35560.4514.130
PersonalLLMDynamic5,961.40738.8519.507

Comments on the paper

1 comment
AuthorsOpenReview ↗

Sincere Appreciation for Your Time and Thoughtful Consideration

Dear Area Chair,

Thank you for reviewing our work. Our core contribution is PersonaFactor, an explicit factor-based framework for personalized generation. It distills reusable and interpretable personalization factors from heterogeneous user interactions and dynamically selects the factors relevant to the current task and user history. Representation analyses show that user history and PersonaFactors induce partially overlapping low-dimensional shifts in the model’s internal representation space, while their combination produces additional shifts beyond either source alone.

In the rebuttal, we added evaluation on the PersonaMem benchmark and introduced two memory-based baselines, MemoryBank and A-MEM. PersonaFactor achieves the best results on the new benchmark and generally outperforms both memory-based methods. Furthermore, both Static and Dynamic PersonaFactor continue to provide additional gains on the stronger GPT-6.1-sol model, further supporting the effectiveness and generality of our approach.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.