acceptodds
Under review as a conference paper at ICLR 2027

Bayesian User-State Inference for Online LLM Personalization

Abstract

Personalizing large language models (LLMs) online requires inferring user preferences from sparse histories and refining these estimates as new interactions arrive. Existing approaches typically select or summarize histories without explicitly modeling uncertainty about preferences. Yet observed user behavior reflects both context and personal preferences, making limited evidence ambiguous and complicating how much to revise estimates. To address this ambiguity and refine preference estimates as evidence accumulates, we propose Bayesian User-State Inference (BayUSI), a framework that treats deviations from input-conditioned population predictions as personal evidence and combines them with shared preference patterns. A learned mixture prior captures recurring preferences across users without annotated groups, while Bayesian updates incorporate each user’s history into a posterior over their preferences. As interactions accumulate, posterior uncertainty governs how strongly new evidence updates preference estimates, providing query-specific conditioning for a frozen LLM. Training through the LLM’s prediction loss aligns preference inference with personalized generation without per-user fine-tuning. Experiments on four LaMP tasks show consistent improvements under sparse-history and evolving-history protocols, demonstrating the value of uncertainty-aware preference inference for online LLM personalization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
large language models, online personalization, Bayesian inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xPHUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This work aims to improve the performance of a frozen LLM in classification, rating and title-generation tasks. Authors propose BayUSI, which extracts a user's preference information from the their history data and incorporate it into LLM input. The user preference information is recorded by a weighted sum ∑i=1Kγu,t,kMu,t,k⊤fθ(xu,t)\sum_{i=1}^K\gamma_{u, t, k}\mathbf{M}_{u, t, k}^{\top}\mathbf{f}_{\theta}(x_{u, t}), whose weights Mu,t,k⊤\mathbf{M}_{u, t, k}^{\top} and γu,t,k\gamma_{u, t, k} are updated using Bayesian inference by collecting the user's sequence of history input.

Critical strengths
  1. The clarity of this work is good. Math expressions are correct and match authors' framework in Figure 1. Authors provide detailed explanations on how their math expressions represent and manipulate user preference.

  2. Authors provide detailed empirical study on a variety of tasks, datasets and benchmarks. They also provide ablation study showing the effectiveness of the individual components in their framework.

  3. Personalization has been a core problem in recommender systems, since a recommender model makes personalized recommendation by identifying the personal interests of different users. Authors' work on LLMs could be a new solution in this area.

Critical weaknesses
  1. The mathematical intuition behind authors' framework is unclear.
  • Why should the user preference be learned by a weighted sum ∑i=1Kγu,t,kMu,t,k⊤fθ(xu,t)\sum_{i=1}^K\gamma_{u, t, k}\mathbf{M}_{u, t, k}^{\top}\mathbf{f}_{\theta}(x_{u, t}) with Bayesian update on weights? What are the benefits? Can it be learned instead by using a shallow neural network with backpropagation update on weights?

  • Why can ∑i=1Kγu,t,kMu,t,k⊤fθ(xu,t)\sum_{i=1}^K\gamma_{u, t, k}\mathbf{M}_{u, t, k}^{\top}\mathbf{f}_{\theta}(x_{u, t}) extract and store a user's preference information from their sparse history and evolving history? What is the mechanism behind it?

  • Suppose you have learned the weights (Mu,t,k,γu,t,k)k=1K(\mathbf{M}_{u, t, k}, \gamma_{u, t, k})_{k=1}^K. How do you interpret them? For example, by observing their coefficients, can you tell whether this user likes something or dislikes something?

  1. Limited scope in the study of LLMs. Classification, rating and title-generation are simple tasks and are usually handled using traditional machine learning models. While LLMs can also handle these tasks, they are more frequently used in complex tasks such as reasoning, math and coding, and evaluated using complex benchmarks such as LiveBench (White et al., 2025). It is unclear how authors' study on user preference can be adapted to these complex tasks.

Minor:

In abstract, "As interactions accumulate, posterior uncertainty governs how strongly new evidence updates preference estimates", what does the "uncertainty" mean here? The uncertainty of a distribution typically means its entropy in information theory. I think authors intent to mean "the posterior captures user preference".

Reference:

Colin White, et al. LiveBench: A challenging, contamination-limited LLM benchmark. ICLR, 2025.

Questions for authors
  1. (Critical) Can authors show an example of the LLM input in Figure 1, and explain how it reflects the user's preference?

  2. In Line 135, Wu∈Rd×p\mathbf{W}_u \in \mathbb{R}^{d\times p}, what do dd and pp mean? And how do you determine their values?

4 comments
AuthorsOpenReview ↗

Response to Reviewer xPHU

Thank you for recognizing the clear formulation and detailed empirical study. In response to your comments, we provide a worked LLM-input example and explicit explanations of the inference steps, and add comparisons with neural and SGD alternatives, a CMV evaluation, and dimension-sensitivity results.

Q1. A concrete LLM input and how it reflects user preferences

Following your request for a concrete LLM-input example, we provide a LaMP-3 walkthrough from a historical rating to residual computation, user-state inference, and prefix conditioning. The LLM receives the current text prompt plus continuous key–value prefix vectors at every transformer layer. The reviews, population prediction, and example state values are illustrative.

  1. Observe an input-response pair (review-user’s rating).

    Suppose the user assigned 5 stars to:

    “Average quality, but it does the job.”

    This review–rating pair becomes part of the observed history.

  2. Compute the user’s response deviation from a population prediction.

    Ratings are normalized by (Eq. 17):

    a(y)=y−32.a(y)=\frac{y-3}{2}.

    A separate population predictor, fitted on training users and then frozen, predicts this normalized rating from the review alone. Suppose it predicts −0.25, equivalent to 2.5 stars. The observed residual is:

    r=a(5)−f0(x)=1−(−0.25)=1.25.r=a(5)-f_0(x)=1-(-0.25)=1.25.

    This positive residual means the user rated that review more favorably than the population prediction.

  3. Infer the relationship between review features and response deviations.

    BayUSI pairs historical review features with their residuals to infer coefficients that map input features to expected response deviations (Eq. 4; lines 168–182):

    E[r∣x,Wu]=Wu⊤fθ(x).\mathbb E[r\mid x,W_u]=W_u^\top f_\theta(x).

    Bayesian conditioning estimates these coefficients and their uncertainty; their posterior means are the matrices in Eq. 5. For example, repeated positive residuals for mixed reviews can support a more favorable rating tendency for similar reviews.

  4. Read out the expected deviation for the current review.

    An illustrative current text prompt is:

    What is the score of the following review on a scale of 1 to 5? just answer with 1, 2, 3, 4, or 5 without further explanation. review: The quality is average, but it works as expected.

    Evaluating the inferred relationship at this review gives the query-specific user state in Eq. 10. For LaMP-3, this is a scalar expected rating deviation. For illustration, a state of +0.5 would represent a one-star upward deviation from the population prediction under the residual model. Only historical interactions inform this estimate; the current rating is unknown.

  5. Convert the state into the actual LLM conditioning.

    A learned mapping converts the state into key–value prefix vectors. At each transformer layer, these are added to a separate history prefix summarizing query-relevant history. The combined vectors enter the layer’s attention as additional keys and values, and the frozen LLM generates the rating (Eqs. (11)–(12)).

    The same text prompt can therefore receive different prefixes for users with different histories. The shared feature map, prior, and conditioning mappings are trained through the LLM's prediction loss; deployment adaptation uses Bayesian conditioning without per-user fine-tuning.

AuthorsOpenReview ↗

Response to Reviewer xPHU

Q2. Why use the Bayesian posterior-weighted state?

To clarify the mathematical intuition, we express the weighted state as a posterior expectation and explain how it supports adaptation from sparse and growing histories. BayUSI models a user’s preferences through coefficients that relate input features to response deviations from the population prediction. Limited history leaves these coefficients uncertain, so we average the predicted deviation over their posterior:

=\sum_{k=1}^{K}\gamma_{u,t,k}M_{u,t,k}^{\top}f_\theta(x_{u,t}).$$ Each coefficient mean supplies a user-adapted prediction, weighted by its component's posterior probability (Eq. (10)). This averages plausible patterns when the evidence does not clearly distinguish them. The Bayesian construction supports this estimate in three ways: 1. **A shared starting point for sparse histories.** The learned prior regularizes the coefficients toward recurring patterns among training users, allowing estimation from limited personal evidence. It captures both typical coefficients and variation around them, without requiring annotated user groups (Eq. 2; lines 143–157). 2. **Progressive adaptation as history grows.** Within each component, new observations revise the coefficients according to the prediction discrepancy and remaining coefficient uncertainty (Eq. 8). 3. **Evidence-based weighting of shared patterns.** Component probabilities change according to how well each pattern explains the observed history, integrating over coefficient uncertainty (Eqs. 6–7). Thus, personal evidence refines both **the prediction within each pattern** and **the relative support for different patterns**, producing an expected response deviation tailored to the current query. ## **Q3. Benefits of BayUSI over a shallow neural network** **To evaluate the neural alternative you raise, we compare BayUSI with a shallow MLP state predictor and gradient-based coefficient updates, highlighting both the empirical benefits and theoretical guarantees of Bayesian user-state inference.** The shallow MLP maps the observed history and current query directly to a user state, using the same LLM conditioning interface: **Evolving-history** | Dataset | Metric | MLP | BayUSI | | --- | --- | --- | --- | | LaMP-2 | Accuracy ↑ | 0.641 | **0.732** | | LaMP-2 | Macro-F1 ↑ | 0.634 | **0.722** | | LaMP-4 | ROUGE-1 ↑ | 0.205 | **0.217** | | LaMP-4 | ROUGE-L ↑ | 0.184 | **0.197** | **Sparse-history** | Dataset | Metric | MLP | BayUSI | | --- | --- | --- | --- | | LaMP-2 | Accuracy ↑ | 0.674 | **0.696** | | LaMP-2 | Macro-F1 ↑ | 0.618 | **0.656** | | LaMP-4 | ROUGE-1 ↑ | 0.203 | **0.206** | | LaMP-4 | ROUGE-L ↑ | 0.184 | **0.190** | **BayUSI improves every reported metric in both settings**, supporting the practical benefit of explicit posterior inference over the evaluated neural mapping. We also compare online coefficient updates against validation-tuned SGD over three seeds: **Evolving-history** | Dataset | Metric | SGD | BayUSI (K=1) | | --- | --- | --- | --- | | LaMP-2 | Accuracy ↑ | 0.692 | **0.741** | | LaMP-2 | Macro-F1 ↑ | 0.668 | **0.708** | | LaMP-4 | ROUGE-1 ↑ | 0.210 | **0.211** | | LaMP-4 | ROUGE-L ↑ | 0.190 | **0.191** | **Sparse-history** | Dataset | Metric | SGD | BayUSI (K=1) | | --- | --- | --- | --- | | LaMP-2 | Accuracy ↑ | 0.688 | **0.702** | | LaMP-2 | Macro-F1 ↑ | 0.636 | **0.654** | | LaMP-4 | ROUGE-1 ↑ | 0.205 | **0.207** | | LaMP-4 | ROUGE-L ↑ | 0.187 | **0.188** | With a single component, BayUSI achieves higher mean scores in all eight SGD comparisons, including a 4.9-percentage-point gain in evolving-history LaMP-2 Accuracy. **Beyond these empirical gains, BayUSI provides exact closed-form posterior updates, with adaptation strength governed by posterior uncertainty (Eq. 8), and guarantees equivalence between sequential updates and batch conditioning under fixed shared parameters (Appendix C.3).** **Together, these results support the benefits of principled Bayesian user-state inference over the evaluated neural and gradient-based alternatives.**
AuthorsOpenReview ↗

Response to Reviewer xPHU

Q4. How is preference information extracted and retained?

We trace how preference information is extracted and retained through accumulated statistics, posterior updates, and query-specific readout.

The posterior retains preference information; Eq. (10) reads it out for the current query. Each observed input–response pair contributes its input features and response residual to the accumulated statistics in Eq. (5):

\qquad \sum_{i<t} f_{u,i}f_{u,i}^{\top}.$$ The first associates inputs with response deviations; the second records the feature directions covered by the observed history. With the learned prior, these statistics determine the coefficient means and covariance (lines 186–199). Component probabilities update according to how well each component explains the residuals (Eqs. (6)–(7)). Eq. 10 then evaluates the inferred feature-to-deviation relationship at the current query: **one stored posterior can produce different adjustments for different queries**, and new observations refine it. ## **Q5. How should the coefficients and component weights be interpreted?** **To make the learned quantities interpretable, we distinguish coefficient means from component probabilities and illustrate their combined meaning through rating deviations.** The quantities have distinct meanings: - **Coefficient means:** the estimated mapping from input features to the user's response deviation. The intercept captures an input-independent tendency; the other coefficients capture variation with the input. - **Component probabilities (Eq. 7):** the history's relative support for each shared preference pattern. A larger probability indicates stronger support for that pattern, rather than stronger liking. Learned feature coordinates and components have no predefined semantic labels. Interpretation therefore comes from their combined prediction for a given input. In LaMP-3, a positive state means an expected rating above the population prediction for that review; a negative state means an expected rating below it. A state of +0.5 corresponds to a one-star upward deviation under the residual model (Eq. (17)). This describes **contextual, relative preferences**; a negative deviation alone does not imply absolute dislike. ## **Q6. Extension to more complex tasks: reasoning, mathematics, coding, and LiveBench.** **To investigate how user preference inference extends to reasoning and technical question answering, including mathematics and programming, we evaluate BayUSI on two complementary benchmarks: CMV personalized argument ranking [1] and CoPA personalized answer generation [2].** These benchmarks provide user histories and preference-sensitive evaluation targets, allowing us to assess how inferred preferences guide response selection and generation. This complements correctness-oriented benchmarks such as LiveBench, where correctness alone does not isolate adaptation to individual users. **CMV evaluates reasoning about individual users’ responses.** Given a user’s observed history and current post, the model assesses competing arguments and ranks replies by their likelihood of persuading that individual. |Method|MRR ↑|MRR@3 ↑|Hit@1 ↑|Hit@3 ↑|DCG@3 ↑| |---|---|---|---|---|---| |PPlug|41.13|32.24|18.65|50.53|36.92| |PPlug+kNN|39.87|30.88|16.02|51.35|36.11| |BayUSI (K=1)|42.62|34.84|18.65|**57.44**|**40.60**| |BayUSI (K=4)|42.46|34.36|19.62|54.36|39.47| |BayUSI (K=8)|**42.93**|**35.05**|**21.05**|54.21|39.94| **BayUSI variants achieve the best results across all five metrics**. With K=1, Hit@3 improves by **6.09 percentage points** and DCG@3 by **10.0%** over the strongest baseline for each metric. **CoPA evaluates open-ended personalized question answering across domains, including mathematics and programming.** Constructed from Stack Exchange, it evaluates whether generated answers align with individual users’ preferences. BayUSI infers a user state from historical interactions and uses it to condition the LLM’s answer to a new question. We assess generated responses using CoPA’s six-factor preference-alignment rubric with GPT-4o as the judge. |Method|Preference alignment ↑ (GPT-4o judge)| |---|---| |PPlug|0.053| |PPlug+kNN|0.056| |BayUSI (K=1)|0.055| |BayUSI (K=4)|**0.072**| |BayUSI (K=8)|0.067| BayUSI (K=4) achieves a **29.4% higher preference-alignment score** than the strongest baseline, PPlug+kNN. Together with CMV, these results address applicability beyond LaMP: **inferred user states guide reasoning about which competing arguments are most likely to persuade each user and personalized answer generation in domains including mathematics and programming.** [1] Zhang, X. F., Beauchamp, N., and Wang, L. (2025). PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought Process. EMNLP. [2] Su, H., et al. (2026). CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors. Findings of ACL.
AuthorsOpenReview ↗

Response to Reviewer xPHU

Q7. What is posterior uncertainty, and how does it govern updating?

The posterior represents both an estimated preference relationship and uncertainty about that estimate. Within a component, the coefficient mean gives the estimate, while the covariance measures how uncertain that relationship remains :

\qquad v=f^\top Sf.$$ This directional variance measures uncertainty in the **expected deviation at the observed input**. Identical preference estimates can have different uncertainty when supported by different histories. Entropy is one scalar summary of uncertainty. For a Gaussian component, it depends on the covariance determinant. BayUSI uses covariance directly because updating requires information about *which feature directions* remain uncertain. This has an explicit computational effect. Eq. 8 and Proposition 1 show that, within a component, the prediction at the newly observed input changes by $$(M^{+}-M)^\top f = \frac{v}{\sigma^2+v}(r-M^\top f)$$ Here, **v/(σ²+v)** is the uncertainty-dependent update weight, and **r − Mᵀf** is the prediction discrepancy. The same discrepancy produces a larger correction when uncertainty is greater relative to observation noise. **Uncertainty therefore governs the inference update; the LLM receives the deterministic posterior-mean readout**, rather than the covariance itself (Eq. (10)). ## **Q8. What do** $d$ **and** $p$ **represent in** $W_u\in\mathbb{R}^{d\times p}$**, and how are their values determined?** **To address your question about the dimensions, we explain their definitions and selection rationale, and add a sensitivity analysis on LaMP-4.** In the coefficient matrix, the input dimension includes the learned features and constant intercept; the output dimension is that of the response representation and residual: $$W_u\in\mathbb R^{d\times p},\quad f_\theta(x)\in\mathbb R^d,\quad a(y),r,W_u^\top f_\theta(x)\in\mathbb R^p.$$ These dimensions are fixed across users. We use eight input features: seven learned coordinates plus one intercept. The response dimensions are: | Task | $p$ | Response representation | | --- | --- | --- | | LaMP-2 | 14 | Orthonormal contrasts for 15 categorical labels | | LaMP-3 | 1 | Rating normalized to ([-1,1]) | | LaMP-4 and LaMP-5 | 8 | Frozen response embeddings projected onto eight whitened PCA components | The classification and rating dimensions follow their response representations. The input dimension and generation PCA rank are capacity hyperparameters; PCA uses training responses only (Appendix A). We vary the input and response dimensions independently over 4, 8, and 32 on LaMP-4, holding the other at eight: | Input dimension $d$ | Response dimensions $p$ | Evolving ROUGE-1 ↑ | Evolving ROUGE-L ↑ | Sparse ROUGE-1 ↑ | Sparse ROUGE-L ↑ | | --- | --- | --- | --- | --- | --- | | 8 | 8 | **0.217** | **0.197** | 0.206 | **0.190** | | 4 | 8 | 0.213 | 0.192 | **0.207** | 0.188 | | 32 | 8 | 0.214 | 0.191 | 0.205 | 0.187 | | 8 | 4 | 0.211 | 0.191 | 0.205 | 0.187 | | 8 | 32 | 0.206 | 0.185 | 0.205 | 0.186 | The default achieves the highest scores on three of four metrics; increasing either dimension to 32 does not improve it in this experiment. We will clarify the dimensions at first use and include these sensitivity results in the revision.
Reviewer_7SdrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Bayesian User-State Inference for online LLM personalization, addressing the fact that existing retrieval- and plug-in-based methods select or summarize histories without modeling uncertainty in preference estimates. BayUSI has three parts: a learned mixture prior over recurring preference patterns, evidence construction that treats each response's deviation from an input-conditioned population prediction as personal evidence, and closed-form Bayesian updates that yield a posterior over preference coefficients and component membership. The posterior predictive mean conditions a frozen Llama-3.1-8B-Instruct through lightweight prefixes, with the trainable personalization modules trained through the LLM's response prediction loss without per-user fine-tuning. Experiments on four LaMP tasks under evolving-history and sparse-history protocols report improvements over the baselines.

Critical strengths
  1. The motivation is clear and well-grounded. Existing retrieval- and plug-in-based methods determine which observations to emphasize, but not how strongly those observations support a preference, and this distinction matters when histories are sparse. The method is technically coherent. Closed-form Gaussian and mixture updates give a principled account of how strongly new evidence should revise preference estimates, and deployment adaptation needs neither optimizer steps nor shared-parameter updates.

  2. The theoretical analysis gives some formal grounding. Proposition 1 characterizes how new evidence changes the predicted residual at a query, and the batch-sequential equivalence in Appendix C.3 also make sense. In addition, the paper is honest about its own limitations. It explains why LaMP-1/6/7 are excluded, acknowledges that the protocol differs from LaMP's official evaluation, and reports mixed ablation results instead of hiding them.

Critical weaknesses
  1. Novelty is limited. The core is a mixture extension of ALPaCA (Harrison et al., 2018), which already learns a Gaussian prior for closed-form online Bayesian regression, applied here to condition an LLM. The gains are concentrated on classification. Under evolving history, on LaMP-5 the method ties PPlug+kNN, and on LaMP-4 the improvement is small.

  2. The online evaluation protocol is custom and diverges from LaMP's official fixed-target evaluation, so the numbers are not directly comparable to published LaMP results. No code is released, and there is no report of inference latency or computational overhead. The main results lack significance testing.

Questions for authors
  1. Why not compare against the retrieval-optimization baselines of Salemi et al. (2024a), cited here and evaluated on LaMP? Would the gains survive that comparison?

  2. The mixture prior shows no consistent advantage over K=1, particularly under sparse history. What would establish that the multi-component structure is necessary, and how sensitive is the method to K?

  3. Under evolving history on LaMP-5, the method ties PPlug+kNN. Can you characterize when Bayesian preference inference helps versus when history aggregation alone suffices?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 7Sdr

Thank you for recognizing the theoretical grounding of BayUSI’s adaptation mechanism, particularly Proposition 1’s characterization of how new evidence changes query-specific residual predictions and the batch–sequential equivalence established in Appendix C.3. We also appreciate your recognition of our central motivation: distinguishing history selection from how strongly observations support a preference.

Q1. Novelty beyond ALPaCA

BayUSI formulates online personalization as Bayesian user-state inference over deviations from input-conditioned population predictions. It adopts the conjugate regression structure used in ALPaCA, with a learned mixture prior whose components represent shared preference patterns. The feature map, prior, and conditioning interface are jointly trained through the frozen LLM’s response prediction loss.

Eq. (40) highlights what distinguishes BayUSI’s adaptation from ALPaCA’s: observed responses refine coefficients within each component and update the posterior probabilities of shared preference patterns. With fixed input features, a single Gaussian prior yields a posterior mean that is affine in observed residuals. BayUSI’s response-dependent component probabilities allow the combined state to depend nonlinearly on those residuals, introducing an adaptation mechanism absent from ALPaCA’s single Gaussian formulation. Appendix C.3 establishes that this flexibility preserves exact sequential inference equivalent to batch conditioning when shared parameters are fixed.

Q2. When does preference inference help beyond history aggregation on LaMP-5?

To investigate when preference inference provides additional value beyond history aggregation, we analyze BayUSI’s gains across history lengths on LaMP-5 and add two complementary evaluations: CMV personalized argument ranking [1] and CoPA personalized question answering [2].

In LaMP-5, history aggregation alone may already capture reusable terminology and phrasing from historical titles, limiting the additional gains from preference inference under ROUGE. We first examine whether these gains vary with observed history length.

We stratify evolving-history episodes by observed history length and compare BayUSI against PPlug+kNN. Positive differences favor BayUSI.

Observed historyΔROUGE-1 [95% CI]ΔROUGE-L [95% CI]
All+0.00087 [−0.00069, +0.00237]+0.00036 [−0.00113, +0.00190]
10+0.00387 [−0.00100, +0.00853]+0.00291 [−0.00200, +0.00774]
20+0.00074 [−0.00424, +0.00600]+0.00035 [−0.00448, +0.00576]
30–50−0.00096 [−0.00382, +0.00175]−0.00095 [−0.00367, +0.00181]
60++0.00145 [−0.00072, +0.00362]+0.00067 [−0.00159, +0.00272]

The largest numerical gains occur at 10 interactions, but all confidence intervals include zero, so this analysis does not establish a history-length regime with reliable additional gains.

We next examine preference inference when evaluation directly reflects individual responses, using CMV, a benchmark for long-context personalization. Given a user’s interaction history and current post, the task requires reasoning about competing arguments and ranking replies by their likelihood of persuading that individual, as indicated by observed feedback. We compare against the same history aggregation baselines.

MethodMRR ↑MRR@3 ↑Hit@1 ↑Hit@3 ↑DCG@3 ↑
PPlug41.1332.2418.6550.5336.92
PPlug+kNN39.8730.8816.0251.3536.11
BayUSI (K=1)42.6234.8418.6557.4440.60
BayUSI (K=4)42.4634.3619.6254.3639.47
BayUSI (K=8)42.9335.0521.0554.2139.94

Across all three K settings, BayUSI exceeds both baselines on MRR, MRR@3, Hit@3, and DCG@3. With K=1, Hit@3 improves by 6.09 percentage points and DCG@3 by 10.0% over the strongest baseline for each metric.

To examine whether the benefit also extends to generation, we evaluate CoPA personalized question answering. CoPA assesses how open-ended answers to Stack Exchange questions align with individual users’ preferences through a six-factor rubric, rather than reference overlap. We apply this rubric with GPT-4o as the judge.

MethodPreference alignment ↑ (GPT-4o judge)
PPlug0.053
PPlug+kNN0.056
BayUSI (K=1)0.055
BayUSI (K=4)0.072
BayUSI (K=8)0.067

BayUSI (K=4) achieves a 29.4% higher preference-alignment score than the strongest baseline, PPlug+kNN.

Together, CMV and CoPA provide complementary evidence that Bayesian user-state inference adds value beyond history aggregation in both personalized argument ranking and open-ended generation evaluated against individual preferences.

[1] Zhang, X. F., Beauchamp, N., and Wang, L. (2025). PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought Process. EMNLP.

[2] Su, H., et al. (2026). CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors. Findings of ACL.

AuthorsOpenReview ↗

Response to Reviewer 7Sdr

Q3. Reproducibility of the Online Evaluation Protocol

To address your concerns about the custom protocol and reproducibility, we clarify the evaluation schedules, data partitions, and comparability with official LaMP scores. Our protocol evaluates adaptation as evidence accumulates, so its results are not directly comparable with LaMP's official fixed-target scores. All methods receive the same histories and prediction targets. Using interactions in their supplied order:

  • Evolving-history: reveal 10 interactions and predict one deterministically selected target from the next unrevealed block of up to 10.
  • Sparse-history: reveal one interaction and predict the next response, evaluating histories of 1–10 interactions.

Section 4.1 specifies this protocol. Evaluation episodes use validation profiles; preparation, optimization, and checkpoint selection use separate partitions within the official training data (Appendix H, Table 7). We will publicly release our code upon acceptance to facilitate reproducibility.

Q4. Additional inference latency

To address the missing computational-overhead measurements, we have added matched latency comparisons and a breakdown of observation encoding and Bayesian-update costs. All experiments are conducted on an NVIDIA A100 80GB GPU with batch size 1 and matched query and history schedules. Measurements include observation/query encoding, online updates, and prefix construction, excluding LLM execution. BayUSI incurs an additional 8.56–34.42 ms/query in personalization latency compared with the baselines.

Additional personalization latency, including encoding (ms/query).

DatasetSettingΔ vs. PPlugΔ vs. PPlug+kNN
LaMP-2Evolving-history+23.12+22.03
LaMP-2Sparse-history+9.95+8.56
LaMP-4Evolving-history+34.42+33.69
LaMP-4Sparse-history+20.28+19.56

Δ denotes BayUSI minus the corresponding baseline.

Separate diagnostic profiling gives the following absolute BayUSI observation-processing costs:

BayUSI observation-processing costs (ms/query).

DatasetSettingObservation encodingResidualization + posterior update
LaMP-2Evolving-history37.502.14
LaMP-2Sparse-history14.001.59
LaMP-4Evolving-history42.862.07
LaMP-4Sparse-history24.971.54

Observation encoding includes paired-history encoding, also performed by PPlug, plus the additional input/response encodings needed for residual inference. Encoding dominates this processing; residualization and posterior updating take 1.54–2.14 ms/query. Reusing paired-history representations through lightweight feature mappings is a potential optimization that remains to be evaluated.

Q5. Absence of significance testing

Following your comment, we have added paired significance tests against the strongest baseline for each metric across all four datasets and both history protocols. We use dependency-block permutation tests with 10,000 permutations and Holm correction across 16 comparisons. For each metric, the comparator is the strongest baseline method in Table 1. The table reports two-sided, Holm-adjusted p-values.

DatasetMetricEvolving-history p (Holm)Sparse-history p (Holm)
LaMP-2Accuracy0.01441.0000
Macro-F10.25800.1353
LaMP-3MAE0.00160.0016
RMSE0.00160.0016
LaMP-4ROUGE-11.00001.0000
ROUGE-L0.53900.5390
LaMP-5ROUGE-11.00001.0000
ROUGE-L1.00001.0000

The improvements are significant for LaMP-3 MAE and RMSE under both protocols, and for evolving-history LaMP-2 Accuracy. The remaining comparisons do not meet the adjusted 0.05 threshold.

AuthorsOpenReview ↗

Response to Reviewer 7Sdr

Q6. Comparison with retrieval-optimization baselines

Following your suggestion, we have added comparisons with the retrieval-optimization baselines of Salemi et al. (2024a): ROPG-KD, ROPG-RL, RSPG-PRE, and RSPG-POST.

BayUSI achieves the best score in every reported comparison below, extending the evidence beyond standard retrieval and plug-in baselines.

Evolving History

DatasetMetricROPG-KDROPG-RLRSPG-PRERSPG-POSTBayUSI
LaMP-2Accuracy ↑0.4890.4120.4310.5280.732
F1 ↑0.4450.3710.3800.4890.722
LaMP-3MAE ↓0.2910.3360.3100.3150.218
RMSE ↓0.6170.6870.6340.6500.509
LaMP-4ROUGE-1 ↑0.1760.1720.1780.1950.217
ROUGE-L ↑0.1560.1560.1600.1780.197
LaMP-5ROUGE-1 ↑0.3670.3240.3730.4320.472
ROUGE-L ↑0.3170.2810.3230.3710.430

Sparse History

DatasetMetricROPG-KDROPG-RLRSPG-PRERSPG-POSTBayUSI
LaMP-2Accuracy ↑0.4620.4460.4570.5190.696
F1 ↑0.4100.3970.4040.4610.656
LaMP-3MAE ↓0.3180.3260.3200.3230.238
RMSE ↓0.6600.6720.6520.6640.540
LaMP-4ROUGE-1 ↑0.1670.1670.1680.1800.206
ROUGE-L ↑0.1470.1480.1500.1610.190
LaMP-5ROUGE-1 ↑0.3270.3420.3360.3790.473
ROUGE-L ↑0.2830.2960.2910.3260.431

These comparisons support explicit user-state inference as a useful complement to optimized history selection.

Q7. When are multiple prior components useful, and how sensitive is performance to K?

To address your questions about necessity and sensitivity to K, we clarify when multiple components can help and extend the appendix’s single-seed comparison to three random seeds.

Multiple components help when distinct shared patterns provide useful guidance and user evidence can distinguish them. Sparse histories increase reliance on the prior, but do not alone establish a need for a mixture: K=1 already performs Bayesian coefficient adaptation.

We extend the sensitivity analysis to three-seed means for K=1, 4, and 8:

Evolving-history

DatasetMetricK=1K=4K=8
LaMP-2Accuracy ↑0.7410.7360.721
F1 ↑0.7080.7240.702
LaMP-4ROUGE-1 ↑0.2110.2140.213
ROUGE-L ↑0.1910.1950.193

Sparse-history

DatasetMetricK=1K=4K=8
LaMP-2Accuracy ↑0.7020.7080.710
F1 ↑0.6540.6640.661
LaMP-4ROUGE-1 ↑0.2070.2060.206
ROUGE-L ↑0.1880.1890.188

Sparse-history LaMP-2 benefits from additional components, consistent with useful prior structure under limited evidence. LaMP-4's small gains occur mainly with evolving histories, suggesting that exploiting this structure may require more evidence. K=8 provides no consistent improvement over K=4, and evolving-history LaMP-2 Accuracy favors K=1. The results establish conditional usefulness.

Reviewer_bwdCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Bayesian User-State Inference (BayUSI), a framework for online LLM personalization. The method models user preferences through a learned mixture prior and performs Bayesian posterior updates using observed user interactions. The inferred user states are incorporated into a frozen LLM through learnable prefixes, enabling personalization without per-user fine-tuning. Experiments on four LaMP tasks demonstrate improvements over existing personalization baselines under sparse-history and evolving-history settings.

Critical strengths

Reasonable methodology. Combining Bayesian linear regression, learned mixture priors, and prefix-based LLM conditioning is intuitive and well-motivated. The uncertainty-aware posterior updates provide a principled way to incorporate new user evidence.

Critical weaknesses

W1. Lack of evaluation under preference drift. BayUSI assumes stationary user preferences, while real user preferences may evolve over time. As posterior uncertainty decreases with accumulated evidence, the model may struggle to adapt to preference shifts. Experiments under changing user preferences would strengthen the claims about online personalization.

W2. Unclear necessity of the mixture prior. The K=1 variant performs comparably to or even better than K=4 in several settings (Appendix Table 4). This raises questions about whether multiple shared preference components are necessary and how much they contribute to the overall improvement.

Questions for authors

Q1. Contribution of Bayesian uncertainty. How much of the improvement comes specifically from uncertainty-aware updates rather than simpler deterministic user-state updates? A stronger comparison would help isolate this contribution.

Q2. Limited gains on generation tasks. Improvements on LaMP-4 and LaMP-5 are relatively small, especially compared with classification and rating tasks. What might explain this difference?

Q3. Generalizability. All experiments are conducted on LaMP tasks. Could the authors evaluate BayUSI on additional personalization benchmarks to demonstrate broader applicability?

2 comments
AuthorsOpenReview ↗

Response to Reviewer bwdC

Thank you for recognizing the motivation and principled incorporation of new user evidence. We address the roles of Bayesian updating and the mixture prior, and add evaluations of preference drift and personalization beyond LaMP.

Q1. Adaptation to preference drift

Following your suggestion to evaluate changing preferences, we have added a controlled LaMP-3 experiment that introduces a rating shift while retaining all pre-shift history. We reveal 100 interactions with original ratings, followed by 50 with ratings shifted downward by one point and clipped at 1:

y′=max⁡(1,y−1).y'=\max(1,y-1).

All methods predict before observing each new rating, using existing checkpoints without retraining or change-point information.

MethodOverall post-shift MAE ↓First-10 MAE ↓Last-10 MAE ↓
PPlug1.0651.0701.072
PPlug+kNN1.0561.0621.058
BayUSI0.9271.0060.869

BayUSI reduces overall post-shift MAE by 12.2% relative to PPlug+kNN. To account for changing target difficulty, we also compare BayUSI with a stationary control that applies the shifted rating rule throughout the stream. Excess MAE is the drift-stream MAE minus this control’s MAE at the corresponding post-shift targets.

BayUSI comparisonFirst 10 post-shiftLast 10 post-shiftReduction
Excess MAE ↓0.4990.36826.3%

The decreasing excess error supports continued adaptation to the shifted ratings, even while all pre-shift observations remain in the history.

Q2. Contribution of the mixture prior beyond K=1K=1

To address this concern about the mixture prior’s necessity, we clarify its role and extend the appendix’s single-seed comparison to three random seeds.

With K=1, observations refine coefficients around one prior center. Multiple components also update the relative support for different shared preference patterns, allowing the state to depend nonlinearly on observed residuals while preserving closed-form inference.

Our extended comparison of K=1, 4, and 8 under both history protocols shows a conditional benefit. Increasing K from 1 to 4 improves LaMP-2 Macro-F1 from 0.708 to 0.724 under evolving history and 0.654 to 0.664 under sparse history. However, evolving-history Accuracy favors K=1, LaMP-4 gains are small, and K=8 offers no consistent further improvement. The mixture therefore provides useful prior flexibility; multiple components are not necessary in every setting.

Evolving-history

DatasetMetricK=1K=4K=8
LaMP-2Accuracy ↑0.7410.7360.721
F1 ↑0.7080.7240.702
LaMP-4ROUGE-1 ↑0.2110.2140.213
ROUGE-L ↑0.1910.1950.193

Sparse-history

DatasetMetricK=1K=4K=8
LaMP-2Accuracy ↑0.7020.7080.710
F1 ↑0.6540.6640.661
LaMP-4ROUGE-1 ↑0.2070.2060.206
ROUGE-L ↑0.1880.1890.188

Q3. Contribution of uncertainty-aware Bayesian updates

To address your request for a stronger comparison with simpler user-state updates, we have added a three-seed evaluation of BayUSI (K=1) against SGD coefficient updates. SGD uses a validation-selected learning rate; the tables report mean performance under both history protocols.

Evolving-history

DatasetMetricSGDBayUSI (K=1)
LaMP-2Accuracy ↑0.6920.741
LaMP-2Macro-F1 ↑0.6680.708
LaMP-4ROUGE-1 ↑0.2100.211
LaMP-4ROUGE-L ↑0.1900.191

Sparse-history

DatasetMetricSGDBayUSI (K=1)
LaMP-2Accuracy ↑0.6880.702
LaMP-2Macro-F1 ↑0.6360.654
LaMP-4ROUGE-1 ↑0.2050.207
LaMP-4ROUGE-L ↑0.1870.188

BayUSI improves every reported metric, with the largest gain on evolving-history LaMP-2 Accuracy (+4.9 percentage points). SGD scales gradients by a scalar learning rate; BayUSI uses posterior covariance to determine the direction and strength of coefficient updates (Eq. (8)). The K=1 comparison supports this Bayesian update rule over tuned SGD independently of mixture reweighting.

AuthorsOpenReview ↗

Response to Reviewer bwdC

Q4. Smaller gains on generation tasks

To explain the smaller generation gains, we clarify how the information supplied by history aggregation differs across the evaluated tasks. In LaMP-4/5, historical titles provide reusable terminology and phrasing, which may already supply much of the information rewarded by ROUGE. Tagging and rating depend more directly on how a user’s past labels or scores relate to their inputs; BayUSI models these relationships through response deviations from population predictions. This provides a plausible explanation for the larger gains on tagging and rating, while the generation results indicate a smaller additional benefit beyond history conditioning.

Q5. Applicability beyond LaMP

Following your suggestion to evaluate beyond LaMP, we have added two complementary benchmarks: CMV personalized argument ranking [1] and CoPA personalized question answering [2]. These results demonstrate BayUSI’s broader applicability across personalized ranking and open-ended question answering.

CMV is a benchmark for long-context personalization. Given a user’s interaction history and current post, the model assesses competing arguments and ranks replies by their likelihood of persuading that individual.

MethodMRR ↑MRR@3 ↑Hit@1 ↑Hit@3 ↑DCG@3 ↑
PPlug41.1332.2418.6550.5336.92
PPlug+kNN39.8730.8816.0251.3536.11
BayUSI (K=1)42.6234.8418.6557.4440.60
BayUSI (K=4)42.4634.3619.6254.3639.47
BayUSI (K=8)42.9335.0521.0554.2139.94

All three BayUSI variants improve MRR, MRR@3, Hit@3, and DCG@3 over both PPlug and PPlug+kNN. K=8 achieves the highest MRR, MRR@3, and Hit@1; K=1 achieves the highest Hit@3 and DCG@3. For K=1, Hit@3 increases from 51.35 to 57.44 (+6.09 percentage points) and DCG@3 from 36.92 to 40.60 (+10.0%), relative to the strongest baseline for each metric.

CoPA evaluates open-ended personalized question answering on Stack Exchange across diverse domains, including mathematics and programming. BayUSI infers a user state from historical interactions to condition answer generation. We assess responses using CoPA’s six-factor preference-alignment rubric with GPT-4o as the judge.

MethodPreference alignment ↑ (GPT-4o judge)
PPlug0.053
PPlug+kNN0.056
BayUSI (K=1)0.055
BayUSI (K=4)0.072
BayUSI (K=8)0.067

BayUSI (K=4) achieves a 29.4% higher preference-alignment score than the strongest baseline, PPlug+kNN. These gains extend the empirical support for Bayesian user-state inference beyond LaMP to long-context personalized argument ranking and open-ended question answering.

[1] Zhang, X. F., Beauchamp, N., and Wang, L. (2025). PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought Process. EMNLP.

[2] Su, H., et al. (2026). CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors. Findings of ACL.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.