- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This work aims to improve the performance of a frozen LLM in classification, rating and title-generation tasks. Authors propose BayUSI, which extracts a user's preference information from the their history data and incorporate it into LLM input. The user preference information is recorded by a weighted sum , whose weights and are updated using Bayesian inference by collecting the user's sequence of history input.
Critical strengths
-
The clarity of this work is good. Math expressions are correct and match authors' framework in Figure 1. Authors provide detailed explanations on how their math expressions represent and manipulate user preference.
-
Authors provide detailed empirical study on a variety of tasks, datasets and benchmarks. They also provide ablation study showing the effectiveness of the individual components in their framework.
-
Personalization has been a core problem in recommender systems, since a recommender model makes personalized recommendation by identifying the personal interests of different users. Authors' work on LLMs could be a new solution in this area.
Critical weaknesses
- The mathematical intuition behind authors' framework is unclear.
-
Why should the user preference be learned by a weighted sum with Bayesian update on weights? What are the benefits? Can it be learned instead by using a shallow neural network with backpropagation update on weights?
-
Why can extract and store a user's preference information from their sparse history and evolving history? What is the mechanism behind it?
-
Suppose you have learned the weights . How do you interpret them? For example, by observing their coefficients, can you tell whether this user likes something or dislikes something?
- Limited scope in the study of LLMs. Classification, rating and title-generation are simple tasks and are usually handled using traditional machine learning models. While LLMs can also handle these tasks, they are more frequently used in complex tasks such as reasoning, math and coding, and evaluated using complex benchmarks such as LiveBench (White et al., 2025). It is unclear how authors' study on user preference can be adapted to these complex tasks.
Minor:
In abstract, "As interactions accumulate, posterior uncertainty governs how strongly new evidence updates preference estimates", what does the "uncertainty" mean here? The uncertainty of a distribution typically means its entropy in information theory. I think authors intent to mean "the posterior captures user preference".
Reference:
Colin White, et al. LiveBench: A challenging, contamination-limited LLM benchmark. ICLR, 2025.
Questions for authors
-
(Critical) Can authors show an example of the LLM input in Figure 1, and explain how it reflects the user's preference?
-
In Line 135, , what do and mean? And how do you determine their values?
4 comments
Response to Reviewer xPHU
Q2. Why use the Bayesian posterior-weighted state?
To clarify the mathematical intuition, we express the weighted state as a posterior expectation and explain how it supports adaptation from sparse and growing histories. BayUSI models a user’s preferences through coefficients that relate input features to response deviations from the population prediction. Limited history leaves these coefficients uncertain, so we average the predicted deviation over their posterior:
=\sum_{k=1}^{K}\gamma_{u,t,k}M_{u,t,k}^{\top}f_\theta(x_{u,t}).$$ Each coefficient mean supplies a user-adapted prediction, weighted by its component's posterior probability (Eq. (10)). This averages plausible patterns when the evidence does not clearly distinguish them. The Bayesian construction supports this estimate in three ways: 1. **A shared starting point for sparse histories.** The learned prior regularizes the coefficients toward recurring patterns among training users, allowing estimation from limited personal evidence. It captures both typical coefficients and variation around them, without requiring annotated user groups (Eq. 2; lines 143–157). 2. **Progressive adaptation as history grows.** Within each component, new observations revise the coefficients according to the prediction discrepancy and remaining coefficient uncertainty (Eq. 8). 3. **Evidence-based weighting of shared patterns.** Component probabilities change according to how well each pattern explains the observed history, integrating over coefficient uncertainty (Eqs. 6–7). Thus, personal evidence refines both **the prediction within each pattern** and **the relative support for different patterns**, producing an expected response deviation tailored to the current query. ## **Q3. Benefits of BayUSI over a shallow neural network** **To evaluate the neural alternative you raise, we compare BayUSI with a shallow MLP state predictor and gradient-based coefficient updates, highlighting both the empirical benefits and theoretical guarantees of Bayesian user-state inference.** The shallow MLP maps the observed history and current query directly to a user state, using the same LLM conditioning interface: **Evolving-history** | Dataset | Metric | MLP | BayUSI | | --- | --- | --- | --- | | LaMP-2 | Accuracy ↑ | 0.641 | **0.732** | | LaMP-2 | Macro-F1 ↑ | 0.634 | **0.722** | | LaMP-4 | ROUGE-1 ↑ | 0.205 | **0.217** | | LaMP-4 | ROUGE-L ↑ | 0.184 | **0.197** | **Sparse-history** | Dataset | Metric | MLP | BayUSI | | --- | --- | --- | --- | | LaMP-2 | Accuracy ↑ | 0.674 | **0.696** | | LaMP-2 | Macro-F1 ↑ | 0.618 | **0.656** | | LaMP-4 | ROUGE-1 ↑ | 0.203 | **0.206** | | LaMP-4 | ROUGE-L ↑ | 0.184 | **0.190** | **BayUSI improves every reported metric in both settings**, supporting the practical benefit of explicit posterior inference over the evaluated neural mapping. We also compare online coefficient updates against validation-tuned SGD over three seeds: **Evolving-history** | Dataset | Metric | SGD | BayUSI (K=1) | | --- | --- | --- | --- | | LaMP-2 | Accuracy ↑ | 0.692 | **0.741** | | LaMP-2 | Macro-F1 ↑ | 0.668 | **0.708** | | LaMP-4 | ROUGE-1 ↑ | 0.210 | **0.211** | | LaMP-4 | ROUGE-L ↑ | 0.190 | **0.191** | **Sparse-history** | Dataset | Metric | SGD | BayUSI (K=1) | | --- | --- | --- | --- | | LaMP-2 | Accuracy ↑ | 0.688 | **0.702** | | LaMP-2 | Macro-F1 ↑ | 0.636 | **0.654** | | LaMP-4 | ROUGE-1 ↑ | 0.205 | **0.207** | | LaMP-4 | ROUGE-L ↑ | 0.187 | **0.188** | With a single component, BayUSI achieves higher mean scores in all eight SGD comparisons, including a 4.9-percentage-point gain in evolving-history LaMP-2 Accuracy. **Beyond these empirical gains, BayUSI provides exact closed-form posterior updates, with adaptation strength governed by posterior uncertainty (Eq. 8), and guarantees equivalence between sequential updates and batch conditioning under fixed shared parameters (Appendix C.3).** **Together, these results support the benefits of principled Bayesian user-state inference over the evaluated neural and gradient-based alternatives.**Response to Reviewer xPHU
Q4. How is preference information extracted and retained?
We trace how preference information is extracted and retained through accumulated statistics, posterior updates, and query-specific readout.
The posterior retains preference information; Eq. (10) reads it out for the current query. Each observed input–response pair contributes its input features and response residual to the accumulated statistics in Eq. (5):
\qquad \sum_{i<t} f_{u,i}f_{u,i}^{\top}.$$ The first associates inputs with response deviations; the second records the feature directions covered by the observed history. With the learned prior, these statistics determine the coefficient means and covariance (lines 186–199). Component probabilities update according to how well each component explains the residuals (Eqs. (6)–(7)). Eq. 10 then evaluates the inferred feature-to-deviation relationship at the current query: **one stored posterior can produce different adjustments for different queries**, and new observations refine it. ## **Q5. How should the coefficients and component weights be interpreted?** **To make the learned quantities interpretable, we distinguish coefficient means from component probabilities and illustrate their combined meaning through rating deviations.** The quantities have distinct meanings: - **Coefficient means:** the estimated mapping from input features to the user's response deviation. The intercept captures an input-independent tendency; the other coefficients capture variation with the input. - **Component probabilities (Eq. 7):** the history's relative support for each shared preference pattern. A larger probability indicates stronger support for that pattern, rather than stronger liking. Learned feature coordinates and components have no predefined semantic labels. Interpretation therefore comes from their combined prediction for a given input. In LaMP-3, a positive state means an expected rating above the population prediction for that review; a negative state means an expected rating below it. A state of +0.5 corresponds to a one-star upward deviation under the residual model (Eq. (17)). This describes **contextual, relative preferences**; a negative deviation alone does not imply absolute dislike. ## **Q6. Extension to more complex tasks: reasoning, mathematics, coding, and LiveBench.** **To investigate how user preference inference extends to reasoning and technical question answering, including mathematics and programming, we evaluate BayUSI on two complementary benchmarks: CMV personalized argument ranking [1] and CoPA personalized answer generation [2].** These benchmarks provide user histories and preference-sensitive evaluation targets, allowing us to assess how inferred preferences guide response selection and generation. This complements correctness-oriented benchmarks such as LiveBench, where correctness alone does not isolate adaptation to individual users. **CMV evaluates reasoning about individual users’ responses.** Given a user’s observed history and current post, the model assesses competing arguments and ranks replies by their likelihood of persuading that individual. |Method|MRR ↑|MRR@3 ↑|Hit@1 ↑|Hit@3 ↑|DCG@3 ↑| |---|---|---|---|---|---| |PPlug|41.13|32.24|18.65|50.53|36.92| |PPlug+kNN|39.87|30.88|16.02|51.35|36.11| |BayUSI (K=1)|42.62|34.84|18.65|**57.44**|**40.60**| |BayUSI (K=4)|42.46|34.36|19.62|54.36|39.47| |BayUSI (K=8)|**42.93**|**35.05**|**21.05**|54.21|39.94| **BayUSI variants achieve the best results across all five metrics**. With K=1, Hit@3 improves by **6.09 percentage points** and DCG@3 by **10.0%** over the strongest baseline for each metric. **CoPA evaluates open-ended personalized question answering across domains, including mathematics and programming.** Constructed from Stack Exchange, it evaluates whether generated answers align with individual users’ preferences. BayUSI infers a user state from historical interactions and uses it to condition the LLM’s answer to a new question. We assess generated responses using CoPA’s six-factor preference-alignment rubric with GPT-4o as the judge. |Method|Preference alignment ↑ (GPT-4o judge)| |---|---| |PPlug|0.053| |PPlug+kNN|0.056| |BayUSI (K=1)|0.055| |BayUSI (K=4)|**0.072**| |BayUSI (K=8)|0.067| BayUSI (K=4) achieves a **29.4% higher preference-alignment score** than the strongest baseline, PPlug+kNN. Together with CMV, these results address applicability beyond LaMP: **inferred user states guide reasoning about which competing arguments are most likely to persuade each user and personalized answer generation in domains including mathematics and programming.** [1] Zhang, X. F., Beauchamp, N., and Wang, L. (2025). PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought Process. EMNLP. [2] Su, H., et al. (2026). CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors. Findings of ACL.Response to Reviewer xPHU
Q7. What is posterior uncertainty, and how does it govern updating?
The posterior represents both an estimated preference relationship and uncertainty about that estimate. Within a component, the coefficient mean gives the estimate, while the covariance measures how uncertain that relationship remains :
\qquad v=f^\top Sf.$$ This directional variance measures uncertainty in the **expected deviation at the observed input**. Identical preference estimates can have different uncertainty when supported by different histories. Entropy is one scalar summary of uncertainty. For a Gaussian component, it depends on the covariance determinant. BayUSI uses covariance directly because updating requires information about *which feature directions* remain uncertain. This has an explicit computational effect. Eq. 8 and Proposition 1 show that, within a component, the prediction at the newly observed input changes by $$(M^{+}-M)^\top f = \frac{v}{\sigma^2+v}(r-M^\top f)$$ Here, **v/(σ²+v)** is the uncertainty-dependent update weight, and **r − Mᵀf** is the prediction discrepancy. The same discrepancy produces a larger correction when uncertainty is greater relative to observation noise. **Uncertainty therefore governs the inference update; the LLM receives the deterministic posterior-mean readout**, rather than the covariance itself (Eq. (10)). ## **Q8. What do** $d$ **and** $p$ **represent in** $W_u\in\mathbb{R}^{d\times p}$**, and how are their values determined?** **To address your question about the dimensions, we explain their definitions and selection rationale, and add a sensitivity analysis on LaMP-4.** In the coefficient matrix, the input dimension includes the learned features and constant intercept; the output dimension is that of the response representation and residual: $$W_u\in\mathbb R^{d\times p},\quad f_\theta(x)\in\mathbb R^d,\quad a(y),r,W_u^\top f_\theta(x)\in\mathbb R^p.$$ These dimensions are fixed across users. We use eight input features: seven learned coordinates plus one intercept. The response dimensions are: | Task | $p$ | Response representation | | --- | --- | --- | | LaMP-2 | 14 | Orthonormal contrasts for 15 categorical labels | | LaMP-3 | 1 | Rating normalized to ([-1,1]) | | LaMP-4 and LaMP-5 | 8 | Frozen response embeddings projected onto eight whitened PCA components | The classification and rating dimensions follow their response representations. The input dimension and generation PCA rank are capacity hyperparameters; PCA uses training responses only (Appendix A). We vary the input and response dimensions independently over 4, 8, and 32 on LaMP-4, holding the other at eight: | Input dimension $d$ | Response dimensions $p$ | Evolving ROUGE-1 ↑ | Evolving ROUGE-L ↑ | Sparse ROUGE-1 ↑ | Sparse ROUGE-L ↑ | | --- | --- | --- | --- | --- | --- | | 8 | 8 | **0.217** | **0.197** | 0.206 | **0.190** | | 4 | 8 | 0.213 | 0.192 | **0.207** | 0.188 | | 32 | 8 | 0.214 | 0.191 | 0.205 | 0.187 | | 8 | 4 | 0.211 | 0.191 | 0.205 | 0.187 | | 8 | 32 | 0.206 | 0.185 | 0.205 | 0.186 | The default achieves the highest scores on three of four metrics; increasing either dimension to 32 does not improve it in this experiment. We will clarify the dimensions at first use and include these sensitivity results in the revision.
Response to Reviewer xPHU
Thank you for recognizing the clear formulation and detailed empirical study. In response to your comments, we provide a worked LLM-input example and explicit explanations of the inference steps, and add comparisons with neural and SGD alternatives, a CMV evaluation, and dimension-sensitivity results.
Q1. A concrete LLM input and how it reflects user preferences
Following your request for a concrete LLM-input example, we provide a LaMP-3 walkthrough from a historical rating to residual computation, user-state inference, and prefix conditioning. The LLM receives the current text prompt plus continuous key–value prefix vectors at every transformer layer. The reviews, population prediction, and example state values are illustrative.
Observe an input-response pair (review-user’s rating).
Suppose the user assigned 5 stars to:
This review–rating pair becomes part of the observed history.
Compute the user’s response deviation from a population prediction.
Ratings are normalized by (Eq. 17):
a(y)=2y−3.
A separate population predictor, fitted on training users and then frozen, predicts this normalized rating from the review alone. Suppose it predicts −0.25, equivalent to 2.5 stars. The observed residual is:
r=a(5)−f0(x)=1−(−0.25)=1.25.
This positive residual means the user rated that review more favorably than the population prediction.
Infer the relationship between review features and response deviations.
BayUSI pairs historical review features with their residuals to infer coefficients that map input features to expected response deviations (Eq. 4; lines 168–182):
E[r∣x,Wu]=Wu⊤fθ(x).
Bayesian conditioning estimates these coefficients and their uncertainty; their posterior means are the matrices in Eq. 5. For example, repeated positive residuals for mixed reviews can support a more favorable rating tendency for similar reviews.
Read out the expected deviation for the current review.
An illustrative current text prompt is:
Evaluating the inferred relationship at this review gives the query-specific user state in Eq. 10. For LaMP-3, this is a scalar expected rating deviation. For illustration, a state of +0.5 would represent a one-star upward deviation from the population prediction under the residual model. Only historical interactions inform this estimate; the current rating is unknown.
Convert the state into the actual LLM conditioning.
A learned mapping converts the state into key–value prefix vectors. At each transformer layer, these are added to a separate history prefix summarizing query-relevant history. The combined vectors enter the layer’s attention as additional keys and values, and the frozen LLM generates the rating (Eqs. (11)–(12)).
The same text prompt can therefore receive different prefixes for users with different histories. The shared feature map, prior, and conditioning mappings are trained through the LLM's prediction loss; deployment adaptation uses Bayesian conditioning without per-user fine-tuning.