- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LatentPersonal, a framework for few-shot LLM personalization that learns a shared low-dimensional adaptation space across users. Given a small set of interactions from an unseen user, the method infers a latent representation using a variational preference encoder and uses it to modulate shared PEFT parameters. Its main implementation inserts a user-dependent diagonal matrix between shared LoRA factors, enabling personalization without user-specific gradient updates at inference time.
Critical strengths
The paper addresses a meaningful problem and the proposed LoRA parameterization provides a simple and computationally efficient solution. This design is technically reasonable and provides a useful alternative to generating or optimizing separate adaptation parameters for each user. The empirical evaluation demonstrates that LatentPersonal performs competitively across four personalization benchmarks, with particularly strong results in the one-shot setting.
Critical weaknesses
-
The proposed approach is closely related to existing methods that generate personalized parameters from user representations. Although using latent coefficients to modulate shared LoRA components is a simple and efficient design, it is not clear what specific benefits this parameterization offers over other shared adaptation mechanisms. The current ablations do not fully address this question, particularly since the w/o Navigation variant also changes the optimization procedure.
-
It is unclear to what extent the learned user representations capture preferences rather than differences in interaction content. The preference encoder relies on mean-pooled interaction representations, which may also encode user-specific topics or content patterns. Since support and query samples are drawn from the same users, the model could potentially benefit from these correlations without learning stable preferences.
-
The analysis of the information bottleneck and learned latent space could be more convincing. While removing the IB term leads to a substantial performance drop, it is unclear whether this reflects the intended information compression or simply the benefits of regularization. Similarly, the decreasing latent displacement in Figures 3 and 6 could partly be explained by averaging more interaction representations, rather than by a particular property of the learned latent space.
-
Some of the reported improvements over Profile-to-PEFT are relatively small, and the gains are less consistent with the second backbone. Paired statistical comparisons would make it easier to assess how reliable these differences are. I also found the evaluation setup somewhat unclear, particularly how personalized reference responses are constructed from PRISM. Since the LLM-based evaluator has access to both user histories and reference responses, it would be useful to clarify how well the resulting scores distinguish preference alignment from reference similarity.
Questions for authors
Please refer to weaknesses
8 comments
W2: Preference Learning and Content Correlations
W2(a): Distinction Between Preferences and Interaction Content
User preferences and interaction content are not mutually exclusive in LLM personalization.
For example, a user's long-standing interest in artificial intelligence topics can itself constitute a topic preference. User preferences encompass not only tone, formatting, and response length, but also content selection, topic interests, and information emphasis. Previous studies [1, 2] have also recognized topical interests and content preferences as important aspects of LLM personalization.
Consistent with this perspective, our Content Personalization (CP) metric explicitly considers topics, examples, information selection, and emphasis. Therefore, our objective is not to eliminate content information from , but to preserve information relevant to personalized adaptation while discarding irrelevant interaction-specific details. This is consistent with the information bottleneck formulation in Section 3.2, which encourages the learned representation to retain preference-relevant information that generalizes across interactions.
[1] Personalized Benchmarking: Evaluating LLMs by Individual Preferences. Findings of ACL, 2026.
[2] Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personas. ACL, 2025.
W2(b): Role of Mean Pooling in Preference Learning
We acknowledge that mean-pooled interaction representations may contain topical and content information. However, mean pooling serves only as a feature aggregation step, rather than directly determining the learned user preferences.
Specifically, the aggregated features are processed by a learnable variational preference encoder, whose optimization is governed by two complementary objectives (Eq. 12):
The adaptation loss encourages to preserve information useful for personalization on held-out query interactions, while the compression term discourages encoding unnecessary interaction-specific information. Together, these objectives encourage to capture information relevant to personalization across interactions, rather than indiscriminately preserving all information contained in the mean-pooled representations.
W2(c): Support–Query Correlations
PIR and PersonaMem-v2 contain diverse interaction content associated with user-specific preferences.
For example, PIR includes questions on distinct topics, such as cooking, weight management, and general knowledge, under the same persona. Similarly, PersonaMem-v2 incorporates different interaction scenarios, including email editing, translation, and question answering, while maintaining user-level preference information.
These dataset designs allow us to evaluate personalization across diverse interaction contexts rather than only repeated or similar topics. Furthermore, training and test users are strictly disjoint, and support and query interactions do not overlap.
Additional evidence comes from our User-Specificity Analysis (Figure 4), where matched user representations consistently outperform swapped and population-mean representations under identical evaluation queries. This demonstrates that the inferred latent representations contain user-specific information that contributes to personalization, rather than merely providing generic adaptation.
Overall, our objective is to learn compact representations that preserve information useful for user-specific adaptation, including content-related preferences where relevant. The support–query objective, diverse evaluation scenarios, and user-specificity analysis collectively provide evidence that LatentPersonal learns representations useful for personalization beyond simple interaction matching.
W3: Information Bottleneck and Latent Space Analysis
W3(a): Information Bottleneck vs. Regularization
We would like to clarify that information bottleneck (IB) and regularization are not mutually exclusive. In fact, IB is implemented through a specific form of regularization that encourages information compression.
Specifically, our KL regularization term provides a variational upper bound on the mutual information between user interactions and the learned latent representation, as derived in Appendix A.1:
Therefore, unlike generic regularization, our IB objective explicitly constrains the information retained from user interactions, encouraging the encoder to preserve information relevant to personalization while suppressing unnecessary interaction-specific details.
To further illustrate the specific role of the IB constraint, we additionally introduce an L2 regularization baseline by replacing the KL term with , while keeping the remaining model architecture and training procedure unchanged. The comparison on HiCUPID is presented below.
| Variant | 1-shot R-1 | MET. | CP | 2-shot R-1 | MET. | CP | 4-shot R-1 | MET. | CP |
|---|---|---|---|---|---|---|---|---|---|
| LatentPersonal | 0.4162 | 0.4133 | 5.95 | 0.4160 | 0.4142 | 5.92 | 0.4156 | 0.4137 | 5.94 |
| No Regularization | 0.2684 | 0.3423 | 4.57 | 0.2703 | 0.3422 | 4.65 | 0.2695 | 0.3444 | 4.74 |
| L2 Regularization | 0.3560 | 0.3826 | 5.34 | 0.3530 | 0.3899 | 5.38 | 0.3534 | 0.3811 | 5.46 |
The results show that L2 regularization improves performance over the no-regularization baseline, but consistently underperforms the full IB objective across all evaluated shot settings and metrics. This suggests that the benefits of IB extend beyond those of conventional weight regularization. Unlike L2, which primarily constrains the magnitude of latent representations, the variational IB objective additionally regularizes the posterior distribution and provides an explicit upper bound on the information retained from user interactions. These findings support the effectiveness of information-bottleneck regularization in learning compact representations for few-shot personalization.
W3(b): Latent Displacement vs. Mean Pooling
We would like to clarify that the decreasing latent displacement in Figures 3 and 6 is intended to illustrate the progressive stabilization of user representations as additional interactions become available, rather than to serve as the sole evidence of the learned latent space's properties.
To further distinguish the effect of mean pooling from that of the learned preference encoder, we introduce an additional comparison between the mean-pooled interaction representations and the learned latent representations. Specifically, for each user, we measure the displacement between consecutive shot settings (1→2, 2→4, and 4→8):
d_K^{z}=|z_u^{(2K)}-z_u^{(K)}|_2.$$ where $K \in {1,2,4}$ denotes the number of observed user interactions. Since the two representations have different dimensions and scales, we normalize their displacements by the corresponding 1→2-shot displacement to compare their relative stabilization trends. |Dataset| Representation|1→2|2→4|4→8| |---|---|---|---|---| |HiCUPID|Mean Pooling ($h_u$)|1.00|0.81|0.69| ||Learned Latent ($z_u$)|1.00|0.67|0.46| |PRISM|Mean Pooling ($h_u$)|1.00|0.89|0.71| ||Learned Latent ($z_u$)|1.00|0.74|0.50| |PersonaMem | Mean Pooling ($h_u$)|1.00|0.79|0.68| ||Learned Latent ($z_u$)|1.00|0.70|0.50| |PIR|Mean Pooling ($h_u$)|1.00|0.79|0.68| ||Learned Latent ($z_u$)|1.000|0.65|0.53| The results show that both mean-pooled and learned latent representations exhibit decreasing displacement as additional interactions become available. However, the learned latent representations consistently demonstrate a more pronounced relative reduction across all four datasets. This suggests that the observed stabilization is not solely attributable to the statistical effect of mean pooling, but also reflects the influence of the learned preference encoder. Together with the user-specificity and interpolation analyses in Figure 4, these findings provide complementary evidence that the learned latent space supports both stable representations and meaningful user-specific adaptation.
W1: Clarifying the Benefits of Our Parameterization
We sincerely thank you for your thoughtful and constructive feedback on LatentPersonal. We appreciate your recognition of the value and efficiency of our approach. Your comments have helped us improve the clarity and rigor of our work. We address your concerns in detail below.
W1(a): Distinction from Existing Methods
Among existing methods, Profile-to-PEFT is one of the most closely related to our work, as both follow a similar pipeline: User History→User Representation→Personalized Adaptation.
However, their adaptation mechanisms are fundamentally different. Profile-to-PEFT adopts parameter generation, using a hypernetwork to generate personalized LoRA parameters. In other words, it uses one neural network to generate the parameters of another neural network. In contrast, LatentPersonal adopts parameter composition within a shared low-dimensional adaptation space:
ΔWu=Bdiag(zu)A,
where zu determines their combination weights. This distinction leads to different parameter spaces and inductive biases: rather than generating user-specific LoRA factors, LatentPersonal combines shared adaptation directions through a compact latent representation.
We have also discussed this distinction in the "Personalized Adaptation of LLMs" subsection of our Related Work.
W1(b): Advantages of Parameter Composition
Regarding the specific benefits of our parameterization, as clarified in W1(a), LatentPersonal performs personalization through composition rather than parameter generation. This design offers two main advantages: (1) Parameter efficiency: only a compact latent vector is required to personalize shared LoRA factors, avoiding the generation of user-specific parameter matrices. (2) Few-shot generalization: constraining user-specific adaptations to a shared low-dimensional space, reducing the degrees of freedom for user-specific adaptation and enabling knowledge sharing across users, thereby reducing overfitting risk in low-interaction settings.
W1(c): Clarification and Extension of Ablation Studies
Our original intention of w/o Navigation was to examine the role of user-specific latent representations in personalization. Specifically, w/o Latent removes the latent coefficients, while w/o Navigation retains the inferred coefficients but fixes them and optimizes the LoRA factors instead. However, since w/o Navigation also changes the optimization procedure, this design may have caused confusion about its purpose and does not fully address the reviewer's concern regarding the benefits of our parameterization.
To address this concern while preserving our original experimental objective, we introduce a Joint Optimization variant that updates both the latent coefficients and LoRA factors during personalization. This allows us to further examine whether additional parameter optimization improves latent-guided personalization, providing a more comprehensive comparison of the different adaptation strategies.
The results show that Joint Optimization consistently underperforms LatentPersonal across all evaluated shot settings and metrics. This suggests that jointly optimizing the latent coefficients and LoRA factors does not necessarily improve few-shot personalization. Together with the w/o Navigation results, these comparisons provide additional evidence that personalization through inferred latent coefficients over fixed shared adaptation directions can be more effective than direct user-specific parameter optimization under limited interactions.
Overall, the key advantage of LatentPersonal lies in its shared low-dimensional adaptation space, which enables efficient few-shot personalization through compact latent coefficients rather than generating or optimizing user-specific LoRA parameters. The methodological distinction, empirical performance, and additional ablation results collectively support the effectiveness of this parameterization, particularly when user interactions are limited.