acceptodds
Under review as a conference paper at ICLR 2027

Learning Latent Preference Distributions for VAE Recommenders

Abstract

Variational autoencoder (VAE)-based collaborative filtering (CF) has emerged as a powerful framework for recommendation, efficiently inferring users' latent representations from user-item interactions without maintaining user-specific parameters. However, existing methods typically adopt the standard Gaussian prior to learn latent user representations, limiting their ability to model multifaceted user preferences. Moreover, existing VAE-based CF methods overlook potential collaborations among users, leaving sparse users with limited supervision signals and further limiting the quality of their latent representations. To address these limitations, we first theoretically analyze how Kullback-Leibler (KL) divergence regularization and reconstruction loss in vanilla VAE-based CFs constrain users' posterior distributions, limiting the overall performance of the methods. Building on these theoretical analyses, we propose LPD-VAE, a VAE-based recommendation framework designed to learn Latent Preference Distributions (LPD) and explore latent preference collaborations among users. First, we initialize a set of preference components based on the Gaussian Mixture Model prior and devise mixture-prior KL regularization to force user posterior distribution to approach these components, thereby learning each user's LPD to characterize rich user preferences. Then, based on the learned user LPDs, we reckon latent preference collaborations among users to discover potential positive samples from uninteracted items, alleviating the sparse interaction issue of VAE-based CFs. Finally, we design a preference-alignment loss supplementing to the original ELBO to encourage user latent representations to align with potential positive samples in the latent space, so as to refine user LPDs. Extensive experiments on three benchmark datasets demonstrate consistent gains over competitive SOTAs. Furthermore, experiments across users with varying interaction densities show that LPD-VAE maintains consistent performance gains under different levels of sparsity. The ablation experiment demonstrated that, compared to the interaction distance in the interaction space, the latent preference distance in the latent space is more effective in learning user preferences.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Recommender Systems, Collaborative Filtering, Variational Autoencoder

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kGwvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LPD-VAE, a VAE-based collaborative filtering framework. It addresses the problems that the standard Gaussian prior limits the diversity of user preferences and that sparse users receive little supervision. To solve this, the authors replace the prior with a Gaussian mixture, use each user’s assignment probabilities over the mixture components to find similar users, and add an alignment loss on potential positive items from these users. Experiments on three datasets show improvements when LPD is added to several VAE backbones.

Critical strengths
  • The paper proposes to find user collaborations in the latent space through mixture-prior assignments and uses them to augment sparse interactions.
  • LPD consistently improves three VAE backbones on three large benchmark datasets.
Critical weaknesses
  • The GMM prior and its ELBO directly follow VaDE (Appendix D.3 even refers to “VaDE”), and the augmentation is a neighbor-based pseudo-positive strategy. The methodological novelty is therefore limited.
  • Theorem 2.1 also holds for a GMM prior, since a GMM satisfies the T1 inequality as well, so it does not explain why the mixture prior helps. Theorem 2.2 is a standard Jensen gap result and is not used to design the alignment loss.
  • The improvements over PIA are small (0.3%–1.7%), and no standard deviations or significance tests are reported. Table 2 does not include the full model, and Multi-VAE + GMM already gives most of the gain, so the contribution of the alignment loss is unclear.
  • There is a lack of comparison with closely related baselines such as MacridVAE and DiffRec, and the paper does not report any comparison of training cost or efficiency with existing methods.
Questions for authors

Please refer to the weaknesses.

Reviewer_r7utOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper identifies key limitations of existing VAE-based recommendation models: the standard Gaussian prior lacks user-specific characteristics and overlooks user-user collaborative signals. The paper first provides a theoretical analysis of how KL divergence and reconstruction loss constrain users' posterior distributions in VAE-based collaborative filtering (CF). Building on this analysis, the paper introduces LDP-VAE, which encourages user posterior distributions to approach a Gaussian Mixture Model (GMM) prior through mixture-based KL regularization. The learned latent distribution of user preferences (LDP) is leveraged to discover collaborative signals among users, alleviating the effects of sparse interactions. Finally, a preference alignment loss is designed to encourage user latent representations to align with potential positive samples in the latent space. Experimental results demonstrate that LDP-VAE outperforms competitive baselines on benchmark datasets.

Critical strengths
  1. The paper is well-motivated and addresses an important limitation of VAE-based recommenders: the standard Gaussian prior is limited in its ability to model user preferences, motivating the need for richer prior distributions.
  2. The proposed LDP-VAE model is conceptually sound and supported by both theoretical analysis and empirical results. The GMM prior enables richer modeling of user preferences, while the discovery of user-user collaborative signals helps alleviate the effects of sparse interactions.
  3. LDP-VAE achieves stronger performance than competitive baselines on benchmark datasets, demonstrating its effectiveness.
  4. The paper includes ablation studies that provide helpful insights into the contributions of different components of the proposed method.
Critical weaknesses
  1. Despite its theoretical and empirical support, the proposed method appears incremental compared with PIA [a], which also aligns masked posteriors with user-specific targets. The GMM prior appears to be an extension of this approach, but the distinction and its advantages are not sufficiently justified. The moderate improvements over PIA reported in Table 1 warrant a more in-depth investigation.
  2. The paper lacks comparisons and discussions of closely related works [b, c, d], which integrate LLM knowledge to infer more expressive priors or learn richer distributions of user preferences. These methods provide complementary perspectives on LDP-VAE and should be considered as competitive baselines.
  3. The proposed method is evaluated only on single-interest VAE models, limiting the assessment of LDP's effectiveness. Disentangled VAE recommenders [e, f] capture multiple user-interest factors, each modeled by a distribution. Incorporating these models would help verify whether the proposed GMM prior provides additional benefits or preserves the ability to model multiple interests.
  4. The paper lacks comparisons with diffusion-based collaborative filtering models [g], which use diffusion processes to address limitations associated with the prior distributions of VAEs.
  5. No statistical significance tests are reported in Table 1, raising concerns about the robustness of the reported improvements. Furthermore, recommendation performance is evaluated at relatively large cutoffs (Top-20, Top-50, and Top-100). Reporting results at smaller cutoffs, such as Top-5 or Top-10, would further validate the models' ability to retrieve relevant items and rank them near the top of the recommendation list.
  6. The experiments in Section 4.3 do not evaluate genuine cold-start items, such as items unseen during training, limiting the assessment of the proposed method's generalizability.
  7. The paper lacks an analysis of key hyperparameters, including the number of GMM components (K), the number of top users (M), and the coefficient lambda controlling the preference alignment loss in Equation 11. A systematic sensitivity analysis would help clarify the roles of these components and their influence on model performance.

[a] On the Mechanisms of Collaborative Learning in VAE Recommenders. ICLR 2026.

[b] Towards Distribution Matching between Collaborative and Language Spaces for Generative Recommendation. SIGIR 2025.

[c] Bridging LLM Embeddings and VAE Parameters for Disentangled Recommendation. SIGIR 2026.

[d] TEARS: Text Representations for Scrutable Recommendations. TheWebConf 2025.

[e] Learning Multi-Faceted Prototypical User Interests. ICLR 2024.

[f] Learning Disentangled Representations for Recommendation. NeurIPS 2019.

[g] Diffusion Recommender Model. SIGIR 2023.

Questions for authors

Please see the review.

6 comments

We sincerely appreciate the reviewer's time and effort in evaluating our work, as well as the valuable comments and constructive suggestions. Below, we provide our responses to the concerns raised.

Question 1: Despite its theoretical and empirical support, the proposed method appears incremental compared with PIA [a], which also aligns masked posteriors with user-specific targets. The GMM prior appears to be an extension of this approach, but the distinction and its advantages are not sufficiently justified. The moderate improvements over PIA reported in Table 1 warrant a more in-depth investigation.

Response:

We thank you for the valuable comments. We acknowledge that both LPD-VAE and PIA address the limitation of VAE-CF in exploiting global collaborative information. However, our work differs fundamentally from PIA in terms of research objectives and the methods designed to address the underlying problems.

(1) Differences between LPD-VAE and PIA.

PIA models the correlations among users' observed interactions in the interaction space. In contrast, we compute each user's LPD over the GMM components in the latent space and use it to discover latent collaboration among users in that space, that is, to find items that a given user has not interacted with but may be interested in, so as to more effectively alleviate the two limitations of vanilla VAE-CF.

PIA targets the insufficient use of far-but-related users in conventional VAE-CF. It introduces an Anchor Regularizer, derived from interaction behaviors, to align user posteriors with item embeddings and thereby promote global information sharing. However, PIA focuses on aligning user posteriors with item embeddings, and this strategy remains limited in uncovering relations between users who share little interaction history but may have similar latent preferences.

LPD-VAE uses the posterior responsibilities of the GMM components to construct a user's latent preference distribution (LPD), and thereby identifies users with similar latent preferences over the global latent preference components. It then mines candidate items from the neighbors' interaction histories, filters them with the target user's decoder to obtain potential positive samples, and introduces cross-user supervision through a preference alignment loss. As a result, LPD-VAE can further establish associations between users based on latent preferences, and more efficiently exploit the collaborative information provided by users with little interaction overlap.

Finally, we emphasize that the above description of the differences between LPD-VAE and PIA will be added to the experimental analysis and related work sections of the revised manuscript.

(2) The role of the GMM prior and its relationship to the proposed mechanism.

Unlike Multi-VAE+GMM, which only replaces the standard Gaussian prior with a GMM prior, LPD-VAE further leverages the GMM posterior probability of assignments to construct LPDs and establish latent preference collaborationss among users in the latent preference space. This enables the model to overcome the limitation of prior distribution modeling alone, which cannot explicitly capture collaborative relationships between users with similar latent preferences but limited overlap in their interaction histories. By exploiting collaborative information from such users, LPD-VAE provides additional samples for users with sparse feedback.

Finally, we will incorporate this distinction between Multi-VAE+GMM and LPD-VAE into the experimental analysis section of the revised manuscript.

(3) Additional experiments and analyses.

To further distinguish the contribution of the GMM prior from that of the subsequent collaborative mechanism, and to verify the effectiveness and stability of the proposed method, we add the following experiments (detailed results are given in our responses to the corresponding comments of Reviewer EGJa). First, in the comparison of user similarity representations (Table 1), we compare, under the same mining procedure, neighbor retrieval based on zuz_u (L2L_2 and cosine), Soft-Prototype, and γu\gamma_u. γu\gamma_u performs best on both Multi-VAE and RecVAE. Second, regarding the quality of potential positives (Table 3), the proportion of selected potential positives that appear in the users’ test sets ranges from 0.095 to 0.128. Third, regarding stability and robustness, the hyperparameter sensitivity analysis shows that Recall@20 varies by no more than 0.015 (Table 2); the Jaccard similarity of the neighbor and potential positive sets across training stages reaches 0.98 after epoch 40 (Table 4); the start timing of mining has little effect on performance (Table 5); and all GMM components are used, with no component collapse (Table 7). Finally, regarding computational efficiency (Table 9), LPD takes less per-epoch training time than the corresponding PIA variants (Multi-VAE: 6.12 s vs. 10.46 s; RecVAE: 12.36 s vs. 12.57 s).

Question 2: The paper lacks comparisons and discussions of closely related works [b, c, d], which integrate LLM knowledge to infer more expressive priors or learn richer distributions of user preferences. These methods provide complementary perspectives on LDP-VAE and should be considered as competitive baselines.

Response:

We thank you for pointing out these related works. We agree that leveraging LLM semantic knowledge to enhance VAE-based recommenders and learn richer user preference representations is a valuable direction in generative recommendation, and it is related to the goals of LPD-VAE.

DMRec [b], LampVAE [c], and TEARS [d] all rely on additional textual semantic information (e.g., item descriptions, LLM-generated summaries, or LLM embeddings), whereas LPD-VAE uses only the user-item interaction matrix. Because of the different information sources, a direct comparison cannot separate gains due to the model mechanism from those due to the extra text. These methods also involve LLM text processing and different VAE architectures, making faithful reproduction and fair comparison costly, so we do not treat them as direct baselines. We select TEAR as the LLM-based recommendation backbone to evaluate the performance gains brought by LPD-VAE. TEARS uses LLM-generated user summaries to enable scrutable and controllable recommendation, so its goal is interpretability, whereas our goal is to alleviate the lack of supervision caused by interaction sparsity; the two objectives are different but they are comparable under the same metrics. We follow the same evaluation setting as TEARS (ML-1M, 5 random seeds, with summaries generated by LLaMA and GPT). The results are as follows.

Table 10: Comparison of model performance across the ML-1M datasets on NDCG and recall at k∈{20,50}. We label LLAMA models with ∞ and GPT models with ■. Each model is evaluated using five different seeds.

ModelRecall@20NDCG@20Recall@50NDCG@50
RecVAE0.297± 0.0040.265± 0.0020.358± 0.0020.272± 0.001
RecVAE+LPD0.306 ± 0.0020.268± 0.0030.367± 0.0010.280± 0.004
■ TEARS RecVAE +LPD0.309± 0.0030.271± 0.0010.375 ± 0.0020.287± 0.001
∞ TEARS RecVAE +LPD0.318 ± 0.0010.280± 0.0030.365± 0.0020.289± 0.001

The results show that LPD is also effective on ML-1M and can be combined with TEARS. First, introducing LPD into the RecVAE backbone improves all four metrics: Recall@20 and Recall@50 both increase by 0.009, NDCG@20 by 0.003, and NDCG@50 by 0.008. This shows that the gain of LPD also holds on ML-1M. Second, applying LPD on top of the TEARS-enhanced RecVAE yields overall better performance than RecVAE + LPD under both LLM summary sources. With GPT-generated summaries, the four metrics improve by 0.003, 0.003, 0.008, and 0.007, respectively; with LLaMA-generated summaries, Recall@20, NDCG@20, and NDCG@50 improve by 0.012, 0.012, and 0.009, respectively. This demonstrates that the supervision signals introduced by LPD through potential positive sample mining can further improve the recommendation performance of LLM-based models.

Question 3: The proposed method is evaluated only on single-interest VAE models, limiting the assessment of LDP's effectiveness. Disentangled VAE recommenders [e, f] capture multiple user-interest factors, each modeled by a distribution. Incorporating these models would help verify whether the proposed GMM prior provides additional benefits or preserves the ability to model multiple interests.

Response:

We thank you for this valuable suggestion. We acknowledge that comparing LPD-VAE with VAE-based recommenders that explicitly model multiple interests helps evaluate the effectiveness of combining a unified latent representation with GMM-based probabilistic mixture modeling. To this end, we added two representative disentangled VAE recommenders, MacridVAE [f] and FacetVAE [e], and compare them from three perspectives: modeling motivation, recommendation performance, and computational complexity.

(1) Modeling Motivation

Existing methods [e, f] capture the multi-interest structure of users by explicitly decomposing the user latent representation (e.g., modeling a separate distribution for each interest factor). LPD-VAE takes a different approach: it keeps a unified user latent representation, constructs each user’s latent preference distribution from the GMM posterior responsibilities, and establishes collaborative relations among users based on the similarity of these distributions. The GMM responsibilities are soft assignments over the mixture performance components, so a user with several interests will have high probabilities on multiple components. The resulting distribution therefore captures the user’s multi-interest structure in the form of a mixture probability. At the same time, LPD does not need to explicitly build and maintain multiple interest representations, which avoids the extra interest-decomposition mechanism and the extra model complexity.

(2) Recommendation Performance

On ML-20M, MacridVAE achieves Recall@20 = 0.396, Recall@50 = 0.529, and NDCG@100 = 0.425, while FacetVAE achieves Recall@20 = 0.415, Recall@50 = 0.551, and NDCG@100 = 0.443. Both are outperformed by RecVAE + LPD (Recall@20 = 0.423, Recall@50 = 0.562, NDCG@100 = 0.449). These results show that, even without explicitly constructing multiple interest representations, LPD-VAE can still achieve better recommendation performance by modeling latent preferences as a GMM-based mixture probability and building user collaboration from the similarity between preference distributions.

(3) Computational Complexity

Given the latent representations of a mini-batch of BB users, computing their GMM responsibilities requires O(BKd)O(BKd) time, where KK is the number of mixture components and dd is the latent dimension. We randomly sample SS users as query users and compute their L1L_1 distances to all BB users in the current mini-batch, resulting in a complexity of O(BKd)O(BKd). Selecting the top-(M) nearest neighbors requires at most O(SBlog⁡B)O(SB\log B) time using a sorting-based upper bound. Therefore, the main cost of the batch-wise LPD neighbor search is O(SBK+SBlog⁡B)O(SBK+SB\log B).

In addition, we compared the actual training time of different models to evaluate the computational overhead introduced by LPD. The detailed complexity analysis and training-time comparison are provided in our response to Reviewer EGJa, Question 7.

Reviewer_EGJaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Summary

This paper proposes LPD-VAE, a VAE-based collaborative filtering framework that replaces the standard Gaussian prior with a Gaussian Mixture Model (GMM) prior and uses the resulting component responsibilities to construct a Latent Preference Distribution (LPD) for each user.

For a user uu, the encoder still produces a single Gaussian posterior qϕ(z∣xu)=N(μu,diag⁡(σu2))q_\phi(z\mid x_u)=\mathcal N(\mu_u,\operatorname{diag}(\sigma_u^2)). Given a latent representation zuz_u, LPD-VAE computes the responsibility of each global GMM component as γu(c)=p(c∣zu)\gamma_u(c)=p(c\mid z_u), and represents the user by the vector γu=[γu(1),…,γu(K)]\gamma_u=[\gamma_u(1),\ldots,\gamma_u(K)].

The method measures user similarity using the L1L_1 distance between LPD vectors, retrieves the MM nearest users, collects items interacted with by these users but not by the target user, and uses the current decoder to select the top HH candidates as pseudo-positive samples. These pseudo-positives provide an additional preference-alignment loss on top of the GMM-based VAE ELBO.

The paper also provides theoretical analyses arguing that a shared prior can restrict posterior diversity and that reconstruction loss can discourage posterior overlap for users with different interaction patterns.

The implemented object appears closer to a soft assignment of a single user representation to global latent prototypes. Under this interpretation, the method is more naturally understood as a combination of GMM-VAE, soft prototype representation, user KNN, and pseudo-positive self-training.

Critical strengths

Strengths

  1. Latent-space collaboration is an interesting direction. The paper attempts to discover collaborative signals beyond direct overlap in sparse interaction vectors.

  2. Using a more flexible prior is reasonable. Replacing a single Gaussian prior with a mixture prior provides additional flexibility in organizing the latent space, and the GMM-only ablation suggests that this modification can provide some benefit independently.

  3. The paper attempts to analyze the mechanism theoretically. The analyses of KL regularization and reconstruction-induced posterior overlap provide motivation for possible limitations of vanilla VAE-CF, even though they do not fully justify the specific LPD-VAE design.

Critical weaknesses

Weaknesses

  1. The central claim that LPD represents a user-specific multifaceted preference distribution is not sufficiently established.

    The user-specific posterior remains a single Gaussian, qϕ(z∣xu)=N(μu,diag⁡(σu2))q_\phi(z\mid x_u)=\mathcal N(\mu_u,\operatorname{diag}(\sigma_u^2)). The proposed LPD is instead the responsibility vector of one latent representation over several global GMM components, γu(c)=p(c∣zu)\gamma_u(c)=p(c\mid z_u).

    This appears closer to soft cluster membership or soft prototype assignment than to a genuine user-specific multimodal preference distribution. The model does not learn multiple user-specific latent modes or multiple interest representations, and prediction still relies on a single latent representation through the VAE decoder.

    Therefore, the key conceptual step from "soft membership over global GMM components" to "multifaceted user preference distribution" needs stronger justification. The lack of semantic, structural, or stability analysis of the GMM components further weakens this interpretation.

  2. The necessity of the GMM-derived LPD is not isolated from simpler alternatives.

    The GMM plays two roles: providing a more flexible prior and mapping a continuous user representation into a KK-dimensional soft assignment vector for KNN. The GMM-only ablation partially evaluates the first role, but the second is not isolated.

    A crucial missing baseline is to retain exactly the same neighbor mining, candidate construction, decoder filtering, and preference-alignment pipeline, but perform KNN directly on the encoder latent representation, for example using ∥μu−μv∥2\|\mu_u-\mu_v\|_2 or cosine similarity.

    The existing dxd_x variant is insufficient because it compares LPD-space retrieval with sparse raw interaction-space retrieval rather than with another learned latent representation.

    Another useful baseline is a simpler soft-prototype representation such as au=softmax⁡(Wzu/τ)a_u=\operatorname{softmax}(Wz_u/\tau), followed by the same KNN and pseudo-positive pipeline. This would test whether the Gaussian-specific means, variances, and mixture weights provide additional value beyond a generic soft categorical/prototype representation.

    Without these controls, it is unclear whether the gains come from the flexible prior, latent-space KNN, a prototype bottleneck, or specifically from GMM responsibilities.

  3. The pseudo-positive mining procedure creates a potentially serious self-reinforcement loop.

    The same model produces zuz_u, constructs γu\gamma_u, retrieves neighbors, collects candidate items, uses its own decoder πi(zu)\pi_i(z_u) to select the top HH pseudo-positives, and is then trained to further increase the scores of those selected items.

    This creates a feedback loop of the form current model belief→pseudo labels→reinforcement of current model belief\text{current model belief} \rightarrow \text{pseudo labels} \rightarrow \text{reinforcement of current model belief}.

    If early latent representations, GMM assignments, or decoder predictions are wrong, the procedure may reinforce these errors. Small MM, HH, and λ\lambda may reduce the influence of noisy pseudo-labels, but they are heuristic controls rather than evidence that the loop is stable.

    This should be validated through sensitivity to MM, HH, λ\lambda, and KK; pseudo-positive precision or held-out hit rate; neighbor and pseudo-label stability across epochs; robustness across random seeds; robustness to injected pseudo-label noise; or comparisons with warm-up or teacher-based variants.

  4. The theoretical analysis motivates limitations of vanilla VAE-CF, but does not derive the concrete LPD-VAE algorithm.

    The theory supports the general observations that a shared prior can restrict posterior diversity and that reconstruction can penalize posterior overlap between users with different interaction statistics. However, it does not establish why the solution should specifically be GMM→γu→L1-KNN→neighbor items→decoder Top-H→pseudo-positive loss\text{GMM} \rightarrow \gamma_u \rightarrow L_1\text{-KNN} \rightarrow \text{neighbor items} \rightarrow \text{decoder Top-}H \rightarrow \text{pseudo-positive loss}.

    In particular, the theory does not establish why GMM responsibilities are the correct preference representation, why L1L_1 distance is the appropriate similarity metric, or why neighbors selected under this metric provide reliable positive items. The theory therefore motivates the general problem more than the particular algorithm.

  5. GMM initialization and component stability are insufficiently specified and validated.

    The method jointly trains the encoder, decoder, and GMM parameters {ωc,μc,σc2}c=1K\{\omega_c,\mu_c,\sigma_c^2\}_{c=1}^K, but the paper does not clearly describe how component means, variances, and mixture weights are initialized, whether a pretrained VAE or warm-up stage is used, or whether pseudo-positive mining starts immediately.

    This matters because early GMM assignments directly determine user neighbors and therefore affect later pseudo-label supervision.

    The paper also lacks diagnostics such as component utilization, mixture-weight distribution, assignment entropy, component separation, effective number of active components, and stability across random seeds. These analyses are important because the main semantic interpretation depends on the GMM components representing meaningful and stable preference structure.

  6. The computation of LPD for user retrieval is not sufficiently clear.

    The paper defines γu(c)=p(c∣zu)\gamma_u(c)=p(c\mid z_u), while zuz_u comes from a stochastic variational posterior. It is unclear whether KNN uses one sampled zuz_u, the posterior mean μu\mu_u, or an expectation over multiple samples.

    If a stochastic sample is used, the resulting LPD and nearest-neighbor set may themselves be unstable. Since these neighbors determine the pseudo-positive supervision, this is an important implementation detail.

  7. The scalability of the global user-KNN stage is unclear.

    The algorithm repeatedly computes LPDs and retrieves nearest users on datasets containing hundreds of thousands of users. A naive implementation would require pairwise comparisons with complexity on the order of O(U2K)O(U^2K). The paper should clarify how this stage is implemented efficiently and report its additional training-time and memory cost.

Questions for authors

Questions for Authors

1 How is γu\gamma_u computed for KNN in practice during inference time? Is it computed from one sampled zuz_u, from the posterior mean μu\mu_u, or by averaging over multiple samples?

2 Please also address the concerns in the weakness part.

9 comments

We sincerely appreciate the reviewer's time and effort in evaluating our work, as well as the constructive feedback and valuable suggestions. Below, we present our responses to the points raised and describe the corresponding revisions.

Question 1: The central claim that LPD represents a user-specific multifaceted preference distribution is not sufficiently established.

Response:

We thank you for the thoughtful comments. We agree with that the technical characterization of LPD: the user's variational posterior qϕ(z∣xu)=N(μu,diag(σu2))q_\phi(z\mid x_u)=\mathcal{N}(\mu_u,\mathrm{diag}(\sigma_u^2)) remains a single Gaussian. LPD is the posterior probability of assignment γu(c)=p(c∣zu)\gamma_u(c)=p(c\mid z_u) of this latent representation over the global GMM components. It is a probability vector that is closer to soft prototype assignment than to a user-specific multimodal preference distribution. We acknowledge that the description of LPD in the paper is not sufficiently precise, and we will revise it in our manuscript. At the same time, we would like to explain the motivation behind this design and the theoretical and empirical support for it.

(1) The role of the mixture prior. Although the user posterior is a single Gaussian, the prior is a GMM composed of multiple preference components. The objective includes a novel KL divergence term with respect to the mixture prior, which no longer pulls all users toward a single shared prior, but allows each user's latent representation to align with components close to their preferences, thereby preserving finer-grained personalized information. Theorem 2.1 in Section 2 proves that aligning all users to a single shared prior restricts posterior diversity and compresses user preference representations, which limits the recommendation performance. Introducing multiple preference components is intended to alleviate this problem, and LPD records the degree to which a user belongs to each preference component and is used to retrieve users with similar preferences.

(2) Comparison with multi-factor models. Our goal is not to learn multiple interest representations for each user, but use global mixture components to characterize preference patterns shared across users, then examine the role of such a representation in mining user interactions. Furhter, following the suggestion of Reviewer r7ut, we added two explicit multi-factor disentangled VAE methods, MacridVAE and FacetVAE, for comparison. On MovieLens-20M, both (Recall@20 of 0.396 and 0.415, respectively) are lower than RecVAE + LPD (0.423). The full results, modeling motivation, and complexity analysis are provided in our response to Question 3 of Reviewer r7ut. In addition, Table 1 also shows that, under the same mining procedure, the user retrieval based on γu\gamma_u consistently outperforms retrieval based on zuz_u, indicating that the latent preference space defined by the GMM is better suited for the neighbor retrieval task than the original latent space.

(3) Structural and stability analysis of the GMM components. Regarding your concern about the lack of component analysis, we have added, in our response to Question 5, the utilization of each component (also see Table 7: all components are used and no component collapse occurs) and the stability of the neighbor and potential positive sets during training (Table 4). Overall, the GMM components remain in use throughout training without collapse, and the assignments they induce become stable as training proceeds.

(4) Revisions to the paper. We will adjust the description of LPD in the introduction to be consistent with that in the contributions of our manuscript, and clarify that LPD is a probability vector over the global GMM preference components aiming at characterizing rich user preferences rather than a user-specific multimodal distribution.

Question 2: The necessity of the GMM-derived LPD is not isolated from simpler alternatives.

Response:

We thank you for raising this important point. We acknowledge that the ablation in the original submission does not fully separate the two roles of the GMM: providing a more flexible prior, and mapping a continuous user representation into a K-dimensional soft assignment vector for neighbor retrieval. Comparing only with Multi-VAE + GMM evaluates the first role partially, and is insufficient to establish the necessity of the GMM-derived LPD over other user-similarity representations.

To further distinguish the roles of different user-similarity representations, we added three control methods. The first two are Latent-KNN (L2) and Latent-KNN (Cosine), which retrieve neighbors directly on the user’s continuous latent representation zuz_u using Euclidean distance and cosine similarity, respectively. The third one is Soft-Prototype KNN, which introduces learnable prototype vectors, converts the inner products between the user latent representation and the prototypes into a K-dimensional soft assignment vector via softmax, and retrieves neighbors based on this vector. All control methods use exactly the same neighbor retrieval process, candidate item set construction, decoder filtering, and preference alignment pipeline as LPD setting, and only differ in the user-similarity representation and its calculation, which ensures a fair comparison.

Table 1: Comparison of Different User Similarity Representations on MovieLens-20M.

VAE BackboneUser Similarity MethodRecall@20Recall@50NDCG@100
Multi-VAELatent-KNN (L2)0.4020.5380.432
Multi-VAELatent-KNN (Cosine)0.4050.5440.436
Multi-VAESoft-Prototype KNN0.4080.5460.436
Multi-VAELPD0.4100.5500.439
RecVAELatent-KNN (L2)0.4150.5540.443
RecVAELatent-KNN (Cosine)0.4170.5560.445
RecVAESoft-Prototype KNN0.4200.5590.447
RecVAELPD0.4230.5620.449

With both backbones, LPD achieves the best results on all three metrics. First, LPD outperforms Latent-KNN (L2 and Cosine), which retrieves neighbors directly on continuous latent representations, indicating that computing user similarity on the latent representation alone has limitations. Second, LPD also consistently outperforms Soft-Prototype KNN, showing that the gain does not come merely from converting a continuous representation into a K-dimensional soft assignment vector. Compared with Soft-Prototype KNN, the GMM-derived LPD further exploits the means, variances, and mixture weights of the preference components, and computes each user’s posterior responsibilities from the Gaussian mixture density. This yields a more effective user-similarity representation, which improves the subsequent neighbor retrieval and potential positive samples mining.

Question 3: The pseudo-positive mining procedure creates a potentially serious self-reinforcement loop.

Response:

We thank you for pointing out the potential feedback bias and stability issues in the potential positive mining procedure. We first clarify the terminology and design goal of our work, then explain the mining procedure in more detail, and finally analyze its dependence on key hyperparameters through additional experiments.

(1) Definition and design goal of potential positive samples. Following the prior work [1], we define items that a user has not yet interacted with but may be interested in as potential positive samples. Our purpose is to leverage the historical interactions of similar users to supplement the target user's limited feedback, thereby alleviating the data sparsity problem in VAE-CF. We emphasize that these samples represent potential user preferences inferred from similar users, rather than pseudo-positive labels assigned to unobserved items. They are incorporated as auxiliary preference signals alongside observed interactions to help the model capture users' latent preferences.

[1] Beyond instance-level alignment and uniformity: Semantic factor learning for collaborative filtering. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, KDD' 26.

(2) Mining procedure and stability validation of potential positive samples. The mining procedure is designed to progressively narrow down a large pool of uninteracted items and identify the most likely potential positives. It consists of four steps. First, the encoder produces the user latent representation zuz_u, from which we compute the posterior assignment probabilities γu\gamma_u over the GMM components; Second, based on γu\gamma_u, we retrieve MM neighbors within the mini-batch. The candidates are the items that these neighbors have interacted with but the target user has not. Third, we rank the candidates by the decoder score πi(zu)\pi_i(z_u) and keep the top HH as potential positives. Finally, we increase the predicted scores of these potential positives with weight λ\lambda.

We agree that the current model's predictions influence the selection of potential positives, which in turn participate in subsequent training. Accordingly, we incorporate two constraints into the design: first, candidate items are collected only from the real interaction records of similar users; second, the model is always optimized with the reconstruction loss on real interactions, and the potential positives only provide an additional auxiliary preference signal. On this basis, we further examine the stability of this procedure during training through the following experiments:

Table 2: Study on hyperparameter sensitivity on MovieLens-20M for Multi-VAE+LPD.

ParameterValueRecall@20Recall@50NDCG@100
MM10.3960.5350.428
30.4020.5360.431
50.4100.5500.438
100.4010.5380.432
HH10.3990.5370.431
30.4080.5480.439
60.4010.5370.434
100.3980.5350.427
λ\lambda0.010.3950.5370.428
0.10.4000.5380.430
0.50.4100.5490.439
10.4020.5360.431
KK50.4030.5380.432
100.4090.5500.438
200.3980.5370.431
500.3950.5330.424

Due to space limitations, the submitted text reports only the best hyperparameter settings. Table 2 presents the sensitivity analysis of MM, HH, λ\lambda, and KK. Within the tested ranges, the variation in Recall@20 never exceeds 0.015, and the fluctuations in Recall@50 and NDCG@100 are similarly small. This indicates that the performance of LPD-VAE is insensitive to changes in these hyperparameters, and that the potential positive samples mining procedure is reasonably robust.

Table 3: Quality analysis of mined potential positive samples on MovieLens-20M for Multi-VAE+LPD.

MMHHλ\lambdaKKHit Rate@10
310.0150.095
530.0150.109
530.150.115
530.5100.128
531100.103
530.5200.117
1030.5100.112
1060.5100.106

To evaluate the quality of the mined potential positive samples, we calculate the proportion of selected samples that appear in the users' test sets, denoted as Hit Rate@10. Table 3 reports the results under different hyperparameter settings. Across the evaluated configurations, the hit rate ranges from 0.095 to 0.128. The highest hit rate (0.128) is achieved with (MM =5), (HH=3), (λ\lambda=0.5), and (KK=10), which is consistent with the setting that yields the best recommendation performance in Table 2. These results indicate that the mined potential positive samples overlap with a certain proportion of ground-truth interactions unseen during training, and that their quality varies consistently with the final recommendation performance.

Table 4: Stability of neighbor retrieval and potential positive sample mining across training epochs on MovieLens-20M (Multi-VAE+LPD). Each entry reports the average Jaccard similarity between the corresponding sets obtained at the two compared epochs. Higher values indicate greater temporal consistency.

Epoch comparisonNeighbor JaccardPotential positive Jaccard
1 vs. 100.720.65
10 vs. 200.850.81
20 vs. 300.930.84
30 vs. 400.970.96
40 vs. 500.980.98
50 vs. 600.980.98

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.