acceptodds
Under review as a conference paper at ICLR 2027

Debiasing Message Passing to Mitigate Popularity Bias in GNN-based Collaborative Filtering

Abstract

Collaborative filtering (CF) models based on graph neural networks (GNNs) achieve strong performance in recommender systems by propagating user-item signals over interaction graphs. However, they are susceptible to popularity bias, since skewed interactions and repeated message passing across high-order neighborhoods amplify the influence of popular items while suppressing long-tail ones. Existing debiasing approaches, including re-weighting objectives, regularization, causal methods, and post-processing, are less effective in GNN-based settings because they do not directly counteract bias propagated through the aggregation process, and recent in-aggregation weighting methods often rely on static heuristics or unstable embedding estimates. We propose *Debiasing Popularity Amplification in Aggregation* (DPAA), a popularity debiasing framework for GNN-based CF that integrates adaptive, representation-aware interaction weighting and layer-wise weighting directly into message passing. DPAA assigns interaction-level weights from a representation-based popularity signal, stabilized by a smooth transition from pre-trained to evolving model embeddings during training. It further introduces a layer-wise weighting that amplifies higher-order neighborhoods, surfacing long-range interactions with diverse and underexposed items. Experiments on real-world and semi-synthetic datasets show that DPAA outperforms state-of-the-art popularity bias correction methods for GNN-based CF.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
graph neural networks, collaborative filtering, popularity bias

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_U2nROpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates popularity bias amplification during message passing in GNN-based collaborative filtering and proposes Debiasing Popularity Amplification in Aggregation (DPAA) to address this issue. DPAA combines adaptive interaction-level weighting based on cosine similarity with layer-wise weighting that emphasizes higher-order collaborative signals, while incorporating pre-trained embeddings to stabilize weight estimation. Experiments on three real-world datasets and semi-synthetic data demonstrate improvements in recommendation accuracy under varying levels of popularity bias.

Critical strengths
  1. The proposed framework directly addresses popularity amplification during GNN message passing by jointly incorporating interaction-level and layer-wise weighting mechanisms.
  2. The evaluation on datasets with unbiased test sets, together with semi-synthetic experiments under varying levels of popularity bias, provides evidence of the method's effectiveness across different bias conditions.
Critical weaknesses
  1. The proposed IIW estimates interaction strength using cosine similarity between L2-normalized embeddings, which discards embedding magnitude information that may encode item popularity [1]. I am concerned that the paper does not sufficiently justify this design choice or compare it with alternative interaction-weighting strategies that preserve magnitude information.
  2. Although the paper reports separate recommendation results for popular and long-tail items, I remain concerned about whether DPAA effectively reduces popularity dominance in the final learned representations. Further analysis of the popularity direction in item embeddings [2] and more detailed long-tail recommendation results [3] would strengthen the empirical evidence.
  3. The paper should compare DPAA with more recent baseline methods.

[1] Donald et al. On the Role of Weight Decay in Collaborative Filtering: A Popularity Perspective. SIGKDD'25.

[2] Liu et al. Rethinking Popularity Bias in Collaborative Filtering via Analytical Vector Decomposition. SIGKDD'26.

[3] Wu et al. Self-supervised graph learning for recommendation. SIGIR'21.

Questions for authors
  1. Could the authors explain why cosine similarity is preferred for estimating interaction strength, given that embedding magnitudes may contain popularity-related information? Further comparison with other weighting strategies is necessary.
  2. Could the authors further analyze how DPAA affects the popularity direction in the final item embeddings and provide more detailed results on long-tail items?
7 comments
AuthorsOpenReview ↗

Rebuttal Part 1

Thank you for your valuable comments. We have provided point-wise responses below and updated the paper where necessary, with changes highlighted in blue.

W1. The proposed IIW estimates interaction strength using cosine similarity between L2-normalized embeddings, which discards embedding magnitude information that may encode item popularity [1]. I am concerned that the paper does not sufficiently justify this design choice or compare it with alternative interaction-weighting strategies that preserve magnitude information.

Q1: Could the authors explain why cosine similarity is preferred for estimating interaction strength, given that embedding magnitudes may contain popularity-related information? Further comparison with other weighting strategies is necessary.

We chose cosine similarity for estimating interaction strength in IIW for the following reasons:

Normalized and bounded interaction weighting: L2 normalization projects user and item embeddings onto the unit sphere, bounding cosine similarity within [-1, 1] and the corresponding IIW within [0, 2]. This ensures consistent scaling of interaction weights and prevents high-norm embeddings from disproportionately influencing similarity estimates due to their magnitudes. In contrast, directly using the dot product produces unbounded similarity scores whose scales vary across interactions, potentially allowing certain interactions to dominate message passing and representation learning. Cosine similarity avoids this issue by ensuring that interaction weights remain bounded and comparable across user–item pairs.

Empirical relationship between cosine similarity and item popularity: As shown in Figure 5 in the Appendix, by grouping items into different popularity quartiles, we observe that more popular item groups generally exhibit higher user–item cosine similarity. Therefore, cosine similarity retains a useful signal for identifying popularity-associated interactions while keeping the scale consistent, from which we assign smaller aggregation weights to interactions with higher user–item similarity.

Empirical comparison with alternative weighting strategies: To further validate our design choice, we conducted additional experiments using two alternative interaction-weighting strategies: (i) raw dot-product similarity, which preserves embedding magnitude information, and (ii) sigmoid-transformed dot-product similarity, which bounds the resulting similarity scores while retaining embedding magnitude information. We replaced cosine similarity with each alternative in the IIW computation while keeping the remaining components of DPAA unchanged. The results show that cosine similarity achieves the best recommendation performance across all datasets, followed by sigmoid-transformed dot-product similarity.

Table: Performance comparison of DPAA using different similarity functions for estimating interaction-level weights (IIW).

DatasetMethodRecall@20NDCG@20HR@20
CoatDPAA (Cosine similarity)0.25650.20650.6234
DPAA (Sigmoid)0.25600.20530.6163
DPAA (Dot product)0.25550.20490.6123
KuaiRecDPAA (Cosine similarity)0.03490.05860.6201
DPAA (Sigmoid)0.02800.03250.3228
DPAA (Dot product)0.02210.02410.2598
Yahoo! R3DPAA (Cosine similarity)0.14790.06590.2221
DPAA (Sigmoid)0.14230.06500.2165
DPAA (Dot product)0.13910.06430.2093

Raw dot-product similarity does not produce good performance, particularly on KuaiRec and Yahoo! R3. This degradation may be attributed to the unbounded interaction weights produced by raw dot-product similarity. We have included this analysis in Section 4.3 of the main body and Appendix A.5.4 of the revised version, with the additions highlighted in blue.

AuthorsOpenReview ↗

Rebuttal Part 2

W2. Although the paper reports separate recommendation results for popular and long-tail items, I remain concerned about whether DPAA effectively reduces popularity dominance in the final learned representations. Further analysis of the popularity direction in item embeddings [2] and more detailed long-tail recommendation results [3] would strengthen the empirical evidence.

Q2. Could the authors further analyze how DPAA affects the popularity direction in the final item embeddings and provide more detailed results on long-tail items?

Regarding popularity dominance, we examine whether DPAA reduces popularity dominance in final item embeddings and analyze their alignment with the popularity direction, following [2]. Specifically, we define the popularity direction as the normalized difference between the mean embeddings of popular (head) and unpopular (tail) items. We then project each item's final embedding onto this direction and compute the Pearson and Spearman correlations between these projections and item popularity, measured by training interaction frequency. We compare these correlations between DPAA and its LightGCN backbone, which does not correct for popularity bias, to assess whether DPAA reduces popularity-related alignment in the learned representations. Lower absolute values indicate weaker alignment between item popularity and the embeddings projections onto the popularity direction, suggesting reduced popularity dominance in the learned representations.

Table: Comparison of popularity-direction correlations in final item embeddings between DPAA and LightGCN. Lower absolute values indicate weaker alignment between item popularity and final item representations.

DatasetMethodPearson correlation coefficient ↓Spearman correlation coefficient ↓
CoatDPAA0.35910.2941
LightGCN0.39040.3094
KuaiRecDPAA0.65170.6922
LightGCN0.95490.9701
Yahoo! R3DPAA0.80790.7247
LightGCN0.88050.8327

Across all three datasets, DPAA achieves lower Pearson and Spearman correlation coefficients than the LightGCN backbone, indicating weaker alignment between item popularity and the final item representations learned by DPAA. These findings provide empirical evidence that DPAA reduces popularity dominance in learned item representations. We have included this analysis in Section 4.3 of the main body and Appendix A.5.5 of the revised version, with the additions highlighted in blue.

Regarding more detailed analysis of the long-tail items, similar to [3], we evaluate DPAA across different item popularity levels based on interaction frequencies and dividing them into four quartiles. Q4 contains the most-popular items accounting for 25% of interactions, followed by Q3 and Q2, while Q1 contains the least-popular items accounting for the remaining 25%. DPAA generally outperforms the corresponding best-performing baselines across different popularity groups. These results demonstrate that DPAA maintains strong recommendation performance for highly popular items while also improving performance across different long-tail item popularity groups. We have included this analysis in RQ3 in Section 4.2 of the main body and Appendix A.4.5 of the revised version, with the additions highlighted in blue.

Table: Performance comparison between DPAA and the best-performing baseline across item popularity quartiles. Each quartile accounts for approximately 25% of total training interactions, with Q4 comprising the most-popular and Q1 comprising the least-popular items.

DatasetQuartileMethodRecall@20NDCG@20HR@20
CoatQ4DPAA0.56050.33350.6005
PRISM0.47370.29780.4945
Q3DPAA0.31840.17490.4034
PRISM0.27340.16340.3621
Q2DPAA0.16790.13140.2748
PRISM0.16650.12750.2621
Q1DPAA0.10820.09460.1794
PRISM0.11010.09820.1864
KuaiRecQ4DPAA0.07640.06290.4756
PPAC0.05990.04890.4726
Q3DPAA0.00240.00260.0321
PPAC0.00000.00000.0000
Q2DPAA0.00030.00060.0076
PPAC0.00000.00000.0000
Q1DPAA0.00020.00030.0008
PPAC0.00000.00000.0000
Yahoo! R3Q4DPAA0.65090.30970.6613
NAVIP0.64500.29370.6590
Q3DPAA0.33170.13390.3449
NAVIP0.31060.13080.3267
Q2DPAA0.11310.03830.1239
NAVIP0.10090.03640.1121
Q1DPAA0.01960.00560.0225
NAVIP0.01940.00620.0233
AuthorsOpenReview ↗

Rebuttal Part 3

W3. The paper should compare DPAA with more recent baseline methods.

Regarding the baselines, we compare our method against representative popularity-debiasing methods from major methodological families. In the table below, we additionally include two of the most recent popularity-debiasing baselines, PRISM [1] and DDC [2], to provide a more up-to-date comparison with DPAA. DPAA continues to outperform the newly added, most recent baselines across all three datasets. We have incorporated these additional results into Table 1 of the revised version, with the additions highlighted in blue.

Table: Recommendation performance on three real-world datasets compared with two additional recent popularity debiasing methods.

DatasetMethodRecall@20NDCG@20HR@20
CoatDPAA (Ours)0.25650.20650.6234
DDC0.23730.17340.6085
PRISM0.25010.20050.6112
KuaiRecDPAA (Ours)0.03490.05860.6201
DDC0.01570.01680.2469
PRISM0.01810.01540.2342
Yahoo! R3DPAA (Ours)0.14790.06590.2221
DDC0.13610.06080.2065
PRISM0.12360.05990.1908

[1] Donald et al. On the Role of Weight Decay in Collaborative Filtering: A Popularity Perspective. SIGKDD'25.
[2] Liu et al. Rethinking Popularity Bias in Collaborative Filtering via Analytical Vector Decomposition. SIGKDD'26.

Reviewer_T3w8OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates popularity bias in Graph Neural Network-based Collaborative Filtering, where message passing tends to amplify signals from high-degree popular items across neighborhoods. To address this, the authors propose DPAA. DPAA introduces an interaction-level weighting mechanism based on normalized embedding similarity, stabilized by a transition from pre-trained to evolving embeddings, alongside a layer-wise weighting mechanism to amplify higher-order neighborhood signals. The method also incorporates an initial residual connection to preserve node-specific information. The authors validate the method on three real-world datasets with unbiased test sets and a semi-synthetic dataset with controllable bias levels.

Critical strengths

The experiments evaluate the method on real-world datasets with unbiased test splits and introduce a controllable semi-synthetic setup to examine robustness across different popularity bias severities.

Critical weaknesses
  1. The paper cites Zhou et al. (2023) extensively, and the core concept of downweighting message passing based on user-item embedding similarity is directly adopted from APDA. The main additions in DPAA appear to be a pre-training convex combination for warm-up, a polynomial layer weight, and an initial residual connection borrowed from GCNII. This makes the contribution look like an engineering combination of heuristics rather than a fundamentally distinct methodology. The conceptual gap between DPAA and APDA is not clearly articulated.

  2. The paper does not directly validate its claims about controlling over-correction and over-smoothing. These claims are central to the motivation and contribution statements, but the evaluation mainly reports recommendation performance and component ablations. The results do not directly show how DPAA changes representation smoothing, popularity amplification, or the trade-off between correcting bias and preserving useful collaborative signals. Without such diagnostics, it remains unclear whether the proposed mechanisms achieve the claimed effects or simply improve the final ranking metrics on the selected datasets.

Questions for authors
  1. Could the authors explicitly clarify the fundamental methodological difference between DPAA and APDA (Zhou et al., 2023)? Specifically, if APDA is supplemented with the same pre-trained warm-up and residual connection, how much unique advantage remains in DPAA?

  2. Could you provide empirical evidence showing that DPAA effectively mitigates over-smoothing? For instance, you could report performance comparisons across different numbers of aggregation layers (such as increasing propagation depth from 2 to 8 layers) to demonstrate whether DPAA resists performance degradation better than the backbone models.

5 comments
AuthorsOpenReview ↗

Rebuttal Part 1

Thank you for your valuable comments. We have provided point-wise responses below and updated the paper where necessary, with changes highlighted in blue.

W1. The paper cites Zhou et al. (2023) extensively, and the core concept of downweighting message passing based on user-item embedding similarity is directly adopted from APDA. The main additions in DPAA appear to be a pre-training convex combination for warm-up, a polynomial layer weight, and an initial residual connection borrowed from GCNII. This makes the contribution look like an engineering combination of heuristics rather than a fundamentally distinct methodology. The conceptual gap between DPAA and APDA is not clearly articulated.

Q1. Could the authors explicitly clarify the fundamental methodological difference between DPAA and APDA (Zhou et al., 2023)? Specifically, if APDA is supplemented with the same pre-trained warm-up and residual connection, how much unique advantage remains in DPAA?

General principle of interaction reweighting: While DPAA shares the high-level idea of interaction weighting with APDA, the underlying concept of interaction reweighting originates from applying inverse propensity weighting (IPW) to unbiased learning-to-rank [1], where interactions are weighted inversely proportional to their popularity or exposure probabilities in the objective function to mitigate various forms of bias. Numerous propensity-based debiasing methods (e.g., [2, 3, 4]) across different types of recommender systems build upon this common principle while introducing methodological contributions to address the limitations of earlier propensity-based approaches. Similarly, DPAA builds upon the general idea of interaction weighting while addressing limitations of existing aggregation-based debiasing methods for GNN-based collaborative filtering.

Methodological differences between DPAA and APDA: The conceptual gap between DPAA and APDA is in interaction-weight estimation, accounting for layer-wise sources of popularity bias, unified weighting, and controlling under- and over-correction. First, APDA estimates interaction weights from evolving embeddings, which can be unreliable during early training. DPAA addresses this by initializing weights from stable pre-trained embeddings and gradually transitioning to current-model estimates, improving their stability, especially during early training epochs. Second, unlike APDA, which does not account for layer-wise sources of popularity bias in shallow layers, DPAA addresses this limitation by integrating layer-wise weighting with interaction-level weighting in a unified formulation. Furthermore, APDA applies the same interaction weights across all layers, potentially over-correcting popularity bias for genuinely relevant popular items. DPAA addresses this by flexibly applying interaction weight either at the initial layer or across all layers to balance under- and over-correction. We further demonstrate the importance of each component through the ablation studies presented in the paper.

Additional empirical comparison with APDA: We have conducted additional experiments by supplementing APDA with the same combination of pre-trained and current-model interaction weights and an initial residual connection, denoted as APDA-PRC. The results show that APDA-PRC generally improves upon APDA by incorporating DPAA's interaction-weight estimation strategy. However, DPAA still achieves the best performance, demonstrating the additional benefits of its layer-wise weighting mechanism, unified formulation of IIW and LW, and control over applying the IIW across propagation layers. We have included this analysis in Section 4.3 of the main body and Appendix A.5.6 of the revised version, with additions highlighted in blue.

Table: Performance comparison of DPAA, APDA, APDA-PRC (APDA with IIW and residual connection). APDA-PRC extends APDA by incorporating pre-trained and current-model interaction weights, as used in DPAA, along with the residual connection.

DatasetMethodRecall@20NDCG@20HR@20
CoatDPAA (Ours)0.25650.20650.6234
APDA0.15870.07450.4626
APDA-PRC0.15940.07580.4698
KuaiRecDPAA (Ours)0.03490.05860.6201
APDA0.01270.01130.1082
APDA-PRC0.01590.02030.1834
Yahoo! R3DPAA (Ours)0.14790.06590.2221
APDA0.14140.06440.2111
APDA-PRC0.14290.06460.2123

[1] Joachims et al. Unbiased learning-to-rank with biased feedback. WSDM'17.
[2] Ai et al. Unbiased learning to rank with unbiased propensity estimation. SIGIR'18.
[3] Luo et al. Model-based unbiased learning to rank. WSDM'23.
[4] Luo et al. Unbiased learning-to-Rank needs unconfounded propensity estimation. SIGIR'24.

AuthorsOpenReview ↗

Rebuttal Part 2

W2. The paper does not directly validate its claims about controlling over-correction and over-smoothing. These claims are central to the motivation and contribution statements, but the evaluation mainly reports recommendation performance and component ablations. The results do not directly show how DPAA changes representation smoothing, popularity amplification, or the trade-off between correcting bias and preserving useful collaborative signals. Without such diagnostics, it remains unclear whether the proposed mechanisms achieve the claimed effects or simply improve the final ranking metrics on the selected datasets.

Q2. Could you provide empirical evidence showing that DPAA effectively mitigates over-smoothing? For instance, you could report performance comparisons across different numbers of aggregation layers (such as increasing propagation depth from 2 to 8 layers) to demonstrate whether DPAA resists performance degradation better than the backbone models.

Over-smoothing: We have run additional experiments which showed that DPAA maintains robust performance at different propagation depths compared to baselines, suggesting its potential to mitigate over-smoothing while preserving useful collaborative information. More specifically, we vary the number of GNN aggregation layers (2, 4, and 8) and compare DPAA with the best-performing baseline on each dataset (PRISM for Coat, PPAC for KuaiRec, and NAVIP for Yahoo! R3). On Coat and Yahoo! R3, recommendation performance generally decreases as the number of aggregation layers increases, potentially reflecting the effects of over-smoothing at greater propagation depths. However, DPAA maintains higher performance across all propagation depths by mitigating over-smoothing. In contrast, on KuaiRec, recommendation performance improves with increasing propagation depth for both DPAA and PPAC, with DPAA achieving its best performance with 8 layers and outperforming PPAC across all metrics at 4 and 8 layers. We have included these additional results under RQ5 in Section 4.2 of the paper.

Table: Effect of varying the number of GNN propagation layers on the recommendation performance of DPAA and the best-performing baseline for each dataset. Bold values indicate the best performance across methods and layer configurations for each dataset and metric.

DatasetMetric2L DPAA2L PRISM4L DPAA4L PRISM8L DPAA8L PRISM
CoatRecall@200.25650.25010.24620.22610.22920.2223
NDCG@200.20650.20050.18230.18010.15600.1778
HR@200.62340.61120.60870.57050.58090.5663
DatasetMetric2L DPAA2L PPAC4L DPAA4L PPAC8L DPAA8L PPAC
KuaiRecRecall@200.02210.02270.03490.03130.06420.0561
NDCG@200.02310.02390.05860.04740.10710.0989
HR@200.22650.21730.62010.33400.72030.7013
DatasetMetric2L DPAA2L NAVIP4L DPAA4L NAVIP8L DPAA8L NAVIP
Yahoo! R3Recall@200.14790.14520.14520.14410.14360.1429
NDCG@200.06590.06450.06550.06470.06420.0639
HR@200.22210.21750.21620.21090.21430.2120

Over-correction: We had compared DPAA with DPAA-γ=0 in Table 9 of the Appendix A.5.1, keeping all other components fixed. We observe that DPAA with initial-layer IIW achieves better performance at most popularity bias severity levels. These findings suggest that restricting IIW to the initial layer is generally more effective, as applying it across all layers may excessively suppress useful higher-order collaborative signals from relevant popular items. We have further clarified this point in blue text Appendix A.5.1 (Ablation Studies).

Thank you for the detailed response. The additional APDA-PRC comparison addresses my concern about the methodological distinction between DPAA and APDA. In particular, it provides evidence that DPAA benefits from its weight-estimation and layer-wise weighting designs beyond the original APDA formulation.

The response to the over-correction concern is also helpful, especially the ablation in Table 9. However, the evidence for mitigating over-smoothing remains incomplete. The experiment compares DPAA with only one, different baseline on each dataset, and DPAA’s performance still decreases as the number of layers increases on Coat and Yahoo! R3. Moreover, being better than a selected baseline at each depth does not by itself demonstrate greater robustness to over-smoothing. This claim would be stronger with multiple baselines under the same layer settings or with representation-level smoothing measures. The authors may also consider softening this claim in the revised paper.

Reviewer_aSumOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper investigates the popularity bias problem in GNN-based collaborative filtering and proposes Debiasing Popularity Amplification in Aggregation (DPAA), a framework designed to mitigate popularity amplification during graph aggregation. Specifically, the method combines adaptive embedding-dependent bias-correction weights with layer-dependent weights to address popularity bias at both the interaction and aggregation-layer levels. Experiments on real-world and semi-synthetic datasets demonstrate improvements in recommendation performance under various settings.

Critical strengths

The paper provides several complementary evaluations, including results averaged over five runs on real-world datasets, controlled experiments with varying levels of popularity bias, evaluation on an additional backbone, and ablation studies on the weighting components and pretrained-model quality. The reported improvements on Coat and KuaiRec are substantial for several evaluation metrics.

Critical weaknesses

A major concern is that the experimental evaluation relies on relatively dated benchmarks and baseline methods. The lack of comparisons with more recent popularity-debiasing approaches makes it difficult to assess the effectiveness and competitiveness of DPAA against current state-of-the-art methods. Evaluations on more recent or diverse recommendation datasets would also provide more convincing evidence of the proposed method's effectiveness.

Questions for authors

In Table 2, the Niche Recall and NDCG scores on KuaiRec remain extremely low (0.0002 and 0.0003). Although the reported relative improvements may appear substantial, the absolute gains in performance are very limited. How should these near-zero values be interpreted, and to what extent do the improvements translate into meaningful benefits for niche-item recommendation?

4 comments
AuthorsOpenReview ↗

Rebuttal Part 1

Thank you for your valuable comments. We have provided point-wise responses below and updated the paper where necessary, with changes highlighted in blue.

W1. A major concern is that the experimental evaluation relies on relatively dated benchmarks and baseline methods. The lack of comparisons with more recent popularity-debiasing approaches makes it difficult to assess the effectiveness and competitiveness of DPAA against current state-of-the-art methods. Evaluations on more recent or diverse recommendation datasets would also provide more convincing evidence of the proposed method's effectiveness.

Recent datasets and baselines: We selected Coat, Yahoo! R3, and KuaiRec because they are publicly available real-world recommendation datasets that support unbiased evaluation. Such evaluation data is crucial for assessing debiasing methods, as conventional recommendation benchmarks typically construct test sets from the same biased logged-interaction distribution as the training data, making it difficult to reliably assess debiasing effectiveness. While Coat and Yahoo! R3 are older datasets, KuaiRec is from 2022. If the reviewer is aware of other recent datasets that are appropriate for this task and have unbiased evaluation, please let us know and we would be happy to run the algorithms on them.

We compare our method against representative popularity-debiasing methods from major methodological families. In the table below, we additionally include two of the most recent popularity-debiasing baselines, PRISM [1] and DDC [2], to provide a more up-to-date comparison with DPAA. DPAA continues to outperform the newly added, most recent baselines across all three datasets. We have incorporated these additional results into Table 1 of the revised version, with the additions highlighted in blue.

Table: Recommendation performance on three real-world datasets compared with two additional recent popularity debiasing methods.

DatasetMethodRecall@20NDCG@20HR@20
CoatDPAA (Ours)0.25650.20650.6234
DDC0.23730.17340.6085
PRISM0.25010.20050.6112
KuaiRecDPAA (Ours)0.03490.05860.6201
DDC0.01570.01680.2469
PRISM0.01810.01540.2342
Yahoo! R3DPAA (Ours)0.14790.06590.2221
DDC0.13610.06080.2065
PRISM0.12360.05990.1908

[1] Donald et al. On the Role of Weight Decay in Collaborative Filtering: A Popularity Perspective. SIGKDD'25.
[2] Liu et al. Rethinking Popularity Bias in Collaborative Filtering via Analytical Vector Decomposition. SIGKDD'26.

AuthorsOpenReview ↗

Rebuttal Part 2

Q1. In Table 2, the Niche Recall and NDCG scores on KuaiRec remain extremely low (0.0002 and 0.0003). Although the reported relative improvements may appear substantial, the absolute gains in performance are very limited. How should these near-zero values be interpreted, and to what extent do the improvements translate into meaningful benefits for niche-item recommendation?

To better understand these near-zero values and assess the benefits of DPAA for niche-item recommendation on KuaiRec, we conduct two additional analyses.

Impact of interaction frequency on niche recommendation: To understand the near-zero niche recommendation performance on KuaiRec out-of-distribution evaluation data, we group niche items based on their total number of interactions in the test set. Specifically, we divide niche items into seven groups according to their total interaction frequencies in the test data: 1, 2, 3, 4, 5, 6–10, and more than 10 interactions. For each group, we evaluate recommendation performance using only the relevant test interactions associated with items in that group, while retaining the entire item catalog as ranking candidates. The results show that recommendation performance remains zero for niche items with 1–4 test interactions, while non-zero performance is observed for groups with more interactions. This suggests that the limited number of interactions associated with many niche items, combined with the distribution shift between the biased training data and the unbiased out-of-distribution test data, may contribute to the extremely low overall niche recommendation performance on KuaiRec. We additionally provide more detailed long-tail performance (see our responses of W2 and Q2 for reviewer U2nR).

Table: Niche-item recommendation performance on KuaiRec grouped by the total number of test interactions per item.

Number of interactions per niche itemNumber of items in this groupRecall@20NDCG@20HR@20
1910.00000.00000.0000
2600.00000.00000.0000
3570.00000.00000.0000
4270.00000.00000.0000
5270.00860.00210.0086
6–101550.00110.00080.0075
>109070.00140.00200.0381

Impact of candidate set size on niche recommendation and and meaningful performance gains: Another possible reason for the low absolute performance is the difficulty of retrieving relevant niche items when ranking over the entire large item catalog, particularly when evaluation is conducted on unbiased, out-of-distribution test data that follows a different distribution from the training data. To better understand this issue, we conduct an additional experiment. Instead of using the entire item catalog as the candidate set, we restrict the ranking to popular items for popular-item evaluation and to niche items for niche-item evaluation. This allows us to assess how well each model ranks relevant items within each popularity group by excluding items belonging to the other group. Under this setting, DPAA outperforms the best-performing baseline (PPAC) by a good margin across both groups, even when evaluated on out-of-distribution test data. These results suggest that the low niche performance in the original evaluation is partly related to the difficulty of identifying relevant niche items from a large candidate set. Moreover, they demonstrate that DPAA can identify relevant niche items more effectively than the baseline, achieving larger and more meaningful performance gains when candidates for ranking are restricted to niche items.

Table: Recommendation performance on KuaiRec for popular and niche item groups. For each group, only items belonging to that group are considered as ranking candidates.

DatasetItem groupMethodRecall@20NDCG@20HR@20
KuaiRecPopularDPAA0.13690.24830.9272
PPAC0.11990.22870.8387
NicheDPAA0.06310.05630.3925
PPAC0.04270.04890.3481

Thank the authors for the detailed response. I still have a question regarding the results on KuaiRec. In Table 1 and the additional results provided in the rebuttal, the compared methods achieve performance close to that of DPAA on Coat and Yahoo! R3. However, a substantial performance gap remains on KuaiRec, even with the inclusion of more recent baselines. Could the authors provide further insights into the reasons for this discrepancy?

Thank you for your positive feedback and follow-up question.

We believe that the larger performance gap between DPAA and the baselines on KuaiRec can be attributed primarily to the more severe popularity skew in its training data compared to Coat and Yahoo! R3 (see Section A.3.2 and Figure 5 in the Appendix). While the baselines improve recommendation performance on KuaiRec by mitigating popularity bias, DPAA is more effective under severe popularity bias, leading to larger performance gains in unbiased evaluation. This explanation is further supported by the following findings. Please refer to the revised version of the paper for the table and figure numbers described below.

  • Empirical validation under severe popularity bias: Our semi-synthetic experiments with controlled levels of popularity bias (RQ2 and Figure 1 in Section 4.2, and Table 6 in the Appendix) demonstrate that DPAA's performance gains over the baselines generally increase as popularity bias becomes more severe.

  • Greater reduction of popularity dominance in the representation space under severe bias: Our ablation studies (Section 4.3 and Section A.5.5 with Table 14 in the Appendix) show that, relative to LightGCN, DPAA achieves the largest reduction in Pearson and Spearman correlations between item popularity and the projections of final item representations onto the popularity direction on KuaiRec compared to Coat and Yahoo! R3. This demonstrates DPAA's greater effectiveness in reducing popularity dominance in the representation space under severe bias.

  • Effectiveness of DPAA's components under severe popularity bias: As shown in Table 1, the baselines generally outperform LightGCN on KuaiRec, suggesting their effectiveness in mitigating popularity bias. However, DPAA achieves larger gains by jointly addressing interaction-level and layer-level sources of bias directly within the aggregation process, while accounting for under-correction, over-correction, and potential over-smoothing. Our ablation studies (Section 4.3 and Appendix A.5.1 with Figure 7 and Table 11) further demonstrate the importance of combining pre-trained and current model estimates for IIW, incorporating LW, and controlling over-correction across different bias levels, highlighting their collective effectiveness under severe popularity bias.

  • Effectiveness of layer-wise weighting under severe bias: As suggested by Lemma 3.2 (Section 3.3), in scenarios where interactions are heavily dominated by popular items, a larger η\eta can be particularly beneficial. Consistent with this theoretical insight, our sensitivity analysis (Section 4.3 and Figure 9 in Appendix A.5.3) shows that KuaiRec benefits more from emphasizing higher-order collaborative signals through LW.

  • Theoretical justification for mitigating bias based on severity: Lemma 3.1 (Section 3.3) theoretically characterizes that IIW reduces the relative contribution of popular items during aggregation depending on the severity of popularity bias.

Together, these theoretical and empirical findings suggest that DPAA is particularly effective under severe popularity bias, which may explain its relatively larger performance gains on KuaiRec. Furthermore, KuaiRec is one of the few large-scale recommendation datasets with nearly fully observed evaluation data, enabling reliable unbiased evaluation of popularity bias correction methods and providing further confidence in the observed performance gains.

Thank you again for your valuable suggestions and comments that have helped improve our paper. We hope that our responses and revisions have adequately addressed your concerns. We would be happy to provide further clarification if needed and welcome any additional suggestions you may have.

Comments on the paper

3 comments

Dear Reviewer aSum,

Thank you again for your comments! As the deadline is approaching, we would really appreciate your thoughts on the rebuttal.

Regards,

Authors

Thank you for the response. My main concerns have been addressed and I will raise my score.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you very much for your response! We are glad that we are able to address your concerns. We appreciate your constructive feedback and reconsideration of your rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.