- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates popularity bias amplification during message passing in GNN-based collaborative filtering and proposes Debiasing Popularity Amplification in Aggregation (DPAA) to address this issue. DPAA combines adaptive interaction-level weighting based on cosine similarity with layer-wise weighting that emphasizes higher-order collaborative signals, while incorporating pre-trained embeddings to stabilize weight estimation. Experiments on three real-world datasets and semi-synthetic data demonstrate improvements in recommendation accuracy under varying levels of popularity bias.
Critical strengths
- The proposed framework directly addresses popularity amplification during GNN message passing by jointly incorporating interaction-level and layer-wise weighting mechanisms.
- The evaluation on datasets with unbiased test sets, together with semi-synthetic experiments under varying levels of popularity bias, provides evidence of the method's effectiveness across different bias conditions.
Critical weaknesses
- The proposed IIW estimates interaction strength using cosine similarity between L2-normalized embeddings, which discards embedding magnitude information that may encode item popularity [1]. I am concerned that the paper does not sufficiently justify this design choice or compare it with alternative interaction-weighting strategies that preserve magnitude information.
- Although the paper reports separate recommendation results for popular and long-tail items, I remain concerned about whether DPAA effectively reduces popularity dominance in the final learned representations. Further analysis of the popularity direction in item embeddings [2] and more detailed long-tail recommendation results [3] would strengthen the empirical evidence.
- The paper should compare DPAA with more recent baseline methods.
[1] Donald et al. On the Role of Weight Decay in Collaborative Filtering: A Popularity Perspective. SIGKDD'25.
[2] Liu et al. Rethinking Popularity Bias in Collaborative Filtering via Analytical Vector Decomposition. SIGKDD'26.
[3] Wu et al. Self-supervised graph learning for recommendation. SIGIR'21.
Questions for authors
- Could the authors explain why cosine similarity is preferred for estimating interaction strength, given that embedding magnitudes may contain popularity-related information? Further comparison with other weighting strategies is necessary.
- Could the authors further analyze how DPAA affects the popularity direction in the final item embeddings and provide more detailed results on long-tail items?
7 comments
Rebuttal Part 2
W2. Although the paper reports separate recommendation results for popular and long-tail items, I remain concerned about whether DPAA effectively reduces popularity dominance in the final learned representations. Further analysis of the popularity direction in item embeddings [2] and more detailed long-tail recommendation results [3] would strengthen the empirical evidence.
Q2. Could the authors further analyze how DPAA affects the popularity direction in the final item embeddings and provide more detailed results on long-tail items?
Regarding popularity dominance, we examine whether DPAA reduces popularity dominance in final item embeddings and analyze their alignment with the popularity direction, following [2]. Specifically, we define the popularity direction as the normalized difference between the mean embeddings of popular (head) and unpopular (tail) items. We then project each item's final embedding onto this direction and compute the Pearson and Spearman correlations between these projections and item popularity, measured by training interaction frequency. We compare these correlations between DPAA and its LightGCN backbone, which does not correct for popularity bias, to assess whether DPAA reduces popularity-related alignment in the learned representations. Lower absolute values indicate weaker alignment between item popularity and the embeddings projections onto the popularity direction, suggesting reduced popularity dominance in the learned representations.
Table: Comparison of popularity-direction correlations in final item embeddings between DPAA and LightGCN. Lower absolute values indicate weaker alignment between item popularity and final item representations.
| Dataset | Method | Pearson correlation coefficient ↓ | Spearman correlation coefficient ↓ |
|---|---|---|---|
| Coat | DPAA | 0.3591 | 0.2941 |
| LightGCN | 0.3904 | 0.3094 | |
| KuaiRec | DPAA | 0.6517 | 0.6922 |
| LightGCN | 0.9549 | 0.9701 | |
| Yahoo! R3 | DPAA | 0.8079 | 0.7247 |
| LightGCN | 0.8805 | 0.8327 |
Across all three datasets, DPAA achieves lower Pearson and Spearman correlation coefficients than the LightGCN backbone, indicating weaker alignment between item popularity and the final item representations learned by DPAA. These findings provide empirical evidence that DPAA reduces popularity dominance in learned item representations. We have included this analysis in Section 4.3 of the main body and Appendix A.5.5 of the revised version, with the additions highlighted in blue.
Regarding more detailed analysis of the long-tail items, similar to [3], we evaluate DPAA across different item popularity levels based on interaction frequencies and dividing them into four quartiles. Q4 contains the most-popular items accounting for 25% of interactions, followed by Q3 and Q2, while Q1 contains the least-popular items accounting for the remaining 25%. DPAA generally outperforms the corresponding best-performing baselines across different popularity groups. These results demonstrate that DPAA maintains strong recommendation performance for highly popular items while also improving performance across different long-tail item popularity groups. We have included this analysis in RQ3 in Section 4.2 of the main body and Appendix A.4.5 of the revised version, with the additions highlighted in blue.
Table: Performance comparison between DPAA and the best-performing baseline across item popularity quartiles. Each quartile accounts for approximately 25% of total training interactions, with Q4 comprising the most-popular and Q1 comprising the least-popular items.
| Dataset | Quartile | Method | Recall@20 | NDCG@20 | HR@20 |
|---|---|---|---|---|---|
| Coat | Q4 | DPAA | 0.5605 | 0.3335 | 0.6005 |
| PRISM | 0.4737 | 0.2978 | 0.4945 | ||
| Q3 | DPAA | 0.3184 | 0.1749 | 0.4034 | |
| PRISM | 0.2734 | 0.1634 | 0.3621 | ||
| Q2 | DPAA | 0.1679 | 0.1314 | 0.2748 | |
| PRISM | 0.1665 | 0.1275 | 0.2621 | ||
| Q1 | DPAA | 0.1082 | 0.0946 | 0.1794 | |
| PRISM | 0.1101 | 0.0982 | 0.1864 | ||
| KuaiRec | Q4 | DPAA | 0.0764 | 0.0629 | 0.4756 |
| PPAC | 0.0599 | 0.0489 | 0.4726 | ||
| Q3 | DPAA | 0.0024 | 0.0026 | 0.0321 | |
| PPAC | 0.0000 | 0.0000 | 0.0000 | ||
| Q2 | DPAA | 0.0003 | 0.0006 | 0.0076 | |
| PPAC | 0.0000 | 0.0000 | 0.0000 | ||
| Q1 | DPAA | 0.0002 | 0.0003 | 0.0008 | |
| PPAC | 0.0000 | 0.0000 | 0.0000 | ||
| Yahoo! R3 | Q4 | DPAA | 0.6509 | 0.3097 | 0.6613 |
| NAVIP | 0.6450 | 0.2937 | 0.6590 | ||
| Q3 | DPAA | 0.3317 | 0.1339 | 0.3449 | |
| NAVIP | 0.3106 | 0.1308 | 0.3267 | ||
| Q2 | DPAA | 0.1131 | 0.0383 | 0.1239 | |
| NAVIP | 0.1009 | 0.0364 | 0.1121 | ||
| Q1 | DPAA | 0.0196 | 0.0056 | 0.0225 | |
| NAVIP | 0.0194 | 0.0062 | 0.0233 |
Rebuttal Part 3
W3. The paper should compare DPAA with more recent baseline methods.
Regarding the baselines, we compare our method against representative popularity-debiasing methods from major methodological families. In the table below, we additionally include two of the most recent popularity-debiasing baselines, PRISM [1] and DDC [2], to provide a more up-to-date comparison with DPAA. DPAA continues to outperform the newly added, most recent baselines across all three datasets. We have incorporated these additional results into Table 1 of the revised version, with the additions highlighted in blue.
Table: Recommendation performance on three real-world datasets compared with two additional recent popularity debiasing methods.
| Dataset | Method | Recall@20 | NDCG@20 | HR@20 |
|---|---|---|---|---|
| Coat | DPAA (Ours) | 0.2565 | 0.2065 | 0.6234 |
| DDC | 0.2373 | 0.1734 | 0.6085 | |
| PRISM | 0.2501 | 0.2005 | 0.6112 | |
| KuaiRec | DPAA (Ours) | 0.0349 | 0.0586 | 0.6201 |
| DDC | 0.0157 | 0.0168 | 0.2469 | |
| PRISM | 0.0181 | 0.0154 | 0.2342 | |
| Yahoo! R3 | DPAA (Ours) | 0.1479 | 0.0659 | 0.2221 |
| DDC | 0.1361 | 0.0608 | 0.2065 | |
| PRISM | 0.1236 | 0.0599 | 0.1908 |
[1] Donald et al. On the Role of Weight Decay in Collaborative Filtering: A Popularity Perspective. SIGKDD'25.
[2] Liu et al. Rethinking Popularity Bias in Collaborative Filtering via Analytical Vector Decomposition. SIGKDD'26.
Rebuttal Part 1
Thank you for your valuable comments. We have provided point-wise responses below and updated the paper where necessary, with changes highlighted in blue.
We chose cosine similarity for estimating interaction strength in IIW for the following reasons:
Normalized and bounded interaction weighting: L2 normalization projects user and item embeddings onto the unit sphere, bounding cosine similarity within [-1, 1] and the corresponding IIW within [0, 2]. This ensures consistent scaling of interaction weights and prevents high-norm embeddings from disproportionately influencing similarity estimates due to their magnitudes. In contrast, directly using the dot product produces unbounded similarity scores whose scales vary across interactions, potentially allowing certain interactions to dominate message passing and representation learning. Cosine similarity avoids this issue by ensuring that interaction weights remain bounded and comparable across user–item pairs.
Empirical relationship between cosine similarity and item popularity: As shown in Figure 5 in the Appendix, by grouping items into different popularity quartiles, we observe that more popular item groups generally exhibit higher user–item cosine similarity. Therefore, cosine similarity retains a useful signal for identifying popularity-associated interactions while keeping the scale consistent, from which we assign smaller aggregation weights to interactions with higher user–item similarity.
Empirical comparison with alternative weighting strategies: To further validate our design choice, we conducted additional experiments using two alternative interaction-weighting strategies: (i) raw dot-product similarity, which preserves embedding magnitude information, and (ii) sigmoid-transformed dot-product similarity, which bounds the resulting similarity scores while retaining embedding magnitude information. We replaced cosine similarity with each alternative in the IIW computation while keeping the remaining components of DPAA unchanged. The results show that cosine similarity achieves the best recommendation performance across all datasets, followed by sigmoid-transformed dot-product similarity.
Table: Performance comparison of DPAA using different similarity functions for estimating interaction-level weights (IIW).
Raw dot-product similarity does not produce good performance, particularly on KuaiRec and Yahoo! R3. This degradation may be attributed to the unbounded interaction weights produced by raw dot-product similarity. We have included this analysis in Section 4.3 of the main body and Appendix A.5.4 of the revised version, with the additions highlighted in blue.