- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper examines whether target-item diffusion recommenders retain stochastic variation across repeated samples for the same user history. DARTS separates sequence-autoencoder pretraining, latent diffusion in a frozen representation space, and decoder fine-tuning. The paper evaluates diversity before and after decoding and reports improvements in Recall and NDCG on three Amazon Reviews 2023 categories. Averaging predictions from multiple samples further improves ranking accuracy without increasing the recommendation list size.
Critical strengths
- The focus on repeated-sample behavior is useful. Ranking accuracy alone cannot tell whether a diffusion recommender behaves as a stochastic predictor. The diagnostics in Section 4.3 examine variation in both latent space and the resulting recommendation lists.
- DARTS improves ranking accuracy across all three datasets and outperforms baselines using the same hybrid item representations. The evaluation also distinguishes gains from larger recommendation lists from those obtained at a fixed list size.
Critical weaknesses
- The contribution of the generated latent to ranking accuracy is not directly tested. As shown in Appendix A.3, the decoder processes the right-shifted interaction history and attends to the encoded history states . However, I could not find a history-only variant that retains these inputs but removes the generated target latent. The existing ablations retain the generation branch, making it difficult to separate its contribution from that of the pretrained encoder–decoder and subsequent ranking fine-tuning.
- The evidence that different samples provide relevant alternative recommendations remains limited. The diagnostics establish variation, and the aggregation results show some predictive value, but neither tells us whether different samples recover distinct relevant future items. The authors acknowledge that leave-one-out evaluation provides only one observed future item per history. It would be useful to evaluate whether different samples recover different held-out future interactions for the same history.
Questions for authors
- How does DARTS compare with a history-only decoder that is fine-tuned under the same ranking objective?
- Do different samples recover different relevant future items for the same user history?
- On the Scientific dataset, Table 1 reports Recall@5 and Recall@10 of 0.0329 and 0.0524, while Table 10 reports 0.0344 and 0.0545 at . Since union recall reduces to ordinary recall in this setting, were different checkpoints, sampling runs, or evaluation settings used?
W1&Q1:
To isolate the contribution of diffusion, we conducted an additional ablation replacing Stage-2 diffusion training with a deterministic latent predictor, while keeping the Stage-1 representations, decoder, and Stage-3 objective unchanged. Importantly, the deterministic baseline has exactly the same number of parameters as DARTS, ensuring a fair comparison. Its deterministic variant performs substantially worse than DART (43–44% lower across all four ranking metrics). This demonstrates that the gains cannot be attributed solely to MLM pretraining and BPR fine-tuning, and highlights the contribution of diffusion to our framework. This ablation is present in the revised version of the paper in Appendix J
W2 & Q2:
We thank the reviewer for raising this important point. Directly evaluating multimodality is challenging in sequential recommendation, as datasets typically provide only one observed next item rather than multiple valid recommendations for a given context.
To investigate recommendation diversity, as a proxy we conducted an additional experiment using sequences of 15 items, holding out 5 items, considering that they are all possible valid recommendations. We have evaluated how our model recovers these items across multiple independent draws. Coverage@10 increases from 4.49% with one draw to 6.41% with seven draws, indicating that different draws can recover different relevant items.
Note that these coverage values should not be directly compared with our standard Recall@10 results. This experiment uses shorter training sequences and a different evaluation objective: the model is still trained for next-item prediction, but evaluated on its ability to recover the five held-out items across multiple draws. Training directly to recover these five items would change the learning objective and evaluate a different model from DART.
This ablation is present in the revised version of the paper in appendix M.
While we acknowledge that this experiment does not establish the existence of multiple distinct modes in the learned conditional distribution, it provides strong empirical evidence that DART generates diverse, relevant recommendations across independent draws, supporting its ability to go beyond deterministic next-item prediction. This motivates a complementary question: does the diversity across draws also translate into diversity in the final recommendations, and does it extend beyond differences in item identity to differences in item semantics?
To investigate this, we performed four independent inference draws for 10,000 test histories and analyzed the top-1 recommendations after Stage 3. At the item level, the top-1 recommendation differs across draws for 61.8% of histories. To assess semantic diversity, we used Amazon's level-2 product categories, corresponding to the second level of its product-category hierarchy and providing a more fine-grained grouping of items. We found that 35.22% of histories receive recommendations spanning multiple level-2 categories, with an average of 1.434 distinct categories across the four draws.
These results show that latent diversity translates into diversity in the final recommendations, both at the item and semantic-category levels.
We have added this ablation to the paper in Appendix L.
Q3:
We thank the reviewer for identifying this inconsistency. The discrepancy resulted from inadvertently mixing results from experiments conducted under different settings. We have corrected this error in the revised manuscript to ensure consistency between Tables 1 and 10.