- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a coarse-to-fine learning for diffusion recommenders, consisting of a semantic cloud construction module and a log-SNR-coupled temperature schedule. The motivation is straightforward and makes sense. My major concerns lie in the experimental evaluation (please refer to the weaknesses and my question). It seems that the baselines had not been tuned well, the results are much lower than those in their original papers.
Critical strengths
-
The motivation of the proposed method is easy to understand and makes sense.
-
Extensive theoretical analysis is provided to support the model design.
Critical weaknesses
-
The results of the baselines present in this paper is not convicing to me. The results of the strongest baseline PreferDiff are significantly lower than those in the original paper, even under the same datasets and settings. This makes me concerned about whether the baselines are tuned properly.
-
The baseline methods are not SOTA. The strongest baseline is from 2025 while most others are before 2023, which I think cannot sufficiently evaluate the effectiveness of the proposed method in such rapid-developed topic.
Questions for authors
-
What does the performance of the proposed model change if you use a fixed cloud (maybe in the initial step or some middle steps) for every denoising steps?
-
Could you show the results of you method, TIGER and DiffuRec under the leave-one-out evaluation setting? It seems that in this paper, these two methods' performances are much lower than in their original papers.
-
Since you use a very large negtive pool (typically 128 and 256) in your method, I am curious about the number of negatives in PreferDiff in your implementation. I would also suggest to conduct hyperparameter analysis in terms of the number of negative to figure out whether the peroformance gain largely from sampling more negative samples.
-
You use LLMs to construct soft positive, this makes me concerned about the training cost. Did you try any simple methods to sample soft positives?
-
You use LLM-derived embeddings in your method. Did you also use these embeddings for other methods or simply use ID embeddings for them?
6 comments
Response to Reviewer U7rh (2/2)
Response to Q2: Leave-One-Out Evaluation.
The performance gap mainly stems from different evaluation protocols. While the original DiffuRec and TIGER papers adopt leave-one-out (LOO) evaluation, our experiments follow a user-split protocol. Under LOO, earlier interactions from test users are available during training, whereas user-split evaluation holds out entire users, requiring models to generalize to unseen users. These differences make our setting more challenging and help explain the lower baseline performance compared with the original papers.
| Method | Metric | Beauty | Toys | Sports | Yelp |
|---|---|---|---|---|---|
| DiffuRec | N@10 | 0.0454 | 0.0407 | 0.0240 | 0.0184 |
| R@10 | 0.0758 | 0.0651 | 0.0393 | 0.0289 | |
| TIGER | N@10 | 0.0347 | 0.0297 | 0.0177 | 0.0120 |
| R@10 | 0.0599 | 0.0531 | 0.0328 | 0.0222 | |
| PreferDiff | N@10 | 0.0423 | 0.0464 | 0.0211 | 0.0174 |
| R@10 | 0.0748 | 0.0852 | 0.0378 | 0.0264 | |
| CloudDiff | N@10 | 0.0447 | 0.0488 | 0.0232 | 0.0189 |
| R@10 | 0.0792 | 0.0900 | 0.0406 | 0.0313 |
Under the standard LOO setting, CloudDiff achieves the best R@10 across all four datasets and the best N@10 on Toys and Yelp, while remaining competitive on Beauty and Sports. These results demonstrate that CloudDiff maintains strong performance under both evaluation protocols.
Response to Q3: Negatives.
PreferDiff uses 256 in-batch negatives in our implementation. Reducing CloudDiff's negatives (main: 128 / 256) to 32 or 64:
| # Negatives | Metric | Beauty | Toys | Sports | Yelp |
|---|---|---|---|---|---|
| 32 | N@10 | 0.0372 | 0.0329 | 0.0139 | 0.0109 |
| R@10 | 0.0547 | 0.0468 | 0.0231 | 0.0189 | |
| 64 | N@10 | 0.0377 | 0.0334 | 0.0137 | 0.0111 |
| R@10 | 0.0554 | 0.0485 | 0.0228 | 0.0180 | |
| Main (128 / 256) | N@10 | 0.0384 | 0.0347 | 0.0158 | 0.0118 |
| R@10 | 0.0560 | 0.0484 | 0.0249 | 0.0209 |
CloudDiff maintains competitive performance even with substantially fewer negatives. While larger negative pools generally improve performance, the gains are modest, suggesting that CloudDiff’s improvements are not primarily driven by negative pool size.
Response to Q4: Other Soft Positives.
Appendix G.3 demonstrates that the cloud can be constructed without LLMs, using SPPMI co-occurrence or SASRec item-embedding neighbors.
| Neighbor Space | Beauty | Toys | Sports | Yelp |
|---|---|---|---|---|
| LLM | 0.0384 | 0.0347 | 0.0158 | 0.0118 |
| SPPMI-SVD | 0.0361 | 0.0315 | 0.0116 | 0.0097 |
| SASRec | 0.0359 | 0.0342 | 0.0130 | 0.0107 |
The results demonstrate robustness across different soft-positive constructions, with LLM-derived semantics being one option rather than a requirement. Our core contribution lies in noise-adaptive supervision, rather than the specific neighborhood construction. Also, LLM generation incurs only a one-time offline cost.
Response to Q5: LLM Embeddings For Other Methods.
Baselines were implemented following their original designs. Methods using LLM-derived embeddings (e.g., LLM-ESR) were provided with the same embeddings as CloudDiff, while ID-based methods retained their trainable ID embeddings. CloudDiff also uses trainable ID embeddings for recommendation, with LLM-derived embeddings providing auxiliary supervision through semantic neighborhood and denoiser representation alignment.
References
[1] Liu, Zhao, et al. Diffgrm: Diffusion-based generative recommendation model. In: Proceedings of the ACM Web Conference 2026. 2026. p. 5853-5864.
[2] Hu, Guoqing, et al. Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation. Advances in Neural Information Processing Systems, 2025, 38: 135766-135804.
Thanks for your detailed responses.
My concern on the training cost of LLM-derived soft positive (or other methods) is still not addressed.
Another question is whether the soft positives are selected from all items?
Response to Reviewer U7rh (1/2)
We sincerely thank the reviewer for their thorough evaluation and constructive suggestions, particularly for identifying important areas to strengthen our experiments.
Response to W1: PreferDiff Reproduction.
In our original experiments, we searched embedding dimensions up to 1024 for all methods (see App. E.4.1), reporting results with 512 for CloudDiff and 1024 for PreferDiff. However, as noted in the original PreferDiff paper, PreferDiff is sensitive to embedding dimensions and considers up to 3072, which explains the lower performance reported in our paper. To address this, we additionally evaluated both methods at dimensions of 2048 and 3072 under the same settings, as shown below.
As reported in the original PreferDiff paper, larger embedding dimensions substantially improve its performance. Nevertheless, CloudDiff outperforms PreferDiff on all four datasets at 2048 dimensions and three at 3072, with particularly strong gains on sparse Yelp. More importantly, CloudDiff achieves strong performance with only 512 hidden dimensions, without relying on larger representations and their associated computational costs.
Response to W2: Add Recent SOTA.
We additionally compare CloudDiff against two recent diffusion-based recommendation methods, DiffGRM [1] (WWW'26) and PreferGrow [2] (NeurIPS'25). CloudDiff achieves the highest NDCG@10 across all four datasets and the highest Recall@10 on three datasets, demonstrating its competitiveness against recent SOTA baselines.
Response to Q1: Fixed Cloud.
Fig. 4 with p = 0 is exactly a cloud whose weight is held constant over all denoising steps, and the "cloud + static τ" variant of Fig. 3 additionally fixes the temperature; both are below the full model, i.e. adapting the cloud with the noise level is what helps. We will make this correspondence explicit in the text.
We additionally evaluate CloudDiff with the cloud fixed at the midpoint timestep across all denoising steps.
Fixing cloud supervision at an intermediate timestep yields lower performance than full CloudDiff across all datasets. This suggests that while semantic-cloud supervision is beneficial, adapting its strength to the diffusion noise level is important.