- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes COMPOTE, a sequential recommender that outputs M hypotheses per user history. A shared causal transformer feeds M heads, each producing a cosine/temperature softmax over the catalogue. Training uses the annealed loss, which interpolates between uniform supervision (high τ) and winner-takes-all (τ→0). An auxiliary confidence head is trained with a BCE loss. At inference, items are ranked by max-pooling over heads. They show this can be done on raw cosine similarities, without the per-head softmax normalizer. Empirically, on three Amazon-2023 subsets, COMPOTE is best in 11 of 12 columns against ID-based baselines. Ablation studies were performed to argue the gains come from the annealed objective. Diagnostics are performed to argue that heads specialize into a near-static tiling of the catalogue.
Critical strengths
-
Controlled evidence that the multi-head parameterization matters, plus a clear collapse diagnosis. Table 6 shows EnsCE and Router heads collapse completely (S=1.00), while COMPOTE’s heads do not (S≈0.6). Table 7 shows the oracle gap is zero for collapsed models and large for COMPOTE. This is a convincing demonstration that equal-weight and routed multi-head training collapses in this setting, and that the COMPOTE objective yields distinct heads. It is the paper’s most valuable empirical contribution.
-
A practical inference rule, honestly validated, with transparent limitations. Dropping the per-head softmax normalizer costs nothing (Table 2). Appendix C quantifies why: the cross-head spread of log Z_m is small relative to the cosine gaps that matter. Appendix M openly reports that the learned tiling is nearly static, which the theory does not predict. This candor makes the empirical story easier to trust and gives the community a concrete open question.
Critical weaknesses
- A large share of the improvement over SASRec comes from the backbone and loss recipe, not from multiple hypotheses. For example, Game N@5 goes 3.31 (SASRec) → 3.78 (MCT-SASRec) → 4.01 (COMPOTE). Multi-interest baselines trained by the authors (e.g., MIND 3.22 H@5 on Game) land far below SASRec, which is atypical and suggests they were under-tuned or trained with a weaker (sampled-softmax) objective than COMPOTE’s full softmax.
2.The deployment claims (sublinear ANN, “key deployment result”) are never tested. Training requires a full-catalogue softmax per head, and no cost or latency numbers are given.
Questions for authors
- Key experimental details are missing from the main text: the annealing schedule (initial τ, decay, final τ), β, architecture size, tuning budget, seeds, and whether model selection is on validation or test.
- The proof of Thm 7 cites “a direct application of Theorem 7” (circular).
- S(x) in Sec. 5.1 sums over j,k=1..M including j=k, while normalizing by M(M−1).
- Appendix J is an empty heading.
Dear Reviewer bFPG, thank you for your thoughtful and constructive feedback on our paper.
W1: We would like to clarify that we do not claim that the entire improvement over SASRec comes from the multiple-hypothesis formulation: the backbone and training recipe also contribute to performance. Our claim is that COMPOTE provides additional gains beyond these components.
Following the reviewer's suggestion, we conducted additional experiments on Yelp, comparing COMPOTE with SASRec, TIGER, and several multi-hypothesis ablations (MCT-SASRec, MCT-Router, and MCT-EnsCe). COMPOTE achieves the best results across all four metrics, reaching 3.40% Hit@5, 4.80% Hit@10, 2.47% NDCG@5, and 2.92% NDCG@10. Relative to the strongest non-COMPOTE baseline for each metric, these correspond to improvements of 34.4%, 17.9%, 25.4%, and 35.8%, respectively.
These additional results support our central claim: while the backbone and training recipe matter, COMPOTE's multiple-hypothesis learning and retrieval approach provides further improvements over the considered alternatives. We will include these results in the revised paper at appendix Q.
W1 bis: We have checked our implementations, the runs of MIND and we can confirm that there is no implementation errors and that they have been trained using full softmax and for a sufficiently large number of iterations.
We also want to note that there exists other works in the literature that report worse performance of MIND and COMIREC compared to SASRec, albeit in a bit different context than ours.
[1] Nguyen et al, BACH: A Bayesian Admixture of Contrastive Heads for Multi-Interest Two-Tower Retrieval, https://arxiv.org/pdf/2607, 2026.
[2] Zhang et al, Controllable Generative Recommendation via Guided Token Refinement, https://arxiv.org/pdf/2610.28642, 2026.
[3] Si et al, Generative Retrieval with Semantic Tree-Structured Identifiers and Contrastive Learning, https://arxiv.org/pdf/2309.13375, 2023
[4] Zhang et al Divide and Conquer: Towards Better Embedding-base, Retrieval for Recommender Systems from a Multi-task Perspective, https://arxiv.org/pdf/2302.02657, 2023
W2: COMPOTE does require a full-catalogue softmax for each hypothesis during training. At inference, however, as described in Section 4.1 (Fast Retrieval for COMPOTE), our max-similarity scoring rule avoids the full-catalogue softmax and enables retrieval through M independent approximate nearest-neighbor (ANN) searches, one per hypothesis. These searches can be executed in parallel, yielding sublinear query time in the catalogue size under standard ANN indexing, although the total computational work still scales linearly with the number of hypotheses M. Thus, our sublinearity claim concerns inference latency, not total FLOPs or training complexity.
To make it clearer, we have updated section 4.1.
Q1: We thank the reviewer for noting that these details should be more accessible. We have added Appendix H to provide a consolidated description of the model architecture and training hyperparameters. In particular, we report the embedding dimension d=128, the annealing schedule \tau$: 1.0 $\rightarrow$ 0$, with linear decay), the learning-rate schedule ($10^{-3}$), with linear decay), weight decay ($10^{-3}$), catalogue temperature \beta=0.07, and score-loss weight $$\alpha=0.1.
We also clarify in appendix G the experimental protocol and validation-based model selection, to make the setup fully reproducible.
Q2: Thanks for pointing this out. The correct reference was "Th. 1 from Ruppert et al."". The proof has been edited.
Q3: We have corrected the typo and removed the case j=k.
Q4 The section contains the two tables about the head collapse (Tables 6 and 7 in the submitted version). We have now added some context to this section, as well as some insights from the tables.