- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes FlySCoRe, a fly-inspired sparse retrieval framework for next-basket recommendation. The key idea is to perform candidate-dependent sparse addressing over historical observations, retrieve relevant purchase evidence from memory, and combine it with a user-level signal for candidate scoring. The authors develop three variants based on connectome-constrained, probabilistic, and neural addressing, and evaluate them on four repeat-purchase datasets.
Critical strengths
-
Interesting candidate-dependent retrieval formulation. The idea of retrieving different historical evidence for different candidate products is intuitive and potentially useful for next-basket recommendation.
-
Strong empirical performance. The proposed variants achieve substantial improvements over the considered baselines across four datasets.
-
Multiple implementations and analyses. The paper investigates the proposed retrieval mechanism through connectome-constrained, probabilistic, and neural variants, together with several ablation and reconstruction experiments.
Critical weaknesses
- Efficiency of candidate-dependent addressing. The candidate-dependent addressing mechanism may introduce non-trivial inference overhead, since the model needs to perform the addressing process for individual candidate products when ranking the catalog. The paper would benefit from reporting the time complexity and empirical inference latency, preferably compared with the baseline methods.
- The necessity of sparse selection is not sufficiently justified. The paper does not clearly establish the advantage of sparse addressing over dense addressing. In fact, the retrained dense-addressing results reported in the Appendix achieve better NDCG@20 on three of the four datasets. This raises the question of whether sparse selection provides a meaningful benefit beyond the biological motivation.
- Insufficient sensitivity analysis for the sparsity level. The effect of the sparse rate, or equivalently the number of activated memory units, is not sufficiently studied. Since this is a central design choice of the proposed method, a sensitivity analysis would help clarify the robustness of the method and the trade-off between sparsity and recommendation performance.
- No analysis of memory-unit utilization and load balance. The paper does not discuss whether memory units are utilized uniformly. Since the learned addressing repeatedly selects a limited subset of memory units, some units may receive substantially more support than others. This could potentially reduce the effective memory pool size and lead to repeated selection of the same memory units. An analysis of memory-unit utilization would help clarify whether such an imbalance occurs and whether it affects the reported results.
Questions for authors
- Efficiency: What is the computational complexity and empirical inference latency of the candidate-dependent addressing mechanism, especially when ranking the full catalog? How does it compare with the baseline methods?
- Sparsity: Have the authors systematically studied how the sparse rate (i.e., the number of activated memory units) affects recommendation performance? How sensitive are the results to this choice?
- Memory-unit utilization: Have the authors analyzed the distribution of memory-unit utilization/support? In particular, is there a potential rich-get-richer effect where a small subset of memory units is repeatedly selected, making the effective memory pool substantially smaller than the nominal pool size? If such an imbalance exists, how does it affect the learned memory representation and recommendation performance?
3 comments
Thank you for clarifying several points that were unclear to me in the original paper. While I appreciate these explanations, some concerns including the lack of sparsity sensitivity analysis and efficiency evaluation remain insufficiently addressed. I will maintain my original score.
Additional Results on Sparsity and Full-Catalog Efficiency
Thank you for your follow-up. Standard benchmarks in the cited next-basket recommendation literature compare models using ranking accuracy (Recall@K and NDCG@K) under a common protocol, which is why our main experiments focus on these metrics. To address your questions directly, we also ran the requested analyses of sparsity sensitivity, memory coverage, and inference latency under the same full-catalog test setup.
1. Sensitivity to the sparsity level
We varied the number of activated memory units using and evaluated full-catalog ranking performance:
| Dataset | Metric | (selected) | ||||
|---|---|---|---|---|---|---|
| CJ | NDCG@20 | 0.4761 | 0.4852 | 0.4841 | 0.4897 | 0.4902 |
| CJ | Recall@20 | 0.8620 | 0.8620 | 0.8620 | 0.8620 | 0.8620 |
| TF | NDCG@20 | 0.4213 | 0.4124 | 0.4326 | 0.4328 | 0.4323 |
| TF | Recall@20 | 0.8309 | 0.8162 | 0.8603 | 0.8456 | 0.8456 |
| DC | NDCG@20 | 0.8443 | 0.8425 | 0.8435 | 0.8386 | 0.8390 |
| DC | Recall@20 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
| IC | NDCG@20 | 0.6338 | 0.6358 | 0.6357 | 0.6350 | 0.6353 |
| IC | Recall@20 | 0.8759 | 0.8780 | 0.8762 | 0.8772 | 0.8776 |
Performance stays stable across all tested values of . The selected setting () is within 0.0009 of the best tested NDCG@20 on TF, DC, and IC, and within 0.0061 on CJ, showing that it works reliably across datasets without per-dataset tuning.
2. Effective memory coverage at
Across the constructed outcome memory, 1,620 of the 4,064 memory units receive nonzero support on CJ, 1,062 on TF, 1,028 on DC, and 1,824 on IC, corresponding to 25.3%-44.9% of the full memory pool. Activating units for each pair therefore controls per-candidate sparsity, while different history-candidate pairs populate a broad set of memory units rather than a fixed seven-unit subset.
3. Full-catalog inference latency
We measured full-catalog inference latency per query on the same machine using the selected checkpoints, reporting the median over three test-set runs:
| Method | Average latency (ms/query) |
|---|---|
| FlySCoRe-PROB | 7.40 |
| FlySCoRe-CNS | 17.31 |
| Triple2Vec | 9.65 |
| Sets2Sets | 10.44 |
| DREAM | 17.27 |
| TIDE | 72.08 |
| DNNTSP | 72.50 |
| SFCNTSP | 72.79 |
With connectome-constrained addressing, FlySCoRe-CNS stays under 45 ms/query on every dataset and averages 17.31 ms/query, more than four times faster than the temporal-set models TIDE, DNNTSP, and SFCNTSP. Its probabilistic variant, FlySCoRe-PROB, further reduces average latency by 57.3% to 7.40 ms/query, making it faster than all listed learned history and sequence baselines.
We hope these additional results address your remaining concerns and support reconsideration of your score.
Thank you for your review. Below, we clarify the computational scope, the exact empirical comparison between sparse and dense addressing, the fixed operating point, and memory utilization.
W1 / Q1: We construct memory offline and do not rescan histories during inference.
FlySCoRe pre-aggregates eligible historical support into S/N outcome memory via Eqs. (2)--(3). Once memory is populated, candidates are encoded, sparsely addressed, and scored directly from stored memory statistics without rescanning all historical observations for every product in the catalog. Offline memory construction from training histories is thus separated from inference-time ranking. Our primary empirical claim focuses on quantifying the ranking-accuracy contribution of candidate-dependent evidence under a unified full-catalog protocol, whereas wall-clock latency benchmarks depend on hardware, batching, and low-level implementation choices and represent a separate systems evaluation.
W2: Sparse CNS outperforms retrained dense addressing on all four datasets.
We would like to clarify the exact numbers from Tables 2 and 5: sparse CNS actually outperforms retrained dense addressing in NDCG@20 on all four datasets, rather than trailing on three: CJ: 0.4841 (Sparse CNS) vs. 0.4731 (Retrained dense) TF: 0.4326 (Sparse CNS) vs. 0.4152 (Retrained dense) DC: 0.8435 (Sparse CNS) vs. 0.8412 (Retrained dense) IC: 0.6357 (Sparse CNS) vs. 0.6315 (Retrained dense)
As specified in Appendix E.5, this control "removes sparse selection and refits the encoder and signal strength under the original training protocol and matched initialization." These matched results directly validate the empirical advantage of sparse, selective addressing over a refitted dense alternative across every evaluated dataset, while the K bypass complements this comparison by measuring the joint contribution of sparse addressing and the personal signal.
W3 / Q2: We use the same fixed k=7 across all four datasets.
In our experiments, we deliberately fix the number of active units to k=7 across all four datasets without per-dataset tuning (Appendix B.3) and compare this single operating point against its matched retrained dense counterpart. The four-dataset evaluation therefore tests a unified sparse regime rather than dataset-specific capacity tuning. Because the same sparse addressing rule governs both memory writing and reading, k defines the coupled memory interface rather than an inference-time threshold; changing it requires rebuilding and refitting both sides of that interface.
W4 / Q3: Address selection does not depend on accumulated memory support.
As defined in Eqs. (1), (4), and (5), memory routing is computed solely from current history-candidate features and learned parameters prior to reading accumulated S/N counts. Address selection is therefore independent of how much support a memory unit already holds, while non-uniform occupancy across units naturally reflects the uneven prevalence of underlying shopping situations in the data. From a modeling standpoint, the relevant property is candidate discrimination: as shown in Section 3.3.1, if all candidates shared an identical address, they would retrieve identical totals and leave the personal-signal ranking unchanged. Both the matched M bypass and the sparse-versus-dense comparison directly verify that candidate-dependent memory access improves ranking performance.
We hope these clarifications address your concerns, and we would be happy to discuss any remaining points.