- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper dynamically adjusts user and item embedding sizes in streaming recommender systems where interactions arrive continuously without replay. Existing dynamic methods either expand embeddings based solely on total interaction count or shrink embeddings to fit memory budgets.
To solve this, TAMRec uses a reinforcement learning controller built on three ideas:
Time-Aware State: It tracks total interaction count, current embedding size, and time elapsed since the last interaction to separate bursty clicks from genuine interest changes.
Grow-Only Actions: It restricts decisions to Maintain or Enlarge (from 4 up to 128 dimensions), eliminating shrink-induced forgetting. Peer-Group Reward Baseline: It rewards expansion only when prediction loss improves over a moving baseline, using same-frequency peers as a fallback reference for cold-start entities.
Tested across five streaming datasets and two backbones (MLP and NeuMF), TAMRec improves accuracy, avoids forgetting, and saves memory on sparse datasets. However, its appendix proofs are mathematically flawed, and projecting all embeddings to size 128 before prediction means smaller embeddings do not reduce runtime.
Critical strengths
Decoupling of temporal recency from cumulative frequency and shrink-induced forgetting are well motivated.
The authors have conducted multi-backbone streaming evaluation covering accuracy, storage, retention, and runtime breakdown (Section 4 & Appendices C–H), which are very comprehensive. The eval is under a strict inference-then-training streaming protocol across five datasets (MovieLens-20M, MovieLens-Latest, Netflix, Amazon Beauty, LastFM-1K) and two backbones (MLP and NeuMF), and transparently report cases where TAMRec is not the most compact (LastFM-1K user ratio 0.9900) or does not achieve the lowest cross-entropy loss.
Critical weaknesses
Although the abstract and Section 3.3 claim six theoretical guarantees (convergence, unbiased variance-reduced gradients, zero forgetting, on-demand optimality, and bounded storage growth), the theorem in Appendix A.3 seems to be mathematically flawed or contradicts the main text.
I also find internal numerical contradictions between main-text and appendix tables: for example
MovieLens-20M (MLP) Accuracy & CE Loss: Reported as 0.4539 / 1.2477 in Table 1 (Line 280), but as 0.4540 / 1.2486 in Section 4.5 (Line 380) and Table 13 (Line 1521). There are also implausibly rounded baseline numbers in table 2 and table 3. Below analysis is from LLM:
1. Mathematically Impossible HR vs. NDCG Pairs in Table 2 (MLP Baselines)
In single-target next-item ranking (Amazon Beauty and LastFM-1K), let be the rank of the ground-truth item for a test interaction:
- .
- .
Because every rank has discount weight strictly bounded between (at ) and (at ), every valid ranking evaluation must satisfy:
In Table 2 (Lines 293–297), 9 out of 12 MLP baseline entries violate this mathematical bound:
Dataset (MLP) | Method | HR@10 | HR@20 | NDCG@10 | NDCG@20 | Ratio | Mathematical Verdict | ||
|---|---|---|---|---|---|---|---|---|---|
| Amazon Beauty | AutoEmb | 0.1161 | 0.1244 | 0.0083 | 0.0605 | 0.0696 | 0.0091 | 1.0964 | ❌ Impossible (!) |
| Amazon Beauty | ESAPN | 0.1267 | 0.1373 | 0.0106 | 0.0652 | 0.0718 | 0.0066 | 0.6226 | ❌ Impossible ( max bound 0.2789) |
| Amazon Beauty | MultiEmb | 0.1210 | 0.1311 | 0.0101 | 0.0794 | 0.0831 | 0.0037 | 0.3663 | ❌ Impossible () |
| Amazon Beauty | SCALL(c=32) | 0.1251 | 0.1583 | 0.0332 | 0.0720 | 0.0815 | 0.0095 | 0.2861 | ❌ Exceeds 0.2789 |
| LastFM-1K | MultiEmb | 0.1784 | 0.2553 | 0.0769 | 0.1050 | 0.1300 | 0.0250 | 0.3251 | ❌ Impossible () |
| LastFM-1K | AutoEmb | 0.1420 | 0.2200 | 0.0780 | 0.0800 | 0.1050 | 0.0250 | 0.3205 | ❌ Impossible () |
| LastFM-1K | SCALL(c=32) | 0.1650 | 0.2455 | 0.0805 | 0.0952 | 0.1204 | 0.0252 | 0.3130 | ❌ Impossible () |
| LastFM-1K | FIXED | 0.1474 | 0.2304 | 0.0830 | 0.0850 | 0.1100 | 0.0250 | 0.3012 | ❌ Impossible () |
| LastFM-1K | ESAPN | 0.1509 | 0.2380 | 0.0871 | 0.0850 | 0.1100 | 0.0250 | 0.2870 | ❌ Impossible () |
- Look closely at
AutoEmb (MLP)onAmazon Beauty: is larger than , which is impossible even for rank-1 hits, let alone ranks – (where the maximum possible is ). - Look at
LastFM-1K (MLP): five different baselines (FIXED,DARTS,ESAPN,AutoEmb,MultiEmb) all haveNDCG@10andNDCG@20rounded to exact multiples of0.0050, and all five have the exact same difference !
2. Cross-Table Contradictions for the Exact Same Model (TAMRec)
| Configuration & Metric | Location A | Location B | Location C | Discrepancy |
|---|---|---|---|---|
MovieLens-20M (MLP, TAMRec) Accuracy / CE Loss | Table 1 (Line 280):<br>0.4539 / 1.2477 | Section 4.5 (Line 380) & Table 13 (Line 1521):<br>0.4540 / 1.2486 | — | Accuracy differs by +0.0001 and CE Loss by +0.0009 for the same baseline run. |
Amazon Beauty (MLP, TAMRec) User / Item Embedding Ratio | Table 3 (Line 334) & Section 4.3 (Line 316):<br>User = 0.0409 ()<br>Item = 0.1168 () | Tables 10–12 (Lines 1202, 1210, 1220) & Table 14 (Line 1532):<br>User = 0.0365 ()<br>Item = 0.1219 () | Table 8 (Line 1121) & Section 4.6 (Line 431):<br>User = 0.0352 ()<br>Item = 0.1273 () | Three different User/Item ratios across the paper for the exact same Amazon Beauty model (HR@10=0.1316, NDCG@10=0.0713, HR@20=0.1700, NDCG@20=0.0810). |
| Training Batch Size & Learning Rate | Appendix B.1 (Lines 984–990):<br>Batch size = 500, LR = 1e-4 | Appendix C.1 (Lines 1091–1092):<br>Batch size = 1,024, LR = 1e-3 | — | batch size and learning rate mismatch despite claiming "identical hardware and optimization settings". |
3. Contradictions Between Figures (Figure 2 & Figure 3) and the Main/Appendix Text
Figure 3(RQ5, 60077_img-020.png) vs. Section 4.6 (Line 396) & Appendix F (Line 1420):- On
Figure 3itself, the green value printed directly aboveTAMRecat6-20 (Long-tail)is0.3888(just aboveSCALL's0.3823). - However, both Section 4.6 (
Line 396) and Appendix F (Line 1420) reportTAMRec's accuracy on6–20 interactionsas0.3988(+0.0100higher than the plotted value).
- On
Figure 2(a)(RQ3, 60077_img-016.png) vs.Eq. (3)(Line 159) & Section 4.4 (Line 361):Eq. (3)defines , so means frequently accessed () and means rarely accessed ().- Yet the x-axis printed on
Figure 2(a)is labeledTime Interval (0=Frequently Accessed, 1=Rarely Accessed)—the exact opposite ofEq. (3)! - Even worse, on
Figure 2(a), the curve is~24.6atx = 0.0(~32.6atx = 0.05), peaks at~36.2atx = 0.6, and drops to~7.7atx = 1.0. In Section 4.4 (Line 361), the authors write: "Capacity is low near 1 (frequent recent interactions), peaks around 0.6, and is lower near 0 (prolonged inactivity)"—contradicting both the x-axis label onFigure 2(a)and the relative heights at0(~25) vs.1(~7.7).
Figure 2(b)(60077_img-018.png) vs.Theorem 6(Line 909):Figure 2(b)shows that at , the average allocated dimension reaches128.0( of ), contradictingTheorem 6's claim that dimensions saturate strictly below .
4. Implausible Rounding, Constant-Multiplier Rows, and Identical Ratios (Tables 2, 3, 10–14)
- Trailing-Zero Clusters & Constant-Grid Numbers in
Table 2andTable 3:- Despite averaging over
2,572,765test interactions onLastFM-1K, the baseline rows inTable 2contain 17 four-decimal numbers ending in0(including0.0850,0.1100,0.2320,0.0750,0.1000,0.2380,0.0850,0.1100,0.1420,0.2200,0.0800,0.1050,0.1650,0.1050,0.1300,0.2760,0.2530,0.2400,0.3100,0.1480,0.1650). - In
Table 3(NeuMF, line 338),SCALL(c=32)'s User/Item ratios acrossLastFM-1K,MovieLens-20M,MovieLens-Latest, andNetflixare0.2440 / 0.2375,0.2420 / 0.2360,0.2460 / 0.2410, and0.2430 / 0.2450—7 out of 8 numbers end in0, and the four User ratios are literally{0.2420, 0.2430, 0.2440, 0.2460}.
- Despite averaging over
- Exact Scalar-Multiple Row in
Table 14(LastFM-1KAblation, Line 1540):- Dividing the four metrics of
w/o Time-Aware(0.1569, 0.0913, 0.2174, 0.1066) byTAMRec(0.1892, 0.1101, 0.2621, 0.1285) gives: All four metrics (HR@10,NDCG@10,HR@20,NDCG@20) were scaled by the exact same factor0.82940.0002!
- Dividing the four metrics of
- Identical Rows and Constant Shifts in
Table 10andTable 12(Amazon BeautySensitivity):- In
Table 12(Lines 1218 & 1220), the entire 5-column row for (0.1316, 0.0713, 0.1700, 0.0810, 0.1219) is 100% identical to . - At (
Line 1221), bothHR@10andHR@20drop by the exact same-0.0011(0.1316 -> 0.1305and0.1700 -> 0.1689), while bothNDCG@10andNDCG@20drop by the exact same-0.0007(0.0713 -> 0.0706and0.0810 -> 0.0803). Similarly, inTable 10(K = 10, line 1203), bothNDCG@10andNDCG@20drop by the exact same-0.0026(0.0713 -> 0.0687and0.0810 -> 0.0784).
- In
- Identical User and Item Ratios in
Table 3andTable 13:- In
Table 13(Line 1524),w/o Group PrioronMovieLens-20MreportsUser Ratio / Item Ratio = 0.2936 / 0.2936(identical to 4 decimal places despite138,493users vs.27,278items). - In
Table 3(Line 332),SCALL(c=32)(MLP) reports identical User ratios (0.1846) onLastFM-1KandMovieLens-Latest, and identical User and Item ratios (0.2350 / 0.2350) onNetflix.
- In
5. Arithmetic Errors in Appendices C and D
- Appendix C.3 (
Line 1133vs.Table 9,Line 1142): Line 1133 claims "RS-side policy lookup and embedding transfer: +1.45 ms", whereasTable 9listsPolicy Fwd(RS) = 1.73 msandEmb Transfer(RS) = 1.54 ms(1.73 + 1.54 = 3.27 ms, not1.45 ms). - Appendix C.2 (
Line 1108vs.Table 8,Line 1121): InTable 8,TAMRechas and , which sum to (not ), and the percentage difference is (not as claimed in Line 1108). - Appendix D.2 (
Line 1227vs.Tables 10–11): Line 1227 states: "The largest gap from the default is 0.0024 for (at ), 0.0006 for (at ), and 0.0011 for (at )." In reality, from the default0.1316, the gap at (0.1291) is0.0025(not0.0024), and the gap at (0.1307) is0.0009(not0.0006, which is the gap at whereHR@10 = 0.1322).
Questions for authors
see above
3 comments
Q3.2 Figure 2(a) vs. Eq. (3) & Section 4.4: Eq. (3) defines Δτ=exp(−Δτ_raw/τ_day)+ε, so Δτ≈1 means frequently accessed and Δτ≈0 means rarely accessed. Yet the x-axis printed on Figure 2(a) is labeled Time Interval (0=Frequently Accessed, 1=Rarely Accessed)—the exact opposite of Eq. (3).
Thank you for pointing out this directional inconsistency. According to our formula : corresponds to recent/frequent access, while means long-term inactivity. The labeling on the x-axis of Figure 2(a) was indeed reversed. We will rename the x-axis to time decay feature, with annotations: long-term inactive, recent access. The curve peaks around 0.6 with lower values at both ends; this curve shape is consistent with our qualitative description that “capacity is lower at both ends”. Only the x-axis labels need correction, and the experimental curve itself remains unchanged. This similar issue was also raised by Reviewer 1.
Q3.3 Figure 2(b) vs. Theorem 6: Figure 2(b) shows that at log₁₀(Frequency)=4.0, the average allocated dimension reaches 128.0 (100% of d_max), contradicting Theorem 6’s claim that dimensions saturate strictly below d_max.
Thank you for identifying this inconsistency between our theory and figure. Figure 2(b) plots the conditional average allocated dimension per group, and the mean reaches for high-frequency groups. The original claim in Theorem 6 that the saturation value is strictly smaller than is indeed too strong and conflicts with experimental observations. We revise Theorem 6 by removing the statement “strictly smaller than ” and only retain the capacity upper bound derived from the discrete, monotone, bounded action space. After revision, the fixed point is allowed to equal , consistent with the observations in Figure 2(b).
Q4.1 "Trailing-Zero Clusters & Constant-Grid Numbers in Table 2 and Table 3: Despite averaging over 2,572,765 test interactions on LastFM-1K, the baseline rows in Table 2 contain 17 four-decimal numbers ending in 0. In Table 3, SCALL (c=32) User/Item ratios across several datasets also contain many values ending in 0, with four user ratios literally {0.2420, 0.2430, 0.2440, 0.2460}.*
Thank you for carefully examining the numerical distribution. Table 2 and Table 3 uniformly report values to four decimal places. Trailing zeros and clustered values within a narrow range are merely visual artifacts caused by display precision and do not indicate problematic data. For SCALL (, maximum dimension ), its total capacity budget is fixed, so the normalized dimension ratio is naturally constrained around 0.25. The observed set arises from the intrinsic mechanism of this baseline, rather than artificial grid-based construction. In the revised manuscript, we will add descriptions of the metric calculation pipeline and precision convention to clearly distinguish numerical patterns induced by algorithm constraints from manual post-processing.
Q4.2 Exact Scalar-Multiple Row in Table 14 (LastFM-1K Ablation): Dividing the four metrics of w/o Time-Aware (0.1569, 0.0913, 0.2174, 0.1066) by TAMRec (0.1892, 0.1101, 0.2621, 0.1285) gives 0.8293, 0.8292, 0.8295, and 0.8296. All four metrics were scaled by almost the same factor 0.8294±0.0002.
Thank you for your careful numerical verification. The results for w/o Time-Aware in Table 4 are obtained from a complete and independent ablation experiment. The ratios of the four metrics are highly close, which indicates that after removing the time-aware module, HR and NDCG exhibit nearly consistent relative degradation under these two K values. Mathematically, there is no constraint requiring these relative ratios to be distinct from each other. This observation is an empirical result under this ablation setting, and the close ratios alone cannot be taken as evidence that the metrics were artificially scaled by a fixed factor. We will add a note in the revised manuscript: this ablated component induces approximately proportional performance degradation across different ranking metrics.
Q4.3 Identical Rows and Constant Shifts in Table 10 and Table 12 (Amazon Beauty Sensitivity): In Table 12, the entire 5-column row for β=0.005 is 100% identical to β=0.5. At β=1.0, HR@10 and HR@20 drop by the exact same −0.0011, while both NDCG@10 and NDCG@20 drop by the exact same −0.0007. Similarly, in Table 10, NDCG@10 and NDCG@20 drop by the exact same amount.
Thank you for carefully checking our sensitivity tables. All groups with different and in Table 10 and Table 12 are from independent full runs. The duplicate value comes from a table typo: the entry labeled was originally the result for . This explains why its numbers match those of . On the Amazon Beauty dataset, the regularizer exerts negligible influence on the allocation strategy for , forming a flat parameter plateau with steady metrics. The consistent drop in HR@10/HR@20 and NDCG@10/NDCG@20 at reflects the model’s empirical behavior under this strong regularization. We will correct to and proofread all table transcriptions throughout the manuscript.
Q4.4 "Identical User and Item Ratios in Table 3 and Table 13: Table 13 reports User Ratio / Item Ratio = 0.2936 / 0.2936 for MovieLens-20M w/o Group Prior despite 138,493 users vs. 27,278 items. In Table 3, SCALL (c=32) also reports identical user ratios across different datasets, and identical user/item ratios on Netflix."
Thank you for pointing out this numerical observation. Given different total entity counts, there is no requirement that the average dimension ratios for users and items must differ. The average ratio is the mean of allocated dimensions over all entities, an independent statistic; the user-side and item-side means can coincide. For MovieLens-20M w/o Group Prior: after removing the group prior, the allocation strategies for users and items converge to similar average ratios, which round to 0.2936. For SCALL (): its fixed total capacity budget constrains normalized dimension ratios within a narrow range, so similar or identical ratios across datasets naturally occur. This empirical outcome arises from algorithmic constraints plus statistical averaging and contains no mathematical contradiction. We will add explanations for the allocation constraints of these two baselines in the revised manuscript and clarify that average ratios are not one-to-one tied to total entity counts.
Q5.1 Appendix C.3 vs. Table 9: Line 1133 claims ‘RS-side policy lookup and embedding transfer: +1.45 ms’, whereas Table 9 lists Policy Fwd (RS)=1.73 ms and Emb Transfer (RS)=1.54 ms (1.73+1.54=3.27 ms, not 1.45 ms).
Thank you for identifying the ambiguous timing label between the appendix and Table 9. The textual description “RS-side policy lookup and embedding transfer: 1.45 ms” was incorrect. This 1.45 ms value was not the direct sum of Policy Fwd (RS) + Emb Transfer (RS) in Table 9. It originally represented the net incremental overhead after subtracting the controller update cost, but its label was misleading and conflicted with the component-wise breakdown. In the revision, we strictly separate the two timing definitions:
• Table 9 reports standalone measured latency for each module (Policy Fwd, Emb Transfer).
• Appendix C.3 will no longer label this net residual as lookup+transfer. If we cannot reliably reproduce this residual value with repeated timing runs, we will remove the 1.45 ms entry entirely and drop this ambiguous decomposition.
Q5.2 Appendix C.2 vs. Table 8: TAMRec has T_train=0.41 min and T_test=16.19 min, which sum to 16.60 min (not T_total=16.61 min), and the percentage difference is (16.61−16.85)/16.85=−1.42% (not −1.47% as claimed).
Thank you for catching this arithmetic error in Appendix C.2. Given min and min, their correct sum is min, while the manuscript incorrectly wrote min. This mistake propagated to the derived percentage difference. We will correct the total runtime, recalculate the percentage gap, and update all dependent values in Appendix C.2 accordingly.
Q5.3 Appendix D.2 vs. Tables 10–11: The claimed largest gaps do not match the displayed values; for example, the gap for K, λ, and β is miscomputed from the corresponding HR@10 entries.
Thank you for identifying the inconsistency in Appendix D.2. The reported maximum performance gaps for contain transcription and calculation errors and do not match the raw HR@10 values in Tables 10–11. We will recalculate all maximum gaps strictly from the measured table values and revise the descriptions in the appendix. We will also add a note that these three sensitivity experiments were run independently. The table results themselves remain unchanged; only the summary gaps in the text will be corrected.
Q1:
Thank you for your great analysis.
The bound derived by the reviewer is valid under its stated premise. In our evaluation, each test interaction contains only one positive item and full-item ranking is adopted. However, for Table 2, all methods and baselines run a complete streaming training and evaluation pipeline separately for (K=10) and (K=20). Each experiment reinitializes the model, and the model is updated with the batch immediately after prediction and evaluation for each test batch (Appendix B.3). Accordingly, the @10 and @20 metrics in the table are not sliced from the same Top-20 ranking; they come from two independent online experiments.
Under this evaluation protocol, the ratio provided by the reviewer is a per-sample identity for a single ranking and cannot be directly used to claim algebraic contradictions in Table 2, since the numerator and denominator of this relation correspond to two distinct experimental trajectories.
Q2.1: Cross-Table Contradictions for the Exact Same Model (TAMRec) MovieLens-20M (MLP, TAMRec) Accuracy / CE Loss is reported as 0.4539 / 1.2477 in Table 1, but as 0.4540 / 1.2486 in Section 4.5 and Table 13. Amazon Beauty (MLP, TAMRec) User / Item Embedding Ratio is also reported differently across Table 3, Section 4.3 and Tables 10–12, and Table 8 / Section 4.6.
Thank you for pointing out this issue.
These experiments share identical hyperparameter configurations but are independent repeated runs, rather than reusing the same trained model instance. The main performance experiments, ablation studies, and efficiency experiments are conducted separately to address different research questions, with the model and policy controller reinitialized in every run. The sampling and online update procedures in reinforcement learning are stochastic, so minor variations in metrics and average embedding dimensions are expected random fluctuations, rather than transcription errors in the manuscript.
For example, the two sets of results on MovieLens-20M: 0.4539 / 1.2477 and 0.4540 / 1.2486 come from two separate streaming runs. The accuracy gap is only 0.0001 and the CE loss gap is only 0.0009. For Amazon Beauty across three independent runs:
• User average embedding ratio: 0.0409, 0.0365, 0.0352, with a maximum difference of 0.0057
• Item average embedding ratio: 0.1168, 0.1219, 0.1273, with a maximum difference of 0.0105
These discrepancies stem from random variance across independent training trajectories and do not represent numerical inconsistencies within a single model instance.
We will add a unified note in the paper to clarify that different tables correspond to separate experiments.
Q2.2 Training Batch Size & Learning Rate: Appendix B.1 reports Batch size = 500, LR = 1e-4, whereas Appendix C.1 reports Batch size = 1,024, LR = 1e-3, despite claiming ‘identical hardware and optimization settings’.
Thank you for pointing out this ambiguous statement in the appendix. The actual configuration is as follows: Amazon Beauty and LastFM-1K use a batch size of 1024; the remaining three datasets use a batch size of 500. The phrase “identical hardware and optimization settings” in our manuscript means that within the same dataset, TAMRec and all baselines adopt exactly the same configurations for fair comparison. It does not require a unified batch size or learning rate across all five datasets. This is a writing error in Appendix B.1. We will correct the parameters in Appendix B.1 and C.1, and explicitly clarify in the revised paper that hyperparameters are aligned among methods within one dataset, while separate tuned hyperparameters can be used across different datasets.
Q3.1 Figure 3 vs. Section 4.6 & Appendix F: On Figure 3 itself, the green value printed directly above TAMRec at 6–20 (Long-tail) is 0.3888 (just above SCALL’s 0.3823). However, both Section 4.6 and Appendix F report TAMRec’s accuracy on 6–20 interactions as 0.3988 (+0.0100 higher than the plotted value).
Thank you for your careful check. The correct value for the long-tail segment with 6–20 interactions in Figure 3 is 0.3888. The value 0.3988 in Section 4.6 and Appendix F is a transcription typo. We will update both the main text and appendix consistently to 0.3888. This correction does not alter our core conclusion: TAMRec (0.3888) remains slightly better than SCALL (0.3823).