acceptodds
Under review as a conference paper at ICLR 2027

TAMRec: Time-Aware and Monotonic Reinforcement Learning for On-Demand Embedding Dimensions in Streaming Recommendation

Abstract

In deep streaming recommender systems, fixed embedding dimensions are inefficient for streaming data, where user-item interactions and activity patterns continuously evolve. Existing dynamic adjustment methods, including reinforcement learning-based approaches, suffer from three critical issues: parameter inflation, catastrophic forgetting caused by shrink operations, and unstable rewards for cold-start. To address these challenges, this paper introduces TAMRec, employing a time-aware policy that decouples interaction frequency from capacity demand, and adopts a monotonic Maintain/Enlarge action space to eliminate shrink-induced forgetting. While monotonic expansion risks uncontrolled parameter growth, an adaptive reward mechanism encourages on-demand allocation: enlargement is favored only when it yields prediction performance above a moving baseline. A group-prior fallback further stabilizes learning for cold-start entities with sparse feedback. Theoretically, we prove that the policy converges to a local optimum with near-zero catastrophic forgetting, on-demand optimality, and bounded storage growth under mild assumptions. Experiments on five benchmark datasets demonstrate that TAMRec delivers competitive or better recommendation performance with compact embeddings and less forgetting in streaming settings. We release the datasets and core codes at: https://anonymous.4open.science/r/STARLA-8887YZ/

Primary area:
reinforcement learning
Keywords:
Streaming recommendation, Embedding dimension allocation, Reinforcement learning, Catastrophic forgetting, Cold-start

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RooXOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper dynamically adjusts user and item embedding sizes in streaming recommender systems where interactions arrive continuously without replay. Existing dynamic methods either expand embeddings based solely on total interaction count or shrink embeddings to fit memory budgets.

To solve this, TAMRec uses a reinforcement learning controller built on three ideas:

Time-Aware State: It tracks total interaction count, current embedding size, and time elapsed since the last interaction to separate bursty clicks from genuine interest changes.

Grow-Only Actions: It restricts decisions to Maintain or Enlarge (from 4 up to 128 dimensions), eliminating shrink-induced forgetting. Peer-Group Reward Baseline: It rewards expansion only when prediction loss improves over a moving baseline, using same-frequency peers as a fallback reference for cold-start entities.

Tested across five streaming datasets and two backbones (MLP and NeuMF), TAMRec improves accuracy, avoids forgetting, and saves memory on sparse datasets. However, its appendix proofs are mathematically flawed, and projecting all embeddings to size 128 before prediction means smaller embeddings do not reduce runtime.

Critical strengths

Decoupling of temporal recency from cumulative frequency and shrink-induced forgetting are well motivated.

The authors have conducted multi-backbone streaming evaluation covering accuracy, storage, retention, and runtime breakdown (Section 4 & Appendices C–H), which are very comprehensive. The eval is under a strict inference-then-training streaming protocol across five datasets (MovieLens-20M, MovieLens-Latest, Netflix, Amazon Beauty, LastFM-1K) and two backbones (MLP and NeuMF), and transparently report cases where TAMRec is not the most compact (LastFM-1K user ratio 0.9900) or does not achieve the lowest cross-entropy loss.

Critical weaknesses

Although the abstract and Section 3.3 claim six theoretical guarantees (convergence, unbiased variance-reduced gradients, zero forgetting, on-demand optimality, and bounded storage growth), the theorem in Appendix A.3 seems to be mathematically flawed or contradicts the main text.

I also find internal numerical contradictions between main-text and appendix tables: for example

MovieLens-20M (MLP) Accuracy & CE Loss: Reported as 0.4539 / 1.2477 in Table 1 (Line 280), but as 0.4540 / 1.2486 in Section 4.5 (Line 380) and Table 13 (Line 1521). There are also implausibly rounded baseline numbers in table 2 and table 3. Below analysis is from LLM:

1. Mathematically Impossible HR vs. NDCG Pairs in Table 2 (MLP Baselines)

In single-target next-item ranking (Amazon Beauty and LastFM-1K), let r∈{1,2,… }r \in \{1, 2, \dots\} be the rank of the ground-truth item for a test interaction:

  • ΔHR:=HR@20−HR@10=P(11≤r≤20)\Delta\text{HR} := \text{HR@20} - \text{HR@10} = \mathbb{P}(11 \le r \le 20).
  • ΔNDCG:=NDCG@20−NDCG@10=E ⁣[I(11≤r≤20)log⁡2(r+1)]\Delta\text{NDCG} := \text{NDCG@20} - \text{NDCG@10} = \mathbb{E}\!\left[\frac{\mathbb{I}(11 \le r \le 20)}{\log_2(r+1)}\right].

Because every rank r∈{11,…,20}r \in \{11, \dots, 20\} has discount weight 1log⁡2(r+1)\frac{1}{\log_2(r+1)} strictly bounded between 1log⁡2(21)≈0.2277\frac{1}{\log_2(21)} \approx \mathbf{0.2277} (at r=20r=20) and 1log⁡2(12)≈0.2789\frac{1}{\log_2(12)} \approx \mathbf{0.2789} (at r=11r=11), every valid ranking evaluation must satisfy: 0.2277≤NDCG@20−NDCG@10HR@20−HR@10≤0.2789.0.2277 \le \frac{\text{NDCG@20} - \text{NDCG@10}}{\text{HR@20} - \text{HR@10}} \le 0.2789.

In Table 2 (Lines 293–297), 9 out of 12 MLP baseline entries violate this mathematical bound:

Dataset (MLP)MethodHR@10HR@20ΔHR\Delta\text{HR}NDCG@10NDCG@20ΔNDCG\Delta\text{NDCG}Ratio ΔNDCGΔHR\frac{\Delta\text{NDCG}}{\Delta\text{HR}}Mathematical Verdict
Amazon BeautyAutoEmb0.11610.12440.00830.06050.06960.00911.0964❌ Impossible (ΔNDCG>ΔHR\Delta\text{NDCG} > \Delta\text{HR}!)
Amazon BeautyESAPN0.12670.13730.01060.06520.07180.00660.6226❌ Impossible (>2.2×> 2.2\times max bound 0.2789)
Amazon BeautyMultiEmb0.12100.13110.01010.07940.08310.00370.3663❌ Impossible (>0.2789> 0.2789)
Amazon BeautySCALL(c=32)0.12510.15830.03320.07200.08150.00950.2861❌ Exceeds 0.2789
LastFM-1KMultiEmb0.17840.25530.07690.10500.13000.02500.3251❌ Impossible (>0.2789> 0.2789)
LastFM-1KAutoEmb0.14200.22000.07800.08000.10500.02500.3205❌ Impossible (>0.2789> 0.2789)
LastFM-1KSCALL(c=32)0.16500.24550.08050.09520.12040.02520.3130❌ Impossible (>0.2789> 0.2789)
LastFM-1KFIXED0.14740.23040.08300.08500.11000.02500.3012❌ Impossible (>0.2789> 0.2789)
LastFM-1KESAPN0.15090.23800.08710.08500.11000.02500.2870❌ Impossible (>0.2789> 0.2789)
  • Look closely at AutoEmb (MLP) on Amazon Beauty: ΔNDCG=0.0091\Delta\text{NDCG} = 0.0091 is larger than ΔHR=0.0083\Delta\text{HR} = 0.0083, which is impossible even for rank-1 hits, let alone ranks 1111–2020 (where the maximum possible ΔNDCG\Delta\text{NDCG} is 0.2789×0.0083=0.00230.2789 \times 0.0083 = 0.0023).
  • Look at LastFM-1K (MLP): five different baselines (FIXED, DARTS, ESAPN, AutoEmb, MultiEmb) all have NDCG@10 and NDCG@20 rounded to exact multiples of 0.0050, and all five have the exact same difference NDCG@20−NDCG@10=0.0250\text{NDCG@20} - \text{NDCG@10} = 0.0250!

2. Cross-Table Contradictions for the Exact Same Model (TAMRec)

Configuration & MetricLocation ALocation BLocation CDiscrepancy
MovieLens-20M (MLP, TAMRec) Accuracy / CE LossTable 1 (Line 280):<br>0.4539 / 1.2477Section 4.5 (Line 380) & Table 13 (Line 1521):<br>0.4540 / 1.2486—Accuracy differs by +0.0001 and CE Loss by +0.0009 for the same baseline run.
Amazon Beauty (MLP, TAMRec) User / Item Embedding RatioTable 3 (Line 334) & Section 4.3 (Line 316):<br>User = 0.0409 (dˉu=5.24\bar{d}_u = 5.24)<br>Item = 0.1168 (dˉi=14.95\bar{d}_i = 14.95)Tables 10–12 (Lines 1202, 1210, 1220) & Table 14 (Line 1532):<br>User = 0.0365 (dˉu=4.67\bar{d}_u = 4.67)<br>Item = 0.1219 (dˉi=15.60\bar{d}_i = 15.60)Table 8 (Line 1121) & Section 4.6 (Line 431):<br>User = 0.0352 (dˉu=4.5\bar{d}_u = 4.5)<br>Item = 0.1273 (dˉi=16.3\bar{d}_i = 16.3)Three different User/Item ratios across the paper for the exact same Amazon Beauty model (HR@10=0.1316, NDCG@10=0.0713, HR@20=0.1700, NDCG@20=0.0810).
Training Batch Size & Learning RateAppendix B.1 (Lines 984–990):<br>Batch size = 500, LR = 1e-4Appendix C.1 (Lines 1091–1092):<br>Batch size = 1,024, LR = 1e-3—2×2\times batch size and 10×10\times learning rate mismatch despite claiming "identical hardware and optimization settings".

3. Contradictions Between Figures (Figure 2 & Figure 3) and the Main/Appendix Text

  1. Figure 3 (RQ5, 60077_img-020.png) vs. Section 4.6 (Line 396) & Appendix F (Line 1420):
    • On Figure 3 itself, the green value printed directly above TAMRec at 6-20 (Long-tail) is 0.3888 (just above SCALL's 0.3823).
    • However, both Section 4.6 (Line 396) and Appendix F (Line 1420) report TAMRec's accuracy on 6–20 interactions as 0.3988 (+0.0100 higher than the plotted value).
  2. Figure 2(a) (RQ3, 60077_img-016.png) vs. Eq. (3) (Line 159) & Section 4.4 (Line 361):
    • Eq. (3) defines Δt=exp⁡(−λ(τt−τlast)/Tday)+ϵ\Delta_t = \exp(-\lambda(\tau_t - \tau_{\text{last}})/T_{\text{day}}) + \epsilon, so Δt≈1\Delta_t \approx 1 means frequently accessed (Δτ≈0\Delta\tau \approx 0) and Δt≈0\Delta_t \approx 0 means rarely accessed (Δτ≫0\Delta\tau \gg 0).
    • Yet the x-axis printed on Figure 2(a) is labeled Time Interval (0=Frequently Accessed, 1=Rarely Accessed)—the exact opposite of Eq. (3)!
    • Even worse, on Figure 2(a), the curve is ~24.6 at x = 0.0 (~32.6 at x = 0.05), peaks at ~36.2 at x = 0.6, and drops to ~7.7 at x = 1.0. In Section 4.4 (Line 361), the authors write: "Capacity is low near 1 (frequent recent interactions), peaks around 0.6, and is lower near 0 (prolonged inactivity)"—contradicting both the x-axis label on Figure 2(a) and the relative heights at 0 (~25) vs. 1 (~7.7).
  3. Figure 2(b) (60077_img-018.png) vs. Theorem 6 (Line 909):
    • Figure 2(b) shows that at log⁡10(Frequency)=4.0\log_{10}(\text{Frequency}) = 4.0, the average allocated dimension reaches 128.0 (100%100\% of dmax⁡d_{\max}), contradicting Theorem 6's claim that dimensions saturate strictly below dmax⁡d_{\max}.

4. Implausible Rounding, Constant-Multiplier Rows, and Identical Ratios (Tables 2, 3, 10–14)

  1. Trailing-Zero Clusters & Constant-Grid Numbers in Table 2 and Table 3:
    • Despite averaging over 2,572,765 test interactions on LastFM-1K, the baseline rows in Table 2 contain 17 four-decimal numbers ending in 0 (including 0.0850, 0.1100, 0.2320, 0.0750, 0.1000, 0.2380, 0.0850, 0.1100, 0.1420, 0.2200, 0.0800, 0.1050, 0.1650, 0.1050, 0.1300, 0.2760, 0.2530, 0.2400, 0.3100, 0.1480, 0.1650).
    • In Table 3 (NeuMF, line 338), SCALL(c=32)'s User/Item ratios across LastFM-1K, MovieLens-20M, MovieLens-Latest, and Netflix are 0.2440 / 0.2375, 0.2420 / 0.2360, 0.2460 / 0.2410, and 0.2430 / 0.2450—7 out of 8 numbers end in 0, and the four User ratios are literally {0.2420, 0.2430, 0.2440, 0.2460}.
  2. Exact Scalar-Multiple Row in Table 14 (LastFM-1K Ablation, Line 1540):
    • Dividing the four metrics of w/o Time-Aware (0.1569, 0.0913, 0.2174, 0.1066) by TAMRec (0.1892, 0.1101, 0.2621, 0.1285) gives: 0.15690.1892=0.8293,0.09130.1101=0.8292,0.21740.2621=0.8295,0.10660.1285=0.8296.\frac{0.1569}{0.1892} = \mathbf{0.8293}, \quad \frac{0.0913}{0.1101} = \mathbf{0.8292}, \quad \frac{0.2174}{0.2621} = \mathbf{0.8295}, \quad \frac{0.1066}{0.1285} = \mathbf{0.8296}. All four metrics (HR@10, NDCG@10, HR@20, NDCG@20) were scaled by the exact same factor 0.8294 ±\pm 0.0002!
  3. Identical Rows and Constant Shifts in Table 10 and Table 12 (Amazon Beauty Sensitivity):
    • In Table 12 (Lines 1218 & 1220), the entire 5-column row for β=0\beta = 0 (0.1316, 0.0713, 0.1700, 0.0810, 0.1219) is 100% identical to β=0.5\beta = 0.5.
    • At β=1.0\beta = 1.0 (Line 1221), both HR@10 and HR@20 drop by the exact same -0.0011 (0.1316 -> 0.1305 and 0.1700 -> 0.1689), while both NDCG@10 and NDCG@20 drop by the exact same -0.0007 (0.0713 -> 0.0706 and 0.0810 -> 0.0803). Similarly, in Table 10 (K = 10, line 1203), both NDCG@10 and NDCG@20 drop by the exact same -0.0026 (0.0713 -> 0.0687 and 0.0810 -> 0.0784).
  4. Identical User and Item Ratios in Table 3 and Table 13:
    • In Table 13 (Line 1524), w/o Group Prior on MovieLens-20M reports User Ratio / Item Ratio = 0.2936 / 0.2936 (identical to 4 decimal places despite 138,493 users vs. 27,278 items).
    • In Table 3 (Line 332), SCALL(c=32) (MLP) reports identical User ratios (0.1846) on LastFM-1K and MovieLens-Latest, and identical User and Item ratios (0.2350 / 0.2350) on Netflix.

5. Arithmetic Errors in Appendices C and D

  • Appendix C.3 (Line 1133 vs. Table 9, Line 1142): Line 1133 claims "RS-side policy lookup and embedding transfer: +1.45 ms", whereas Table 9 lists Policy Fwd(RS) = 1.73 ms and Emb Transfer(RS) = 1.54 ms (1.73 + 1.54 = 3.27 ms, not 1.45 ms).
  • Appendix C.2 (Line 1108 vs. Table 8, Line 1121): In Table 8, TAMRec has Ttrain=0.41 minT_{\text{train}} = 0.41\text{ min} and Ttest=16.19 minT_{\text{test}} = 16.19\text{ min}, which sum to 16.60 min16.60\text{ min} (not Ttotal=16.61 minT_{\text{total}} = 16.61\text{ min}), and the percentage difference (16.61−16.85)/16.85(16.61 - 16.85) / 16.85 is −1.42%-1.42\% (not −1.47%-1.47\% as claimed in Line 1108).
  • Appendix D.2 (Line 1227 vs. Tables 10–11): Line 1227 states: "The largest gap from the default is 0.0024 for KK (at K=1K = 1), 0.0006 for λ\lambda (at λ=20\lambda = 20), and 0.0011 for β\beta (at β=1.0\beta = 1.0)." In reality, from the default 0.1316, the gap at K=1K = 1 (0.1291) is 0.0025 (not 0.0024), and the gap at λ=20\lambda = 20 (0.1307) is 0.0009 (not 0.0006, which is the gap at λ=2\lambda = 2 where HR@10 = 0.1322).
Questions for authors

see above

3 comments

Q1:

Thank you for your great analysis.

The bound derived by the reviewer is valid under its stated premise. In our evaluation, each test interaction contains only one positive item and full-item ranking is adopted. However, for Table 2, all methods and baselines run a complete streaming training and evaluation pipeline separately for (K=10)(K=10) and (K=20)(K=20). Each experiment reinitializes the model, and the model is updated with the batch immediately after prediction and evaluation for each test batch (Appendix B.3). Accordingly, the @10 and @20 metrics in the table are not sliced from the same Top-20 ranking; they come from two independent online experiments.

Under this evaluation protocol, the ratio provided by the reviewer is a per-sample identity for a single ranking and cannot be directly used to claim algebraic contradictions in Table 2, since the numerator and denominator of this relation correspond to two distinct experimental trajectories.

Q2.1: Cross-Table Contradictions for the Exact Same Model (TAMRec) MovieLens-20M (MLP, TAMRec) Accuracy / CE Loss is reported as 0.4539 / 1.2477 in Table 1, but as 0.4540 / 1.2486 in Section 4.5 and Table 13. Amazon Beauty (MLP, TAMRec) User / Item Embedding Ratio is also reported differently across Table 3, Section 4.3 and Tables 10–12, and Table 8 / Section 4.6.

Thank you for pointing out this issue.

These experiments share identical hyperparameter configurations but are independent repeated runs, rather than reusing the same trained model instance. The main performance experiments, ablation studies, and efficiency experiments are conducted separately to address different research questions, with the model and policy controller reinitialized in every run. The sampling and online update procedures in reinforcement learning are stochastic, so minor variations in metrics and average embedding dimensions are expected random fluctuations, rather than transcription errors in the manuscript.

For example, the two sets of results on MovieLens-20M: 0.4539 / 1.2477 and 0.4540 / 1.2486 come from two separate streaming runs. The accuracy gap is only 0.0001 and the CE loss gap is only 0.0009. For Amazon Beauty across three independent runs:

• User average embedding ratio: 0.0409, 0.0365, 0.0352, with a maximum difference of 0.0057

• Item average embedding ratio: 0.1168, 0.1219, 0.1273, with a maximum difference of 0.0105

These discrepancies stem from random variance across independent training trajectories and do not represent numerical inconsistencies within a single model instance.

We will add a unified note in the paper to clarify that different tables correspond to separate experiments.

Q2.2 Training Batch Size & Learning Rate: Appendix B.1 reports Batch size = 500, LR = 1e-4, whereas Appendix C.1 reports Batch size = 1,024, LR = 1e-3, despite claiming ‘identical hardware and optimization settings’.

Thank you for pointing out this ambiguous statement in the appendix. The actual configuration is as follows: Amazon Beauty and LastFM-1K use a batch size of 1024; the remaining three datasets use a batch size of 500. The phrase “identical hardware and optimization settings” in our manuscript means that within the same dataset, TAMRec and all baselines adopt exactly the same configurations for fair comparison. It does not require a unified batch size or learning rate across all five datasets. This is a writing error in Appendix B.1. We will correct the parameters in Appendix B.1 and C.1, and explicitly clarify in the revised paper that hyperparameters are aligned among methods within one dataset, while separate tuned hyperparameters can be used across different datasets.

Q3.1 Figure 3 vs. Section 4.6 & Appendix F: On Figure 3 itself, the green value printed directly above TAMRec at 6–20 (Long-tail) is 0.3888 (just above SCALL’s 0.3823). However, both Section 4.6 and Appendix F report TAMRec’s accuracy on 6–20 interactions as 0.3988 (+0.0100 higher than the plotted value).

Thank you for your careful check. The correct value for the long-tail segment with 6–20 interactions in Figure 3 is 0.3888. The value 0.3988 in Section 4.6 and Appendix F is a transcription typo. We will update both the main text and appendix consistently to 0.3888. This correction does not alter our core conclusion: TAMRec (0.3888) remains slightly better than SCALL (0.3823).

Q3.2 Figure 2(a) vs. Eq. (3) & Section 4.4: Eq. (3) defines Δτ=exp(−Δτ_raw/τ_day)+ε, so Δτ≈1 means frequently accessed and Δτ≈0 means rarely accessed. Yet the x-axis printed on Figure 2(a) is labeled Time Interval (0=Frequently Accessed, 1=Rarely Accessed)—the exact opposite of Eq. (3).

Thank you for pointing out this directional inconsistency. According to our formula Δτ=exp⁡(−τ/τbase)\Delta\tau = \exp(-\tau/\tau_{\text{base}}): Δτ→1\Delta\tau \to 1 corresponds to recent/frequent access, while Δτ→0\Delta\tau \to 0 means long-term inactivity. The labeling on the x-axis of Figure 2(a) was indeed reversed. We will rename the x-axis to time decay feature, with annotations: (0=)(0=) long-term inactive, (1=)(1=) recent access. The curve peaks around 0.6 with lower values at both ends; this curve shape is consistent with our qualitative description that “capacity is lower at both ends”. Only the x-axis labels need correction, and the experimental curve itself remains unchanged. This similar issue was also raised by Reviewer 1.

Q3.3 Figure 2(b) vs. Theorem 6: Figure 2(b) shows that at log₁₀(Frequency)=4.0, the average allocated dimension reaches 128.0 (100% of d_max), contradicting Theorem 6’s claim that dimensions saturate strictly below d_max.

Thank you for identifying this inconsistency between our theory and figure. Figure 2(b) plots the conditional average allocated dimension per group, and the mean reaches dmax=128d_{\text{max}}=128 for high-frequency groups. The original claim in Theorem 6 that the saturation value is strictly smaller than dmaxd_{\text{max}} is indeed too strong and conflicts with experimental observations. We revise Theorem 6 by removing the statement “strictly smaller than dmaxd_{\text{max}}” and only retain the capacity upper bound derived from the discrete, monotone, bounded action space. After revision, the fixed point is allowed to equal dmaxd_{\text{max}}, consistent with the observations in Figure 2(b).

Q4.1 "Trailing-Zero Clusters & Constant-Grid Numbers in Table 2 and Table 3: Despite averaging over 2,572,765 test interactions on LastFM-1K, the baseline rows in Table 2 contain 17 four-decimal numbers ending in 0. In Table 3, SCALL (c=32) User/Item ratios across several datasets also contain many values ending in 0, with four user ratios literally {0.2420, 0.2430, 0.2440, 0.2460}.*

Thank you for carefully examining the numerical distribution. Table 2 and Table 3 uniformly report values to four decimal places. Trailing zeros and clustered values within a narrow range are merely visual artifacts caused by display precision and do not indicate problematic data. For SCALL ((c=32)(c=32), maximum dimension 128128), its total capacity budget is fixed, so the normalized dimension ratio is naturally constrained around 0.25. The observed set 0.2420,0.2430,0.2440,0.2460{0.2420, 0.2430, 0.2440, 0.2460} arises from the intrinsic mechanism of this baseline, rather than artificial grid-based construction. In the revised manuscript, we will add descriptions of the metric calculation pipeline and precision convention to clearly distinguish numerical patterns induced by algorithm constraints from manual post-processing.

Q4.2 Exact Scalar-Multiple Row in Table 14 (LastFM-1K Ablation): Dividing the four metrics of w/o Time-Aware (0.1569, 0.0913, 0.2174, 0.1066) by TAMRec (0.1892, 0.1101, 0.2621, 0.1285) gives 0.8293, 0.8292, 0.8295, and 0.8296. All four metrics were scaled by almost the same factor 0.8294±0.0002.

Thank you for your careful numerical verification. The results for w/o Time-Aware in Table 4 are obtained from a complete and independent ablation experiment. The ratios of the four metrics are highly close, which indicates that after removing the time-aware module, HR and NDCG exhibit nearly consistent relative degradation under these two K values. Mathematically, there is no constraint requiring these relative ratios to be distinct from each other. This observation is an empirical result under this ablation setting, and the close ratios alone cannot be taken as evidence that the metrics were artificially scaled by a fixed factor. We will add a note in the revised manuscript: this ablated component induces approximately proportional performance degradation across different ranking metrics.

Q4.3 Identical Rows and Constant Shifts in Table 10 and Table 12 (Amazon Beauty Sensitivity): In Table 12, the entire 5-column row for β=0.005 is 100% identical to β=0.5. At β=1.0, HR@10 and HR@20 drop by the exact same −0.0011, while both NDCG@10 and NDCG@20 drop by the exact same −0.0007. Similarly, in Table 10, NDCG@10 and NDCG@20 drop by the exact same amount.

Thank you for carefully checking our sensitivity tables. All groups with different β\beta and KK in Table 10 and Table 12 are from independent full runs. The duplicate value comes from a table typo: the entry labeled β=0.005\beta=0.005 was originally the result for β=0\beta=0. This explains why its numbers match those of β=0.5\beta=0.5. On the Amazon Beauty dataset, the regularizer exerts negligible influence on the allocation strategy for β∈[0,0.5]\beta \in [0,0.5], forming a flat parameter plateau with steady metrics. The consistent drop in HR@10/HR@20 and NDCG@10/NDCG@20 at β=1.0\beta=1.0 reflects the model’s empirical behavior under this strong regularization. We will correct β=0.005\beta=0.005 to β=0\beta=0 and proofread all table transcriptions throughout the manuscript.

Q4.4 "Identical User and Item Ratios in Table 3 and Table 13: Table 13 reports User Ratio / Item Ratio = 0.2936 / 0.2936 for MovieLens-20M w/o Group Prior despite 138,493 users vs. 27,278 items. In Table 3, SCALL (c=32) also reports identical user ratios across different datasets, and identical user/item ratios on Netflix."

Thank you for pointing out this numerical observation. Given different total entity counts, there is no requirement that the average dimension ratios for users and items must differ. The average ratio is the mean of allocated dimensions over all entities, an independent statistic; the user-side and item-side means can coincide. For MovieLens-20M w/o Group Prior: after removing the group prior, the allocation strategies for users and items converge to similar average ratios, which round to 0.2936. For SCALL ((c=32)(c=32)): its fixed total capacity budget constrains normalized dimension ratios within a narrow range, so similar or identical ratios across datasets naturally occur. This empirical outcome arises from algorithmic constraints plus statistical averaging and contains no mathematical contradiction. We will add explanations for the allocation constraints of these two baselines in the revised manuscript and clarify that average ratios are not one-to-one tied to total entity counts.

Q5.1 Appendix C.3 vs. Table 9: Line 1133 claims ‘RS-side policy lookup and embedding transfer: +1.45 ms’, whereas Table 9 lists Policy Fwd (RS)=1.73 ms and Emb Transfer (RS)=1.54 ms (1.73+1.54=3.27 ms, not 1.45 ms).

Thank you for identifying the ambiguous timing label between the appendix and Table 9. The textual description “RS-side policy lookup and embedding transfer: 1.45 ms” was incorrect. This 1.45 ms value was not the direct sum of Policy Fwd (RS) + Emb Transfer (RS) in Table 9. It originally represented the net incremental overhead after subtracting the controller update cost, but its label was misleading and conflicted with the component-wise breakdown. In the revision, we strictly separate the two timing definitions:

• Table 9 reports standalone measured latency for each module (Policy Fwd, Emb Transfer).

• Appendix C.3 will no longer label this net residual as lookup+transfer. If we cannot reliably reproduce this residual value with repeated timing runs, we will remove the 1.45 ms entry entirely and drop this ambiguous decomposition.

Q5.2 Appendix C.2 vs. Table 8: TAMRec has T_train=0.41 min and T_test=16.19 min, which sum to 16.60 min (not T_total=16.61 min), and the percentage difference is (16.61−16.85)/16.85=−1.42% (not −1.47% as claimed).

Thank you for catching this arithmetic error in Appendix C.2. Given Ttrain=0.41T_\text{train}=0.41 min and Ttest=16.19T_\text{test}=16.19 min, their correct sum is (16.60)(16.60) min, while the manuscript incorrectly wrote (16.61)(16.61) min. This mistake propagated to the derived percentage difference. We will correct the total runtime, recalculate the percentage gap, and update all dependent values in Appendix C.2 accordingly.

Q5.3 Appendix D.2 vs. Tables 10–11: The claimed largest gaps do not match the displayed values; for example, the gap for K, λ, and β is miscomputed from the corresponding HR@10 entries.

Thank you for identifying the inconsistency in Appendix D.2. The reported maximum performance gaps for K,λ,βK, \lambda, \beta contain transcription and calculation errors and do not match the raw HR@10 values in Tables 10–11. We will recalculate all maximum gaps strictly from the measured table values and revise the descriptions in the appendix. We will also add a note that these three sensitivity experiments were run independently. The table results themselves remain unchanged; only the summary gaps in the text will be corrected.

Reviewer_7ZnBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TAMRec, employing a time-aware policy that decouples interaction frequency from capacity demand, and adopts a monotonic Maintain/Enlarge action space to eliminate shrink-induced forgetting. While monotonic expansion risks uncontrolled parameter growth, an adaptive reward mechanism encourages on-demand allocation: enlargement is favored only when it yields prediction performance above a moving baseline. A group-prior fallback further stabilizes learning for cold-start entities with sparse feedback.

Critical strengths
  1. The paper addresses an interesting and challenging problem in streaming recommendation.
  2. Experiments on five benchmark datasets demonstrate that TAMRec delivers competitive or better recommendation performance with compact embeddings and less forgetting in streaming settings.
Critical weaknesses
  1. The policy gradient update in the framework figure uses r_t directly, whereas Equation (8) uses r_t-b(s_t). Could the authors clarify which formulation is actually used for policy optimization and ensure consistency between the figure and the equations?
  2. Equation (7) uses immediate rewards in the policy gradient, while the objective is defined as maximizing cumulative rewards. Could the authors clarify why immediate rewards are used instead of cumulative returns and how this formulation accounts for the long-term effects of dimension adjustment?
  3. The reward formulation in Equations (6)–(8) is confusing. Equation (6) defines the reward as r(s_t,a_t)=\bar{L}{base,t}-L_t, while Line 203 defines the adaptive baseline as b(s_t)=\bar{L}{base,t}. Substituting these definitions into Equation (8) gives r(s_t,a_t)-b(s_t)=-L_t, meaning that the adaptive baseline cancels out completely. Could the authors clarify the rationale behind using the same baseline in both the reward function and the policy gradient estimator?
  4. Although the embedding dimensions are capped at 128, could the authors clarify whether the policy learns to stop expanding based on diminishing performance gains, rather than simply reaching the predefined maximum dimension?
Questions for authors
  1. The policy gradient update in the framework figure uses r_t directly, whereas Equation (8) uses r_t-b(s_t). Could the authors clarify which formulation is actually used for policy optimization and ensure consistency between the figure and the equations?
  2. Equation (7) uses immediate rewards in the policy gradient, while the objective is defined as maximizing cumulative rewards. Could the authors clarify why immediate rewards are used instead of cumulative returns and how this formulation accounts for the long-term effects of dimension adjustment?
  3. The reward formulation in Equations (6)–(8) is confusing. Equation (6) defines the reward as r(s_t,a_t)=\bar{L}{base,t}-L_t, while Line 203 defines the adaptive baseline as b(s_t)=\bar{L}{base,t}. Substituting these definitions into Equation (8) gives r(s_t,a_t)-b(s_t)=-L_t, meaning that the adaptive baseline cancels out completely. Could the authors clarify the rationale behind using the same baseline in both the reward function and the policy gradient estimator?
  4. Although the embedding dimensions are capped at 128, could the authors clarify whether the policy learns to stop expanding based on diminishing performance gains, rather than simply reaching the predefined maximum dimension?
1 comment

Q1:

Thank you for identifying the inconsistency between the figure and the formula. Our implementation aligns with the framework diagram and directly uses the immediate reward rtr_t: the policy network outputs action logits, which go through softmax for sampling Maintain / Enlarge. The baseline loss BtB_t adopts time-decayed historical loss for data-abundant entities, and the average loss within the same frequency group for cold-start entities. The reward is defined as rt=clip(Bt−Lt,−Rmax,Rmax)r_t = \text{clip}(B_t - L_t, -R_{\text{max}}, R_{\text{max}}). The optimization objective of the controller is the cross-entropy multiplied by the reward: Lpolicy=−∑trtlog⁡πθ(at∣st)\mathcal{L}{\text{policy}} = -\sum_t r_t \log \pi\theta(a_t|s_t). No extra term b(st)b(s_t) is subtracted in our implementation. The framework diagram is correct; the term rt−b(st)r_t - b(s_t) in Equation (8) is a typo. We will correct Equation (8), add the description of reward clipping, and unify the framework diagram, main-text formulas and pseudocode.

Q2:

In streaming environments, long-term returns tend to conflate distribution shift and interference from model updates. For this reason, we adopt immediate rewards and optimize the single-step average reward Jstep(θ)=E[rt]J_{\text{step}}(\theta) = \mathbb{E}[r_t], without multi-step credit assignment. Note that the dimension state persists into subsequent steps. The original manuscript incorrectly described this objective as the standard cumulative return. We will revise the objective description and remove the corresponding claim of long-term optimality. The algorithm and experimental results remain unchanged.

Q3:

Thank you for the careful algebraic verification. The derivation provided by the reviewer holds for the formulas in the original manuscript, yet this cancellation artifact only arises from the erroneous notation in our draft and does not reflect the logic of our actual implementation. In our code, we perform only one comparison: we first compute the reference loss BtB_t, then construct the clipped reward rtr_t using Bt−LtB_t - L_t. We directly weight the negative log-probability of the action with rtr_t, without an extra subtraction of b(st)b(s_t).

The root cause lies in confusing notation in the original manuscript: the same BtB_t was used both as the reference for reward construction and separately defined as the baseline b(st)b(s_t) in the policy gradient, which leads to complete algebraic cancellation. In the revised version, we will consistently refer to BtB_t as the reference loss. We will no longer set b(st)=Btb(s_t)=B_t, and we avoid additional centering outside rtr_t. We also remove the theoretical claim that “the auxiliary baseline ensures unbiasedness and reduces gradient variance”. The only claim supported by our implementation is that the time-decayed historical loss and group-wise prior provide a stable loss reference for sparse feedback.

Q4:

Yes. Our method contains two layers: soft stopping and hard upper bound.

Before reaching the maximum dimension dmaxd_{\text{max}}, both Maintain and Enlarge actions are sampled by the policy network. If the loss decreases relative to the reference loss after choosing Enlarge, the reward rtr_t becomes positive, and the policy will increase the sampling probability of this action. Conversely, a negative reward suppresses the probability of Enlarge. Therefore, when the gains from expansion diminish and no positive reward is persistently obtained, the policy naturally tends to select Maintain, achieving soft stopping.

When the dimension hits dmaxd_{\text{max}}, action masking in our code forces the action to Maintain, forming a hard boundary to ensure the dimension never exceeds 128.

Note that dmaxd_{\text{max}} serves only as an upper capacity bound. The single-step reward mechanism can encourage the policy to stop expansion before this upper limit, yet it cannot guarantee that every entity finds the optimal stopping point prior to dmaxd_{\text{max}}. In our experiments, the average dimension of most entities is below 128, while some high-frequency entities eventually hit this upper bound, which aligns with the behavior of this mechanism.

Reviewer_9ptYOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TAMRec, a recommendation framework, which adjusts embedding dimensions to reduce memory usage. The framework only maintains or enlarges each embedding dimension and never shrinks it, since shrinking could cause the loss of learned representations. A policy network decides whether to maintain or enlarge a dimension based on three input features: interaction frequency, the current embedding dimension and a time-decayed interval feature. The policy is trained by reinforcement learning and receives a reward when it reduces the loss.

Critical strengths
  • This paper addresses an important problem, which is the resource bottleneck of embedding tables. Fixed dimensions can allocate excessive capacity to sparse entities and insufficient capacity to dense entities, and the proposed method addresses this by enabling adaptive dimension allocation.

  • The method achieves comparable or better recommendation performance while using memory efficiently. For instance, it shows comparable performance compared to baselines on Amazon Beauty even with about 4-12% size compared to fixed embeddings.

Critical weaknesses
  • I have several concerns about the experiments. First, the method is evaluated on only two backbones, both of which are outdated(2017). The authors should include experiments on more recent backbones to demonstrate the method’s practicality regarding that there was a substantial progress in recommendation models since then. Second, the computational efficiency results are reported only on the Amazon Beauty dataset. According to Table 3, this is the dataset with the smallest overall embedding size, which is about 4–12% of the fixed-dimension size. The authors state that “The policy adds computation, but compact embeddings reduce the recommender’s cost; the total measured runtime remains comparable.” However, the embedding dimensions are larger on the other datasets, so the computational overhead may be much heavier in those cases. This is a significant concern since recommendation is a latency-sensitive task.

  • The presentation could be improved. For example, the x-axis of Figure 2 does not match its description in Section 4.4.

Questions for authors
  • Please refer to the weaknesses section.
1 comment

Q1: First, the method is evaluated on only two backbones, both of which are outdated (2017). The authors should include experiments on more recent backbones to demonstrate the method’s practicality regarding that there was a substantial progress in recommendation models since then.

Thank you for your comment. Our core goal is not to benchmark backbone predictive performance, but to validate whether dynamic dimension allocation improves the accuracy–storage trade-off for streaming recommendation. We select MLP and NeuMF as representative interaction paradigms: pure nonlinear interaction and MF-MLP fusion, respectively. TAMRec yields consistent trends on both, showing our mechanism’s gains are not tied to one specific interaction function.

TAMRec’s controller uses only interaction frequency, current embedding dimension and time interval. All dynamically sized embeddings go through a unified hierarchical projection before entering the downstream backbone, and the controller is agnostic to backbone internals. The framework is thus pluggable to other recommendation models. We acknowledge that testing on newer backbones would expand coverage, yet the two selected backbones adequately support our main conclusion: adaptive dimension allocation improves the storage–accuracy trade-off under our streaming protocol. We will note this limitation and future research direction in the revised discussion.

Q2: Second, the computational efficiency results are reported only on the Amazon Beauty dataset. According to Table 3, this is the dataset with the smallest overall embedding size, which is about 4–12% of the fixed-dimension size. The authors state that ‘The policy adds computation, but compact embeddings reduce the recommender’s cost; the total measured runtime remains comparable.’ However, the embedding dimensions are larger on the other datasets, so the computational overhead may be much heavier in those cases. This is a significant concern since recommendation is a latency-sensitive task.

Thank you for pointing out this critical issue. We first distinguish between embedding storage overhead and end-to-end latency overhead. The average raw embedding dimension of TAMRec on Amazon Beauty is indeed relatively low: under the NeuMF setup, the user and item embedding sizes are only 3.58% and 7.65% of Fixed-128, respectively. This result supports our conclusion that the embedding table storage is substantially reduced.

However, after hierarchical projection, the dynamic embeddings are projected to a unified 128-dimensional input for MLP / NeuMF. The main computation of the downstream backbone does not shrink alongside the raw embedding dimension. Meanwhile, the policy network and projection module introduce extra overhead. Therefore, the original statement “compact embeddings reduce the recommender’s computational cost” is imprecise.

Amazon Beauty is indeed the dataset with the smallest embedding size among all our datasets. The observation that TAMRec achieves comparable end-to-end runtime to Fixed-128 on this dataset is only valid for this dataset under the current hardware and implementation setup. As the reviewer noted, on other datasets with larger embedding tables, the extra overhead from the controller and projection modules may substantially increase end-to-end latency. We have not yet conducted timed experiments with a unified measurement protocol on those datasets, so this risk cannot be ruled out.

Q3: The presentation could be improved. For example, the x-axis of Figure 2 does not match its description in Section 4.4.

Thank you for pointing out this presentation issue. The time-decay feature adopted in this paper follows the rule that it takes larger values for dense recent interactions and smaller values when there has been no interaction for a long period. The horizontal axis of the original Figure 2 was labeled as the normalized raw time interval, defined such that 0 corresponds to frequent visits and 1 corresponds to rare visits. This direction is opposite to the description of the time-decay feature in Section 4.4, leading to inconsistency.

We will adjust the ordering of the horizontal axis in the revised manuscript and explicitly label it as the time-decay feature: 0 denotes long-term inactivity, and 1 denotes a recent interaction. The caption will also be aligned with the textual description in Section 4.4. This revision only modifies the axis presentation order and label explanations; the experimental statistics for each group remain unchanged.

This concern was also raised by Reviewer 3.

We have also addressed other issues, including formatting and presentation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.