acceptodds
Under review as a conference paper at ICLR 2027

IDShare: Learned Sharing as Regularization for Long-Tailed ID Embeddings

Abstract

Recommendation models often use ID features with many distinct values. Features such as user IDs and ad IDs can have long-tailed frequency distributions. Long-tailed IDs have few training examples, making their embeddings difficult to learn reliably. We introduce IDShare, a simple end-to-end method that learns which IDs share an embedding. Each shared embedding is updated using training examples from all IDs assigned to it. We find that this learned sharing provides effective regularization for long-tailed IDs. On two recommendation benchmarks, IDShareachieves higher mean AUC than independent embeddings with tuned global L2, without using that penalty. Further analysis shows that IDShare improves predictions for long-tailed IDs. Its performance remains stable across different numbers of shared embeddings. It also retains its advantage in deeper models. These findings support learned sharing as a simple and stable default for training long-tailed ID embeddings.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Representation Learning, Recommender Systems, Long-Tailed ID Embeddings

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_kYEVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes IDShare, which learns to assign multiple IDs to shared embeddings to alleviate overfitting in long-tailed recommendation. The assignments and embeddings are jointly optimized using the recommendation objective. Experiments on TAAC and KuaiRand demonstrate improvements over independent embeddings and several regularization baselines.

Critical strengths
  1. The paper addresses a practical problem in recommendation systems, as long-tailed IDs often have too few interactions to learn reliable embeddings.

  2. The proposed method is simple and intuitive. Sharing embeddings seems like a reasonable way to reduce overfitting without relying on L2 regularization.

  3. The experiments cover two large-scale datasets and several recommendation backbones, making the results more convincing.

  4. I appreciate the additional analyses on codebook size, model depth, and ID frequency, which help better understand the method's behavior.

Critical weaknesses
  1. The novelty is somewhat limited. Similar ideas have been explored in previous work, such as Clustered Embedding Learning, and the differences are not clearly explained.

  2. The paper would benefit from comparisons with existing learned embedding-sharing methods, rather than mainly focusing on standard regularization and fixed-sharing baselines.

  3. Continuous and IDShare use different OOV representations, which may affect the fairness of the unseen-ID comparison. It is unclear how much of the reported improvement comes from this difference.

  4. The improvements over tuned L2 baselines are relatively small. The high variance across runs on TAAC also raises concerns about the consistency of these gains.

  5. The paper provides little analysis of computational costs. It would be useful to report training time, memory usage, and inference latency.

Questions for authors

See weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer kYEV

Thank you for your review. Below we address Weaknesses 1–5 with new experiments and measurements (highlighted in blue in the revised manuscript).

W1 & W2. Differences and comparisons with CEL and learned sharing. Unlike CEL's alternating cluster-reassignment and splitting stages or CCE's sketched reclustering, IDShare updates fixed-KK single-code routes and shared vectors jointly on every step via hard nearest-code selection and soft straight-through gradients, with no private residual (Table 1 in §2.5). Under matched inputs, RankMixer backbones, d=16d=16, seeds, and λ=0\lambda=0 (§4.6, Table 6; Appendix I.1, Table 18), IDShare outperforms adapted CCE (matched KdKd shared floats) and CEL-Lite (pure shared clusters under our Adam setup) on all three seeds on both datasets: +0.492 ± 0.217 pp over CCE and +2.850 ± 0.150 pp over CEL-Lite on TAAC (77.526% vs. 77.035% and 74.676%), and +0.107 ± 0.058 pp over CCE and +4.304 ± 0.498 pp over CEL-Lite on KuaiRand (75.445% vs. 75.337% and 71.141%).

W3. Fairness of OOV representations. We trained Tuned Continuous (λ=3×10−9\lambda=3\times 10^{-9}) and IDShare (λ=0\lambda=0) on TAAC with identical 1% random OOV masks on real target and history lookups, giving Continuous's OOV row and IDShare's b0b_0 matched fallback training exposure (Appendix H, Table 17). Under symmetric OOV exposure, the unseen-target AUC gap closes (-0.0257 ± 0.0738 pp, 1/3 seeds; unseen logloss still favors IDShare on 3/3 seeds), confirming your point on unseen ranking. Meanwhile, IDShare's gain on seen 1–10 lookup IDs increases to +0.3517 ± 0.0863 pp (3/3 seeds) and overall AUC stays higher by +0.0405 ± 0.0130 pp (3/3 seeds) (zeroing only b0b_0 at evaluation in Table 16 also leaves +0.4334 pp overall over Continuous(00)). Thus, our seen long-tail improvements persist under matched fallback exposure rather than relying on OOV asymmetry.

W4. Gain over tuned L2 and cross-seed variance on TAAC. Appendix I.3, Table 21 reports full paired statistics: while the single 3-seed TAAC/RankMixer overall contrast vs. Tuned Continuous (+0.0738 ± 0.1959 pp) has a wide interval, IDShare(00) exceeds grid-tuned Continuous in all four dataset/backbone settings without a 7-point L2 search (including +0.4411 ± 0.0964 pp, p=0.0156p=0.0156 on TAAC/DIN, and +0.1518 pp, p=0.0102p=0.0102 across the four settings under independent-stratum assumptions), and leads on all three paired seeds at every tested K∈[10K,80K]K \in [10\text{K}, 80\text{K}] and depth 2,4,82, 4, 8 (Appendix D). Looking within frequency groups, the gain is much clearer on sparse items (with a -0.0118 pp trade-off on 501+ head items): on 1–10 lookup target IDs, IDShare beats Tuned-L2 Continuous on all three seeds by +0.2164 ± 0.1429 pp (Table 19, overall +0.1554 ± 0.0507 pp) and Matched-OOV Tuned-L2 Continuous on all three seeds by +0.3517 ± 0.0863 pp (Table 17).

W5. Training time, memory usage, and inference latency. In isolated single-GPU benchmarks on TAAC (§5; Appendix I.3, Table 22), native training at batch 2,048 takes 66.86 ms per step (30.6K ex/s, 7.65 GiB peak allocated) vs. 28.94 ms (70.4K ex/s, 6.85 GiB) for Tuned Continuous (2.31×2.31\times per-step cost, while avoiding a 7-run L2 sweep). Exporting shrinks the saved full-model state_dict by 72.3% (1,476.4 → 409.3 MB; 93.6% on the identity payload) and cuts inference peak allocated GPU memory by 61.5%–70.2% vs. Continuous (1.39 to 0.41 GiB at batch 1; 1.40 to 0.43 GiB at batch 128; 1.59 to 0.61 GiB at batch 2,048), with exported p50 latency within 4.6%–5.4% of Continuous (2.18 / 2.50 / 3.35 ms vs. 2.07 / 2.38 / 3.20 ms).

AuthorsOpenReview ↗

Brief follow-up on our new experiments

As the discussion period nears its close, we wanted to check whether our new experiments and revised manuscript address Weaknesses 1–5, or if anything remains unclear.

One brief observation from Appendix Table 18 further clarifies W1–W2: on both benchmarks, adapted CCE and CEL-Lite actually finish below even unregularized Continuous(0) in mean AUC (77.035% and 74.676% vs. 77.070% on TAAC; 75.337% and 71.141% vs. 75.370% on KuaiRand), whereas IDShare(0) outperforms L2-tuned Continuous without an L2 sweep, retains +0.352 pp on seen 1–10 lookup IDs under matched 1% OOV masking (Table 17), and cuts inference GPU memory by 61.5%–70.2% (Table 22). Thus, whether embedding sharing helps or hurts generalization depends directly on joint, residual-free single-code training.

Please let us know if any further details would be helpful in the remaining discussion window.

Area_Chair_SgakOpenReview ↗

Gentle Reminder on Rebuttal Discussion

Dear Reviewer,

As the discussion deadline is approaching, I would kindly encourage you to review the authors' rebuttal and share your follow-up comments, particularly on whether your concerns have been adequately addressed.

Your feedback would be greatly appreciated and helpful for the final assessment. Thank you for your time and contribution!

Reviewer_UZ69OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper considers the problem of learning embeddings for IDs with fewer examples which are usually difficult to learn independently. The authors introduce a method called IDShare that learns embeddings in a shared manner where some of the IDs share a single embedding. A routing table, which maps the IDs to embeddings, and an embedding codebook are learned jointly. Learning the embeddings in a shared manner provides regularization for the long tailed IDs. The performance of the proposed method is evaluated on two recommendation datasets, where this method outperforms baselines comprising of independent embeddings, L2 and adaptive regularization, and fixed sharing methods. The experiments also find a stable performance advantage across backbones, different codebook sizes and deeper models. They also find that the predictions are improved for unseen IDs and long tailed IDs, and also study the frequency structure of the learned sharing.

Critical strengths
  1. The problem considered is of practical significance and the paper discuss detailed experiments, including prediction performance and provides insights about how the proposed method benefits long tailed and unseen IDs, and that the long tailed IDs are tied mainly to other lower frequency IDs.

  2. The superior performance of the proposed method over L2 tuned continuous is impressive, without needing any additional regularization term.

Critical weaknesses
  1. The experiments don’t involve any comparison with other learned sharing methods. The only baselines compared are the independent embeddings with and without regularization, and fixed sharing methods.
Questions for authors
  1. How does the performance of the proposed method compare against other learned sharing methods such as Clustered Embedding Learning, Clustering the Sketch, AutoAssign+?

  2. Why isn’t the performance on a separate held out test set considered? In some of the experiment settings, the validation set is used to select the best performing model and thus, evaluating generalization using a separate test set would be a stronger result.

1 comment
AuthorsOpenReview ↗

Response to Reviewer UZ69

Thank you for your positive review (Weak Acceptance). Below we answer both of your questions directly.

Q1. Comparison with learned sharing methods (W1, Q1). Keeping inputs, RankMixer backbones, d=16d=16, seeds, and λ=0\lambda=0 fixed, we ran three-seed comparisons on both benchmarks against adapted Clustering the Sketch (CCE, matching KdKd shared floats) and CEL-Lite (growing up to KK shared clusters without private residuals; §4.6, Table 6; Appendix I.1, Table 18, with AutoAssign+ also contrasted in Table 1). IDShare wins every seed in all four comparisons:

BenchmarkBaselineBaseline AUC (%)IDShare AUC (%)Δ\Delta AUC (pp)Seeds
TAAC (K=20KK=20\text{K})CCE (adapted)77.035 ± 0.04077.526 ± 0.179+0.492 ± 0.2173/3
TAAC (K=20KK=20\text{K})CEL-Lite (adapted)74.676 ± 0.18077.526 ± 0.179+2.850 ± 0.1503/3
KuaiRand (K=40KK=40\text{K})CCE (adapted)75.337 ± 0.05575.445 ± 0.014+0.107 ± 0.0583/3
KuaiRand (K=40KK=40\text{K})CEL-Lite (adapted)71.141 ± 0.48775.445 ± 0.014+4.304 ± 0.4983/3

Q2. Why a separate held-out test set was not used (Q2). Our experiments used fixed chronological two-way splits (May 23–28 train / May 29 validation on TAAC; chronological split on KuaiRand, Appendix A) without a third temporal holdout. Validation selection favors the baseline rather than IDShare: in revised Table 4 / Figure 2, Continuous uses the validation set to pick its best λ∗\lambda^* across 7 grid points, whereas IDShare uses fixed λ=0\lambda=0 with no L2 search (TAAC also evaluates the fixed one-epoch final checkpoint without early stopping, and KuaiRand uses the same epoch-end checkpoint rule for all methods). Because splitting the already-evaluated validation set post-hoc would not produce an untouched test set, we keep the split fixed and clarify this scope in the revision; across all newly added CCE, CEL-Lite, Tuned-L2, and Matched-OOV runs under this fixed protocol, IDShare's gains remain consistent.

Reviewer_jWYpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper proposes IDShare, a learnable embedding-sharing method designed to mitigate overfitting in high-cardinality, long-tailed ID features in recommendation and ranking systems. IDShare assigns each ID a learnable routing state and maps IDs to a shared codebook using nearest-neighbor assignment and a straight-through estimator. The method is evaluated on TAAC and KuaiRand, primarily with RankMixer, with additional experiments on DIN and TokenMixer-Large.The problem is practically important, and the proposed approach is simple and potentially compatible with existing recommendation architectures. However, the paper does not sufficiently establish its novelty over prior embedding clustering and sharing methods. In addition, the current experiments do not fully isolate the contribution of learnable routing, and some reported gains are relatively small.

Critical strengths
  1. Long-tailed ID embedding overfitting is an important challenge in large-scale RSs.

  2. The codebook-based sharing mechanism is modular and can potentially be integrated into different recommendation backbones.

  3. The method is evaluated on two datasets and multiple model architectures.

  4. Frequency-based evaluations, codebook-capacity experiments, and additional diagnostics provide useful insights into the method's behavior.

  5. Sharing embeddings may reduce embedding-table storage, although practical deployment benefits have not yet been measured.

Critical weaknesses
  1. The methodological differences from existing embedding clustering and sharing approaches, such as Clustered Embedding Learning, Clustering the Sketch, and AutoAssign+, are insufficiently established. Direct comparisons with the closest methods are needed.

  2. The experiments do not adequately separate the effects of the shared codebook, learnable routing, nearest-neighbor assignment, and the straight-through estimator. More carefully controlled experiments are needed to attribute the gains to the proposed components.

  3. For RankMixer without L2 regularization, IDShare improves AUC by approximately 0.4565 percentage points on TAAC but only 0.0743 percentage points on KuaiRand. Given the limited number of runs, confidence intervals or appropriate statistical tests would help establish the reliability of these improvements.

  4. Fine-grained frequency-based results are primarily reported for TAAC. Moreover, the OOV comparison uses different fallback behaviors, potentially confounding the interpretation of the results.

  5. The current diagnostics provide suggestive evidence but do not conclusively establish that the observed gains arise from the proposed sharing mechanism.

  6. The paper does not report actual serving memory, embedding storage, throughput, or latency. Therefore, the claimed practical efficiency benefits remain insufficiently validated.

Questions for authors
  1. What are the fundamental methodological differences between IDShare and the closest embedding clustering or sharing methods? Can the authors provide direct comparisons?

  2. Can controlled ablations isolate the effects of learnable routing from those of the shared codebook and other optimization components?

  3. Are the reported improvements statistically significant, particularly on KuaiRand? Can the authors provide confidence intervals or appropriate significance tests?

  4. Can the authors report frequency-stratified results on KuaiRand and use consistent OOV handling across baselines?

  5. What are the actual reductions in embedding storage and serving memory, and how does IDShare affect training and inference latency?

3 comments
AuthorsOpenReview ↗

Response to Reviewer jWYp

Thank you for your detailed review. To improve readability (Presentation: 1), we added Table 1 (§2.5) comparing assignment rules, per-ID outputs, auxiliary inputs, and L2 needs across method families, and clarified training routing (R,CR, C) vs. exported serving in §2.2, §5, and Appendix A.

Q1. Closest baselines and direct comparisons (W1, Q1). CEL alternates gradient updates with cluster splitting and reassignment; Clustering the Sketch (CCE) sums sketched tables with periodic reclustering; AutoAssign+ uses an RL controller to toggle unique vs. shared slots (Table 1). IDShare emits one shared vector per ID with no private residual, updating routes and codes jointly on every step without reclustering stages or frequency states. Under matched backbones, d=16d=16, seeds, and λ=0\lambda=0 (§4.6, Table 6; Appendix I.1, Table 18), IDShare wins all three seeds in every comparison: +0.492 ± 0.217 pp over CCE (77.526% vs. 77.035%, matched KdKd) and +2.850 ± 0.150 pp over CEL-Lite (74.676%) on TAAC (20K20\text{K}), and +0.107 ± 0.058 pp over CCE (75.445% vs. 75.337%) and +4.304 ± 0.498 pp over CEL-Lite (71.141%) on KuaiRand (40K40\text{K}) (validation picks CCE's epoch-1 checkpoint before reclustering; 1–10 seed directions vs. CCE are 2/1).

Q2. Ablations of codebook tying and routing (W2, W5, Q2). Three controls separate single-code tying from compositional capacity and fixed assignment. First, splitting d=16d=16 into M=2M=2 or M=4M=4 subspaces (20K20\text{K} codes/group on TAAC; Appendix G, Tables 14–15) cuts 1–10 lookup complete-tuple sharing from 100.0% to 6.0% and 0.0% and lowers overall mean AUC from 77.609% (M=1M=1) to 77.489% and 77.555% (M=4M=4 lowers 1–10 AUC by -0.113 pp; M=2M=2 is +0.015 pp higher). Second, post-training untying (Appendix F, Table 13) fixes the backbone and routes: untying raises training-probe AUC (85.10% vs. 81.98%) while lowering validation AUC by 0.134 pp (77.33% vs. 77.47%, 3/3 seeds; logloss favors untying). Third, FrozenHash (revised Table 3) pairs fixed hash assignments with an identity straight-through path instead of learned routing and soft STE (Appendix B), trailing IDShare by 0.586 pp on TAAC (p=0.0284p=0.0284) and 0.123 pp on KuaiRand. Nearest-neighbor selection, soft STE, temperature, radius floor, and loss weights were kept fixed across settings rather than individually ablated.

Q3. Statistical significance on KuaiRand (W3, Q3). Appendix I.3, Table 21 reports paired tt-tests and 95% CIs (df=2df=2) for all 12 contrasts in submitted Tables 2–3 (revised Tables 3–4). On KuaiRand/RankMixer, all three paired gains over Continuous (λ∗=0\lambda^*=0) are positive (+0.125, +0.081, +0.017 pp; mean +0.0743 ± 0.0539 pp, 95% CI [-0.0595, +0.2082] pp, p=0.1395p=0.1395). On KuaiRand, IDShare also beats AdamAR (+0.1355 pp, p=0.0015p=0.0015), TailShare (+0.1316 pp, p=0.0521p=0.0521), FrozenHash (+0.1233 pp, p=0.0604p=0.0604), CCE (+0.1072 pp, 3/3), and CEL-Lite (+4.3041 pp, p=0.0044p=0.0044). Across the four backbones (revised Table 4), TAAC/DIN reaches +0.4411 pp (p=0.0156p=0.0156), and an exploratory four-setting Welch–Satterthwaite mean gives +0.1518 pp (95% CI [+0.0563, +0.2473] pp, p=0.0102p=0.0102 under independent strata).

Q4. KuaiRand frequencies and OOV handling (W4, Q4). On KuaiRand (Table 20, λ∗=0\lambda^*=0), IDShare improves unseen-ID AUC on 4.04M examples by +0.2163 ± 0.0704 pp (3/3 seeds, p=0.0336p=0.0336), with smaller positive means on seen 1–5 (+0.0420 pp) and 1–10 (+0.0144 pp; mixed across counts) and -0.012 to -0.070 pp above 10 lookups. Under identical 1% target/history OOV masks in TAAC training (Table 17), the unseen AUC gap vs. Tuned Continuous (λ=3×10−9\lambda=3\times 10^{-9}) closes (-0.0257 pp; unseen logloss favors IDShare on 3/3 seeds), while seen 1–10 lookup gain rises to +0.3517 ± 0.0863 pp (3/3 seeds) and overall AUC remains +0.0405 ± 0.0130 pp (3/3).

Q5. Storage, GPU memory, throughput, and latency (W6, Q5). Dense identity storage drops 93.6% analytically on TAAC (1,119.2 → 71.2 MB) and KuaiRand (1,601.3 → 102.6 MB). On TAAC, saved whole-model state_dict drops 72.3% from 1,476.4 MB (Continuous: 1,457.3 MB) to 409.3 MB (exported, diff ≤8.94×10−8\le 8.94\times 10^{-8}). In single-GPU runs (§5; Table 22), batch-2,048 training takes 66.86 ms (30.6K ex/s, 7.65 GiB) vs. 28.94 ms (70.4K ex/s, 6.85 GiB) for Tuned Continuous, while inference compares as follows:

Inference (TAAC, 1× GPU)Batch 1 (p50 / Alloc.)Batch 128 (p50 / Alloc.)Batch 2,048 (p50 / Throughput / Alloc.)
Continuous2.07 ms / 1.39 GiB2.38 ms / 1.40 GiB3.20 ms / 627.9K ex/s / 1.59 GiB
Native IDShare4.35 ms / 1.43 GiB4.93 ms / 1.59 GiB11.65 ms / 176.0K ex/s / 1.71 GiB
Exported IDShare2.18 ms / 0.41 GiB (-70.2%)2.50 ms / 0.43 GiB (-69.7%)3.35 ms / 601.3K ex/s / 0.61 GiB (-61.5%)
AuthorsOpenReview ↗

Quick reading guide to the revised manuscript for Reviewer jWYp

Thank you again for your thorough review. Because your initial feedback noted that the submission was difficult to follow (Presentation: 1), we wanted to share a quick index to the blue-highlighted additions we made specifically to address your readability, mechanism, and systems questions, and ask if anything still needs clarification:

Topic from your reviewWhere it is clarifiedMain empirical result
Positioning vs. CEL, CCE & AutoAssign+New Table 1 (§2.5) & §4.6 (Table 6)IDShare wins 3/3 seeds vs. CCE (+0.492 / +0.107 pp) & CEL-Lite (+2.850 / +4.304 pp)
Isolating sharing vs. OOV & KuaiRand tail§4.7 & Appendices H, I.2 (Tables 17, 20)KuaiRand unseen +0.216 pp (p=0.034); TAAC seen 1–10 rises to +0.352 pp under 1% matched OOV
Training vs. exported serving path§2.2, §5 & Appendix I.3 (Table 22)Export shrinks model state by 72.3% and serving peak GPU memory by 61.5%–70.2% (+4.6%–5.4% p50 latency)

If any aspect of the presentation or experimental controls could still be clearer, please let us know while the discussion remains open.

Area_Chair_SgakOpenReview ↗

Gentle Reminder on Rebuttal Discussion

Dear Reviewer,

As the discussion deadline is approaching, I would kindly encourage you to review the authors' rebuttal and share your follow-up comments, particularly on whether your concerns have been adequately addressed.

Your feedback would be greatly appreciated and helpful for the final assessment. Thank you for your time and contribution!

Reviewer_amEtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the problem of overfitting in long tail ID embeddings in recommender systems. It proposes IDShare, a method based on learned embedding sharing. The method uses learnable routing representations to assign IDs to their nearest codewords in a shared codebook. IDs assigned to the same codeword share the same embedding. A straight through estimator is used to jointly optimize the routing representations and the codebook. This achieves regularization through parameter sharing.

Critical strengths
  1. The paper uses learned parameter sharing as a form of regularization. This provides a simple approach to reducing overfitting in long tail ID embeddings.
  2. Across four combinations of datasets and backbone models on TAAC and KuaiRand, IDShare achieves higher mean validation AUC than independent embeddings with tuned L2 regularization.
  3. The authors analyze the performance of the method and the role of the sharing mechanism. They examine codebook size, network depth, performance across different ID frequencies, and the removal of parameter sharing after training.
Critical weaknesses
  1. The incremental novelty of the method over existing research remains unclear. The paper does not sufficiently explain the substantive contributions of its parameterization, routing mechanism, and training procedure compared with existing learned embedding sharing methods.

  2. Direct comparisons with existing learned embedding sharing methods are missing. The experiments mainly compare IDShare with independent embeddings, regularization methods, and fixed sharing strategies. These comparisons are insufficient to establish its performance advantages over existing learned sharing methods.

  3. The experiments for unseen IDs use different representation learning strategies. The OOV representation in Continuous does not receive embedding table updates during training. In contrast, the zero codeword in IDShare can receive training signals from seen IDs. This difference may affect the fairness of the comparison.

  4. The frequency group experiments lack direct comparisons with a tuned baseline. The experiments mainly compare IDShare with Continuous without L2 regularization. It remains unclear whether IDShare still outperforms the tuned baseline for low frequency IDs.

Questions for authors
  1. What are the key differences between IDShare and existing learned embedding sharing methods in parameterization, routing mechanism, and training procedure? What substantive contributions do these differences provide?

  2. Could the authors provide direct comparisons with representative learned embedding sharing methods to verify the relative performance advantages of IDShare?

  3. How is the zero codeword used during training? Could the authors provide controlled experiments with more consistent representation learning strategies for unseen IDs? This would help clarify how much of the improvement comes from the learned sharing mechanism.

  4. Does IDShare still outperform Continuous across different ID frequency groups when Continuous uses tuned L2 regularization? Could the authors provide groupwise AUC and Logloss results to verify its actual benefits for long tail ID representation learning?

3 comments
AuthorsOpenReview ↗

Response to Reviewer amEt

Thank you for your clear review and experimental suggestions. Full tables are added in blue in the revised manuscript.

Q1. Differences from prior learned sharing and contribution (W1, Q1). Rather than proposing a new quantization operator, we show that residual-free single-code sharing regularizes sparse IDs without an additional global L2 search (Table 1, §2.5). Each ID emits one shared vector with no private residual (unlike multi-group PQ's KMK^M combinations or CEL's post-clustering personalization); nonzero codes use a radius floor (0.1+softplus⁡(ρk)0.1 + \operatorname{softplus}(\rho_k)) to stay separated from c0=0c_0=0, while b+b_+ shifts outputs outside ∥ri−ck∥22\|r_i - c_k\|_2^2 without distorting routing. Private states RR and fixed-KK codes CC update jointly on every step via hard nearest-code selection and annealed soft STE, avoiding the alternating cluster-splitting or reclustering stages in CEL and CCE while keeping quantizer hyperparameters fixed across all settings.

Q2. Direct comparisons with learned-sharing methods (W2, Q2). Keeping inputs, RankMixer backbones, d=16d=16, seeds, and λ=0\lambda=0 fixed, IDShare beats adapted Clustering the Sketch (CCE) and CEL-Lite on all three seeds on both datasets (§4.6, Table 6; Appendix I.1, Table 18):

BenchmarkBaselineBaseline AUC (%)IDShare AUC (%)Δ\Delta AUC (pp)Seeds
TAAC (20K20\text{K})CCE (matched KdKd)77.035 ± 0.04077.526 ± 0.179+0.492 ± 0.2173/3
TAAC (20K20\text{K})CEL-Lite (pure sharing)74.676 ± 0.18077.526 ± 0.179+2.850 ± 0.1503/3
KuaiRand (40K40\text{K})CCE (matched KdKd)75.337 ± 0.05575.445 ± 0.014+0.107 ± 0.0583/3
KuaiRand (40K40\text{K})CEL-Lite (pure sharing)71.141 ± 0.48775.445 ± 0.014+4.304 ± 0.4983/3

CCE matches IDShare's KdKd shared floats across two tables in four subspaces (reclustering in TAAC's epoch and before epochs 2–8 on KuaiRand, where validation picks epoch 1); CEL-Lite uses loss reassignment and gradient-PCA splitting under Adam without private residuals (Table 18).

Q3. Zero codeword and consistent unseen-ID control (W3, Q3). In training, c0=0c_0=0 is a fixed origin anchor (padding/missing use separate rows), while b0∈Rdb_0 \in \mathbb{R}^d updates when a lookup routes to q(i)=0q(i)=0. To match fallback exposure, we trained Tuned Continuous (λ=3×10−9\lambda=3\times 10^{-9}) and IDShare (00) on TAAC with identical 1% random OOV masks on real target/history lookups (Appendix H, Table 17):

TAAC (1% OOV Mask)Cont. AUC (%)IDShare AUC (%)Δ\Delta AUC (pp)Δ\Delta LoglossSeeds
Overall (21.95M)77.550 ± 0.06777.590 ± 0.059+0.0405 ± 0.0130-0.000399 ± 0.0000773/3
Seen 1–10 (1.01M)75.488 ± 0.10675.840 ± 0.021+0.3517 ± 0.0863-0.000020 ± 0.0005433/3
Unseen (0) (5.58M)76.245 ± 0.09876.220 ± 0.029-0.0257 ± 0.0738-0.001264 ± 0.0001911/3

Matching OOV exposure closes the unseen AUC gap (-0.0257 pp, while unseen logloss favors IDShare on 3/3 seeds), confirming your point on unseen ranking. On seen 1–10 lookup IDs, IDShare's gain rises to +0.3517 pp (3/3 seeds, winning buckets 1, 2–5, and 6–10), showing that seen long-tail gains persist under matched fallback exposure rather than relying on unequal OOV updates (zeroing only b0b_0 at evaluation in Table 16 also leaves +0.4334 pp overall and +0.1107 pp unseen over Continuous(00)).

Q4. Groupwise AUC and logloss vs. Tuned-L2 Continuous (W4, Q4). IDShare(00) retains higher AUC than Tuned Continuous (λ=3×10−9\lambda=3\times 10^{-9}) overall, on unseen IDs, and on pooled 1–10 lookup IDs on all three seeds (Appendix I.2, Table 19):

TAAC LookupsTuned Cont.IDShare (00)Δ\Delta AUC (pp)Δ\Delta Logloss
Overall (21.95M)77.4534%77.6088%+0.1554 ± 0.0507 (3/3)-0.000932 ± 0.000397
0 (Unseen) (5.58M)76.0360%76.2654%+0.2295 ± 0.0429 (3/3)-0.003215 ± 0.001314
1–10 (Pooled) (1.01M)75.5577%75.7741%+0.2164 ± 0.1429 (3/3)+0.000412 ± 0.000612
11–50 (1.07M)77.0644%77.1599%+0.0955 ± 0.0460 (3/3)-0.000166 ± 0.000550
51–500 (51–100 / 101–500)76.975 / 76.936%77.031 / 76.984%+0.056 / +0.048 (3/3)-0.000311 / -0.000150
501+ (Head) (11.63M)78.3711%78.3594%-0.0118 ± 0.0105 (0/3)-0.000195 ± 0.000056

Table 19 also lists buckets 1, 2–5 (one seed -0.0129 pp), and 6–10; 1–10 logloss favors Tuned Continuous on 2/3 seeds. On KuaiRand (Table 20, λ∗=0\lambda^*=0), IDShare gains +0.2163 pp on unseen IDs (3/3) and +0.0420 pp on 1–5 lookups.

Thank you for your detailed response and additional experiments. Your response has addressed some of my concerns. However, my concern about the incremental novelty of the method remains. I will therefore maintain my original rating. Good luck!

AuthorsOpenReview ↗

Thank you for your follow-up

Thank you for taking the time to review our response and additional experiments, and we respect your assessment.

We are glad the new learned-sharing baselines, matched-OOV control, and tuned-L2 frequency results addressed your empirical questions. On novelty, we fully agree that nearest-code routing and straight-through estimation are established components. We kept the parameterization intentionally simple because our goal is not a new quantization operator, but establishing learned single-vector sharing as a structural regularizer for long-tailed IDs at 10M+ scale. While prior quantized and clustered tables typically add multi-group subcodes (M>1M>1), private residuals, or staged reclustering to preserve capacity, our study shows that restricting each ID to one jointly learned shared vector is what curbs rare-ID overfitting without a global L2 search, outperforming both M>1M>1 coding and adapted CCE/CEL-Lite.

Thank you again for your feedback, which helped make the positioning and empirical controls of the revised paper much clearer.

Comments on the paper

5 comments
AuthorsOpenReview ↗

Response to Reviewer wkZc

Thank you for the specific questions on paired significance tests, matched CEL and Clustering the Sketch baselines, MI4Rec, and design/runtime trade-offs.

Q1. Paired significance tests across seeds for submitted Tables 2 and 3 (W3, Q1). Appendix I.3, Table 21 reports paired differences, sample SDs, 95% tt-intervals, and unadjusted two-sided pp-values (df=2df=2) from unrounded seed AUCs (submitted Tables 2–3 become revised Tables 3–4 after inserting Table 1):

Submitted Table 2 (λ=0\lambda=0)TAAC Δ\Delta ± SD (95% CI, pp)KuaiRand Δ\Delta ± SD (95% CI, pp)
vs. Continuous (00)+0.4565 ± 0.1975 ([-0.034, +0.947], p=0.0571p=0.0571)+0.0743 ± 0.0539 ([-0.060, +0.208], p=0.1395p=0.1395)
vs. AdamAR+0.0621 ± 0.1798 ([-0.385, +0.509], p=0.6104p=0.6104)+0.1355 ± 0.0090 ([+0.113, +0.158], p=0.0015p=0.0015)
vs. TailShare+0.5606 ± 0.2333 ([-0.019, +1.140], p=0.0532p=0.0532)+0.1316 ± 0.0542 ([-0.003, +0.266], p=0.0521p=0.0521)
vs. FrozenHash+0.5862 ± 0.1749 ([+0.152, +1.021], p=0.0284p=0.0284)+0.1233 ± 0.0550 ([-0.013, +0.260], p=0.0604p=0.0604)
Submitted Table 3 (vs. Tuned Cont.)Mean Δ\Delta ± SD (pp)95% tt-CI (pp)Two-sided pp (df=2df=2)
TAAC / RankMixer (λ∗=3×10−9\lambda^*=3\times10^{-9})+0.0738 ± 0.1959[-0.4128, +0.5604]0.5809
TAAC / DIN (λ∗=10−9\lambda^*=10^{-9})+0.4411 ± 0.0964[+0.2016, +0.6806]0.0156
KuaiRand / RankMixer (λ∗=0\lambda^*=0)+0.0743 ± 0.0539[-0.0595, +0.2082]0.1395
KuaiRand / TokenMixer-L (λ∗=10−11\lambda^*=10^{-11})+0.0181 ± 0.1100[-0.2552, +0.2913]0.8028

With three seeds (df=2df=2, t0.975,2=4.303t_{0.975,2}=4.303), per-setting intervals are wide even when all seeds are positive; TAAC/DIN (p=0.0156p=0.0156), TAAC/FrozenHash (p=0.0284p=0.0284), and KuaiRand/AdamAR (p=0.0015p=0.0015) reach p<0.05p<0.05 individually, while an exploratory equal-weight Welch–Satterthwaite summary across the four submitted Table 3 settings gives +0.1518 pp (95% CI [+0.0563, +0.2473] pp, p=0.0102p=0.0102 under independent-stratum assumptions).

Q2. Comparison with CEL and Clustering the Sketch at matched KK (W1, Q2). Table 1 (§2.5) contrasts IDShare against multi-group coding (M>1M>1), MGQE, CEL, CCE, and AutoAssign+, and §4.6, Table 6 (Appendix I.1, Table 18) compares IDShare with adapted CCE (two K/2K/2-row tables across four subspaces, matching KdKd shared floats) and CEL-Lite (loss reassignment and gradient-PCA splitting up to KK clusters without private residuals, under our Adam setup). On TAAC (K=20KK=20\text{K}), IDShare (77.5260%) beats CCE (77.0345%) by +0.4915 ± 0.2170 pp (3/3 seeds) and CEL-Lite (74.6764%) by +2.8496 ± 0.1495 pp (3/3 seeds). On KuaiRand (K=40KK=40\text{K}), IDShare (75.4446%) beats CCE (75.3374%) by +0.1072 ± 0.0581 pp (3/3 seeds) and CEL-Lite (71.1405%) by +4.3041 ± 0.4980 pp (3/3 seeds) (validation selects CCE's epoch-1 checkpoint before reclustering; 1–10 lookup seed directions vs. CCE are 2/1).

Q3. Relationship with MI4Rec and item content features (W2, Q3). Thank you for pointing out MI4Rec (Zheng et al., CIKM 2025), now cited and compared in Table 1 and §6. Both represent sparse or cold-start items via shared learnable vectors, but MI4Rec uses item text and a pretrained LM encoder to form continuous attention mixtures over meta-item tokens, whereas IDShare learns hard single-code assignments (K∈[10K,80K]K \in [10\text{K}, 80\text{K}]) from interaction gradients alone (also applying directly to user and anonymized ad IDs without text). When item content features exist, the two are complementary: backbones can ingest content/meta-item features alongside IDShare's collaborative ID branch (our TAAC and KuaiRand models already use categorical side features), and content projections could initialize or condition routing states rir_i in future work.

Design choices and computational cost (W4). Splitting d=16d=16 into M=2M=2 or M=4M=4 subspaces (§4.5; Appendix G, Tables 14–15) cuts 1–10 lookup complete-tuple sharing from 100.0% to 6.0% and 0.0% and lowers overall mean AUC (77.6088% vs. 77.4892% and 77.5548%). Temperature schedule, hash initialization, radius floor, loss weights, and b0,b+b_0, b_+ are held fixed across all datasets, backbones, KK, and depths without retuning, though holding them fixed does not replace single-factor ablations. On a single GPU (§5; Appendix I.3, Table 22), native training takes 66.86 ms per step vs. 28.94 ms for Tuned Continuous (2.31×2.31\times, +11.7% peak alloc.), while exporting shrinks saved full-model state by 72.3% (1,476.4 → 409.3 MB) and cuts inference peak allocated memory by 61.5%–70.2% vs. Continuous (+4.6%–5.4% p50 latency).

Dear Reviewer wkZc, with a couple of days left in the discussion period, we wanted to check whether the paired significance tests (Table 21), the matched CCE and CEL-Lite runs (Table 6), and the MI4Rec comparison (§6) resolve your questions.

On your point regarding VQ and DPQ building blocks, Appendix Tables 14–15 add one direct check. Keeping total width fixed at d=16, we split single-code IDShare (M=1) into M=2 and M=4 DPQ-style subcodes. Complete-tuple sharing on 1–10 lookup IDs drops from 100.0% to 6.0% and 0.0%, and overall AUC falls from 77.609% to 77.489% and 77.555%. Higher compositional capacity hurts rather than helps here, because full single-vector tying is what regularizes rare IDs. Please let us know if anything in the statistical tables or baseline setups remains unclear.

Area_Chair_SgakOpenReview ↗

Reminder: Reviewer–Author Discussion

Thank you to the reviewers who have already engaged in the discussion and provided valuable feedback. As the discussion deadline is approaching, I would kindly encourage the remaining reviewers to review the authors' responses and share their follow-up comments, particularly on whether their concerns have been adequately addressed.

Authors are also encouraged to respond promptly to any further questions. Thank you all for your time and contributions to a constructive discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.