acceptodds
Under review as a conference paper at ICLR 2027

WORD-CTR: Write Once, Read Deep for Efficient Long-Sequence Click-Through Rate Prediction

Abstract

Click-through rate (CTR) prediction often scores many candidate items for the same user, making repeated access to long behavior histories costly. Reusing a candidate-independent user state can reduce this repeated work, provided that different candidates can still obtain distinct information from it. We present WordCTR (Write Once, Read Deep), which represents the retained behavior history as a fixed-size, multi-head associative memory. Its writer fits a linear key-to-value map in each head using content- and recency-weighted ridge regression. Stacked reader blocks combine non-sequential user, candidate, and context fields and make successive queries to the same memory without revisiting the behavior sequence. For fixed model dimensions, candidate-side reading is independent of history length once the memory is available, while memory construction remains history-dependent. We evaluate WordCTR against 17 baselines on three public datasets and an InHouse dataset that includes a behavior stream of up to 1024 events. WordCTR obtains the best reported AUC, GAUC, and NE on the public datasets, and the best AUC and NE on InHouse. We also examine cached and uncached inference across history lengths and candidate counts. In the measured single-request GPU workload, candidate-side read-and-score time varies little across tested history lengths, while caching reduces total latency with a larger benefit at higher tested candidate counts.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Click-through rate prediction, long user behavior modeling, user memory, long-sequence recommendation, multi-interest modeling, efficient inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TZdZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes WordCTR, a CTR predictor that compresses retained user histories into candidate-independent, fixed-size multi-head associative memories. Each writer fits a content- and recency-weighted ridge-regression key-to-value map. Stacked reader blocks combine user, candidate, and context fields and repeatedly query the same memory without revisiting the raw history. For fixed model dimensions, candidate-side reading is independent of history length once the memory is available, while writing remains history-dependent.

The authors compare predictive performance against 17 baselines on three public datasets and an InHouse dataset with a behavior stream of up to 1024 events. WordCTR has the best reported public-dataset metrics and InHouse AUC and NE, but not InHouse GAUC. Single-request GPU measurements show weak history-length dependence in candidate-side scoring and benefits from cross-candidate reuse under the tested workload.

Critical strengths
  1. The architecture coherently addresses repeated history processing when scoring many candidates for one user. A candidate-independent matrix memory supports distinct candidate-conditioned readouts, while stacked readers refine field interactions without revisiting the sequence. The explicit separation of history-dependent writing from history-independent reading is useful and analytically clear.

  2. The reported predictive results are promising across four datasets and 17 baselines. The public-dataset results suggest that fixed-size compression can retain useful predictive information, and the inference breakdown and depth/temporal ablations help characterize practical tradeoffs. The paper also acknowledges that writing is not free and that InHouse GAUC does not improve over the strongest baseline.

Critical weaknesses
  1. The predictive advantage is not yet sufficiently supported by protocol and uncertainty information. The train/validation/test chronology, history construction, negative sampling, and baseline search budgets need clearer documentation. The InHouse AUC lead is only 0.0005 and the KuaiVideo lead is 0.0019, making repeated-run uncertainty particularly important. Public datasets retain only 50–100 events, so long-history accuracy evidence rests on the private dataset. Moreover, the existing ablations do not isolate the ridge solve from a capacity-matched simpler associative writer, or content gating from temporal decay. These gaps limit confidence in both the reported advantage and its attribution to the proposed combination.

  2. The efficiency comparison does not isolate performance under equivalent user-state reuse. Table 3 shows that the measured MIMN candidate-side cost is lower than WordCTR’s; WordCTR’s total advantage comes from user-side construction. MIMN also supports pre-maintained user states, so rebuilding its history within each request is different from evaluating a cache-hit scoring path. The cached WordCTR measurements still include precompute, leaving cross-candidate versus cross-request cache semantics unclear. The current measurements support the tested implementation and workload, but do not establish a serving advantage when both user states are already available.

Questions for authors

The following clarification requests could materially affect my assessment:

  1. Please provide the train/validation/test chronology, history construction and negative-sampling protocols, baseline search budgets, and repeated-seed uncertainty for WordCTR and the strongest baselines.

  2. Does cached mean reuse within one candidate batch or a cache hit across requests? Can you compare candidate scoring when both methods’ user states are already available, and report write/refresh costs separately under identical assumptions?

  3. Can a focused capacity-matched comparison with a simple associative-memory writer isolate the benefit of the ridge solve? Separating content gating and temporal decay would also clarify which components drive the gains.

Reviewer_oVU3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes WordCTR, a long-sequence CTR prediction model designed for scenarios where many candidate items are scored for the same user state. The key idea is to separate history-dependent computation from candidate-dependent scoring. WordCTR first writes the retained user behavior sequence into a fixed-size, multi-head associative memory using content- and recency-weighted ridge regression. Then, stacked reader blocks repeatedly query this cached memory using non-sequential user, candidate, and context features, without revisiting the raw behavior sequence for each candidate. The paper evaluates WordCTR against 17 baselines on three public datasets and one InHouse dataset with histories up to 1024 events. WordCTR achieves the best reported AUC, GAUC, and NE on the public datasets, and the best AUC and NE on InHouse. The paper also reports cached and uncached inference latency, showing that candidate-side reading is weakly dependent on history length once the memory is available.

Critical strengths

The main strength of the paper is the clear system-aware formulation of CTR prediction with repeated candidate scoring. Instead of treating each user-candidate pair independently, WordCTR explicitly exploits cross-candidate reuse by writing the user history once and reading it multiple times. This is practically relevant for ranking systems where many candidates share the same user state. The proposed architecture is also technically coherent. The weighted ridge-regression memory gives a fixed-size candidate-independent user representation, while the stacked reader preserves candidate-conditioned access. The use of content and recency weights is intuitive for user behavior modeling, and the ablation on uniform writes supports its effectiveness. The empirical evaluation is fairly broad. The paper compares against many representative CTR baselines across public and internal datasets, and the reported accuracy improvements are consistent. The latency breakdown is also useful because it distinguishes user-side memory construction from candidate-side scoring, which is important for understanding the actual serving benefit.

Critical weaknesses

The first concern is that the novelty is somewhat incremental. Candidate-independent user-state caching, memory-based user modeling, and associative memory have all appeared in prior work. The paper combines these components in a reasonable way, but the conceptual contribution may be closer to an engineering integration than a fundamentally new modeling principle. The distinction from MIMN, SDIM, TWIN V2, and other reusable user-state methods could be sharpened further. The second concern is about the fairness and generality of the latency comparison. The efficiency results are mainly compared with one measured MIMN implementation under a single-request GPU workload. Many production CTR systems use batching, CPU/GPU hybrid serving, caching, retrieval indexes, and optimized kernels. Therefore, the reported latency advantage may not fully reflect deployment-level efficiency. The paper is careful about this limitation, but the experimental evidence is still narrow relative to the efficiency claims. A third weakness is that the ablation study is limited. The paper studies reader depth, temporal buckets, and uniform writes, but it does not fully isolate the contribution of the ridge memory itself against simpler fixed-size memory alternatives, such as learned slots, pooled multi-vector representations, low-rank attention summaries, or non-ridge associative maps. This makes it harder to determine whether the gains come specifically from the proposed weighted ridge memory or from the general write-once/read-many design.

Questions for authors
  1. How does WordCTR compare with simpler fixed-size multi-vector user representations, such as learned pooling slots or attention-pooled interest vectors, under the same reader architecture?
  2. Can the authors provide a more detailed comparison with MIMN, SDIM, and TWIN V2 in terms of what is cached, when it is updated, and what computation remains candidate-dependent?
  3. The latency experiments use a single-request GPU workload. How would the results change under batched serving or CPU-side memory construction, which are common in CTR systems?
  4. On the InHouse dataset, WordCTR improves AUC and NE but not GAUC. Can the authors explain why the user-group ranking metric does not improve and whether this suggests uneven gains across users?
  5. How often must the memory be rebuilt in a realistic online system as new user behaviors arrive, and what is the amortized cost under frequent user-state updates? Suggested Scores
Reviewer_haYxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper is built upon the retrieval-based CTR model. However, I can not see a significant difference or novelty between this paper and the retrieval-based CTR models. Moreover, evidence for predictive gains on long histories is limited, as the public datasets retain only 50–100 events and the 1024-event setting is evaluated only on internal data. Accuracy-versus-history-length curves and repeated runs with confidence intervals would help establish the benefits and robustness of the method.

Critical strengths
  1. This paper has strong results over multiple baseline methods.
Critical weaknesses
  1. The novelty is not clear. The difference or novelty between this paper and the retrieval-based CTR models should be further clarified.
  2. Evidence for predictive gains on long histories is limited, as the public datasets retain only 50–100 events and the 1024-event setting is evaluated only on internal data. Accuracy-versus-history-length curves and repeated runs with confidence intervals would help establish the benefits and robustness of the method.
Questions for authors
  1. Could you clarify the methodological novelty relative to retrieval-based CTR models and existing memory-based approaches? In particular, what advantages arise from the proposed ridge-regression memory and stacked readers beyond candidate-independent history reuse?
  2. Could you report predictive performance across different retained history lengths, particularly beyond 100 events? Since the longest-history setting is evaluated only on internal data, additional public-data experiments and repeated runs with confidence intervals would strengthen the evidence.
  3. What benefits does ridge-regression memory provide over a simpler weighted linear associative memory when using the same reader architecture?
  4. How does the number of reader layers affect the accuracy–latency trade-off? Do additional reads provide sufficient gains to justify their cost?
  5. How does WordCTR compare with MIMN and other memory-based methods when both start with cached user states, excluding full-history reconstruction?
  6. How is the memory updated when new behaviors arrive or old behaviors leave the retained window? What are the associated computation and storage costs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.