acceptodds
Under review as a conference paper at ICLR 2027

TRAIL: An Efficient Industrial Ranking Framework via Interest Evolution and Target-Relevant Aggregation

Abstract

Modeling long user behavior sequences is crucial for click-through rate (CTR) prediction, which requires efficiently capturing user interest evolution over long histories and target-relevant behaviors. Target-aware attention methods are efficient and directly model the relevance between target items and historical behaviors, but they mainly focus on matching the target item with individual behaviors, leaving interest evolution underexplored. Self-attentive models capture behavior interactions, but their sequence modeling is usually performed without conditioning on the target item and incurs quadratic computational cost. We propose TRAIL, an efficient framework for long sequence CTR prediction via interest evolution and target-relevant aggregation. TRAIL first applies the State-Compressed Gated DeltaNet (SC-GDN) to model interest-state evolution over the raw behavior sequence with linear complexity. It then uses Segmented Target Attention (STA) to construct target-relevant segment representations, reducing sequence length while preserving target-relevant behavioral signals. Over the compressed sequence, Global Stacked Target Attention (GSTA) further aggregates target-relevant information with low computational cost. Finally, a Multi-resolution Interest Fusion module integrates multi-level interest representations for CTR prediction. Experiments show that TRAIL achieves the best AUC and GAUC while using only about 15% of the FLOPs required by the full-attention model. Online A/B testing further yields +2.51% advertiser value and +2.36% revenue gains, demonstrating TRAIL's effectiveness and efficiency in industrial long sequence CTR modeling.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Long Sequence Modeling, Recommender Systems, Scaling Law

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rLtZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TRAIL, an efficient framework for long-sequence CTR prediction. It combines Gated DeltaNet-based interest evolution, segmented target attention, and global target-aware aggregation to capture user interests from long behavior histories. Experiments on KuaiRand-1K and an industrial dataset show improvements in AUC and GAUC with relatively low computational cost. The reported online A/B test also shows positive business gains. I find the practical motivation reasonable, though the novelty of the proposed architecture is less clear.

Critical strengths
  1. The paper addresses an important problem in industrial recommendation. Modeling long user behavior sequences under strict serving constraints is practically meaningful.

  2. The experimental results are encouraging. TRAIL achieves competitive AUC and GAUC while substantially reducing computation compared with full attention.

  3. I appreciate the online A/B evaluation. The reported 2.51% improvement in advertiser value and 2.36% revenue gain provide useful evidence beyond offline benchmarks.

Critical weaknesses
  1. Novelty is somewhat limited. Interest evolution and target-aware attention have been explored in DIEN, STCA, and LONGER. TRAIL seems more like a combination of existing techniques.

  2. The individual contributions of the modules are not fully convincing. Some ablation gains are quite small, and I would like to see stronger architectural controls.

  3. Efficiency comparisons could be more complete. FLOPs are reported, but actual serving latency and memory usage would be more informative.

  4. The online gains are promising, although the seven-day A/B test lacks statistical significance information. This makes the business impact somewhat difficult to judge.

Questions for authors
  1. What is the key difference between TRAIL and a simpler combination of Gated DeltaNet-2 and STCA under the same computational budget?

  2. Could the authors clarify whether the smaller ablation improvements are statistically significant across multiple seeds?

  3. How does TRAIL compare with LONGER and STCA in actual inference latency, memory consumption, and throughput?

  4. Could the authors provide confidence intervals or significance tests for the online A/B improvements?

6 comments
AuthorsOpenReview ↗

Response to rLtZ (1/2): Architectural Comparisons

Thank you for recognizing TRAIL’s practical relevance. We respectfully disagree with the characterization of TRAIL as a straightforward combination of existing techniques. Our contribution is a hierarchical architecture that encodes behavior transitions before compression, uses the target to guide local aggregation, and performs global interaction over the compressed representations. This design combines behavior-level temporal modeling with target-aware compression while controlling the cost of repeated global interaction, improving prediction accuracy and inference efficiency under industrial serving constraints. The revised manuscript adds architectural comparisons, measured inference results at 10K, and statistical significance results for the ablation and online gains. We address each question below.

Q1. Architectural difference from combining Gated DeltaNet-2 with STCA.

TRAIL introduces target-aware local aggregation between behavior-level interest encoding and stacked global attention. A direct combination of GDN2 and STCA applies global attention to all LL encoded behaviors. TRAIL instead uses the target to construct a representation within each temporal window, then performs global interaction over the resulting L/wL/w representations. This design allows temporal context and target relevance to jointly inform local aggregation while reducing the cost of repeated global interaction under industrial serving constraints.

Architectural distinction from prior work. We would like to clarify that TRAIL is not merely a straightforward combination of existing techniques. DIEN introduces target relevance into its interest-evolution layer. TRAIL first encodes the full history independently of the target, allowing these representations to be cached and reused during ranking. STA subsequently uses the target as the query to weight and aggregate temporally enriched behaviors within local windows.

LONGER uses local InnerTransformer interactions for token merging. STA uses the target to weight and aggregate behaviors already enriched with temporal context, producing a target-specific representation for each segment. GSTA then performs repeated global aggregation over these segment representations.

Controlled architectural comparisons. We evaluate the following variants:

VariantAUCGAUCMFLOPs
TRAIL0.82120.7390199.3
w/o STA0.82070.7383251.5
SC-GDN + GSTA (matched budget)0.82050.7380199.6
STA-Mean0.82000.7370158.8

The w/o STA variant applies stacked global attention directly to the full encoded history, keeping the encoder and downstream configurations unchanged. TRAIL improves AUC/GAUC by 0.05/0.07 percentage points while reducing computation from 251.5M to 199.3M FLOPs, a 20.8% reduction. This comparison demonstrates improved prediction quality with lower computational cost.

For the requested comparison at matched computational budgets, we additionally evaluate SC-GDN followed by two layers of target cross-attention over all 1,000 behaviors. We keep L=1000L=1000, d=128d=128, MIF, input features, and the prediction head unchanged, and set SC-GDN’s state dimensions to dk′=16d_k'=16 and dv′=32d_v'=32, compared with 16/6416/64 in TRAIL. This yields 199.6M FLOPs, closely matching TRAIL’s 199.3M, and achieves 0.8205 AUC / 0.7380 GAUC. TRAIL therefore improves AUC/GAUC by 0.07/0.10 percentage points at essentially the same computational budget, supporting its allocation of computation to interest encoding and target-aware local aggregation before global interaction.

The STA-Mean variant preserves the temporal windows, compressed sequence length, and segment-level FFN, replacing target attention with mean pooling within each segment. TRAIL improves AUC/GAUC by 0.12/0.20 percentage points, supporting target-aware construction of segment representations at the same compression ratio.

The w/o STA and STA-Mean comparisons are reported in Section 4.3, Table 2.

AuthorsOpenReview ↗

Response to rLtZ (2/2): Statistical Significance and Inference Efficiency

Q2. Statistical significance of ablation gains across random seeds.

The ablation gains are consistent across all five random seeds, and the paired comparisons achieve p<0.05p<0.05 for both AUC and GAUC, including variants with smaller average differences.

We evaluate complete TRAIL and each variant using the same five seeds (42, 3407, 1234, 5678, and 2026) and perform a two-sided paired t-test across seeds. The resulting p-values are:

Variant compared with TRAILAUC p-valueGAUC p-value
STA-Mean0.00002570.00000930
w/o STA0.0003640.000379
w/o SC-GDN0.00004550.000689
GSTA-1L0.005410.000689
w/o MIF0.0007990.00000834
STA first0.0006890.000394
Conv (k=3k=3)0.0003640.000220
Conv (k=10k=10)0.0008640.0000145
minGRU0.0006890.000348

These results support the consistency and statistical significance of the improvements. The comparisons of encoding order, encoder choice, and target participation in local aggregation provide complementary evidence for the individual architectural decisions.

The ablation and architectural comparison results are reported in Section 4.3, Table 2.

Q3. Inference latency, throughput, and memory consumption.

At 10K, TRAIL achieves higher AUC and GAUC than STCA and LONGER while reducing measured P99 latency by 28.7% and 20.0%, respectively. Its throughput reaches 899.78 requests/s, exceeding both baselines, with 5.28 GiB peak GPU memory.

The comparison on the same industrial dataset at L=10,000L=10{,}000 gives:

ModelAUCGAUCP99 latency (ms)Throughput (requests/s)Peak GPU memory (GiB)
STCA0.78210.723476.82618.386.31
LONGER0.78270.723968.54751.944.56
TRAIL0.78490.726554.81899.785.28
TRAIL w/o SC-GDN0.78390.725049.62987.693.46

Measurements use the same NVIDIA H20 hardware, batch size 64, 50 candidates per request, and BF16 precision. Standard attention operations use FlashAttention, while SC-GDN uses Flash Linear Attention for chunkwise parallel computation. Model-specific history caching precomputes and reuses candidate-independent representations, including SC-GDN’s history encoding in TRAIL.

TRAIL achieves 0.7849 AUC / 0.7265 GAUC at 54.81 ms P99 latency, with a throughput of 899.78 requests/s, compared with 618.38 for STCA and 751.94 for LONGER. TRAIL w/o SC-GDN yields slightly lower AUC and GAUC than complete TRAIL, but still outperforms both baselines with lower latency, higher throughput, and lower peak memory. These results demonstrate TRAIL’s effectiveness and efficiency in industrial ranking with 10K behavior histories.

The inference settings and measured results are reported in Section 4.5, Table 5.

Q4. Statistical significance of the online A/B improvements.

The online improvements in ADVV, revenue, and CTR are statistically significant at the 0.05 level, with p-values of 0.0389, 0.0326, and 0.0401, respectively.

MetricRelative liftp-value
ADVV+2.51%0.0389
Revenue+2.36%0.0326
CTR+1.93%0.0401

The experiment runs for seven days on a platform serving on the order of 100 million daily active users. Users are randomly assigned by UserID, and 10% of total traffic is allocated to the experimental group. The experimental group replaces the production control model with TRAIL, with all other settings identical to those of the control group.

At the platform’s current revenue scale, sustaining the observed 2.36% revenue lift would translate into an estimated annual revenue increase on the order of hundreds of millions. The randomized assignment and identical settings apart from the model replacement support attributing the observed business improvements to TRAIL.

The online experimental setup is described in Section 4.6, with relative improvements and p-values reported in Table 7.

The architectural comparisons clarify how target participation in local aggregation distinguishes TRAIL from directly combining recurrent encoding with stacked target attention. The paired experiments support the significance of the ablation gains, while the measured inference results and significant online improvements demonstrate practical value. We hope these additions address the reviewer’s questions about novelty, component contributions, efficiency, and statistical significance, and support a favorable reassessment of our work.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer rLtZ,

Thank you again for your feedback. We would like to briefly follow up on our rebuttal, in which we clarified the architectural differences from combining GDN2 with STCA and provided controlled comparisons examining target participation in local aggregation. We also addressed the statistical significance of the ablation results across five random seeds.

We would appreciate your feedback on whether these results and clarifications address your main concerns. Thank you for your time and consideration.

Reviewer_WBdKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TRAIL is a four-stage long-sequence CTR pipeline. SC-GDN, a Gated DeltaNet-2 recurrence with reduced key and value dimensions, encodes interest evolution over the raw sequence. STA then compresses it ten to one using segment-level target attention. GSTA stacks two target-to-segment cross-attention layers over the compressed sequence, and MIF fuses global, pooled, recent and context features through RankMixer. Evaluated on KuaiRand-1K and a private advertising dataset at L = 1000 against DIN, TWIN, HSTU, STCA and LONGER, with ablations, sensitivity analyses, scaling curves, reported per-sample FLOPs, and a seven-day A/B test on 10% of traffic (ADVV +2.51%, Revenue +2.36%).

Critical strengths
  • Method design. Placing the recurrent encoder before compression inverts the usual pipeline, in which retrieval or pooling shortens the history before any dependency modeling happens. The paper pairs this choice with an explicit complexity analysis per module, so the accuracy and cost consequences of the ordering can be discussed directly rather than argued qualitatively. Keeping all target-conditioned interaction on the compressed sequence is a sensible way to buy depth cheaply.

  • Experimental completeness. The evidence runs from offline AUC and GAUC through hyperparameter sensitivity for two modules, scaling behavior in both sequence length and model depth, a per-sample compute account, and finally a production A/B test. Carrying the claim all the way to online business metrics is uncommon in submissions of this type and is the most substantive part of the paper.

  • Quality of the ablation design. The STA-Mean variant keeps the segmentation structure and the compressed length fixed and changes only how tokens inside a segment are aggregated. This separates the benefit of shortening the sequence from the benefit of choosing what to keep, which is more informative than simply deleting a module and reporting the drop.

Critical weaknesses
  • By the complexity analysis in Sec. 3.4, SC-GDN is the only module still operating at the uncompressed length L, while the other three run on L/w segments, so it necessarily dominates the reported cost. Removing it, however, costs only 0.06% AUC. The most expensive component is thus the least productive one, which is the opposite of what an efficiency thesis requires.

  • STA collapses each ten-token segment immediately after SC-GDN produces per-token states, so most within-segment detail is averaged away by the target-attention weights. The natural control, running the same recurrence on the segment sequence instead of the raw one, is never reported, although it is the direct test of the paper's central ordering claim.

  • The ablation deltas sum to 0.31% AUC while the end-to-end gain over LONGER is 0.17%. The component-wise attribution is therefore not internally consistent: either the modules are substantially redundant, or the ablations and the main table are not measured against a common reference.

  • Novelty is limited at the component level. SC-GDN is GDN2 with smaller key and value dimensions, and the sensitivity study shows AUC is unchanged across that range, so the compression buys compute rather than modeling capacity. GSTA is stacked target-to-history attention, which the paper itself attributes to STCA, applied to a compressed sequence. STA follows the TWIN and LONGER line, and RankMixer is used unmodified.

  • The claim of using about 15% of the FLOPs is not established under a stated protocol, since the main text does not say whether all methods are counted the same way. No latency, throughput or memory figure is measured either, although a recurrent scan over a thousand tokens and batched attention over a hundred segments behave very differently at equal FLOPs.

  • The abstract claims histories of up to ten thousand interactions, but all main results use L = 1000. The 10k point appears only in a scaling curve with no baseline, and the segment size is never retuned even though the paper states it should scale with L.

  • All ablations and sensitivity studies are run on a public dataset of a thousand users, while the large-scale evidence sits in a private dataset with no code release, so none of the design conclusions is externally checkable.

Questions for authors
  • What are the AUC, GAUC and FLOPs of applying the same recurrence to the compressed segment sequence instead of the raw sequence?

  • How do you reconcile the summed ablation deltas of 0.31% with the end-to-end gain of 0.17% over LONGER?

  • Under what convention are the baseline FLOPs counted, and can you report measured P99 latency, QPS and peak memory on the same hardware?

  • Can you provide the comparison against LONGER and STCA at L = 10000 with the segment size retuned for that length?

  • Does replacing SC-GDN with a generic linear-cost recurrent block of matched cost preserve the gain, which would attribute the benefit to interest-evolution encoding rather than to GDN2 specifically?

  • For the online experiment, what was the control model, what happened to online CTR and CVR, and what was the serving-cost delta relative to the incumbent?

6 comments
AuthorsOpenReview ↗

Response to WBdK (1/4): Architectural Contributions

We thank the reviewer for recognizing TRAIL’s design and practical relevance. We respectfully disagree with the assessment that TRAIL offers limited architectural novelty. Its contribution lies in coordinating behavior-level interest encoding, target-aware compression, and global aggregation to improve prediction accuracy and inference efficiency under industrial serving constraints. SC-GDN does not yield the smallest component gain: removing it reduces AUC/GAUC by 0.06/0.09 percentage points. Its candidate-independent representations are cached across ranking requests. At 10K, SC-GDN improves AUC/GAUC by 0.10/0.15 percentage points, while complete TRAIL remains faster than STCA and LONGER. The revised manuscript adds architectural comparisons, measured inference efficiency and baseline results at 10K, and further details of the online experiment. We address each question below.

Architectural contributions and differences from prior work. TRAIL addresses two connected challenges in industrial ranking: modeling behavior-level temporal dependencies before history compression and controlling the cost of repeated global interaction over long histories. Our contribution is a hierarchical architecture that first encodes behavior-level interest evolution, then uses the target to guide local compression, and finally performs repeated global interaction over the compressed representations. This ordering allows temporal context to inform target-aware compression, while reducing the sequence length processed by subsequent global attention layers. These design choices aim to improve prediction accuracy and inference efficiency under industrial serving constraints.

SC-GDN. SC-GDN encodes behavior-level interest evolution before compression, allowing subsequent aggregation to use temporally enriched behaviors. It retains the GDN2 update rule in compact key/value subspaces to control computation, and its candidate-independent representations are cached for reuse during ranking. The state-dimension analysis, encoding-order comparison, and convolutional and minGRU replacements examine the efficiency, placement, and choice of this encoder.

STA and GSTA. TWIN retrieves top-KK original behaviors from the full history, while LONGER uses local InnerTransformer interactions for token merging. STA lets the target directly guide aggregation within each consecutive temporal window, constructing a target-specific summary from the encoded behaviors. GSTA then performs stacked target attention over these L/wL/w summaries, whereas STCA reads the full history at each layer. This design preserves temporal context before compression and concentrates repeated global interaction on compact representations relevant to the target. The w/o STA and STA-Mean comparisons evaluate the value of this intermediate aggregation and its use of target information.

MIF. MIF uses RankMixer to fuse complementary views, including global target interest, mean and max pooling, recent segments, and contextual features. Its architectural role is to retain information at multiple resolutions for prediction, with the MIF ablation evaluating the contribution of this fusion.

These choices form a coordinated architecture for improving ranking effectiveness within industrial serving constraints. The architectural comparisons and component ablations examine its key decisions, while the 10K measurements demonstrate improved ranking quality with lower latency than STCA and LONGER. The online experiment further establishes its value in production.

AuthorsOpenReview ↗

Response to WBdK (2/4): Encoding Order and Ablation Interpretation

Q1. Applying SC-GDN after sequence compression.

Applying SC-GDN after compression achieves 0.8209 AUC / 0.7384 GAUC at 98.3M FLOPs. Encoding before compression achieves higher prediction accuracy because behavior-level temporal context is available when STA constructs candidate-relevant segment representations.

We evaluate an STA-first variant that moves the same SC-GDN module after STA, keeping its configuration and the downstream modules unchanged:

Encoding orderAUCGAUCMFLOPs
SC-GDN → STA → GSTA (TRAIL)0.82120.7390199.3
STA → SC-GDN → GSTA0.82090.738498.3

Encoding before compression improves AUC/GAUC by 0.03/0.06 percentage points. SC-GDN enriches individual behaviors with their preceding interest states before aggregation. When applied after STA, it models transitions between segment summaries, after individual behaviors have already been aggregated. The comparison supports encoding behavior-level interest evolution before compression.

SC-GDN representations before compression are also candidate-independent and cached across ranking requests, avoiding repeated full-history encoding. The reported FLOPs describe per-sample analytical computation; the measured inference results presented below demonstrate complete TRAIL’s latency advantage over STCA and LONGER.

The encoding-order comparison is reported in Section 4.3, Table 2, and the measured inference results are provided in Section 4.5, Table 5.

Q2. The summed ablation drops and the improvement over LONGER.

The 0.31 percentage points and 0.17 percentage points measure different comparisons and are not expected to be equal.

Each row in Table 2 independently modifies one component of complete TRAIL, measuring its effect while keeping the other components unchanged. The 0.31 percentage points sum five such AUC drops, including two alternative modifications of STA: replacing target-aware aggregation with mean pooling and removing STA entirely. These drops quantify the effects of individual changes within the complete architecture, where components interact, rather than additive contributions.

The 0.17 percentage points is the directly measured difference between TRAIL and LONGER (0.8212−0.81950.8212 - 0.8195). Since the ablations are separate modifications of TRAIL rather than successive steps that transform it into LONGER, their summed drops need not equal this difference.

The overall model comparison is reported in Section 4.2, Table 1, and the independent component ablations are reported in Section 4.3, Table 2.

AuthorsOpenReview ↗

Response to WBdK (3/4): Inference Efficiency and 10K Evaluation

Q3. FLOPs accounting and measured inference efficiency.

All models use the same FLOPs accounting scope. We additionally measure P99 latency and peak GPU memory on the same hardware at 10K. TRAIL achieves higher AUC and GAUC than STCA and LONGER while reducing P99 latency by 28.7% and 20.0%, respectively.

Consistent FLOPs accounting. We count one MAC as two FLOPs and report per-sample forward computation for the sequence modeling components. Input embeddings and the final prediction tower use common configurations across models and are excluded. Normalization, masking, and pooling are also omitted consistently across all models as lower-order operations.

Measured inference efficiency. The comparison on the industrial dataset at L=10,000L=10{,}000 gives the following results:

ModelAUCGAUCP99 latency (ms)Throughput (requests/s)Peak GPU memory (GiB)
STCA0.78210.723476.82618.386.31
LONGER0.78270.723968.54751.944.56
TRAIL0.78490.726554.81899.785.28
TRAIL w/o SC-GDN0.78390.725049.62987.693.46

Measurements use the same NVIDIA H20 hardware, batch size 64, 50 candidates per request, and BF16 precision. Standard attention operations use FlashAttention, while SC-GDN uses Flash Linear Attention for chunkwise parallel computation. Model-specific history caching precomputes and reuses candidate-independent representations, including SC-GDN’s history encoding in TRAIL.

Complete TRAIL achieves 0.7849 AUC / 0.7265 GAUC at 54.81 ms P99 latency, with a throughput of 899.78 requests/s and 5.28 GiB peak GPU memory. Its throughput exceeds that of STCA (618.38 requests/s) and LONGER (751.94 requests/s). TRAIL w/o SC-GDN yields lower AUC and GAUC than complete TRAIL, but still outperforms both baselines with lower latency, higher throughput, and lower peak memory. These results demonstrate TRAIL’s effectiveness and efficiency in industrial ranking with 10K behavior histories.

The FLOPs accounting and detailed derivation are described in Appendix A. The measured inference settings and results are reported in Section 4.5, Table 5.

Q4. Segment size adjustment and baseline comparisons at 10K.

We evaluate segment sizes of 10, 20, 50, and 100 at L=10,000L=10{,}000. A segment size of 10 achieves the highest prediction quality, while larger segments offer lower computational cost. At the same history length, TRAIL outperforms STCA and LONGER on both AUC and GAUC with lower measured P99 latency.

Segment size evaluation. On the industrial dataset, varying segment size while keeping all other settings unchanged gives:

Segment size wwCompressed length L′L'AUCGAUCMFLOPs
1010000.78490.72651927.7
205000.78470.72621714.2
502000.78440.72571586.1
1001000.78390.72501543.4

Increasing ww from 10 to 20 reduces FLOPs by 11.1%, with AUC/GAUC decreases of only 0.02/0.03 percentage points. Increasing ww to 100 reduces FLOPs by 19.9%. Segment size therefore provides a practical deployment choice: retaining finer segment representations for higher prediction accuracy or increasing compression to accommodate a smaller computational budget.

Baseline comparisons. With w=10w=10, TRAIL achieves 0.7849 AUC / 0.7265 GAUC, exceeding STCA by 0.28/0.31 percentage points and LONGER by 0.22/0.26 percentage points. Its P99 latency is 54.81 ms, compared with 76.82 ms for STCA and 68.54 ms for LONGER under the same evaluation settings, as detailed in Q3.

The segment-size evaluation and baseline comparisons are reported in Section 4.5, Tables 6 and 5, respectively.

Reviewer_xhDYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TRAIL is a framework for click-through rate prediction from long user histories. It applies a state-compressed Gated DeltaNet encoder before target-aware segment aggregation, then uses stacked target attention and multi-resolution fusion with RankMixer for prediction. The paper reports improvements over DIN, TWIN, HSTU, STCA, and LONGER on KuaiRand-1K and an industrial advertising dataset. Its default configuration uses approximately 199M modeling FLOPs, compared with 1354M for HSTU and 172M for LONGER. A seven-day online experiment reports gains of 2.51% in advertiser value and 2.36% in revenue.

Critical strengths
  1. Encoding the full history before target-aware compression is a sensible design for retaining information that early selection could discard. The STA-Mean comparison is useful: it preserves the segmentation and compressed length while replacing target-aware aggregation with averaging, and performance declines on both metrics. The appendix also makes the recurrent update and the analytical computation costs explicit.
  2. The reported gains are consistent across the public and industrial datasets and include comparisons with recent efficient models. On KuaiRand-1K, TRAIL improves AUC/GAUC over LONGER from 0.8195/0.7365 to 0.8212/0.7390. The five-seed evaluation and the online business results give the work practical relevance beyond a single offline comparison.
Critical weaknesses
  1. The benefit of the full-history recurrent encoder needs to be assessed against its cost. SC-GDN retains the GDN2 update [1] with smaller state dimensions, and stacked target aggregation is already present in STCA [2]. The contribution therefore depends substantially on what this combination adds. In Table 2, removing SC-GDN changes AUC/GAUC from 0.8212/0.7390 to 0.8206/0.7381, still above LONGER's 0.8195/0.7365. Appendix A assigns about 112.2M of the total 199.3M FLOPs to SC-GDN; if the other components are unchanged, the ablated model would require approximately 87.1M. The gain may still be valuable, especially for longer histories, but the paper does not report the ablation's cost or separate recurrence from added capacity and local convolution. A focused comparison with a simpler history encoder would help establish why the proposed encoder is the right use of the computation budget.
  2. The evidence for practical efficiency at ultra-long lengths is incomplete. Table 1 establishes lower analytical FLOPs than HSTU and STCA at length 1000, but TRAIL uses more FLOPs than LONGER. Figure 2 extends only TRAIL to length 10000, so it does not show the comparative trade-off there. No measured latency, throughput, or peak memory is reported. This matters because STA makes the compressed representations candidate-dependent, while the shared SC-GDN computation can potentially be reused across candidates. STCA includes request-level sharing and LONGER includes KV caching [2,3], but the comparison does not specify these settings. The per-sample counts, which exclude input embeddings and the final prediction tower, are insufficient to assess the claimed serving advantage.
Questions for authors
  1. Can you report the cost of the existing no-SC-GDN ablation and its performance at a representative longer history length? A comparison with one simpler history encoder, such as a local convolution with a similar computation budget and the same downstream modules, would distinguish the value of recurrence from additional feature transformation.
  2. Which input features, prediction heads, and tuning budgets are shared across the baselines? Please provide a comparable quality-versus-cost comparison with STCA and LONGER at one longer history length, preferably using measured latency or throughput and peak memory. State the candidate batch size, numerical precision, sharing/caching policy, and whether the FLOPs use the same accounting scope for all models.
  3. What is the production control model in the online experiment, and what exactly does ADVV measure? Please specify the randomization unit, the sample size and uncertainty of the reported lifts, and whether changes beyond TRAIL were introduced. Existing experiment details and uncertainty estimates would help determine how strongly the online results support the contribution.
4 comments
AuthorsOpenReview ↗

Response to xhDY (1/3): History Encoder Contributions

We thank the reviewer for recognizing TRAIL’s practical value. Our goal is to improve prediction accuracy and inference efficiency together under industrial serving constraints. To assess the benefit of full-history recurrent encoding alongside its cost, we report that SC-GDN improves AUC/GAUC by 0.10/0.15 percentage points at 10K with 5.19 ms of additional P99 latency, while complete TRAIL remains faster than STCA and LONGER. Its candidate-independent history representations are cached and reused during ranking. The revised manuscript also adds focused encoder comparisons and further details of the baseline and online experimental settings. We address the three questions below.

Q1. Cost and contribution of full-history recurrent encoding.

The model without SC-GDN requires 87.1M FLOPs, and adding SC-GDN improves AUC/GAUC by 0.06/0.09 percentage points. SC-GDN also outperforms local convolutions and minGRU at comparable computational costs. At a history length of 10K, it improves AUC/GAUC by 0.10/0.15 percentage points over the model without SC-GDN, while complete TRAIL retains lower measured P99 latency than STCA and LONGER.

The role of encoding before compression. SC-GDN enriches each behavior with its preceding interest state before STA constructs candidate-relevant segment summaries and GSTA aggregates information globally. These modules serve complementary roles: modeling interest evolution, constructing local representations relevant to the candidate, and integrating information across segments. SC-GDN retains the GDN2 update rule in compact key/value states to control computation. Its candidate-independent history representations are cached and reused across ranking requests.

Encoder cost and simpler alternatives. We compare the following encoders while keeping the downstream modules unchanged:

Model / encoderAUCGAUCMFLOPs
TRAIL with SC-GDN0.82120.7390199.3
w/o SC-GDN0.82060.738187.1
Local convolution, k=3k=30.82070.7381186.5
Local convolution, k=10k=100.82080.7382189.2
minGRU0.82090.7382185.4

Removing SC-GDN reduces AUC/GAUC by 0.06/0.09 percentage points. The convolutional replacements use width 192, and minGRU uses width 128. At comparable computational costs, the convolutions recover little of the improvement, while minGRU recovers part of it but remains below SC-GDN on both metrics. These results support both encoding interest evolution before compression and the choice of SC-GDN over local transformations and the simpler recurrent encoder.

Interaction with compression. Applying the same SC-GDN module after STA achieves 0.8209 AUC / 0.7384 GAUC at 98.3M FLOPs, compared with 0.8212 / 0.7390 at 199.3M FLOPs for TRAIL. This supports encoding individual behavior transitions before aggregation, allowing temporal context to inform STA. Encoding before STA also permits caching the candidate-independent representations.

Longer-history evaluation. On the industrial dataset at L=10,000L=10{,}000, SC-GDN improves AUC/GAUC from 0.7839/0.7250 to 0.7849/0.7265, corresponding to gains of 0.10/0.15 percentage points. These gains highlight the importance of encoding full-history interest evolution when modeling 10K industrial behavior sequences. SC-GDN adds 5.19 ms of P99 latency and increases peak GPU memory from 3.46 to 5.28 GiB. Complete TRAIL achieves 54.81 ms P99 latency, below STCA (76.82 ms) and LONGER (68.54 ms). Thus, full-history interest encoding improves prediction accuracy while preserving the framework’s measured latency advantage over both baselines.

The encoder and encoding-order comparisons are reported in Section 4.3, Table 2; the 10K prediction and inference results are reported in Section 4.5, Table 5.

AuthorsOpenReview ↗

Response to xhDY (2/3): Baseline Comparability and 10K Efficiency

Q2. Baseline comparability and practical efficiency at longer histories.

All models share the same input features, preprocessing, and prediction MLP configuration within each dataset, use common training hyperparameter search ranges alongside architecture-specific searches, and follow the same FLOPs accounting scope. At 10K, TRAIL achieves higher AUC and GAUC than STCA and LONGER while reducing measured P99 latency by 28.7% and 20.0%, respectively, and achieving higher throughput than both baselines.

Shared inputs and prediction heads. Inputs include historical-item and candidate-item IDs and user/context features, with additional multimodal embeddings used on the industrial dataset. Common settings include embedding and hidden dimensions of 128, two attention layers and two heads where applicable, and dropout of 0.1.

Hyperparameter search. All models use the same learning-rate search {10−4,3×10−4,10−3}\{10^{-4}, 3\times10^{-4}, 10^{-3}\} and weight-decay search {0,10−6,10−5,10−4}\{0, 10^{-6}, 10^{-5}, 10^{-4}\}. Model-specific searches cover DIN’s activation-unit width, TWIN’s retrieval size and representation dimension, HSTU’s Q/K projection dimension and relative-position range, STCA’s FFN expansion ratio, and LONGER’s merge factor and retained-query count.

Measured prediction quality and inference efficiency at 10K. We compare all four variants on the same industrial dataset at L=10,000L=10{,}000:

ModelAUCGAUCP99 latency (ms)Throughput (requests/s)Peak GPU memory (GiB)
STCA0.78210.723476.82618.386.31
LONGER0.78270.723968.54751.944.56
TRAIL0.78490.726554.81899.785.28
TRAIL w/o SC-GDN0.78390.725049.62987.693.46

Measurements use the same NVIDIA H20 hardware, batch size 64, 50 candidates per request, and BF16 precision. Standard attention operations use FlashAttention, while SC-GDN uses Flash Linear Attention for chunkwise parallel computation. Model-specific history caching precomputes and reuses candidate-independent history representations, including SC-GDN’s history encoding in TRAIL.

TRAIL exceeds STCA by 0.28/0.31 percentage points in AUC/GAUC and LONGER by 0.22/0.26 percentage points, with 54.81 ms P99 latency and 5.28 GiB peak GPU memory. Its throughput reaches 899.78 requests/s, compared with 618.38 for STCA and 751.94 for LONGER. TRAIL w/o SC-GDN also outperforms both baselines on AUC and GAUC, with lower latency, higher throughput, and lower peak memory. These results demonstrate TRAIL’s effectiveness and efficiency in industrial ranking with 10K behavior histories.

Consistent FLOPs accounting. All models use the same per-sample forward accounting scope, counting one MAC as two FLOPs. We focus on the sequence modeling components and exclude input embeddings and the final prediction tower, which use common configurations across models. Normalization, masking, and pooling are also omitted consistently across all models as lower-order operations.

The shared experimental settings are described in Section 4.1, hyperparameter search ranges in Appendix B, measured 10K results and inference settings in Section 4.5, Table 5, and the detailed FLOPs derivation in Appendix A.

AuthorsOpenReview ↗

Response to xhDY (3/3): Online Experiment Details

Q3. Production control, ADVV, and online experimental details.

The production control combines self-attention and target attention over the 100 most recent behaviors. The experimental group replaces this model with TRAIL, with all other settings identical to those of the control group. ADVV measures advertiser value as the sum of each advertising unit’s target cost per conversion multiplied by its actual number of conversions.

Experimental setup. The experiment runs for seven days on a platform with DAU on the order of 100 million, with users randomly assigned by UserID and 10% of total traffic allocated to the experimental group. Each user remains in the same group throughout the experiment.

Online results and statistical significance. TRAIL achieves the following relative improvements over the production control:

MetricRelative liftp-value
ADVV+2.51%0.0389
Revenue+2.36%0.0326
CTR+1.93%0.0401

All three improvements have p-values below 0.05. At the platform’s current revenue scale, sustaining the observed 2.36% revenue lift would translate into an estimated annual revenue increase on the order of hundreds of millions. These gains accompany a 5% increase in normalized production serving cost. The randomized user assignment and identical settings apart from the model replacement support attributing the observed business improvements to TRAIL.

The production control, experimental setup, and ADVV definition are described in Section 4.6, with online improvements and p-values reported in Table 7.

The additional experiments demonstrate the contribution of full-history interest encoding and TRAIL’s favorable balance between prediction accuracy and computational cost at a history length of 10K, supported by positive online business results. We hope these results and clarifications address the reviewer’s questions and support a favorable reassessment of our work.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer xhDY,

Thank you again for your thoughtful feedback. We would like to briefly follow up on our rebuttal, in which we added encoder comparisons and evaluations at a sequence length of 10K. We also clarified the experimental settings and online evaluation.

We would appreciate your feedback on whether these additions address your main concerns. Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.