- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TRAIL, an efficient framework for long-sequence CTR prediction. It combines Gated DeltaNet-based interest evolution, segmented target attention, and global target-aware aggregation to capture user interests from long behavior histories. Experiments on KuaiRand-1K and an industrial dataset show improvements in AUC and GAUC with relatively low computational cost. The reported online A/B test also shows positive business gains. I find the practical motivation reasonable, though the novelty of the proposed architecture is less clear.
Critical strengths
-
The paper addresses an important problem in industrial recommendation. Modeling long user behavior sequences under strict serving constraints is practically meaningful.
-
The experimental results are encouraging. TRAIL achieves competitive AUC and GAUC while substantially reducing computation compared with full attention.
-
I appreciate the online A/B evaluation. The reported 2.51% improvement in advertiser value and 2.36% revenue gain provide useful evidence beyond offline benchmarks.
Critical weaknesses
-
Novelty is somewhat limited. Interest evolution and target-aware attention have been explored in DIEN, STCA, and LONGER. TRAIL seems more like a combination of existing techniques.
-
The individual contributions of the modules are not fully convincing. Some ablation gains are quite small, and I would like to see stronger architectural controls.
-
Efficiency comparisons could be more complete. FLOPs are reported, but actual serving latency and memory usage would be more informative.
-
The online gains are promising, although the seven-day A/B test lacks statistical significance information. This makes the business impact somewhat difficult to judge.
Questions for authors
-
What is the key difference between TRAIL and a simpler combination of Gated DeltaNet-2 and STCA under the same computational budget?
-
Could the authors clarify whether the smaller ablation improvements are statistically significant across multiple seeds?
-
How does TRAIL compare with LONGER and STCA in actual inference latency, memory consumption, and throughput?
-
Could the authors provide confidence intervals or significance tests for the online A/B improvements?
6 comments
Response to rLtZ (2/2): Statistical Significance and Inference Efficiency
Q2. Statistical significance of ablation gains across random seeds.
The ablation gains are consistent across all five random seeds, and the paired comparisons achieve for both AUC and GAUC, including variants with smaller average differences.
We evaluate complete TRAIL and each variant using the same five seeds (42, 3407, 1234, 5678, and 2026) and perform a two-sided paired t-test across seeds. The resulting p-values are:
| Variant compared with TRAIL | AUC p-value | GAUC p-value |
|---|---|---|
| STA-Mean | 0.0000257 | 0.00000930 |
| w/o STA | 0.000364 | 0.000379 |
| w/o SC-GDN | 0.0000455 | 0.000689 |
| GSTA-1L | 0.00541 | 0.000689 |
| w/o MIF | 0.000799 | 0.00000834 |
| STA first | 0.000689 | 0.000394 |
| Conv () | 0.000364 | 0.000220 |
| Conv () | 0.000864 | 0.0000145 |
| minGRU | 0.000689 | 0.000348 |
These results support the consistency and statistical significance of the improvements. The comparisons of encoding order, encoder choice, and target participation in local aggregation provide complementary evidence for the individual architectural decisions.
The ablation and architectural comparison results are reported in Section 4.3, Table 2.
Q3. Inference latency, throughput, and memory consumption.
At 10K, TRAIL achieves higher AUC and GAUC than STCA and LONGER while reducing measured P99 latency by 28.7% and 20.0%, respectively. Its throughput reaches 899.78 requests/s, exceeding both baselines, with 5.28 GiB peak GPU memory.
The comparison on the same industrial dataset at gives:
| Model | AUC | GAUC | P99 latency (ms) | Throughput (requests/s) | Peak GPU memory (GiB) |
|---|---|---|---|---|---|
| STCA | 0.7821 | 0.7234 | 76.82 | 618.38 | 6.31 |
| LONGER | 0.7827 | 0.7239 | 68.54 | 751.94 | 4.56 |
| TRAIL | 0.7849 | 0.7265 | 54.81 | 899.78 | 5.28 |
| TRAIL w/o SC-GDN | 0.7839 | 0.7250 | 49.62 | 987.69 | 3.46 |
Measurements use the same NVIDIA H20 hardware, batch size 64, 50 candidates per request, and BF16 precision. Standard attention operations use FlashAttention, while SC-GDN uses Flash Linear Attention for chunkwise parallel computation. Model-specific history caching precomputes and reuses candidate-independent representations, including SC-GDN’s history encoding in TRAIL.
TRAIL achieves 0.7849 AUC / 0.7265 GAUC at 54.81 ms P99 latency, with a throughput of 899.78 requests/s, compared with 618.38 for STCA and 751.94 for LONGER. TRAIL w/o SC-GDN yields slightly lower AUC and GAUC than complete TRAIL, but still outperforms both baselines with lower latency, higher throughput, and lower peak memory. These results demonstrate TRAIL’s effectiveness and efficiency in industrial ranking with 10K behavior histories.
The inference settings and measured results are reported in Section 4.5, Table 5.
Q4. Statistical significance of the online A/B improvements.
The online improvements in ADVV, revenue, and CTR are statistically significant at the 0.05 level, with p-values of 0.0389, 0.0326, and 0.0401, respectively.
| Metric | Relative lift | p-value |
|---|---|---|
| ADVV | +2.51% | 0.0389 |
| Revenue | +2.36% | 0.0326 |
| CTR | +1.93% | 0.0401 |
The experiment runs for seven days on a platform serving on the order of 100 million daily active users. Users are randomly assigned by UserID, and 10% of total traffic is allocated to the experimental group. The experimental group replaces the production control model with TRAIL, with all other settings identical to those of the control group.
At the platform’s current revenue scale, sustaining the observed 2.36% revenue lift would translate into an estimated annual revenue increase on the order of hundreds of millions. The randomized assignment and identical settings apart from the model replacement support attributing the observed business improvements to TRAIL.
The online experimental setup is described in Section 4.6, with relative improvements and p-values reported in Table 7.
The architectural comparisons clarify how target participation in local aggregation distinguishes TRAIL from directly combining recurrent encoding with stacked target attention. The paired experiments support the significance of the ablation gains, while the measured inference results and significant online improvements demonstrate practical value. We hope these additions address the reviewer’s questions about novelty, component contributions, efficiency, and statistical significance, and support a favorable reassessment of our work.
Follow-up on Our Rebuttal
Dear Reviewer rLtZ,
Thank you again for your feedback. We would like to briefly follow up on our rebuttal, in which we clarified the architectural differences from combining GDN2 with STCA and provided controlled comparisons examining target participation in local aggregation. We also addressed the statistical significance of the ablation results across five random seeds.
We would appreciate your feedback on whether these results and clarifications address your main concerns. Thank you for your time and consideration.
Response to rLtZ (1/2): Architectural Comparisons
Thank you for recognizing TRAIL’s practical relevance. We respectfully disagree with the characterization of TRAIL as a straightforward combination of existing techniques. Our contribution is a hierarchical architecture that encodes behavior transitions before compression, uses the target to guide local aggregation, and performs global interaction over the compressed representations. This design combines behavior-level temporal modeling with target-aware compression while controlling the cost of repeated global interaction, improving prediction accuracy and inference efficiency under industrial serving constraints. The revised manuscript adds architectural comparisons, measured inference results at 10K, and statistical significance results for the ablation and online gains. We address each question below.
Q1. Architectural difference from combining Gated DeltaNet-2 with STCA.
TRAIL introduces target-aware local aggregation between behavior-level interest encoding and stacked global attention. A direct combination of GDN2 and STCA applies global attention to all L encoded behaviors. TRAIL instead uses the target to construct a representation within each temporal window, then performs global interaction over the resulting L/w representations. This design allows temporal context and target relevance to jointly inform local aggregation while reducing the cost of repeated global interaction under industrial serving constraints.
Architectural distinction from prior work. We would like to clarify that TRAIL is not merely a straightforward combination of existing techniques. DIEN introduces target relevance into its interest-evolution layer. TRAIL first encodes the full history independently of the target, allowing these representations to be cached and reused during ranking. STA subsequently uses the target as the query to weight and aggregate temporally enriched behaviors within local windows.
LONGER uses local InnerTransformer interactions for token merging. STA uses the target to weight and aggregate behaviors already enriched with temporal context, producing a target-specific representation for each segment. GSTA then performs repeated global aggregation over these segment representations.
Controlled architectural comparisons. We evaluate the following variants:
The w/o STA variant applies stacked global attention directly to the full encoded history, keeping the encoder and downstream configurations unchanged. TRAIL improves AUC/GAUC by 0.05/0.07 percentage points while reducing computation from 251.5M to 199.3M FLOPs, a 20.8% reduction. This comparison demonstrates improved prediction quality with lower computational cost.
For the requested comparison at matched computational budgets, we additionally evaluate SC-GDN followed by two layers of target cross-attention over all 1,000 behaviors. We keep L=1000, d=128, MIF, input features, and the prediction head unchanged, and set SC-GDN’s state dimensions to dk′=16 and dv′=32, compared with 16/64 in TRAIL. This yields 199.6M FLOPs, closely matching TRAIL’s 199.3M, and achieves 0.8205 AUC / 0.7380 GAUC. TRAIL therefore improves AUC/GAUC by 0.07/0.10 percentage points at essentially the same computational budget, supporting its allocation of computation to interest encoding and target-aware local aggregation before global interaction.
The STA-Mean variant preserves the temporal windows, compressed sequence length, and segment-level FFN, replacing target attention with mean pooling within each segment. TRAIL improves AUC/GAUC by 0.12/0.20 percentage points, supporting target-aware construction of segment representations at the same compression ratio.
The w/o STA and STA-Mean comparisons are reported in Section 4.3, Table 2.