acceptodds
Under review as a conference paper at ICLR 2027

LoopCTR: Unlocking the Loop Scaling Power for Click-Through Rate Prediction

Abstract

Scaling Transformer-based click-through rate (CTR) models by stacking more parameters brings growing computational and storage overhead, creating a widening gap between scaling ambitions and the stringent industrial deployment constraints. We propose **LoopCTR**, which introduces a loop scaling paradigm that increases training-time computation through recursive reuse of shared model layers, decoupling computation from parameter growth. LoopCTR adopts a sandwich architecture enhanced with Hyper-Connected Residuals and Mixture-of-Experts, and employs process supervision at every loop depth to encode multi-loop benefits into the shared parameters. This enables a train-multi-loop, infer-zero-loop strategy where a single forward pass without any loop already outperforms all baselines. Experiments on three public benchmarks and one industrial dataset demonstrate state-of-the-art performance. Oracle analysis further reveals 0.02–0.04 AUC of untapped headroom, with models trained with fewer loops exhibiting higher oracle ceilings, pointing to a promising frontier for adaptive inference.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Click-Through Rate, Recommender Systems, Long Sequence Modeling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_RimKOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces LoopCTR for click-through rate prediction

The main idea is to increase training-time computation by repeatedly applying the same Transformer block, instead of stacking more layers with separate parameters. The model uses an Entry–Loop–Exit structure together with Hyper-Connected Residuals, Mixture-of-Experts, and supervision at every loop depth. This allows the authors to train with multiple loops but skip the Loop Block entirely at inference time

it evaluates the method on three public datasets and one industrial dataset Inhouse, reporting strong AUC and NE results, including at zero-loop inference. The paper also includes loop-depth experiments, ablations, comparisons with a stacked-layer variant, efficiency measurements, and an oracle analysis of adaptive loop depth.

Critical strengths
  1. The train-multi-loop, infer-zero-loop result is practically interesting. The model trained with 3 loops performs strongly even when the Loop Block is completely skipped at inference, and this pattern appears across all four datasets. If validated more rigorously, this could offer a useful way to spend additional computation during training without increasing serving latency

  2. The empirical evaluation covers several useful angles. The paper evaluates three public datasets and one industrial dataset, and includes loop-depth sweeps, component ablations, efficiency measurements, and an iso-FLOPs comparison with a stacked-layer variant. This makes the main empirical phenomenon reasonably easy to identify, even though some important controls are still missing

Critical weaknesses

I am not convinced that the zero-loop improvement is caused by loop-based reasoning. At zero-loop inference, the Loop Block is not executed at all. The deeper losses can still update the shared Entry and Exit Blocks, so the improvement may instead come from auxiliary supervision, regularization, or additional training computation. The paper lacks the most important controls, such as a compute-matched L=0 model and zero-loop results with and without process supervision

Questions for authors
  1. How many runs or random seeds were used for Table 2, and what observations were paired in the reported paired t-test?
  2. Does the efficiency eval include long-sequence compression, MoE routing, and cache construction? Were the same preprocessing and caching opportunities provided to every baseline?
  3. The L=0 model is already quite strong, while loop training adds only a relatively small gain. Without a compute-matched L=0 baseline, it is unclear whether the improvement really comes from loop scaling rather than the strong entry–exit backbone, extra training compute, or deep supervision
  4. Could the authors report zero-loop performance with and without process supervision, together with an L=0 baseline matched for total training computation?
1 comment

We sincerely appreciate your time and constructive comments. We address your concerns point by point below.


[W1 / Q3–Q4] Does the zero-loop gain come from recursive training, extra compute or supervision?

Response: The zero-loop gain is a training benefit: losses at depths 0–L update the shared Entry and Exit Blocks, which remain active when the Loop Block is skipped at inference. Section 3.5 describes the multi-depth objective and zero-loop inference.

(1) Supervision control (Added experiment). On Amazon, we compare zero-loop performance with and without process supervision under the same settings:

Training lossAUC ↑GAUC ↑NE ↓
Final depth only0.84970.85010.7044
All depths 0–30.87150.87000.6594

Multi-depth supervision improves zero-loop AUC by 0.0218.

(2) Weight-sharing control (Section 4.2 comparison). We extend the Section 4.2 StackCTR comparison to zero-loop inference under matched Amazon settings and the same depth supervision:

ModelZero-loop AUC ↑
Untied StackCTR0.8680
Shared LoopCTR0.8715

(3) Matched training FLOPs (Added experiment). For a fair comparison, we match the training compute at approximately 7.33×10¹⁵ forward/backward matrix FLOPs on Amazon.

Amazon, i=0AUC ↑GAUC ↑NE ↓
L=00.82760.82442.091
LoopCTR, L=30.87150.87000.6594

FLOPs count forward/backward matrix multiplications and attention operations.

[Q1] How many runs were used, and what observations were paired in the t-test?

Response: For the significance analysis, we use three runs per model and perform two-sided paired t-tests between LoopCTR and best baseline methods.

[Q2] Does timing include compression, routing and cache construction for all methods?

Response: The cached-scoring timer includes the Loop and Exit blocks, their routing operations, and prediction. User encoding, including long-sequence compression and user-side Entry processing, is timed separately. Candidate embeddings and Entry outputs are prepared before scoring. Thus, cached-scoring latency excludes cache construction and candidate preparation.

The cached benchmark also provides user-side preprocessing reuse for OneTrans, HSTU and MTGR, with candidate preparation outside their scoring timers. The cached stages follow each model's architecture.

This consistent timing convention supports a fair comparison of cached scoring latency between LoopCTR and these baselines.

Changes in the revised manuscript

Page and line numbers refer to the revised PDF.

[W1 / Q3–Q4] Zero-loop controls: added the loop-depth and weight-sharing comparisons Appendix E.4, p. 22, Table 10.

[Q2] Request cost: corrected Table 5 to include the user-side cache construction cost p. 19, lines 973–985.


We hope these additional experiments and analyses address your concerns and provide a basis for reconsidering your score. Thank you again for your time and constructive feedback. We would be happy to clarify any remaining questions.

Reviewer_YwA4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed LoopCRT, a loop scaling paradigm to imporve training computation via recursive reuse of shared model structures, aiming at decoupling computation from the growth of the model scale. The model supports multi-loop training and zero-loop inference. Experiments on three public datasets and one industrial dataset show improvements over existing CTR baselines.

Critical strengths

The proposed approach is useful for industrial CTR systems with strict inference latency requirements. Also, the method achieves consistent improvements across four datasets. The ablation studies and efficiency comparisons also provide useful evidence.

Critical weaknesses
  1. Several main ideas have been explored in previous studies. AlgoFormer (TMLR 2025) already proposed a similar pre-loop-post architecture with supervision across loop iterations. LARES (2025) explored recurrent depth scaling in recommendation systems, and LayerSkip (ACL 2024) studied intermediate supervision for early-exit inference. These relevant works are not discussed. The paper needs to better clarify its technical novelty beyond combining existing methods for CTR prediction.

  2. The improvements may come from additional training computation and supervision rather than the loop architecture itself. Although StackCTR provides an iso-FLOPs comparison, the paper does not include a training-compute-matched comparison for the zero-loop inference model. This makes it difficult to understand how much benefit comes specifically from recursive training.

Questions for authors
  1. What are the main technical differences between LoopCTR and AlgoFormer, especially regarding the sandwich architecture and multi-depth supervision? How does LoopCTR differ from LARES?

  2. Can you compare the zero-loop LoopCTR model against a non-loop model trained with the same training FLOPs? This would help validate the benefits of loop scaling.

3 comments

We sincerely appreciate your time and constructive comments. We address your concerns point by point below.


[W1 / Q1] How does LoopCTR differ from AlgoFormer, LARES and LayerSkip?

Response:

LoopCTR uses multi-loop training to improve a CTR predictor that can be deployed without executing the Loop Block. Its architecture and supervision are designed around this training and deployment setting.

The comparison below summarizes the methods' architectures, training objectives and inference use.

DimensionAlgoFormer [1]LARES [2]LayerSkip [3]LoopCTR
ArchitecturePre/loop/post Transformers with shared loopPre-block and shared recurrent coreDistinct Transformer layers with a shared exitEntry/shared Loop/Exit Blocks with HCR and MoE
TrainingLosses over a configurable range of loop depthsAlignment pre-training and reinforcement post-trainingIntermediate supervision and layer dropoutCTR losses at all depths
InferenceIterative computation with depth extrapolationRecurrent refinement for next-item predictionEarly exit and self-speculative decodingSkip the Loop Block and retain the Entry–Exit predictor

(1) Sandwich architecture. AlgoFormer and LoopCTR share the pre/loop/post structure. AlgoFormer uses standard Transformers to model iterative algorithms. LoopCTR's Entry Block encodes heterogeneous CTR features, the shared Loop Block combines token-dependent MoE transformations with HCR stream mixing, and the Exit Block predicts clicks. On Amazon, removing HCR or MoE lowers AUC by 0.0201 or 0.0072, respectively (Figure 3).

(2) Multi-depth supervision. AlgoFormer averages losses over a configurable range of loop depths. LoopCTR applies the click-prediction loss at every depth from 0 to L. Each loss updates the shared Entry and Exit Blocks, including the direct loss at depth 0. These blocks are retained when the entire Loop Block is skipped at inference. On Amazon, the zero-loop AUC rises from 0.8497 to 0.8715 with multi-depth supervision under the same settings. The compute-matched L=0 comparison below further examines the role of training computation.

(3) Recommendation and early exit. LARES studies recurrent refinement for sequential recommendation, with alignment pre-training and reinforcement post-training to improve iterative inference. LoopCTR instead applies CTR supervision at each depth and evaluates the Entry–Exit predictor without the loop at deployment. LayerSkip trains early exits in a stack of distinct language-model layers, whereas LoopCTR repeatedly uses a shared block during training and can bypass that block at inference.

The Training Objective describes multi-depth supervision and zero-loop inference. We have clarified the relationship to prior work in Related Work.

[W2 / Q2] Does the zero-loop gain remain against L=0 with matched training FLOPs?

Response:

At matched training FLOPs, LoopCTR (0/3) achieves 0.8715 AUC versus 0.8276 for L=0.

(1) Matched training FLOPs (Added experiment).

For a fair comparison on Amazon, we match the training compute at approximately 7.33×10¹⁵ forward/backward matrix FLOPs.

Amazon, i=0AUC ↑GAUC ↑NE ↓
L=00.82760.82442.091
LoopCTR, L=30.87150.87000.6594

FLOPs count forward/backward matrix multiplications and attention operations.

(2) Weight-sharing control (Section 4.2 comparison). We extend the Section 4.2 StackCTR comparison to zero-loop inference under matched Amazon settings and the same multi-depth supervision:

ModelZero-loop AUC ↑
Untied StackCTR0.8680
Shared LoopCTR0.8715

These comparisons characterize zero-loop performance under matched compute and under shared versus untied training.

References

[1] Gao et al. AlgoFormer: An Efficient Transformer Framework with Algorithmic Structures, TMLR 2025, Sections 2.2–2.3.

[2] Liu et al. LARES: Latent Reasoning for Sequential Recommendation, arXiv:2505.16865, 2025, Sections 2.2–2.4.

[3] Elhoushi et al. LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding, ACL 2024.

Changes in the revised manuscript

Page and line numbers refer to the revised PDF.

[W1 / Q1] Abstract: aligned the contribution summary with the training and deployment setting Abstract, p. 1.

[W1 / Q1] Related work: added the three requested works and clarified their relationship to LoopCTR §5, p. 9, lines 470–479.

[W1–W2 / Q1–Q2] Zero-loop controls: added the loop-depth and weight-sharing comparisons Appendix E.4, p. 22, Table 10.


We hope these additional experiments and analyses address your concerns and provide a basis for reconsidering your score. Thank you again for your time and constructive feedback. We would be happy to clarify any remaining questions.

Thanks for your response and more experiment results. The results are clear and partially address my novelty concern. Raise the score to 3.

Thank you for considering our responses and additional experiments, and for raising your score. We sincerely appreciate your time and constructive feedback, which have helped us clarify the paper’s contribution.

Reviewer_QLWUOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LoopCTR, a loop-scaling paradigm for CTR prediction that increases training-time computation by recursively reusing a shared-parameter Loop Block instead of stacking more layers. The architecture follows a sandwich design (Entry–Loop–Exit) augmented with Hyper-Connected Residuals (HCR), Mixture-of-Experts (MoE) in attention and FFN, and process supervision across loop depths, enabling train-multi-loop, infer-zero-loop. Experiments on three public datasets and a large in-house dataset show consistent gains over strong baselines, with notable zero-loop inference performance and an oracle analysis suggesting further headroom for adaptive inference.

Critical strengths
  • Introduces “loop scaling” for CTR, decoupling computation from parameter growth via recursive reuse of a shared Loop Block.
  • Evaluates across four datasets (including a challenging industrial dataset with both short- and long-term sequences).
  • Presents oracle-depth analysis and per-loop diagnostics that help interpret loop behavior and diminishing returns.
Critical weaknesses
  • Oracle analysis occasionally extrapolates to more inference loops than the model was trained with (L<3 but i=3), raising questions about fairness and stability of the estimated ceiling.
  • The masking scheme is called “PrefixAttn” but deviates from standard prefix-LM terminology, raising potential confusion and a lack of theoretical grounding for this specific asymmetry.
  • Ablation results contain surprising magnitudes (e.g., massive GAUC drop without HCR on Amazon) that merit deeper investigation or replication checks.
Questions for authors
  • What are the exact MoE settings (experts per layer, top-k, capacity factor, router architecture, auxiliary loss weight), and how sensitive are the results to these choices? Can you report token-level expert utilization and any routing specialization analysis?
  • The ablation “w/o HCR” on Amazon shows extremely large GAUC/NE degradations. Can you verify those numbers with multiple seeds and provide diagnostics to rule out confounds?
  • For the oracle analysis, why is i=3 used when L<3 (extrapolation beyond the trained loop depth)? Does training with L=1 or L=2 and evaluating with i>L materially bias the oracle ceiling? Please include oracle results constrained to i≤L.
  • How many learnable query tokens are used for long-term sequence compression, and how does performance/latency vary with this number? Could you add an ablation comparing learnable queries vs. simpler pooling baselines?

If the authors address the above concerns, I will consider raising my score.

9 comments

We sincerely appreciate your time and constructive comments. We address your concerns point by point below.


[Q1] What are the MoE settings, and how sensitive are the results to routing choices?

Response:

(1) MoE settings. Across all four datasets, all MoE modules use 4 experts and top-2 routing. Each router is a bias-free linear map from the current token state to expert logits. We select the top two logits and normalize their weights with softmax. Attention and FFN routers have separate parameters. The same Loop Block routers are reused across iterations.

Capacity factor: Not used. All tokens retain their top-2 expert assignments.

Auxiliary loss. The auxiliary-loss weight is 0.01 on Amazon/TaobaoAds and 0 on KuaiVideo/InHouse.

(2) Expert configurations (Existing experiments). We reproduce the Table 4 results below. k/E denotes activated/total experts.

Amazon

k/EAUC ↑GAUC ↑NE ↓
1/40.86600.86480.6678
3/40.86900.86780.6631
4/40.86700.86550.6716
2/20.87190.87130.6492
2/30.87050.86820.6576
2/50.86880.86740.6607
2/4 (selected)0.87260.87130.6560

KuaiVideo

k/EAUC ↑GAUC ↑NE ↓
1/40.74400.66160.8795
3/40.74280.66240.8810
4/40.74370.66370.8792
2/20.74410.66000.8816
2/30.74230.65930.8833
2/50.74460.66240.8759
2/4 (selected)0.74480.66380.8774

(3) Auxiliary-loss sensitivity (Added experiments). We evaluated the same grid, λ ∈ {0, 0.01, 0.1, 1}, on Amazon and KuaiVideo. Within each dataset, we vary only λ and report the L=3 model at i=3.

Amazon

Auxiliary-loss weightAUC ↑GAUC ↑NE ↓
00.86930.86760.6621
0.010.87260.87130.6560
0.10.86630.86430.6703
10.86880.86720.6620

KuaiVideo

Auxiliary-loss weightAUC ↑GAUC ↑NE ↓
00.74480.66380.8774
0.010.74420.66290.8790
0.10.74510.66370.8777
10.74500.66330.8781

Among the tested values, λ=0.01 gives the best Amazon results. On KuaiVideo, λ=0.1 gives the highest AUC, while λ=0 gives the highest GAUC and lowest NE.

(4) Token-level expert utilization (Added analysis). We report first-loop attention dispatch frequencies by token type, normalized over top-2 assignments.

Amazon

Token typeExpert 1Expert 2Expert 3Expert 4
History31.9%27.0%8.9%32.2%
User ID21.2%29.9%25.9%23.0%
Target item28.2%10.5%30.6%30.8%

KuaiVideo

Token typeExpert 1Expert 2Expert 3Expert 4
Positive history20.3%25.0%16.9%37.8%
Negative history20.3%22.1%26.7%30.9%
User ID40.5%30.6%13.8%15.2%
Target item ID36.8%12.8%27.4%22.9%
Target item features20.3%17.3%25.8%36.6%

On Amazon, expert 3 receives 30.6% of target-item assignments but only 8.9% of history assignments. On KuaiVideo, expert 1 receives 40.5% of user-ID assignments versus 20.3% of positive-history assignments. These differences indicate distinct expert preferences across token types.

[W3 / Q2] Can the large GAUC/NE degradation without HCR be verified across seeds and checked for confounds?

Response:

The HCR benefit persists across three paired seeds on AUC, GAUC and NE.

On Amazon, the evaluated w/o HCR checkpoint gives 0.8502 GAUC and 0.6937 NE, compared with 0.8713 GAUC and 0.6560 NE for Full. This corresponds to a GAUC decrease of 0.0211 and an NE increase of 0.0377.

HCR ablation across seeds. The original experiments used three seeds. For further verification, we evaluated Full and w/o HCR with three additional paired seeds. The w/o HCR model replaces HCR with standard pre-RMSNorm residual connections while retaining MoE. Each pair uses the same Amazon data, optimizer, training and evaluation procedure, and MoE and loss settings. Both models use three training and inference loops. Results are reported separately for each additional seed:

SeedModelAUC ↑GAUC ↑NE ↓
43w/o HCR0.85810.85660.6800
43Full0.87070.86920.6567
44w/o HCR0.85850.85600.6780
44Full0.87120.86980.6535
45w/o HCR0.85520.85340.6884
45Full0.86590.86600.6717

Full improves all three metrics in every pair. Averaged across the three seeds, it improves AUC by 0.0120 and GAUC by 0.0130 and reduces NE by 0.0215. These matched runs show that the advantage persists when the checked training and MoE settings are aligned, rather than depending on a single seed.

[W1 / Q3] Why evaluate i>L, and does the oracle trend hold when i≤L?

Response:

(1) Why evaluate beyond the training depth? LoopCTR encodes an inductive bias for iterative refinement through a shared update. This update can be applied recursively beyond the training depth, which can be viewed as latent reasoning in representation space. Evaluating i>L therefore tests whether further refinement improves predictions using additional computation with the same parameters. The oracle assesses the potential of choosing the inference depth for each sample.

(2) Oracle comparison on four datasets (Added analysis). Full range uses i∈{0,1,2,3}, as reported in Table 3. Restricted range uses i≤L.

Without reasoning extrapolation, the restricted oracle still exceeds the best fixed-depth prediction by 0.0056–0.0231 AUC across all 12 settings.

All scores are AUC. Best fixed selects a single depth within i≤L for the model trained at the L in each row. Both ΔAUC columns subtract this same Best fixed score from the corresponding oracle score.

DatasetLBest fixed within i≤LFull range oracleΔAUC (Full)Restricted oracle (Added)ΔAUC (Restricted)
Amazon10.86870.8885+0.01980.8743+0.0056
Amazon20.87010.8865+0.01640.8824+0.0123
Amazon30.87280.8858+0.01300.8858+0.0130
TaobaoAds10.64360.6744+0.03080.6535+0.0099
TaobaoAds20.64380.6701+0.02630.6636+0.0198
TaobaoAds30.64410.6672+0.02310.6672+0.0231
KuaiVideo10.74470.7675+0.02280.7525+0.0078
KuaiVideo20.74450.7614+0.01690.7565+0.0120
KuaiVideo30.74500.7588+0.01380.7588+0.0138
InHouse10.69850.7306+0.03210.7098+0.0113
InHouse20.70020.7201+0.01990.7149+0.0147
InHouse30.70070.7195+0.01880.7195+0.0188

The smaller-L advantage appears with the full range. Under i≤L, oracle AUC instead increases with L on all four datasets. At L=3, the two ranges coincide.

[Q4] How many queries are used, and how do they compare with pooling?

Response: We use 16 learned queries, as specified in Appendix D.1.

(1) Query count (Existing checkpoints, Added timing). With the same training configuration, the existing q=8, q=16, and q=32 checkpoints give the following zero-loop results. We added full-forward timing for each checkpoint:

QueriesAUC ↑GAUC ↑Added timing, p50 / p95 (ms)
80.69780.568547.17 / 47.73
160.70070.568751.14 / 52.24
320.69980.569256.01 / 56.95

Increasing q from 8 to 32 raises median latency from 47.17 to 56.01 ms. The selected q=16 gives the highest AUC at an intermediate latency, while q=32 gives slightly higher GAUC.

Timings include embeddings, compression, routing and the complete forward pass. We use batch 2048, one exclusive H20, FP32, TF32 off, 30 warm-up passes and 200 synchronized measurements. Input transfer is excluded and no user states are cached.

The original environment used H20-3e GPUs (approximately 140 GiB of GPU memory), whereas these follow-up measurements use H20 GPUs (approximately 96 GiB). This hardware difference may contribute to the latency differences. (We are currently unable to rent H20-3e GPUs for a hardware-matched reproduction.)

(2) Learned queries vs. pooling (Added experiment). We compared masked segment-mean pooling with learned queries, using the same 16 output tokens and matched training settings:

CompressionAUC ↑GAUC ↑NE ↓
Segment-mean pooling0.695450.563880.92414
Learned queries0.700650.568670.91873

With zero-loop inference, learned queries improve AUC by 0.0052 and GAUC by 0.0048, and reduce NE by 0.0054. With the same output token count, task-trained cross-attention provides a more effective compression than fixed segment averaging.

[W2] How does the mask differ from prefix-LM attention, and why use this asymmetry?

Response: We have renamed PrefixAttn to asymmetric attention (AsymAttn) to distinguish it from standard prefix-LM attention. Sequence tokens attend only to sequence tokens, while global tokens attend to all tokens. Interactions among global tokens are bidirectional because they represent observed CTR features rather than an autoregressively generated sequence.

This asymmetry lets global tokens aggregate behavior history while keeping sequence states independent of candidate features, supporting reuse across candidates when other sequence-side operations preserve that independence. We clarify this rationale in §3.2 and use the revised terminology consistently in Figure 1 and Appendix D.2. The mask itself is unchanged.

Changes in the revised manuscript

Page and line numbers refer to the revised PDF.

[W2] Attention terminology and rationale: renamed PrefixAttn to asymmetric attention (AsymAttn) throughout the text, Eq. (3) and Figure 1, and clarified global-token interactions and the rationale for candidate-independent sequence states §3.2, p. 4, lines 171–180, Figure 1, p. 3, caption lines 121–127 and Appendix D.2, p. 19, lines 988–990.

[W1 / Q3] Oracle: clarified label-dependent selection and added the comparison without extrapolation §4.2, p. 7, lines 361–365 and Appendix E.1, p. 21, Table 7.

[Q1] MoE: specified router and auxiliary-loss settings and added the Amazon and KuaiVideo auxiliary-loss sensitivity results Appendix B.1, p. 16, lines 842–843 and Appendix E.3, p. 22, Table 9.

[Q4] Compression comparison: added the learned-query and pooling results Appendix E.2, p. 21, Table 8.


We hope these additional experiments and analyses address your concerns and provide a basis for reconsidering your score. Thank you again for your time and constructive feedback. We would be happy to clarify any remaining questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General responses and manuscript revision summary

Dear PCs, SACs, ACs, and reviewers,

We sincerely thank the PCs, SACs, and ACs for their time and effort in coordinating the review process, and all reviewers for their careful reading and constructive comments on our manuscript. We know that we are now approaching the end of the author-reviewer discussion. During the rebuttal period, we have been focusing on these beneficial suggestions from the reviewers and doing our best to add several experiments and revise our manuscript. We believe our current carefully revised manuscript can address the reviewers' concerns.

As reviewers highlighted, we believe our paper tackles an important and relevant problem for latency-sensitive CTR prediction (Reviewer YwA4, Reviewer RimK), introduces a practically interesting train-multi-loop, infer-zero-loop strategy (Reviewer RimK), and achieves consistent improvements across four datasets (Reviewer QLWU, Reviewer YwA4). In addition, all the reviewers recognized the breadth of our empirical evaluation, including loop-depth analyses, ablation studies, and efficiency comparisons.

After considering our additional experiments and clarifications, all three reviewers (QLWU, YwA4, and RimK) have raised their ratings to 3. We sincerely appreciate their positive reassessment of our work's contribution and quality. We also appreciate that the additional experiments provided new evidence of synergy, with HCR more effective in the loop-based configuration (Reviewer QLWU). The results were clear and partially addressed the novelty concern (Reviewer YwA4), while the zero-loop StackCTR comparison was helpful and the supervision ablation addressed part of the concern (Reviewer RimK).

Moreover, we thank the reviewers for their suggestions on clarifying the relationship to prior work (Reviewer YwA4), strengthening the zero-loop controls (Reviewer RimK), and verifying the HCR ablation and oracle analysis (Reviewer QLWU). In response to these comments, we have carefully revised and enhanced our manuscript with the following important changes and added experiments, together with clarifications in our responses:

  • [Reviewer YwA4, Reviewer QLWU] We add a discussion of the relationship and differences between LoopCTR and AlgoFormer, LARES, and LayerSkip (Section 5), and clarify the role of HCR in loop-based CTR modeling in our follow-up responses.

  • [Reviewer RimK, Reviewer YwA4, Reviewer QLWU] To investigate the sources of zero-loop gains, we add comparisons with final-depth supervision, untied StackCTR, and a training-FLOPs-matched L=0L=0 model (Appendix E.4, Table 10). We also explain the early-stopping and fixed-compute checkpoint protocols in our follow-up responses.

  • [Reviewer QLWU] We add oracle results restricted to i≤Li\leq L across all four datasets and clarify their interpretation as potential adaptive-inference headroom (Appendix E.1, Table 7).

  • [Reviewer QLWU] We specify the MoE settings, add auxiliary-loss sensitivity experiments, and report token-level expert utilization (Appendices B.1 and E.3, Table 9, and reviewer response).

  • [Reviewer QLWU] We compare learned queries with segment-mean pooling (Appendix E.2, Table 8) and report the accuracy–latency trade-off for different query counts in our response.

  • [Reviewer QLWU] We further clarify the attention mask's rationale and rename PrefixAttn to asymmetric attention (AsymAttn) (Section 3.2, Figure 1, and Appendix D.2).

  • [Reviewer RimK] We clarify the timing scope, cache construction cost, and caching opportunities provided to the baselines (Appendix D.2, Table 5, and reviewer response).

These updates are temporarily highlighted in "red" for facilitating checking.

We hope our responses and revisions have addressed the reviewers' concerns.

Thanks again,

Submission12056 Authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.