- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Maglev restructures recommendation models into hardware-local sequential stages. Soft Indexers learn bounded feature projections; Hard Indexers convert learned affinities into feature assignments that respect storage budgets and coverage. Induct carries a gated residual digest between stages, and Rail schedules stage-local embedding lookups to fill pipeline warm-up slots while retaining exact synchronous gradient clipping. The evaluation combines a 28B-sample proprietary study of Wukong and DHEN, five additional backbones on three public datasets, systems measurements, and reported production A/B tests. Table 2 reports 116 of 116 wins against comparable same-backbone frontiers and a median 0.15 percentage-point RLL margin. Public Soft-Indexer experiments use 12 paired seeds per setting, with positive mean gains in all 15 settings. The main contribution is a practical model and systems design that improves the observed quality and compute trade-off.
Critical strengths
- The evaluation supports a useful architectural contribution beyond one backbone or benchmark. Sections 4.2-4.3 compare quality under explicit compute and sparse-capacity budgets. Table 1 provides paired confidence intervals across five backbones and three datasets; Appendix A.12.2 additionally tests Hard Indexers, including an unfavorable FiGNN result on Criteo. Appendix A.11 checks non-interpolated frontiers, additional converged runs, and nearby budgets. These checks materially strengthen the argument that the gains are not simply an artifact of one selected comparison.
- The model and systems components address a coherent deployment constraint. Learned feature placement makes lookups local, Induct restores useful cross-stage conditioning, and Rail uses the resulting lookup independence to improve scheduling. The analysis explicitly limits zero bubbles to a specified latency model and excludes the common optimizer tail, while measured throughput includes that tail. Appendix A.4 reports 6.3%-11.7% schedule gains across 12 configurations, and A.12.5 separates the schedule contribution from sharding and communication overlap. This separation makes the practical contribution more credible than a single aggregate speedup.
Critical weaknesses
- The strongest scaling conclusions remain specific to a partially reproducible industrial campaign. Each grid configuration has one seed. The repeatability check in A.10.1 compares four-layer Wukong with eight-stage Maglev-DHEN at 38.6% different FLOPs on later data; the authors correctly state that it does not estimate uncertainty for the frontier. Thus, it cannot resolve the smallest ablation gains or uncertainty in fitted slope ratios. The public experiments offer useful complementary evidence but do not reproduce the industrial scaling regime. The appendix also omits consequential settings: industrial train/validation/test construction and temporal leakage controls, numerical optimizer and learning-rate settings, completion criteria, and some co-tuned widths. Public configurations list depth and Indexer width without the full model and training configuration. These omissions limit independent assessment of the scaling claims. A representative repeated, near-matched industrial contrast and complete public configurations would materially strengthen the evidence; repeating the entire industrial grid is not necessary.
- The representation analysis supports a coefficient approximation bound, but some interpretation goes beyond what it establishes. In Section 3.2, U = B-dagger W is called the optimal linear readout. Appendix A.9 shows that it minimizes ||W-BU||_F, whereas the stated output is ZBU; it does not generally minimize ||Z(W-BU)||_F. For example, with Z = [[2,1],[1,2]], B = [1,0]^T and W = [0,1]^T, the proposed U is 0, but prediction least squares gives U = 0.8. This does not invalidate the displayed upper bound for the chosen U, but the optimality claim should be qualified. Similarly, the spectral tail is a lower bound on coefficient residual, not automatically a lower bound on prediction error. A fixed width O also leaves the orientation of the learned subspace relevant. The Lipschitz statement in A.9 appropriately disclaims a full prediction guarantee, but an explicit mapping from the deployed nonlinear digest to the surrogate would be needed to justify stronger claims of recovered interactions.
Questions for authors
- Clarification of reproducibility: Please provide the complete public experiment configurations, including embedding dimensions, all backbone widths, Induct width, optimizer, learning rates, regularization, stopping rule and tuning search. For the industrial study, what are the temporal splits, history cutoffs and completion criteria? How are leakage from future events and overlap between training and evaluation controlled?
- Evidence that could affect the scaling assessment: Can you provide paired repetitions for one or two same-backbone, approximately matched-FLOP industrial comparisons, including a larger sparse-capacity setting? Please distinguish uncertainty of those fixed contrasts from the descriptive slopes fitted to the selected grid. This would help assess the smaller effects without requiring another full campaign.
- Clarification of the theory: Is U = B-dagger W intended to optimize the coefficient residual only? What conditions would make it optimal for the Z-weighted prediction objective, and how is B constructed from the actual gated stages? Please also separate the coefficient spectral-tail statement from a prediction-error floor.
- Clarification of implementation: A.7 distributes ordered sequence chunks across participating stages, while Section 3.1 states that each industrial feature field is assigned exactly once. What unit is partitioned for sequence fields, how do chunks contribute to the storage budgets, and how are stage order and unequal sequence lookup costs chosen? Does Listing 1's update of the digest before the current backbone block match the exact implementation described in Section 3.2?
We thank the reviewer again for recognizing our work. We hope the rebuttal submitted earlier was to your satisfaction and you would consider raising our score as you kindly offered. We're happy to provide any further clarifications if needed.