acceptodds
Withdrawn from ICLR 2027

Train on the States You Deploy: Closed-Loop Hard Routing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A 31M learned sparse-attention model with native perplexity reaches perplexity when deployed with the hard routes it was trained to produce. Soft-only training omits the hard deployment-state distribution; because deployed routes are generated layer by layer, that distribution is closed-loop. We introduce *Closed-Loop Hard Routing* (CLHR), which keeps a differentiable soft path for router learning while training the backbone on the hard trajectory induced by its own decisions. In an exact reproduction of a public 31M-parameter protocol, CLHR reduces closed-loop deployment excess from to nats across three seeds, changing the hard/native perplexity ratio from to . With the configuration frozen, the reduction transfers to 300M models on WikiText-103 () and FineWeb-Edu (); on LAMBADA, standard hardening reduces accuracy from to , whereas CLHR changes it from to . A matched causal ladder shows that hard exposure, open-loop replay, and shuffled routing do not suffice. Across the tested settings, successful soft-to-hard deployment requires task supervision on useful, self-induced hard states.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Sparse Attention, Learned Routing, Conditional Computation, Training-Inference Mismatch, Mixture of Experts, Routing Exposure Bias, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_9g2hOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper identifies that learned sparse attention is trained with differentiable soft gates but deployed with discrete top-k routes. Because deployed routes are generated layer by layer (each hard decision changes the hidden state that feeds every later router), the backbone has never seen the state trajectory it encounters at deployment.

The authors propose Closed-Loop Hard Routing. Each step runs two paths. The soft path is the ordinary gated-attention model and is the only path that trains the router. The hard path selects detached top-k masks layer by layer from its own current hard state, so the backbone is trained on its own self-induced deployment trajectory.

With the configuration frozen, the reduction transfers to 300M models on WikiText-103 (0.796 → 0.049) and FineWeb-Edu (0.513 → 0.027); on LAMBADA, standard hardening reduces accuracy from 13.2% to 3.7%, whereas CLHR changes it from 13.9% to 12.9%.

Critical strengths
  • Causal identification of the failure mode. The six-condition ladder (standard, soft replay, historical, open-loop hard, shuffled closed-loop, CLHR) isolates which ingredient matters. This is the paper’s core contribution. It turns “soft-to-hard mismatch” from an intuition into a mechanistically specific claim.
  • Careful measurement and transparent reporting. The paper defines closed-loop versus open-loop evaluation, uses an independent evaluator with mask-cardinality tests, reports per-seed values, and discloses a bug (the evaluator mask bug and the MoBA top-k=full-block configuration bug).
Critical weaknesses
  • The paper is poorly presented and hard to follow. The introduction and background are not provided sufficiently
  • CLHR trains on the backbone’s own hard trajectory with a soft loss. This is close to SSA-style dual-stream training, and the paper’s own adapted SSA reaches essentially the same excess (0.0033 vs 0.0035). CLHR’s advantage is only about 0.042 nats in absolute NLL.
  • Full CLHR costs about 1.96x standard training, and the benchmark is dense-emulated masking with no sparse kernel. The “near-lossless deployment” therefore has no demonstrated end-to-end quality-speed result.
  • The sparsity and scale regime is narrow: 31M and 300M, sequence length 512, with the 300M model trained on WikiText-103 repeated to 4B tokens.
Questions for authors
  • For the native-hard comparison, was the same tuning effort (learning rate, warmup, and hard-routing-specific tricks) given to every baseline?
  • For SSA-style, how was the alignment weight chosen, and is the adapted version faithful enough to be called independent confirmation? Was removing alignment tested at 300M?
Reviewer_RmFgOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces ClosedLoop Hard Routing (CLHR), which keeps a differentiable soft path for router learning while training the backbone on the hard trajectory induced by its own decisions.

Critical strengths

The paper is poorly organized, with its research problem, proposed methods, and contributions presented unclearly, making it difficult for the reviewer to identify its strengths.

Critical weaknesses
  1. The paper writing is very poor: Section 1 does not mention background and problem then jump in “ CLHR retains the jointly learned router but covers its complete”, what is CLHR here?
  2. Section 1 shows many results such as “31M WikiText-103 : 1.782 → 0.0035”, what does the results mean? They should be introduced as an experiment section.
  3. The paper sticks to the 31M learned sparse-attention model, but why must the model 31M?
  4. Code and paper show consistent names: "Train Soft, Roll Out Hard: Near-Lossless Deployment of Learned Sparse Attention"
  5. Section 8 - Related Work does not adequately discuss the most relevant prior studies or clearly position the paper within the existing literature.
  6. What are the main baselines of the method? And what are their results in the experiment section?
  7. What are the key novelty and impact of the proposed method?
  8. The experiment scale is very limited (300M model).
  9. There are no results of trained models on downstream tasks.
  10. Numbers writing are very inconsistent: “The 324,803,584-parameter FineWeb-Edu test processes 999,948,288” or “from 1.782 to 0.00346 (individual values 0.00375, 0.00299, and 0.00364)”
Questions for authors

Please refer to the above weakness.

Reviewer_uHtLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Closed-Loop Hard Routing (CLHR) to reduce the performance degradation caused by switching learned sparse attention from soft training to hard Top-K deployment. CLHR jointly trains soft and hard trajectories, with each hard route generated from its own current hidden state. The router learns through the soft path, while the backbone receives task supervision from both paths. Experiments on 31M–300M models trained on 0.4–4B tokens demonstrate improved performance.

Critical strengths

None

Critical weaknesses
  • The novelty over existing dual-stream training appears limited. Section 5 shows that the SSA-style baseline already achieves comparable deployment excess, and removing its alignment loss has no measurable effect. This leaves CLHR’s contribution primarily in how frequently and consistently hard-path supervision is applied, rather than establishing a fundamentally new training principle.

  • The experimental scale is limited. The main positive results stop at 300M parameters and sequence length 512, while the 4B-token WikiText experiment repeatedly uses the same corpus (Section 3). Training uses dense-emulated masked attention, without demonstrating an end-to-end quality–speed benefit (Appendices I–J). Moreover, native-hard training is competitive in the NSA-style setting, and the frozen-Qwen adaptation provides no reliable benefit (Table 2).

Questions for authors
  • Does CLHR outperform SSA-style training without alignment under matched compute budgets? As described in Section 5, CLHR runs two complete forward passes, whereas SSA-style runs only one.
  • How does CLHR compare with thoroughly tuned native-hard baselines at 300M scale? The learning-rate sweep in Appendix G.1 covers only the 31M model.
  • Can CLHR demonstrate joint quality–speed benefits at longer contexts using actual sparse kernels? The main positive results use length 512, while the long-context timing results lack corresponding model-quality evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.