- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes a method called Dense Intermediate Consistency for Endpoints (DICE). It is essentially a weighted deep supervision method, where the final readout is applied to every hidden state and those intermediate predictions are used in the loss function as well (with a particular weighting schedule). The authors provide evaluations on prefix sums, maze solving, and bAbi question answering. The authors also provide some mechanistic analysis studying the gradients in endpoint supervision and DICE.
Critical strengths
- DICE shows improvement over the chosen baselines on the datasets.
- Section 4.5 and 4.6 provide some mechanistic analysis giving intuition on why the method works better than endpoint supervision.
Critical weaknesses
1. Minimal experimental suite. The current evaluations on prefix sums, maze solving, and question answering are limited. I would appreciate a much larger experimental suite on more datasets and tasks.
Concrete suggestions:
- a) Since the authors cite TRM, which evaluates on ARC-AGI, further evaluations on ARC-AGI would strengthen this work.
- b) The authors can perform evaluations in a language modeling setup, similar to Saunshi et al. 2025 (Reasoning with Latent Thoughts...). In that work, the authors pretrain on language modeling with the Pile dataset and evaluate on Closed book QA, open book QA, math word problems, etc. (See Table 3). There are a number of tasks in each setting in that work.
I think the experimental setup is too minimal as it stands.
2) Lacking novelty. The authors mention that in prior work, "a loss is computed only at the final iterate, and gradients are propagated back through the entire chain of updates". However, this is not entirely true. The authors cite the work of Bansal et al. 2022 ("End-to-end algorithm synthesis with recurrent networks: Logical extrapolation without overthinking"). That work proposes a "progressive loss", which is like a one-state deep supervision instead of an every-state supervision (this work). The motivations for both are very similar as well (Bansal et al. 2022 mentions "incremental progress", which is similar to the "unconstrained intermediate iterations" problem in this work). I could not find where in the paper the authors mention this similarity; if the authors could point this out, I would appreciate it because I may have missed it.
Some works to distinguish from, each of which has similar deep supervision losses:
- a) "End-to-end algorithm synthesis with recurrent networks: Logical extrapolation without overthinking" Bansal et al. 2022 (see above)
- b) "LOOPED TRANSFORMERS ARE BETTER AT LEARNING LEARNING ALGORITHMS" by Yang et al. 2024.
- c) "Scaling Latent Reasoning via Looped Language Models" by Zhu et al. 2025.
Questions for authors
As I understand it, DICE is a major contribution of this work; however, I'm having a hard time understanding what is novel about DICE. Can the authors explain what makes DICE novel compared to prior work? If DICE itself is not the contribution, can the authors clarify what the main contribution of this work is?
3 comments
I thank the authors for including the experiments I requested. I have updated my score to 2 to reflect this. My concerns about novelty remain. As the authors mention, supervising intermediate iterates is already established (Bansal et al., 2022, Yang et al., 2024, Zhu et al., 2025, TRM). By the rebuttal’s own comparison, DICE differs from these mainly in which iterates it supervises and how it weights them. If the primary contribution is a controlled study of how intermediate supervision is implemented, these prior losses should be compared with tuned baselines across all tasks in the main paper. Currently, this comparison is too minimal, so I cannot recommend acceptance.
Thank you for raising your score. We agree that, given the prior losses, the comparison must be broader. Below we extend it to another setting, and we will include the comparison against Bansal et al., Yang et al., and Zhu et al. with tuned baselines across all tasks in the main paper.
Table 1: Universal Transformer on bAbI (10k). Test accuracy (%) averaged over the 20 tasks, ± std over 5 seeds.
| Method | Mean acc | Tasks 1–5 | # Tasks below 95% accuracy |
|---|---|---|---|
| Endpoint | 86.57±1.35 | 90.02 | 7 |
| Progressive loss α=1 (Bansal) | 74.92±3.39 | 76.72 | 13 |
| Window loss T=3 (Yang 2024) | 76.23±1.98 | 81.91 | 13 |
| Exit-gate loss β=0.05 (Zhu 2025) | 86.33±1.64 | 85.16 | 7 |
| DICE exponential w_k | 87.97±1.22 | 92.94 | 4 |
DICE has the best mean over 20 tasks and tasks 1–5 accuracy, and has smallest number of tasks the perform less than 95% accuracy.
Thank you for your careful review and the concrete suggestions on experiments and related work.
W1a — ARC-AGI-1 with TRM (10k epochs, official hyper-parameters, EMA)
ARC-AGI-1 evaluation set (400 puzzles), %. pass@k from the official voting evaluator.
This is a limited-budget ARC result, at 10% of the official training schedule. It establishes promising applicability, instead of performance at convergence.
W1b — Language-model setting
We applied DICE to a 3.5B-parameter recurrent-depth language model (Huginn), continued-pretrained on wikitext-103 at K=8 recurrent steps, 3 seeds, with a single uniform weight schedule shared by every method that requires intermediate supervision. Evaluation uses a fixed seed for the latent initialization, so every model is scored on identical inputs and initial states; the metric is top-1 next-token accuracy when forced to emit after 1 recurrent step.
Since DICE is trained to decode at every step while the baselines are not, a step-1 number alone cannot show whether standard LM quality improves. In domain, at every depth (3 seeds, perplexity / top-1):
Off-distribution and downstream. We evaluated at every depth k=1…8 on six held-out corpora (LAMBADA, C4, OpenWebText, arXiv, PubMedQA, GitHub) and zero-shot on HellaSwag/PIQA (3 seeds). At k=1 DICE lowers mean out-of-domain perplexity by 4.54±0.05 (77.83→73.29), leading on 4–5 of 6 corpora in every seed, and gains +3.2±0.3 pp on HellaSwag and +2.1±0.9 pp on PIQA. GRPO in particular degrades sharply off-distribution (49.0 vs endpoint's 19.5 perplexity at k=8), consistent with our Related Work argument that RL post-training must undo what the base model already learned.
In conclusion, DICE substantially improves the early-exit quality and compute–quality trade-off of looped language models, both in-domain and out-of-domain, while approximately preserving full-depth performance in-domain. This suggests DICE is particularly useful when variable inference depth or early exiting is important.
W2 — Progressive loss (Bansal), looped-TF loss (Yang), LoopLM loss (Zhu) vs DICE
See the 7MKQ Q1 table and the 7vrs design-space table.
W2 / Q — Novelty and main contribution
We will add the comparison to Bansal et al., Yang et al., and Zhu et al. in Related Work. The losses differ as follows:
Our main contribution is not intermediate supervision itself, but a controlled study of how it is implemented: which supervision designs (which states are supervised, how they are weighted, whether gradients flow across steps, shared vs. separate readout) are effective for optimization and test-time extrapolation, why they help, and when the benefit persists across architectures (recurrent nets, recursive/Universal Transformers, TRM, a 3.5B looped LM). For example, on 512-bit prefix sums under one network, uniform DICE reaches 96.2% versus 30.3% for progressive loss and 30.9% for exit-gate loss, so intermediate supervision alone does not guarantee extrapolation.
The detailed design-space table and the language-model ablation are in our response to Reviewer 7vrs.