acceptodds
Under review as a conference paper at ICLR 2027

COARSE-GRAINING RECURSIVE COMPUTATION: CONTINUATION-RELATIVE INTERFACES AND FAILURE MODES

Abstract

A learned substitute can replace a run of blocks or iterations in a deep or recursive network at lower cost. Replacement methods fit the substitute to reproduce the output of what it replaces and accept it on that fidelity. Fidelity, however, is measured at the replacement boundary, and usefulness is decided after it, by the continuation: the computation that runs on the substitute’s output, and whether it still solves the task at lower total cost. We give an evaluation contract that judges a replacement by its continuation. The contract separates boundary-state agreement, path statistics, executed computation, and task loss, and states what agreement at each level certifies about the next; it decomposes the continuation gap exactly into a fixed-action term and a reallocation term; and it requires the control that replacement methods omit, the reduced network trained from scratch on the task. Under the contract, fidelity is neither necessary nor sufficient, and three settings show both directions. On GPT-2, a substitute trained through the continuation with no fidelity target beats a published block-replacement method’s fidelity optimum at all five boundaries tested in distribution; under distribution shift the fidelity optimum keeps the lower loss. On a pretrained recursive model, the continuation-trained substitute is 0.36–0.37 nats below the un-adapted model in distribution at 24% fewer recursion MACs, with up to thirty times the fidelity optimum’s boundary error. On an eight-expert prototype over six synthetic data families, a from-scratch control beats every arm that inherits the trained weights on all 60 family, model, and split combinations, and anchoring expert occupancy to the original path cuts the distance from 0.78 to 0.26 without consistently lowering error. A replacement claim thereby becomes a question its continuation can answer: what it still achieves, at what cost, against which control.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
conditional computation, recursive computation, coarse-graining, model compression, continuation-aware learning, quality–cost trade-offs

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_yi8iOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes evaluation rules for the setting where a segment of computation in a deep or recursive network is replaced by a cheaper substitute. It argues that reporting how similar the substitute is to the original computation is not enough. Such experiments should also report end-to-end downstream task performance and OOD performance. On GPT-2, LoopFormer, and a synthetic prototype, the paper shows that similarity and task performance do not move in the same direction. The paper itself is not a methods paper and does not propose a new compression method.

Critical strengths

The paper puts a lot of effort into spelling out the boundaries and limitations of its claims, and this kind of candor deserves credit.

Critical weaknesses
  1. Fidelity fitting plus downstream benchmark evaluation is already what ReplaceMe and LLM-Streamline do, and using the reduced network trained from scratch as a control is also existing practice. What the paper actually does is closer to consolidating these evaluation practices.
  2. The writing puts a heavy burden on the reader. There are very few figures, the theoretical part feels disconnected from the experiments, and the paper introduces many self-defined terms that are used heavily before they are ever defined.
Questions for authors

Proposition 1 lists six conditions in the main text, but four of them are marked as vacuous, not applicable, or untested. After reading the whole paper, I still don't understand what role these conditions actually play in this work.

Reviewer_xZLvOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to evaluate a cheap substitute for several layers or recursive steps. It separates agreement with the replaced computation from downstream task quality and deployment cost, and examines these distinctions using a synthetic expert model, GPT-2, and LoopFormer.

Critical strengths
  • The paper studies the replacement of multiple computation steps with a cheaper module and usefully distinguishes replacement fidelity from downstream task quality and computational cost.

  • The GPT-2 and LoopFormer experiments show that matching intermediate states more closely does not always lead to better task performance.

Critical weaknesses
  • My main concern is the incremental contribution. As the paper acknowledges, task-based module replacement, the separation between fidelity and task performance, and training the reduced network from scratch all have precedents. I found it difficult to identify what the proposed evaluation contract establishes beyond bringing these existing ideas together. The loss decomposition is useful diagnostically, but does not seem to resolve this concern.

  • The reported improvement over the original LoopFormer is measured against an unadapted model. Without a comparable adaptation baseline on the original model, it is unclear how much of the gain comes from replacement and how much comes from fitting the target domain. The reversed ordering under distribution shift makes this distinction particularly relevant.

Questions for authors
  • Would the LoopFormer gains persist if the original model were adapted on the same data under a comparable training budget?
Reviewer_pi3yOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that learned replacements ("macros") for layers or recursive computations should be evaluated by the downstream continuation they enable ("fidelity"), rather than how closely they reproduce the intermediate state they replace.

To measure that, it proposes a continuation-relative evaluation contract that jointly considers i) downstream quality, ii) executed computation, iii) deployment cost and iv) same-graph direct-training controls.

The evaluation is based on a synthetic routed-expert prototype, GPT-2 block replacement and LoopFormer recursion and attempts to answer whether continuation-optimized replacements can outperform fidelity-optimized ones, under matched quality/cost criteria.

Critical strengths
  • In the realm of dynamic models and adaptive computation (MoEs, looped transformers, etc.), the topic of how downstream semantics and performance are preserved under compressed intermediary steps is very important, as errors tend to accumulate and alter generation trajectories.
Critical weaknesses
  • The manuscript is extremely difficult to follow. The central point is conceptually straightforward, but it is obscured by lack of narrative, methodological descriptions and improper use of new terms. Unfortunately, the evaluation is equally confusing to the reader. A substantially more focused presentation would improve both accessibility and the ability to assess the contribution.
    • Examples of improperly used and/or inadequately defined terms: macro, fine program, fidelity, interface, prototype, phase, span, bank ...
    • The evaluation setup is completely missing from the manuscript.
  • The theoretical contribution should also be stated carefully. Many of the individual propositions, e.g., that occupancy does not determine the order of non-commuting operations, that omitted state can prevent deterministic closure, and that local approximation errors can accumulate, are elementary or already established. The manuscript would benefit from making this contribution boundary more prominent and concise.
  • The empirical evidence is narrower than the broader routed/coarse-graining framing may initially suggest. The GPT-2 study provides useful evidence for the separation between boundary fidelity and downstream loss in a pretrained model, while the LoopFormer study adds a genuinely recursive setting, but neither pretrained model contains learned expert routing.
Questions for authors
  • What do the authors consider the primary contribution of the paper: the general coarse-graining/path-signature framework, or the continuation-relative evaluation contract?
  • The paper explicitly states that no trained sparse-expert model is evaluated and gives a prospective replication protocol in the Appendix. How central do the authors consider validation on a real learned-routing model to the claims suggested by the current framing?
  • Several components of the general formulation, including ordered path signatures and broader multi-scale/horizon-wide control, are not empirically instantiated. Could the authors clarify which parts of the formal framework are intended as contributions supported by the current experiments vs proposed extensions or diagnostics?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.