acceptodds
Under review as a conference paper at ICLR 2027

Skipping Recomputation During Training: from Recurrent Models to Model Predictive Control and Beyond

Abstract

Sequential model inference can often be accelerated with a binary ”skip or compute” gate. At each step of the sequential computation, the gate determines if an expensive operation is performed, or if it is skipped. The setup is shared across e.g. dynamical systems, diffusion models, sequence classification, and replanning in Model Predictive Control (MPC). During training, however, current methods still perform the costly updates at every step to obtain gradient signals needed for learning the gate. We address this limitation by showing that the gate can be learned from sparse execution while still receiving dense supervision over the skipped steps, without ever having used the gate to control recomputation. The gate learning signal is derived from the computational graph of the sequence model by combining downstream loss cotangents with the input sensitivity of the expensive operation. This decouples learning of the gate network from the sequential model’s recomputation schedule, enabling efficient batch-synchronized training with sparse expensive updates. We demonstrate how this approach can be applied in three distinct scenarios: autonomous dynamics, sequence classification with input-skipping recurrent models and replanning in MPC, with substantial reductions in training time and comparable or improved accuracy.

Primary area:
learning on time series and dynamical systems
Keywords:
surrogate gradient, surrogate gradients, model predictive control, training-time efficiency, sequential, sequence, replanning, planning, computational gate, hard gating, straight-through estimation, RNNs, dynamical systems, diffusion, caching

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_g6zDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces how expensive recomputation can be avoided by learning a gate signal in sequential computations, in a decoupled way. The authors show the capabilities of their proposed approach in different setups.

Critical strengths
  • The paper is well written and easy to follow.
  • The paper addresses an important problem of reducing training time by learning a signal for gates to enable skipping computations in scenarios when they are not needed. This is especially useful when the model needs to perform expensive computations, as in Model Predictive Control.
  • The authors integrate the learning of this gate into the training of the model as a concurrent training objective, but also allow fully decoupled posthoc learning as well.
  • The results on the MPC example are encouraging in terms of training time.
Critical weaknesses
  • A dedicated Related Work section would help to compare the presented approach more thoroughly with previous work (at the moment, it is part of the introduction only, in a paragraph from L054-063).
  • The results on the recurrent neural networks example (Section 5.3) are not convincing; the authors also point out that it is a suboptimal setup. Even though they show big improvements in the construction times, the accuracy differences are still substantial, and I do not see that it would be used in practice. The 40% target version is somewhat closer in accuracy but gives much higher inference time. It is also not clear why that target was chosen, it looks very arbitrary.
  • The authors write in the abstract that: "We demonstrate how this approach [...] with substantial reductions in training time and comparable or improved accuracy." I do not find where it actually leads to improved accuracy.
  • More experimental evaluation would be needed to strengthen the paper.
  • In general, I am not convinced that ICLR is the most suitable venue for this paper.
Questions for authors
  • Can the authors elaborate more on the results of the RNN example (5.3), and clarify the 40% target updates?
  • Could the authors point out the scenario where their approach not only gives improved training time but also improved accuracy?
  • Could the authors provide additional examples where their approach shows benefits?
1 comment

Thank you for your review and time!

"A dedicated Related Work section would help"

  • We believe the text is easier to follow when introduction to the problem and related work are interleaved, but to address concerns of contextualization, we have expanded the related works covered in the introduction. See blue text parts in Section 1.

"Even though they show big improvements in the construction times, the accuracy differences are still substantial"

  • To show there exists a trade-off in favor of our proposed method, we added another run at a denser training-time refresh rate (30% instead of only the original 20%) to Table 3, which now achieves the best accuracy across compared methods, while still being faster to train than all the baseline models.
  • This establishes a trade-off where we can demonstrably either train much faster at a small cost in accuracy for matched inference budget (the original result), or train a bit slower (still faster than all baselines), achieving best accuracy.
  • Additionally, to level the field against SkipRNN at inference timings (as it is deciding how many future steps to skip in advance, which may not be appropriate in other scenarios with surprising inputs), we also tailor our gate to this problem specifically, by evaluating our gate in parallel, as each closed-step interval can do so when inputs are known in advance. Details are in Appendix TODO.

"The 40% target version is somewhat closer in accuracy but gives much higher inference time. It is also not clear why that target was chosen, it looks very arbitrary" and "clarify the 40% target updates"

  • This is a good point, we agree it seemed arbitrary when no justification was given, we will clarify this also in the paper Section 5.4. TODO
  • Briefly: the 30% and 40% inference-time refresh targets were chosen to offer both a lower and higher refresh rate than the baselines natural skip-rates, SkipRNN (31.87%) and DenseSTE (40%) showing how our method works around those operating points.

"I do not find where it actually leads to improved accuracy."

  • We agree this had very thin justification in the original draft, before the improvements in the RNN scenario's accuracy added in this new draft.
  • As that accuracy difference is in our opinion not interesting anyway, we will edit the wording.

"Could the authors provide additional examples where their approach shows benefits?"

  • We have added Section 5.1, demonstrating that the RecompUtility signal generalizes to even more applications than the three already covered in the original draft, bringing the total to five different promising application areas (dynamics, RNNs, Diffusion, Looped Transformers, MPC) for the general method proposed in this work. We hope that such breadth should be sufficient to demonstrate that the proposed method is promising, while we do not think it is within the scope of our paper communicating a new general concept, to implement every possible application of it. We leave that for future work.
Reviewer_7Bn5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper looks at sequential models where an expensive object (an RNN state update, an MPC plan, a dynamics propagator) is cached and reused for several steps, and a binary gate decides when to recompute. The authors' premise is that training such a gate with a straight-through estimator forces the expensive computation at every training step. Their label, RecompUtility, is a first-order estimate of how much the loss would change if the cached object were refreshed at a given step, built from the backpropagated gradient and the Jacobian at the cached input. It avoids evaluating the expensive map at every skipped step and does not depend on the gate, so the gate is fit afterwards from trajectories run under random, batch-synchronised schedules.

Critical strengths

Post-hoc gate learning from a fixed teacher is not new (the paper cites Tao et al. 2019 and Zhao et al. 2026); a teacher label you can compute for skipped steps without running the expensive map is, as far as I know. That is what lets label collection run sparse under a batch-synchronised schedule, and it is the part I found most useful. The MPC experiment supports it: the learned gate matches dense STE on the designed scenarios (96/96), with task cost close to re-planning at every step, in about 1/28 of the reported training time (Table 1).

Critical weaknesses
  1. The utility estimate is never validated directly. The gate is trained from RecompUtility, but the paper never tests how well it predicts the benefit of recomputing. The STE comparison in Section 5.1 checks that two gradient calculations agree; it says nothing about the accuracy of the estimated loss reduction. The dense STE run already solves the new plan at every closed step (A.1), so the estimate could have been compared with the utility from the exact new action. This matters because the gate learns a threshold on the estimate: if the sign or the ordering is off in some region, the gate learns a different rule and the downstream numbers will not show it. The MPC results show the signal is useful there, not how accurate it is.

  2. The MNIST comparison leaves the efficiency advantage unclear. Training is faster, but accuracy is lower and the update rate higher than for SkipRNN and dense STE (Table 2), and no training time at a common accuracy is reported. Against simple schedules, the learned gate wins at low budgets but periodic refresh is better at 20% (Table 4), and neither periodic nor random is tested at the 40% operating point of Table 2. So the claim that "the gate learns a better policy than random or periodic refreshing" is shown only at low budgets, and outside the MPC toy this is the experiment the abstract's accuracy claim rests on.

  3. The RNN utility appears to omit the effect of the anchor update itself. In Section 4.3 the gate decides whether to update the hidden state or keep it. The utility used, U_t = −λᵀJ_aΔu_t, only accounts for how the current input differs from an anchor input; the term for the anchor update itself, λᵀ[h(z_c, u_a) − z_c], is dropped without explanation. A quick check: when the current input equals the anchor input the utility is zero, although updating can still change the state and the loss. The dropped term also differs between intervals while the label threshold ρ (A.2) is global. Please explain why it can be omitted and how the anchor input is chosen. This only affects the RNN construction, but that is the one behind the MNIST results.

Questions for authors
  1. Which STE does Section 5.1 compare against? If it is the input-space relaxation of Section 3.2, the closed-step forward is h(x_c), which is already cached, so what does "always-recomputing" recompute? If it is the output-space kind (Campos et al., Eq. 3), how can the gradients agree to floating-point precision when h is nonlinear?

  2. Sequential MNIST: what do periodic and random refresh give at the 40% operating point? What does "Construction (s)" include, and how long does each method take to reach a common accuracy?

1 comment

Thank you for the careful reading, and particularly the equal-input example in Section 4.3 and the distinction between matching an STE derivative and predicting an actual loss change. These observations led to a correction in the RNN derivation and a new experiment.

"The gate is trained from RecompUtility, but the paper never tests how well it predicts the benefit of recomputing"

  • We agree an empirical evaluation of the quality of the estimator improves the paper, and have added a new experimental section 5.1 to demonstrate exactly this.

"The RNN utility appears to omit the effect of the anchor update itself"

  • Thank you for noticing this, this term was indeed mistakenly omitted from the text/equation 3, although the reported experiments and the code in the supplementary material of the original submission already correctly implemented this. We apologize for the mistake and have corrected it in Equation (3) in Section 4.3.

"Which STE does Section 5.1 compare against? If it is the input-space relaxation of Section 3.2, the closed-step forward is h(x_c), which is already cached, so what does "always-recomputing" recompute?"

  • It is the input-space relaxation of Section 3.2. You are correct that at a closed step the forward value is the already cached h(zc)h(z_c). The dense baseline evaluates hh again at that same input so that ordinary automatic differentiation can propagate the gate derivative through hh, as is typically done with STE in practice. The cached output alone gives the forward value, but computing the derivative also requires the sensitivity of hh at zcz_c. Since that value is shared across the skipped steps, we can reuse its derivative information across the interval. This is precisely the reuse that RecompUtility exploits.

"What do periodic and random refresh give at the 40% operating point?" On the original submitted backbones, at approximately 40% updates, learned, periodic, and random placement achieved 97.64%, 98.38%, and 97.38%, respectively. On the new backbones trained at 30% mean refresh, the corresponding results are 98.76%, 98.88%, and 98.23%. Thus the learned gate beats random placement at this operating point, while periodic placement has the highest mean accuracy.

“What does ‘Construction (s)’ include, and how long does each method take to reach a common accuracy?"

  • Table 3 includes backbone training through validation-based stopping, validation monitoring, and, for our method, utility-label collection and one gate fit. Compilation and warm-up, penalty search, inference-rate calibration, and test evaluation are excluded. The closest observed comparison is 98.76% in 483.5 seconds for the denser-training model and 98.67% in 740.3 seconds for Dense STE. These are completed-model measurements. We did not record time to first reach a common accuracy threshold.
Reviewer_bAGxOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RecompUtility, a signal for training gates that decide whether to recompute an operation during sequential inference. The signal estimates the benefit of refreshing a cached value, allowing the gate to skip some computations. The authors evaluate the method on autonomous dynamics, synthetic model predictive control (MPC), and sequential MNIST. They report substantial reductions in training time, including a reduction from 136.6 to 4.8 minutes compared with Dense straight-through estimator (STE) in the synthetic MPC experiment. On Sequential MNIST, training is faster than Dense STE, but accuracy is slightly lower and inference is not faster.

Critical strengths
  1. RecompUtility offers a new way to train recomputation gates and accelerate sequentially trained sequence models without evaluating at every step.
  2. The experiments show meaningful training-time reductions, particularly on the MPC task, while maintaining strong task performance in the evaluated settings.
Critical weaknesses
  1. On Sequential MNIST, the proposed method has lower test accuracy than Dense STE and similar inference time. This makes the practical trade-off less compelling in this setting, where longer training could produce a more accurate model.
  2. The empirical evaluation is limited to an artificial dynamics, a synthetic MPC, and Sequential MNIST. These tasks could be useful for benchmarking, but do not demonstrate the method’s value in practical applications. The authors also acknowledge that Sequential MNIST is not a realistic sequence-processing scenario. Although diffusion models are among the broader motivations for this work, the method is not evaluated on a realistic task in that setting.
Questions for authors
  1. Could the authors evaluate RecompUtility on a larger, realistic sequential task or on a diffusion-model application, one of the settings motivating the work? This would help establish whether the reported training-time gains extend beyond the current demonstrations.
  2. How closely does RecompUtility’s estimate match the actual effect of recomputing an operation? For example, could the authors compare the estimate with the change in loss when they recompute at steps where the gate would otherwise skip?
  3. How might RecompUtility relate to methods that enable parallel training of sequence models, such as state space models, xLSTM, and DeltaNet? Clarifying how the proposed approach fits alongside these methods could help readers understand its potential role.
1 comment

Thank you for your review and time!

"On Sequential MNIST, the proposed method has lower test accuracy than Dense STE and similar inference time. This makes the practical trade-off less compelling in this setting, where longer training could produce a more accurate model."

  • This was an artefact of our experimental setting where we used a low average refresh rate during training to show improvements in training time, our main goal in this experiment, with what we interpreted as negligible accuracy cost. To show that in fact there exists a trade-off in our favor, we added another run at a denser refresh rate (30% instead of only the original 20%), which achieves the highest mean test accuracy among the trained models in Table 3 at the 40% inference refresh target while still being faster to train than all the baseline models, although with a smaller margin. This should address your concern, and establishes a trade-off where we can demonstrably either train much faster at a small cost in accuracy for matched inference budget, or train a bit slower (still faster than all baselines), then achieving best accuracy. Additionally, to level the field against SkipRNN at inference timings, we added a small improvement evaluating our gate in parallel, as each closed-step interval can do so when inputs are known in advance.

"The empirical evaluation is limited [..] Although diffusion models are among the broader motivations for this work, the method is not evaluated on a realistic task in that setting"

  • Section 5.1 evaluates RecompUtility on a pretrained diffusion transformer, and Appendix A.5 compares it with TeaCache and SenCache at equal refresh budget

"How closely does RecompUtility’s estimate match the actual effect of recomputing an operation?"

  • This is an excellent point and we agree evaluations of this substantially improve the paper, and hence introduced Section 5.1 also for this purpose. This demonstrates that RecompUtility signal correlates more strongly with the true usefulness of the recomputation than various problem-specific baselines.
Reviewer_wCSvOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a training approach for sequential tasks using a batch-synchronized random schedule, ordinary backprop, for skipped steps a per-step label following a first-order Taylor that estimates how much the loss would drop if the expensive model h were re-run with the current input (RecompUtility). The work demonstrates the proposed approach on sequential MNIST, replanning MPC and autonomous systems.

Critical strengths

The addressed problem (sparse training of RNN models) is is of great interest to the AI community. The presented idea of the paper is simple and "clean", gate independent labels allow arbitrary training schedules.

Critical weaknesses

Major concerns (Related work): The paper misses crucial related work it needs to cite, relate to, in some cases benchmark against:

RNNs with time-, clock- or event-gated updates. Clockwork RNN (Koutník et al., 2014) and Phased LSTM (Neil et al., 2016). Phased LSTM explicitly claims faster training and an order-of-magnitude reduction in runtime computation. Delta Networks (Neil et al., 2016) and EGRU (Subramoney et al., ICLR 2023) update only when a change crosses a threshold. EGRU is especially relevant because it shows that the backward pass (BPTT) also becomes sparse and scales with the number of events.

Learned skipping or partial updates in RNNs. HM-RNN (Chung et al., 2017) learns binary boundary/update decisions with a straight-through estimator. This is the same STE baseline the paper compares against in Sec. 3.2/5.1. Variable Computation RNN (Jernite et al., 2017) learns how much of the state to update at each step. LSTM-Jump (Yu et al., 2017) and Skim-RNN (Seo et al., ICLR 2018) learn to skip or skim inputs using policy gradients and Gumbel-softmax. Zoneout (Krueger et al., 2017) is the stochastic "keep previous state" counterpart and is a natural ablation for the gating scheme.

Adaptive computation and conditional compute, including LLMs. ACT (Graves, 2016) and PonderNet (Banino et al., 2021) learn how much computation to spend per step with differentiable or probabilistic halting. SkipNet (Wang et al., ECCV 2018) learns to skip whole blocks with a gating network trained by hybrid supervised/RL learning. In transformers and LLMs, CALM (Schuster et al., NeurIPS 2022), Mixture-of-Depths (Raposo et al., 2024) and LayerSkip (Elhoushi et al., ACL 2024) decide per token whether to run expensive layers. Mixture-of-Depths is notable because it keeps a static compute graph and lowers training FLOPs as well, which is the property the paper highlights as "batch-synchronized training". The survey by Han et al. (2021) organizes this area (instance-, spatial-, and temporal-wise dynamic networks) and would help the authors place their contribution.

Gradient estimators for binary gates. The paper compares only against the STE. Gumbel-softmax/Concrete relaxations (Jang et al., 2017) and REINFORCE-type estimators (used by LSTM-Jump and SkipNet) are the standard alternatives. At minimum they should be discussed, and ideally they should be included as baselines.

Skipping and caching in diffusion models. The abstract names diffusion models as a motivating setting, but none of the diffusion caching literature is cited. DeepCache (Ma et al., 2023) and TeaCache (Liu et al., CVPR 2025) reuse cached features between denoising steps. TeaCache uses a cheap input-difference signal to decide when to recompute, which is conceptually close to the input-sensitivity term in this paper. Learning-to-Cache (Ma et al., NeurIPS 2024) trains a differentiable router that decides which layers to recompute or cache, which is essentially the same "skip-or-compute gate" problem.

Long-horizon learned planning and the question of when to replan. The paper's MPC instantiation is framed only against classical and differentiable MPC. It does not cite the large body of learned long-horizon planners in which deciding whether to re-generate a plan is a central design question. Diffuser (Janner et al., ICML 2022), Decision Diffuser (Ajay et al., 2022), and Hierarchical Diffuser (Chen et al., 2024) plan full trajectories on the D4RL Maze2D/AntMaze long-horizon navigation tasks (Fu et al., 2020) and execute them in receding-horizon fashion. Hierarchical Diffuser explicitly targets the computational cost of replanning. Most directly, Zhou et al. (2023), "Adaptive Online Replanning with Diffusion Models", propose a principled criterion for when to replan, based on the model's likelihood of the current plan. They report 38% gains on Maze2D and note that replanning at every step is both costly and harmful. This is the same skip-or-recompute decision that the paper's MPC gate learns, so it is the most important missing comparison. In visuomotor control, Diffusion Policy (Chi et al., RSS 2023) on Push-T and ACT/action chunking (Zhao et al., 2023) execute action chunks open-loop and replan at fixed intervals. Bidirectional Decoding (Liu et al., 2024) analyses the trade-off between chunk length (consistency) and replanning frequency (reactivity). Learned world-model planners such as TD-MPC2 (Hansen et al., ICLR 2024) and DINO-WM (Zhou et al., 2024) perform MPC on Push-T, 2D mazes (PointMaze), and Wall environments, where each replanning step is a costly model rollout. These works also supply the standard long-horizon testbeds that the paper's evaluation is missing.

Major Point: As indicated in the references before, also central insightful benchmark that are specifically interesting for the proposed approach are missint, e.g., long-horizon planning, Puth-T, PointMaze, etc.

Questions for authors

One semi-clarification question: Please discuss / argue how the written novelty claim that the gate is learned "without ever having used the gate to control recomputation" distinguishes it from related work and why / how this actually helps. This line of argument should be flanked with suitable ablations.

Please address the central papers of the mentioned missing related work and benchmarks.

1 comment

Thank you for the detailed references. In particular, the examples of sparse backward computation, fixed-capacity routing, and adaptive replanning helped us sharpen which part of our training procedure is new. We expanded Section 1, “Introduction and prior work,” and the discussion of decoupled gate learning in Section 3.3.

“Please discuss / argue how the written novelty claim that the gate is learned ‘without ever having used the gate to control recomputation’ distinguishes it from related work and why / how this actually helps.”

  • Post-hoc gate fitting has precedents, including Tao et al. The contribution we want to emphasize is how we obtain its training target. For a step where the expensive operation was skipped, we use its sensitivity at the last refresh and the downstream task-loss gradient to estimate how much refreshing would have helped. This lets us collect supervision for skipped steps while executing a sparse schedule chosen in advance. In particular, we can synchronize expensive updates across a minibatch during training and still fit a gate that makes individual decisions at inference. Section 3.3 explains that benefit, the STE timing experiment isolates its computational effect, and Table 1 tests the quality of the resulting labels.

“Mixture-of-Depths is notable because it keeps a static compute graph and lowers training FLOPs as well.”

  • We agree that efficient batched sparse execution has clear precedents. Mixture-of-Depths selects a fixed number of tokens per layer and can train an auxiliary router to predict those selections during autoregressive inference. LayerSkip uses layer dropout and intermediate exit losses, and supports early exit and self-speculative decoding. Our specific supervision question is how to estimate the downstream task-loss benefit of a computation that was skipped. We have expanded the comparison in Sections 1 and 3.3.

“RNNs with time-, clock- or event-gated updates” and “learned skipping or partial updates in RNNs.”

  • We agree that Phased LSTM and EGRU are relevant and add citations. Phased LSTM uses a time gate to produce sparse updates, and EGRU exploits event sparsity in the backward pass as well as the forward pass.
  • However, neither method while theoretically lowering FLOPs, in practice are not a good fit for efficient training on a GPU, as evidenced by the papers reporting no or very modest wall-time improvements. In addition, even the official code for both methods in fact executes densely during training. We cite these mechanisms in revised Section 1 addressing these shortcomings of existing methods.

“Skipping and caching in diffusion models.”

  • We agree that diffusion caching provides a direct test of the proposed signal. DeepCache reuses features, Learning-to-Cache learns routing decisions, and TeaCache uses changes in the model input to estimate when a fresh evaluation is needed. In Table 1 and the Diffusion appendix, we compare the signals RecompUtility, TeaCache, and SenCache on the same pretrained model and the same refresh decisions. RecompUtility tracks the measured benefit more closely, and the steps it selects give a greater mean LPIPS reduction at matched refresh budgets.

“Adaptive Online Replanning with Diffusion Models […] is the most important missing comparison” and “central insightful benchmarks […] are missing.”

  • We agree that Zhou et al. address the same decision of when to replan as MPC replanning on a very general level, but comparing to various other planning architectures is far beyond the scope of this paper, and we believe including such comparisons would not make the contribution here more clear. Our MPC experiment holds the planner fixed across methods so that the effect of our contribution is isolated.

Comments on the paper

1 comment

We thank the reviewers for their time and feedback. Following two reviewers’ suggestion, we added Section 5.1 to test how well RecompUtility predicts the measured benefit of refreshing. Across all five applications, including the new diffusion and looped transformer experiments, RecompUtility has the highest Pearson and Spearman correlations among the signals tested on each task.

Table 3 also shows a training-time–accuracy trade-off on Sequential MNIST. At the same inference refresh budget, sparser training favors speed, while denser training achieves the highest mean accuracy among the models in Table 3 and still trains faster than every baseline. We also expanded the prior-work discussion in Section 1 and clarified the caption of Figure 1.

Additions to the draft during rebuttal phase are indicated by blue text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.