Major concerns (Related work): The paper misses crucial related work it needs to cite, relate to, in some cases benchmark against:
RNNs with time-, clock- or event-gated updates. Clockwork RNN (Koutník et al., 2014) and Phased LSTM (Neil et al., 2016). Phased LSTM explicitly claims faster training and an order-of-magnitude reduction in runtime computation. Delta Networks (Neil et al., 2016) and EGRU (Subramoney et al., ICLR 2023) update only when a change crosses a threshold. EGRU is especially relevant because it shows that the backward pass (BPTT) also becomes sparse and scales with the number of events.
Learned skipping or partial updates in RNNs. HM-RNN (Chung et al., 2017) learns binary boundary/update decisions with a straight-through estimator. This is the same STE baseline the paper compares against in Sec. 3.2/5.1. Variable Computation RNN (Jernite et al., 2017) learns how much of the state to update at each step. LSTM-Jump (Yu et al., 2017) and Skim-RNN (Seo et al., ICLR 2018) learn to skip or skim inputs using policy gradients and Gumbel-softmax. Zoneout (Krueger et al., 2017) is the stochastic "keep previous state" counterpart and is a natural ablation for the gating scheme.
Adaptive computation and conditional compute, including LLMs. ACT (Graves, 2016) and PonderNet (Banino et al., 2021) learn how much computation to spend per step with differentiable or probabilistic halting. SkipNet (Wang et al., ECCV 2018) learns to skip whole blocks with a gating network trained by hybrid supervised/RL learning. In transformers and LLMs, CALM (Schuster et al., NeurIPS 2022), Mixture-of-Depths (Raposo et al., 2024) and LayerSkip (Elhoushi et al., ACL 2024) decide per token whether to run expensive layers. Mixture-of-Depths is notable because it keeps a static compute graph and lowers training FLOPs as well, which is the property the paper highlights as "batch-synchronized training". The survey by Han et al. (2021) organizes this area (instance-, spatial-, and temporal-wise dynamic networks) and would help the authors place their contribution.
Gradient estimators for binary gates. The paper compares only against the STE. Gumbel-softmax/Concrete relaxations (Jang et al., 2017) and REINFORCE-type estimators (used by LSTM-Jump and SkipNet) are the standard alternatives. At minimum they should be discussed, and ideally they should be included as baselines.
Skipping and caching in diffusion models. The abstract names diffusion models as a motivating setting, but none of the diffusion caching literature is cited. DeepCache (Ma et al., 2023) and TeaCache (Liu et al., CVPR 2025) reuse cached features between denoising steps. TeaCache uses a cheap input-difference signal to decide when to recompute, which is conceptually close to the input-sensitivity term in this paper. Learning-to-Cache (Ma et al., NeurIPS 2024) trains a differentiable router that decides which layers to recompute or cache, which is essentially the same "skip-or-compute gate" problem.
Long-horizon learned planning and the question of when to replan. The paper's MPC instantiation is framed only against classical and differentiable MPC. It does not cite the large body of learned long-horizon planners in which deciding whether to re-generate a plan is a central design question. Diffuser (Janner et al., ICML 2022), Decision Diffuser (Ajay et al., 2022), and Hierarchical Diffuser (Chen et al., 2024) plan full trajectories on the D4RL Maze2D/AntMaze long-horizon navigation tasks (Fu et al., 2020) and execute them in receding-horizon fashion. Hierarchical Diffuser explicitly targets the computational cost of replanning. Most directly, Zhou et al. (2023), "Adaptive Online Replanning with Diffusion Models", propose a principled criterion for when to replan, based on the model's likelihood of the current plan. They report 38% gains on Maze2D and note that replanning at every step is both costly and harmful. This is the same skip-or-recompute decision that the paper's MPC gate learns, so it is the most important missing comparison. In visuomotor control, Diffusion Policy (Chi et al., RSS 2023) on Push-T and ACT/action chunking (Zhao et al., 2023) execute action chunks open-loop and replan at fixed intervals. Bidirectional Decoding (Liu et al., 2024) analyses the trade-off between chunk length (consistency) and replanning frequency (reactivity). Learned world-model planners such as TD-MPC2 (Hansen et al., ICLR 2024) and DINO-WM (Zhou et al., 2024) perform MPC on Push-T, 2D mazes (PointMaze), and Wall environments, where each replanning step is a costly model rollout. These works also supply the standard long-horizon testbeds that the paper's evaluation is missing.
Major Point: As indicated in the references before, also central insightful benchmark that are specifically interesting for the proposed approach are missint, e.g., long-horizon planning, Puth-T, PointMaze, etc.
Thank you for your review and time!
"A dedicated Related Work section would help"
"Even though they show big improvements in the construction times, the accuracy differences are still substantial"
"The 40% target version is somewhat closer in accuracy but gives much higher inference time. It is also not clear why that target was chosen, it looks very arbitrary" and "clarify the 40% target updates"
"I do not find where it actually leads to improved accuracy."
"Could the authors provide additional examples where their approach shows benefits?"