acceptodds
Under review as a conference paper at ICLR 2027

CausalCache: Aligning Computation with Action in Diffusion Policies

Abstract

Accelerating diffusion robot policies requires identifying computation that can be reused without compromising control. Feature similarity offers a convenient signal, but overlooks how approximation errors are transformed before they reach the robot: the remaining denoising network can absorb or amplify a local perturbation, while robot kinematics can turn similar joint-command errors into different end-effector displacements and rotations. We therefore view cache allocation as preserving task-space fidelity rather than matching intermediate features. We introduce CausalCache, a training-free framework that compiles cache policies from these propagated task-space consequences. Single-block interventions recompute the full network suffix, and an end-effector pose (EE-Pose) objective measures the resulting position and orientation errors. A fixed-budget compiler minimizes the tail risk of intervention costs using , while conditional pair refinement evaluates interacting changes in the context of the compiled policy. Calibration requires neither policy retraining nor task-success labels; deployment executes a frozen step-layer matrix with no online selection overhead. On RoboTwin2, CausalCache retains near-full task success with block reuse, delivering action-denoising and complete-policy inference speedups. Eight calibration episodes perform comparably to forty, while SmolVLA experiments on a physical SO-101 demonstrate the advantage of task-space scoring across three manipulation tasks. These results establish propagated action consequences as a practical basis for allocating computation in diffusion robot policies.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, diffusion policies, VLA, inference acceleration, caching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_k453OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CausalCache, an offline method for selecting which computations to reuse in diffusion robot policies. Cache decisions are scored by their effect on final end-effector poses, followed by budget-constrained optimization and local refinement. On RoboTwin2, the method achieves 91.9% success versus 92.1% for the full policy, with a 2.83× reduction in complete-policy inference time. Experiments on SO-101 provide additional support for the pose-based objective.

Critical strengths
  1. Evaluating the final action consequence is a sensible improvement over local feature similarity. The matched-budget ablation shows a substantial benefit from recomputing the denoising suffix.
  2. The acceleration results are practically useful. Evaluation covers 50 simulation tasks, including 46 tasks absent from calibration, and three physical manipulation tasks.
Critical weaknesses
  1. The baseline comparison is incomplete. CausalCache is evaluated across several budgets, while BAC, Uniform, and reduced-NFE inference each have only one reported setting. More comparable parameter sweeps are needed to establish the claimed advantage.
  2. Some components have limited supporting evidence. Exact Pair matches the simpler Conditional method at every budget in Table 4. The benefit of EE-Pose also varies with budget: it is negligible at B230/B240, positive at B245, and negative at B250. The physical results are encouraging, but 20 trials per configuration leave considerable uncertainty.
Questions for authors
  1. How were the baseline settings selected, particularly BAC’s S4/BUA0? Do the conclusions hold after tuning the baselines to similar inference latencies?
  2. Do Exact Pair and Conditional select the same masks? What are the closed-loop results for the Additive variant? What varies across the five simulation runs? Please provide paired confidence intervals for the main comparisons and refinement gains.
Reviewer_ZebmOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CausalCache, a training-free method to accelerate diffusion/flow robot policies by reusing transformer-block residuals across denoising steps according to a fixed step–layer mask. The mask is built offline. Each block is cached in isolation and the rest of the network is replayed to measure the effect on the final action. This effect is scored in end-effector space via forward kinematics, combining position and orientation errors. A mixed-integer program then selects the blocks to cache at a fixed budget by minimizing the tail risk (CVaR) of the summed per-block costs over calibration episodes, and a small set of budget-neutral swaps refines the result. On RoboTwin2 with Fast-WAM, the method reuses 76.7% of blocks with essentially unchanged success (91.9% vs. 92.1%), for a 3.9× speedup of the action module and 2.8× end to end. Physical experiments with SmolVLA on an SO-101 arm compare end-effector scoring with action-space scoring.

Critical strengths

The central idea, judging cache decisions by their propagated effect on the action rather than by local feature similarity, seems sound and well-supported. The comparison between local-similarity and suffix-replay selection is striking: 28.9% vs. 91.4% success at the same budget. Scoring in task space with orientation is sensible for manipulation, and the per-task results on orientation-sensitive tasks illustrate this nicely. The experimental protocol looks careful: paired seeds, five-run means, bootstrap intervals, a calibration-size study showing that eight episodes suffice, transfer to tasks unseen during calibration, and a clean timing methodology. The resulting masks add no online overhead, which is a real practical advantage.

Critical weaknesses

The compiled masks reveal a simple structure that the paper does not exploit. They compute essentially all of the first denoising step and most of the last one, and reuse the first-step residuals everywhere in between. The B240 mask is exactly "compute steps 1 and 10, cache everything else" and already reaches 90.9% success. This is close to a two-evaluation sampler. The natural baselines are therefore few-step samplers with a tuned time grid (NFE = 2–3), this simple rule-based schedule, and few-step or distilled policies (e.g., Consistency Policy, Prasad et al., RSS 2024). Only uniform NFE = 6 is reported. Without these comparisons it is unclear how much the elaborate calibration and compilation add.

There also seems to be a mismatch between what is calibrated and what is deployed. Per-block costs are measured by reusing the residual from the immediately preceding step, whereas in the deployed masks residuals are typically reused from the first step, many steps earlier. These costs are then summed as if additive over more than 200 cached blocks, and the refinement explores only a handful of local swaps. The evidence for the end-effector objective is limited. In simulation its advantage over action-space scoring appears only near the collapse point (B242–B245, a single layer exchange). It is absent at B230 and reversed by 11 points at B250, a result relegated to the appendix. The success differences against BAC and TeaCache (0.5–0.7 points) are within the binomial uncertainty of 1,000 episodes, and most hardware differences amount to one or two trials out of twenty. The local sensitivity analysis (Eqs. 8–9) is a first-order expansion that adds little.

Finally, the writing is dense and relies on idiosyncratic terminology ("compile", "promotions", "cohorts", "execution receipts", "matrix hashes", etc.) that obscures a conceptually simple method, and key findings, such as the structure of the compiled masks and the B250 reversal, are buried in the appendix.

Minor: flow matching was introduced concurrently and independently by Lipman et al. (2023), Liu et al. (2023, rectified flow), and Albergo & Vanden-Eijnden (2023, stochastic interpolants), all at ICLR 2023; these works should be cited together.

X. Liu, C. Gong, Q. Liu. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. ICLR 2023. M. S. Albergo, E. Vanden-Eijnden. Building Normalizing Flows with Stochastic Interpolants. ICLR 2023.

Questions for authors

How does CausalCache compare, at matched latency, with a 2- or 3-step sampler using a tuned time grid, with the simple "compute first and last step" schedule, and with a distilled few-step policy?

How well do singleton costs predict the measured loss of the full deployed mask, given that deployed residuals are reused from much earlier steps?

Can you explain the reversal at B250, where end-effector scoring is much worse than action-space scoring?

Can you report statistical tests or confidence intervals for the main-table comparisons with BAC and TeaCache and for the hardware results?

Reviewer_YxGPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CausalCache, a training-free inference acceleration framework for diffusion-style robot policies (Action-DiT for Fast-WAM, action expert for SmolVLA). It determines which blocks to actually compute and which to reuse cached residuals on a "denoising step × layer" grid, based not on feature similarity but on the final consequences of caching decisions in task space (end-effector pose). The process consists of three steps: (1) perform a single-point intervention on each block, reuse the residual from that step, and recalculate the complete suffix (subsequent layers, scheduler, denoising step), scoring with the forward kinematics-post-processed EE-Pose error (position + geodesic rotation angle); (2) compile a static mask using CVaR90 mixed integer programming under precise budget constraints; (3) refine in context through budget-neutral conditional pairing and exchange (measured joint actions, capturing compensation/disturbance). During deployment, only one frozen mask is read, without online routers, retraining, or success labels.

Critical strengths
  • It reframes cache selection from feature similarity to task-space causal consequence, with a Jacobian decomposition rigorously showing why local similarity is insufficient.

  • Fully training-free, label-free, and router-free, with a frozen mask at deployment; it reports control–latency trade-offs in both simulation and on hardware, plus independent timing replication and offline amortization costs.

Critical weaknesses
  • CVaR90 estimates the average of the worst 10% of episodes, yet RoboTwin2 calibration has only 40 episodes, so the worst 10% is just 4 episodes; SO-101 has even fewer—8 episodes—where "empirical CVaR90 equals the maximum episode cost" (App. E.2), i.e., CVaR90 degenerates to the worst single episode, giving a very high-variance tail estimate highly sensitive to individual-episode randomness.

  • The main text's failure-mode analysis is thin: there is no breakdown of collision rates, grasp failure rates, or object-drop rates across masks, nor an analysis of what the worst-case caching outcome actually is (pose offset, gripper timing error, or task-level failure).

Questions for authors

See weaknesses above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.