acceptodds
Under review as a conference paper at ICLR 2027

Tiny Recursive Policy: Recursive Latent Refinement for Visuomotor Control

Abstract

Behaviour cloning has become a powerful approach for robotic manipulation, but many modern policies reconstruct their internal representation from a fixed observation window at each prediction and do not maintain persistent latent context throughout execution. We introduce Tiny Recursive Policy (TRP), a parameter-efficient behaviour-cloning policy that combines persistent temporal state with recursive latent refinement. At each control step, TRP repeatedly updates high- and low-level latent states using a shared computation module before decoding an action sequence, and propagates the resulting state to subsequent predictions. We evaluate TRP across LIBERO-100, MetaWorld ML45, and the LIBERO-Mem benchmarks. TRP performs competitively across multi-task and few-shot settings and shows significant gains on long-horizon and context-dependent tasks, while using significantly fewer parameters than large generative baselines. Execution-time perturbations show that reliance on the persistent carry increases with temporal-context demand, while training with carry perturbations substantially improves robustness to both carry and visual corruptions. Ablations reveal a strong interaction between persistent state and recursive refinement: the benefit of carry depends strongly on the amount of within-step computation, and the resulting performance cannot be explained by parameter count alone.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Behavior Cloning, Few-shot Imitation Learning, Recursive Refinement

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DBEDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a behavior-cloning policy that updates two recurrent states using a small shared network. These states persist across control steps, and the paper also studies training-time perturbations of the carried state. Evaluation covers multi-task learning, few-shot adaptation, and history-dependent manipulation on three benchmarks. The experiments report competitive performance with a compact backbone and examine the ablation effects of persistent state, recursion depth, model size, and temporal gradient propagation.

Critical strengths
  1. I believe the performance is good, especially considering the small model capacity. TRP achieves 91.9% success on LIBERO-LONG with a 4.2M-parameter backbone. Its LIBERO-Mem subgoal-completion score is 0.597, compared with approximately 0.41 for the strongest reported non-TRP baseline.

  2. Also, the ablations provide useful insights beyond aggregate performance. The execution-time interventions distinguish whether carry is needed during execution from whether it helps during training. An interesting point I found is that the paper also shows that carry is not uniformly beneficial: its effect depends strongly on recursion depth. I find this interaction more informative than simply demonstrating that a recurrent policy uses historical information.

Critical weaknesses
  1. I think the architectural novelty is limited and the validation of the two-level recursion is insufficient. TRP largely adapts existing HRM/TRM-style recursion to stateful control. While Table 3a shows that carry and recursion interact, the comparison between H3L6 and H1L1 also changes the computation budget (21 vs. 2 updates). Thus, it remains unclear whether the gains come from the two-level architecture or simply additional computation. A compute-matched single-loop baseline would help establish the value of the proposed design.

  2. The source of the parameter-efficiency advantage remains unclear. The 4.2M-versus-336.3M comparison in Table 1 is a genuine and significant parameter difference, but it combines different backbone choices with different computation schemes; it does not quantify the contribution of weight sharing itself. My another concern is whether a straightforward, similarly sized recurrent MLP/Transformer, with the same setup (such as observation encoder, action horizon), could achieve a comparable performance–parameter trade-off. This distinction matters because compactness alone does not establish the value of the proposed recursive design.

I would be happy to raise my score if the authors address my concerns.

After reading the rebuttal: I've raised my rating, see reasons below.

Questions for authors
  1. How does TRP compare with a single-loop recurrent policy using the same shared computation block, persistent carry, and comparable per-prediction computation, while keeping the encoder, action horizon, data, and training setup fixed? A focused comparison on LIBERO-LONG and LIBERO-Mem would directly address whether the two-level update schedule provides additional value.

  2. Could you separate the parameters in the shared computation core from the remaining backbone parameters and give the corresponding count for an otherwise identical unshared version? For performance attribution, how does a similarly sized, simple recurrent MLP/Transformer perform under the same observation and training setup?

Reviewer_vCcKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a recursive policy which maintains a temporal state during the rollout and recursively refines it to produce the final actions. The policy keeps a high-level and a low-level latent carry across timesteps, and applies two levels of refinement to iteratively generate actions at each timestep. The paper evaluates the multi-task learning and few-shot generalization abilities of the policy under LIBERO and MetaWorld benchmarks, and showcases the strength of the proposed latent carry in completing the memory-dependent tasks.

Critical strengths

(1) The proposed method is simple yet effective, and it performs well under several benchmarks and settings with a backbone containing only 4.2M parameters. (2) The paper presents a broad set of experiments on the multi-task learning, few-shot learning and memory-dependent tasks settings. And the ablations on several key design choices of the model are performed. These experiments provide a clear demonstration of the model's strengths against the baselines.

Critical weaknesses

(1) The design details of the proposed method are not clearly explained in the paper. Firstly, the motivation and necessity of the proposed two levels of the latent carry lack explanation. A single-state recurrent refinement baseline would help isolate the benefit of the two-state design. Secondly, the specific details of the compute module are not provided, and the paper only mentions that it can be implemented as a transformer block, without further elaboration. Thirdly, the specific architecture of the action head is not discussed, as it is a critical component to build a robot policy. (2) Some critical experiments are missing to support the proposed method. First, the evaluation lacks real-world experiments to validate the model's performance in the physical world. Second, the underlying significance of the 'latent carry' remains unclear. The paper lacks an in-depth discussion and experiments to analyze what this variable represents and what specific information it captures.

Questions for authors

(1) Have you evaluated the proposed policy on physical robots, especially in memory-dependent and long-horizon tasks? If so, how does it perform relative to the baselines? (2) What motivates the use of two levels of the latent carry and the two levels of the recursive refinements? What distinct roles do these two levels serve? (3) How are the compute module and the action head implemented in the TRP policy? (4) Have any in-depth experiments been conducted to investigate the interpretability of the proposed latent carry?

Reviewer_sVWVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is a behavior-cloning method for visuomotor control that combines recursive latent refinement with persistent temporal memory. At each policy step, a shared transformer block repeatedly updates high and low-level latent states before predicting an action chunk. The resulting latent states are carried across policy evaluations. The authors also propose carry perturbation training to improve robustness. Experiments on LIBERO-100, MetaWorld ML45, and LIBERO-Mem show competitive performance with a compact 4.2M-parameter backbone (with the largest improvements on long-horizon and memory-dependent tasks).

Critical strengths
  1. TRP achieves the largest margin on LIBERO-Mem, the one benchmark explicitly designed for non-Markovian manipulation. This is important because it is direct evidence that a compact stateful policy can outperform much larger stateless ones precisely where current-observation information is insufficient (which is the paper’s core motivation).

  2. The paper provides useful ablations beyond the main performance comparisons. Carry-wiping and visual-blackout experiments reveal how the policy uses persistent state, while carry perturbation substantially improves robustness on LIBERO-LONG with little loss in clean performance.

Critical weaknesses
  1. The paper does not sufficiently establish how much of TRP's improvement comes from persistent temporal memory, recursive refinement, or benefits of carry propagation during training.

On MetaWorld, wiping the carry at execution barely affects performance (0.943 -> 0.941), yet training without it gives 0.742. This 20-percentage-point gap is entirely a training-time phenomenon, which the authors leave unexplained beyond a brief "warm start" suggestion in Section 4.4.

On LIBERO-LONG, TRP-perturbed nearly matches TRP's clean performance (0.913 vs. 0.919), while losing only 5 percentage points when the carry is wiped, compared with 34 points for standard TRP. Thus, TRP-perturbed achieves strong long-horizon performance even when the carry is reset at test time. This raises an alternative explanation. Rather than learning a more robust memory representation, carry perturbation may simply encourage the policy to rely less on memory. The experiments do not distinguish between these two explanations.

Table 3b reports that propagating gradients through the carry substantially degrades performance (LIBERO-Mem: 0.597 -> 0.441 -> 0.370 -> 0.326 as the TBPTT horizon increases). However, the table lacks sufficient implementation details to interpret this result. Section 3 states that the first R−1 recursive cycles are executed without gradient tracking and their outputs are detached (which already blocks gradients from reaching the incoming carry). Therefore, simply removing the stop-gradient in Equation 13 would not enable temporal gradient propagation. The authors should clarify what modifications were made to implement TBPTT and whether the reported degradation can be attributed specifically to temporal gradient propagation.

  1. Experiments and reporting are not rigorous enough to support the strength of the claims.

The paper does not clarify whether baseline results were reproduced under the same experimental conditions or taken from previous publications. Furthermore, the statement that all models use the same vision encoder "where possible" is ambiguous, since the authors do not specify which baselines use different encoders. This makes it difficult to assess the fairness of the comparisons.

No experiment distinguishes the benefit of persistent recurrent memory from that of simply using a longer observation history. The observation horizon (n_o) is unspecified, and no long-window baseline is evaluated. The recurrent baseline LSTM-GMM is absent from Table 1 and the MetaWorld comparisons and achieves just 0.019 on LIBERO-Mem, raising concerns about the adequacy and tuning of this baseline. The paper also does not investigate what information the persistent carry actually encodes. LIBERO-Mem provides the clearest evidence for the importance of persistent temporal memory in genuinely non-Markovian manipulation tasks, but the broader claims remain insufficiently isolated experimentally.

Although several baselines have comparable parameter counts (ResNet-T, VQ-BeT, ACT), the paper lacks a compute-matched comparison with a conventional recurrent policy. Table 3a includes a shallow stateful H1L1 variant (whose performance on LIBERO-LONG drops substantially when carry is enabled (0.642 -> 0.206)). However, this only demonstrates that carry is harmful in this particular shallow architecture; it does not establish that persistent state generally requires recursive refinement. A conventional recurrent policy (e.g., GRU/LSTM-based, with the same observation encoder and action chunking) would provide a more informative comparison. Separately, the comparison between TRP's 4.2M parameters and the 336.3M parameter generative baselines does not establish computational efficiency. The default H3L6 configuration requires 21 sequential evaluations of the shared block per prediction, yet no FLOPs, inference latency, or control-frequency measurements are provided. Furthermore, the non-monotonic model-size results (performance peaking at 4.2M parameters and declining for larger models) could reflect insufficient tuning or overfitting rather than demonstrating that recursive computation is inherently more effective than increasing model capacity.

The paper provides no appendix and omits essential implementation details, including the observation and prediction horizons, execution horizon, latent dimension, Transformer configuration, perturbation hyperparameters, optimizer settings, training budget, carry propagation during training, and alignment of action-slot latents across successive timesteps. Finally, no standard deviations or confidence intervals are reported despite the stated three seeds, while several headline improvements are small: LIBERO-90 (0.929 vs. 0.918), MetaWorld ML45 (0.946 vs. 0.945), and few-shot LIBERO-LONG (0.704 vs. 0.688). Without uncertainty estimates, it is difficult to determine whether these improvements are statistically meaningful.

Questions for authors

Were the baseline results in Table 1 and Figure 1 reproduced under your protocol (same demonstrations, observation inputs, seeds, and rollouts) or taken from Mete et al. (2024) and Pan et al. (2026)? Which baselines do not share the vision encoder? If the comparisons are not matched, the headline results cannot be reliably assessed.

Does TRP-no-carry with a longer observation window close the gap on LIBERO-LONG and LIBERO-Mem? The paper's central claim is that persistent memory contributes to the long-horizon and context-dependent gains, but no longer-window control is evaluated, so memory is not separated from simply observing more history.

Can the authors provide a GRU/LSTM baseline using the same observation encoder and action chunking, with matched parameters and compute, particularly on LIBERO-LONG and LIBERO-Mem? The H1L1-with-carry row shows that carry harms that particular architecture, not that recursive refinement is necessary for effective persistent state. How was LSTM-GMM tuned on LIBERO-Mem, where it achieves only 0.019?

During training, is the carry rolled forward through trajectories in temporal order, or are timesteps sampled iid? Since the carry is indexed by future action slot, how is it aligned when the policy advances by the execution horizon shifted or reused as-is? Please specify the observation, prediction, and execution horizons, as these details are necessary to interpret the proposed temporal memory mechanism.

Section 3 states that the first R−1 cycles are evaluated without gradient tracking and detached, which already prevents gradients from reaching the incoming carry when R>1. How was TBPTT in Table 3b implemented? If gradients were enabled through all R cycles within each step, the table conflates removing the within-step gradient truncation with propagating temporal gradients. Was a control with full within-step gradients but detached temporal carry evaluated?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.