acceptodds
Under review as a conference paper at ICLR 2027

Past2Next: Past Action Conditioning Policy with Data-Augmented Tokenization

Abstract

Discrete-token autoregressive policies have emerged as a compelling paradigm for robot learning. Their downstream performance depends on two axes: the action tokenizer's reconstruction fidelity and the policy's reliability in predicting action tokens. Prior work mostly targets the former, yet reconstruction gains often fail to reach downstream success, a compression gap we trace to the latter, under-addressed axis. We introduce Past2Next, a discrete-token autoregressive policy that addresses this bottleneck by using past action conditioning for token prediction, then decoding the action tokens into a continuous action chunk via a geometry-aware tokenizer. During tokenizer training, we apply data augmentation to improve codebook coverage on the \(SO(3)\) action manifold. During policy learning, Past2Next conditions token prediction on observations, a short history of executed actions, and their first- and second-order finite differences to acquire dynamic features. Our method exploits locally smooth action trajectories governed by the equations of motion. Specifically, finite-difference signals capture the local tangent and higher-order motion trends of the action stream, narrowing the feasible set of next-step actions to a compact neighborhood around its extrapolated trajectory and reducing next-token prediction uncertainty. Across simulation and real-robot experiments, Past2Next reduces predictive Shannon entropy and improves task success rates over the evaluated baselines. Moreover, by such a mechanism, increased codebook capacity improves downstream performance rather than degrading autoregressive learning, allowing finer action quantization to translate into better control performance. Modeling local dynamics and respecting action geometry thus makes an autoregressive policy both accurate and predictable.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Imitation Learning, Robot Learning, Robotics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yj82OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses action reconstruction and token predictability through two complementary interventions. First, SO(3) augmentation during tokenizer training aims to improve coverage of the action manifold and quantization fidelity. Second, the policy conditions on raw action history and its first- and second-order finite differences. The authors motivate these features through local motion continuity and polynomial extrapolation. Their ablations suggest that explicitly providing the difference features improves success rates beyond conditioning on raw history alone. The method is evaluated on both simulation and real hardware.

Critical strengths
  1. The proposed conditioning mechanism is simple and practically useful. Table 1 compares conditioning variants using a shared tokenizer and policy architecture, providing evidence that explicitly exposing finite differences can help policy learning.

  2. The experiments investigate several aspects of the method, including action-history conditioning, codebook capacity, chunk size, and tokenizer augmentation. Evaluations on both simulation and real-world tasks, together with experiments adding history conditioning to continuous policies, suggest broader utility.

Critical weaknesses
  1. Proposition 4.2 requires strict band-limiting, which does not follow from smooth robot motion alone. Although Appendix A acknowledges this approximation, increasing the retained spectral energy from 99% to 99.9% removes the shrinking-bound guarantee. Without quantifying the omitted frequencies, the bound’s relevance to the original demonstrations remains uncertain. Furthermore, the analysis excludes gripper commands, limiting its ability to explain predictability of the complete action-token sequence.

  2. Table 1 does not report uncertainty, and the number of independent training seeds is unclear. Consequently, the robustness of the improvement from raw-history conditioning (62.2%) to history plus finite differences (68.8%) is difficult to assess. Repeated training runs and confidence intervals on the improvement would strengthen this evidence.

  3. Section 6.1 reports the best mean success across checkpoints evaluated on test episodes. It is unclear whether checkpoint selection uses separate validation episodes. Selecting checkpoints using the reported test results could produce optimistic performance estimates. An independent final evaluation and clearer reporting of training, tuning, and checkpoint-selection budgets are needed to assess the substantial gains.

Questions for authors
  1. How many independent training seeds were used for Table 1? Since these finite differences are deterministic functions of the history, this comparison is particularly relevant to establishing the benefit of their explicit representation. Please report uncertainty across independent training runs and confidence intervals on the improvement.

  2. Section 6.1 states that checkpoints are evaluated using 50 test episodes per task and that the best mean success rate is reported. Were checkpoints selected using separate validation episodes, followed by an independent final test evaluation? Please clarify the checkpoint-selection procedure, the number of evaluated checkpoints, and how the reported standard errors were calculated. Given the substantial improvements, comparable training, tuning, and checkpoint-selection budgets across methods are important for interpreting the results.

3 comments
AuthorsOpenReview ↗

W1: Scope and Validity of the Theoretical Assumptions

We clarify the physical motivation, the scope of Proposition 4.2, and its connection to the complete action-token prediction problem.

  1. Physical Grounding: Under bounded actuator torques and appropriate regularity conditions, robot trajectories are locally smooth (C2C^2), as established in Lemma C.1. This motivates using recent actions to predict local motion trends. Our power spectral density (PSD) analysis on LIBERO-Long demonstrations further shows that 99% of the spectral energy is concentrated below fmax⁡=1.58f_{\max}=1.58 Hz, yielding ρ=2πfmax⁡Δt≈0.50\rho=2\pi f_{\max}\Delta t\approx0.50 at Δt=0.05\Delta t=0.05 s (App. A, Table 7). Thus, the band-limited approximation is motivated by the measured frequency distribution of the demonstrated actions, rather than assumed without empirical evidence.

  2. Interpretation of the Contraction Bound: Proposition 4.2 establishes that, under strict band-limiting, the K-point extrapolation error satisfies ‖aₜ − âₜ⁽ᴷ⁾‖∞ ≤ ρᴷ ‖a‖∞. For K = 3 and ρ ≈ 0.50, the bound is approximately 12.5% of the action scale. At the stricter 99.9% spectral-energy cutoff, fₘₐₓ = 4.08 Hz and ρ = 1.28, so the sufficient condition ρ < 1 is no longer satisfied. This means the bound no longer guarantees contraction; it does not imply that past actions cease to provide useful predictive information. Importantly, retaining 99% of spectral energy does not establish a worst-case error guarantee for the original signal, since the remaining high-frequency components may contain meaningful contact transitions and corrections. Our theoretical result therefore characterizes the dominant low-frequency motion component rather than guaranteeing contraction for every demonstrated action..

  3. Applicability to Manipulation and Gripper Commands: Past2Next does not explicitly constrain predicted actions to the extrapolated trajectory. Instead, the finite differences provide local motion information that the policy combines with current visual observations, allowing it to respond to contact transitions and abrupt corrections. The smoothness argument applies to the six continuous pose dimensions, while discrete gripper commands are excluded from Proposition 4.2 and modeled directly by the seven-dimensional action tokenizer. Consequently, the theorem explains a source of predictability in the continuous motion channels. The complete policy's effectiveness is established empirically: past-action conditioning reduces measured next-token predictive entropy from 2.87 to 1.60 nats while improving task success across simulation and real-world manipulation tasks.

  4. Finite-Difference Order and Noise: Lemma D.2 additionally characterizes the trade-off between extrapolation order and noise amplification. Under independent per-step noise with variance σ2\sigma^2, the variance of an mm-th order backward difference is (2mm)σ2\binom{2m}{m}\sigma^2, giving 2σ22\sigma^2, 6σ26\sigma^2, and 20σ220\sigma^2 for the first three orders. This motivates our use of first- and second-order differences to capture local acceleration and jerk trends without introducing the larger noise amplification of higher-order features.

Overall, our theoretical contribution establishes when local motion history can provide a contracting prediction neighborhood, while our experiments demonstrate that explicitly exposing this structure improves autoregressive action prediction in practical manipulation settings.

AuthorsOpenReview ↗

W2 and Q1: Detailed results of Table 1

We thank the reviewer for pointing this.

The uncertainty measure in Table 1: We rerun the controlled study of past-action conditioning on LIBERO-10. Success rates are reported as the mean with standard error. All variants share the same OAT tokenizer (codebook size = 1,000) and policy architecture; only the conditioning inputs differ. Results for (b)–(d) are calculated from four independent training seeds. The OAT baseline retains its previously reported mean and standard error.

Conditioning VariantSuccess Rate (%) ↑
(a) OAT (baseline)56.30 ± 1.00
(b) Raw past actions only62.25 ± 0.17
(c) Dynamics features only63.80 ± 0.14
(d) Dynamics + raw past actions (ours)68.70 ± 0.21
AuthorsOpenReview ↗

W3 Q2: Evaluation Protocol and Checkpoint Selection

We thank the reviewer for requesting clarification of our evaluation protocol. We provide the complete procedure below.

  1. Training and Checkpoint Selection: For all baselines, we train each policy for 1,000 epochs and evaluate every 200 epochs, resulting in five evaluated checkpoints per training run. At each evaluation, we sample random initial states from the benchmark and evaluate 50 episodes per task. We select the checkpoint with the highest average success rate across tasks within each training run.

  2. Independent Seeds and Standard Error: We repeat the entire training and evaluation procedure using four independent random seeds. Each seed produces one selected checkpoint and its corresponding success rate. The final reported result is the mean and standard error across these four selected checkpoints. Therefore, the reported uncertainty reflects variation across independent training runs.

Reviewer_pz31OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces past2next, an action tokenizer for autoregressive policies. Building upon OAT, the paper introduces SA(3) based augmentation and past action conditioning. The authors evaluate past2next on libber-10 and robocall atomic, showing that their tokenizer outperforms prior tokenizers.

Critical strengths
  • The predictability vs. distortion framing is clear and useful for AR tokenization. In particular, showing that past-action conditioning reverses the degradation of larger codebooks is an interesting finding
  • Incorporating history into action conditioning
  • Strong gains on Libero-10
Critical weaknesses

The main weaknesses of the paper are its methodological clarity and evaluation protocol.

  • The main architecture is never disclosed. Details such as how images are encoded, how many parameters the model has, and the exact AR transformer architecture are not described.

  • The evaluation protocol mixes several different settings, making it hard to grasp which changes result in which gains.

  • For instance, the initial study in section 3 is hard to interpret without additional details.

  • The evaluation protocol is hard to grasp, making it unclear how important the action horizon is for the method. Instead of choosing a fixed evaluation protocol, the main result is the best score over several chunk sizes and codebook sizes, with SO(3) augmentation added only to the best configuration. This makes the comparison to baselines hard to judge.

  • It is never disclosed how many seeds are run per experiment.

  • It is not disclosed on what data the entropy is measured

  • SO(3) augmentation is ablated only in a single configuration. As such, the claim that SO(3) "improves codebook utilization" and "reducing the reconstruction-distortion term" is not supported. App. A also states that FSQ is near 100% already, leaving little room for gains.

  • Strongest baselines are not evaluated on the real robot (A2A)

  • No connection made how this could be incorporate to larger-scale autoregressive VLAs, which would be the most interesting application. OAT performs this study.

Questions for authors
  • How does Past2Next compare under a fixed action chunk size and execution horizon, shared with all baselines, across both benchmarks?-
  • Does SO(3) augmentation help on its own, for instance when paired with OAT without past-action conditioning?
  • Which observation encoder is used, how many parameters does the policy have, and what is the exact AR transformer architecture?
  • On which data (training or held-out demonstrations) is the conditional entropy in Fig. 7 measured?
6 comments
AuthorsOpenReview ↗

Clarification

We thank Reviewer pz31for your detailed comments and advice. Based on your paper summary, I would like to clarify our contribution. You mentioned: "The paper introduces past2next, an action tokenizer for autoregressive policies." Rather than only improving action tokenization through SO(3) augmentation, our main contribution is on the policy backbone side. We state in the abstract: "We introduce Past2Next, a discrete-token autoregressive policy that addresses this bottleneck by using past action conditioning for token prediction."

AuthorsOpenReview ↗

W1 and Q3 Model Architecture

  1. Observation Encoder: Our Past2Next policy uses two observation frames and seven previous actions to predict a chunk of 16 actions, each with seven dimensions. The observation encoder and data preparation are exactly the same as OAT (Liu et al.): A trainable ResNet-18 is initialized from random weights and includes a robot state encoder (an identity mapping with no learned projection).

  2. AR Transformer Architecture: The AR transformer is also exactly the same as OAT (Liu et al.): it contains four decoder blocks with hidden width 256, four attention heads per attention module, and head dimension 64.

  3. Training and Inference Recipe: Training uses teacher forcing and cross-entropy over the eight target tokens, while inference generates eight tokens sequentially with KV caching, temperature 1.0, and top-k sampling with k=10, excluding BOS from generated outputs. The frozen action decoder, which contains four transformer decoder layers with width 256, four heads, and feed-forward width 1,024, reconstructs all 16 continuous actions in one forward pass. During training, expert history is used for the first 1,000 optimizer updates; afterward it use predicted past actions with probability 0.5, obtained through one detached prediction on the previous observation window.

Model parameter counts and trainability across training stages. Counts exclude stored data-normalization statistics.

ComponentParameters (M)Trainable in Stage 1Trainable in Stage 2
Observation encoder22.394NoYes
Raw-history MLP0.020NoYes
Acceleration MLP0.020NoYes
Jerk MLP0.020NoYes
AR model, including embeddings and condition stem6.049NoYes
OAT action encoder1.581YesNo
OAT action decoder4.223YesNo
FSQ quantizer0.000----
Total parameters34.309
Stage 1 trainable total∗^{*}5.795
Stage 2 trainable total28.504
AuthorsOpenReview ↗

W2, W4, W5, and Q1: Evaluation Protocol and Experiment Detail

  1. Experiment setup and evaluation protocol: For the main result in Table 2, for each baseline, we report the success rate trained by us using their original paper settings. We train each baseline using 4 random seeds and report the mean success rate.

  2. Action chunk horizon: Because task success may vary with the action chunk horizon, this parameter was independently fine-tuned for each baseline. OAT peaks at a chunk size of 32, whereas A2A performs best at 16. To ensure a fair comparison, all reported baseline results reflect each model's top-performing setup.

Reviewer_u8wHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Past2Next, an autoregressive robot manipulation policy that combines past-action conditioning with geometry-aware action tokenization. The authors argue that the effectiveness of discrete action policies depends not only on action reconstruction accuracy but also on how easily the resulting tokens can be predicted. Experiments on LIBERO-Long, RoboCasa, and three real-world manipulation tasks show encouraging improvements over existing autoregressive and continuous-action policies.

Critical strengths
  1. Clear motivation and interesting perspective. The paper identifies an important distinction between action reconstruction fidelity and token predictability. This provides a useful perspective on why better action tokenization does not necessarily translate into better downstream control performance.
  2. Strong experimental results. Past2Next achieves 85.5% success on LIBERO-Long and 66.3% on RoboCasa, outperforming the strongest reported baselines by 10.7 and 3.7 percentage points, respectively. The real-robot experiments also show consistent improvements across three manipulation tasks.
  3. Practical inference efficiency. The real-robot results report nearly identical inference latency for Past2Next and OAT (27.4 ms versus 27.2 ms), suggesting that the additional conditioning features introduce little inference overhead in the evaluated setting.
Critical weaknesses
  1. The methodological novelty is moderate. Using past actions and finite differences as additional conditioning signals is relatively straightforward. The dynamics-based interpretation is useful, but the main technical components are extensions of existing methods. The authors should clarify what fundamentally distinguishes Past2Next from previous history-conditioned policies.

  2. The theoretical assumptions may not hold in complex manipulation scenarios. The theoretical analysis relies on smooth, approximately band-limited action trajectories. However, real-world manipulation often involves abrupt contact changes, discontinuous commands, and rapid corrections. The appendix also shows that the contraction condition does not hold under the stricter 99.9% spectral-energy cutoff. More discussion of these limitations would strengthen the theoretical contribution.

  3. The contribution of SO(3) augmentation needs further validation. The ablation study shows that SO(3) augmentation improves LIBERO-Long success from 79.4% to 85.5%. However, the paper does not provide sufficiently detailed measurements of reconstruction error, codebook coverage, or the effect of augmentation on different motion types. These would help substantiate the claimed mechanism.

  4. Limited real-world evaluation. Although the real-robot experiments are encouraging, they cover only three tasks on one robotic platform, with 20 trials per task. More experiments involving different robots, contact-rich manipulation, or disturbances would provide stronger evidence of robustness and generalization.

Questions for authors

please refer to the weakness part.

3 comments

We thank the reviewer for their careful review. We have conducted additional experiments and will revise the manuscript based on the reviewer's comments.

W1 Contribution novelty and relation to prior work

  1. Predictability vs. Reconstruction Fidelity: The central contribution of Past2Next is not just introducing historical actions as additional inputs. Rather, we identify and address a previously underexplored bottleneck in discrete-token autoregressive policies: token predictability, which is distinct from tokenizer reconstruction fidelity. In the table below, under the same action chunk (H=32), we show that by enlarging the codebook size (1000 to 5000), we achieve lower reconstruction error (0.003 to 0.001 in MSE) without past action conditioning; the OAT baseline has lower token prediction accuracy and a lower general task success rate.
Token Accuracy (%) ↑Token Entropy (nats) ↓Success Rate (%) ↑
Codebook SizeShared Recon. MSE ↓OATPast2NextOATPast2NextOATPast2Next
1,0000.00315.4922.222.871.6056.368.8
5,0000.00113.8820.994.593.3346.973.4
  1. Physical Dynamics vs. History: Prior works treat raw history as only a generic conditioning; e.g., AR-VLA (Hu et al.) uses raw historical robot states; PTP (Torne et al.) uses observation histories; A2A (Jia et al.) constructs a source prior for continuous flow. In contrast, Past2Next derives history conditioning from rigid-body dynamics (Sec. 4). We prove that first- and second-order finite differences (dt(1)d_t^{(1)} acceleration, dt(2)d_t^{(2)} jerk) explicitly form a quadratic local extrapolator a^t(3)=at−1+dt(1)+dt(2)\hat{a}_t^{(3)} = a_{t-1} + d_t^{(1)} + d_t^{(2)} governing the local continuous action stream.

  2. Information-Theoretic Metric (Entropy Reduction): Rather than evaluating history purely as an empirical feature, we explicitly quantify its effect on discrete token predictability. Past-action dynamics conditioning lowers the per-step conditional entropy H(zi∣ct,z<i)\mathcal{H}(z_i \mid c_t, z_{<i}) from 2.87 to 1.60 nats (Sec. 6.3), shrinking the effective next-token search space from ∼18\sim 18 to ∼5\sim 5 candidate tokens. In the table above, we show how this changes with token accuracy and in the 5000 codebook size case.

  3. Empirical Results: Our experiments show that this conditioning reduces predictive token entropy and enables increased codebook capacity to improve downstream performance. These findings explain how local motion structure can contribute to finer action quantization, translating into better control performance.

W2 The scope of theoretical assumptions

  1. Physical Grounding: Under bounded actuator torques and finite control bandwidth, robot motion is smooth (C2C^2) and bounded (Lemma C.1). Power spectral density analysis on LIBERO-Long demonstrations shows a 99% spectral-energy cutoff at fmax⁡=1.58 Hzf_{\max} = 1.58\text{ Hz}, yielding ρ=2πfmax⁡Δt≈0.50\rho = 2\pi f_{\max} \Delta t \approx 0.50 at Δt=0.05 s\Delta t = 0.05\text{ s} (App. A, Table 7).

  2. Geometric Contraction Bound: Proposition 4.2 guarantees that the extrapolation error satisfies ∥at−a^t(K)∥∞≤ρK∥a∥∞\Vert a_t - \hat{a}_t^{(K)} \Vert_{\infty} \leq \rho^K \Vert a \Vert_{\infty}. For K=3K=3, this confines the feasible next action within a compact neighborhood of approximately 12.5% of the action scale.

  3. Noise Amplification Trade-off: By Lemma D.2, the noise variance of an mm-th order finite difference scales as (2mm)σ2\binom{2m}{m}\sigma^2. The noise variances for 1st, 2nd, and 3rd order differences are 2σ22\sigma^2, 6σ26\sigma^2, and 20σ220\sigma^2, respectively. Selecting 1st (dt(1)d_t^{(1)}) and 2nd (dt(2)d_t^{(2)}) order features captures acceleration and jerk trends while avoiding the severe noise amplification (20σ220\sigma^2) of 3rd-order differences (App. A).

W3 The contribution of SO(3) augmentation

  1. SO(3) Ablation on Past2Next and OAT: The existing ablation study (App. A Table 5) already isolates the downstream contribution of SO(3) augmentation. At the same H=16 and 5,000-code configuration, adding augmentation during tokenizer training improves success from 79.4% to 85.5%, a gain of 6.1 percentage points without changing the policy conditioning mechanism. We also present the ablation of SO(3) augmented action tokenizer results for OAT here, showing that even with SO(3) augmentation but without the past action conditioning, it can not perform well. This proves that past action conditioning and SO(3) augmentation are orthogonal gains. | Policy | Codebook Size | Recon. MSE ↓ | Success Rate ↑ | |:---|:---:|:---:|:---:| | OAT+SO(3) | 1000 | 0.003 | 58.8 | | | 5000 | 0.001 | 44.6 |

  2. Rotation Awareness: The idea of SO(3) augmentation is that translation and rotation components possess geometric structure that generic coordinate-wise tokenization does not explicitly respect. Our augmentation applies a shared rotation to the complete action chunk, preserving its temporal motion pattern while exposing the tokenizer to different spatial orientations.

W4 Limited Real-world evaluation

1 Current Results: Our real-world evaluation includes three multi-stage manipulation tasks with different demands: sequential pick-and-place, articulated object manipulation, and contact-rich nut-and-washer removal. These experiments specifically test whether history conditioning helps during multi-stage action execution. Past2Next achieves 18/20, 16/20, and 15/20 successes, consistently outperforming both Diffusion Policy and OAT. The largest improvement occurs in Nut Washer Removal, where the number of turns required varies across trials, and the transition from a secured to a loose nut can be visually difficult to distinguish. Moreover, these improvements require almost no additional inference overhead compared with OAT (27.4 ms versus 27.2 ms). Evaluation on additional hardware platforms and under controlled disturbances would further establish robustness, but the current results already demonstrate that the method is effective across different manipulation settings.

The rebuttal addresses some of my concerns through additional experiments and clarifications, but the limited methodological novelty and remaining concerns about theoretical assumptions and real-world generalization lead me to maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.