acceptodds
Under review as a conference paper at ICLR 2027

ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

Abstract

Action tokenizers play a central role in autoregressive vision-language-action (VLA) models, determining both the targets for policy training and the executable commands recovered from predicted tokens. Tokenizer fidelity is commonly evalu- ated using pointwise reconstruction metrics such as mean squared error (MSE), yet small individual errors do not fully characterize how faithfully action adjustments across demonstrations are preserved. After compression, similar actions may still cluster around a representative motion, while the adjustments needed for different contexts are diminished, distorted, or even reversed. To assess whether similar actions retain their relative physical proximity after reconstruction, we introduce physical rank consistency (PRC), which measures local physical distance-order preservation beyond pointwise reconstruction accuracy. We further present Ac- tionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization. Physical Rank Preservation (PRP) supervises near-far ordering in encoder and quantized feature distances, while Quantization Regularization (QR) applies the same ordering to codeword assign- ment distributions. Together, these objectives guide tokenizer learning to preserve physical distance ordering alongside reconstruction accuracy. Under the Qwen3- VL-4B policy training setup, ActionPiece achieves 94.8% success on LIBERO and 68.8% on unseen LIBERO-Plus, with additional evaluations reaching 71.9% on SimplerEnv and 82.2%, 42.7%, 29.5% on VLA-Arena L0, L1, and L2, respectively. Code is provided in the supplementary material and will be publicly released upon acceptance.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, Action Tokenization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_66PFOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces Physical Rank Consistency (PRC), a metric intended to measure whether an action tokenizer preserves local ordering of physical distances between action chunks after encoding and decoding. It proposes ActionPiece, an RVQ-based action tokenizer trained with a physical rank preservation loss over representation distances and a quantization regularization loss over codeword-assignment distributions, and evaluates it on LIBERO, LIBERO-Plus, SimplerEnv, VLA-Arena, and real-robot insertion tasks.

Critical strengths

The paper identifies a useful limitation of reconstruction-only tokenizer evaluation.

PRC is a sensible, decoder-agnostic evaluation concept. Measuring rank preservation in decoded action space rather than in each tokenizer's latent space makes the comparison meaningful across tokenizers with different vocabularies and sequence lengths. The use of physical translation, SO(3) geodesic rotation, and gripper discrepancies in Equations 4-5 is substantially more appropriate for this setting than a raw Euclidean distance over heterogeneous action coordinates.

The backbone, policy data, prompt, global batch size, action horizon, and policy training steps are held fixed. Under this setup, ActionPiece achieves 94.8% on LIBERO and 68.8% on LIBERO-Plus, outperforming the strongest reported baselines by 1.1 and 4.5 points, respectively. The larger gain under the out-of-distribution LIBERO-Plus shifts is consistent with the paper's hypothesis that preserving fine action distinctions can matter under perturbation.

The implementation details are relatively thorough. The manuscript specifies the action chunk length, number of tokens, codebook size, objective weights, physical-distance normalization, hard-forward/soft-backward codeword estimator, and training protocols.

Critical weaknesses

The central causal claim is not yet fully disentangled from improved reconstruction quality and other representation improvements.

The relation between the training objective and the reported PRC metric is weaker than the terminology suggests. PRP in Equation 8 is an extreme-pair triplet-style objective: for each anchor it selects one batch-local action at the 5th percentile and one at the 95th percentile, and enforces QR in Equation 10 imposes the analogous constraint on Jensen-Shannon divergence. In contrast, PRC@32 evaluates the rank ordering among the 32 nearest original neighbors. Enforcing the order of one near and one far point does not directly supervise the ordering among local neighbors, nor does it guarantee local neighborhood preservation. This distinction matters because the paper frames PRP and QR as physical “rank preservation,” while they supervise only sparse, batch-dependent ordinal constraints. The paper should evaluate PRC over several values of k report neighborhood-overlap or retrieval metrics, and compare against objectives that directly optimize local ordinal consistency or a differentiable rank surrogate.

The physical metric introduces consequential embodiment- and parameterization-dependent assumptions that are not stress-tested.

Since VLA fine-tuning can exhibit nontrivial seed sensitivity, the reliability of the main ranking is difficult to assess. At least three policy seeds for the matched LIBERO and LIBERO-Plus comparisons, accompanied by confidence intervals over evaluation rollouts, would substantially strengthen the evidence.

Questions for authors

Can the authors provide multi-seed results and confidence intervals for the matched Table 1 and Table 4 comparisons? In particular, is the 1.1-point LIBERO difference from ActionCodec statistically distinguishable from training and evaluation variation?

Were any LIBERO-Plus, SimplerEnv, VLA-Arena, or real-world evaluation results used for model selection or hyperparameter selection?

Can the authors provide a capacity- and reconstruction-matched analysis? For example, compare standard RVQ, PRP, QR, and ActionPiece at matched MSE, matched token count, and matched codebook utilization, then test whether PRC remains predictive of downstream success. How sensitive are PRC and ActionPiece to the action-distance design? Please report results under alternative translation/rotation/gripper weights, alternative rotational normalizations.

Given that the paper motivates physical-rank preservation for precise closed-loop behavior, can the authors provide failure analyses showing cases where two nearby demonstrations are reconstructed with reversed or attenuated adjustments by standard RVQ and correctly separated by ActionPiece? Such examples would make the mechanism more tangible than aggregate PRC alone.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 66PF (1/3)

Thank you for your careful review and constructive suggestions. We appreciate your recognition of the limitation of reconstruction-only evaluation, the value of measuring physical ordering in decoded action space, and the controlled comparisons and implementation details.

| W1 Reconstruction Fidelity, Physical Ordering, and Policy Success

Our claim is that physical-order supervision improves tokenizer quality and downstream policy performance, while PRC provides complementary information beyond mean reconstruction MSE. The randomized-label control in Table 10 and the comparisons at similar MSE provide evidence for these two points, respectively.

We retain the original pairs and randomly reverse half of the near–far labels, randomizing the ordering supervision. Using the original physical labels raises PRC@32 from 0.871 to 0.953 and LIBERO-Plus success from 61.7% to 68.8%. At similar reconstruction MSE, the encoder/quantized weighting of 0.50/0.50 achieves MSE 3.47, PRC 0.955, and LIBERO/LIBERO-Plus success of 94.3%/68.9%, compared with MSE 3.50, PRC 0.871, and 91.2%/61.7% for the randomized-label control. At approximately the same mean reconstruction error, randomizing physical-order supervision substantially reduces PRC and downstream success. This shows that mean MSE alone is insufficient to explain the performance difference and supports the interpretation that preserving physical relationships benefits downstream control.

Near–far labelsMSE (×10⁴) ↓PRC@32 ↑Neighbor overlap@32 ↑LIBERO (%) ↑LIBERO-Plus (%) ↑
Randomly reversed near–far labels3.500.87184.76%91.261.7
Physical (default)3.320.95394.87%94.868.8

The similar-MSE comparison answers a different question: what does PRC tell us beyond reconstruction error? As discussed in Sections 3.1–3.2, similar reconstruction error can coexist with different levels of local physical-order preservation. In Tables 6 and 10, SIGReg and the QR-margin 0.05 variant both have an MSE of 3.77 at the reported precision, yet differ in PRC (0.950 vs. 0.932) and LIBERO-Plus success (66.5% vs. 64.3%).

| W2 Local Ordering and Neighborhood Diagnostics

We tested more direct local supervision by sampling 1, 8, or 16 pairs per anchor from its 32 nearest training-set neighbors. The margins, loss weights, and training budget are fixed (100K steps, seed 42), and all settings use the same validation samples and metric implementation. The results are shown below.

Pair selectionPairs per anchorMSE (×10⁴) ↓PRC@32 ↑Neighbor overlap@32 ↑
Default ActionPiece (batch 5th/95th percentiles)13.320.95394.87%
Training-set top-32 neighbors14.220.91892.00%
Training-set top-32 neighbors84.200.94293.28%
Training-set top-32 neighbors164.250.91891.80%

Eight pairs improve PRC and neighbor overlap over one pair at similar MSE, while sixteen pairs do not sustain that improvement. We also report complementary neighborhood metrics below. Appendix I (Table 12) adds a direct analysis of the differences between nearby reconstructed action commands.

MetricPlain RVQActionPiece
Trustworthiness0.99960.9999
Continuity0.99970.9999
Overlap@320.90500.9487
PRC@320.90200.9530
AuthorsOpenReview ↗

Response to Reviewer 66PF (2/3)

| Q1 Run-to-Run Variability and Robustness Checks

Thank you for asking whether the LIBERO gain is robust to training and evaluation variability. Our LIBERO evaluation uses repeated rollouts across different initial layouts for each task. We repeated the matched LIBERO comparison three times under the same protocol. The table reports success rates (%) as mean ± sample standard deviation across runs. Overall is the equal-weight mean of the four suites within each run.

TokenizerSpatialObjectGoalLongOverall
ActionCodec93.27 ± 0.6199.00 ± 0.4092.40 ± 0.6089.80 ± 0.6093.62 ± 0.35
ActionPiece96.13 ± 0.7098.60 ± 0.4093.67 ± 0.4690.67 ± 0.5094.77 ± 0.10

The average Overall gain is 1.15 percentage points. The lowest ActionPiece Overall score (94.65%) exceeded the highest ActionCodec score (93.95%), so the advantage was consistent across the three runs.

LIBERO is already approaching saturation in this comparison, so the remaining headroom is limited. The gains are larger on LIBERO-Plus and SimplerEnv WidowX: ActionPiece achieves 68.8% versus 64.2% for ActionCodec on LIBERO-Plus (+4.6 percentage points, Table 1), and 71.9% versus 65.2% for ActionCodec-BAR on SimplerEnv WidowX (+6.7 points, Table 2).

Appendix G (Table 10) reports sensitivity to margins, loss weights, encoder–quantized mixing, and near–far sampling. We additionally report matched tokenizer comparisons on Gemma-4 E2B and Qwen2.5-4B below. ActionPiece improves LIBERO and LIBERO-Plus success over Plain RVQ with each backbone.

BackboneTokenizerLIBERO (%)LIBERO-Plus (%)
Gemma-4 E2BPlain RVQ90.461.3
Gemma-4 E2BActionPiece93.265.1
Qwen2.5-4BPlain RVQ88.758.2
Qwen2.5-4BActionPiece92.563.6

We did not use LIBERO-Plus, SimplerEnv, VLA-Arena, or real-world evaluation results for model selection or hyperparameter tuning. The LIBERO-Plus results in Table 10 are reported as additional evaluations of the hyperparameter settings.

|Q2 Matched Comparisons and the Physical Distance Definition

Appendix H (Table 11) separates depth from token budget. At a fixed budget of 16 tokens per chunk, we compare one level with 16 slots against two levels with eight slots, for both Plain RVQ and ActionPiece. We also report the 32-token setting that keeps all 16 slots at two levels. Tables 6 and 10 provide comparisons at similar reconstruction MSE, including the comparison discussed in W1. These comparisons control the token budget or compare similar MSE.

ActionPiece is also effective beyond EEF action parameterizations. On RoboCasa-GR1, we tokenize joint commands directly and use Euclidean distance between normalized joint-action chunks to select near–far pairs. With the same Qwen3-VL-4B backbone, ActionPiece improves mean success over FAST and Plain VQ by 5.17 and 6.25 percentage points, respectively (Appendix F, Table 9).

Tokenizer (Qwen3-VL-4B, 100K steps)RoboCasa-GR1 mean success across 24 tasks (%)
Plain VQ46.25
FAST47.33
ActionPiece52.50
AuthorsOpenReview ↗

Response to Reviewer 66PF (3/3)

|Q3 Preserving Action Differences Between Nearby Demonstrations

Thank you for suggesting a direct reconstruction analysis. Appendix I (Table 12) reports results over the full LIBERO tokenizer validation split. We recomputed the physical top-32 neighbors and report 14,638 same-task, cross-episode pairs and 181,039 cross-task pairs.

For a pair of chunks A,BA,B, we measure how much the reconstructed eight-step translation difference B^−A^\hat B-\hat A deviates from the original difference B−AB-A, relative to its magnitude: Erel=∣∣(B^−A^)−(B−A)∣∣2∣∣B−A∣∣2E_{\mathrm{rel}}=\frac{||(\hat B-\hat A)-(B-A)||_2}{||B-A||_2}.

ActionPiece lowers median error by 54.2% on same-task, cross-episode pairs and by 53.5% on cross-task pairs. It also lowers the local sign-reversal rate for every translation and relative rotation component within the same-task group. This directly tests whether the decoded commands preserve differences between nearby demonstrations.

(a) Relative error in reconstructed translation differences

MetricSame task, different episodesCross-task neighbors
Number of pairs14,638181,039
Plain RVQ median relative error0.2990.297
ActionPiece median relative error0.1370.138
Relative reduction in median error54.2%53.5%

(b) Local sign reversals within same-task, cross-episode pairs

ComponentPlain RVQActionPiece
Translation xx5.22%1.36%
Translation yy5.46%1.97%
Translation zz4.71%0.89%
Relative rotation xx1.63%0.34%
Relative rotation yy2.01%0.48%
Relative rotation zz2.09%0.46%

Compared with Plain RVQ, ActionPiece more than halves the median relative error in action differences for both groups and reduces sign reversals in all six translation and relative rotation components. These results show that ActionPiece more faithfully preserves the magnitude and direction of the adjustments between nearby demonstrations.

Reviewer_aGF5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ActionPiece looks at a gap in action tokenization for autoregressive VLAs: a tokenizer can reconstruct individual actions accurately and still distort the physical relationships between them. The paper introduces Physical Rank Consistency (PRC) as a metric for this and adds relational training objectives to the tokenizer. The evaluation includes controlled tokenizer comparisons, distribution-shift benchmarks and real-robot experiments.

Critical strengths
  1. The paper explains clearly why small pointwise reconstruction errors can still distort the action differences that precise manipulation depends on, which standard reconstruction metrics do not capture. Preserving physical relationships through quantization is a sensible answer to this problem, and PRC is a useful complementary diagnostic.
  2. The empirical comparisons are broad, covering matched LIBERO comparisons, alternative-objective ablations, distribution-shift evaluations and real-robot tasks that need precise alignment and coordination. The component ablations show gains when representation-level and assignment-level supervision are combined.
Critical weaknesses
  1. The paper does not explain well why encoder features, quantized features and codeword assignment distributions each need their own supervision. The objective has several interacting terms, and its sensitivity to the loss weights, the distance mixing and the neighborhood selection is not studied, so the method looks more complicated than its central idea needs.
  2. The experiments do not isolate where the gains come from. All controlled tokenizer comparisons use one Qwen-based backbone, and the SimplerEnv and VLA-Arena comparisons mix systems with different backbones and training setups. ActionPiece also improves reconstruction together with PRC, while the alternative neighborhood objective improves policy success despite lowering PRC. Comparisons on other backbones and at matched reconstruction quality are needed to show that preserving physical rankings is what drives the improvement. Since this is the paper's central claim, it is the main reason for my score.
Questions for authors
  1. How does PRC relate to existing neighborhood-preservation measures from dimensionality reduction, such as trustworthiness and continuity? Similarly, how do PRP and QR differ from earlier distance- or topology-preserving autoencoders and quantizers?
  2. PRP and QR contrast a near chunk with a far chunk in each batch, while PRC measures the ordering among close neighbors. Why should this coarse near-versus-far signal improve the fine local ordering?
  3. What are the main remaining failure modes in the real-robot insertion and handover tasks, and do they come from token prediction errors or from tokenizer reconstruction?
4 comments
AuthorsOpenReview ↗

Response to Reviewer aGF5 (1/3)

Thank you for your careful review and constructive suggestions. We appreciate your recognition of the motivation to preserve differences between actions, the value of PRC as a complementary diagnostic, and the breadth of our empirical evaluation.

| W1 Roles of Feature and Codeword-Assignment Supervision

Thank you for asking what each term contributes. Quantization can change the distance ordering learned by the encoder when nearby features are mapped to different codewords. PRP therefore applies the same ordering criterion before and after quantization, with more weight on the quantized features that the decoder receives. Distances between selected codeword embeddings describe the outcome of code selection. The soft assignment distributions also reflect how the tokenizer scores alternative codes. QR uses this information to supervise the assignment step.

The component ablation in Table 6 shows that PRP gives the larger individual gain over Plain RVQ. Adding QR on top of PRP further improves MSE, PRC, and policy success, including a 3.4-point gain on LIBERO-Plus. The mixing sweep shows a trade-off. Increasing the encoder share improves PRC and slightly raises LIBERO-Plus success, while the default gives the lowest MSE and highest LIBERO success. This sweep compares different balances between the two feature distances. We report the full sensitivity study in Appendix G (Table 10).

All policies below use Qwen3-VL-4B with 30K training steps. Each sensitivity setting varies one factor from the default: margins of 0.1, encoder/quantized mixing of 0.25/0.75, and near/far percentiles of 5%/95%. Loss-weight multipliers are relative to the default.

SettingValueMSE (×10⁴) ↓PRC@32 ↑LIBERO (%) ↑LIBERO-Plus (%) ↑
Component ablation
Plain RVQReconstruction only4.060.90290.960.4
+ PRP3.630.94793.865.4
+ QR3.920.91692.962.4
ActionPiece (default)PRP + QR3.320.95394.868.8
Margin
PRP margin0.0503.620.94094.066.6
PRP margin0.2003.560.94794.667.2
QR margin0.0503.770.93293.464.3
QR margin0.2003.950.92593.162.5
Loss weight
PRP weight0.50×3.900.91693.063.1
PRP weight2.00×3.390.95194.167.5
QR weight0.50×3.730.92593.963.7
QR weight2.00×4.080.91192.562.1
Encoder and quantized distance mixing
Encoder / quantized0.50 / 0.503.470.95594.368.9
Encoder / quantized0.75 / 0.253.690.96093.969.2
Near–far sampling percentiles
Near / far1% / 99%3.790.92792.665.9
Near / far10% / 90%3.370.94594.468.1
AuthorsOpenReview ↗

Response to Reviewer aGF5 (2/3)

| W2.1 Physical Ordering and Policy Success

Our claim is that physical-order supervision improves tokenizer quality and downstream policy performance, while PRC provides complementary information beyond mean reconstruction MSE. The randomized-label control in Table 10 and the comparisons at similar MSE provide evidence for these two points, respectively.

The label control in Appendix G (Table 10) retains the original pairs and randomly reverses half of the near–far labels, randomizing the ordering supervision. Using the original physical labels raises PRC@32 from 0.871 to 0.953 and LIBERO-Plus success from 61.7% to 68.8%. This isolates the effect of using correct physical labels. At similar reconstruction MSE, the encoder/quantized weighting of 0.50/0.50 achieves MSE 3.47, PRC 0.955, and LIBERO/LIBERO-Plus success of 94.3%/68.9%, while the randomized-label control achieves MSE 3.50, PRC 0.871, and 91.2%/61.7%. At approximately the same mean reconstruction error, randomizing physical-order supervision substantially reduces PRC and downstream success. This shows that mean MSE alone is insufficient to explain the performance difference and supports the interpretation that preserving physical relationships benefits downstream control.

Near–far labelsMSE (×10⁴) ↓PRC@32 ↑Neighbor overlap@32 ↑LIBERO (%) ↑LIBERO-Plus (%) ↑
Randomly reversed near–far labels3.500.87184.76%91.261.7
Physical (default)3.320.95394.87%94.868.8

Table 6 shows a boundary case for interpreting tokenizer metrics. The Neighborhood objective (Appendix C, Eq. 14) pulls each anchor's ten physically nearest chunks in the minibatch closer in representation space, with larger weights for closer chunks. The loss can decrease even when the relative ordering of those neighbors changes. Compared with Plain RVQ, the Neighborhood variant yields higher MSE and lower PRC, yet improves success on both LIBERO and LIBERO-Plus. One way this can happen is when physically similar actions are interchangeable in a given state. Changing their relative order can then lower PRC without harming execution. We report this case to show where aggregate tokenizer metrics and measured policy success can diverge.

The comparison at similar reconstruction error illustrates a complementary point. SIGReg and the QR-margin 0.05 variant both have an MSE of 3.77 at the reported precision. They differ in PRC (0.950 vs. 0.932) and LIBERO-Plus success (66.5% vs. 64.3%). This shows that similar average reconstruction error can coexist with different local ordering and different performance under distribution shift. These results motivate reporting MSE, PRC, and policy success together.

| W2.2 Backbone and Benchmark Comparisons

To test whether the tokenizer gains extend beyond Qwen3-VL-4B, we added within-backbone comparisons on Gemma-4 E2B and Qwen2.5-4B. ActionPiece improves success on both LIBERO and LIBERO-Plus with each backbone.

BackboneTokenizerLIBERO (%)LIBERO-Plus (%)
Gemma-4 E2BPlain RVQ90.461.3
Gemma-4 E2BActionPiece93.265.1
Qwen2.5-4BPlain RVQ88.758.2
Qwen2.5-4BActionPiece92.563.6

We additionally compare Plain RVQ and ActionPiece under the same Qwen3-VL-4B backbone on SimplerEnv and VLA-Arena to assess the gains from replacing the tokenizer.

Qwen3-VL-4B tokenizerSimplerEnv (%)VLA-Arena L0 (%)VLA-Arena L1 (%)VLA-Arena L2 (%)
Plain RVQ55.270.528.99.5
ActionPiece71.982.242.729.5
AuthorsOpenReview ↗

Response to Reviewer aGF5 (3/3)

| Q1 Relation to Neighborhood and Structure Preservation

Trustworthiness penalizes neighbors introduced by reconstruction, while continuity penalizes original neighbors that are lost. Both account for how far the affected points move in rank. Overlap@32 measures the fraction of the original 32 nearest neighbors that remain in the reconstructed top-32 set, excluding the anchor in both sets. PRC@32 asks a complementary question: among the original 32 neighbors, how well does reconstruction preserve their distance ordering relative to the anchor? Even with 100% overlap, the order within the neighborhood can change.

MetricPlain RVQActionPiece
Trustworthiness0.99960.9999
Continuity0.99970.9999
Overlap@320.90500.9487
PRC@320.90200.9530

PRP and QR build on the broader idea of structure-preserving regularization. Distance-matching objectives can penalize changes in numerical distances, and topological autoencoders can align topological signatures. ActionPiece uses physical distances between action chunks to define selected near–far orderings. PRP applies these comparisons to encoder and quantized features, while QR applies them to codeword-assignment distributions via Jensen–Shannon divergence. The contribution is to carry the same action-space ordering signal through feature learning and discrete code assignment, and then assess the structure of the decoded actions.

| Q2 From Near–Far Supervision to Local Ordering

Each anchor contributes one near–far comparison per batch, but the comparisons change across batches and update the same encoder and quantizer. The accumulated constraints help separate physically distinct motions, while reconstruction encourages the tokenizer to retain the details that distinguish nearby motions. Together, these signals can improve local relationships that the loss does not directly compare.

The randomized-label control in our response to your W2.1 tests this directly. Retaining the original pairs while randomizing the ordering supervision yields PRC@32 of 0.871, compared with 0.953 using physical labels. Neighbor overlap and policy success also improve with the physical labels.

| Q3 Real-Robot Failure Modes and Reconstruction Replay

The main failures we observed were missed grasps during bimanual handover, where small offsets in height or position affected the receiving hand, and failed insertions when the tube was slightly misaligned with the rack. To isolate reconstruction effects, we ran paired replays on 20 scenes per task. For each scene, we replayed the same ground-truth trajectory either directly or after encoding and decoding it with each tokenizer, under matched initial conditions.

Replay conditionBimanual handoverTube insertion
Original ground-truth actions20/2020/20
Plain RVQ reconstruction12/2014/20
ActionPiece reconstruction19/2020/20

These replays bypass policy token prediction. They show that executing decoded actions can fail even when the tokenizer receives the ground-truth actions and the decoder receives their corresponding tokens. ActionPiece reduced these reconstruction-induced failures in both tasks.

Dear Reviewer aGF5,

Thank you again for your careful review and constructive suggestions. We appreciate your recognition of our motivation to preserve action differences, the complementary value of PRC, and the breadth of our empirical comparisons.

We have posted our point-by-point responses with additional experiments and uploaded the revised manuscript, with changes highlighted in blue. We hope these additions help address your concerns. If you have any further questions or would like clarification on any point, we would be happy to provide additional details and continue the discussion.

Thank you again for your time and consideration.

Reviewer_T2rpOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that pointwise reconstruction error is insufficient to assess whether subtle variations between similar actions are preserved by an action tokenizer. To address this limitation, the authors propose Physical Ranking Consistency (PRC), which selects 32 nearest neighbors for each anchor in the original action space and measures the Spearman rank correlation between pairwise physical distances before and after reconstruction. The physical distance metric combines normalized translation distance, the shortest rotation angle in SO(3), and gripper discrepancy.

To strengthen relative constraints, ActionPiece incorporates two objectives into the action codec. Physical Ranking Preservation (PRP) constrains the relative proximity of encoded and quantized features, while Quantization Ranking (QR) imposes the same ranking constraint on the Jensen–Shannon divergence between codeword assignment distributions. The final configuration encodes 8-step action sequences into 16 tokens, using a codebook of size 512 and a single quantization layer. After freezing the tokenizer, the authors train a Qwen3-VL-4B policy using standard next-token prediction.

Critical strengths
  1. PRC is a simple diagnostic that works across tokenizers. It is computed in the decoded physical action space with a proper SO(3) distance, so it applies across vocabularies, sequence lengths, and decoder types. Other tokenizer papers could adopt it.
  2. Controlled tokenizer comparison and broad evaluation. All tokenizers in Table 1 are compared under one Qwen3-VL-4B policy with identical data, prompt, batch, horizon and training budget, which is rare in the VLA tokenizer literature. ActionPiece is evaluated on four simulation benchmarks plus a real dual-arm Franka task. Its gain on unseen LIBERO-Plus (+4.5 pp over the strongest baseline across 10,030 rollouts) is well beyond rollout-sampling noise, which makes it a useful empirical reference point even though the source of the improvement remains unclear.
Critical weaknesses
  1. The paper's own controls do not support its central claim that preserving physical distance order is what drives the gains.
    • Table 6 shows that +SIGReg reaches nearly the same PRC as +PRP and the full model (0.950 vs 0.947 and 0.953), even though SIGReg is an isotropic-Gaussian regularizer that uses no physical-distance information at all.
    • +SIGReg also scores higher than +PRP on LIBERO-Plus (66.5 vs 65.4). +Temporal matches ActionPiece on LIBERO (94.8).
  2. The training signal does not match the stated goal of local neighbourhood preservation. Sections 1 and 3 motivate ActionPiece by the need to preserve small differences between similar demonstrations, and PRC@32 measures distance ordering among the 32 nearest neighbours. Yet PRP and QR (Sec. 4.3, App. D.4) supervise only one pair per anchor inside a random batch of 128: the 5th-percentile chunk (≈6th nearest) against the 95th-percentile chunk (≈120th nearest), with a margin of 0.1. In a multi-task batch, these two chunks typically belong to different motion phases or even different tasks, so the constraint is a coarse, global triplet margin that is probably satisfied easily. Nothing directly constrains the ordering within the local neighbourhood that PRC evaluates. It is therefore unclear how the proposed losses produce the claimed fine-grained relational fidelity. Ablations with nearer positives/negatives (e.g., ranks within the top-k), multiple pairs per anchor, or a listwise ranking loss over local neighbours would help establish this link.
  3. Limited action-space and embodiment coverage. All evaluations use end-effector (EEF) pose deltas plus a parallel gripper on two arm types: Franka in LIBERO/LIBERO-Plus/VLA-Arena and on the real robot, and WidowX in SimplerEnv. This is not only a gap in experiments. The core physical distance dphysd_{phys} (Eqs. 4–5, 20, 22), which defines the near/far pairs for both PRP and QR as well as the PRC metric, is constructed explicitly from translation, SO(3) rotation and gripper state. The paper does not say how dphysd_{phys} should be defined for joint-space actions. Each choice would change the selected pairs and hence the supervision signal. It is therefore unclear whether the conclusions, and the claim that PRC is a tokenizer-agnostic diagnostic, transfer beyond Cartesian single- or dual-arm control. An experiment on a joint-space benchmark (e.g., RoboCasa-GR1 or Robotwin), with an explicitly specified joint-space dphysd_{phys}, would substantially strengthen the paper.
Questions for authors
  1. How were the margins (mr=mq=0.1m_r = m_q = 0.1), the loss weights (λr=λq=6.25×10−4\lambda_r = \lambda_q = 6.25\times10^{-4}), the encoder/quantized mixing weight (λ=0.25\lambda = 0.25) and the 5th/95th-percentile pair selection chosen? The two margins act on quantities with different ranges (D∈[0,2]D \in [0,2] for unit-normalized features vs dJS∈[0,ln⁡2]d_{JS} \in [0, \ln 2]), yet share the same value. Please report a sensitivity analysis (at least over the margins, the loss weights and the percentile choice), and the magnitude of LrankL_{rank} and LquantL_{quant} relative to LMSEL_{MSE} during training, together with the fraction of triplets that already satisfy the margin.
  2. The paper presents the method as RVQ, but the final model uses Q = 1 (Table 7), i.e., single-level VQ. App. D.2 claims Q = 1 and Q = 2 perform similarly but gives no numbers, while prior work (e.g., FASTerVQ) reports that more residual levels substantially reduce reconstruction error. Could the gains from PRP/QR partly compensate for the limited capacity of a single level? Please report MSE, PRC and success for Q ∈ {1, 2, 3}, with and without PRP+QR, and state the Q used by the "Standard RVQ" baseline.
11 comments
AuthorsOpenReview ↗

Response to Reviewer T2rp (1/3)

Thank you for your careful review and detailed suggestions. We appreciate your recognition of PRC as a diagnostic applicable across tokenizers, the controlled tokenizer comparisons, and the breadth of our evaluation.

| W1 Reconstruction, PRC, and Policy Success in Table 6

Thank you for highlighting these comparisons. Our claim is that physical-order supervision improves tokenizer quality and downstream policy performance, while PRC provides complementary information beyond mean reconstruction MSE. The randomized-label control in Table 10 and the comparisons at similar MSE provide evidence for these two points, respectively.

PRC measures a property of decoded actions and does not require the training objective to explicitly include physical distances. The reconstruction objective already uses ground-truth actions as supervision, and adding representation regularization that does not depend on physical distances may also improve reconstruction and local relationships. SIGReg increasing PRC from 0.902 to 0.950 therefore does not contradict the view that preserving action relationships well may be beneficial. It shows that explicit ordering supervision is one way to obtain this property, but not the only way. The same reasoning explains why we do not directly optimize PRC: training objectives and evaluation metrics do not need to match term by term. PRP and QR impose sparse physical-order constraints, while PRC evaluates local ordering after decoding. These constraints can improve PRC indirectly, without directly optimizing all top-32 orderings.

We therefore interpret the policy success rates in Table 6 alongside both reconstruction MSE and PRC. Compared with standard RVQ, SIGReg reduces MSE from 4.06 to 3.77, increases PRC from 0.902 to 0.950, and raises LIBERO and LIBERO-Plus success from 90.9% to 93.3% and from 60.4% to 66.5%, respectively. Compared with PRP alone, SIGReg has higher PRC (0.950 vs. 0.947), while PRP has lower MSE (3.63 vs. 3.77). The two variants trade places across benchmarks: PRP performs better on LIBERO (93.8% vs. 93.3%), and SIGReg performs better on LIBERO-Plus (66.5% vs. 65.4%). Full ActionPiece achieves lower MSE and higher PRC than Temporal, matches its 94.8% on LIBERO, and improves success by 2.7 percentage points on the more challenging LIBERO-Plus benchmark (68.8% vs. 66.1%).

AuthorsOpenReview ↗

Response to Reviewer T2rp (2/3)

| W2.1 Why Can Sparse Near–Far Supervision Improve Local Ordering?

Thank you for raising this question. Each anchor contributes one near–far comparison per batch, but the comparisons change across batches and update the same encoder and quantizer. The accumulated constraints help separate physically distinct motions, while reconstruction encourages the tokenizer to retain the details that distinguish nearby motions. Together, these signals can improve local relationships that the loss does not directly compare.

To test the physical ordering signal directly, we retain the original pairs and randomly reverse half of the near–far labels, randomizing the ordering supervision. Using the original physical labels raises PRC@32 from 0.871 to 0.953, neighbor overlap from 84.76% to 94.87%, and LIBERO-Plus success from 61.7% to 68.8%. This control shows that correct physical supervision can improve decoded local ordering and policy performance even with one near–far comparison per anchor. Appendix G (Table 10) reports the MSE, PRC, and policy success results.

Near–far labelsMSE (×10⁴) ↓PRC@32 ↑Neighbor overlap@32 ↑LIBERO (%) ↑LIBERO-Plus (%) ↑
Randomly reversed near–far labels3.500.87184.76%91.261.7
Physical (default)3.320.95394.87%94.868.8

| W2.2 How Demanding Are the Selected Comparisons?

Although the selected chunks may come from different tasks or motion phases, each chunk covers only eight action steps, and different tasks can contain similar short translations, rotations, or gripper motions. We therefore measure the physical distances between the selected chunks and their anchors directly. Across 8,192 anchors, the mean physical distance was 1.20 for near pairs and 3.00 for far pairs. The corresponding mean squared differences between the original chunks, computed over their 8×78\times7 normalized action elements, were 0.080 and 0.386, a 4.83-fold difference. To assess how often the learned constraints remain unsatisfied, we evaluated the trained tokenizer from our main experiments on 10,000 randomly sampled test batches of 128 chunks. After training, 5.34% of PRP comparisons and 1.74% of QR comparisons still fell short of their respective margins. The margin of 0.1 applies to the learned representation-distance or Jensen–Shannon divergence gap. The softplus loss remains positive after the margin is satisfied and gradually approaches zero as the learned near–far gap grows.

| W2.3 Local Pairing Controls

We tested closer positives and negatives and multiple comparisons per anchor. For each anchor, we sampled 1, 8, or 16 pairs from its 32 physically nearest neighbors in the full training set and ordered each pair by physical distance to the anchor. We averaged the pair losses within each anchor and held the margins, loss weights, and training budget fixed (100K steps, seed 42). All rows use the same validation samples, metric implementation, and evaluation settings.

Pair selectionPairs per anchorMSE (×10⁴) ↓PRC@32 ↑Neighbor overlap@32 ↑
Default ActionPiece (batch 5th/95th percentiles)13.320.95394.87%
Training-set top-32 neighbors14.220.91892.00%
Training-set top-32 neighbors84.200.94293.28%
Training-set top-32 neighbors164.250.91891.80%

Among the local variants, increasing from one to eight pairs per anchor improved PRC and neighbor overlap at similar reconstruction error. Increasing to sixteen pairs did not sustain the improvement. The default near–far sampling achieved the lowest reconstruction error and the best results on both neighborhood metrics in this comparison, showing that more direct local comparisons do not necessarily lead to better local ordering.

| W3 Joint-Space Control on RoboCasa-GR1

Thank you for this suggestion. We added a RoboCasa-GR1 joint-space experiment in Appendix F (Table 9), covering 24 tasks on a dual-arm platform with dexterous hands.

For RoboCasa-GR1, each coordinate has the same joint-control meaning across action chunks. We therefore use Euclidean distance between normalized joint-action chunks to select near–far pairs.

Tokenizer (Qwen3-VL-4B, 100K steps)RoboCasa-GR1 mean success across 24 tasks (%)
Plain VQ46.25
FAST47.33
ActionPiece52.50

With the same Qwen3-VL-4B backbone, ActionPiece improves mean success over FAST and Plain VQ by 5.17 and 6.25 percentage points, respectively. These results support the applicability of ActionPiece to joint actions within a fixed embodiment and extend our experimental coverage to dual-arm dexterous-hand joint control.

AuthorsOpenReview ↗

Response to Reviewer T2rp (3/3)

| Q1 Hyperparameter Sensitivity

Thank you for suggesting these analyses. We added a sensitivity study in Appendix G (Table 10) that varies the PRP and QR margins and weights, the encoder–quantized mixing coefficient, and the near–far sampling percentiles one setting at a time. All policies use Qwen3-VL-4B with 30K training steps. The default uses margins of 0.1, encoder and quantized weights of 0.25 and 0.75, and 5th/95th-percentile sampling. Loss-weight multipliers are relative to the corresponding default weight.

SettingValueMSE (×10⁴) ↓PRC@32 ↑LIBERO (%) ↑LIBERO-Plus (%) ↑
ActionPiece (default)—3.320.95394.868.8
Margin
PRP margin0.0503.620.94094.066.6
PRP margin0.2003.560.94794.667.2
QR margin0.0503.770.93293.464.3
QR margin0.2003.950.92593.162.5
Loss weight
PRP weight0.50×3.900.91693.063.1
PRP weight2.00×3.390.95194.167.5
QR weight0.50×3.730.92593.963.7
QR weight2.00×4.080.91192.562.1
Encoder and quantized distance mixing
Encoder / quantized0.50 / 0.503.470.95594.368.9
Encoder / quantized0.75 / 0.253.690.96093.969.2
Near–far sampling percentiles
Near / far1% / 99%3.790.92792.665.9
Near / far10% / 90%3.370.94594.468.1

We additionally report LIBERO-Plus results in Table 10 to show the effects of the hyperparameter changes; these results were not used for model selection or hyperparameter tuning. Compared with the encoder/quantized weighting of 0.75/0.25, the default scores 0.9 percentage points higher on LIBERO and 0.4 points lower on LIBERO-Plus. Notably, doubling the PRP weight yields lower MSE, slightly higher PRC (0.951 vs. 0.950), and higher success on both benchmarks than the SIGReg-only variant in Table 6. This also highlights the potential of PRP.

| Q2 RVQ Depth and Token Budget

Thank you for asking us to clarify the effect of residual depth. We now describe the main ActionPiece configuration as single-level vector quantization (VQ, Q=1Q=1) in the Introduction and Sections 4 and 4.1. Appendix D.2 describes the extension to multiple residual levels. The Standard RVQ baseline uses one quantization level (Q=1Q=1), with 16 latent slots and 16 action tokens per chunk. We have clarified in Appendix D.2 that the similar performance at Q=1Q=1 and Q=2Q=2 refers to a fixed 16-token budget. Under that budget, increasing depth from one to two levels reduces the number of slots from 16 to 8. We also evaluated a second setting that keeps all 16 slots at Q=2Q=2, expanding the sequence to 32 tokens. We report both comparisons in Appendix H (Table 11).

MethodQSlotsTokensMSE (×10⁴) ↓PRC@32 ↑LIBERO (%) ↑LIBERO-Plus (%) ↑
Fixed budget (16 tokens)
Plain RVQ116164.060.90290.960.4
ActionPiece116163.320.95394.868.8
Plain RVQ28164.180.90790.661.1
ActionPiece28163.420.95194.768.8
Expanded budget (32 tokens)
Plain RVQ216323.5800.93794.264.3
ActionPiece216322.8850.97396.369.2

At a fixed budget, ActionPiece's LIBERO success differs by 0.1 percentage points between depths, and LIBERO-Plus success is unchanged. With 32 tokens, both Plain RVQ and ActionPiece improve in reconstruction and policy success, and ActionPiece retains an advantage over Plain RVQ at each matched setting. The longer sequence also requires more autoregressive decoding steps and more token predictions per action chunk. We therefore keep 16 tokens for the main comparison, matching the budget used for ActionCodec and FASTerVQ in our evaluation. Due to compute constraints, we did not evaluate Q=3Q=3. The Q=1Q=1 and Q=2Q=2 results show that increasing depth has little effect on policy success at a fixed 16-token budget, whereas increasing depth while keeping all 16 slots improves performance with a 32-token budget.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Dear Reviewers and Area Chairs,

Thank you very much for your careful reviews and constructive suggestions. We appreciate your recognition of PRC as a diagnostic applicable across tokenizers (@T2rp and @66PF), our motivation to preserve action differences (@aGF5 and @66PF), and the controlled comparisons and breadth of our evaluation (@T2rp, @aGF5, and @66PF).

Your feedback has helped us strengthen the experimental evidence and improve the clarity of our paper. We have revised the manuscript and provided point-by-point responses to each reviewer. Added and revised text is marked in blue. The updates include, but are not limited to, the following:

1. Tested physical-order supervision (Section 5.4 and Appendix G, Table 10).
Addresses: @T2rp (W1, W2); @aGF5 (W2, Q2); @66PF (W1, W2).
The randomized-label control shows that correct physical labels improve PRC from 0.871 to 0.953 and LIBERO-Plus success from 61.7% to 68.8%.

2. Clarified training supervision and evaluation (Sections 3.2 and 4.3).
Addresses: @T2rp (W1, W2); @aGF5 (Q2); @66PF (W2).
We clarified PRC’s role as a decoded-action diagnostic and explained how accumulated near–far constraints and reconstruction jointly support local relationship preservation.

3. Added hyperparameter analyses (Appendix G, Table 10).
Addresses: @T2rp (Q1); @aGF5 (W1).
We evaluated margins, loss weights, feature-distance mixing, and sampling percentiles.

4. Added joint-space evaluation (Appendix F, Table 9).
Addresses: @T2rp (W3); @66PF (Q2).
On 24 RoboCasa-GR1 tasks, ActionPiece achieves 52.50% success, exceeding FAST and Plain VQ by 5.17 and 6.25 percentage points.

5. Clarified VQ configuration and token budgets (Section 4 and Appendices D.2 and H, Table 11).
Addresses: @T2rp (Q2); @66PF (Q2).
We clarified the single-level VQ configuration and compared quantization depths under fixed 16-token and expanded 32-token budgets.

6. Added direct action-difference analysis (Section 3.1 and Appendix I, Table 12).
Addresses: @66PF (Q3).
Full-validation-set analysis shows over 50% lower median translation-difference error and reduced sign reversals across all six translation and relative rotation components.

Thank you again for your time and constructive feedback. We hope our responses and revisions address your concerns. If you have any further questions or would like clarification on any point, we would be happy to provide additional details and continue the discussion.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.