ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

Action tokenizers play a central role in autoregressive vision-language-action (VLA) models, determining both the targets for policy training and the executable commands recovered from predicted tokens. Tokenizer fidelity is commonly evalu- ated using pointwise reconstruction metrics such as mean squared error (MSE), yet small individual errors do not fully characterize how faithfully action adjustments across demonstrations are preserved. After compression, similar actions may still cluster around a representative motion, while the adjustments needed for different contexts are diminished, distorted, or even reversed. To assess whether similar actions retain their relative physical proximity after reconstruction, we introduce physical rank consistency (PRC), which measures local physical distance-order preservation beyond pointwise reconstruction accuracy. We further present Ac- tionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization. Physical Rank Preservation (PRP) supervises near-far ordering in encoder and quantized feature distances, while Quantization Regularization (QR) applies the same ordering to codeword assign- ment distributions. Together, these objectives guide tokenizer learning to preserve physical distance ordering alongside reconstruction accuracy. Under the Qwen3- VL-4B policy training setup, ActionPiece achieves 94.8% success on LIBERO and 68.8% on unseen LIBERO-Plus, with additional evaluations reaching 71.9% on SimplerEnv and 82.2%, 42.7%, 29.5% on VLA-Arena L0, L1, and L2, respectively. Code is provided in the supplementary material and will be publicly released upon acceptance.