acceptodds
Under review as a conference paper at ICLR 2027

Flexible Action-Oriented Image Tokenization

Abstract

Vision-language-action (VLA) models commonly process hundreds of patch-level visual features at every control step, creating long visual contexts. These visual contexts constitute the largest computational bottleneck in VLAs. Existing VLA compression methods may discard control-relevant information at low token budgets or learn representations tied to a predetermined token count, limiting inference-time adaptation. We introduce Flexi-Act, a visual tokenizer that compresses patch features into a compact, ordered sequence of continuous tokens. This sequence preserves control-relevant information under aggressive compression and supports different visual-token budgets through its prefixes. To learn this ordering, we pretrain the tokenizer with nested dropout and a temporary reconstruction decoder that reconstructs frozen visual-encoder features from sampled prefixes, encouraging earlier tokens to capture broadly useful information while later tokens specialize. We then integrate the pretrained tokenizer into a downstream VLA and jointly fine-tune the tokenizer and policy across prefix lengths, allowing the same policy to switch between supported budgets at inference without retraining. At 16 tokens per camera view, Flexi-Act maintains strong control performance across LIBERO, LIBERO-Plus, and SIMPLER while using 16× fewer visual tokens and 4.5×–5.8× lower estimated FLOPs than OpenVLA-OFT. On LIBERO and LIBERO-Plus, it maintains near-lossless performance using only 1 visual token per camera view.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLAs, Image Compression, Vision-Language

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_XQE9OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Flexi-Act, a visual tokenizer that compresses pretrained image features into an ordered space of continuous tokens. The tokenizer is trained to reconstruct complete visual features from sampled prefixes. It is then fine-tuned with a VLA across different token budgets. The paper evaluates the approach with OpenVLA-OFT and Pi_0 on LIBERO, LIBERO-Plus, and SIMPLER.

Critical strengths
  • This paper is well written and easy to follow.
  • The motivation is reasonable. Visual tokens account for much of the input processed by a VLA, and reducing that sequence could lower inference cost. Evaluating the method on two policy backbones is useful.
Critical weaknesses
  1. The methodological novelty is limited. FlexTok has already shown how learned registers and nested dropout can produce ordered visual tokens that work at different prefix lengths. Flexi-Act applies this recipe to VLA features, but the paper does not identify a new insight about visual tokenization for control.
  2. The claim that the tokenizer is action-oriented is not convincingly supported. Its first stage reconstructs frozen visual features without actions or instructions. Action supervision enters only through the subsequent policy fine-tuning. The paper does not show that the learned prefixes preserve information relevant to control any better than a generic visual compressor would.
  3. The main design choices are not isolated experimentally. There is no downstream comparison with a simple learned resampler trained across the same token budgets, or with variants that remove feature-reconstruction pretraining or nested dropout. Because the tokenizer and policy are fine-tuned together, it is difficult to tell which part of the method produces the reported results.
  4. The practical value of ordering remains unclear. The paper explains that prefixes let one checkpoint operate at different token budgets, but it does not show when a deployed policy would need to change its budget. The budget is selected externally rather than adapted to the scene or task.
Questions for authors
  • What does the proposed tokenizer offer over a standard learned resampler trained with the same action objective and token-budget sampling?
  • How much of the downstream performance depends on the first-stage reconstruction training and nested dropout?
  • What evidence shows that early tokens prioritize information needed for control, rather than information useful for visual feature reconstruction?
  • In what deployment setting would the token budget change, and how does one ordered tokenizer compare with separately trained fixed-budget tokenizers at matched token counts?
3 comments

We thank the reviewer for their time and thoughtful feedback, and for acknowledging the clarity of the paper, the motivation for reducing visual context, and the value of evaluating two policy backbones. We address each point below.

Learned-Resampler Comparison and Training Ablations

To address both the contribution beyond a standard learned resampler and the role of nested dropout, we compare Flexi-Act with a conventional Q-Former using the same architecture and other training settings, but a fixed token budget during both stages. At matched evaluation budgets, Flexi-Act outperforms this baseline by around 6 percentage points on LIBERO and 8.5 percentage points on SIMPLER. These results demonstrate the benefit of ordered, variable-budget training over a fixed-budget standard learned resampler for downstream control.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act897.598.897.994.097.1
OpenVLA-OFT + Q-Former only (no nested dropout)192.991.590.282.289.2
OpenVLA-OFT + Q-Former only (no nested dropout)893.893.193.086.991.7
MethodTokens/viewPick Coke CanMove NearOpen/Close DrawerTotal
OpenVLA-OFT25669.768.353.264.6
OpenVLA-OFT + Flexi-Act161.363.451.159.0
OpenVLA-OFT + Flexi-Act866.266.552.762.6
OpenVLA-OFT + Q-Former only (no nested dropout)152.253.144.850.4
OpenVLA-OFT + Q-Former only (no nested dropout)857.356.547.154.2

We separately examine reconstruction pretraining by initializing the Q-Former from scratch and training it only during Stage 2, retaining the action-prediction objective and token-budget sampling. Across OpenVLA-OFT and π₀ on LIBERO, removing pretraining lowers average success by around 14 percentage points. This shows that reconstruction-based initialization provides a substantial benefit beyond learning the resampler through action supervision alone.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act1698.099.098.294.297.4
OpenVLA-OFT w/o reconstruction pretraining183.381.182.276.080.7
OpenVLA-OFT w/o reconstruction pretraining1686.585.287.078.884.4
π₀25696.898.895.885.294.2
π₀ + Flexi-Act196.698.395.283.993.5
π₀ + Flexi-Act1696.998.795.885.094.1
π₀ w/o reconstruction pretraining181.780.378.669.877.6
π₀ w/o reconstruction pretraining1685.985.183.173.481.9

We also compare against CLS-token selection and feature pooling under the same downstream training setup. Flexi-Act outperforms these simpler compression baselines by around 22 percentage points on LIBERO, demonstrating that simply selecting or pooling pretrained features does not recover the performance of the learned representation.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act1698.099.098.294.297.4
OpenVLA-OFT + CLS Token181.090.185.238.673.7
OpenVLA-OFT + Feature pooling182.192.286.340.075.2
OpenVLA-OFT + Feature pooling1685.192.986.941.176.5

Novelty discussion and Action-Oriented Representations

To our knowledge, we are the first to demonstrate the effectiveness of nested dropout and ordered tokenization for compressing visual features used as inputs to VLA policies, extending their application beyond image-generation tokenization as studied by Bachmann et al. Our findings show that ordered training does more than enable flexible token budgets: it improves downstream control performance over a conventional fixed-budget Q-Former at matched token counts. Reconstruction pretraining provides an important initialization, while subsequent action-supervised fine-tuning adapts the representation for control. Together, these findings demonstrate how a single policy can retain strong performance under aggressive visual compression while supporting multiple token budgets, placing Flexi-Act on the control-performance–efficiency Pareto frontier among the efficient VLA methods compared.

To address whether the learned tokens prioritize control rather than visual-feature reconstruction, we distinguish the objectives of the two training stages. Stage 1 learns general compressed visual features through reconstruction, while Stage 2 jointly updates the tokenizer and policy using the action-prediction loss. As shown below, reconstruction cosine similarity decreases at every evaluated budget after Stage 2, even though the resulting policies achieve strong control performance. These results suggest that policy fine-tuning adapts the representation to information useful for action prediction, rather than simply preserving fidelity to the original visual features.

Tokens/viewBefore Stage 2 Cosine similarity ↑After Stage 2 Cosine similarity ↑
10.790.63
160.820.62
320.830.65
640.830.64

Settings for Token-Budget Adaptation

Flexible token budgets allow a single policy to allocate more visual tokens to harder tasks and fewer to easier ones, potentially maintaining strong control performance with a lower average token budget. Predicting task difficulty before execution is beyond the scope of this work. To illustrate the potential benefit, we conduct a simulation with OpenVLA-OFT + Flexi-Act on SIMPLER, using evaluation outcomes to select the smallest evaluated budget between 1 and 32 tokens that permits successful execution for each task. This achieves 64.3% success with an average of 4.3 tokens per view, closely matching the fixed 16-token policy’s 64.4% success at nearly 1/4th the token budget. This analysis demonstrates the potential of task-dependent budgets.

Budget policyAvg. tokens/viewPick Coke CanMove NearOpen/Close DrawerTotal
Fixed K = 1161.363.451.159.0
Fixed K = 161669.668.053.364.4
Task-dependent (1–32)4.369.567.953.164.3

Thank you for the detailed response. Although the additional experiments and discussion address some of my concerns, the explanation of the paper’s novelty and motivation remains unconvincing. Given that all reviewers have raised similar concerns, I have decided to maintain my original rating.

Reviewer_LoUjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Flexi-Act, a flexible visual tokenizer for reducing the computational cost of VLA models. The method uses a Q-Former to compress visual features into an ordered sequence of tokens, trained with nested dropout and feature reconstruction so that register tokens retain useful information. The tokenizer is then jointly finetuned with a VLA policy across different token budgets, allowing for inference at different levels of compression w/o retraining. Experiments on LIBERO, LIBERO-Plus, and SIMPLER show that Flexi-Act maintains strong performance with substantially fewer visual tokens and lower computational cost across 2 VLA backbones.

Critical strengths
  • Strong efficiency improvements with minimal performance loss. Flexi-Act achieves strong performance with even just one visual token per camera view on LIBERO and LIBERO-Plus. It also reduces inference latency by $\sim$2x compared with OpenVLA-OFT, demonstrating practical benefits beyond theoretical FLOP reductions.
  • Flexible inference across token budgets and VLA architectures. One finetuned policy supports multiple token budgets without retraining. The method is evaluated on both OpenVLA-OFT and OpenPI, showing that the approach works with different action prediction architectures.
Critical weaknesses
  • Limited methodological novelty and unclear VLA-specific contribution. Flexi-Act primarily combines existing techniques (Q-Former-based compression, nested dropout, feature reconstruction, multi-budget finetuning). In particular, the tokenizer is pretrained to reconstruct frozen visual features without any action-related supervision, making it unclear what is specifically action-oriented about the learned representation? While the efficiency improvements are impressive, the paper provides limited new insights into how visual compression can better support robot control.
  • Insufficient ablations and generalization experiments. The paper lacks comparisons with simpler alternatives, such as a standard Q-Former trained at fixed token budgets or a tokenizer trained w/o reconstruction pretraining or nested dropout. These comparisons/ablations are important since a single visual token already achieves near-baseline performance on LIBERO. Furthermore, LIBERO policies are finetuned separately for each evaluation suite, while LIBERO-Plus also uses additional finetuning on its training data (lines 367-374). This limits the evidence that the learned representations generalize to unseen tasks or environments.
Questions for authors
  • How does Flexi-Act compare with a standard Q-Former trained at fixed token budgets? How much do nested dropout and reconstruction pretraining contribute to downstream control performance?
  • What makes the learned representations specifically action-oriented, rather than general compressed visual features? Could the authors provide an analysis of what control-relevant information is preserved at different token budgets?
  • How well does Flexi-Act generalize to unseen tasks or environments without additional finetuning, particularly under the perturbations introduced by LIBERO-Plus?
5 comments

We thank the reviewer for their time and thoughtful feedback, and for recognizing FlexiAct’s strong control performance under aggressive compression, the measured latency improvements, and the flexibility across token budgets and VLA architectures. We address each point below.

Novelty Discussion and Training Ablations

To our knowledge, we are the first to demonstrate the effectiveness of nested dropout and ordered tokenization for compressing visual features used as inputs to VLA policies, extending their application beyond image-generation tokenization as studied by Bachmann et al. Our findings show that ordered training does more than enable flexible token budgets: it improves downstream control performance over a conventional fixed-budget Q-Former at matched token counts. Reconstruction pretraining provides an important initialization, while subsequent action-supervised fine-tuning adapts the representation for control. Together, these findings demonstrate how a single policy can retain strong performance under aggressive visual compression while supporting multiple token budgets, placing Flexi-Act on the control-performance–efficiency Pareto frontier among the efficient VLA methods compared.

To isolate the contribution of Stage 1 reconstruction pretraining, we initialize the tokenizer’s Q-Former from scratch and train it only during downstream policy fine-tuning. Across OpenVLA-OFT and π₀ on LIBERO, removing pretraining lowers average success by around 14 percentage points, showing that the reconstruction objective is important for learning representations that transfer effectively to downstream control.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act1698.099.098.294.297.4
OpenVLA-OFT w/o reconstruction pretraining183.381.182.276.080.7
OpenVLA-OFT w/o reconstruction pretraining1686.585.287.078.884.4
π₀25696.898.895.885.294.2
π₀ + Flexi-Act196.698.395.283.993.5
π₀ + Flexi-Act1696.998.795.885.094.1
π₀ w/o reconstruction pretraining181.780.378.669.877.6
π₀ w/o reconstruction pretraining1685.985.183.173.481.9

To isolate the contribution of nested dropout, we train a conventional Q-Former at a single fixed token budget during both Stage 1 and Stage 2. This removes prefix-ordered training and requires a separate model for each budget. At matched budgets on LIBERO, Flexi-Act outperforms this fixed-budget baseline by around 6 percentage points.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act897.598.897.994.097.1
OpenVLA-OFT + Q-Former only (no nested dropout)192.991.590.282.289.2
OpenVLA-OFT + Q-Former only (no nested dropout)893.893.193.086.991.7

The same trend holds on SIMPLER, where Flexi-Act outperforms the fixed-budget Q-Former by around 8.5 percentage points at matched budgets. This shows that the benefit of prefix-ordered training extends beyond LIBERO.

MethodTokens/viewPick Coke CanMove NearOpen/Close DrawerTotal
OpenVLA-OFT25669.768.353.264.6
OpenVLA-OFT + Flexi-Act161.363.451.159.0
OpenVLA-OFT + Flexi-Act866.266.552.762.6
OpenVLA-OFT + Q-Former only (no nested dropout)152.253.144.850.4
OpenVLA-OFT + Q-Former only (no nested dropout)857.356.547.154.2

To test whether learned tokenization offers benefits beyond simple compression, we compare Flexi-Act with CLS-token selection and feature pooling under the same downstream training setup. Across the evaluated LIBERO budgets, Flexi-Act outperforms these baselines by around 22 percentage points, showing that its ordered representation retains substantially more task-relevant visual information.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act1698.099.098.294.297.4
OpenVLA-OFT + CLS Token181.090.185.238.673.7
OpenVLA-OFT + Feature pooling182.192.286.340.075.2
OpenVLA-OFT + Feature pooling1685.192.986.941.176.5

Generalization to LIBERO-Plus

To evaluate transfer without additional adaptation, we train both OpenVLA-OFT and π₀ policies on LIBERO and evaluate them zero-shot on LIBERO-Plus, with and without Flexi-Act. We find that Flexi-Act’s generalization is bounded by the base visual features on which it is trained. Across both backbones, Flexi-Act follows the generalization profile of the corresponding uncompressed policy in the identical setting, suggesting that the tokenizer largely preserves the backbone’s transfer behavior while compressing its visual representation.

MethodTokens/viewCameraRobotLanguageLightBackgroundNoiseLayoutTotal
OpenVLA-OFT25655.621.781.092.791.078.668.767.9
OpenVLA-OFT + Flexi-Act152.124.564.891.891.980.366.365.2
OpenVLA-OFT + Flexi-Act1653.325.667.192.593.482.668.266.8
π₀25613.86.058.885.081.479.068.953.6
π₀ + Flexi-Act113.04.756.684.480.076.465.451.8
π₀ + Flexi-Act1613.95.657.985.080.878.167.753.1

Action Oriented Representations

To address what makes the representation action-oriented, we distinguish its reconstruction-based initialization from its downstream training objective. Stage 1 learns general compressed visual features, while Stage 2 updates both the tokenizer and policy through the action-prediction loss, adapting the representation to downstream control. We examine this adaptation across token budgets by measuring reconstruction cosine similarity before and after Stage 2. Similarity decreases at every evaluated budget despite strong downstream control performance, suggesting that the representation moves beyond general visual-feature reconstruction toward features suited to action prediction.

Tokens/viewBefore Stage 2 Cosine similarity ↑After Stage 2 Cosine similarity ↑
10.790.63
160.820.62
320.830.65
640.830.64

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the new ablations and zero-shot LIBERO-Plus results, which substantially strengthen the story.

I am still somewhat undecided about the methodological novelty and VLA-specific contribution. While the new experiments demonstrate the benefits of reconstruction pretraining and nested dropout, these are established techniques for learning compressed visual representations. In particular, if I understand correctly, the decrease in reconstruction similarity after Stage 2 may not necessarily demonstrate that the learned features prioritize action-relevant information, as it could simply reflect changes from downstream finetuning?

Could the authors elaborate on what new insight Flexi-Act provides specifically for VLA models? I would appreciate further clarification on this point.

Reviewer_B6kNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Flexi-Act, a flexible visual tokenizer for vision-language-action models. It converts visual patch features into an ordered sequence of continuous tokens. The tokenizer adopts a Perceiver-style architecture and is pretrained with nested dropout and visual feature reconstruction. It is then jointly fine-tuned with the policy using different token budgets. This allows a single checkpoint to support multiple visual token budgets during inference without retraining. Experiments are conducted with OpenVLA-OFT and pi0 on LIBERO, LIBERO-Plus, and SIMPLER. The results show competitive success rates under token compression and reduced estimated FLOPs.

Critical strengths
  1. Practical compression with measured latency gains. At 16 tokens per view, the OpenVLA-OFT variant obtains 97.4% average success on LIBERO, compared with the reported 97.1% reference, while reducing estimated FLOPs from 3.95T to 0.68T. These results suggest that substantial visual-context compression can be useful in practice.

  2. The same tokenizer and policy checkpoint support multiple prefix lengths without separate fine-tuning for each budget. Evaluating this setup on both regression-based OpenVLA-OFT and flow-matching pi0, across three benchmark settings, provides useful evidence that the interface is not tied to a single action head.

Critical weaknesses
  1. Nested dropout and flexible visual tokenization build on prior work by Rippel et al. (2014) and Bachmann et al. (2025). The paper lacks ablations of reconstruction pretraining and nested dropout. It also lacks a simpler tokenizer at a matched budget. Thus, the effect of ordering cannot be separated from the additional training data and tokenizer capacity.

  2. Practical efficiency evidence is incomplete. Appendix D reports latency for OpenVLA-OFT, but not for pi0 or other efficient baselines. Reducing the budget from 16 to 1 token per view lowers latency only from 78.0 ms to 75.2 ms. The tokenizer still computes all 256 registers. Action throughput, peak GPU memory, and training time are also missing. These omissions make the practical efficiency gains difficult to assess.

  3. The experiments include pi0 but not the newer pi0.5, although the latter is cited. Why was pi0.5 omitted?

  4. All control experiments use simulation benchmarks. Although SIMPLER approximates real-world settings, the paper reports no physical robot evaluation. The effects of visual compression under real visual changes, physical contact, and control constraints remain unclear. Real-robot experiments would strengthen the practical claims.

Questions for authors

See weaknesses.

2 comments

We thank the reviewer for their time and thoughtful feedback, and for recognizing the practical compression and latency gains, the flexibility of a single checkpoint across token budgets, and the evaluation across different VLA architectures. We address each point below.

Novelty Discussion and Training Ablations

To our knowledge, we are the first to demonstrate the effectiveness of nested dropout and ordered tokenization for compressing visual features used as inputs to VLA policies, extending their application beyond image-generation tokenization as studied by Bachmann et al. Our findings show that ordered training does more than enable flexible token budgets: it improves downstream control performance over a conventional fixed-budget Q-Former at matched token counts. Reconstruction pretraining provides an important initialization, while subsequent action-supervised fine-tuning adapts the representation for control. Together, these findings demonstrate how a single policy can retain strong performance under aggressive visual compression while supporting multiple token budgets, placing Flexi-Act on the control-performance–efficiency Pareto frontier among the efficient VLA methods compared.

To isolate the contribution of Stage 1 reconstruction pretraining, we initialize the tokenizer’s Q-Former from scratch and train it only during downstream policy fine-tuning. Across OpenVLA-OFT and π₀ on LIBERO, removing pretraining lowers average success by around 14 percentage points, showing that the reconstruction objective is important for learning representations that transfer effectively to downstream control.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act1698.099.098.294.297.4
OpenVLA-OFT w/o reconstruction pretraining183.381.182.276.080.7
OpenVLA-OFT w/o reconstruction pretraining1686.585.287.078.884.4
π₀25696.898.895.885.294.2
π₀ + Flexi-Act196.698.395.283.993.5
π₀ + Flexi-Act1696.998.795.885.094.1
π₀ w/o reconstruction pretraining181.780.378.669.877.6
π₀ w/o reconstruction pretraining1685.985.183.173.481.9

To isolate the contribution of nested dropout, we train a conventional Q-Former at a single fixed token budget during both Stage 1 and Stage 2. This removes prefix-ordered training and requires a separate model for each budget. At matched budgets on LIBERO, Flexi-Act outperforms this fixed-budget baseline by around 6 percentage points.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act897.598.897.994.097.1
OpenVLA-OFT + Q-Former only (no nested dropout)192.991.590.282.289.2
OpenVLA-OFT + Q-Former only (no nested dropout)893.893.193.086.991.7

The same trend holds on SIMPLER, where Flexi-Act outperforms the fixed-budget Q-Former by around 8.5 percentage points at matched budgets.

MethodTokens/viewPick Coke CanMove NearOpen/Close DrawerTotal
OpenVLA-OFT25669.768.353.264.6
OpenVLA-OFT + Flexi-Act161.363.451.159.0
OpenVLA-OFT + Flexi-Act866.266.552.762.6
OpenVLA-OFT + Q-Former only (no nested dropout)152.253.144.850.4
OpenVLA-OFT + Q-Former only (no nested dropout)857.356.547.154.2

Comparison with Simple Compression Baselines

To test whether learned tokenization offers benefits beyond simple compression, we compare Flexi-Act with CLS-token selection and feature pooling under the same downstream training setup. Across the evaluated LIBERO budgets, Flexi-Act outperforms these baselines by around 22 percentage points, showing that its ordered representation retains substantially more task-relevant visual information.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act1698.099.098.294.297.4
OpenVLA-OFT + CLS Token181.090.185.238.673.7
OpenVLA-OFT + Feature pooling182.192.286.340.075.2
OpenVLA-OFT + Feature pooling1685.192.986.941.176.5

Results on π₀.₅

We train a version of Flexi-Act with π₀.₅ as the base model on LIBERO and SIMPLER. We use the same Stage 1 pretraining and Stage 2 fine-tuning procedures described in the paper. The results are below:

MethodTokens/viewSpatialObjectGoalLongAvg.
π₀.₅25698.898.298.092.496.9
π₀.₅ + Flexi-Act197.197.396.589.195.0
π₀.₅ + Flexi-Act1698.498.397.992.296.7
MethodTokens/viewPick Coke CanMove NearOpen/Close DrawerTotal
π₀.₅25669.168.855.165.0
π₀.₅ + Flexi-Act160.760.050.757.6
π₀.₅ + Flexi-Act1668.968.555.064.8

Efficiency Measurements

We provide additonal efficiency results for both OpenVLA-OFT and π₀ below. Although Flexi-Act produces all 256 register embeddings internally, only the selected prefix enters the policy backbone. Because the language backbone performs substantially more computation than the visual encoder and tokenizer, shortening the visual sequence processed by every language-model layer produces most of the inference-time savings.

MethodTokens/viewForward latency (ms) ↓Throughput (actions/s) ↑Peak GPU memory (GB) ↓
OpenVLA-OFT256159.948.815.7
+ Flexi-Act175.284.516.2
+ Flexi-Act1678.086.716.2
π₀256310.0161.17.2
+ Flexi-Act1139.3327.87.6
+ Flexi-Act16146.5322.37.6

The latency difference between one and 16 tokens is modest because both settings have already substantially compressed the original patch sequence, leaving fixed model costs to dominate. The primary gain comes from replacing the uncompressed visual sequence with a compact prefix rather than reducing an already short 16-token prefix to one token. This difference should become more pronounced as VLA backbones grow and deployments incorporate additional camera views, increasing the visual context processed at every control step [1, 2].

[1] Qwen Team. Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments. 2026.

[2] Wei Wu, Fan Lu, Yunnan Wang, et al. A Pragmatic VLA Foundation Model. 2026.

Reviewer_XsqCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Flexi-Act, a continuous visual tokenizer for reducing the visual context processed by vision-language-action models. A Q-Former-style module aggregates patch features into an ordered token sequence. The tokenizer is pretrained through visual-feature reconstruction with nested dropout, and subsequently fine-tuned jointly with the policy across multiple prefix lengths. This enables a single policy checkpoint to operate at different visual-token budgets. The method compresses visual observations rather than tokenizing actions.The authors evaluate Flexi-Act with OpenVLA-OFT and π₀ on LIBERO, LIBERO-Plus, and SIMPLER.

Critical strengths

1.A useful capability for computation-constrained VLA inference. The method allows a single policy checkpoint to operate across several visual-token budgets without retraining. This is practically meaningful because it avoids maintaining a separately trained policy for each supported budget.

2.Encouraging compression results across two policy architectures. Evaluation with both regression-based OpenVLA-OFT and flow-matching π₀ provides evidence beyond a single action-generation architecture. The measured full-model GPU forward speedup in Appendix D also provides practical evidence of acceleration beyond estimated FLOP reductions.

Critical weaknesses
  1. The contribution of the proposed ordered, multi-budget representation is insufficiently isolated. The method combines established visual resampling and nested-dropout mechanisms, but the paper does not compare against a conventional fixed-budget Q-Former or simple feature pooling under comparable training conditions. It also lacks downstream ablations removing reconstruction pretraining or nested dropout, and does not quantify the trade-off between one multi-budget policy and separately trained fixed-budget policies. Consequently, it remains unclear how much the proposed ordering and training strategy contribute beyond learned visual compression and additional training.

  2. The comparative evaluation does not sufficiently establish the claimed advantages over alternative compression approaches. Table 1 includes relevant visual-token reduction methods, but imports their results from Compressor-VLA and compares selected operating points at different token budgets. This is informative but does not establish a controlled performance–efficiency frontier under a common training and measurement protocol. The paper also does not report uncertainty across training seeds, limiting interpretation of small success-rate differences. Moreover, GPU forward timing and simulated rollouts provide limited support for deployment claims involving real robot control loops.

Questions for authors

1.Under comparable training data and optimization budgets, how does Flexi-Act compare with simple feature pooling and a conventional fixed-budget Q-Former at the same output token count? A comparison using measured full-model latency would help establish whether the additional tokenizer complexity is justified.

2.How do downstream success rates change when reconstruction pretraining or nested dropout is removed? At representative budgets, how does the multi-budget checkpoint compare with separately trained fixed-budget counterparts?

3.Which settings are shared between Flexi-Act and the externally reported baselines in Table 1, including policy adaptation, training data, evaluation protocol, and FLOP accounting? Can the authors provide a controlled comparison with a representative learned compression baseline, together with uncertainty estimates for the principal success-rate results?

2 comments

We thank the reviewer for their time and thoughtful feedback, and for recognizing the practical value of flexible token budgets, the compression results across two policy architectures, and the measured inference speedups. We address each point below.

Ablations on reconstruction pretraining and Nested dropout

We first ablate Stage 1 reconstruction pretraining by initializing the tokenizer’s Q-Former from scratch and training it only during downstream policy fine-tuning. We evaluate this variant with both OpenVLA-OFT and π₀ on LIBERO. Across the two backbones and evaluated token budgets, removing reconstruction pretraining lowers average success by around 14 percentage points, demonstrating that Stage 1 pretraining is important for learning compressed visual representations that transfer effectively to downstream control.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act1698.099.098.294.297.4
OpenVLA-OFT w/o reconstruction pretraining183.381.182.276.080.7
OpenVLA-OFT w/o reconstruction pretraining1686.585.287.078.884.4
π₀25696.898.895.885.294.2
π₀ + Flexi-Act196.698.395.283.993.5
π₀ + Flexi-Act1696.998.795.885.094.1
π₀ w/o reconstruction pretraining181.780.378.669.877.6
π₀ w/o reconstruction pretraining1685.985.183.173.481.9

We next isolate the contribution of nested dropout by training a conventional Q-Former at a single fixed token budget in both Stage 1 and Stage 2. This removes prefix-based training and requires a separate model for each budget. At matched budgets, the fixed-budget Q-Former trails Flexi-Act by around 6 percentage points on LIBERO and around 8.5 percentage points on SIMPLER. These results show that training the tokenizer across prefix lengths with nested dropout improves downstream performance.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act897.598.897.994.097.1
OpenVLA-OFT + Q-Former only (no nested dropout)192.991.590.282.289.2
OpenVLA-OFT + Q-Former only (no nested dropout)893.893.193.086.991.7
MethodTokens/viewPick Coke CanMove NearOpen/Close DrawerTotal
OpenVLA-OFT25669.768.353.264.6
OpenVLA-OFT + Flexi-Act161.363.451.159.0
OpenVLA-OFT + Flexi-Act866.266.552.762.6
OpenVLA-OFT + Q-Former only (no nested dropout)152.253.144.850.4
OpenVLA-OFT + Q-Former only (no nested dropout)857.356.547.154.2

These results validate both the Stage 1 reconstruction pretraining and the nested dropout objective showing that they improve the resulting policy across different base models and benchmarks.

Comparison with Simple Compression Baselines

Beyond the fixed-budget Q-Former evaluated above, we compare Flexi-Act with two simpler compression strategies: directly using the visual encoder’s CLS representation and mean-pooling its patch features to the target token budget. We train all methods using the same downstream setup as Flexi-Act. On LIBERO, Flexi-Act at the corresponding budgets, outperforms these simple baselines by around 22 percentage points. These results show that Flexi-Act’s learned ordered representation retains substantially more task-relevant visual information than directly selecting or pooling pretrained visual features.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act1698.099.098.294.297.4
OpenVLA-OFT + CLS Token181.090.185.238.673.7
OpenVLA-OFT + Feature pooling182.192.286.340.075.2
OpenVLA-OFT + Feature pooling1685.192.986.941.176.5

Multi-Budget versus Fixed-Budget Fine-Tuning

We consider two ways to train a policy for a fixed token budget. The conventional fixed-budget Q-Former keeps its token count fixed during both Stage 1 pretraining and Stage 2 fine-tuning and, as shown above, performs substantially worse than Flexi-Act. To isolate the effect of multi-budget policy fine-tuning, we instead retain nested-dropout pretraining in Stage 1 and fine-tune separate policies at a single prefix length in Stage 2. At their training budgets, these fixed-budget policies closely match the multi-budget Flexi-Act policy. However, when the policy trained with 16 tokens is evaluated with one token, its success falls to 86.7% on LIBERO and 51.9% on SIMPLER, compared with 96.6% and 59.0%, respectively, for the multi-budget policy. These results show that nested dropout during reconstruction pretraining is important for compression quality, while multi-budget Stage 2 fine-tuning enables a single policy to operate reliably across token budgets.

MethodTokens/viewSpatialObjectGoalLongAvg.
OpenVLA-OFT25697.698.497.994.597.1
OpenVLA-OFT + Flexi-Act (multi-budget)197.498.597.593.196.6
OpenVLA-OFT + Flexi-Act (multi-budget)1698.099.098.294.297.4
OpenVLA-OFT + Flexi-Act (fixed-budget)197.398.597.593.296.6
OpenVLA-OFT + Flexi-Act (fixed-budget)1698.199.198.094.097.3
OpenVLA-OFT + Flexi-Act (fixed-budget) (trained with budget 16, evaluated with 1 budget 1)189.190.088.379.486.7
MethodTokens/viewPick Coke CanMove NearOpen/Close DrawerTotal
OpenVLA-OFT25669.768.353.264.6
OpenVLA-OFT + Flexi-Act (multi-budget)161.363.451.159.0
OpenVLA-OFT + Flexi-Act (multi-budget)1669.668.053.364.4
OpenVLA-OFT + Flexi-Act (fixed-budget)161.062.950.858.8
OpenVLA-OFT + Flexi-Act (fixed-budget)1669.468.153.264.4
OpenVLA-OFT + Flexi-Act (fixed-budget) (trained with budget 16, evaluated with 1 budget 1)154.755.144.451.9

Baseline Settings and Error Bars

Table 1 includes two categories of compression baselines. FastV, SparseVLM, SpecPrune-VLA, and SP-VLA apply post-hoc token pruning or scheduling at inference, while VLA-Cache reuses cached features. These methods operate on an already fine-tuned OpenVLA-OFT policy and require no additional downstream training. By contrast, Compressor-VLA and Flexi-Act introduce trainable compression modules to the base OpenVLA module, that are then optimized jointly with the OFT policy. OpenVLA-OFT, Compressor-VLA, and Flexi-Act all use the same training data, number of optimization steps, and policy adaptation settings, following the original OpenVLA-OFT fine-tuning recipe.

We additionally report error bars over three independently trained seeds for OpenVLA-OFT and Flexi-Act. Across the four LIBERO suites, the reported error bars range from ±0.1 to ±0.3 percentage points for both methods, indicating low variation across runs.

MethodTokens/viewSpatialObjectGoalLong
OpenVLA-OFT25697.7 +- 0.398.4 +- 0.197.9 +- 0.394.5 +- 0.2
OpenVLA-OFT + Flexi-Act1698.0 +- 0.299.0 +- 0.298.2 +- 0.394.2 +- 0.2

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

We thank all reviewers for their thoughtful and constructive feedback. We appreciate the recognition of FlexiAct's flexible inference from a single policy checkpoint (XsqC, B6kN, LoUj), strong compression performance and measured latency gains (XsqC, B6kN, LoUj), and the motivation for reducing visual context and evaluating across different policy backbones (XQE9). We have expanded the revised manuscript with experiments and analyses addressing the following concerns:

  • Training contributions and compression baselines (XsqC, B6kN, LoUj, XQE9): Added reconstruction-pretraining ablations and comparisons with a fixed-budget Q-Former, CLS-token selection, and pooling. Appendix F.

  • Multi-budget versus fixed-budget policies (XsqC, XQE9): Added comparisons showing that multi-budget fine-tuning preserves matched-budget performance while supporting reliable budget switching. Appendix F.

  • Representations for control (LoUj, XQE9): Added reconstruction analysis before and after policy fine-tuning to examine adaptation beyond visual-feature reconstruction. Appendix B.

  • Practical inference efficiency (B6kN, XsqC): Added latency, action-throughput, and peak-memory measurements across both backbones, alongside discussion of the measured trade-offs. Appendix D.

  • Additional backbone evaluation (B6kN): Added experiments with π₀.₅ on LIBERO and SIMPLER. Appendix G.

  • Zero-shot transfer (LoUj): Added LIBERO-to-LIBERO-Plus evaluations without additional fine-tuning for both backbones. Appendix H.

  • Token-budget adaptation (XQE9): Added an outcome-informed simulation illustrating the potential of task-dependent budgets, while distinguishing it from online budget selection. Appendix I.

We provide detailed responses to each reviewer below and would be happy to answer further questions or clarify any remaining concerns. We hope the additional results and discussion strengthen your assessment of the paper and encourage you to consider revising your scores upward.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.