- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Flexi-Act, a visual tokenizer that compresses pretrained image features into an ordered space of continuous tokens. The tokenizer is trained to reconstruct complete visual features from sampled prefixes. It is then fine-tuned with a VLA across different token budgets. The paper evaluates the approach with OpenVLA-OFT and Pi_0 on LIBERO, LIBERO-Plus, and SIMPLER.
Critical strengths
- This paper is well written and easy to follow.
- The motivation is reasonable. Visual tokens account for much of the input processed by a VLA, and reducing that sequence could lower inference cost. Evaluating the method on two policy backbones is useful.
Critical weaknesses
- The methodological novelty is limited. FlexTok has already shown how learned registers and nested dropout can produce ordered visual tokens that work at different prefix lengths. Flexi-Act applies this recipe to VLA features, but the paper does not identify a new insight about visual tokenization for control.
- The claim that the tokenizer is action-oriented is not convincingly supported. Its first stage reconstructs frozen visual features without actions or instructions. Action supervision enters only through the subsequent policy fine-tuning. The paper does not show that the learned prefixes preserve information relevant to control any better than a generic visual compressor would.
- The main design choices are not isolated experimentally. There is no downstream comparison with a simple learned resampler trained across the same token budgets, or with variants that remove feature-reconstruction pretraining or nested dropout. Because the tokenizer and policy are fine-tuned together, it is difficult to tell which part of the method produces the reported results.
- The practical value of ordering remains unclear. The paper explains that prefixes let one checkpoint operate at different token budgets, but it does not show when a deployed policy would need to change its budget. The budget is selected externally rather than adapted to the scene or task.
Questions for authors
- What does the proposed tokenizer offer over a standard learned resampler trained with the same action objective and token-budget sampling?
- How much of the downstream performance depends on the first-stage reconstruction training and nested dropout?
- What evidence shows that early tokens prioritize information needed for control, rather than information useful for visual feature reconstruction?
- In what deployment setting would the token budget change, and how does one ordered tokenizer compare with separately trained fixed-budget tokenizers at matched token counts?
3 comments
Novelty discussion and Action-Oriented Representations
To our knowledge, we are the first to demonstrate the effectiveness of nested dropout and ordered tokenization for compressing visual features used as inputs to VLA policies, extending their application beyond image-generation tokenization as studied by Bachmann et al. Our findings show that ordered training does more than enable flexible token budgets: it improves downstream control performance over a conventional fixed-budget Q-Former at matched token counts. Reconstruction pretraining provides an important initialization, while subsequent action-supervised fine-tuning adapts the representation for control. Together, these findings demonstrate how a single policy can retain strong performance under aggressive visual compression while supporting multiple token budgets, placing Flexi-Act on the control-performance–efficiency Pareto frontier among the efficient VLA methods compared.
To address whether the learned tokens prioritize control rather than visual-feature reconstruction, we distinguish the objectives of the two training stages. Stage 1 learns general compressed visual features through reconstruction, while Stage 2 jointly updates the tokenizer and policy using the action-prediction loss. As shown below, reconstruction cosine similarity decreases at every evaluated budget after Stage 2, even though the resulting policies achieve strong control performance. These results suggest that policy fine-tuning adapts the representation to information useful for action prediction, rather than simply preserving fidelity to the original visual features.
| Tokens/view | Before Stage 2 Cosine similarity ↑ | After Stage 2 Cosine similarity ↑ |
|---|---|---|
| 1 | 0.79 | 0.63 |
| 16 | 0.82 | 0.62 |
| 32 | 0.83 | 0.65 |
| 64 | 0.83 | 0.64 |
Settings for Token-Budget Adaptation
Flexible token budgets allow a single policy to allocate more visual tokens to harder tasks and fewer to easier ones, potentially maintaining strong control performance with a lower average token budget. Predicting task difficulty before execution is beyond the scope of this work. To illustrate the potential benefit, we conduct a simulation with OpenVLA-OFT + Flexi-Act on SIMPLER, using evaluation outcomes to select the smallest evaluated budget between 1 and 32 tokens that permits successful execution for each task. This achieves 64.3% success with an average of 4.3 tokens per view, closely matching the fixed 16-token policy’s 64.4% success at nearly 1/4th the token budget. This analysis demonstrates the potential of task-dependent budgets.
| Budget policy | Avg. tokens/view | Pick Coke Can | Move Near | Open/Close Drawer | Total |
|---|---|---|---|---|---|
| Fixed K = 1 | 1 | 61.3 | 63.4 | 51.1 | 59.0 |
| Fixed K = 16 | 16 | 69.6 | 68.0 | 53.3 | 64.4 |
| Task-dependent (1–32) | 4.3 | 69.5 | 67.9 | 53.1 | 64.3 |
Thank you for the detailed response. Although the additional experiments and discussion address some of my concerns, the explanation of the paper’s novelty and motivation remains unconvincing. Given that all reviewers have raised similar concerns, I have decided to maintain my original rating.
We thank the reviewer for their time and thoughtful feedback, and for acknowledging the clarity of the paper, the motivation for reducing visual context, and the value of evaluating two policy backbones. We address each point below.
Learned-Resampler Comparison and Training Ablations
To address both the contribution beyond a standard learned resampler and the role of nested dropout, we compare Flexi-Act with a conventional Q-Former using the same architecture and other training settings, but a fixed token budget during both stages. At matched evaluation budgets, Flexi-Act outperforms this baseline by around 6 percentage points on LIBERO and 8.5 percentage points on SIMPLER. These results demonstrate the benefit of ordered, variable-budget training over a fixed-budget standard learned resampler for downstream control.
We separately examine reconstruction pretraining by initializing the Q-Former from scratch and training it only during Stage 2, retaining the action-prediction objective and token-budget sampling. Across OpenVLA-OFT and π₀ on LIBERO, removing pretraining lowers average success by around 14 percentage points. This shows that reconstruction-based initialization provides a substantial benefit beyond learning the resampler through action supervision alone.
We also compare against CLS-token selection and feature pooling under the same downstream training setup. Flexi-Act outperforms these simpler compression baselines by around 22 percentage points on LIBERO, demonstrating that simply selecting or pooling pretrained features does not recover the performance of the learned representation.