acceptodds
Under review as a conference paper at ICLR 2027

SCEVLA: SPARSE COLLABORATIVE EXITS WITH SPECIALIZED TOKENS FOR DYNAMIC VISION LANGUAGE-ACTIONMODELS

Abstract

Vision-Language-Action (VLA) models have become a cornerstone of embodied intelligence, but their large-scale architectures introduce substantial inference latency, limiting real-time robotic deployment. Existing dynamic VLA approaches alleviate this issue by allocating computation adaptively. However, dense exit supervision and conflicting updates to shared parameters can impair intermediate predictions. In this work, we propose SceVLA: a dynamic-depth VLA framework that enables efficient inference via sparse collaborative exits with specialized tokens. SceVLA introduces sparsely distributed exits at different backbone depths, where each exit is equipped with dedicated action tokens to specialize in predictions at its corresponding representation level. To encourage effective cross-depth collaboration, we design a collaborative attention mechanism that allows exits to exploit complementary information from different layers, while blocking gradients through cross-exit feature transfers. Furthermore, we develop a lightweight router that dynamically selects an exit for each input through unidirectional attention over exit-specific tokens and backbone representations. On CALVIN, SceVLA achieves a policy-query speedup with AvgLen decreasing from 4.42 to 4.36. On RoboTwin 2.0, it achieves a speedup with 72.7% success, compared with 85.5% for the full-depth base policy, revealing a benchmark-dependent accuracy–latency trade-off.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Adaptive Computation, Early-Exit Inference, Cross-Exit Collaboration, Robotic Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_apfwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SceVLA, a dynamic-depth VLA framework with sparse exits, specialized action-token streams, and directed cross-exit attention. Deeper-exit streams reuse shallower-stream features through detached connections, while a separately trained router decides when to stop computation. Experiments on CALVIN and RoboTwin 2.0 report over 2× policy-query speedups, with a small performance drop on CALVIN and a larger drop on RoboTwin.

Critical strengths
  1. Clear practical motivation. The paper addresses VLA inference latency by improving intermediate action predictions and allocating computation across a small set of exits.
  2. Useful component comparisons. The token and attention ablations show that the complete model achieves better adaptive performance than shared-token and non-collaborative variants on both benchmarks.
Critical weaknesses
  1. Marginal routing gains in the main CALVIN result. With VLM-only initialization, the fixed shallow exit already achieves 4.34 AvgLen at 64.3 ms, compared with 4.36 at 68.9 ms for adaptive inference. The average exit depth of 8.1 further shows that the adaptive policy operates almost entirely at the shallowest exit. Thus, the headline CALVIN speedup primarily demonstrates the strength of the shallow exit, while the incremental contribution of dynamic routing is marginal.
  2. Substantial performance loss on RoboTwin. The reported 2.13× speedup reduces success from 85.5% to 72.7%. Accuracy–latency curves and comparisons at matched computational budgets are needed to assess whether this trade-off is consistently favorable.
  3. Unspecified routing thresholds. Neither the main paper nor Appendix A reports the values or selection procedure for the training error tolerances and inference stopping thresholds. These settings determine when the policy exits and are necessary to reproduce the reported accuracy–latency trade-off.
  4. Missing real-world validation of latency benefits. The paper reports lower policy-query latency only in simulation, without demonstrating benefits in physical robot execution. Real-robot experiments should establish whether these reductions improve task throughput or responsiveness, particularly compared with asynchronous execution strategies such as RTC [1], where applicable.

[1] Real-time execution of action chunking flow policies. Advances in Neural Information Processing Systems, 2026.

Questions for authors

See weaknesses.

Reviewer_kU7xOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents a dynamic VLA architecture. The proposed architecture, named SceVLA, places 3 exits in backbone networks (layers 8/16/24 on CALVIN with VLA-Adapter, 12/24/36 on RoboTwin 2.0 with Qwen-OFT) and uses action token streams at each exits to decide when to exit. The training is two-staged: joint L1 action supervision for all action exits, then the BCE router. Evaluation are mainly conducted on manipulation benchmarks: CALVIN and RoboTwin. The results indicate that although SceVLA largely reduces the inference latency and accelerate the policy by over 2x (2.28x on CALVIN, and 2.13x on RoboTwin), the method shows there are consequential drops in the primary metrics (AvgLen from 4.42 to 4.36, and success rate from 85.5% to 72.7% on RoboTwin).

Critical strengths
  1. Clear and practical architecture. The proposed architecture is simple and easy to implement, and meaningful: replacing a shared token stream with per-exit streams plus collaboration raises AvgLen from 4.12 to 4.36 on CALVIN and success rate from 68.9% to 72.7% on RoboTwin at similar latency. This is the clearest evidence that the architecture itself matters.

  2. Efficient yet with competitive performance. SceVLA shows a promising accuracy-latency trade-off on CALVIN (2.28x speedup with AvgLen decreases from 4.42 to 4.36) and RoboTwin (2.13x speedup with success rate drops from 85.5% to 72.7%).

Critical weaknesses
  1. Insufficient attribution to benefits from proposed components. The experiments do not convincingly establish that dynamic routing combined with collaborative attention outperform substantially than a strong fixed shallow exit; and the extra decoder parameters added for separate action token streams also indicate the comparison is not under the same computational budget, further blurs the attribution.
  2. Latency figures are internally inconsistent. An adaptive policy that always stops at one of three exits cannot be faster on average than the corresponding mixture of its own fixed-exit latencies. Using Table 3's own fixed-exit latencies and average depths, the minimum achievable mean latency is about 86.9 ms for CALVIN SceVLA(R) (reported 78.8 ms), about 79.4 ms for RoboTwin SceVLA(R) (reported 70.7 ms) and about 85.2 ms for RoboTwin SceVLA(V) (reported 79.6 ms), before any router overhead. Three of the four headline speedups therefore cannot be reconciled with the paper's other numbers. Separately, Figure 4 is described as "approximately reconstructed from the supplied visualization, not raw measurements", with individual values replaced by neighbor averages. Together these undermine confidence in the reported results.
Questions for authors
  1. On CALVIN, VLM-only SceVLA exits at an average depth of 8.1 and matches its fixed layer-8 score (AvgLen 4.36 vs. 4.34). Could the authors report a VLM-only model truncated to 8 layers and trained alone with a single action head? Without it, the CALVIN speedup cannot be attributed to multi-exit design or routing mechanism. That is, could an 8-layer backbone for this benchmark already achieve competitive performance and show similar accuracy-latency trade-off?
  2. Could the authors explain how the adaptive latencies in Table 3 can fall below every mixture of the fixed-exit latencies at the reported average depths. How are latency and average depth measured (per query or per action, warm-up, KV cache, router overhead)? Please give a per-component latency breakdown, including the vision encoder.
  3. Why does Figure 4 use curves reconstructed from a visualization rather than raw logged values? Please provide the raw measurements and confirm that no other figure or table was produced this way.
  4. Could the authors provide a detailed description of experiments as well as the “preliminary models” used in Figure 1? The three-exit (8/16/24) curve (≈4.04 AvgLen at layer 8) does not match either SceVLA setting in Table 1 (4.15 / 4.34). Moreover, because the loss is averaged over K exits under a fixed training budget, denser configurations receive less supervision per exit. As it stands, Figure 1 does not adequately support the claim that sparse exits are preferable to dense ones, which is a core motivation of the paper.
  5. VLM-only SceVLA reaches 77.6% success at full depth on RoboTwin 2.0 Clean (Tables 2–3). It is initialized without any robot action data and, per Appendix A, trained only on Clean demonstrations. This is substantially higher than comparable Qwen3-VL-OFT results under similar data: 61.7% in Table 2 of this paper (from VLAct, Base/Clean), and 50.38% in the official StarVLA release for a model trained on 50 clean demonstrations per task (https://github.com/starVLA/VLAct/blob/main/examples/Robotwin/README.md). Could the authors specify: (i) the number of Clean demonstrations per task used for training; (ii) the backbone and its size; and (iii) any differences in training schedule or evaluation protocol that could account for this gap of 16–27 points? If the gain comes from the SceVLA architecture itself, that would be a notable result in its own right and should be analyzed explicitly. If it comes from more data or a different setup, the comparison with the published baselines in Table 2 should be qualified accordingly.
  6. The current comparison only involves DeeR-VLA and MoLe-VLA. Strong dynamic depth baselines (namely DySL-VLA and LoopVLA, as mentioned in Related work) should also be included in comparison.
Reviewer_BqDkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SceVLA combines a VLA backbone with sparse exits, separate action-token streams, and directed collaborative attention. Deeper streams read detached features from shallower streams, while a separately trained router chooses when to stop. Evaluation on CALVIN and RoboTwin 2.0 reports over 2× policy-query speedups, with a small CALVIN performance decrease and a substantially larger RoboTwin decrease relative to the base policies.

Critical strengths

The proposed information flow is technically sensible: collaboration reuses already available features, and routing does not require evaluating future backbone layers. This makes the acceleration mechanism credible.

Table 5 shows an improved observed accuracy–latency trade-off over shared-token and non-collaborative variants on both benchmarks. The use of two backbones and manipulation settings is a useful breadth of evaluation.

Critical weaknesses

On CALVIN, most of the benefit is already obtained by a fixed shallow exit: layer 8 reaches 4.34 AvgLen at 64.3 ms, versus adaptive routing at 4.36 and 68.9 ms. Without uncertainty estimates, the small adaptive gain is difficult to assess. On RoboTwin, adaptive success remains 12.8 percentage points below the full-depth base policy, so the usefulness of routing needs a fuller accuracy–latency curve.

The supplied 12-page PDF repeatedly refers to Appendix A for training, thresholds, action chunks, and timing details, but contains no such appendix. Figure 4 is explicitly reconstructed from a visualization with some values replaced, and the text does not verify that the displayed recommendations controlled the rollout. It therefore provides limited empirical support for the routing interpretation.

Questions for authors

Please supply the referenced Appendix A, including router tolerances, stopping thresholds, threshold-selection data, training budgets, and the precise timing boundary.

Across seeds and threshold settings, does adaptive routing significantly outperform fixed-depth policies at comparable latency? Can you provide raw execution logs supporting the behavior illustrated in Figure 4?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.