acceptodds
Under review as a conference paper at ICLR 2027

When Visual Compression Breaks Control: Efficient Multi-View VLA Inference via Retaining Evidence for Control-Aligned Policy Recovery

Abstract

Aggressive visual-token compression is increasingly important for efficient Vision-Language-Action (VLA) inference. However, we find that compression does more than discard visual information: it also induces a representation shift that can substantially misalign a policy trained on dense visual inputs. We therefore formulate efficient multi-view VLA inference as two coupled problems—preserving action-relevant physical evidence and recovering policy behavior under compression-induced representation shift. To this end, we propose RECAP-VLA, short for Retaining Evidence for Control-Aligned Policy Recovery, as a fixed-budget compression-and-recovery framework. RECAP-VLA organizes cross-view tokens into geometry-grounded physical-evidence components and allocates the compact token budget according to action relevance while preserving global context and residual evidence. It further introduces Compression Amplification Profiling (CAP) to localize policy regions most associated with compression-induced control error, and Control-Aligned Counterfactual Policy Adaptation (CAPA) to recover dense-policy behavior using lightweight same-state dense–compact supervision. At a fixed 512-to-256 token operating point, direct compression reduces success from 91% to 46%, whereas RECAP-VLA restores it to 92%. On 200 matched LIBERO Spatial episodes, RECAP-VLA achieves 91.5% success versus 90.5% for the dense policy while reducing median end-to-end latency by 8.44%. The same recipe remains within 2.5 percentage points of dense performance across additional OpenVLA-family checkpoints and LIBERO tasks, with approximately 8.3% median-latency reduction.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action, efficient VLA inference, multi-view perception, visual token compression, robot learning, embodied AI, policy adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EtpwOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes RECAP-VLA, a framework for reducing two-view VLA inputs from 512 to 256 visual tokens while preserving closed-loop control. Its compactor uses simulator metric depth and calibrated camera geometry to associate cross-view tokens into physical-evidence components. A protected global-context scaffold, a learned action-relevance predictor, and bounded residual aggregation then determine the compact representation.

Critical strengths

The paper includes useful component and recovery controls.

The large improvement from action-level recovery is an interesting empirical observation.

Critical weaknesses

The paper seems like an engineering trick paper, without many theoretical contribution,

The performance improvement is marginal. Reducing 512 tokens to 256 halves the downstream visual sequence, not the entire inference cost: the full visual encoder still runs at deployment. The median saving is approximately 8.092 ms, but mean latency improves by only 2.53%, while p95 latency and peak memory remain essentially unchanged. This benefit may still be useful, but a stage-level breakdown is needed to show the costs of visual encoding, geometry planning, compact policy inference, and action decoding, and whether depth acquisition, transfer, and synchronization are included in “end-to-end” timing.

The main paper does not clearly define which tokens Hard-256 retains, how its budget is allocated across the two views, or how its selection and positional conventions compare with the other variants. It also does not show the key comparison against a simple balanced two-view selector or a 2D importance-based selector equipped with the same CAPA procedure. If Hard-256 uses a particularly weak selection rule, its 45-point collapse cannot establish that reasonable visual-token compression generally causes a failure of comparable magnitude.

Questions for authors

Does CAP improve on fixed final-layer adaptation? Please explain the rule selecting blocks 30–31, provide the existing 96-state diagnostic, and compare a small number of nearby layer choices or random layer pairs while keeping q/v projections, rank, and training budget fixed.

Reviewer_1B71OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

RECAP-VLA compresses visual tokens for two-view VLAs under a fixed budget. Its compactor uses simulator depth and camera calibration to group tokens across views into physical-evidence components. It keeps a 64-token scaffold, fills the rest of the budget by utility-weighted multi-cover, and folds dropped tokens into kept ones as residuals. A recovery stage (CAP/CAPA) picks late blocks with a correlation diagnostic and trains rank-8 LoRA on blocks 30 and 31 to match the dense teacher's actions. On LightVLA/LIBERO-Spatial at 512 to 256 tokens, it reaches 91.5% vs. 90.5% for the dense model, with 8.44% lower p50 latency.

Critical strengths
  1. The ablations are well controlled. Going from Hard-256 (46%) to Evidence (52%) to Hard+CAPA (84%) to the full method (92%) in Table 4 separates compaction from recovery. All recovery baselines get the same parameter count and data (Table 2).
  2. The objective mismatch result will be useful to practitioners. SmoothL1 lowers its own loss but raises control deviation (0.631 to 0.718). Training on D_ctrl with the same adapter raises success from 77% to 92%.
Critical weaknesses
  1. No existing VLA pruning method is compared, and the efficiency gain is small. The only compressed baseline is Hard-256, and its selection rule is not given. EfficientVLA, SpecPrune-VLA, BFA++ and DepthCache are cited but not run at the same budget. Halving the tokens cuts p50 latency by 8.44% and mean latency by 2.53%, and p95 latency and memory are essentially unchanged. The method also needs ground-truth simulator depth and calibration, and a 3% depth perturbation already costs 5 pp.
  2. The statistics are too thin for the claims. The ablations use 100 episodes each, so the 95% CI is roughly ±6 to 10 pp, and the gaps the argument rests on (87, 88 and 86 vs. 92) fall inside it. The headline comparison itself has CI [−4, 6] and p=0.78. CAP layer selection uses only 24 states, all results are in LIBERO simulation, and Supplementary Tables 6, 8 and 10 are cited but not included.
Questions for authors
  1. How does Hard-256 select tokens? LightVLA already prunes tokens itself, so what does "Dense-512" mean here?
  2. Please compare against at least two published pruning methods at 256 tokens, each followed by CAPA.
  3. Please give CIs or multiple seeds for the Table 2 and Table 4 ablations, using the 200-episode paired protocol.
  4. Can you break latency down into encoder, compactor and LLM? Does the method still work with estimated rather than ground-truth depth?
  5. How does it compare with single-view baselines at 256 tokens, such as dropping the wrist view?
Reviewer_UmvjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RECAP-VLA, a framework for efficient multi-view Vision-Language-Action (VLA) inference. The method combines geometry-aware visual token compression with control-aligned policy adaptation to address information loss and representation shift caused by compression. Experiments on LIBERO demonstrate that RECAP-VLA can reduce visual tokens by 50% while maintaining comparable task success rates and reducing inference latency.

Critical strengths

1.The paper addresses an important practical problem in efficient VLA inference by considering both action-relevant information preservation and policy adaptation under visual token compression. 2.The authors conduct extensive experiments, including component ablations, comparisons with different policy adaptation strategies, and evaluations across multiple model checkpoints and LIBERO task suites.

Critical weaknesses

1.Insufficient motivation. Several key claims lack convincing evidence. For example, the impact of compression-induced representation shift (Page 1, Line 42) is not clearly distinguished from information loss. The introduction also focuses too heavily on method details rather than establishing the motivation. 2.Missing strong baselines. The paper lacks direct comparisons with recent action-aware and multi-view VLA pruning methods, making it difficult to assess its advantages over existing approaches. 3.Overly complex design. Evidence-preserving compression alone improves success from 46% to only 52%, while basic compression with policy adaptation already achieves 84%. The necessity of the complex compression mechanism is not sufficiently justified. 4.Limited inference speedup. Despite reducing visual tokens by 50%, the method achieves only 8.44% lower median latency and 2.53% lower mean latency, with no detailed runtime breakdown.

Questions for authors

See above weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.