acceptodds
Under review as a conference paper at ICLR 2027

The Layer Mystery of VLA: An Information-Theoretical Analysis of VLA Latent Interface

Abstract

Vision-language-action (VLA) policies connect a pretrained vision-language backbone to an action head through a latent interface, but which backbone layers this interface should expose remains unclear. We study single-layer selection and multi-layer fusion for frozen backbones across three pretrained models and two manipulation benchmarks, LIBERO and CALVIN, with three policy-training seeds per configuration. Across three fusion mechanisms and three layer-subset strategies, 47 of 54 configurations underperform the best observed single-layer policy. Our stastical analysis further confirms that fusion's advantage is very limited. However, the best layer varies substantially across backbones and benchmarks, making layer selection consequential and exhaustive policy sweeps expensive. We further derive a reweighting equivalence between the proposed information-bottleneck objectives for action-conditioned InfoNCE and action prediction, motivating InfoNCE as a proxy for layer quality. Empirically, InfoNCE provides the most consistent positive association with policy success among four evaluated proxies. Selecting the layer with the highest InfoNCE score requires 9–33 times less GPU compute than exhaustive policy sweeps and reduces mean selection regret from 17.89 percentage points for deepest-layer selection to 3.71 points across six settings. Its mean regret is close to the 3.28–3.50 points achieved by fixed-layer heuristics optimized retrospectively using all six oracle sweeps, without requiring closed-loop evaluations during selection.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language Action Model, Interpretability, Learning Theory

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_toxuOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates layer selection and fusion for the latent interface of VLA policies. Through systematic experiments, the authors find that layer fusion generally underperforms the best single layer, while the optimal layer varies across backbones and benchmarks. To enable efficient layer selection, they propose InfoNCE as a layer-quality proxy, motivated by an information-bottleneck equivalence. Experiments show that InfoNCE-based selection reduces GPU compute by 9–33× and achieves substantially lower selection regret than deepest-layer selection.

Critical strengths
  • Systematic Evaluation of Layer Selection and Fusion: The paper provides a comprehensive comparison of single-layer interfaces and three existing fusion methods across three pretrained backbones and two manipulation benchmarks, using multiple training seeds and statistical analysis. This offers more reliable empirical evidence than prior studies with limited layer comparisons and single-seed evaluations.
  • Practical and Efficient Layer Selection: The proposed InfoNCE-argmax selector avoids expensive policy training and closed-loop evaluations for every layer. It reduces GPU compute by 9–33× compared with exhaustive oracle sweeps while achieving substantially lower selection regret than deepest-layer selection.
  • Consistent Empirical Validation of InfoNCE: The paper evaluates four layer-quality proxies and shows that InfoNCE has the most consistent positive correlation with downstream policy success across the evaluated backbones and benchmarks. This supports its practical utility for identifying effective latent interfaces.
Critical weaknesses
  • Gap Between IB Equivalence and Task Performance: Theorem 1 proves equivalence between the IB objectives (Eqs. 2 and 4), but neither objective is shown to be equivalent to its corresponding task loss (Eqs. 1 and 3). Moreover, the proposed InfoNCE-based selector does not explicitly optimize the IB compression term. Thus, the theoretical equivalence does not establish why a higher InfoNCE score should predict better downstream policy performance.

  • Potentially Under-optimized Fusion Baselines: The paper concludes that fusion generally underperforms the best single layer, yet does not demonstrate sufficient architecture-specific hyperparameter tuning. Extremely large performance drops (e.g., −53.0 percentage points in Table 1) also lack diagnostic analysis. These issues make it difficult to determine whether fusion is inherently less effective or simply insufficiently optimized.

  • Missing Comparisons with Simple Action-Aware Probes: Although InfoNCE is compared with DiME and entropy-based metrics, the paper does not evaluate simple action-conditioned probes, such as linear action regression. Without such baselines, it remains unclear whether InfoNCE itself provides an advantage over simpler methods that exploit the same action supervision.

  • Limited Methodological Novelty: The method applies the established InfoNCE objective [1] to intermediate VLM representations, while its theoretical formulation builds on existing IB frameworks [2, 3]. The contribution therefore lies mainly in applying and evaluating existing techniques for VLA layer selection, rather than introducing a fundamentally new methodology.

    References

    [1] van den Oord et al., Representation Learning with Contrastive Predictive Coding, arXiv, 2018.

    [2] Alemi et al., Deep Variational Information Bottleneck, ICLR, 2017.

    [3] Kawaguchi et al., How Does Information Bottleneck Help Deep Learning?, ICML, 2023.

  • Limited Generalizability Beyond Frozen Backbones: All experiments use frozen VLM backbones. Since fine-tuning can change intermediate representations and their usefulness for action prediction, it remains unclear whether the conclusions about single-layer superiority and InfoNCE-based selection extend to end-to-end trained VLA models.

Questions for authors
  • The Data Processing Inequality implies that mutual information between intermediate representations and expert actions cannot increase with layer depth for deterministic backbones. How do the authors explain the non-monotonic InfoNCE scores observed across layers, and what evidence supports their hypothesis that optimization limitations of InfoNCE critics align with those of downstream action heads?
  • What factors explain the substantial variation in optimal layer depth across backbone–benchmark pairs (e.g., Qwen selecting L3 on LIBERO versus L16 on CALVIN)? Can these differences be attributed to backbone pretraining, task characteristics, or representation properties?
  • Can the authors provide direct evidence that adding layers introduces action-irrelevant information, as hypothesized in Section 3.1, rather than merely making representations harder for the action head to decode or optimize?
Reviewer_nT95OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates how best to incorporate visual and instructional information from pretrained Vision-Language Models (VLMs) into the action expert of Vision-Language-Action Models (VLA) through the lens of information theory. In particular, the paper examines whether representations from a single VLM layer are sufficient or whether other fusion approaches are needed. Additionally, the paper proposes a method for selecting the best VLM layer for a VLA using the InfoNCE loss, motivated by the information bottleneck theory.

The process of selecting and potentially processing the information to be included in the action expert is referred to as the latent interface. First, the paper shows that different fusion-based interfaces cannot outperform a single optimal layer across three base models and two simulation benchmarks, namely LIBERO and CALVIN. Then the authors confirm that the deepest layer does not provide the optimal choice for any of the base models. Lastly, experiments comparing the InfoNCE loss per layer versus the performance of using each VLM layer as a latent interface correlate and can therefore be helpful for layer selection.

Critical strengths
  1. Interesting Information-Theoretical Lens
    The paper addresses an interesting question relevant to VLAs: How much information (and exactly which) is needed from the VLM? The paper aims to provide a theoretical explanation and verifies it through empirical observations. Figure 1 shows a strong correlation between the InfoNCE-based loss term and the success rate, demonstrating that the proposed metric can be used to select a well-performing layer.
Critical weaknesses
  1. Frozen VLM assumption
    The paper assumes that the VLM remains frozen during training, which is consistent with some prior work. However, strong models like pi0 or pi0.5 do not freeze the VLM. As a result, it is reasonable to assume that the representations can more closely adapt to the needs of the action expert. In addition, pi0 or pi0.5 use all layers of the VLM in a mixture-of-experts setting. While the paper mentions this scenario in its "even" case in Table 1, it remains unclear what the exact setting was to produce these results. Lastly, Figure 2 in the Appendix illustrates how widely performance across different layers can vary with the training seed, raising the question of why layers would be so sensitive to the training process. Even across different datasets, the optimal layers can vary widely, which raises the question of how these differences could be so extreme.

  2. Writing and Presentation
    For the most part, the paper is hard to read. In particular, frequent typographical (e.g., "the the", line 188 or "Wo compare", line 412) and grammatical mistakes (e.g., "Our contribution of can be broken down as follow", line 80) make the submission appear more like a rough draft. At the same time, there is a clear break in style, as most of the introduction seems to have been written by an LLM, containing typical AI-like formulations, such as "The decision is consequential: " (l. 38). Overall, it took significant effort to understand the content of the paper.

Questions for authors
  • What exactly is meant by the "even" and "best" scenarios in Table 1?
  • How is the InfoNCE critic trained? There seems to be almost no information regarding the architecture and other important details.
  • Do the GPU hours in the InfoNCE sweep account for the training time of the InfoNCE critic?
  • Why was the training step budget set to 8000 steps? Is there any chance some layers have not converged yet?
Reviewer_BgS8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies which layers of a frozen vision-language backbone should provide representations to a robot's action head. It conducts exhaustive layer sweeps across three backbones on LIBERO and CALVIN, compares single-layer interfaces with adapted fusion methods, and evaluates an InfoNCE-based layer selector. The paper also develops information-theoretic arguments to motivate the analysis.

The strongest contributions are the systematic layer study and the practical evaluation of the selector. The theoretical tightness claim and the scope of the fusion conclusions require qualification.

Critical strengths
  1. The paper provides a systematic empirical study across three backbones and two benchmarks, using three policy-training seeds. This offers useful evidence about a consequential interface choice in frozen-backbone VLA systems.

  2. The fusion comparison produces a clear finding within the tested framework. Table 1 reports that the adapted fusion methods perform below the best observed single layer in 47 of 54 configurations.

  3. InfoNCE is evaluated as an actionable selector rather than only as a correlated diagnostic. Section 4.3 and Table 5 report mean regret of 3.71 percentage points, compared with 17.89 for selecting the deepest layer. Table 2 reports that the exhaustive policy sweep costs approximately 9–33 times as much compute as the InfoNCE sweep. The selected action head still requires training, but the reduction in selection cost is practically relevant.

Critical weaknesses
  1. The fitted-critic diagnostic does not establish the claimed tightness.

Section 4.2 states that Appendix C confirms tightness of the InfoNCE mutual-information bound. Appendix C starts from an optimal density-ratio critic and then substitutes ratios obtained from a fitted critic. The resulting correction does not control critic approximation or optimization error.

For example, if Z=AZ=A is a balanced bit, a constant critic gives a zero score and a zero fitted correction, although the true mutual information is log⁡2\log 2. This is a logical counterexample to the claimed diagnostic interpretation, not a description of the experimental critics.

Figure 4 illustrates the concern: a shallow-layer score plus its correction lies below a deeper-layer score. If these were population quantities and the correction upper-bounded the complete gap, this ordering would contradict Equation (6) and data processing. The held-out estimates illustrate this tension, but do not themselves prove a numerical population violation.

Theorem 1 remains valid, but relates the proposed information-bottleneck objectives. Appendix A.4 motivates an additional action-information term beyond the classification target; it does not establish that the fitted critic optimizes that objective. The tightness claim should therefore be corrected, and the theoretical analysis should be presented as motivation for the empirically evaluated selector.

  1. The fusion conclusion should remain specific to the tested adaptations.

Appendix E.6 specifies fixed layer assignment without learned fusion gates for VLA-Adapter, modified attention and training for APT, and eight selected layers with a changed backbone/head setting for BEHAVIOR.

The shared framework is a reasonable controlled comparison. However, these adaptations do not establish a general limitation of the complete original methods. Appendix B's architecture-only APT control is also restricted to ABot/CALVIN. The main conclusions should explicitly preserve these boundaries.

Questions for authors
  1. Can the authors revise the tightness claim to distinguish the optimal-critic result from the fitted-critic diagnostic, explicitly accounting for the uncontrolled approximation and optimization gap?

  2. Can the main text state more clearly that the fusion findings concern the evaluated adaptations, and summarize the consequential differences from the original methods?

  3. As an optional practical comparison, how does InfoNCE selection perform against a prespecified inexpensive layer-selection baseline? This would sharpen the efficiency assessment, but I do not consider it a prerequisite for acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.