acceptodds
Under review as a conference paper at ICLR 2027

A Phenomenology of Neural Geometry in Vision-Language-Action Models

Abstract

Robotic manipulation is inherently geometric, but what shape does the world take inside the policy that performs it? Prior work on vision-language-action (VLA) policies establishes that task-relevant variables are linearly readable from activations, yet readability certifies only that a variable is present and says nothing about the shape its values trace, which may be curved and multi-dimensional. Using controlled sweeps, natural rollouts, linear probes, and activation interventions, we build a taxonomy of seven task-relevant concepts in VLA models. We find that realized geometry is predominantly multi-dimensional and is shaped by the constraints under which the policy acts. Physical joint limits, for example, leave wrist orientation an open arc rather than a circle, and distance to the target, a single scalar, is realized in two orthogonal subspaces, a coarse ruler for the approach and a fine ruler for the grasp. The same geometries reappear in two further VLA models on real-world robot data, indicating that they are neither model-specific nor artifacts of simulation. We demonstrate how the findings are actionable by steering along the geometry a concept realizes. Overall, this work lays out an initial phenomenology of realized geometry in VLA policies and demonstrates that a concept's shape carries as much importance as its readability.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, VLA, vision language action models, geometry, representations, neural geometry

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_NyLWOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies neural geometry in vision-language-action models. The authors explore whether a concept can be retrieved from activations in the model, and the actual geometry of the concept in the activation space. To be specific, the authors study variables including end-effector position, target position, distance, contact, task phase, wrist orientation, and gripper, showing the interpretability and geometry of all these factors in vision-language-action models. The authors also study some of these observations on real-world data.

Critical strengths
  1. The setting of the paper is meaningful. Studying the geometry and interpretability of vision-language-action models can facilitate further understanding of their internal representations for environment understanding and action generation.
  2. The experiment is overall comprehensive, studying most of the factors related to the environment and manipulation phases. A lot of analysis methods are leveraged to verify different representations.
  3. Some of the observations are insightful. Also, the authors extend some of the observations to real-world settings and other models.
Critical weaknesses
  1. The real-world and cross-model validation is still somewhat limited. Although the authors explore this part in the paper in Sec. 4, such as π0.5 and MolmoAct2 on real-world DROID data, some of the stronger causal analyses in Sec. 3 cannot be reproduced in real-world settings or on other models.
  2. The paper contains a lot of interesting figures, covering a variety of analyses, but some of them are not clearly explained or self-contained in the main text. Some figures are related to important observations in the paper, but the explanations of the basic settings, how such experiments are conducted, and how the results support the claims are too brief in the main text. A reader needs to frequently check the appendix for more information. For example, Figs. 2 and 8. I think it would be better if more important information could be included in the main text, while moving some less important text to the appendix. The authors can better balance the content between the two parts.
Questions for authors

Could the authors clarify how much of the similarity across models may come from using the same labels, rather than from the models actually learning similar representations?

Reviewer_cSiPOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper argues that for vision-language-action (VLA) policies, linear readability of a control variable does not determine the structure of its activations, and that geometry—not readability—governs steerability. Using MolmoBot in simulation, the authors build a taxonomy of seven variables via controlled sweeps, teleported interventions, probes, and activation edits: EE and target positions occupy near-Euclidean rank-3 subspaces; distance is realized in two nearly orthogonal, range-specific subspaces; direction needs only one direction; contact is linearly decodable and survives off-schedule interventions; task phase is decodable but inferred from motion; wrist orientation is an open arc rather than a circle; gripper aperture is steered by a near-1D pullback direction. They further distinguish "belief" edits from "action" edits. Four variables are re-examined in π₀.₅ and MolmoAct2 on real-world DROID data, with cross-model Procrustes similarities above shuffled nulls.

Critical strengths
  1. Iterative erasure and the synthetic single-axis control in Figure 3, together with early-touch and induced-slip interventions in Figure 4, address alternative explanations based on correlated variables.
  2. Compensatory wrist motion and joint leakage during gripper editing are informative observations
Critical weaknesses
  1. Some conclusions conflate readout dimensionality with the dimensionality of the concept’s representation. Section 3.1 interprets position-probe rank curves as showing that three dimensions are necessary and sufficient. Appendix B.3, however, constructs these curves by truncating the principal directions of three-dimensional predictions, whose rank cannot exceed three. This establishes the usefulness of three directions for the chosen linear readout, but not that position-related information occupies only a three-dimensional activation subspace. Displacement and curvature tests provide additional evidence about the selected projection, yet a useful readout subspace should remain distinct from the concept’s full realized geometry.
  2. Near- and far-range distance separation is supported more directly than its proposed explanation through finite numerical precision. Figure 3 and Appendix B.4 show that different ranges can be read along different directions and that one remains readable after erasing directions associated with the other. These results do not identify whether the separation arises from precision constraints, visual scale, training distributions, or task phase. The causal wording on page 3 is stronger than the hypothesis presented in Section 3.1. The evidence supports range-dependent distance coding without establishing its origin.
  3. The cross-model similarity measure partly inherits the structure of shared supervision. Appendix B.10 projects each model through probes trained on the same variables before computing Procrustes similarity and explicitly acknowledges that some agreement is built in. Exceeding a shuffled-label null in Figure 8 shows that the models recover related variables, but does not fully separate shared geometry from shared readout targets. The real wrist sweep also spans only approximately −79° to 84°. An open arc over that interval cannot distinguish part of a closed circle from a nonperiodic curve. The real-data results support recurring trends more strongly than matching topology.
  4. The belief-versus-action distinction remains an interpretation of intervention responses rather than an independently identified mechanism. Compensatory motion in Figure 6 is consistent with a belief edit, but the intervention is constructed through a circle-constrained autoencoder and has a magnitude of approximately 19% of the activation norm. The reported results do not establish whether other action variables remain stable. Gripper edits are directly optimized through action-output gradients, so changing the gripper command is not independent validation of the proposed representation role. Appendix B.9 also states that these gripper action chunks are not executed. The experiments demonstrate different control responses to different edits, while stronger claims about internal roles and closed-loop control remain less established.
Questions for authors
  1. Are the Section 3.1 rank curves obtained solely by truncating three-dimensional predictions? Does “three dimensions are necessary and sufficient” refer to readout dimensionality or intrinsic concept dimensionality in activation space?
  2. Beyond the range-specific probes, erasure tests, and synthetic single-axis control, is there existing evidence distinguishing finite precision from other explanations of two-scale coding? Otherwise, is finite precision intended as an untested hypothesis?
  3. Which cross-model results establish geometric agreement beyond decoding shared labels? Is the real wrist sweep intended to support a conclusion about open topology?
  4. Were changes in other joints, translation, and gripper outputs recorded during wrist interventions? Could existing measurements clarify whether the compensatory response is variable-specific?
Reviewer_cszCOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how seven manipulation variables are organized in the activations of MolmoBot. It separates three questions: whether a variable can be read out, what geometry its representation traces, and how editing that representation affects the policy. Controlled simulator sweeps, held-out-house probes, erasure tests and activation edits reveal position readout subspaces, separate near/far distance information, an open wrist arc, and task-related structure. Wrist edits produce compensatory rotations. Edits derived from the action output can flip decoded gripper commands, but low-dimensional edits also affect the arm outputs. The authors then look for related structures in MolmoBot, π0.5 and MolmoAct2 on DROID recordings. I read the contribution as an empirical study of existing VLA representations.

Critical strengths
  1. The two-scale distance finding is the strongest part of the paper. It combines held-out-house decoding, reciprocal erasure, near/far probes fitted without clipped labels, a cutoff sweep and a synthetic single-axis control (PDF pp. 5, 18–19, 24–25; Figs. 3, 12–14). Together, these checks support separable linear information about near and far distances. The result goes beyond the appearance of the L-shaped plot. It gives useful evidence about how distance information is organized in a trained VLA, even though the reason this structure forms remains open.

  2. The behavioral and output diagnostics make the study informative beyond probe accuracy. Contact is tested off schedule, phase queries are tested with the current frame replacing the history frame, and wrist edits produce a compensatory response (PDF pp. 6–8, 19–21; Figs. 4–7). The gripper experiment is also useful: a dominant edit direction can flip the decoded command, but it changes arm outputs as well. These results show concrete limits of reading and editing a control variable. I think this is a worthwhile empirical contribution without requiring a new policy or higher task success.

Critical weaknesses
  1. The paper should be more precise about which geometry it has established. The position rank curve in App. B.3 is computed on three-dimensional predictions, whose rank is at most three by construction. It does not establish that all position-related activation variation is three-dimensional (PDF pp. 4–5, 18; Fig. 2). The controlled paths and displacement tests do support geometry within the selected readout subspace, and I would keep that result. The cross-model claims need similar care. Procrustes compares coordinates supervised on the same labels, so some agreement is built in, as App. B.10 notes (PDF pp. 9, 22). For real-data distance, the paper shows related readout patterns but does not report the reciprocal erasure or direction comparisons used in simulation (PDF p. 35; Fig. 31). These results support recurring readable structures more clearly than a shared full representation geometry.

  2. The wrist edits support a compensation effect, but the selective belief interpretation remains uncertain. The original activation sweep forms an open arc; the circular autoencoder supplies coordinates for constructing the edit. Its edit has a mean norm of 19% of the activation norm (PDF pp. 7–8, 20; Fig. 6 and App. B.8). Held-out angle calibration, zero-degree cancellation and paired unedited rollouts are useful controls. Still, the paper does not independently check that the edit changes the wrist estimate while leaving other represented variables fixed, or compare with a matched edit that leaves the angle unchanged. The opposite-sign response is consistent with a changed state estimate, but does not isolate that explanation from other effects on the decoder. The one-shot result also uses seven scenes after one exclusion. I would keep the compensation finding and qualify the stronger belief/action conclusion.

Questions for authors
  1. Does "three dimensions are necessary and sufficient" refer only to the chosen position readout?

  2. For the wrist edits, is there an independent check of the post-edit angle and other represented variables, or a matched control that separates a selective state edit from other effects on the decoder?

  3. For the DROID distance codes, are the near/far direction comparisons and reciprocal erasure results available?

Comments on the paper

1 comment

General response

We sincerely thank all three reviewers for their careful reading and constructive feedback. Below, we summarize the new experiments and the main points of discussion.

New Experiments

1. Shared labels

  • Question 1: Could the agreement between models on real DROID data come from fitting the same labels?
  • Exp 1: We ran the same erasure and direction tests inside MolmoBot, π0.5 and MolmoAct2 on DROID data. The results are in Appendix Figure 37.
  • Result 1: In each model, we fit one probe that predicts near distances from activations and one probe that predicts far distances from activations. We then removed from the activations the directions that the far probe uses. After this, the far distances could no longer be read by the probe (e.g. MolmoAct2: R² 0.74 → 0.00), but near distances could still be read as well as before (R² 0.79 → 0.78). Removing the near directions instead had the opposite effect. So, each model stores near and far distances in different directions. In π0.5 and MolmoAct2, these two sets of directions are nearly orthogonal. As just fitting probes on the same labels cannot create this geometry, this supports the claim that the agreement between models is not just from the labels.
  • Question 2: Which cross-model results go beyond decoding the same labels?
  • Experiment 2: We analyzed the gripper edits of App. Fig. 33 in π0.5 and MolmoAct2. These edits are fit through the action output, without labels.
  • Result 2: In each model, as in MolmoBot, the edits point the same way across episodes. This shared direction follows each model's own gripper command. This again points to a similar geometric structure appearing without any labels.

2. Effects of the wrist edit

  • Question: Does the edit change the model's estimate of its wrist angle without changing other variables? Does this effect depend on the autoencoder we trained, or would any edit of that size do the same?
  • Exp: We applied the wrist edit in 34 new scenes from held-out houses. Then, we measured the edit's effect using linear probes that do not use the autoencoder. We compared this effect with a random edit of the same size. We also recorded every joint and the gripper, to see whether the edit leaks into other actions. The results are in Appendix Figure 38.
  • Result: The probes read the edit like the wrist is really rotated at that angle, while other measured variables barely change. The robot turns against the edit mainly at the wrist joint, while other joints and the gripper barely change. Therefore, this wrist edit is isolated to the wrist. The random edit does not shift the probed wrist angle, and the robot does not turn against it.

3. Gripper edits run in a closed loop.

  • Question: Do the gripper edits still work when the robot executes them? App. B.9 measured them only on predicted actions.
  • Exp: We ran the gripper edits in the simulator and had the robot execute them.
  • Result: At layer 9, the edits flip the gripper in 308 of 308 runs. At layer 32, they flip the gripper in 81% of 290 runs.

Clarifications of wording/scope

Other comments, mainly from reviewers cszC and cSiP, helped us state our results more precisely. We will make these clarifications in our camera-ready version:

  • Position: "Three dimensions are necessary and sufficient" refers to a linear readout. Our claim is that the model contains a 3-D, approximately Euclidean position code in a subspace of the activations. We do not claim that position appears nowhere else.
  • Distance: Finite precision is the idea that motivated the distance experiment. It is not a claim or result of the paper. The result is the orthogonal near/far distance subspaces.
  • Wrist: The open wrist arc is shown in simulation, not on the real-robot sweep.
  • Transfer to real data: Sec. 4 tests whether the models learn a similar geometry, and procrustes is just one measure of this. We do not claim that models share the same representation space.

Main findings the reviewers agree with

  • cszC: The reviewer calls the two-scale distance finding "the strongest part of the paper", and writes that "these checks support separable linear information about near and far distances." The controlled paths and displacement tests "do support geometry within the selected readout subspace, and I would keep that result." Overall, the reviewer calls the study "a worthwhile empirical contribution."
  • cSiP: The reviewer agreed that our two-scale distance result is supported, saying: "The evidence supports range-dependent distance coding." The reviewer also mentioned that some of our experiments provide informative observations and do a good job addressing alternative explanations.
  • NyLW: "The experiment is overall comprehensive." Also, "some of the observations are insightful."

We thank the reviewers for their time. Their feedback has helped us make our claims more precise and bolster the experimental evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.