acceptodds
Under review as a conference paper at ICLR 2027

The Linear Representation Hypothesis for Vision-Language-Action Models

Abstract

The linear representation hypothesis (LRH) has become a standard lens for measuring and intervening on semantic information through the internal representations of large language models (LLMs). A growing body of work has begun extending this perspective to vision-language-action (VLA) models, but the dynamical nature of embodied interaction introduces an additional challenge. Unlike semantic attributes commonly studied in LLMs, such as gender or language, a physical quantity of interest (QoI) in a VLA evolves jointly with the system dynamics: the representation influences the actions selected by the policy, which alter the physical state and, in turn, the next representation. In this paper, we develop a theoretical, signature-based formulation of the LRH for VLA that unifies representations and policies. On the representation side, we establish the existence of representations from which the future evolution of a QoI under a candidate action trajectory can be recovered via linear probing. On the policy side, we introduce a signature generalized linear model for stochastic action chunks. This structure yields a monotonic change in the expected future QoI along linear paths in natural parameter space, enabling linear steering. We construct an explicit oracle representation in a planar control-affine navigation experiment and verify the predicted linear probing and steering mechanisms.

Primary area:
interpretability and explainable AI
Keywords:
vision-language-action models, linear representation hypothesis, mechanistic interpretability, path signature method

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_QQbzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper develops a control-theoretic formulation of the linear representation hypothesis for vision-language-action models. It distinguishes probing a current physical quantity from probing its future value under a candidate action trajectory, and defines state-conditioned linear steering through changes in the expected propagated quantity. Under uniform Chen–Fliess conditions, the paper constructs shared finite-dimensional state representations that approximately support action-conditioned linear probes for multiple quantities of interest. On the policy side, it introduces a signature exponential-family abstraction and shows that an exact probe induces an affine natural-parameter path along which the expected quantity changes monotonically. A planar control-affine example with an analytically constructed oracle representation and 671 candidate action chunks numerically verifies the probing identity and steering mechanism. The results concern a constructed idealized system rather than empirical validation on learned VLA representations.

Critical strengths

1. A useful formulation of the distinction between static representation probing and action-conditioned physical prediction.

The paper points out an important limitation of applying the standard static probing view directly to embodied policies: the physical quantity relevant to a decision may depend on both the current state and the candidate action trajectory. The action-conditioned measurement map and state-conditioned steering direction make this dependence explicit. I also found the distinction between decodability and intervention useful: a quantity being linearly readable from a representation does not by itself mean that editing the representation will influence that quantity through the downstream policy. This is a useful way to separate probing from steering.

2. The idealized construction is explicit and easy to inspect.

The existence argument is constructive. Appendix B.1 stacks truncated Lie-derivative coefficients for the selected quantities, and Appendix A gives a uniform truncation-error bound. Under the exact-probe and exponential-family assumptions, the steering analysis relates the response derivative to the variance of the propagated quantity. The numerical example also specifies the dynamics, representation transformation, candidate actions, and base measure, so the reported mean shifts and KL costs are relatively easy to check. The example is clearly presented as an oracle verification rather than as evidence from a learned policy.

Critical weaknesses

1. The existence result does not establish realizability through the observation interface used to define the VLA representation.

The setup defines the representation as an observation-based map, λ(x)=λˉ(h(x),ℓ)\lambda(x)=\bar{\lambda}(h(x),\ell), so the model does not have direct access to the physical state. However, Theorem 4.2 constructs a state-dependent representation from Lie derivatives of the quantities of interest, without requiring these features to factor through the observation map hh. The theorem therefore gives a full-state feature construction, but it does not show that the same construction is realizable under the observation restriction used in the VLA formulation.

This matters independently of how the representation is learned. If two physical states produce the same observation but different propagated quantities under the same candidate action trajectory, any observation-based representation must give them the same probe prediction. The worst-case error is then at least half the difference between the two propagated quantities, regardless of representation dimension. The paper clearly presents the construction as an oracle representation, but the connection to the VLA setting still seems to require an observation-sufficiency assumption, a formulation based on observation histories, or a clearer separation between state-space existence and observation-realizable representations.

2. The probing-to-steering connection relies on an additional compatibility assumption between the representation and the policy.

Section 5 identifies the representation used for probing with the natural parameter of a signature exponential-family policy. This does not seem to follow from the existence result in Section 4 or from signature expressivity alone. In particular, representing an action distribution in signature coordinates does not imply that those coordinates depend affinely on the hidden representation being edited. Under the assumed parameterization and an exact probe, the intervention produces an exponential tilt by the propagated quantity, which is what gives the nonnegative-variance derivative.

The numerical example constructs both the exact probe and the compatible policy by design, so it does not test whether this compatibility holds for a learned representation and policy head. It is therefore still unclear how directly the steering result carries over to learned VLA models. I do not think a large-scale robot benchmark is necessary here, but either a compatibility result for an observation-conditioned policy or a small experiment on a learned representation and action head would make this connection much more convincing.

Questions for authors

1. Observation-based realizability.

Under what assumptions does the representation in Theorem 4.2 factor through the observation map hh, or through an observation history? The theorem constructs a full-state representation from Lie derivatives, while the original VLA formulation restricts the representation to observations. How should the result be interpreted when two physical states have the same observation but different propagated quantities under the same action trajectory?

2. Compatibility between learned activations and policy natural parameters.

What property of the action head would make its signature natural parameters affine in the representation being edited? This seems to be an additional assumption rather than something implied by signature expressivity. Is there a condition that could be checked for a learned policy, without replacing it with the oracle exponential-family policy?

3. Minimality and the dual-coordinate argument.

Appendix C.1 assumes a regular, minimal exponential family and uses the inverse relation between natural and expectation coordinates. However, the signature contains S∅(u)=1S_{\emptyset}(u)=1, and for a fixed horizon some pure-time coordinates are also constant across action paths. These directions should not change the normalized action distribution. How are they removed when defining the minimal exponential family used in the argument? I do not think this affects the variance-based monotonicity result, but it seems relevant to the strict-convexity and inverse-coordinate claims.

4 comments

(Part 1/2)

🎬 Demonstration website (anonymous): https://teal-smakager-b0c294.netlify.app Side-by-side videos of the closed-loop experiments: the original and three rewritten instructions, fixed steering vectors, and our steering.


We thank the reviewer for the thoughtful assessment of our work, particularly for recognizing the distinction between linear decodability and behavioral intervention. We have uploaded a substantially revised manuscript. The new Section 6 directly tests these predictions on pretrained VLAs.

1. Validation on Pretrained VLAs (W1–W2)

We particularly appreciate the reviewer's suggestion: "I do not think a large-scale robot benchmark is necessary here, but either a compatibility result for an observation-conditioned policy or a small experiment on a learned representation and action head would make this connection much more convincing."

We fully agree, and we have directly addressed this concern by extending our experiments to two pretrained VLA models, OpenVLA-OFT and SmolVLA. These models employ substantially different action-generation mechanisms: a deterministic MLP action head and a stochastic flow-matching action expert. Our revised Section 6 investigates whether the two central properties of our framework can be observed in their learned representations.

Action-conditioned linear probing (Figure 2, Table 4). Our signature-based linear probes predict the chunk-induced changes in physical QoIs with held-out R2=0.90R^2=0.90–0.980.98 on OpenVLA-OFT and 0.790.79–0.960.96 on SmolVLA, outperforming conventional direct and additive probes. More importantly, even at matched parameter counts, nonlinear MLP readouts and unstructured MLPs improve the mean R2R^2 by at most 0.0050.005 on OpenVLA-OFT and 0.0010.001 on SmolVLA, and widening them to 256256 hidden units adds at most 0.0080.008 and 0.0070.007, respectively, providing empirical evidence for the proposed linear structure in learned VLA representations. Please see lines 385–389 for detailed descriptions of each method.

Probe-derived linear steering (Tables 1–2). We further demonstrate that this information can be used to modify the actual behavior of pretrained VLAs. By accounting for the action head through its Jacobian, our method produces monotonic responses in 138/140 held-out states on OpenVLA-OFT, with cosine similarity 0.980.98–1.001.00 to the simulator oracle gradient. On SmolVLA, it achieves monotonic responses in 83% of the 53 oracle-steerable states, with median cosine similarity 0.920.92.

Interestingly, these results directly support the reviewer's observation that decodability does not necessarily imply successful intervention. Conventional static probe directions often steer physical quantities in the wrong direction, whereas our action-conditioned, head-aware directions are much more successful at intervention. Furthermore, our closed-loop experiments demonstrate meaningful behavioral modification without additional fine-tuning or data collection (Section 6.3 and Figure 3). In a separate behavioral redirection experiment, representation steering successfully manipulates an alternative mug in 19/2519/25 trials, whereas modifying the language instruction alone fails in 75/7575/75 trials (three rewritten instructions). We further compare with fixed steering vectors from language models, CAA (Rimsky et al., 2024) and the ITI probe direction (Li et al., 2023), applied at the same site with the same step size. Built from the same information as ours, they succeed in at most 10/25 at their best strength, against 19/25 for ours; a CAA vector built from hand-labelled task phases reaches 16/25. Ours is also ahead of every baseline at α=32\alpha = 32 and 4040. This mirrors the reviewer's point: directions that encode the target but ignore the current state and action steer less reliably than our state- and action-dependent direction. For more details, please check our demonstration website (https://teal-smakager-b0c294.netlify.app) showcasing our new experimental results.

We believe these additions directly address the central limitation identified in the review: the proposed probing and steering mechanisms are no longer demonstrated only in a simulator oracle construction, but are empirically validated on pretrained VLA models.

(continued in the next comment)

(Part 2/2)

2. Representation–Policy Compatibility (W2, Q2)

We agree that directly identifying the representation with the natural parameter was restrictive in our original formulation. We have therefore generalized the signature exponential-family parameterization by introducing a differentiable mapping η(x)=g(λ(x))\eta(x)=g(\lambda(x)) (Definitions 5.1–5.2), separating the learned VLA representation from the policy natural parameter. This is motivated by the universal approximation properties of path signatures, which allow continuous path functionals, including suitable policy log-density ratios, to be approximated through linear signature functionals. Theorem 5.3 preserves monotonicity along the natural-parameter path, while the new Corollary 5.4 provides a local steering direction in representation space.

Crucially, our experiments do not require the actual action heads to exactly satisfy the signature exponential-family assumption or explicitly identify gg. Instead, we work with the PCA-reduced representation z(x)=(1,Pλ(x))z(x)=(1,P\lambda(x)) and directly estimate the representation-to-expected-signature map ϕ(z)=Eu∼π(⋅∣z)[S≤2(u)]\phi(z)=\mathbb{E}_{\mathbf{u}\sim\pi(\cdot\mid z)}[S_{\leq 2}(\mathbf{u})]. The steering direction is obtained by pulling back the probe direction through the action head:

zψ(x)=Dϕ(z(x))⊤z~ψ(x),z~ψ(x)=Lψ⊤z(x).z_\psi(x)=D\phi(z(x))^\top\widetilde z_\psi(x), \qquad \widetilde z_\psi(x)=L_\psi^\top z(x).

This provides a practical connection between our theoretical steering principle and learned VLA action heads, without asserting exact exponential-family compatibility.

3. Additional Clarifications (W1, Q1, Q3)

Observation-based realizability. Our framework does not require reconstructing the entire physical state, but only retaining sufficient information about task-relevant QoIs and their action-conditioned evolution. For Theorem 4.2, a sufficient condition is that the relevant Lie-derivative coefficients LIψj(x)L_I\psi_j(x) factor through the observation map hh. Such dynamical information may also be available through sufficiently informative observation histories, as motivated by classical observability and delay-embedding theory. For example, a single image of a pendulum may reveal its angle, whereas consecutive observations can additionally reveal its direction of motion and angular velocity. We agree that observationally indistinguishable states with different propagated QoIs cannot admit an exact observation-based probe. Our new experiments provide empirical evidence that the required dynamical information is nevertheless accessible in the evaluated pretrained representations.

Minimality of the exponential family. We thank the reviewer for identifying this technical point. The empty-word and fixed-horizon pure-time signature coordinates are constant across action paths and therefore do not affect the normalized policy distribution. More generally, minimality requires restricting to identifiable parameter directions, or equivalently quotienting out all directions that contribute only action-independent constants. We clarify this restriction in the beginning of Appendix C.1. Importantly, the variance-based monotonicity result itself does not require minimality, since it follows directly from the exponential-tilting covariance identity.

We hope these substantially extended experiments and theoretical clarifications address the reviewer's main concerns and strengthen the connection between our framework and practical pretrained VLA models.

References

  • Li, K., Patel, O., Viégas, F., Pfister, H., and Wattenberg, M. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model. NeurIPS, 2023.
  • Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. M. Steering Llama 2 via Contrastive Activation Addition. ACL, 2024.
Reviewer_QQbzOpenReview ↗

Raise my score from 2 to 3

Thank you for the substantial revisions and additional experiments. The evaluations on OpenVLA-OFT and SmolVLA are particularly helpful and directly address my concern that the original results were demonstrated only through an oracle construction. I also appreciate the comparisons between static probe directions and directions that account for the learned action head, which make the distinction between decodability and effective intervention clearer.

The revised policy parameterization and the clarification of non-identifiable signature coordinates also address important parts of my theoretical questions.

My remaining reservations mainly concern the scope of the conclusions. The state-space existence result still requires an observation-sufficiency condition to connect it to the VLA observation interface, while the representation-space steering guarantee is local. The new experiments provide encouraging evidence in the evaluated setting, and I would encourage the authors to keep these distinctions explicit when relating the theoretical results to pretrained VLAs.

Overall, the additional experiments and theoretical clarifications substantially address my main concerns and strengthen the paper. Some reservations remain regarding observation-based realizability and the scope of the steering results, but the revisions have improved my assessment. I am therefore raising my score from 2 to 3.

We sincerely thank the reviewer for the careful reading of our revision and for raising the score.

We agree with both remaining points. Theorem 4.2 is an existence result in state space, and relating it to the VLA observation interface requires the observation-sufficiency condition discussed above; our probing results provide empirical evidence for the evaluated representations rather than a verification of that condition. We will keep these distinctions explicit when relating the theory to pretrained VLAs.

Thank you again for the constructive feedback, which has clearly improved the paper.

Reviewer_gxqqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper extends the linear representation hypothesis (LRH) from LLMs to vision-language-action (VLA) models. Its central conceptual move is to argue that embodied quantities of interest (QoIs) should be probed in their propagated form, psi(F_u(x)): the value of a physical quantity after executing a candidate action trajectory u from state x.

On the probing side, Theorem 4.2 uses the Chen-Fliess expansion. It shows that for analytic control-affine dynamics there exist a representation lambda(x) and linear measurement maps L_psi such that psi(F_u(x)) is approximated, uniformly on compact sets, by a bilinear pairing <lambda(x), L_psi S(u)>. Here S(u) is the path signature of the action trajectory.

On the steering side, the paper models the policy as a signature exponential family, pi_lambda proportional to mu * exp<lambda, S(u)>, with the representation serving as the natural parameter. Theorem 5.3 shows that moving lambda along lambda_psi = L_psi* lambda changes the expected propagated QoI monotonically, with derivative equal to Var[psi(F_u(x))]. Appendix C.1 shows that this path arises as a KL projection onto constant-QoI hyperplanes.

Validation consists of an oracle construction on a planar control-affine navigation system (dp_y/dt = theta * v) for which the Chen-Fliess series terminates exactly. The policy is a Boltzmann distribution over 671 hand-designed action chunks. The experiments confirm the exact probe, the monotone steering response, the variance-slope identity, and saturation at the extremal candidate. No learned representation or VLA model is evaluated.

Critical strengths
  1. A novel and well-motivated framing. Action-conditioned probing addresses a real limitation of the static probes used in existing VLA work. Embodied decision-making depends on the predicted consequences of actions, not just on the current state. Using path signatures as the action-side analogue of unembedding vectors is a natural and elegant way to formalize this. This framing is the main reason the paper is worth engaging with, and it could productively shape future empirical probing studies.

  2. A concrete, testable prediction from the variance-slope identity. The result that the policy's variance in predicted QoI governs both the rate of steering and where it saturates gives practitioners a diagnostic for when representation steering will or will not be effective. This could be checked on real models.

Critical weaknesses
  1. A novel and well-motivated framing. Action-conditioned probing addresses a real limitation of the static probes used in existing VLA work. Embodied decision-making depends on the predicted consequences of actions, not just on the current state. Using path signatures as the action-side analogue of unembedding vectors is a natural and elegant way to formalize this. This framing is the main reason the paper is worth engaging with, and it could productively shape future empirical probing studies.

  2. A concrete, testable prediction from the variance-slope identity. The result that the policy's variance in predicted QoI governs both the rate of steering and where it saturates gives practitioners a diagnostic for when representation steering will or will not be effective. This could be checked on real models.

Questions for authors
  1. In Theorem 4.2, lambda lies in R^{p*d_N}, while Section 5 pairs lambda in H_S directly with S(u). For p > 1, how does the stacked multi-QoI representation serve as the natural parameter of the policy? What inner product on H_S is intended?

  2. Why should the backbone representation coincide with the natural parameter of the action distribution, rather than being related to it by an unknown nonlinear map? Is there an argument that diffusion or flow-matching policies are approximately of signature exponential-family form?

  3. Can you state a prediction of the framework that a pretrained VLA could fail? Without one, it is hard to assess the paper as an LRH for VLAs rather than as an existence result.

Suggestions (these could raise my score, but I do not expect them in a rebuttal):

  1. A robustness result for approximate probes looks straightforward. Suppose psi = g + e with |e| <= epsilon, and the policy is tilted along the probe direction g. Then dJ/dalpha >= sd_alpha(g) * (sd_alpha(g) - epsilon). Monotonicity therefore holds wherever the policy's spread in predicted QoI exceeds the probe error.

  2. Under exact probes, steering along psi_1 changes E[psi_2] at rate Cov(psi_1, psi_2). Characterizing this cross-QoI interference would connect to causal separability (Park et al., 2024) and would substantiate the multi-QoI claim.

  3. An experiment with a learned policy, even a small one on this navigation task, comparing action-conditioned and static probes on held-out trajectories.

5 comments

(Part 1/3)

🎬 Demonstration website (anonymous): https://teal-smakager-b0c294.netlify.app Side-by-side videos of the closed-loop experiments: the original and three rewritten instructions, fixed steering vectors, and our steering.


We sincerely thank the reviewer for the thoughtful and detailed review. We have uploaded a substantially revised manuscript. The revision directly addresses the reviewer's main questions through a generalized representation–policy formulation, new falsifiable predictions, and experiments on pretrained VLAs.

1. Representation–Policy Compatibility and Signature Exponential Family (Q1–Q2)

We appreciate these insightful questions, which identify an important ambiguity in our original formulation. We have substantially revised Section 5 to distinguish the VLA representation from the natural parameter of the policy.

In the original manuscript, we directly identified the representation λ\lambda with the natural parameter of the signature exponential family. As the reviewer correctly points out, this identification is restrictive, particularly for the stacked multi-QoI representation of Theorem 4.2 and the nonlinear action heads used in modern VLAs.

To address this, we now introduce a differentiable mapping gg such that

η(x)=g(λ(x)),g:Rd→HS,\eta(x) = g(\lambda(x)), \qquad g:\mathbb{R}^d \rightarrow \mathcal{H}_S,

where λ(x)\lambda(x) is the VLA representation and η(x)\eta(x) is the natural parameter of the signature exponential family (Definitions 5.1–5.2). This separates the representation space from the policy's natural-parameter space, allowing nonlinear relationships between them. Importantly, the main theoretical argument is preserved: Theorem 5.3 establishes monotonicity in natural-parameter space, while the new Corollary 5.4 provides a local linear steering direction in representation space.

The motivation for this parameterization of VLA policies comes from the universal approximation properties of path signatures. For a fixed state xx, the signature map S(u)S(\mathbf{u}) provides a rich feature representation of action trajectories. Under suitable regularity conditions, continuous path functionals can be approximated by linear functionals of these features. In particular, taking a reference path measure μ\mu, we consider approximating the policy log-density ratio as

log⁡dπ(⋅∣x)dμ(u)≈⟨g(λ(x)),S(u)⟩HS−A(g(λ(x))).\log \frac{\mathrm{d}\pi(\cdot\mid x)}{\mathrm{d}\mu}(\mathbf{u}) \approx \langle g(\lambda(x)),S(\mathbf{u})\rangle_{\mathcal{H}_S} - A(g(\lambda(x))).

This motivates the signature exponential family as a flexible statistical abstraction of action distributions, including those induced by diffusion- or flow-based policies. We emphasize that signature universality motivates the expressive form of this approximation, rather than guaranteeing that a particular pretrained action head exactly satisfies it. We have clarified this motivation in the revised Section 5 (lines 284–323).

(continued in the next comment)

(Part 2/3)

2. Falsifiable Predictions and Validation on Pretrained VLAs (Q3, Suggestion 6)

We agree that falsifiable predictions in pretrained VLAs are essential for establishing the practical relevance of our framework. This was a key limitation of our original submission, and we have replaced the oracle-only numerical verification with extensive experiments on two pretrained VLAs: OpenVLA-OFT and SmolVLA. We would particularly appreciate the reviewer's attention to the new Section 6.

Our framework makes two concrete empirical predictions:

First, action-conditioned probing (Figure 2, Table 4). If the proposed linear structure is present in learned representations, the action-conditioned evolution of a physical QoI should be accurately predicted by a bilinear pairing between the representation and the action signature. The proposed probes achieve held-out R2=0.90R^2=0.90–0.980.98 on OpenVLA-OFT and 0.790.79–0.960.96 on SmolVLA, outperforming direct and additive probes. Even at matched parameter counts, nonlinear MLP readouts and unstructured MLPs improve the mean R2R^2 by at most 0.0050.005 on OpenVLA-OFT and 0.0010.001 on SmolVLA, and widening them to 256256 hidden units adds at most 0.0080.008 and 0.0070.007, respectively. Please see lines 385–389 for detailed descriptions of each method. This provides evidence that learned representations encode information about the dynamical consequences of candidate actions.

Second, probe-derived steering (Tables 1–2). If this information is behaviorally meaningful, the probe-derived direction, appropriately transported through the action head, should induce the predicted local change in the physical QoI. We test this by comparing the resulting direction with the oracle simulator gradient. On OpenVLA-OFT, our method produces monotonic responses in 138/140138/140 held-out states, with cosine similarity 0.980.98–1.001.00 to the oracle gradient (Table 2). On SmolVLA, a stochastic flow-matching policy, it achieves monotonic responses in 83% of the 53 oracle-steerable state–QoI pairs, with median cosine similarity 0.920.92 (Table 3).

Importantly, these predictions are falsifiable and need not hold universally across pretrained VLAs. Poor held-out action-conditioned prediction would challenge the proposed linear structure, while systematic disagreement between probe-derived steering and the oracle would reveal limitations of its behavioral interpretation. Indeed, our SmolVLA experiments identify such limitations in states with bimodal physical outcomes. Near contact-mode transitions, even small variations in action chunks can produce sharply different outcomes, challenging the smooth signature-based approximation and leading to unreliable steering directions. At these states, the probe's predictions carry almost no information about the realized outcome (rank correlation 0.020.02), and the failure is not caused by gradient-estimation error. These findings characterize a concrete failure regime that we observed in experiments. We further describe the details in Appendix D.4.

Furthermore, our closed-loop experiments demonstrate meaningful behavioral changes without additional policy fine-tuning or data collection (Section 6.3, Figure 3). We further compare with fixed steering vectors from language models, CAA (Rimsky et al., 2024) and the ITI probe direction (Li et al., 2023), applied at the same site with the same step size. Built from the same information as ours, they succeed in at most 10/25 at their best strength, against 19/25 for ours; a CAA vector built from hand-labelled task phases reaches 16/25. Ours is also ahead of every baseline at α=32\alpha = 32 and 4040. We have also prepared additional experimental and demonstration materials at https://teal-smakager-b0c294.netlify.app. We believe these results directly address the reviewer's concern that the framework was previously supported only by an oracle construction.

(continued in the next comment)

(Part 3/3)

3. Robustness and Cross-QoI Interactions (Suggestions 4–5)

We appreciate these suggestions, which identify meaningful extensions of the theoretical framework.

Regarding approximate probes, the reviewer correctly observes that a useful robustness characterization follows from the covariance identity. If the true propagated QoI is ψ(Fu(x))\psi(F_\mathbf{u}(x)) and its approximation is ψ^(Fu(x))\widehat \psi(F_\mathbf{u}(x)), with ∣ψ−ψ^∣≤ϵ|\psi-\widehat \psi|\leq\epsilon, then under exact exponential tilting by ψ^\widehat \psi,

ddαEπα[ψ]=Cov⁡πα(ψ,ψ^)≥σπα(ψ^)(σπα(ψ^)−ϵ).\frac{\mathrm{d}}{\mathrm{d}\alpha}\mathbb{E}_{\pi_\alpha}[\psi] =\operatorname{Cov}_{\pi_\alpha}(\psi,\widehat \psi) \geq \sigma_{\pi_\alpha}(\widehat \psi)\bigl(\sigma_{\pi_\alpha}(\widehat \psi)-\epsilon\bigr).

where σπα(ψ^):=Var⁡πα[ψ^]\sigma_{\pi_\alpha}(\widehat{\psi}) := \sqrt{\operatorname{Var}_{\pi_\alpha}[\widehat{\psi}]} denotes the standard deviation of the predicted QoI under the steered policy. Thus, monotonicity is preserved whenever the variation in the predicted QoI sufficiently exceeds the approximation error. A more general treatment of policy-family mismatch remains an important direction for future work.

We also agree that cross-QoI interactions deserve further investigation. Under the exact signature exponential-family formulation, steering along the natural-parameter direction associated with ψ1\psi_1 changes the expected value of ψ2\psi_2 according to

ddαEπα[ψ2]=Cov⁡πα(ψ1,ψ2).\frac{\mathrm{d}}{\mathrm{d}\alpha}\mathbb{E}_{\pi_\alpha}[\psi_2] =\operatorname{Cov}_{\pi_\alpha}(\psi_1,\psi_2).

Our construction in Theorem 4.2 establishes existence by stacking the Lie-derivative coefficients of multiple QoIs, without attempting to exploit their shared structure. In practice, these coefficients may overlap substantially, suggesting the possibility of more compact representations and a systematic characterization of cross-QoI interactions. We view this as a promising extension of the present framework.

We hope that the revised theoretical formulation and, particularly, the new experiments on pretrained VLAs address the main concerns raised in the review and clarify the broader applicability of our approach.

References

  • Li, K., Patel, O., Viégas, F., Pfister, H., and Wattenberg, M. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model. NeurIPS, 2023.
  • Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. M. Steering Llama 2 via Contrastive Activation Addition. ACL, 2024.
Reviewer_qWmEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adapts the linear representation hypothesis to vision-language-action models. It defines action-conditioned linear probing and steering for physical quantities of interest (QoIs), uses the Chen–Fliess expansion to show that a finite-dimensional representation with signature-based linear probes exists (Theorem 4.2), and proves that steering along a linear path monotonically increases the expected QoI when the policy is a signature exponential family (Theorem 5.3). A planar navigation example with an oracle representation illustrates both results. The theory is clean, but it is not yet connected to actual VLA models.

Critical strengths
  1. The paper is easy to read for a theory paper. Definitions 3.1 and 3.2 make clear what probing and steering mean for a VLA before any construction is introduced, and Figures 2 and 3 tie the exponential-tilting argument to the navigation example.

  2. Action-conditioned probing is a useful idea. A static probe reads a quantity from the current state, whereas for a VLA what matters is how that quantity evolves under a candidate action chunk. Theorem 4.2 (Appendix B.1) builds one shared representation for a finite set of QoIs, with uniform approximation control over K×UHK \times \mathcal{U}_H, and identifies the trajectory signature as the action-dependent part of the readout.

  3. The steering result is clean, and its assumptions are stated explicitly. Under an exact probe and a signature exponential-family policy, Theorem 5.3 gives ddαJψ,x(α)=Var⁡πλα[ψ(Fu(x))]≥0\frac{d}{d\alpha}J_{\psi,x}(\alpha)=\operatorname{Var}_{\pi_{\lambda_\alpha}}[\psi(F_{\mathbf{u}}(x))]\geq 0. Figures 2 and 3 check the predicted relation between variance and slope numerically, instead of only showing favorable trajectories.

Critical weaknesses
  1. The theoretical representation may require information that a VLA cannot observe. Theorem 4.2 builds a representation from the full physical state. A VLA only receives observations, such as images. Different physical states can produce the same observation but lead to different outcomes under the same action. The paper does not explain when the required representation can be recovered from these observations. This leaves a gap between the existence result and the VLA setting.

  2. The steering result relies on an assumption that has not been verified in actual VLAs. Theorem 5.3 assumes a signature exponential-family policy, with the representation as its natural parameter. The paper does not show whether diffusion- or flow-based VLA policies satisfy this assumption. It also provides no practical test for checking it. The scope of the steering guarantee therefore remains unclear for learned VLA policies.

  3. The experiment tests a hand-built example rather than a learned VLA. The representation is constructed from known dynamics and the chosen quantity of interest. The policy is also designed to satisfy the theory. This verifies the proposed construction, but does not show that a trained VLA learns the same structure. A small test on a trained VLA would help establish that connection. Otherwise, the claims should be limited to the idealized setting studied here.

Questions for authors
  1. What condition on hh would make λN\lambda_N factor through the observations, and can Theorem 4.2 be restated under that condition? This would show whether the result applies to partially observed settings or only to full-state control.

  2. Is there a testable condition, for example on local log-density ratios, under which a diffusion or flow-matching policy approximately satisfies the assumptions of Theorem 5.3? Such a condition would make the practical scope of the steering result much clearer.

5 comments

(Part 1/3)

🎬 Demonstration website (anonymous): https://teal-smakager-b0c294.netlify.app Side-by-side videos of the closed-loop experiments: the original and three rewritten instructions, fixed steering vectors, and our steering.


We appreciate the reviewer's careful and constructive feedback. To address the main concerns, we have substantially revised the manuscript. We hope these materials provide useful context for the responses below, particularly the new experimental evidence presented in Section 6.

1. Validation on Pretrained VLAs (W2–W3)

We particularly appreciate the reviewer's concluding observation in the paper summary: "The theory is clean, but it is not yet connected to actual VLA models." We agree that establishing this connection was the most important missing piece in our original submission. To directly address this concern, we have substantially revised Section 6 to validate the proposed linear probing and steering framework on two pretrained VLA models, OpenVLA-OFT and SmolVLA.

Action-conditioned linear probing (Figure 2, Table 4). Our signature-based linear probe, ⟨λ(x),LψS(u)⟩\langle \lambda(x),L_\psi S(\mathbf{u})\rangle, predicts the future consequences of candidate action chunks for various physically meaningful quantities of interest (QoIs). On held-out episodes, the proposed approach achieves R2=0.90R^2=0.90–0.980.98 on OpenVLA-OFT and 0.790.79–0.960.96 on SmolVLA, outperforming conventional direct and additive probing methods. Notably, even at matched parameter counts, nonlinear MLP readouts and unstructured MLPs improve the mean R2R^2 by at most 0.0050.005 on OpenVLA-OFT and 0.0010.001 on SmolVLA, and widening them to 256256 hidden units adds at most 0.0080.008 and 0.0070.007, respectively. This indicates that the proposed linear structure captures most of the predictable action-conditioned variation in the evaluated QoIs. Please see lines 385–389 for detailed descriptions of each method.

Probe-derived linear steering (Tables 1–2 and Figure 3). We further demonstrate that the probed information can be used to steer the actual behavior of pretrained VLAs. By incorporating the effect of the action head through its Jacobian, we construct a local representation-space steering direction that systematically changes the resulting physical QoI. On OpenVLA-OFT, our method produces monotonic responses at 138/140 held-out states, with cosine similarity 0.980.98–1.001.00 to the simulator oracle gradient. On SmolVLA, which employs a stochastic flow-matching action expert, it achieves monotonic responses in 83% of the 53 oracle-steerable states, with median cosine similarity 0.920.92.

Moreover, our closed-loop experiments demonstrate meaningful behavioral modification without additional fine-tuning or data collection (Section 6.3). In particular, representation steering succeeds in redirecting the policy to manipulate an alternative mug in 19/25 trials, whereas changing the language instruction alone fails in 75/75 trials (three rewritten instructions). We further compare with fixed steering vectors from language models, CAA (Rimsky et al., 2024) and the ITI probe direction (Li et al., 2023), applied at the same site with the same step size. Built from the same information as ours, they succeed in at most 10/25 at their best strength, against 19/25 for ours; a CAA vector built from hand-labelled task phases reaches 16/25. For more information, please refer to the accompanying demonstration website (https://teal-smakager-b0c294.netlify.app).

We believe these results directly address the central concern that the proposed framework was previously verified only through an oracle construction. The revised experiments provide empirical evidence that both action-conditioned linear decodability and probe-derived steering are present in the evaluated learned VLA representations.

(continued in the next comment)

(Part 2/3)

2. Observation-Based Representation and Partial Observability (W1, Q1)

We would first like to clarify that our formulation does not require the VLA representation to reconstruct the entire physical state. Although we use the notation λ(x)\lambda(x) to emphasize its dependence on the underlying physical state, the actual representation is obtained from observations o=h(x)o=h(x) and language instructions ℓ\ell through the VLA backbone, i.e., λ(x)=λˉ(h(x),ℓ)\lambda(x)=\bar{\lambda}(h(x),\ell), as defined in Section 1.

Our central hypothesis is that the representation encodes task-relevant QoIs and sufficient dynamical information to predict their evolution under candidate actions. This is closely related to classical observability and delay-embedding theory, where sufficiently informative partial observation histories can recover the true state's dynamical information unavailable from a single observation. For example, a single image of a pendulum may reveal its angle, whereas consecutive observations can additionally reveal its direction of motion and angular velocity.

More precisely, the relevant condition is not necessarily the injectivity of the observation map hh itself, but whether the dynamical information required for predicting a particular QoI is preserved. For instance, if two states produce the same observation and also satisfy

h(x)=h(x′)⟹ψ(Fu(x))=ψ(Fu(x′)),∀u∈UH,h(x)=h(x') \quad \Longrightarrow \quad \psi(F_{\mathbf{u}}(x))=\psi(F_{\mathbf{u}}(x')),\qquad \forall \mathbf{u}\in\mathcal{U}_H,

then the propagated QoI can be characterized through the observation without requiring complete state reconstruction. For the finite-order construction in Theorem 4.2, a sufficient condition is that the corresponding Lie-derivative coefficients LIψj(x)L_I\psi_j(x) factor through the observation map. Alternatively, an informative observation history can serve as the observation variable under appropriate observability conditions.

Importantly, whether this information is actually encoded in learned VLA representations remains an empirical question. Our new experiments in Figure 2 and Table 4 directly investigate this hypothesis: the strong action-conditioned probing results on OpenVLA-OFT and SmolVLA provide evidence that their learned representations retain substantial information about the dynamical consequences of candidate actions.

(continued in the next comment)

(Part 3/3)

3. Policy Compatibility and the Signature Exponential Family (W2, Q2)

We appreciate this question, as the relationship between the learned representation and the policy natural parameter was indeed restrictive in our original formulation. We have therefore revised Section 5 to introduce a differentiable map η=g(λ)\eta=g(\lambda), rather than identifying the representation λ\lambda directly with the natural parameter.

The motivation comes from the universal approximation properties of path signatures. For a fixed state, the signature map provides a rich representation of candidate action trajectories, allowing continuous path functionals to be approximated by linear functionals of S(u)S(\mathbf{u}) under suitable conditions. In particular, for a stochastic policy and a suitable reference measure μ\mu, this motivates approximating its log-density ratio as

log⁡dπ(⋅∣λ(x))dμ(u)≈⟨g(λ(x)),S(u)⟩HS−A(g(λ(x))).\log\frac{\mathrm{d}\pi(\cdot\mid \lambda(x))}{\mathrm{d}\mu}(\mathbf{u}) \approx \langle g(\lambda(x)),S(\mathbf{u})\rangle_{\mathcal{H}_S}-A(g(\lambda(x))).

This provides a flexible signature exponential-family abstraction while allowing nonlinear relationships between the backbone representation and the policy distribution. Theorem 5.3 preserves the monotonicity result along the natural-parameter path, while the new Corollary 5.4 characterizes the corresponding local steering direction in representation space. We have clarified this motivation and the revised parameterization in Section 5 (lines 284–323).

The formulation also suggests a concrete diagnostic for stochastic policies. Under an exact signature exponential-family representation, the log-density ratio between policies conditioned on two representations λ\lambda and λ′\lambda' is affine in the signature:

log⁡dπ(⋅∣λ′)dπ(⋅∣λ)(u)=⟨g(λ′)−g(λ),S(u)⟩HS−[A(g(λ′))−A(g(λ))].\log\frac{\mathrm{d}\pi(\cdot\mid\lambda')}{\mathrm{d}\pi(\cdot\mid\lambda)}(\mathbf{u}) = \langle g(\lambda')-g(\lambda),S(\mathbf{u})\rangle_{\mathcal{H}_S}-\bigl[A(g(\lambda'))-A(g(\lambda))\bigr].

This relation can be tested approximately by examining whether estimated log-density ratios admit accurate linear signature fits on held-out action trajectories. We emphasize that this is a diagnostic implied by the model, rather than a property that we assume has already been verified for arbitrary pretrained VLA architectures.

Crucially, our new experiments do not require explicitly identifying gg or assuming exact signature exponential-family compatibility. Instead, we work with the PCA-reduced latent representation z(x)=(1,Pλ(x))∈Rqz(x)=(1,P\lambda(x))\in\mathbb{R}^q, where PP is fitted on training episodes. Using the pretrained action head, we directly construct the representation-to-expected-signature map ϕ(z)=Eu∼π(⋅∣z)[S≤2(u)]\phi(z)=\mathbb{E}_{\mathbf{u}\sim\pi(\cdot\mid z)}[S_{\leq 2}(\mathbf{u})] and obtain the local steering direction

zψ(x)=Dϕ(z(x))⊤z~ψ(x),z~ψ(x)=Lψ⊤z(x).z_\psi(x)=D\phi(z(x))^\top\widetilde z_\psi(x), \qquad \widetilde z_\psi(x)=L_\psi^\top z(x).

This enables probe-derived steering for both deterministic and stochastic action heads, without explicitly identifying the natural-parameter map gg.

References

  • Li, K., Patel, O., Viégas, F., Pfister, H., and Wattenberg, M. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model. NeurIPS, 2023.
  • Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. M. Steering Llama 2 via Contrastive Activation Addition. ACL, 2024.
Reviewer_Yga7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how the Linear Representation Hypothesis (LRH), originally studied in large language models, can be extended to vision-language-action (VLA) models. The key motivation is that physical quantities in embodied systems are inherently dynamic: their future values depend not only on the current state but also on the actions generated by the policy and the underlying system dynamics.

To address this challenge, the authors introduce an action-conditioned formulation of linear probing and steering. On the representation side, the paper leverages the Chen–Fliess expansion and path signatures to construct representations that support approximately linear prediction of future quantities of interest (QoIs) under candidate action trajectories. On the policy side, the authors introduce a signature-based exponential-family formulation and establish conditions under which linear interventions in representation space monotonically change the expected future QoI.

The theoretical results are illustrated through a planar control-affine navigation example, where an oracle representation and a compatible stochastic policy are explicitly constructed. The experiments verify the predicted probing and steering behavior, including the relationship between the steering response and the variance of the QoI.

Overall, the paper provides an interesting theoretical perspective on representation interpretability in embodied systems. However, the connection between the proposed framework and representations learned by practical VLA models remains insufficiently established in the evaluated submission.

Critical strengths
  1. The paper highlights the importance of action-conditioned probing in VLA models, where future physical quantities depend on both the current state and the executed actions. This provides an interesting perspective beyond conventional static representation probing.

  2. The use of path signatures and the Chen–Fliess expansion provides a principled connection between representations, system dynamics, and policy interventions. The variance-based characterization of monotonic steering is particularly interesting.

Critical weaknesses
  1. The theoretical results rely on full-state information and a specific exponential-family policy whose natural parameters are directly linked to the representation. However, practical VLA models operate on partial observations and typically employ complex nonlinear action heads. It remains unclear whether the constructed representations and the proposed steering guarantees can be realized in learned VLA models.

  2. The experiments use a simplified navigation system with explicitly constructed representations and policies that satisfy the theoretical assumptions. While useful for verifying the mathematical framework, they provide limited evidence that the proposed probing and steering mechanisms generalize to learned VLA models. Even a small-scale experiment with a learned representation and policy would substantially strengthen the paper.

Questions for authors
  1. The representation constructed in Theorem 4.2 depends on physical states and their associated dynamical properties. Under what assumptions can this representation be obtained from the partial observations available to a VLA model? Would the theoretical guarantees remain meaningful when the observation function is not injective?

  2. What motivates the assumption that the internal representation can serve directly as the natural parameter of the signature-based exponential-family policy? Are there conditions under which this assumption can be justified or approximately satisfied by commonly used VLA action heads?

  3. How robust is the proposed monotonic steering mechanism to approximation errors in the linear probe or deviations from the assumed policy family? Could the authors provide theoretical bounds or characterize scenarios in which steering may fail?

  4. Can the authors clarify which conclusions should be interpreted as general properties of VLA representations and which hold only for the constructed oracle representation and policy? In particular, what empirical observations would falsify the proposed hypothesis in a learned VLA model?

  5. Could the authors provide additional evidence using a learned representation or policy, even on a simplified control task, to demonstrate that the predicted action-conditioned probing and steering properties are not exclusive to the oracle construction?

5 comments

(Part 1/3)

🎬 Demonstration website (anonymous): https://teal-smakager-b0c294.netlify.app Side-by-side videos of the closed-loop experiments: the original and three rewritten instructions, fixed steering vectors, and our steering.


We thank the reviewer for the constructive feedback and thoughtful assessment of our theoretical contributions. We have uploaded a substantially revised manuscript. We refer the reviewer to these materials, particularly the new experimental evidence presented in Section 6, when considering our responses below.

1. Main Concern: Validation on Pretrained VLAs (W1–W2, Q4–Q5)

We particularly appreciate the reviewer's concluding summary: "However, the connection between the proposed framework and representations learned by practical VLA models remains insufficiently established." We fully agree that this was the most important limitation of our original submission, and addressing it has been our central priority in the revision. To directly address this concern, we have substantially extended our experiments to two pretrained VLA models: OpenVLA-OFT and SmolVLA.

Our experiments directly investigate the two central predictions of the proposed framework:

  • Action-conditioned linear probing (Figure 2). On held-out episodes, our linear probes achieve R2=0.90R^2=0.90–0.980.98 on OpenVLA-OFT and 0.790.79–0.960.96 on SmolVLA, consistently outperforming conventional direct and additive probes. Even at matched parameter counts, nonlinear MLP readouts and unstructured MLPs improve the mean R2R^2 by at most 0.0050.005 on OpenVLA-OFT and 0.0010.001 on SmolVLA, and widening them to 256256 hidden units adds at most 0.0080.008 and 0.0070.007, respectively. This suggests that the proposed linear structure captures most of the predictable action-conditioned variation in the evaluated QoIs. Please see lines 385–389 for detailed descriptions of each method.
  • Probe-derived linear steering (Tables 1–2). By transporting the probe-derived direction through the representation-to-action interface, we can steer physical QoIs without modifying the pretrained policy weights. On OpenVLA-OFT, our direction produces monotonic responses in 138/140 held-out states, with cosine similarity 0.980.98–1.001.00 to the simulator oracle gradient. On SmolVLA, which employs a stochastic flow-matching action expert, it achieves monotonic responses in 83% of the 53 oracle-steerable states, with median cosine similarity 0.920.92. These learned-VLA experiments test the corresponding local representation-space consequence of the framework rather than the exact SEF assumptions of Theorem 5.3.

Beyond these local interventions, our closed-loop experiments (Section 6.3 and Figure 3) demonstrate meaningful behavioral changes through representation steering. In a separate behavioral redirection experiment, steering succeeds in manipulating the alternative mug in 19/25 trials, whereas changing the language instruction alone fails in 75/75 trials (three rewritten instructions), without additional policy fine-tuning or demonstration collection. We further compare with fixed steering vectors from language models, CAA (Rimsky et al., 2024) and the ITI probe direction (Li et al., 2023), applied at the same site with the same step size. Built from the same information as ours, they succeed in at most 10/25 at their best strength, against 19/25 for ours; a CAA vector built from hand-labelled task phases reaches 16/25. Ours is also ahead of every baseline at α=32\alpha = 32 and 4040. We also attach a demonstration website (https://teal-smakager-b0c294.netlify.app) showcasing our new experimental results.

Together, these results provide evidence that the proposed probing and steering mechanisms are not exclusive to the constructed oracle representation and policy.

(continued in the next comment)

(Part 2/3)

2. Partial Observability and Representation Realizability (W1, Q1)

We would first like to clarify the intended scope of our representation hypothesis.

Our framework does not require a VLA representation to reconstruct the full physical state. Rather, it requires sufficient information about task-relevant QoIs and their action-conditioned evolution.

Recovering dynamical information from partial observations is closely related to classical observability and delay-embedding theory. Under suitable conditions, sufficiently informative observation histories can reveal otherwise unobserved state variables. For example, a single image of a pendulum may reveal its angle, whereas consecutive observations can additionally provide information about its angular velocity. This intuition is related to classical observability results and the delay embedding theorem, although their applicability depends on the underlying dynamical and observation assumptions.

More importantly, our new experiments directly investigate whether such dynamical information is encoded in pretrained VLA representations. The strong action-conditioned probing performance on both OpenVLA-OFT and SmolVLA provides empirical support for this hypothesis.

3. Signature Exponential-Family Parameterization (W1, Q2)

Our motivation for introducing the signature exponential family (SEF) is to provide an interpretable statistical geometry connecting representation interventions to changes in the action distribution, rather than to claim that all existing VLA action heads exactly follow this parameterization.

The construction is motivated by the universal approximation properties of path signatures. Under suitable regularity assumptions, continuous functionals u↦F(u)∈R\mathbf{u} \mapsto F(\mathbf u)\in \mathbb R of action trajectories, including policy log-density ratios, can be approximated by linear signature functionals:

log⁡dπηdμ(u)≈⟨η,S(u)⟩HS−A(η).\log\frac{\mathrm d\pi_{\eta}}{\mathrm d\mu}(\mathbf{u}) \approx \langle \eta,S(\mathbf{u})\rangle_{\mathcal H_S} - A(\eta).

Importantly, we have generalized the original formulation by introducing a differentiable mapping g:Rd→HSg : \mathbb{R}^d \to \mathcal{H}_S

η(x)=g(λ(x)),\eta(x) = g(\lambda(x)),

from the learned representation λ\lambda to the policy natural parameter η\eta (revised Section 5, Definition 5.1–5.2). This relaxes the original identification of the representation with the natural parameter, allowing nonlinear representation–policy relationships.

Theorem 5.3 establishes monotonicity along the natural-parameter path

ηα=η(x)+αηψ(x),\eta_\alpha = \eta(x)+\alpha\eta_\psi(x),

under the SEF assumptions, whereas the new Corollary 5.4 and the learned-VLA experiments concern the corresponding local steering direction in representation space.

Crucially, our pretrained-VLA experiments do not require explicitly identifying gg or assuming that the actual action heads exactly satisfy the SEF model.

Instead, we work with the PCA-reduced latent representation z(x)=(1,Pλ(x))∈Rqz(x)=(1,P\lambda(x))\in\mathbb{R}^q, where PP is fitted on training episodes. Using the pretrained action head, we directly construct the representation-to-expected-signature map ϕ(z)=Eu∼π(⋅∣z)[S≤2(u)]\phi(z)=\mathbb{E}_{\mathbf{u}\sim\pi(\cdot\mid z)}[S_{\leq 2}(\mathbf{u})] and obtain the local steering direction

zψ(x)=Dϕ(z(x))⊤z~ψ(x),z~ψ(x)=Lψ⊤z(x).z_\psi(x)=D\phi(z(x))^\top\widetilde z_\psi(x), \qquad \widetilde z_\psi(x)=L_\psi^\top z(x).

This enables probe-derived steering for both deterministic and stochastic action heads, without explicitly identifying the natural-parameter map gg.

(continued in the next comment)

(Part 3/3)

4. Robustness and Scope of the Claims (Q3–Q4)

We agree that quantitative robustness guarantees under probing errors and deviations from the assumed policy family are important directions for further theoretical investigation. Regarding approximate probes, if the true propagated QoI is ψ(Fu(x))\psi(F_\mathbf{u}(x)) and its approximation is ψ^(Fu(x))\widehat \psi(F_\mathbf{u}(x)), with ∣ψ−ψ^∣≤ϵ|\psi-\widehat \psi|\leq\epsilon, then under exact exponential tilting by ψ^\widehat \psi,

ddαEπα[ψ]=Cov⁡πα(ψ,ψ^)≥σπα(ψ^)(σπα(ψ^)−ϵ).\frac{\mathrm{d}}{\mathrm{d}\alpha}\mathbb{E}_{\pi_\alpha}[\psi] =\operatorname{Cov}_{\pi_\alpha}(\psi,\widehat \psi) \geq \sigma_{\pi_\alpha}(\widehat \psi)\bigl(\sigma_{\pi_\alpha}(\widehat \psi)-\epsilon\bigr).

where σπα(ψ^):=Var⁡πα[ψ^]\sigma_{\pi_\alpha}(\widehat{\psi}) := \sqrt{\operatorname{Var}_{\pi_\alpha}[\widehat{\psi}]} denotes the standard deviation of the predicted QoI under the steered policy. Thus, monotonicity is preserved whenever the variation in the predicted QoI sufficiently exceeds the approximation error. However, our primary objective is to establish a principled formulation of action-conditioned probing and steering and to identify sufficient structural conditions under which their theoretical properties hold. A more general treatment of policy-family mismatch remains an important direction for future work.

Moreover, our experiments provide a concrete practical scenario of the proposed hypothesis: failure to predict action-conditioned QoI changes or failure of probe-derived directions to produce the predicted physical effects would challenge its applicability to learned VLA representations. Indeed, our SmolVLA experiments reveal a limitation of the proposed steering mechanism in states with multimodal physical outcomes. In our theoretical result, we have assumed the continuity of the variation of the physical QoI with respect to the action chunk u\mathbf{u}. But, near contact-mode transitions, small variations in action chunks can lead to sharply different outcomes, challenging the smooth signature-based approximation and reducing alignment with the oracle steering direction. At these states, the probe's predictions carry almost no information about the realized outcome (rank correlation 0.020.02), and comparisons with finite-difference and oracle gradients indicate that the failure is not caused by gradient-estimation error. We investigate these failures in Appendix D.4.

We hope that the substantially extended experiments, together with the clarified representation–policy formulation, address the central concerns raised in the review and strengthen the connection between our theoretical framework and practical pretrained VLA models.

References

  • Li, K., Patel, O., Viégas, F., Pfister, H., and Wattenberg, M. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model. NeurIPS, 2023.
  • Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. M. Steering Llama 2 via Contrastive Activation Addition. ACL, 2024.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Revisions and Responses to All Reviewers

🎬 Demonstration website (anonymous): https://teal-smakager-b0c294.netlify.app


We thank all reviewers for their constructive feedback. As the discussion period draws to a close, we just wanted to emphasize some of our most prominent revisions to the main reviewer concerns once more. We have uploaded a substantially revised manuscript.

1. Connection to pretrained VLAs (all reviewers)

The shared concern was that the framework was supported only by an oracle construction. Section 6 now tests its two predictions on two pretrained VLAs with different action heads: OpenVLA-OFT (deterministic) and SmolVLA (stochastic flow matching).

  • Action-conditioned linear probing (Figure 2). Held-out R2=0.90R^2=0.90–0.980.98 on OpenVLA-OFT and 0.790.79–0.960.96 on SmolVLA, above direct and additive probes. MLP probes of the same size improve the mean R2R^2 by at most 0.0050.005, and wider MLPs (256256 units) by at most 0.0080.008.
  • Probe-derived steering (Tables 1–2). Transporting the probe direction through the action head gives monotonic responses in 138/140 held-out states on OpenVLA-OFT (cosine 0.980.98–1.001.00 to the simulator gradient), and in 83% of the 53 oracle-steerable states on SmolVLA (median cosine 0.920.92). Static probe directions often steer in the wrong direction, so decodability alone does not imply effective intervention.
  • Closed-loop redirection (Section 6.3, demo). Steering moves the alternative mug in 19/25 held-out trials, whereas three rewritten instructions fail in all 75 trials. Fixed steering vectors from language models (CAA, Rimsky et al., 2024; ITI probe direction, Li et al., 2023), built from the same information, reach at most 10/25 at their best strength; a CAA vector from hand-labelled task phases reaches 16/25. Videos are on the website.

2. Representation–policy compatibility (all reviewers)

The revision separates the VLA representation from the natural parameter of the signature exponential family through a differentiable map, η(x)=g(λ(x))\eta(x) = g(\lambda(x)). Theorem 5.3 keeps monotonicity in natural-parameter space, and the new Corollary 5.4 gives the local steering direction in representation space. The experiments do not require identifying gg or an exact exponential-family action head.

3. Partial observability (Yga7, qWmE, QQbz)

The framework does not require reconstructing the full state, only information about the task-relevant quantities and their action-conditioned evolution. For Theorem 4.2, a sufficient condition is that the relevant Lie-derivative coefficients factor through the observation map. The probing results are empirical evidence that the evaluated representations retain this information.

4. Robustness and scope (Yga7, gxqq, QQbz)

For an approximate probe ψ^\widehat\psi with ∣ψ−ψ^∣≤ϵ|\psi-\widehat\psi|\leq\epsilon, monotonicity holds whenever the standard deviation of ψ^\widehat\psi under the steered policy exceeds ϵ\epsilon. The steering guarantee is local, and the experiments agree: closed-loop success peaks at α=56\alpha = 56 (19/25) and drops at α=80\alpha = 80 (1/25). On SmolVLA, steering fails at contact-mode transitions, where the outcome changes discontinuously with the action chunk; there the probe carries almost no information about the outcome, and the failure is not a gradient-estimation error (Appendix D.4).

We hope the summary helps clarify the scope of our revisions. We want to thank the reviewers and the AC again for their time and consideration.

References

  • Li et al. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model. NeurIPS, 2023.
  • Rimsky et al. Steering Llama 2 via Contrastive Activation Addition. ACL, 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.