acceptodds
Withdrawn from ICLR 2027

STAR: Steering Action Representations with Proprioception for VLA Inference

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language-action models (VLAs) map language instructions and visual observations to robot actions, showing strong capabilities across robotic manipulation tasks. However, in a probing experiment, we find that action representations near the action output do not fully preserve task-relevant visual grounding. Motivated by this late-layer grounding gap, we introduce STAR, for STeering Action Representation, a training-free inference method for pre-trained VLAs. STAR applies an exponential moving average over late-layer residual updates, allowing later action representations to inherit object-aware update directions from earlier layers without model updates or architectural changes. It further uses recent proprioceptive states to adapt the mixing strength, applying a conservative mixing when execution dynamics change rapidly. Experiments on robot manipulation benchmarks show that STAR improves late-layer object awareness and increases task success rates.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-language-action Model, Robotic Manipulation, Training-Free

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DSFuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes STAR, a training-free inference method for vision-language-action models. Motivated by a decline in the sensitivity of late-layer action representations to target-object masking, STAR applies an exponential moving average to residual updates across the final third of Transformer layers. The mixing coefficient is dynamically determined from recent proprioceptive signals, including directional continuity, translation magnitude, translational change, and orientation change. Experiments on three VLA backbones across four model–benchmark configurations report an average success-rate improvement.

Critical strengths
  1. The method is lightweight and practically motivated. STAR modifies inference without updating model parameters or requiring additional full-model forward passes. Improving existing policies with limited computational overhead is a useful direction, and the reported real-robot latency overhead is modest.

  2. The evaluation covers multiple model architectures. Improvements are reported for GR00T-N1.5, OpenVLA, and π₀.₅ across SimplerEnv and LIBERO-PRO. This provides broader evidence of applicability than evaluation on a single backbone.

  3. The paper includes useful ablations and diagnostic controls. The experiments examine individual proprioceptive cues, fixed versus dynamic coefficients, and intervention depth. The masking analysis includes same-area background and random-pixel controls, while gradient-based analysis examines the sensitivity of the final actions rather than relying solely on attention visualization.

Critical weaknesses
  1. DAMO already uses the same basic EMA residual-update rule to preserve information across layers and includes adaptive activation refinement and coefficient adjustment. STAR’s main methodological distinction therefore appears to be its proprioception-conditioned coefficient schedule and its application to robotic control. However, the main comparison is against DTP, which intervenes through visual-token pruning rather than residual momentum.

  2. The object-awareness score measures the relative change in hidden states after masking the target object. A decrease in this score across depth does not necessarily imply loss of task-relevant information: it could also reflect changes in representation scale or transformations toward action-relevant features. Figure 1(c) also shows higher awareness at some intermediate layers than at the final layer used for action prediction, raising the question of whether the subsequent decline reflects harmful information loss or useful transformation toward action outputs.

  3. Why should rapid motion changes or fine adjustments require weaker cross-layer EMA, while steady motion warrants stronger mixing? These phases may also require more accurate visual grounding. I think the ablations support the overall design but do not directly validate this directional assumption.

  4. STAR motivates intervention through declining late-layer object awareness but fixes ρ = 2/3 without explaining how to identify the appropriate mixing region for a new model or deployment setting. The limited start-layer ablation does not establish such a procedure. I think the key problem of this paper might be: how to find a \rho for different dataset, or even different observation. I think \rho is variable.

Questions for authors

See weakness

Reviewer_tZszOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates a late-layer visual grounding degradation in pretrained Vision-Language-Action (VLA) models, where action hidden states become progressively less sensitive to task-relevant objects in the final Transformer layers. To mitigate this issue without retraining, the authors propose STAR, an inference-time intervention that applies an exponential moving average (EMA) over layer-wise residual updates in the final third of the Transformer layers (mtk=βtmtk−1+(1−βt)Δhtkm _ t^k = \beta _ t m _ t^{k-1} + (1 - \beta _ t)\Delta h _ t^k, h^tk=htk−1+mtk\hat{h} _ t^k = h _ t^{k-1} + m _ t^k), while modulating the mixing coefficient βt∈[βmin⁡,βmax⁡]\beta _ t \in [\beta _ {\min}, \beta _ {\max}] using four hand-crafted end-effector kinematic finite differences from recent proprioceptive states. Experiments with GR00T-N1.5, OpenVLA, and π0.5\pi _ {0.5} on SimplerEnv, LIBERO-PRO, and a single real-robot pick-and-place task show modest improvements in task success rates.

Critical strengths
  1. The diagnostic analyses in Section 1 and Appendix A (comparing object-masking sensitivity and action-gradient saliency against attention maps) clearly illustrate how late-layer hidden states lose sensitivity to task-relevant objects in pretrained VLAs.
  2. The inference-time modification adds negligible latency (3.3% on an RTX 3090) and is tested across three distinct VLA backbones (GR00T-N1.5, OpenVLA, and π0.5\pi _ {0.5}).
Critical weaknesses
  1. The core method and diagnostic narrative are an almost direct transfer of DAMO [1] from VLMs to VLAs, leaving minimal technical novelty. Specifically, the late-layer visual grounding drop in Figure 1(c), the architecture comparison in Figure 2, the residual momentum update in Equations (3)–(4) (mtk=βtmtk−1+(1−βt)Δhtkm _ t^k = \beta _ t m _ t^{k-1} + (1 - \beta _ t)\Delta h _ t^k and h^tk=htk−1+mtk\hat{h} _ t^k = h _ t^{k-1} + m _ t^k, which are identical to Equations (4)–(5) in DAMO [1]), and the restriction to the final third of layers (ρ=2/3\rho = 2/3) all directly mirror DAMO [1], with the heuristic proprioceptive rescaling of βt\beta _ t in Equations (5)–(8) as the only change.
  2. Despite citing DAMO [1] as the closest prior work in Sections 2.3 and 3.2, Tables 1 and 2 omit both DAMO [1] and a Fixed-β\beta EMA baseline, comparing only against native inference and a two-pass token-pruning method (DTP [2]). Worse, the OpenVLA ablations are internally inconsistent: in Table 3, removing all proprioceptive signals (No proprioceptive evidence, where rt=0r _ t = 0 reduces Equation (8) to a constant βt=(βmin⁡+βmax⁡)/2=0.40\beta _ t = (\beta _ {\min} + \beta _ {\max}) / 2 = 0.40) drops average success to 27.8% (below native inference at 30.3%), whereas in Table 4 every Fixed-β\beta variant across [0.25,0.55][0.25, 0.55] achieves 30.3%–31.6%.
  3. Key implementation details are missing from both the main text and the appendix: the paper never reports the values of the proprioceptive weights (wc,wv,wj,wωw _ c, w _ v, w _ j, w _ \omega) in Equation (6), the window duration TwT _ w in Equation (5), or the sharpness κ\kappa in Equation (8), nor does it explain how Equations (3)–(4) are applied across multi-step flow-matching denoising iterations in π0.5\pi _ {0.5} and GR00T-N1.5 versus single-pass decoding in OpenVLA.
  4. The proprioceptive rule in Equations (5)–(8) relies on brittle kinematics heuristics: assuming large translational steps (vtv _ t) and direction continuity (ctc _ t) always require stronger smoothing while rotation changes (ωt\omega _ t) require less smoothing breaks down in continuous curved or rotational tasks (e.g., pouring, wiping, or screwing), and sliding-window normalization becomes ill-conditioned when the robot pauses (σ(Htx)≈0\sigma(H _ t^x) \approx 0).

[1] Wang et al. 2025. DAMO: Decoding by Accumulating Activations Momentum for Mitigating Hallucinations in Vision-Language Models. ICLR 2025.

[2] Li et al. 2026. DTP: A Simple Yet Effective Distracting Token Pruning Framework for Vision-Language Action Models. arXiv:2601.16065.

Questions for authors
  1. Why are DAMO [1] and Fixed-β\beta EMA omitted from Tables 1 and 2 across GR00T-N1.5, OpenVLA, and π0.5\pi _ {0.5}?
  2. What is the exact algorithmic difference between No proprioceptive evidence in Table 3 (27.8%) and the Fixed-β\beta variants in Table 4 (30.3%–31.6%)?
  3. What are the numerical values of wc,wv,wj,wωw _ c, w _ v, w _ j, w _ \omega, TwT _ w, and κ\kappa, and how is Equation (3) applied across iterative flow-matching denoising steps in π0.5\pi _ {0.5} and GR00T-N1.5?
Reviewer_eacEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

STAR is a training-free inference method that uses recent proprioception to modulate an exponential moving average of late-layer residual updates over action representations. An approximately 4.0-point mean gain is achieved across four main configurations

Critical strengths
  1. Comparisons on the same frozen policies cover SimplerEnv, LIBERO-PRO, and three backbones; all four configurations improve, supporting some transferability of the lightweight intervention.
Critical weaknesses
  1. The chosed backbone is too weak to demonstrate effectness of the proposed method. Also, the paper lacks more results on difficult benchmarks, such as robotwin or robodojo.
Questions for authors

1.What are the trial counts, seeds, and confidence intervals for paired success differences? 2.Do improvements in the appendix grounding metrics predict task-level success gains, or do they only indicate greater input sensitivity? 3. Is this method general enough to be applied to world-action model?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.