acceptodds
Under review as a conference paper at ICLR 2027

AfterImage: Object-Grounded Attention for Vision-Language-Action Policies

Abstract

Vision-language-action (VLA) policies must retrieve the right visual evidence for control even as a scene's appearance changes. Most VLA training leaves this selection to action supervision, while recent approaches guide decoder attention with task-relevant labels. Where such guidance should intervene remains a fundamental question. We identify a cross-modal positional asymmetry in rotary position embeddings: broad positional interventions severely impair control, whereas changes confined to action-to-image attention can preserve task performance. This finding motivates AfterImage, powered by a Conditional Object-Region Attention (CORA) mechanism. AfterImage treats the action stage following image encoding as a place to relearn where to look. It separates the selection of image regions from the total attention allocated to images, so object guidance redistributes an existing visual budget. We turn this principle into an adaptation system that learns from region priors during training. Deployment requires no external perception model and incurs negligible parameter/computational overhead. Attention measurements and closed-loop transfer evaluations connect this intervention pathway to more task-directed visual retrieval and improved manipulation robustness.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Robotic Manipulation, Visual Grounding, Attention Routing, Object-Centric Learning, Rotary Positional Embeddings, Spatial Supervision

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_usvYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AfterImage introduces CORA, a small action-query adapter guided by object-region annotations during training. A log-sum-exp correction preserves each intervened row's total visual attention while redistributing it across image tokens, and the region loss is isolated from backbone gradients. Experiments on π0 and π0.5 show gains on LIBERO-Plus and mixed results on LIBERO-PRO, alongside attention measurements and frozen-policy RoPE interventions.

Critical strengths
  1. The distinction between visual attention mass and within-image allocation is useful. The normalization has a clear, exact local guarantee, making the intended intervention easy to understand.

  2. Deployment is lightweight: the adapter adds 294,912 parameters and requires no object masks or external perception model at inference. This makes the training-time supervision practical to use.

  3. The transfer evaluation is substantial and uses matched initial states and action noise. Appendix E also reports paired significance tests, and the paper openly discusses the positional-generalization trade-off.

Critical weaknesses
  1. The experiments compare CORA with Stock, but not with GuidedVLA or a comparable mask-supervised adapter. There are also no training ablations removing budget conservation or gradient isolation. This leaves the benefit of the proposed mechanisms over object supervision itself unresolved.

  2. The causal and frequency-specific claims exceed the evidence. Higher target attention does not establish why control improves, and removing RoPE from a frozen model does not show that learned low-frequency adapters outperform high-frequency or unrestricted ones (Sections 5.4–5.5).

  3. The improvement is less consistent on the stronger backbone: π0.5 gains 1.76 points on LIBERO-Plus but loses 0.51 overall on LIBERO-PRO, including 3.50 points on Position. This limits the claim of broadly improved robustness.

Questions for authors
  1. How does CORA compare with GuidedVLA or an otherwise matched mask-supervised adapter, including variants without the LSE correction and gradient isolation?

  2. With parameter count and supervision fixed, do learned low-frequency adapters outperform high-frequency and full-band adapters? What controlled evidence supports the causal routing claim?

  3. Are the π0.5 gains and Position/Task regressions stable across independent fine-tuning seeds? How should these trade-offs affect the scope of the robustness claim?

Reviewer_QXnpOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CORA for flow-based vision-language-action policies. It adapts low-frequency action-to-image queries, preserves local visual attention mass through an LSE correction, and trains the adapter with region supervision on detached features. CORA improves LIBERO-Plus success with pi0 and pi0.5 and LIBERO-PRO success with pi0. On LIBERO-PRO, pi0.5 gains on Semantic/Object but regresses on Position/Task and slightly declines overall.

Critical strengths
  1. The LSE correction preserves local visual attention mass while region supervision adjusts its distribution within images.
  2. On LIBERO-Plus, CORA improves pi0 and pi0.5 by 4.61 and 1.76 percentage points. Paired rollouts match initial conditions and action noise, and McNemar tests support the gains.
  3. CORA adapts action-to-image queries while leaving shared image keys unchanged, localizing the attention intervention to visual retrieval by the action expert.
  4. Inference uses standard policy inputs without masks or an external perception model. The adapter adds 294,912 parameters.
Critical weaknesses
  1. TAR mixes within-head target selection with changes in head weights. Appendix E pools heads using each policy's own attention probabilities, making TAR a budget-weighted average of per-head target ratios. These weights can differ between Stock and CORA despite local conservation. TAR can rise by allocating more attention to an already object-focused head, even if neither head selects targets better. Table 3 needs a per-head breakdown and a comparison with fixed head weights.
  2. The claim of non-destructive adaptation lacks control-retention evidence. Equation (14) isolates the backbone from direct region-loss gradients, but the adapter receives both action and region gradients, and the backbone continues to adapt through the modified forward pass. The paper does not analyze how these objectives interact on the adapter. Clean-task results for the same checkpoints and phase-specific gradient measurements would help assess control retention and investigate the pi0.5 Position/Task regressions.
  3. Frequency comparisons do not control intervention magnitude. Removing equal numbers of rotary pairs need not produce equal changes in attention. If low-frequency relative phases are small, removing them is a weaker intervention, providing an alternative explanation for preserved performance. Table 4 also shows that both Low-24 and High-24 removal preserve success. Phase ranges and comparisons matched for induced logit or attention changes are needed to support the proposed frequency-based functional separation.
  4. The attention diagnostic does not resolve task roles or execution stages. Appendix E measures initial observations against unions of task-relevant objects, whereas training uses stage-aware targets. Union-based TAR cannot distinguish appropriate attention to the manipulated object during grasping from attention to the destination during placement. Role- and phase-resolved measurements are needed to connect the diagnostic to task-directed retrieval throughout execution.
Questions for authors
  1. Can you report per-head conditional TAR and visual budgets, and compare Stock and CORA using fixed head weights?
  2. How do the action and region gradients interact on the adapter across task phases, and what clean-task performance do the same checkpoints retain?
  3. Do the frequency conclusions hold when interventions are matched for induced logit or attention changes? Please report the relative phase ranges.
  4. Can you report manipulated-object and destination attention across execution stages, and test target switching under different valid instructions in the same scene? I encourage the authors to focus their response on the following two questions: (i) Under the same backbone, region annotations, demonstrations, and training budget, does CORA outperform direct object-attention supervision? (ii) Does removing budget conservation from the action forward pass reduce performance when the adapter, conditional auxiliary loss, base-budget head weights, and training protocol are matched?
Reviewer_RnCuOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AfterImage, which uses a Conditional Object-Region Attention (CORA) adapter to change how action tokens attend to image tokens. The adapter operates in the low-frequency RoPE dimensions of action-to-image attention. It is trained with object-region masks, while attention normalization keeps the total attention assigned to images unchanged. Masks are not needed at inference. The paper evaluates the method on π0 and π0.5 using LIBERO-Plus and LIBERO-PRO, and studies frozen-policy RoPE interventions on LIBERO and DexJoCo.

Critical strengths
  1. The method offers a focused way to introduce object guidance into a VLA policy. It changes action-to-image attention while preserving the total visual attention budget, so the supervision mainly changes where action tokens look within the images. The region loss updates only the adapter, and no mask or external perception model is needed during rollout. I think this makes the method practically interesting.
  2. The paper reports closed-loop evaluations with matched baselines. On LIBERO-Plus, the overall success rate improves by 4.61 percentage points for π0 and 1.76 points for π0.5. Both models also improve on the Semantic and Object categories of LIBERO-PRO. These results support the usefulness of object-guided attention for at least some visual and object shifts.
Critical weaknesses
  1. I am not fully convinced that the low-frequency RoPE dimensions are the best place to train the adapter. Tables 4 and 5 remove RoPE rotations from frozen policies and show how sensitive the existing policies are to these changes. That intervention differs from training an adapter in those dimensions. In Table 4, removing the highest 24 pairs also has little effect on success. A comparison of trained adapters in low, middle, and high-frequency bands under the same setting would better support this central design choice.
  2. The gains are not consistent across transfer categories, and the explanation of the failure cases needs more evidence. For π0.5 on LIBERO-PRO, Position and Task decrease by 3.50 and 2.15 percentage points, and the overall result is slightly below Stock. The paper suggests that supervision on the manipulated object may make the policy attend less to destinations or placement surfaces. This is plausible, but Appendix C says that stage-aware masks are used without explaining which regions are supervised at different stages. The attention analysis in Table 3 also does not include Position or Task. I therefore cannot tell whether the proposed explanation accounts for these drops.
Questions for authors
  1. Clarification that could change my assessment: What object or region is used as the mask target at each stage of a trajectory, especially after the object has been grasped? Are destination or receptacle regions ever supervised? If they are included, how should I understand the explanation that the method only supervises the manipulated object? If they are excluded, please clarify how the original stage-aware annotations were selected or modified.
  2. Additional evidence that could change my assessment: The trained CORA results are on two models from the π family and LIBERO-derived benchmarks. DexJoCo evaluates frozen-policy RoPE changes, but not a trained CORA adapter. Do the authors have a matched Stock-versus-CORA comparison on an independent benchmark, such as RoboTwin or RoboDojo, or on a different RoPE-based VLA under the same benchmark? If this evidence is not available, I think the conclusion about how generally the adapter helps should be stated more narrowly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.