acceptodds
Withdrawn from ICLR 2027

A2DB: Affordance as a Distillation Bottleneck for Compressing Vision-Language-Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language-action (VLA) models offer broad manipulation capabilities but remain costly to train and deploy. Standard knowledge distillation requires compact students to reproduce dense teacher representations that contain control-irrelevant information. We introduce A2DB, an affordance-bottleneck VLA distillation framework that transfers only the interaction affordance information needed to determine where and how to act. Automatic affordance labels supervise the frozen teacher probe, whose scaled leading first-layer singular directions define a compact affordance subspace. A2DB follows a two-stage distillation strategy. Phase I distills an external-view student VLM through selective feature matching, affordance supervision, and language alignment. Phase II freezes this student VLM and trains an affordance-conditioned action expert with behavior cloning, sliced Wasserstein action distillation, and cycle consistency; deployment retains only the student VLM and action expert. Across four LIBERO suites, A2DB reaches 76.7% average success with 0.29 B parameters and 38.6 ms latency, outperforming standard knowledge distillation by 11.6 percentage points while using about 91% fewer parameters and 51% lower latency than the teacher. Phase I controls show that leading probe-weight directions and singular-value scaling outperform rank-matched and alternative extraction rules, with performance approaching a plateau at . Phase II controls identify a near-saturated sampling regime and show that action-distribution guidance and aligned cycle supervision are both important. Affordance predictions, attention maps, label-corruption tests, and failure analysis connect the gains to object, contact, and motion cues, while identifying contact and motion precision as the dominant residual errors and motivating evaluation across seeds, perceptual shifts, embodiments, and real robots. Code will be released after the paper is accepted.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Knowledge Distillation, Affordance Learning, Efficient Robot Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZpY1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes A2DB, an affordance-guided distillation framework for compressing vision-language-action models. It uses a teacher-side affordance probe to select features for student distillation, followed by training an affordance-conditioned action expert. On four LIBERO suites, the 0.29B-parameter student achieves 76.7% average success, outperforming standard knowledge distillation by 11.6 percentage points.

Critical strengths
  1. Relevant compression objective and promising matched-scale results. Selecting manipulation-relevant teacher information is a useful direction for VLA compression. At the same reported student parameter count and inference latency, A2DB improves average success over standard KD by 11.6 percentage points.
  2. Substantial component-level evaluation. The paper includes controls for random and PCA subspaces, leading versus trailing singular directions, singular-value scaling, action sampling, and inverse-model training. These comparisons provide evidence that the selected representation and action-training objectives contribute within the evaluated configuration.
Critical weaknesses
  1. Unexplained teacher performance gap. The teacher achieves 84.4% average success, compared with approximately 94.2% in published π0 results [1]. Its checkpoint and fine-tuning setup are not sufficiently specified to explain this gap.
  2. Unclear baseline training. The BC-only ablation uses an already distilled VLM, but its relationship to the main-table BC baseline is unclear. The training recipes for standard KD and SmolVLA also need clarification.
  3. Insufficient justification for the training complexity. The method combines multiple losses and separate representation and action training stages. The paper does not compare this procedure with joint training or simpler alternatives.
  4. Limited evaluation. Experiments are restricted to LIBERO, with no real-robot evaluation or additional simulation benchmarks. Such evaluations would provide stronger evidence that the method transfers across environments and robotic platforms.

[1] Driess et al., Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better, Table 1.

Questions for authors

See Weaknesses.

Reviewer_fyEbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

A2DB compresses a π₀ vision-language-action teacher into a single-view student through an affordance-guided transfer subspace. Automatically generated object masks, contact-location proxies, and motion directions supervise a probe on frozen teacher features. Singular-value-scaled leading directions of the probe's first layer define the target for student feature matching. A second stage freezes the student representation and trains an affordance-conditioned action expert using demonstrations and teacher action samples. Table 1 reports 76.7% mean LIBERO success for the 0.29B student, compared with 65.1% for standard distillation and 62.2% for behavior cloning. Appendix E.1 specifies a separately trained student for each suite.

Critical strengths
  1. The 0.29B student improves mean success by 11.6 percentage points over standard distillation and 14.5 points over behavior cloning at the same parameter count and reported 38.6 ms per action chunk. It remains 7.7 points below the teacher, with approximately 91% fewer parameters and 51% lower measured chunk inference latency on an RTX 4090. These results establish a useful compression tradeoff in the tested setting and are the strongest reason to consider acceptance.

  2. On LIBERO Spatial, removing selective feature matching while retaining affordance and language supervision reduces success from 76.4% to 65.4%; full hidden-state matching gives 65.6%, random directions 67.8%, and PCA directions 68.2% (Table 2). Appendix Tables 7–9 further examine direction selection and scaling. Direct affordance supervision also matters: removing it gives 63.6%. Together with the Phase II ablations, these controls provide useful knowledge about combining selective transfer and policy training, while leaving the stronger semantic interpretation open.

Critical weaknesses
  1. Appendix E.5 reports one training seed and one simulator seed, and most mechanism controls are restricted to Spatial. The four-suite mean aggregates separate policies and does not establish variation across retraining. The SmolVLA baseline also lacks its weight source, training data, camera inputs, and replanning configuration. Its original paper reports 87.3% mean LIBERO success under a different multitask, per-action replanning protocol (Shukor et al. 2025); this paper reports 72.7%. Those protocols prevent a direct numerical comparison, and the missing recipe makes the claimed 4.0-point advantage difficult to assess. These gaps are fixable, but currently keep my recommendation on the weak-rejection side: the same-size gains are compelling, while their stability and the broader comparison remain insufficiently established.

  2. Equation (2) defines “contact” from the median initial coordinates of tracked object points and motion from first-to-last displacement; neither directly measures gripper contact or manipulation phases. The globally pooled probe supplies directions for positionwise matching in Eq. (8), without a direct localization check. Figure 5 uses targets from the same automatic pipeline, so it cannot establish physical contact accuracy or uniquely attribute failures to geometric error. Empty valid-track sets and near-zero displacements also need separate treatment. The overall pipeline is understandable, but these definitions and key rollout conditions require substantial cross-reference to the appendix. The abstract's “only” affordance-transfer claim should be limited to Phase I feature matching, since language alignment, demonstrations, and teacher action samples also enter training. Related preprints offer useful context: ActDistill uses action-guided selective transfer (Ye et al. 2025), Kong et al.'s AffordVLA aligns features to an external affordance teacher (Kong et al. 2026), and UAD generates affordance targets for distillation (Tang et al. 2025). The probe-SVD pipeline remains a specific contribution; missing comparisons to these preprints are not a basis for my rejection recommendation.

References

Kong, Weijie, Zhian Su, Wei Yu, and Huixu Dong. 2026. “AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignment.” arXiv:2605.17517. https://arxiv.org/abs/2605.17517.

Shukor, Mustafa, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, et al. 2025. “SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics.” arXiv:2506.01844. https://arxiv.org/abs/2506.01844.

Tang, Yihe, Wenlong Huang, Yingke Wang, Chengshu Li, Roy Yuan, Ruohan Zhang, Jiajun Wu, and Li Fei-Fei. 2025. “UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation.” arXiv:2506.09284. https://arxiv.org/abs/2506.09284.

Ye, Wencheng, Tianshi Wang, Lei Zhu, Fengling Li, Guoli Yang, and Hengtao Shen. 2025. “ActDistill: General Action-Guided Self-Derived Distillation for Efficient Vision-Language-Action Models.” arXiv:2511.18082. https://arxiv.org/abs/2511.18082.

Questions for authors
  1. Can you provide per-task success counts and variation across independent training runs, and test a central selective-matching ablation on another suite? This would determine how much weight to place on the reported gains beyond the current seeds and Spatial controls.

  2. Which weights, training data, camera inputs, and replanning settings were used for SmolVLA, and how were they aligned with A2DB's rollout protocol? This could change my assessment of the cross-architecture comparison.

  3. How are empty valid-track sets and near-zero displacement handled? Do held-out annotated contacts/motions or spatial localization measurements support the physical meaning of the targets and projected features? Such evidence could strengthen the claimed mechanism beyond the current automatic-label and success-rate checks.

Reviewer_D9LaOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose A2DB framework for compressing VLA by using affordance masks as an information bottleneck in KD. They extract affordance from a frozen Pi-0 teacher and then distill a student VLM followed by an action expert. Overall this idea is interesting and relatively novel, with some great performance across benchmarks.

Critical strengths
  • The motivation behind using affordance as a bottleneck for distilling knowledge is interesting and novel. The intuition that not all teacher's knowledge is relevant is interesting and clearly stated.

  • Ablation studies are detailed and above average

  • The 91% parameter reduction and 51% latency reduction while maintaining 76.7% success is really a huge achievement

Critical weaknesses
  • All the experiments are performed in sim environment on benchmark datasets, and missing real-robot evaluations. Real-world eval is necessary for sim-to-real transfer and understanding real embodiment performance. Although the authors list this as a limitation and future work section, this significantly hurt the paper's claims

  • The authors have cited GLAD, VITA-VLA, LatBot, CEED-VLA, BitVLA, etc. but none of them have been included in the baseline comparison, whereas they are similar/concurrent VLA distillation methods, could the authors answer why? It'd be interesting to see if the proposed framework outperforms the other standard distilllation approaches (latent distillation and/or action distillation, e.g., GLAD, ActDistill, etc.)

  • The claim that SVD of the first-layer of weight identified affordance-related direction has no significant evidence guarantee. The probe is a 2-layer MLP. The first layer's SVD captures input directions that the probe uses, but there is no guarantee these are the only or best affordance directions. The second layer or the non-linearity could also contribute to those.

  • The affordance labels are generated by grounding Dino to Sam2 to CoTracker, which could basically lead to cascading effect of error propagation. There is no failure analysis of the annotation pipeline. Figure 10 show that random masks and object swaps drop success to 0%. But there is no analysis of how often the automatic pipeline actually fails on LIBERO data. What is the annotation accuracy? How many episodes have incorrect masks, wrong detections or wrong tracking? If the pipeline is near-perfect on LIBERO's clean scenes, the corruption experiments are less informative because they test artificial noise rather than realistic failure modes.

  • Phase 2 of the pipeline has a huge complexity with a marginal gain as compared to simpler alternates. This involves BC, slided distillation, cycle consistency, FiLM, self-attention. The ablation shows that they help, but overall complexity doesn't align with the marginal gain they bring. This makes the readers question which component is actually necessary and which are only heuristically designed.

  • The reproducibility statement says that "one training seed without a validation split." With 500 demonstrations per suite and extensive hyperparameter tuning, there is a huge possibility of overfitting to evaluation.

  • The student receives only the external camera while the teacher receives both external and wrist views. This is shown as a design choice but the information loss from dropping the wrist view is not measured. Is this the only reason for 7.7pp gap?

Questions for authors
  • Can you provide multi-seed results (atleast 3 seeds) for at least the main comparison (Table 1) and the key ablations (Tables 2–3)?

  • Why were no concurrent VLA distillation methods included as baselines?

  • The cycle consistency loss reconstructs the affordance from the action chunk. But the inverse model is a 3-layer MLP. How to ensure that this is recovering useful affordance information and not regularizing the action space?

There are other limitations mentioned in the weakness section. Please address them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.