- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how action heads adapt to pretrained vision-language representations. The authors observe that action heads become increasingly sensitive to high-variance feature channels during training. They propose Adaptive Interface Equalizer (AdaIE), which rescales feature channels according to the evolving sensitivity of the downstream model without introducing additional trainable parameters. Experiments on simulated and real-world robotic manipulation, diffusion policies, and world models show overall performance improvements.
Critical strengths
-
The paper addresses a practically relevant question about the interface between pretrained representations and downstream action models. AdaIE is simple to integrate into existing architectures and introduces modest training overhead.
-
Evaluation spans multiple simulation benchmarks, five real-world tasks, and world models. The comparison between equalizing and reinforcing sensitivity provides useful initial evidence for the proposed approach.
Critical weaknesses
-
The proposed mechanism lacks theoretical grounding and sufficient causal evidence. The paper links the benefits of sensitivity equalization to a mismatch between pretrained representations and downstream tasks. However, it does not establish when sensitivity concentration harms learning or why inverse-sensitivity reweighting should improve optimization or control. The aligned and mismatched settings in Section 2.2 differ in models, datasets, and tasks, making it difficult to attribute the observed differences specifically to alignment. The authors acknowledge this limitation. A theoretical guarantee is not necessary to establish empirical usefulness, but the current evidence supports the effectiveness of reweighting more strongly than the proposed explanation for that effectiveness.
-
Comparisons and ablations do not fully establish the distinctive benefit of AdaIE. The main VLA experiments lack direct comparisons with variance-based reweighting and simple normalization, and the contributions of EMA and norm matching are not isolated. Limited reporting across independent training seeds and only 20 real-world trials per task per method also leave uncertainty about the robustness of the gains.
Questions for authors
-
Is there evidence that representation–task mismatch determines the benefit of equalization when the model architecture and downstream task are held fixed? Can theoretical analysis or additional diagnostics substantiate this explanation?
-
How does AdaIE compare with variance-based reweighting and simple normalization in the main VLA experiments? Does norm matching alone explain part of the improvement?
-
How many independent training seeds were used for Tables 2 and 3? Can the authors report variability across training runs or confidence intervals to clarify the reliability of the gains?
6 comments
Thank you again for your thoughtful review.
With the discussion period approaching its conclusion, we wish to kindly check whether our response has addressed your questions.
If you find our responses helpful, we would greatly appreciate your acknowledgment. If you have any further questions or would like additional clarification, please do not hesitate to reach out. We are committed to providing any further responses during this discussion phase.
We sincerely appreciate your time and thoughtful consideration, and we look forward to your feedback.
Dear Reviewer Z1VD,
Thank you again for your time and consideration.
As the discussion period ends tomorrow, we would greatly appreciate it if you could take a moment to review our rebuttal and additional experiments when you have a chance. We look forward to hearing your feedback.
Best,
The Authors
Thank you for your thoughtful review! We address the concerns below:
1. Mismatch metrics
Thanks for raising this point. We looked through the literature for ways to measure this gap. We found two measures in the transfer learning literature that might be helpful: LogME [1] and linear probing [2]. Specifically, LogME estimates how suitable frozen backbone features are for a downstream task, and linear probing measures how much target variance a simple linear readout can explain. They are used to estimate transferability from a backbone to a downstream task. Higher LogME and linear probing mean that downstream targets are more easily predictable from the frozen features. We report these two metrics below:
The two aligned interfaces have higher scores than the two mismatched interfaces, reflecting that they are easier to adapt to the downstream tasks, empirically. However, a more comprehensive understanding of representation–task alignment, particularly its theoretical foundations, requires further investigation, as acknowledged in our Limitations.
2. Var-based experiments for VLAs and simple normalization.
We conduct additional experiments on a subset of RoboCasa-GR1 (12 random subset tasks), training for 30K steps and evaluating with 60 episodes per task. Due to computational resource constraints, we use this subset for the additional ablations. For the variance-based baseline, we follow the same approach used in our LeWM experiments.
As shown above, the variance-based variant does not improve over the baseline (but instead degrades performance) at either tested α, while AdaIE consistently outperforms the baseline.
Regarding simple normalization, our baseline already includes LayerNorm and a learnable linear projection before the VLM features enter the DiT. AdaIE is applied after these operations, which are shared by both variants. Furthermore, NormMatch only preserves the overall RMS magnitude after channel-wise reweighting. Without reweighting (α=0), NormMatch reduces to the identity transformation. Thus, the improvement cannot be explained by the addition of normalization or norm matching alone.
3. Training seeds were used for Tables 2 and 3
As in our Limitations, due to computational resource constraints, we were only able to conduct multi-seed training for a subset of our experiments. Tables 2 and 3 report results from a single training run, while Tables 1 (which also evaluates RoboCasa-GR1, but on a subset of the tasks in Table 2) and 5 report mean and standard deviation across multiple training seeds. Additionally, our new ablation results above show consistent improvements across different α values. This provides further evidence of AdaIE's reliable gains.
Thank you again for your time and thoughtful review. We look forward to your feedback.
[1] LogME: Practical Assessment of Pre-trained Models for Transfer Learning, ICML 2021.
[2] Scalable Diverse Model Selection for Accessible Transfer Learning, NeurIPS 2021.