- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes EmbodiedMidtrain, a data selection and mid-training pipeline for improving VLM initialization for VLA models. A lightweight classifier trained on frozen VLM features scores candidate vision-language samples by their similarity to robot training data. The method selects 1.28 million samples for VLM mid-training, followed by robot action fine-tuning. Experiments on CALVIN, SimplerEnv, LIBERO-10, and real-world tasks show improvements over the original VLM initializations, with additional analyses of transfer across backbones and action heads, selection methods, and selected data.
Critical strengths
1.The paper studies a meaningful research problem. Improving the transfer from VLMs to VLAs is an important direction in embodied intelligence. The proposed data selection and mid-training pipeline offers a practical way to improve VLM initialization for robot action learning.
2.The experimental analysis is broadly thorough, combining quantitative evaluations with visualizations. Results across multiple benchmarks, together with ablation studies and qualitative examples, provide useful evidence for the effectiveness of the proposed pipeline.
Critical weaknesses
1.The reduction in the data distribution gap is not directly verified after selection. The paper uses MMD and t-SNE to show the gap between the original vision-language data and robot data, but does not report the corresponding comparison after selection. The higher classifier scores in Figure 4 are an expected result of selecting high-scoring samples and do not independently show that the gap has narrowed. I suggest comparing MMD before and after selection in the same feature space, with t-SNE visualizations as supporting evidence.
2.The data sizes and sampling procedures in the ablation experiments are unclear. Although the paper reports the same mid-training compute budget, it does not specify whether the random-selection baseline uses a fixed subset of 1.28 million distinct samples, matching the selected dataset. Please also clarify whether the alternative selection methods in Table 2 each select 1.28 million samples, and report the number of distinct samples used in each setting and how repeated sampling is handled.
Questions for authors
Could the authors respond to the two concerns raised in the weaknesses section?
4 comments
We look forward to hearing from you
As the discussion period is coming to an end, we would greatly appreciate any further feedback you may have. We would be happy to address any further questions or concerns about our work.
Thank you for the detailed responses and for taking the time to address my concerns. The additional MMD analysis and clarification of the data sizes and sampling procedures have largely resolved my questions.
We sincerely thank the reviewer for the feedback and for taking the time to engage with our paper. We are glad that the additional experiments and clarifications addressed your concerns.
Thanks for your feedback on our paper. For your concerns and questions, we would like to respond as follows.
Q1. Distribution gap after data selection.
A1. We computed the MMD between each VLM source and each VLA dataset before and after selection. The table reports the change in MMD after selection, where a negative value means a narrower gap.
The gap narrows for most pairs, showing that selection moves the VLM data toward the VLA domain under a measure that does not depend on the classifier score. The reduction is largest for the general sources LLaVA-Instruct-665k, LAION-400M, and CC-12M, consistent with Appendix K, where selection within general sources gives larger downstream gains than selection within embodied-oriented sources. We have added these results and the t-SNE visualization after selection to the revision.
Q2. Data sizes and sampling in the ablation experiments.
A2. All settings in Table 2 use the same data size. Random selection and the three alternative proximity measurements each use a fixed subset of 1.28M distinct samples, drawn without repetition from the same candidate data as our method. The settings differ only in the 1.28M samples they select. Mid-training runs for 5,000 steps at a batch size of 256, which is one pass over the 1.28M samples, and no sample is repeated. We will state these details in the revision.