acceptodds
Under review as a conference paper at ICLR 2027

StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing

20 views

Abstract

Robot learning continues to explore different pretrained models, approaches to action modeling, and sources of supervision. These choices require infrastructure that accommodates new methods while providing reusable training and evaluation workflows. We present StarVLA, an open-source infrastructure that gives the robot learning community accessible, reusable models, training tools, and benchmark resources. Model-level frameworks let researchers compose pretrained components, including vision-language models (VLMs), with action heads and modify their connections and learning objectives. The infrastructure provides 27 registered policy implementations spanning visuomotor, VLM-based, and world-model-based policies. Scalable, extensible dataloaders and configurable dataset mixtures integrate robot data, human-centered data, and multimodal data. Shared training tools support robot post-training, cross-embodiment training, VLM training, and multimodal co-training. The 13 simulation benchmark integrations cover 10 robot embodiments, spanning single-arm, bimanual, humanoid, and mobile manipulation, as well as navigation. Through these integrations, StarVLA makes baseline training recipes, evaluation entry points, and at least 41 policy checkpoint repositories available for research. A shared policy-serving path supports simulation and real-robot clients while keeping environment dependencies separate from model execution. Community research builds on these resources to extend policy architectures and conduct representation pretraining. StarVLA provides infrastructure for researchers to develop their own models, training procedures, and evaluations.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Robot learning, Open Source, Research Infrastructure, Modular Frameworks

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_yBaZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes StarVLA, a unified framework/infrastructure VLA training and evaluation. The key point is allowing interchangability between models, data methods, training methods, etc. They provide both numerous models and benchmarks, and support many types of training.

Critical strengths

Unifying VLA research into one framework is a good idea. Presently, each research project is an ad-hoc implementation or built on someone else's ad hoc implementation. They may help speed up research, similar to stable baselines for RL. This may help the VLA community build more reproducible and reliable research.

They also provide evidence that their work is already aiding research, which is encouraging.

Critical weaknesses

This is more of an engineering project than a research project. I don't see any contributions other than a well-designed framework. I do see value in this work, but I am not sure if this is the right venue. I guess this is a question for the AC, if the paper needs a research contribution or just engineering contributions to the research community. I have not reviewed for the area "infrastructure, software libraries, hardware, systems, etc." before, so I do not know whats expected here. Once I have feedback from the AC I will update my score.

Questions for authors
  1. What is the purpose of Figure 3? If I understand correctly, StarVLA in this figure is just some combination of configs, but they arent even all the same. Is the point of this to illustrate that mixing and matching models, training methods, etc can perform close to state of the art?

  2. What is the research contribution? What did you discover? A new algorithm, training method? Please just clarify that the sole contribution is the infrastructure.

  3. Not a question, but a nice experiment would be to show that you can come up with novel research simply within your framework, ie perform ablations on common methods and find a nice contribution that would not be possible without your infra. For example, maybe some insight into how training data affects all of these methods and is biasing our interpretation of each method, since different models use different datasets even if their inputs/outputs are identical.

  4. How necessary are these unified packages in the age of Claude/codex? 10 years ago this would have been great for accelerating research, but now people just use AI to handle interfacing between models, training methods, datasets. Make a case for why this is still necessary.

Reviewer_MGurOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

StarVLA presents a comprehensive, modular infrastructure for robot learning, bringing together pretrained backbones, diverse action heads, and reusable training and evaluation tools. With 27 policy implementations and 13 simulation benchmark integrations covering 10 robot embodiments, it provides a substantial foundation for developing and studying robot policies. Competitive results across multiple policy families, scalable training, and concrete examples of adoption in subsequent research demonstrate its practical value. The framework makes a valuable contribution to the community by supporting both strong baseline implementations and flexible development of new methods.

Critical strengths
  1. Broad scope. StarVLA brings diverse policy architectures, training paradigms, datasets, and robot benchmarks into a unified framework. This comprehensive coverage makes it useful for a wide range of robot learning research.
  2. Strong community impact. The framework has been adopted and extended by numerous subsequent works, demonstrating substantial value to the research community. Providing shared infrastructure that enables further research is an important contribution.
  3. Practical usability. Its modular design, ready-to-use training recipes, released checkpoints, and integrated evaluation tools make it easier to develop and experiment with VLA methods. Researchers can build on a substantial working foundation and focus their effort on new ideas.
Critical weaknesses

The paper provides useful comparisons across policy configurations and training settings. Given the framework’s extensive coverage, a deeper discussion of the patterns emerging would make this more informative for future research.

Questions for authors

See Weakness

Reviewer_xz6YOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents StarVLA, an open-source codebase for building, training and evaluating vision-language-action (VLA) policies. Its central idea is a fixed boundary: each policy is one "framework" file that combines a pretrained backbone (a VLM, a video world model, or a visual encoder) with an action head and defines its loss, and it exposes only forward (training loss) and predict_action (action chunk). Data loading, training, configuration and evaluation are shared across all policies. The claimed contributions are this encapsulation (27 registered policy implementations), data and training support for robot, human and vision-language data (four training paradigms), and integrations with 13 simulation benchmarks, served through a common policy server, with reference configurations and 41 checkpoint repositories. Validation relies on previously reported benchmark results, throughput measurements, and adoption by more than 50 follow-up works. No new experiments were run for the paper.

Critical strengths
  • The codebase covers VLM-based, world-model-based and visuomotor policies, several action-head families (regression, FAST tokens, flow matching / diffusion), and 13 benchmark integrations behind one serving interface. More than 50 works have built on it (Fig. 2), and Appendix F shows concrete reuse of model, data, training and evaluation components. This is real external evidence that the resource is useful to the community. It is the main reason I do not rate the paper lower: infrastructure with demonstrated adoption can be a legitimate ICLR contribution.
  • A simple, reusable boundary. Keeping method-specific computation inside one framework file, with the trainer and policy server calling only forward / predict_action, is an easy contract for researchers to adopt. Backbones can be swapped through configuration via the backbone dispatcher. This is what makes the Lego-like composition work in practice, and it is useful to the community.
Critical weaknesses
  • The Lego-like composition works, but the paper does not show what it contributes beyond engineering.
    • Composing pretrained backbones with interchangeable action heads is a sound design, and the codebase does support it. However, component composition is not new in itself: the paper's own Table 1 lists "Component composition" for RoboVLMs, Dexbotic, VLA Foundry and FluxVLA. What could distinguish StarVLA is what its breadth makes possible, and the paper does not demonstrate this.

    • The paper never uses the composability to answer a research question, such as which backbone, action head or supervision mix matters, and under what conditions. Appendix A states that "No new training or benchmark runs were performed during its preparation". The only like-for-like comparison is the LIBERO block of Table 9 (three backbones × four heads, 30k steps). On this saturated benchmark all variants fall between 95.2% and 96.6%, and no seeds or variance are reported.

    • An infrastructure paper needs to show what the infrastructure enables, either new findings or measured savings. Here the evidence is counts and adoption only. This is the main reason for my rating.

  • No clear evidence that the comparisons support the claims of advantage.
    • Fig. 3 shows, for each benchmark, the highest collected StarVLA configuration against three reference methods whose training data, initialization and budgets differ (as the caption itself notes). In some panels the protocols also differ: on RoboCasa365, StarVLA is trained on 34 seen tasks while the references use the full training set, and RoboDojo aggregates the two sides differently. StarVLA is below the best reference in two of the ten panels (RoboCasa365: 40.64 vs 45.1; VLA-Arena: 38.24 vs 39.88). No uncertainty estimates are given.
    • In Table 1, "–" means "unconfirmed, not necessarily unsupported". Most of the discriminating cells for other frameworks are "–", so the table cannot establish that those frameworks lack the listed capabilities.
    • The code warns that the current revision may not reproduce previously released checkpoints. However, the 41 checkpoint repositories are not mapped to specific code revisions, which limits their value as reference resources.
    • Why this matters: the reference results and checkpoints are presented as a main contribution. Without matched protocols and version mapping, they serve as context rather than as reliable baselines.
Questions for authors
  • What can researchers do with StarVLA that is not practical with existing component-composition frameworks such as Dexbotic, RoboVLMs or LeRobot? A concrete example of a study that was only feasible with StarVLA would help me judge the contribution.
  • In Fig. 3, the StarVLA configurations and the reference methods differ in training data, initialization and budget. Under matched settings, which of the reported advantages would remain?
3 comments
AuthorsOpenReview ↗

Response 1/2: Research Enabled by StarVLA and Measured Training Efficiency

Thank you for the careful reading, and apologies for the late reply. We answer the two weaknesses and two questions together.

W1 / Q1. Research enabled beyond component composition.

To be explicit: we do not claim "component composition" as a novelty; every VLA codebase composes components in some form. What matters is what sits under one stack. Our contribution is the infrastructure and the research workflows it supports. In StarVLA, VLM backbones and video world-model backbones share the same action heads, dataloaders, trainers, and evaluation interfaces, and the same stack supports vision-language co-training and 13 benchmarks. To our knowledge, StarVLA is the first framework to combine these: in Table 1, RoboVLMs, Dexbotic, and VLA Foundry have no world-model policies, FluxVLA Engine comes with each policy's own pipeline, and LeRobot integrates policies as plugins rather than as composable backbones and heads.

You ask for new findings or measured savings; we address each below.

1. On research findings: a VLM and a video world model under one pipeline. Section 5.3 (Table 5) brings together the backbone and action-head results from Appendix C.5 (Table 10). The LIBERO and SimplerEnv grid share a data pipeline, training budget, and evaluation protocol.

BenchmarkBackboneFASTOFTPIGR00T
LIBEROQwen2.5-VL-3B (VLM)95.296.195.495.4
Qwen3-VL-4B (VLM)95.496.695.796.5
Cosmos-Predict2-2B (world model)–95.895.595.2
SimplerEnv WidowXQwen3-VL-4B (VLM)–64.660.965.3
Cosmos-Predict2-2B (world model)–61.258.761.6

At 30k steps on LIBERO, the eleven backbone and action-head configurations span 95.2–96.6%. The SimplerEnv results extend the comparison to a second environment, far from saturation, with the same three continuous action heads. With the action head held fixed, the VLM and the world-model backbone differ by at most 1.3 points on LIBERO and 3.7 points on SimplerEnv WidowX. To our knowledge, the two backbone families had not previously been compared with the action head and the evaluation protocol held fixed.

2. On "no seeds or variance": evaluation protocol and training cost. Our evaluation follows the practice of assessing trained policies through repeated rollouts, as in OpenVLA-OFT, π0, and GR00T N1 [1–3]. Each LIBERO entry aggregates 2,000 rollouts across four suites, and each SimplerEnv entry uses five repetitions of the official evaluation. Given the substantial cost of repeating training runs for 2–9B models, we use one training run per configuration and evaluate each policy across multiple tasks and repeated rollouts.

3. On "counts and adoption only": efficiency evidence already in the submission. The submission already reports quantitative training-efficiency measurements in Section 6.2 and Figure 4. On 8×H200 at batch 32 per GPU, the reported step times are 0.78 s for StarVLA, 1.12 s for LingBot-VLA, and 1.98 s for Dexbotic (measurement details in Appendix E.3). Our A100 series scales from 8 to 256 GPUs at batch eight per GPU, reaching 2,200 samples/s and 79.1% scaling efficiency.

To make the research findings more prominent, we have brought the key comparisons and findings from Appendix C.5 into Section 5.3 of the revised main text. We did not foreground the comparison in point in the original submission because we submitted to the infrastructure track, where the contribution is the system. We believe the absence of a new learning algorithm should not, by itself, be grounds for rejection in that track.

[1] Kim et al. Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success. RSS 2025.

[2] Black et al. π0 RSS 2025.

[3] Bjorck et al. GR00T N1.5

AuthorsOpenReview ↗

Response 2/2: Benchmark Comparisons, Framework Coverage, and Checkpoint Provenance

First, we apologize for the unclear explanation. We address each concern below.

W2 / Q2. Soundness of the comparisons and resources.

Figure 3: purpose of the figure. Figure 3 is an overview of reference recipes. For each of ten benchmark settings, it shows that a policy built with StarVLA reaches competitive performance, so researchers have a working starting point and a reference result on each benchmark. It is not a claim of advantage over the reference methods, and the contribution of the paper is the infrastructure behind these recipes rather than the scores. For comparisons under shared settings (Q2), Section 5.3 reports the LIBERO grid with a common data pipeline, a training budget, and the same evaluation protocol. We have revised the caption to state this purpose, and Appendix C.4 gives the benchmark-specific evaluation details.

Table 1: the search criterion and the meaning of "–". The footnote was too cautious. Every "–" means we searched the framework's public code and paper and did not find the capability. These frameworks were actively evolving during submission preparation, so the table reflects the versions we inspected at that time. We have reworded the footnote to "not found in the framework's public code or paper as of September 2026" and will correct any cell you believe is wrong.

Checkpoint provenance: planned code-revision mapping and model-card updates. We have added a dedicated checkpoint-to-code mapping table (Table 14, Appendix H), including model configurations and mapping evidence. The corresponding revision details and evaluation instructions will also be added to the Hugging Face model cards.

AuthorsOpenReview ↗

Follow-up on Our Responses and Manuscript Revisions

Hi, thank you again for your careful review. We would appreciate any feedback you may have on our responses and the revised manuscript. Please let us know if you have any remaining concerns; we would be happy to provide further clarification.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Discussion and Manuscript Revisions

We thank the Program Chairs, the Area Chair, and the reviewers for their time and consideration. We are encouraged that all three reviewers recognized StarVLA's practical value and community impact. The current scores are 4 (MGur), 3 (yBaZ), and 2 (xz6Y).

The discussion focused on contribution scope, research evidence, and the interpretation and reproducibility of reference resources. For your convenience, we summarize our responses and revisions below.

The first concern was the contribution expected of an infrastructure paper. We believe StarVLA is well suited to the submitted primary area, “infrastructure, software libraries, hardware, systems, etc.” Its reusable infrastructure supports research across the robot learning community. Related ICLR precedents include LeRobot (2026) [1] and TorchRL (2024) [2]. Following our clarifications, reviewer yBaZ raised their score from 2 to 3.

The second concern was evidence beyond implementation counts and adoption. The original submission already included backbone–action-head comparisons and training-efficiency measurements. We have moved the key comparisons from Appendix C.5 into Section 5.3, Table 5, highlighting the similar reported performance of VLM and video world-model backbones within the shared infrastructure. Section 6.2 and Figure 4 already report training efficiency and scaling to 256 GPUs.

We also clarified Figure 3's purpose and Table 1's search criterion, and added checkpoint-to-code mappings in Appendix H, Table 14, addressing the comparison and reproducibility concerns.

Although reviewer xz6Y has not yet followed up on our responses, their concern about research value overlaps with the issue discussed with yBaZ. The revisions above also respond to their additional questions about comparisons and checkpoint provenance.

The long-term goal of starVLA is to make robot learning research more accessible, especially to researchers with limited engineering resources. We hope StarVLA lowers the barriers to participation and gives more researchers the opportunity to develop, test, and share their ideas.

[1] Cadène et al. LeRobot: An Open-Source Library for End-to-End Robot Learning. ICLR 2026.

[2] Bou et al. TorchRL: A Data-Driven Decision-Making Library for PyTorch. ICLR 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.