acceptodds
Withdrawn from ICLR 2027

Kimodo-Policy: From Text-to-Motion Generators to Humanoid Vision-Language-Action Policies

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

24 views

Abstract

Humanoid Vision-Language-Action (VLA) models translate multimodal task understanding into coordinated whole body behavior. Existing approaches equip pretrained vision language or video generation backbones with action experts, but their pretraining does not directly establish whole body motion priors. Unified or upper-lower-body action designs also lack dedicated root, body, and hand modeling despite their distinct requirements. These limitations increase the burden of learning precise, coordinated actions from limited task-specific humanoid demonstrations. We introduce Kimodo-Policy, which initializes visually grounded control with pretrained text-to-motion (T2M) priors over inter-joint coordination and temporal structure. Its three stage root-body-hand architecture combines dedicated generators with explicit dependencies: root trajectories condition body generation, while future body features and visual observations guide hand control. We pretrain the policy on diverse humanoid demonstrations, conditioning actions on visual observations, language instructions, and motion history. With 678M total and only 92M trainable parameters, Kimodo-Policy achieves strong performance on HumanoidArena, SIMPLE, and real-world loco-manipulation tasks. Further experiments show performance gains from larger VLA pretraining datasets, greater model capacity, and stronger pretrained T2M priors.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Loco-Manipulation, Vision-Language-Action, Text-to-Motion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cRaWOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This work presents a method for adapting a pre-trained text-to-motion models into a VLA model for humanoid control. First, a pre-trained Kimodo text-to-motion model is augmented with ControlNet adaptor layers to inject additional language and visual inputs into the model. An additional model is introduced to generate hand motions. The ControlNet layers are finetuned with data from various whole-body control tasks. The generated motions are then provided as target motions for a whole-body tracking controller (SONIC), which translates the kinematic motion plan into controls for a humanoid robot (G1). The finding that kinematic motion generation models can be finetuned into an effective VLA model, and the motion generation model seem to offer useful priors for downstream tasks.

While the insight is interesting, the exposition of this submission is rather poor with lots of grammatical errors and confusing sentences. Furthermore, it is difficult to assess the quality of the results without videos to showcase the behaviors of the robots. A more detailed discussion is provided in the following sections.

Critical strengths

This work offers an interesting insight that kinematic motion generation models can be effectively finetuned for whole-body humanoid control, and that the motion generation models provide useful priors for downstream tasks.

The experiments provide fairly good comparisons of the proposed method to previous work.

Critical weaknesses

While the insight that kinematic text-to-motion models can be finetuned into VLA models for humanoid control is quite interesting, the technical novelty seems very limited. The method mainly involves adding controlnet layers to Kimodo and an additional HandNet to control the robot’s hands. A similar controlnet method was also previously used to adapt text-to-motion models in: OmniControl: Control Any Joint at Any Time for Human Motion Generation

The three-stage architecture is not particularly novel, since Kimodo’s architecture already consists of a 2-stage architecture that separates the root and body motions. So the only additional stage introduced in this work is the HandNet.

The lack of videos showing the behaviors of the robot makes it difficult to assess the effectiveness of the proposed model.

The quality of the writing in this submission is quite low, well below the bar that would be expected for ICLR. There are many grammatical errors and confusing sentences littered throughout the paper that make it difficult to understand what the paper is trying to say. Below are just a few of the many examples of such writing issues:

Line 215: “isolate stage optimization” awkward wording

Line 218: “while a token-axis MLP maps the image-token sequence to the future temporal horizon.” Not clear what this means, especially “token-axis MLP”.

Line 223: what is “content-selective visual retrieval”? It’s not explained anywhere what this means.

Line 225: What is “bilateral diffusion head”?

Figure 4: What is “Light brown: observed history” referring to? None of the figures are light brown.

Line 280: this sentence is confusing: “All losses mask padding; hand losses also mask unavailable labels, and transitions require two valid adjacent labels”. What does “all losses mask padding” mean? How does “hand losses” mask unavailable labels? What does it mean for transitions require two valid adjacent labels? What are “adjacent labels”?

Line 307: what is “RTC”?

Line 325: what does “evaluate under SONIC” mean? SONIC is a model, it’s not clear how one evaluates a policy “under” a model.

Line 373: what does “joint-best” mean?

Line 453: The sentence is not grammatically correct. Should be “All ablations are trained on… and evaluated on…”

Major revisions to the writing is necessary.

Questions for authors

Can the authors clarify on the technical innovation of this work? Applying ControlNet to adapt a pre-trained model is not a very significant technical innovation.

Can the authors provide some videos of the behaviors produced by the model?

The experiments in Table 1 and 2 compare mostly to old baselines from 2023. Are there more recent baselines that can act as stronger baselines? For example, Wei et al., was only compared to in the SIMPLE benchmark. Can that method also be applied to HumanoidArena. Furthermore, there are more recent works, such as WholeBodyVLA (Jiang et al., 2026), which could provide more recent baselines to compare to.

Please, clarify the writing confusions listed under “Critical Weaknesses”

Reviewer_11RyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Kimodo-Policy, which adapts a pretrained Text-to-Motion model for humanoid VLA control. It freezes the pretrained Kimodo backbone, incorporates visual information through ControlNet, and generates whole-body actions using a three-stage Root–Body–Hand architecture for closed-loop control.

Critical strengths

See "Questions For Authors".

Critical weaknesses

See "Questions For Authors".

Questions for authors

Overall, I appreciate the effort put into this work. However, I feel that the novelty of the proposed method and its distinction from recent works are somewhat limited.

  1. Novelty and distinction from recent works. WholeBodyWAM, ω-0, and other recent WAM methods have already explored text-conditioned motion generation and motion priors for humanoid control. I am not entirely clear about the key novelty of this work compared with these methods. Although I understand that many of these papers were developed around the same time, a clearer explanation of their differences would help me better evaluate the contribution of this paper and potentially reconsider my score.
  2. Limitations of Text-to-Motion priors. I agree that Text-to-Motion can provide useful motion priors, and I have explored similar ideas myself. However, I feel that the semantic understanding of Kimodo is relatively limited compared with existing VLA models. For example, although BONES-SEED contains a large amount of motion data in terms of duration, as far as I know, it covers only around 4,000 motion categories. In my view, while using a pretrained T2M model is certainly convenient, its generalization ability may not be comparable to that of large-scale VLA models. Traditional VLA approaches can benefit from large and diverse datasets to improve generalization, without necessarily requiring such highly curated motion data. Although this work further incorporates VLA demonstration data for training, I am still concerned that the frozen Kimodo backbone may impose certain limitations on the model's overall capabilities. Nevertheless, I find this direction quite interesting.
  3. Language and task generalization. Following the previous point, I would like to ask: Can the model successfully execute the same task when the instruction is expressed differently? How well does it generalize to unseen instruction combinations, novel objects, or entirely new tasks?
  4. Design choice of the action generator. Why do the authors choose to use Kimodo as the primary action generator, rather than using it to provide motion references or priors for a separate action policy?
  5. Real-world demonstrations and supplementary materials. I could not find any real-world demonstration videos for reference, which leaves me somewhat uncertain about the real-world results. I would appreciate it if the authors could provide such videos. Additionally, I believe the supplementary appendix could be included at the end of the main paper, as permitted by the ICLR submission guidelines.
Reviewer_kxGvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents Kimodo-Policy, which adapts a pretrained text-to-motion model for visually grounded humanoid loco-manipulation. It uses frozen root and body motion generators augmented with visual conditioning through trainable ControlNet branches, together with a separate HandNet for bilateral hand control. Observed motion history supports receding-horizon action prediction, while SONIC executes the generated motion references. The authors additionally perform VLA pretraining on 419 hours of humanoid demonstrations. Evaluation covers HumanoidArena, SIMPLE, and eight real-world tasks, with further analyses of motion priors, pretraining scale, model capacity, and training choices.

Critical strengths
  1. An interesting backbone choice for humanoid policy learning. Reusing a pretrained motion generator provides an explicit source of whole-body coordination and temporal structure. The combination of visual conditioning, motion-history constraints, and specialized hand control offers a concrete approach to adapting this prior for loco-manipulation.

  2. Promising empirical results across complementary settings. The paper evaluates the approach on two simulation benchmarks and physical humanoid tasks. The multi-task HumanoidArena results are particularly encouraging: the VLA-pretrained model achieves 72.14% success, compared with approximately 53.8% for the strongest reported baseline. The ablations also provide evidence that the pretrained motion backbone contributes substantially under the evaluated training protocol.

Critical weaknesses
  1. The choice of an offline motion backbone is insufficiently justified. Kimodo is adapted to closed-loop control through history-conditioned, sliding-window inference, whereas existing models such as DiP in CLoSD and ARDY already support autoregressive or online motion generation. ARDY is especially relevant because its official implementation provides G1 models and its project demonstrates integration with SONIC. The paper does not adequately explain the trade-offs favoring Kimodo or provide a controlled comparison with a streaming motion prior. Consequently, it remains unclear what benefits specifically follow from adapting this offline backbone rather than an online alternative.

  2. The available materials provide limited evidence for assessing real-world execution quality. I could not find demonstration videos in the supplied materials. For humanoid loco-manipulation, continuous recordings are important for assessing temporal coordination, contact stability, execution quality, and failure modes. Static snapshots and success rates provide useful quantitative evidence, but do not make these aspects directly observable or enable a qualitative comparison with baselines. This limits my assessment of the reported physical capabilities.

Questions for authors
  1. Why choose Kimodo over an online motion model such as ARDY or CLoSD? A controlled comparison would help clarify the trade-offs in task performance, latency, and motion continuity.

  2. Are demo videos available? It is difficult to assess real-world execution quality from static images alone. Continuous success and failure examples, ideally alongside baselines, would help.

  3. Figure 2 discusses three backbone families, but there is no video-generation-based baseline. Why not compare with MotionWAM or WholeBodyWAM? If this is impractical, please explain the constraints.

  4. What does “w/o T2M” actually remove? Does it keep the same architecture with random initialization? A parameter-matched unified action head would also help isolate the benefit of the three-stage design.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.