- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SphLA, which uses Spherical Flow Matching to generate latent actions from language and vision-language inputs. These actions are executed by a pretrained BFM actor for humanoid whole-body control.
Critical strengths
See "Questions For Authors".
Critical weaknesses
See "Questions For Authors".
Questions for authors
-
Although the paper presents real-robot experiments, I could not find an accompanying demonstration video in the provided materials. Without such a video, it is difficult to assess the actual motion quality and stability.
-
The paper mainly compares SphLA with standard Euclidean Flow Matching. However, since SphLA explicitly preserves spherical geometry by design, the reduction in radial error is largely expected and does not necessarily demonstrate the superiority of the proposed method.
-
The paper primarily applies existing Riemannian Flow Matching techniques to the spherical latent space of a BFM. Both the key geometric techniques and the idea of directly generating latent actions have been explored in prior work. I would appreciate clarification on what substantial technical innovations this work introduces beyond existing methods.
-
I am not convinced that this work addresses a sufficiently significant problem. Although SphLA improves latent generation performance, the improvements appear relatively modest, and their practical significance remains unclear. Therefore, I have concerns about the overall significance and contribution of this work.
3 comments
Dear Authors,
Thank you for your response. We still have the following concerns:
- UFO is pretrained on LAFAN1, while the generator is trained on BONES-SEED. We find this cross-dataset setup quite puzzling.
- We appreciate the release of the model weights. After downloading and testing the model, we found that some instructions were not executed satisfactorily. This raises concerns that the low-level controller may limit the benefits of high-level motion generation.
- The evaluation uses an author-selected test subset whose representativeness and difficulty remain unclear. This makes the reported results less convincing.
As the rebuttal period draws to a close, we are not requesting additional experiments. I am very sorry, but given the unresolved questions above and my concerns about the paper’s novelty, I find it difficult to revise my score at this stage. I hope you will continue to refine this work, clarify its contributions, and strengthen its validation.
Follow-up to Reviewer Drid: Clarifications on Evaluation and Contribution
Thank you for following up and for testing the released model. We address the remaining concerns and clarify the contribution below.
- Why use LAFAN1 for the controller and BONES-SEED for the generator?
We use UFO's released BFM-Zero-based controller, pretrained using LAFAN1 motion data. BFM-Zero combines online unsupervised RL with motion-data regularization and reports cross-dataset tracking evaluation on 175 AMASS motions (Sec. 3.1).
BONES-SEED supplies paired language descriptions and motions for generator training. We retarget these motions to G1 and encode them with the same frozen backward network to obtain normalized spherical targets. The encoder and actor remain frozen, keeping the representation connecting generation and execution fixed. The controller maps robot states and latents to actions; SphLA learns which latent sequence to generate from language. These objectives do not require identical training datasets.
- Instructions that were not executed satisfactorily.
Thank you for reporting this experience. Execution depends on both generated latents and controller capability. SphLA's generalization can vary with phrasing and the coverage of BONES-SEED training motions. Concrete descriptions of actions, body involvement, and movement direction reflect the training caption format.
We would appreciate the tested instructions, durations, and inference settings to help reproduce these cases and distinguish generation from control limitations. Table 2 keeps the controller fixed and evaluates actual executions, showing that the generator's contribution remains measurable through the same frozen BFM actor network.
- Representativeness and difficulty of the evaluation.
Table 1 uses the full 4,189-sample validation partition with three generation seeds. The text–latent evaluator is trained on reference pairs and kept frozen; generated samples are not used for evaluator training or model selection.
For Table 2, we sample 100 distinct captions outside generator training to balance rollout cost and action coverage. The set covers walking, jogging, jumping, dancing, turning, manipulation, idle behavior, crouching, and stretching, with 8–12 captions per category. Each caption is evaluated at 2, 4, and 6 seconds with two generation seeds, yielding 600 rollouts per method. The motion evaluator is trained on reference trajectories and scores actual MuJoCo executions. All rollouts, including falls, enter the metrics under the same controller and evaluation protocol. Retrieval tests cross-category instruction alignment; FID, Diversity, and fall rates assess motion distribution, variation, and stability.
Appendix E.4 compares SphLA and OMG on 63 motion-name matches between our test set and the BONES-SEED portion of OMG's official test set. Both caption versions independently condition generation with two seeds, and both our evaluator and OMG's released evaluator score the executions. SphLA achieves higher R@1/2/3 across all four caption–evaluator combinations.
Further clarification on the contribution.
The central distinction is the generation target: our joint-space FM baseline, TextOp, and OMG generate kinematic reference trajectories, whereas SphLA directly generates control latents for the frozen BFM actor network. The frozen backward encoder supplies spherical supervision offline, and we formulate conditional sequence generation in this representation using Riemannian FM. Text and vision-language experts reuse the same frozen BFM actor network; the VLA evaluation uses recorded visual inputs and simulated-state feedback.
With the controller and evaluation protocol held fixed, Table 2 shows higher executed-motion R@1/2/3, lower FID, and a lower observed fall rate than joint-space FM and Euclidean latent FM on held-out captions. The OMG cross-evaluation provides further evidence across annotation styles and evaluators. Figure 6(c) demonstrates the same instruction over 2-, 4-, and 6-second horizons; duration can be specified or predicted by an auxiliary head. With resident models on the G1's Orin NX, uncached text-to-plan preparation takes 898.3 ms for 5-second plans and 1716.0 ms for 10-second plans on average. This demonstrates onboard generation substantially faster than the resulting motion's execution duration.
Matched simulation and real-robot videos are now available on our anonymous supplementary website. Using the same text instructions, these videos provide qualitative evidence of consistent motion execution and action transitions across sim2sim and sim2real deployment for SphLA's text route.
Thank you again for considering these clarifications and the additional evidence.
Response to Reviewer Drid: Contribution Clarifications and Supporting Evidence
Thank you for your thoughtful questions about the contribution and its practical value. We address your questions below.
Q1: Demonstration videos.
A: Thank you for highlighting the need for video evidence. We have provided continuous text-conditioned real-robot videos and vision-language body-motion replays in simulation using recorded egocentric video on our anonymous video page. Both use the same frozen BFM actor network as the low-level whole-body controller.
Q2: Evidence beyond norm preservation.
A: Thank you for raising this point. Our evaluation examines whether SphLA's conditional generation in the BFM control representation improves instruction alignment and the resulting whole-body behavior. We compare it with both explicit reference-trajectory generation and Euclidean generation in the same latent space, and further examine whether the control interface can be reused under vision-language conditioning.
In Table 1, SphLA improves R@1 by 1.92 percentage points over the spherical-prior FM baseline, whose outputs are also normalized before evaluation. In Table 2, the joint-space baseline generates kinematic references that are subsequently encoded for execution, while latent-space FM and SphLA directly generate control latents. Using the same frozen BFM actor network and evaluation protocol, SphLA improves executed-motion R@1 by 5.47 and 4.14 percentage points over latent-space and joint-space FM, respectively. These comparisons demonstrate benefits in simulated execution beyond radial-error reduction.
The expanded vision-language evaluation further demonstrates reuse of the same frozen BFM actor network across conditioning modalities. On 70 held-out episodes across three G1 tasks, with matched training conditions and NFE 1, SphLA reduces joint MAE from 10.53° to 9.61° and root-relative MPJPE from 154.84 to 61.28 mm relative to FM. These results measure simulated body-motion fidelity with recorded egocentric video input and current body-state feedback. The text and vision-language experts share the same frozen BFM actor network, supporting whole-body execution across both conditioning modalities.
Q3: Contribution beyond existing geometric and latent-action methods.
A: SphLA applies Riemannian flow matching to latent-action generation for humanoid whole-body control, formulating conditional sequence generation in the fixed spherical control representation of a pretrained Forward–Backward BFM.
Among related text-driven humanoid systems, TextOp and OMG supply kinematic references to a tracker through latent decoding and direct motion-feature generation, respectively. SphLA directly generates inputs for the frozen BFM actor network. Reference motions are encoded offline by the frozen backward network and normalized to form spherical latent targets. At inference, generated latents and robot-state feedback drive the frozen BFM actor network, without an intermediate kinematic reference. This aligns the generation target and its geometry with the existing control interface.
In addition, we examine data scale and motion duration to further characterize text-conditioned generation in this latent space. Figure 7 in Appendix G.2 shows that larger training subsets yield more sustained validation velocity alignment. The motion duration can either be predicted from the text by an auxiliary length head or specified directly by the user. Figure 6(c) illustrates the same instruction executed over requested durations of 2, 4, and 6 seconds. These examples show motion adapting to the requested duration and spanning the full interval, rather than a fixed-duration sequence followed by idle padding.
Q4: Practical significance.
A: The practical objective is to connect high-level conditions to a pretrained controller's existing whole-body capabilities. Table 2 evaluates trajectories actually executed in MuJoCo, so the retrieval gains reflect instruction alignment after control and physical simulation. SphLA also achieves lower executed-motion FID than both baselines. These results link the complete generation formulation to the quality of the resulting whole-body behavior.
For practical deployment, we evaluated SphLA on the Unitree G1's onboard Orin NX. With cached text conditioning, preparing complete 5-second and 10-second motion plans takes 0.866 and 1.683 seconds on average, respectively (Table 4). The independently trained text and vision-language experts also reuse the same frozen BFM actor network. This shared whole-body control interface provides a basis for developing multimodal human-robot interaction while retaining the pretrained controller across the two conditioning routes.
Thank you again for your constructive questions. We hope these clarifications address your concerns.