acceptodds
Under review as a conference paper at ICLR 2027

SphLA: Spherical Latent Action Flow Matching for Humanoid Whole-Body Control

Abstract

Behavioral foundation models (BFMs) provide pretrained whole-body policies for humanoid robots, supporting diverse behaviors through a shared policy. The latent representation of a Forward-Backward BFM conditions whole-body control. We use this representation as a generative action space, translating high-level task specifications into latent actions executed by the BFM's actor network without synthesizing kinematic motion references. However, these latent actions lie on a hypersphere, whose geometry conventional Euclidean flow matching does not preserve. To address this limitation, we introduce SphLA, spherical latent-action flow matching, which formulates conditional generation on this hypersphere. Our approach combines geodesic probability paths, tangent velocity prediction, and exponential-map integration to preserve the latent norm throughout sampling. SphLA supports both language and vision-language conditioning through independently trained action experts that share the same spherical body latent space and actor network. Experiments show that SphLA preserves spherical latent geometry and improves text–action alignment over Euclidean flow matching. Real-robot experiments validate language-directed humanoid whole-body control.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Spherical flow matching, language-conditioned action generation, behavioral foundation models, humanoid control

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DridOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SphLA, which uses Spherical Flow Matching to generate latent actions from language and vision-language inputs. These actions are executed by a pretrained BFM actor for humanoid whole-body control.

Critical strengths

See "Questions For Authors".

Critical weaknesses

See "Questions For Authors".

Questions for authors
  1. Although the paper presents real-robot experiments, I could not find an accompanying demonstration video in the provided materials. Without such a video, it is difficult to assess the actual motion quality and stability.

  2. The paper mainly compares SphLA with standard Euclidean Flow Matching. However, since SphLA explicitly preserves spherical geometry by design, the reduction in radial error is largely expected and does not necessarily demonstrate the superiority of the proposed method.

  3. The paper primarily applies existing Riemannian Flow Matching techniques to the spherical latent space of a BFM. Both the key geometric techniques and the idea of directly generating latent actions have been explored in prior work. I would appreciate clarification on what substantial technical innovations this work introduces beyond existing methods.

  4. I am not convinced that this work addresses a sufficiently significant problem. Although SphLA improves latent generation performance, the improvements appear relatively modest, and their practical significance remains unclear. Therefore, I have concerns about the overall significance and contribution of this work.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Drid: Contribution Clarifications and Supporting Evidence

Thank you for your thoughtful questions about the contribution and its practical value. We address your questions below.

Q1: Demonstration videos.

A: Thank you for highlighting the need for video evidence. We have provided continuous text-conditioned real-robot videos and vision-language body-motion replays in simulation using recorded egocentric video on our anonymous video page. Both use the same frozen BFM actor network as the low-level whole-body controller.

Q2: Evidence beyond norm preservation.

A: Thank you for raising this point. Our evaluation examines whether SphLA's conditional generation in the BFM control representation improves instruction alignment and the resulting whole-body behavior. We compare it with both explicit reference-trajectory generation and Euclidean generation in the same latent space, and further examine whether the control interface can be reused under vision-language conditioning.

In Table 1, SphLA improves R@1 by 1.92 percentage points over the spherical-prior FM baseline, whose outputs are also normalized before evaluation. In Table 2, the joint-space baseline generates kinematic references that are subsequently encoded for execution, while latent-space FM and SphLA directly generate control latents. Using the same frozen BFM actor network and evaluation protocol, SphLA improves executed-motion R@1 by 5.47 and 4.14 percentage points over latent-space and joint-space FM, respectively. These comparisons demonstrate benefits in simulated execution beyond radial-error reduction.

The expanded vision-language evaluation further demonstrates reuse of the same frozen BFM actor network across conditioning modalities. On 70 held-out episodes across three G1 tasks, with matched training conditions and NFE 1, SphLA reduces joint MAE from 10.53° to 9.61° and root-relative MPJPE from 154.84 to 61.28 mm relative to FM. These results measure simulated body-motion fidelity with recorded egocentric video input and current body-state feedback. The text and vision-language experts share the same frozen BFM actor network, supporting whole-body execution across both conditioning modalities.

Q3: Contribution beyond existing geometric and latent-action methods.

A: SphLA applies Riemannian flow matching to latent-action generation for humanoid whole-body control, formulating conditional sequence generation in the fixed spherical control representation of a pretrained Forward–Backward BFM.

Among related text-driven humanoid systems, TextOp and OMG supply kinematic references to a tracker through latent decoding and direct motion-feature generation, respectively. SphLA directly generates inputs for the frozen BFM actor network. Reference motions are encoded offline by the frozen backward network and normalized to form spherical latent targets. At inference, generated latents and robot-state feedback drive the frozen BFM actor network, without an intermediate kinematic reference. This aligns the generation target and its geometry with the existing control interface.

In addition, we examine data scale and motion duration to further characterize text-conditioned generation in this latent space. Figure 7 in Appendix G.2 shows that larger training subsets yield more sustained validation velocity alignment. The motion duration can either be predicted from the text by an auxiliary length head or specified directly by the user. Figure 6(c) illustrates the same instruction executed over requested durations of 2, 4, and 6 seconds. These examples show motion adapting to the requested duration and spanning the full interval, rather than a fixed-duration sequence followed by idle padding.

Q4: Practical significance.

A: The practical objective is to connect high-level conditions to a pretrained controller's existing whole-body capabilities. Table 2 evaluates trajectories actually executed in MuJoCo, so the retrieval gains reflect instruction alignment after control and physical simulation. SphLA also achieves lower executed-motion FID than both baselines. These results link the complete generation formulation to the quality of the resulting whole-body behavior.

For practical deployment, we evaluated SphLA on the Unitree G1's onboard Orin NX. With cached text conditioning, preparing complete 5-second and 10-second motion plans takes 0.866 and 1.683 seconds on average, respectively (Table 4). The independently trained text and vision-language experts also reuse the same frozen BFM actor network. This shared whole-body control interface provides a basis for developing multimodal human-robot interaction while retaining the pretrained controller across the two conditioning routes.

Thank you again for your constructive questions. We hope these clarifications address your concerns.

Dear Authors,

Thank you for your response. We still have the following concerns:

  1. UFO is pretrained on LAFAN1, while the generator is trained on BONES-SEED. We find this cross-dataset setup quite puzzling.
  2. We appreciate the release of the model weights. After downloading and testing the model, we found that some instructions were not executed satisfactorily. This raises concerns that the low-level controller may limit the benefits of high-level motion generation.
  3. The evaluation uses an author-selected test subset whose representativeness and difficulty remain unclear. This makes the reported results less convincing.

As the rebuttal period draws to a close, we are not requesting additional experiments. I am very sorry, but given the unresolved questions above and my concerns about the paper’s novelty, I find it difficult to revise my score at this stage. I hope you will continue to refine this work, clarify its contributions, and strengthen its validation.

AuthorsOpenReview ↗

Follow-up to Reviewer Drid: Clarifications on Evaluation and Contribution

Thank you for following up and for testing the released model. We address the remaining concerns and clarify the contribution below.

  1. Why use LAFAN1 for the controller and BONES-SEED for the generator?

We use UFO's released BFM-Zero-based controller, pretrained using LAFAN1 motion data. BFM-Zero combines online unsupervised RL with motion-data regularization and reports cross-dataset tracking evaluation on 175 AMASS motions (Sec. 3.1).

BONES-SEED supplies paired language descriptions and motions for generator training. We retarget these motions to G1 and encode them with the same frozen backward network to obtain normalized spherical targets. The encoder and actor remain frozen, keeping the representation connecting generation and execution fixed. The controller maps robot states and latents to actions; SphLA learns which latent sequence to generate from language. These objectives do not require identical training datasets.

  1. Instructions that were not executed satisfactorily.

Thank you for reporting this experience. Execution depends on both generated latents and controller capability. SphLA's generalization can vary with phrasing and the coverage of BONES-SEED training motions. Concrete descriptions of actions, body involvement, and movement direction reflect the training caption format.

We would appreciate the tested instructions, durations, and inference settings to help reproduce these cases and distinguish generation from control limitations. Table 2 keeps the controller fixed and evaluates actual executions, showing that the generator's contribution remains measurable through the same frozen BFM actor network.

  1. Representativeness and difficulty of the evaluation.

Table 1 uses the full 4,189-sample validation partition with three generation seeds. The text–latent evaluator is trained on reference pairs and kept frozen; generated samples are not used for evaluator training or model selection.

For Table 2, we sample 100 distinct captions outside generator training to balance rollout cost and action coverage. The set covers walking, jogging, jumping, dancing, turning, manipulation, idle behavior, crouching, and stretching, with 8–12 captions per category. Each caption is evaluated at 2, 4, and 6 seconds with two generation seeds, yielding 600 rollouts per method. The motion evaluator is trained on reference trajectories and scores actual MuJoCo executions. All rollouts, including falls, enter the metrics under the same controller and evaluation protocol. Retrieval tests cross-category instruction alignment; FID, Diversity, and fall rates assess motion distribution, variation, and stability.

Appendix E.4 compares SphLA and OMG on 63 motion-name matches between our test set and the BONES-SEED portion of OMG's official test set. Both caption versions independently condition generation with two seeds, and both our evaluator and OMG's released evaluator score the executions. SphLA achieves higher R@1/2/3 across all four caption–evaluator combinations.

Further clarification on the contribution.

The central distinction is the generation target: our joint-space FM baseline, TextOp, and OMG generate kinematic reference trajectories, whereas SphLA directly generates control latents for the frozen BFM actor network. The frozen backward encoder supplies spherical supervision offline, and we formulate conditional sequence generation in this representation using Riemannian FM. Text and vision-language experts reuse the same frozen BFM actor network; the VLA evaluation uses recorded visual inputs and simulated-state feedback.

With the controller and evaluation protocol held fixed, Table 2 shows higher executed-motion R@1/2/3, lower FID, and a lower observed fall rate than joint-space FM and Euclidean latent FM on held-out captions. The OMG cross-evaluation provides further evidence across annotation styles and evaluators. Figure 6(c) demonstrates the same instruction over 2-, 4-, and 6-second horizons; duration can be specified or predicted by an auxiliary head. With resident models on the G1's Orin NX, uncached text-to-plan preparation takes 898.3 ms for 5-second plans and 1716.0 ms for 10-second plans on average. This demonstrates onboard generation substantially faster than the resulting motion's execution duration.

Matched simulation and real-robot videos are now available on our anonymous supplementary website. Using the same text instructions, these videos provide qualitative evidence of consistent motion execution and action transitions across sim2sim and sim2real deployment for SphLA's text route.

Thank you again for considering these clarifications and the additional evidence.

Reviewer_Z3Y8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents SphLA, a geometry-aware flow-matching method for generating latent action sequences for humanoid BFMs. Since the controller’s latent actions lie on a fixed-radius sphere, SphLA uses geodesic interpolation, tangent-space velocities, and exponential-map integration to preserve this geometry during generation. The method is applied to text-conditioned motion generation and a separate vision-language-conditioned action model. Validation includes controlled comparisons with Euclidean flow matching, physics-based execution, recorded-condition VLA replay, and qualitative deployment on a Unitree G1 robot.

Critical strengths
  1. Well-motivated geometry and a reusable interface. The spherical formulation matches the normalized BFM latent space, is clearly presented, and reuses an existing controller without retraining it.

  2. Meaningful controlled text-generation results. The text route includes controlled comparisons, improving executed-motion retrieval R@1 from 0.7560 to 0.8107. Real-robot deployment further demonstrates practical feasibility.

Critical weaknesses
  1. The experiments do not establish closed-loop, high-level interaction. VLA evaluation uses only ten training episodes from 138 demonstrations. The expert sees recorded images and proprioception; only the low-level actor receives evolving simulated-state feedback, and predicted hand actions are never executed. The results therefore show body-motion replay fidelity, not visually grounded manipulation. Real-robot demonstrations use the text route and do not close this gap. Since that route pre-generates full sequences, it also does not test adaptation to execution deviations. The replay protocol is disclosed, but evidence for closed-loop multimodal control remains limited. (Sec. 4.3; Appendices C.1, C.4, and F.)

  2. Generalization and interactive deployment are also under-validated. Although the paper tests held-out text, overlap between test motions and the frozen BFM’s pretraining data, and whether the real-robot instructions are held out, remain unclear. Hardware results omit trial counts and quantitative task-completion measures. The reported 50 Hz is only the low-level actor rate: onboard generation takes about 0.866 s for 5-second sequences and 1.683 s for 10-second sequences, excluding cached text features and initial encoding. This shows generation is feasible, not that the system responds to new instructions or visual changes; VLA latency is unreported. (Sec. 4.4; Table 4; Appendix G.)

Questions for authors
  1. Are there VLA results on held-out episodes, or closed-loop manipulation results using live images from the system's own execution? If not, please limit the corresponding conclusions to body-motion replay under recorded conditioning.

  2. What motion and text overlap exists between BFM pretraining, generator training, and evaluation? Are the real-robot instructions held out, and are repeated-trial results, failure cases, and quantitative completion rates available?

  3. What is the full instruction-to-action latency for uncached instructions, and the observation-to-action latency for the VLA route? Does the current system update high-level sequences during execution in response to new observations or execution deviations?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Z3Y8: Evaluation Protocols and Additional Results

Thank you for recognizing the motivation of our spherical formulation, the reusable control interface, and the controlled text-generation results. We address your three questions below.

Q1: Held-out VLA evaluation and closed-loop manipulation.

A: Thank you for highlighting the need for held-out evaluation and clarifying the scope of closed-loop control. We have expanded the VLA study to 700 demonstrations across three G1 tasks and evaluated both FM and SphLA on all 70 held-out test episodes, which were excluded from fine-tuning. We have completed the expanded evaluation for Table 3.

Visual inputs come from recorded egocentric video, rather than live images from the system’s own execution. Given our hand/gripper hardware limitations, we focus on body-motion execution in simulation, without executing the separate hand outputs. At each action-chunk update, the expert receives the instruction, a video frame aligned to simulation time, and current simulated body proprioception. The same frozen BFM actor network used by the text route executes the predicted body latents with simulated-state feedback. Thank you for pointing this out. We will revise the manuscript to align the VLA claims with the scope of our evaluation.

Video supplement.

Q2: Data overlap, real-robot instructions, and repeated trials.

A: Thank you for highlighting the importance of clarifying these data relationships. We use UFO’s released frozen backward encoder and actor, with LAFAN1 documented as its motion-training corpus, while our generator uses BONES-SEED, comprising 79,599 training and 4,189 validation pairs. The latent-space evaluator in Section 4.1 and the execution-trajectory evaluator in Section 4.2 use reference pairs from the same corpus, with the generator’s 4,189-sample validation holdout serving as their test partition and source groups kept disjoint across their evaluator splits.

For Table 2, we balance physical-rollout cost and action coverage by sampling 100 distinct captions outside generator training, spanning nine common action categories. Each caption is evaluated at 2, 4, and 6 seconds using generation seeds 0 and 1, yielding 600 simulated rollouts per method under the same frozen BFM actor network and evaluation protocol. All executed trajectories—including falls—contribute to retrieval, FID, and Diversity. A fall is defined as root height dropping below 0.35 m during the task; Table 2 reports 6.00% for SphLA versus 7.33% for each baseline. In Appendix E.4, we identify 63 motion-name matches between the BONES-SEED subset of OMG’s official test set and our test set, each with different OMG and BONES-SEED text annotations. Using two generation seeds, both OMG and SphLA generate and execute motions under each caption version. The resulting trajectories are scored using both OMG’s publicly released evaluator and our execution-trajectory evaluator. SphLA achieves higher R@1/2/3 across all four caption–evaluator combinations. We will clarify the data splits and evaluation-set selection in the appendix.

Q3: Latency and updates during execution.

A: Thank you for the question. We measured both routes with models resident and the frozen BFM actor network running concurrently. For the text route, SphLA generates a complete latent-action sequence from the instruction before execution. Uncached-text timing includes tokenization, T5 encoding, and complete latent-plan generation up to plan readiness, excluding model loading, playback transitions, and physical actuation. On the H20 host, mean/P95 latency is 114.1/152.4 ms for 5-s plans and 159.6/173.4 ms for 10-s plans (100 requests per duration). Mean text-processing times are 10.73 ms and 11.09 ms, with the remaining preparation taking 103.41 ms and 148.48 ms, respectively. We will shortly supplement these results with onboard Unitree G1 measurements under the same uncached-text timing protocol. The text route executes pregenerated plans without automatic high-level replanning. Thank you for highlighting this point; we will document the timing boundaries and update schedules in the appendix.

For the VLA route, in a separate H20 latency benchmark with real-time chunking (RTC) enabled, we configured the expert to update action chunks at 2.5 Hz using current simulated body proprioception, while the frozen BFM actor network was configured to execute at 50 Hz. Mean/P95 steady-state latency from observation assembly to return of the complete latent chunk is 214.9/230.1 ms. Under this protocol, the VLA expert updates high-level action chunks during execution using the latest simulated body-state observations.

Thank you again for your detailed questions. We hope these clarifications and supporting evidence address your concerns and make the evaluated capabilities clearer.

AuthorsOpenReview ↗

Follow-up to Reviewer Z3Y8: Onboard Latency and Real-Robot Videos

Following up on our earlier response, we have now completed the additional uncached-text benchmark on the G1’s onboard Orin. With T5 and DiT kept resident, we cleared the fixed prompt’s text-feature cache before every request. Mean/P95 instruction-to-plan-ready latency is 898.3/901.5 ms for 5-s plans and 1716.0/1727.8 ms for 10-s plans (100 requests per duration).

We have also added real-robot videos to our anonymous supplementary website, using the same text instructions as the three simulation scenarios. The matched simulation and real-robot videos provide qualitative evidence of consistent motion execution and action transitions across sim2sim and sim2real deployment for SphLA’s text route.

Reviewer_Z3Y8OpenReview ↗

Response to the rebuttal

Thank you for the additional held-out evaluation and latency tests. These address several of my concerns. However, the real-robot validation remains limited, and the motions in the videos still look somewhat unstable. I think this is an important research direction, but given these remaining concerns, I will keep my current score.

AuthorsOpenReview ↗

Follow-up to Reviewer Z3Y8: Hardware Evaluation

Thank you for acknowledging that the additional held-out evaluation and latency measurements address several of your concerns. We take your observations on real-robot motion stability seriously. Our central contribution is conditional generation directly in the control representation of a pretrained BFM. The controlled comparisons evaluate different generation formulations through the same frozen BFM actor network and measure the motions actually executed in physics simulation. They therefore provide evidence of improved instruction alignment after execution, alongside the qualitative hardware demonstrations.

The expanded held-out evaluation further supports reuse of this body-control interface under vision-language conditioning, and the uncached onboard measurements establish the feasibility of generating complete motion plans on the G1. We hope these additions help distinguish the evidence supporting the generation method from the remaining questions about hardware robustness. Given the concerns addressed during the discussion, we would appreciate your reconsideration of the overall assessment in light of this strengthened evidence. We would also be happy to clarify any specific motion or transition in the videos that remains concerning. Thank you again for your careful assessment.

Reviewer_MZEfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SphLA, a conditional generative model operating in the spherical latent-action space of a pretrained Forward-Backward behavioral foundation model (BFM). The method combines geodesic interpolation, tangent-space velocity projection, and exponential-map integration to preserve latent norms during generation. A frozen actor converts generated latent sequences into whole-body control actions.

Critical strengths

A coherent design aligned with the downstream control interface. The fixed-norm latent representation naturally motivates spherical generation. The training paths, velocity supervision, and sampling updates form a consistent framework, and the norm-preservation property is clearly explained.

Critical weaknesses
  1. The methodological novelty needs sharper positioning. The paper explicitly builds on Riemannian Flow Matching and its prior applications to robot motion. Geodesic interpolation, tangent projection, and exponential-map integration are established tools. The main contribution therefore appears to be their application to the latent-action interface of a frozen FB-based BFM, together with empirical validation. This combination is useful, but the paper should clarify what challenges specific to this control interface require new methodological developments.

  2. Insufficient vision-language evaluation. Evaluation uses only ten training episodes with recorded observations, without executing hand outputs or reporting manipulation success. This does not establish generalization or closed-loop manipulation capability.

  3. Lack of real world deployment demo videos. Continuous videos would help assess motion quality, stability, and transitions.

Questions for authors

See Weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer MZEf: Clarifications and Additional Evidence

Response to Reviewer MZEf

Thank you for recognizing the consistency of our formulation and its alignment with the downstream control interface. We address your three concerns below.

Comment 1: Methodological novelty and challenges specific to the frozen BFM interface.

Response: Our contribution centers on generating the control latents consumed by a pretrained Forward-Backward BFM. We encode reference motions offline using its frozen backward network and normalize the representations to form spherical supervision targets for SphLA. At inference, SphLA directly generates latent sequences for the frozen BFM actor network, which combines them with robot-state feedback to produce whole-body commands without an intermediate kinematic reference. Table 1 shows a 1.92-percentage-point R@1 gain over Euclidean FM with a matched spherical prior and normalized FM outputs. Under the same frozen BFM actor network and evaluation protocol, Table 2 reports executed-motion R@1 gains of 5.47 and 4.14 points over latent-space and joint-space FM, respectively, with lower FID than both.

The interface-specific challenge is to generate instruction-conditioned temporal sequences in a representation whose semantics and scale are fixed by the pretrained controller. We apply Riemannian FM to humanoid latent-action generation, aligning training paths, velocity targets, and sampling updates with the spherical geometry of the frozen BFM control representation. Figure 4 provides qualitative evidence of local organization by action identity and instructed body involvement. Figure 7 in Appendix G.2 further shows that larger training subsets sustain stronger validation velocity alignment, supporting improved velocity-field learning across text-conditioned latent trajectories. Beyond text, a separately trained vision-language expert uses the same latent interface and frozen BFM actor network. This provides a shared whole-body control interface for both conditioning routes, supporting the development of multimodal human-robot interaction.

Comment 2: Limited vision-language evaluation.

Response: Thank you for the suggestion. We have expanded the submitted ten-training-episode evaluation to three public G1 whole-body demonstration datasets: pillow transport (300 episodes), fruit placement (200), and snack placement (200). These are 700 demonstration episodes across three tasks, split by complete episode into 560 training, 70 validation, and 70 test episodes. We fine-tuned FM and SphLA using the same training data, initialization, and training budget, and evaluated all 70 held-out episodes (30, 20, and 20 per task, respectively). For context, LocoManiLab uses 50 training demonstrations per task in its hardware study, while the official GR00T–SONIC soda-can example uses approximately 150.

At each action-chunk update, the expert receives the instruction, a recorded egocentric video frame aligned to simulation time, and current simulated body proprioception. The same frozen BFM actor network used by the text route executes the predicted body latents. Reference latents are obtained from the held-out demonstrations using the frozen backward network; all three execution sources are scored against the corresponding teleoperated body states. All 210 rollouts produced complete finite trajectories. The three-task macro results are below :

Latent source29-joint MAE (°) ↓Root-relative MPJPE (mm) ↓
Reference latents8.68567.747
FM10.530154.837
SphLA9.60861.275

Values are averaged equally across episodes within each task, then equally across the three tasks. Visual input comes from recorded video, and hand outputs are not executed; these results evaluate body-motion fidelity with high-level proprioceptive feedback, rather than manipulation success. The table above provides the updated aggregate comparison for Table 3; the per-task results will be included in Appendix F.

Comment 3: Continuous deployment videos.

Response: Thank you for highlighting the missing video evidence. To make motion quality, stability, and transitions easier to assess, we have provided continuous text-conditioned real-robot videos and simulation demonstrations on our anonymous video page. We also provide the text-to-motion model weights on the same page.

Thank you again for your careful assessment. We hope these clarifications and supporting evidence address your concerns and make the contribution clearer.

Thank you for the additional evaluation. The larger dataset and evaluation on 70 held-out episodes improve the evidence compared with the original ten-training-episode evaluation. For the vision-language results, I appreciate the clarification that these measure body-motion fidelity. Since the visual input is prerecorded, hand outputs are not executed, and manipulation success is not measured, they do not yet establish visually closed-loop manipulation. Please ensure this distinction is clear in the paper. The frozen BFM interface is useful, but my concern about the methodological contribution beyond applying existing Riemannian flow matching to this interface remains. The response clarifies the motivation and empirical benefits, but does not fully resolve this concern.

AuthorsOpenReview ↗

Follow-up to Reviewer MZEf: Methodological Contribution and Deployment Evidence

Thank you for acknowledging the strengthened held-out evaluation and for clarifying the remaining concern. We further clarify the methodological contribution and its practical implications below.

Our methodological contribution is the formulation of conditional sequence generation directly in a pretrained BFM's fixed control representation. The frozen backward encoder constructs spherical supervision from reference motions offline, and SphLA learns to generate the latent sequences consumed directly by the frozen BFM actor network. At inference, the actor combines these latents with robot-state feedback, without first generating and encoding an intermediate kinematic reference. We use Riemannian FM to formulate generation in this inherited spherical representation. Independently trained text and vision-language experts can use the same frozen BFM actor network through this shared whole-body control interface.

An informative result is that directly generating control latents does not automatically improve performance. Under the same frozen BFM actor network and evaluation protocol, Table 2 reports lower executed-motion R@1 for Euclidean latent FM than for joint-space FM (0.7560 versus 0.7693), whereas SphLA achieves 0.8107 and lower FID than both. Table 1 additionally compares methods with a matched spherical prior and normalizes FM outputs before evaluation. Together, these controlled comparisons support the value of the complete spherical latent-generation formulation beyond changing the prediction target or enforcing the endpoint norm alone.

The practical benefits also extend to generation efficiency. Under a shared timing protocol on the same M4 Pro, using identical prompts and requested motion durations, SphLA prepares complete 5-second and 10-second plans in 0.317 s and 0.514 s on average, compared with 12.394 s and 20.706 s for the released OMG paper-300M model. These measurements cover uncached text processing through complete output availability, using each model's native generation settings.

Related systems such as Kimodo focus on offline motion authoring. Our G1 deployment demonstrates onboard generation of control latents directly from language: with T5 and DiT resident on Orin NX, complete 5-second and 10-second latent plans are ready in 0.8983 s and 1.7160 s on average, including uncached text processing. Together with the controlled execution comparisons, these measurements demonstrate the executed-motion quality and practical efficiency of the complete SphLA formulation.

Following your suggestion, we will explicitly limit the vision-language conclusions in the manuscript to body-motion execution under recorded visual conditioning with proprioceptive feedback, without claiming visually closed-loop manipulation.

We also invite you to view the matched simulation and real-robot videos now available on our anonymous supplementary website. Using the same text instructions, these videos provide qualitative evidence of consistent motion execution and action transitions across sim2sim and sim2real deployment for SphLA's text route.

We would appreciate your reconsideration of the contribution in light of this clarified formulation and the supporting execution and deployment evidence. Thank you again for your careful assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the SphLA Rebuttal

We thank the reviewers for their careful feedback and continued discussion. For ease of reference, we summarize the contribution clarified in our responses, the additional evidence provided during the rebuttal, and the scope of the evaluated capabilities below.

  • Generation in the controller's representation (MZEf, Drid). SphLA formulates conditional sequence generation directly in the fixed spherical control representation of a pretrained BFM, using Riemannian FM. The frozen backward encoder converts retargeted BONES-SEED motions into supervision, while the frozen BFM actor network consumes the generated latents directly with robot-state feedback. This connects language-paired motion data to the released LAFAN1-pretrained controller without an intermediate kinematic reference at inference. Independently trained text and vision-language experts reuse the same frozen BFM actor network.

  • Controlled evidence beyond norm preservation (MZEf, Drid). Table 1 uses a matched spherical prior and normalizes FM outputs before evaluation. In Table 2, under the same controller and evaluation protocol, Euclidean latent FM achieves lower executed-motion R@1 than joint-space FM (0.7560 versus 0.7693), whereas SphLA achieves 0.8107 and lower FID than both. These comparisons support the complete spherical latent-generation formulation through actual execution, beyond changing the prediction target or satisfying the endpoint norm constraint alone.

  • Evaluation coverage and cross-evaluation (Z3Y8, Drid). We clarified the roles of the evaluation sets: Table 1 evaluates the 4,189-sample validation partition with three generation seeds; Table 2 uses 100 captions outside generator training across nine action categories, three durations, and two seeds, yielding 600 MuJoCo rollouts per method. All rollouts, including falls, enter the metrics. Observed fall rates are 6.00% for SphLA and 7.33% for each baseline. Appendix E.4 further compares SphLA and OMG on 63 matched motion names using two caption styles and two evaluators; SphLA achieves higher R@1/2/3 across all four caption-evaluator combinations.

  • Expanded held-out vision-language evaluation (MZEf, Z3Y8). We expanded the study from ten training episodes to 700 demonstrations across three G1 tasks and evaluated all 70 held-out test episodes. With matched fine-tuning conditions, SphLA reduces joint MAE from 10.530 to 9.608 degrees and root-relative MPJPE from 154.837 to 61.275 mm relative to FM. The expert updates action chunks using recorded egocentric video and current simulated body proprioception. These results measure body-motion fidelity with proprioceptive feedback; hand outputs are not executed and visually closed-loop manipulation is not established.

  • Generation latency and update schedules (MZEf, Z3Y8, Drid). On the G1's onboard Orin NX, uncached instruction-to-plan-ready mean/P95 latency is 898.3/901.5 ms for 5-second plans and 1716.0/1727.8 ms for 10-second plans, with resident models and 100 requests per duration. On the same M4 Pro, a shared uncached-text timing protocol gives mean generation times of 0.317/0.514 s for SphLA versus 12.394/20.706 s for the released OMG paper-300M model, using each model's native generation settings. Separately, the H20 VLA benchmark measures 214.9/230.1 ms mean/P95 observation-to-chunk latency, with expert and actor update rates configured at 2.5 and 50 Hz. The text route executes pregenerated plans; the VLA route updates chunks from current simulated body-state feedback.

  • Real-robot evidence and released materials (MZEf, Z3Y8, Drid). Our anonymous supplementary website now provides text-conditioned real-robot videos, matched simulation demonstrations, VLA simulation replays, and text-to-motion model weights. The matched videos use the same instructions to show execution and action transitions across simulation and hardware. They provide qualitative deployment evidence; systematic hardware robustness remains a limitation.

We will incorporate the clarified contribution, evaluation protocols, additional results, and capability boundaries into the revised manuscript. We hope this consolidated summary helps the reviewers and Area Chair assess SphLA's contribution and empirical support in light of the full discussion. As the discussion period draws to a close, we would be happy to answer any remaining questions before it ends. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.