acceptodds
Under review as a conference paper at ICLR 2027

BeJEPA: A Goal-Conditioned Humanoid Behavior World Model via Unsupervised RL

16 views

Abstract

Humanoid control requires coordinating whole-body posture with position and heading in the world frame. Mainstream controllers imitate trajectories in the root frame, but struggle to regain the target world pose after displacement. We introduce BeJEPA, a goal-conditioned Behavior World Model learned through unsupervised reinforcement learning from human motion priors and robot interaction. BeJEPA learns a shared representation of body configuration and world placement through a joint-embedding predictive architecture (JEPA). A Dynamic Predictor predicts the next state embedding under an action, while a Successor Predictor summarizes a goal-conditioned policy's expected discounted future state embeddings in a single vector. During joint training, Dynamic grounds the initial representation in observed physical transitions, while Successor increasingly organizes it around long-term behavior. Their composition guides policy improvement through one physical prediction and one behavioral summary, without long imagined rollouts. In 20-second motion-tracking experiments, BeJEPA with online action refinement using Dynamic outperforms all evaluated baselines in global body-position error, reducing it on AMASS by 61.7% relative to BFM-Zero with world-pose inputs and 58.7% relative to SONIC. Physical deployment on a Unitree G1 demonstrates joint attainment of posture, world position, and heading goals, recovery from the ground, and closed-loop return after disturbances and goal changes.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Humanoid Whole-Body Control, World Models, robotics, Unsupervised Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_dUnNOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

BeJEPA is a goal-conditioned framework for humanoid control that jointly represents body posture, world position, and heading. It combines a one-step latent dynamics model with a successor predictor that summarizes expected future state features, using their composition to train a policy through unsupervised RL with motion priors. The dynamics model also supports online action refinement for improved tracking precision. Experiments demonstrate improved global tracking accuracy and real-world goal reaching, posture control, and recovery from disturbances on a Unitree G1.

Critical strengths
  1. The method introduces a novel combination of a one-step world model and a successor predictor for long-term behavioral prediction. Reusing the world model for online action refinement addresses practical limitations such as imprecise end-effector control and tracking lag, with promising potential for broader downstream applications.

  2. The experiments are solid, with detailed and convincing ablations supporting the design choices. The results show clear performance improvements, complemented by strong demonstrations on a real humanoid robot.

Critical weaknesses

I have no major concerns.

A useful extension would be to test newly specified rewards through both direct policy conditioning and online action refinement. Since BeJEPA learns Dynamic and Successor through prediction and TD losses rather than explicitly fitting the FB factorization, it would be valuable to verify whether it retains broader zero-shot reward-inference capabilities beyond goal reaching and tracking. Reporting the latency of online refinement would also help assess its practicality for real-time control. (Also, current BFMs still lack the precision needed for fine manipulation. Combining wrist-level online refinement with UMI-based manipulation is worth exploring, as it could improve end-effector accuracy while retaining the BFM’s whole-body coordination.)

Questions for authors
  1. What is the computational latency of online action refinement, and what hardware is used?
  2. Just curious: could adding an auxiliary critic for heading or root-position rewards during pretraining achieve similar improvements in global tracking? like what in https://arxiv.org/abs/2610.15833
2 comments

Thank you for your careful and constructive review.

1. Reward inference. We evaluate rewards for velocity tracking, height maintenance, and end-effector (wrist) pose. BeJEPA achieves higher rewards than BFM-Zero (world) in both the 2 s and 10 s evaluations.

Table 1. Reward inference (↑).

Duration (s)BeJEPABFM-Zero (world)
20.27950.2661
100.33490.2797

While reward inference is useful, its benefits for future manipulation research may be less direct. We therefore focus the paper on demonstrating world-coordinate goal reaching, which is relevant to manipulation.

2. UMI and wrist refinement. We agree that limited end-effector precision is a key issue for BFMs. We hypothesize that this may stem from how Forward–Backward representations encode wrist motion: periodic wrist movements during locomotion can be difficult to predict, and the Backward encoder may compress and average out these details.

Our initial motivation was to adapt a BFM-Zero-like low-level controller for robust humanoid loco-manipulation, with UMI data guiding the higher-level policy. This motivated two core innovations of our work: world-coordinate goal reaching and online wrist refinement. We appreciate this suggestion and will continue exploring UMI-based loco-manipulation with our controller.

3. Computational latency. We use GPU acceleration for inference. On a 5090 GPU, end-to-end latency from input receipt through MPC planning to action output is 11.97, 17.76, and 1.35 ms for Balanced, Root xy, and Wrist rotation modes, respectively. All fit within the 20 ms budget of a 50 Hz control cycle.

4. Auxiliary critic for world position and heading. We added an auxiliary critic conditioned on sts_t and sgoals_{\mathrm{goal}} during pretraining. It did not match BeJEPA's world-coordinate goal-reaching performance.

Table 2. Auxiliary critic ablation.

ModelRoot xy min (m) ↓Root xy mean (m) ↓Root yaw min (°) ↓Root yaw mean (°) ↓
BFM-Zero (world) + physical critic0.6261.3036.2935.11
BeJEPA0.1590.6422.4724.02

The cited P-BFM supports world-frame heading control but not reaching specified xy targets. Similarly, our BFM-Zero (world) can track and reach heading goals, but struggles with position goals that require coordinated stepping.

Prior work, such as Advanced Skills by Learning Locomotion and Local Navigation End-to-End, also studies end-to-end position goals reaching with RL. However, it often requires scenario-specific reward design, including arrival rewards, exploration rewards, and stagnation penalties. Such methods tend to focus on particular tasks and are rarely used in recent humanoid whole-body controllers. BeJEPA offers an end-to-end approach to world-coordinate goal reaching through unsupervised RL.

Reviewer_dUnNOpenReview ↗

Response to Rebuttal

Thank you for your thoughtful response. I have no remaining concerns. And thanks for doing the extra credit, even with rating and confidence already maxed out :)

Reviewer_2hpvOpenReview ↗
Rating
4(+1)
Confidence
4(+2)
Informativeness
3(+1)
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

This paper proposes BeJEPA, a goal-conditioned humanoid behavior world model that extends the unsupervised RL paradigm of BFM-Zero. It separates short-term physical prediction (Dynamic) from long-term behavioral prediction (Successor) to improve policy learning without long imagined rollouts. An additional MPC module refines actions during execution. Experiments demonstrate improved global motion tracking, benefits from the predictor decomposition, and successful real-world deployment on Unitree G1.

Critical strengths
  1. Meaningful architectural contribution. The Dynamic–Successor decomposition provides a principled way to combine short-term physical prediction with long-term behavioral reasoning. This is a meaningful extension of existing goal-conditioned humanoid control frameworks.

  2. Strong empirical results. BeJEPA substantially improves global tracking accuracy over BFM-Zero and other baselines. The ablation studies support the effectiveness of the proposed decomposition, and real-world demonstrations further establish its practical feasibility.

Critical weaknesses
  1. Insufficient understanding of why the proposed decomposition works. Although the ablations demonstrate the benefits of Dynamic and Successor, they do not fully establish whether the improvements arise from the proposed decomposition itself, the additional dynamics supervision, or the training schedule. This limits the strength of the paper's central mechanistic claim.

  2. Limited evaluation of challenging long-horizon goal reaching. The paper emphasizes autonomous goal reaching without intermediate motion references, but the strongest quantitative results focus on motion tracking. The fixed-goal experiments mainly involve relatively simple posture and position targets. More challenging goal-reaching evaluations would better establish the advantages of the proposed long-term behavioral prediction.

Questions for authors
  1. Can the authors further isolate the benefits of the Dynamic–Successor decomposition from those of additional dynamics supervision and the Dynamic loss-weight schedule?

  2. Can the authors provide quantitative comparisons with BFM-Zero on more challenging long-horizon goal-reaching tasks that require complex intermediate behaviors?

2 comments

W1 / Q1: Separating the contributions.

Our method has two main contributions: jointly learning a world model with Dynamic and Successor, and training the actor through their composition.

As requested, we added Merged+Dyn: a Dynamic Predictor provides one-step dynamics supervision to the shared encoder, while the actor is still trained through the merged, action-conditioned Successor.

Table 1. Goal-reaching ablation.

ModelRoot xy min (m) ↓Root xy mean (m) ↓Root yaw min (°) ↓Root yaw mean (°) ↓
Merged0.6661.3486.4135.81
Merged+Dyn0.1920.6793.0824.32
BeJEPA0.1590.6422.4724.02

Adding Dynamic substantially improves Merged, supporting the importance of one-step prediction for preserving features useful for long-term behavioral prediction. BeJEPA further improves accuracy and converges faster.

Table 2. Training steps to first reach each error threshold.

MPJPE-G thresholdBeJEPAMerged+Dyn
≤1000 mm7.2M10.8M
≤500 mm10.8M14.4M
≤240 mm32.4M90.0M

We attribute this additional benefit to composing Dynamic and Successor for actor training. The Successor–goal inner product acts as V(z)V(z), whereas BFM-Zero and Merged+Dyn learn Q(z,a)Q(z,a). Sharing continuation values across actions that reach the same latent state can improve learning efficiency (see related value-function work).

We also ablated the Dynamic loss-weight schedule and learning-rate decay. Without them, MPJPE-G still falls below 240 mm, but can rise above 240 mm again with extended training. We therefore use both schedules to stabilize training and assess convergence.

W2 / Q2: Challenging long-horizon goal reaching.

We added a sequential goal-reaching comparison with BFM-Zero (world): (1) stand up from arbitrary fallen poses, (2) move a specified distance, (3) withstand external forces and velocity perturbations, and (4) turn by a specified angle and adopt a target whole-body pose. This tests whether the policy can connect ground recovery, upright balance, locomotion, disturbance rejection, and posture adjustment within one task.

Table 3. Sequential goal-reaching performance.

MethodStand-up success ↑Stand-up time (s) ↓Final position error (m) ↓Final heading error (°) ↓Disturbance-induced position error (m) ↓
BFM-Zero (world)100%1.481.74944.050.392
BeJEPA100%1.170.3932.530.188

These results highlight BeJEPA's versatility in complex goal reaching.

Thank you for the detailed rebuttal and the additional experiments.

I particularly appreciate the authors' careful and transparent ablation study. The new Merged+Dyn experiment clearly distinguishes the contribution of dynamics supervision from that of the Dynamic–Successor decomposition. Rather than overstating the benefits of the proposed architecture, the authors provide an honest analysis of where the improvements come from. The additional convergence results further demonstrate the value of the proposed decomposition.

The new sequential goal-reaching experiments also effectively address my concerns about long-horizon control capabilities.

Overall, I believe the rebuttal has adequately addressed my concerns and significantly strengthened the paper. I am satisfied with the authors' responses and recommend acceptance.

Reviewer_v1onOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BeJEPA combines an action-conditioned Dynamic predictor and a goal-conditioned Successor predictor to train humanoid policies for joint posture and world-pose control. Dynamic also supports online MPC refinement. The problem is relevant, and the reported global tracking gains are substantial.

Critical strengths

Actor-only BeJEPA improves AMASS global MPJPE to 278.0 mm, versus 573.4 mm for world-input BFM-Zero and 531.5 mm for SONIC. MPC further reduces it to 219.5 mm. The physical demonstrations and encoder-gradient ablations are useful.

Critical weaknesses

1. The world-model benefit is not sufficiently isolated. BeJEPA receives persistent world-pose feedback, whereas standard SONIC lacks an equivalent interface. The paper does not clarify whether its SONIC baseline was adapted. Please evaluate a properly trained or fine-tuned SONIC variant with equivalent position/heading feedback, world-anchored targets, and a compatible objective, or a comparably strong model-free tracker. Report training resources and reference look-ahead. SONIC already achieves better AMASS local MPJPE (49.7 mm versus 82.3 mm for BeJEPA); suitable global feedback might narrow the global gap, although this must be tested.

The world-input BFM-Zero comparison is valuable, and MPC demonstrates practical utility. Nevertheless, goal recovery alone does not establish an advantage of predictive modeling. Existing tracking formulations already support global anchor feedback [1]. The motivation should identify the comparative benefit established by the experiments, rather than imply that a world model is required for world-goal control.

2. The decomposition ablation has a confound. Table 2 changes both predictor structure and separately supervised dynamics learning. The stop-gradient variants retain the split architecture. Please compare against a merged action-conditioned successor with the same dynamics auxiliary loss on the shared encoder, matching capacity, data, and training objectives where feasible. This would distinguish the proposed decomposition from the benefits of auxiliary predictive representation learning.

3. Successor's current-policy interpretation needs justification. Equation 9 and the supplied implementation bootstrap a state-only Successor from replay transitions under the collection goal. Even with a fixed encoder, these transitions reflect historical behavior and exploration, not necessarily the current actor. Retaining the goal does not retain the policy. The fixed-policy analysis does not resolve this distinction; action-conditioned off-policy critics instead query the target policy in their backup [2]. Please clarify the evaluated policy and calibrate Successor against fresh rollouts of a frozen actor, including sensitivity to replay age or exploration. This is an interpretation and validation gap, not proof that the algorithm fails.

4. Evaluation provenance and uncertainty are incomplete. Please provide the exact Table 1 motion IDs/time windows, training/test relationships, baseline configurations, and failure/reset scoring. The README identifies a 40-motion LAFAN evaluation archive but does not establish the full Table 1 protocol. Report independent training runs, evaluation counts, and uncertainty for central comparisons, distinguishing policy-level from rollout-level variation. Global/local errors and completion or fall statistics should be reported together. No data-leakage allegation is implied.

Questions for authors

The weakness summarizes my questions, and there are some minor concerns.

  • Distinguish actor-only hardware deployment from MPC evaluation. Report MPC hardware, measured latency, activation frequency, and reproduction entry points; 50 Hz simulation control does not establish real-time inference.
  • Separate the scheduled Dynamic loss-weight decay from the gradient-stage interpretation in Figure 8. Report absolute lag or its distribution alongside mean signed lag in Figure 3b.
2 comments

Thank you for your review. References[1],[2] appear to lack links, so we consulted related work in our response.

W1: Why World Model? World models’ strength lies in goal reaching without reference and data efficiency.

SONIC requires 9,000+ GPU hours, making retraining infeasible. We therefore compared against MimicLite, an open-source model-free tracker with better reported metrics. We added BeJEPA’s world-pose inputs and anchors to MimicLite and retrained it.

SONIC require reference trajectories, whereas BeJEPA can reach goals without intermediate references. For standing up from the ground without such references, SONIC and MimicLite both achieve 0% success, versus over 95% for BeJEPA. The goal-inference comparison below shows that world-pose inputs alone do not enable MimicLite to reach targets without trajectory guidance.

Table1. Goal inference.

MethodRoot xy min(m)↓Root xy mean(m)↓Root yaw min(°)↓Root yaw mean(°)↓
BeJEPA0.1590.6422.4724.02
MimicLite(world)0.6240.89831.3158.18

On AMASS, excluded from both methods' training sets, BeJEPA achieves lower global tracking error using only 1.7% as much motion data as MimicLite.

Table2. AMASS tracking (mm).

MethodMPJPE-G↓MPJPE-L↓
BeJEPA278.082.3
MimicLite(world)401.073.8

MPJPE-L measures local pose matching but does not capture world-position accuracy. Following human motion references at the correct world position can require larger or additional steps, increasing MPJPE-L. Thus MPJPE-G is a better and more meaningful metric for humanoid applications.

W2: Decomposition and dynamics supervision. Our core innovation in unsupervised RL is adding the Dynamic Predictor, which structures world-coordinate representations. We added an auxiliary Dynamic Predictor to Merged solely for encoder supervision(Merged+Dyn). Its substantial improvement over Merged supports the importance of Dynamic Predictor.

Table3. Goal inference with dynamics supervision.

ModelRoot xy min(m)↓Root xy mean(m)↓Root yaw min(°)↓Root yaw mean(°)↓Wrist xy min(cm)↓
BeJEPA0.1590.6422.4724.024.66
Merged+Dyn0.1920.6793.0824.325.01
Merged0.6661.3486.4135.815.15

BeJEPA still performs slightly better and converges faster. We attribute this additional benefit to chaining Dynamic and Successor to train the actor. The Successor–goal inner product acts like V(z)V(z), whereas BFM-Zero and Merged+Dyn learn Q(z,a)Q(z,a). Sharing continuation values across actions reaching the same latent improves learning efficiency (see related value-function work)

Table4. Training steps to first reach each error threshold.

MPJPE-G thresholdBeJEPAMerged+Dyn
≤1000mm7.2M10.8M
≤500mm10.8M14.4M
≤240mm32.4M90.0M

W3: Successor's policy interpretation. Successor predicts discounted future features under the replay-induced, goal-conditioned behavior policy μg\mu_g. Dynamic predicts a candidate action's outcome, and Successor evaluates the subsequent behavior. Chaining them approximates Qμg(s,a)Q_{\mu_g}(s,a): take the candidate action, then follow μg\mu_g. The actor uses these behavior-policy values to improve its actions, without requiring Successor to immediately match the new policy after every actor update.

With the actor and encoder frozen, we further trained the predictors against future features from fresh rollouts. BeJEPA's predictor error decreased only from 0.35 to 0.33 (5.7%), versus 0.33 to 0.31 (6.1%) for BFM-Zero. Thus, even before further calibration, Successor was close to its calibrated accuracy under the frozen actor, with a calibration gap comparable to the action-conditioned baseline.

W4: Evaluation protocol. AMASS is held out from BeJEPA and all baselines; LAFAN is partly included in training. For the paper's Table 1, we split the full datasets into 20-second motion segments and discarded shorter remainders. SONIC and BFM-Zero use their official GitHub implementations. BFM-Zero (world) adds BeJEPA's 4 Dim world-pose inputs and uses the same evaluation configuration. Fall rates are 15.02% for SONIC and 1.64% for BeJEPA.

Q1: MPC latency. On a 5090 GPU, end-to-end latency from input receipt through MPC planning to output is 11.97/17.76/1.35 ms for Balanced/Root xy/Wrist rot mode, respectively, all within a 50 Hz control cycle.

Q2: Figure, Gradient schedule and tracking lag. We will improve the figures.

In Figure 8, the median Dynamic/Successor gradient-norm ratio already drops from 2.63 at 0–5M steps to 0.39 at 15–30M, while the loss weight remains constant throughout the first 30M steps.

For Figure 3b, Balanced MPC reduces mean absolute lag from 94.75 to 34.75 ms on AMASS and from 97.18 to 53.33 ms on LAFAN, confirming that the timing improvement is not merely cancellation of positive and negative lags.

We greatly appreciate your helpful review comments. We have added the ablation experiments as you suggested. As the discussion period is approaching its end, we would greatly appreciate any further feedback on our responses, particularly whether any key concerns remain unresolved.

Thanks for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.