- Rating
- 4
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
BeJEPA is a goal-conditioned framework for humanoid control that jointly represents body posture, world position, and heading. It combines a one-step latent dynamics model with a successor predictor that summarizes expected future state features, using their composition to train a policy through unsupervised RL with motion priors. The dynamics model also supports online action refinement for improved tracking precision. Experiments demonstrate improved global tracking accuracy and real-world goal reaching, posture control, and recovery from disturbances on a Unitree G1.
Critical strengths
-
The method introduces a novel combination of a one-step world model and a successor predictor for long-term behavioral prediction. Reusing the world model for online action refinement addresses practical limitations such as imprecise end-effector control and tracking lag, with promising potential for broader downstream applications.
-
The experiments are solid, with detailed and convincing ablations supporting the design choices. The results show clear performance improvements, complemented by strong demonstrations on a real humanoid robot.
Critical weaknesses
I have no major concerns.
A useful extension would be to test newly specified rewards through both direct policy conditioning and online action refinement. Since BeJEPA learns Dynamic and Successor through prediction and TD losses rather than explicitly fitting the FB factorization, it would be valuable to verify whether it retains broader zero-shot reward-inference capabilities beyond goal reaching and tracking. Reporting the latency of online refinement would also help assess its practicality for real-time control. (Also, current BFMs still lack the precision needed for fine manipulation. Combining wrist-level online refinement with UMI-based manipulation is worth exploring, as it could improve end-effector accuracy while retaining the BFM’s whole-body coordination.)
Questions for authors
- What is the computational latency of online action refinement, and what hardware is used?
- Just curious: could adding an auxiliary critic for heading or root-position rewards during pretraining achieve similar improvements in global tracking? like what in https://arxiv.org/abs/2610.15833
2 comments
Response to Rebuttal
Thank you for your thoughtful response. I have no remaining concerns. And thanks for doing the extra credit, even with rating and confidence already maxed out :)
Thank you for your careful and constructive review.
1. Reward inference. We evaluate rewards for velocity tracking, height maintenance, and end-effector (wrist) pose. BeJEPA achieves higher rewards than BFM-Zero (world) in both the 2 s and 10 s evaluations.
Table 1. Reward inference (↑).
While reward inference is useful, its benefits for future manipulation research may be less direct. We therefore focus the paper on demonstrating world-coordinate goal reaching, which is relevant to manipulation.
2. UMI and wrist refinement. We agree that limited end-effector precision is a key issue for BFMs. We hypothesize that this may stem from how Forward–Backward representations encode wrist motion: periodic wrist movements during locomotion can be difficult to predict, and the Backward encoder may compress and average out these details.
Our initial motivation was to adapt a BFM-Zero-like low-level controller for robust humanoid loco-manipulation, with UMI data guiding the higher-level policy. This motivated two core innovations of our work: world-coordinate goal reaching and online wrist refinement. We appreciate this suggestion and will continue exploring UMI-based loco-manipulation with our controller.
3. Computational latency. We use GPU acceleration for inference. On a 5090 GPU, end-to-end latency from input receipt through MPC planning to action output is 11.97, 17.76, and 1.35 ms for Balanced, Root xy, and Wrist rotation modes, respectively. All fit within the 20 ms budget of a 50 Hz control cycle.
4. Auxiliary critic for world position and heading. We added an auxiliary critic conditioned on st and sgoal during pretraining. It did not match BeJEPA's world-coordinate goal-reaching performance.
Table 2. Auxiliary critic ablation.
The cited P-BFM supports world-frame heading control but not reaching specified xy targets. Similarly, our BFM-Zero (world) can track and reach heading goals, but struggles with position goals that require coordinated stepping.
Prior work, such as Advanced Skills by Learning Locomotion and Local Navigation End-to-End, also studies end-to-end position goals reaching with RL. However, it often requires scenario-specific reward design, including arrival rewards, exploration rewards, and stagnation penalties. Such methods tend to focus on particular tasks and are rarely used in recent humanoid whole-body controllers. BeJEPA offers an end-to-end approach to world-coordinate goal reaching through unsupervised RL.