- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how to add fall recovery to a pretrained humanoid motion tracker while retaining a single deployed decoder and an unchanged state–reference interface. StableBFM continues PPO training on tracking environments and uses a recovery expert to label learner-visited states in a separate recovery branch. Both objectives update the same decoder. The paper considers an MLP and a context-conditioned Transformer, studies the frequency of recovery updates, and introduces cache perturbations for recovery supervision in the latter. Evaluation uses held-out AMASS and Motion Decode motions, a 192-trial fall-to-stand benchmark, and pushed tracking rollouts in MuJoCo after training in Isaac Lab. A Unitree G1 sequence provides qualitative hardware evidence. The MLP variant reports 93.23% eventual stand-up success and 73.60% motion recovery, with a reported post-training cost of two GPU-hours.
Critical strengths
-
The deployment constraint is useful and the recovery gains are substantial. The recovery expert is used during training, while inference retains the existing action decoder and control interface. Table 2 shows that the MLP acquires both stand-up recovery and return to an advancing reference, whereas the released tracker has zero recovery under these criteria.
-
The evaluation distinguishes several behaviors that are often conflated. The paper separates knockdown resistance, root recovery, upright recovery, motion recovery, and clean versus retried stand-up outcomes, and supplies numerical thresholds in Appendix B. The held-out motion sets and the update-frequency/context ablations are valuable. For example, the best setting for eventual stand-up success differs from the best setting for motion recovery (Table 3), which is an informative practical observation.
Critical weaknesses
-
The evidence supports a tracking–recovery trade-off more clearly than preservation of tracking performance. On Motion Decode, the MLP's tracking success decreases from 82.38% to 76.85%, while global MPJPE increases from 547.30 to 1181.90 mm (Table 2(a), p. 8). The context model also drops from its own reproduced baseline's 79.51% to 71.32% on this dataset. Near-preserved AMASS success does not resolve these losses. The harmonic Stability Score combines capabilities but does not establish that the original capability has been retained to an acceptable degree. This matters because preserving a pretrained tracker's behavior is a central motivation, rather than a secondary convenience.
-
The comparison does not yet fully separate recovery capability, evaluation selection, and training resources. Appendix B conditions recovery on each model's actual knockdowns, although knockdown rates differ; consequently the models need not be evaluated on the same subset of disturbed episodes for this metric. Appendix E also gives the STUBBORN reproduction rough-terrain training while the other recovery runs use flat ground. The two-GPU-hour figure is a post-training measurement, and the cost of training the reproduced BFM-Zero expert is not reported in that accounting. These choices do not invalidate the method, but limit the interpretation of direct superiority and efficiency claims. Training-seed variability is not reported, and the hardware evidence in Section 4.5 is qualitative, leaving the reliability of the complete trade-off uncertain.
Questions for authors
-
What level of tracking degradation is considered acceptable for the claimed preservation of the pretrained model? Can you explain the Motion Decode losses in Table 2, including the substantial increase in global MPJPE, and indicate whether a checkpoint or training setting yields a better tracking–recovery trade-off?
-
How many motions remain in the common no-push scored set, and how many pushed trials contribute to each model's conditional recovery denominator? Please report successful continuation over all matched pushed trials, or recovery over an explicitly shared knockdown subset, alongside the existing conditional rates.
-
What is the cost of obtaining the recovery expert, and how does the comparison change when that cost is included or explicitly treated as an amortized prerequisite? How sensitive is the STUBBORN comparison to matching the terrain and post-training conditions?
5 comments
Q2. Common scored set, recovery denominators, and a shared knockdown subset
Thank you for pointing out the potential interpretation bias from differing evaluation denominators. To address the concern that different disturbance resistance leads to different knockdown subsets across models, we explicitly report the complete evaluation counts and add a comparison on a strictly matched shared knockdown subset (Table P).
(a) Common motion set and actual denominators
All models are evaluated on exactly the same benchmark. The common no-push scored set contains 118 motions, with 10 individually matched external pushes per motion, following Section 4.1 and Appendices B.2/B.4 with random seed 20260903. Push magnitude, direction, application point, and timing are matched across all models, giving 1180 disturbed trials. The actual denominator of each model's original conditional recovery rate—its number of actual knockdown trials—is reported in the second column of Table P.
(b) Recovery on exactly the same knockdown trials
For a fully fair comparison, we take the strict intersection of trials in which all eight models are knocked down, yielding 100 disturbed trials. We evaluate recovery using the original continuous one-second motion recovery criterion.
Table P. Original conditional recovery and recovery on the shared knockdown subset.
| Model | Original conditional-recovery denominator (knockdowns) | Original motion recovery (%) ↑ | Recovery on the shared knockdown subset |
|---|---|---|---|
| Pretrained MLP | 809 | 0.00 | 0/100(0%) |
| Pretrained Transformer | 905 | 0.00 | 0/100(0%) |
| Recovery exploration (all environments allocated to recovery) | 819 | 53.70 | 54/100(54%) |
| Split-environment exploration | 800 | 61.80 | 62/100(62%) |
| BFM-Zero | 494 | 36.80 | 37/100(37%) |
| STUBBORN (post-training reproduction) | 824 | 42.10 | 42/100(42%) |
| StableBFM(MLP) | 814 | 73.60 | 74/100(74%) |
| StableBFM(Transformer) | 951 | 74.00 | 73/100(73%) |
On the identical shared knockdown subset, with denominator differences eliminated, StableBFM (MLP) still achieves 74% recovery, exceeding the next-best baseline, split-environment exploration (62%), and STUBBORN (42%) by 12 and 32 percentage points, respectively. This controlled comparison confirms that StableBFM's performance advantage comes from its recovery mechanism rather than evaluation-selection bias.
Q3. Teacher acquisition cost, amortization, and matched STUBBORN comparisons
Thank you for raising this point. Below, we analyze teacher-cost amortization and verify baseline fairness using strictly controlled ablation experiments.
(a) Teacher acquisition cost and reuse across students
StableBFM follows an efficient paradigm of training a teacher once and reusing it for multiple low-cost student upgrades through distillation. Even after fully accounting for teacher training—the BFM-Zero reproduction publicly provided through UFO costs approximately 4.5 GPU-hours, and MLP student post-training costs 2.0 GPU-hours—the total end-to-end cost of the first upgrade is only 6.5 GPU-hours, below the 7.50 GPU-hours required for a complete 4000-step STUBBORN post-training run. More importantly, the teacher policy is fully reusable in subsequent iterations, so each additional student upgrade costs only 2.0 GPU-hours, or 26.7% of STUBBORN's cost. Deployment retains only the original student decoder, with the same model capacity and control interface as the baseline and no additional inference overhead.
(b) Controlling training terrain and optimization steps
In the original experiments (Table C(a)), we match the pretrained tracker, motion pool, trainable modules, and evaluation protocol. We retain the original rough-terrain configuration and provide four times as many optimization steps for STUBBORN to explore recovery from the outset (4000 vs. 1000 steps), allowing sufficient exploration budget. To directly address whether terrain and optimization-step differences dominate the comparison, we add a controlled experiment that matches both flat-ground training and a 1000-step optimization budget (Table C(b)).
Table C(a). Original StableBFM and STUBBORN post-training configurations, following Appendix E.
| Setting | StableBFM(MLP) | STUBBORN (post-training reproduction) |
|---|---|---|
| Robot and control interface | 29-DoF G1; original PD, observation, and tokenizer interfaces | Same |
| Pretrained initialization | Official SONIC MLP, step 41,550 checkpoint | Same |
| Training motion pool | boneseed130k | Same |
| Trainable parameters | Action decoder (g1_dyn); all other modules frozen | Same |
| Total parallel environments | 4096 | 4096 |
| Checkpoint selection | Final checkpoint at the end of the training budget | Same |
| Training terrain | Flat ground | Rough terrain inherited from the pretrained tracker |
| Post-training optimization steps | 1000 | 4000 |
Table C(b). Controlled comparison with matched training terrain and post-training steps.
| Method / configuration | Training terrain | Post-training optimization steps | Motion Decode tracking success (%) ↑ | Eventual stand-up success (%) ↑ | Motion recovery (%) ↑ |
|---|---|---|---|---|---|
| STUBBORN (original configuration) | Rough terrain | 4000 | 73.92 | 83.85 | 42.10 |
| STUBBORN (matched ablation) | Flat ground | 1000 | 75.18 | 82.29 | 42.21 |
| StableBFM (ours) | Flat ground | 1000 | 76.85 | 93.23 | 73.60 |
Table C(b) clearly shows that when STUBBORN uses the same flat terrain and 1000-step optimization budget as StableBFM, its motion recovery remains at 42.21%, essentially unchanged from 42.10% with rough terrain and 4000 steps. A substantial gap of 31.39 percentage points remains relative to StableBFM's 73.60%. This demonstrates that StableBFM's effectiveness stems from its sample efficiency and guided recovery mechanism, rather than terrain or training-step bias.
Additional response to Weakness 2: real-robot reliability
To address real-robot deployment reliability, we provide the complete continuous recording corresponding to the hardware recovery sequence in Section 4.5: real-robot recovery video. The frame sequence in the paper is extracted from this original recording, which shows the full process of the G1 robot being disturbed, falling, autonomously adjusting its posture, and returning to a stable standing position.
To provide quantitative evidence of hardware robustness while accounting for joint wear and motor heating caused by repeated falls, we conducted 10 consecutive independent fall-recovery trials on the G1 under protective conditions. StableBFM stood up autonomously in 9 trials, achieving 90% success and a mean recovery time of 1.3 seconds, measured from torso contact with the ground until stable standing.
| Method | Test conditions | Trials | Autonomous stand-up success (%) | Recovery time (s) |
|---|---|---|---|---|
| StableBFM | G1 hardware tests under protective conditions | 10 | 90%(9/10) | 1.3 |
The continuous hardware video and repeated quantitative tests with a high success rate jointly verify StableBFM's transfer robustness and execution reliability in the physical world. We hope these additional experiments and results fully address your concerns.
Q1. Tracking preservation, Motion Decode errors, and training configurations
(a) Tracking preservation
For joint motion tracking and recovery, the MLP’s 5.53-point change in Motion Decode success remains acceptable. The model retains 76.85% success while achieving 93.23% eventual stand-up success and 73.60% motion recovery. Table 2 shows good preservation of motion quality: local MPJPE increases by only 10.47 mm (53.35 → 63.82 mm), PA-aligned error by 5.43 mm (32.82 → 38.25 mm), and root-height error decreases from 60.51 mm to 58.02 mm. On AMASS, success (82.80%) and local MPJPE (52.64 → 52.43 mm) are nearly unchanged. The pretrained controller’s coordination remains intact.
(b) Global MPJPE growth: dataset amplification
We explain the larger global-error increase on Motion Decode by its sensitivity to accumulated global errors. For the same MLP, MPJPE-G increases by 76.10 mm on AMASS (606.50 → 682.60 mm) and 634.60 mm on MD (547.30 → 1181.90 mm), while MD local and PA errors increase by only 10.47 mm and 5.43 mm. Small persistent velocity or heading deviations accumulate into trajectory-position errors, and heading-induced position errors also depend on motion displacement. Clip duration and motion distribution therefore affect how the same global tracking error appears in the metric. We tested this by injecting identical perturbations into ideal reference trajectories from Motion Decode (1503 clips) and AMASS (1500 clips reconstructed with the appendix’s stratified sampling protocol, seed 20260902).
Table D. Dataset characteristics and sensitivity to global errors.
SR denotes strict-foot diagnostic success after data-level error injection, using a 0.20 m world-coordinate threshold between corresponding feet. Table 2 uses the original no-foot evaluation. MPJPE-G increases are computed relative to the ideal trajectories before error injection.
Table D shows the amplification: MD has longer clips (23.1 s vs. 6.7 s) and a heavier tail of long-distance motions. Under identical heading drift of 1°/s, MPJPE-G increases by 293 mm on MD and 46 mm on AMASS. Thus, the same global tracking change incurs a larger metric cost on MD. Meanwhile, local/PA errors increase by 10.47 / 5.43 mm for the MLP and 20.90 / 12.38 mm for the Transformer. Local execution remains strong, while global deviations are amplified more on MD.
(c) Configuration selection
Our hyperparameters were selected through Pareto-front tradeoff analysis. In Table 3(a), K=6 trades only 0.33 points of AMASS tracking success relative to K=2 for an 8.80-point gain in motion recovery (73.60% vs. 64.80%). Compared with K=3 (69.80%), K=6 better serves the goal of reconnecting with the advancing reference.
We ablated the recovery-environment fraction ρ, behavior-cloning weight β_rec, and initial-pose pool size (Table R).
Table R. Single-parameter ablations on Motion Decode.
Two findings follow: (1) the default is already balanced; weakening recovery supervision reduces both recovery and tracking success, and increasing β_rec to 4 lowers tracking success to 75.96%. (2) Broader state coverage gives a Pareto improvement: expanding the initial-pose pool from 353 to 500 improves all metrics.