- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
Heracles addresses humanoid tracking when the reference differs substantially from the current state. It places a state-conditioned flow-matching model between the reference and a low-level RL tracker. Given the robot’s current configuration and one reference frame, the model generates a 0.2s residual trajectory with eight keyframes and replans at 25Hz. The tracker uses an iFSQ representation and is trained with PPO and adaptive motion sampling. Experiments include tracking and recovery tests in MuJoCo, transfer to SONIC, comparison with HumanUP, and demonstrations on a Unitree G1.
Critical strengths
- The middleware only changes the reference input and can be attached to existing trackers without retraining. It improves SONIC’s recovery results from 0/40 to 18/40 and runs in real time onboard.
- With the same tracker, adding the middleware raises Discrete CR from 26.3% to 92.0%. Recovery using a single standing reference improves from 0/40 to 37/40 configurations.
Critical weaknesses
- The middleware is a conditional motion in-betweening model trained offline on motion capture and separately from the tracker. It connects to the tracker only through reference inputs. On Recovery and Discrete, all baselines directly track the raw reference, without an interpolation baseline.
- iFSQ does not stand out among the baselines. Under identical observations, VQ-VAE achieves higher CR (91.9% vs 86.0%, and 67.1% vs 39.4% on Discrete), although the paper aims to improve success in difficult cases.
- The 0.2s planning horizon and eight-keyframe window are not studied experimentally. Adaptive motion sampling is used only for tracker training and is listed as a contribution, but no ablation is provided.
- The paper does not specify the source sequences of the Discrete keyframes or how long each keyframe is held before switching. It also does not explain how the eight planned frames over 0.2s are passed to the 50Hz tracking policy.
Questions for authors
No other questions.
3 comments
(2/3)
W2. Reference fidelity of the iFSQ tracker and the higher CR of VQ-VAE
Within Heracles, the middleware handles large deviations from the reference, and the tracker executes the planned reference. We therefore chose the tracker by how closely it follows a reference. CR measures this only partly, since it counts a frame as complete whenever root height stays within 0.3 m and root orientation within 1.2 rad of the reference, so a policy that keeps its balance while following the joints loosely still reaches a high CR. Joint error measures this directly.
VQ-VAE has the highest overall joint error of all trackers in Table 1, 1.883 rad. Its comparison with iFSQ-BM isolates the quantizer, since the two share the same encoder-decoder policy and observations and differ only in the latent. iFSQ-BM reduces overall joint error by 29% relative to VQ-VAE, and iFSQ with our observations by 41%. The gap is present even on smooth daily motions, where the joint error of VQ-VAE, 1.674 rad, is about 1.8 times that of iFSQ-BM and 2.1 times that of iFSQ.
We attribute both the higher CR and the higher joint error of VQ-VAE to how it quantizes the reference. VQ-VAE replaces each reference window with the nearest vector of a learned codebook, so the tracker executes a nearby prototype motion rather than the reference itself. iFSQ instead quantizes each latent channel independently on a fixed grid, so the number of representable codes grows exponentially with the latent dimension and the reference is represented at a much finer resolution. Rounding to prototypes limits precision, which explains the gap on daily motions. It also makes the tracker conservative, because abrupt or unusual parts of the reference are smoothed toward familiar motions and the robot keeps its balance more often. This effect is largest on the discretized subset, where VQ-VAE reaches 67.1% CR, while its joint error of 2.320 rad remains above the 2.208 rad of iFSQ and nearly twice the 1.181 rad of Heracles.
In Heracles, smoothing an infeasible reference into a feasible transition is the task of the middleware, and the tracker is expected to execute the planned transition as given. A tracker that rounds its input to a fixed set of prototypes would also distort the planned transition. We therefore use iFSQ, which has lower joint error than VAE, VQ-VAE, SONIC, iFSQ-BM and iFSQ+H in every subset and the lowest joint error of all standalone trackers on the discretized subset.
W4. Construction of the discretized references and the planner-tracker interface
Discretized references. All 150 discretized references come from fall-and-get-up recordings in the proprietary motion-capture collection described in Sec. 4.1, with no overlap with the daily or recovery subsets. Each reference consists of three static poses from the same recording, each a single stable frame with minimal joint motion, and consecutive poses differ in pelvis height by at least 0.30 m. Every reference lasts 550 frames at 50 fps, i.e. 11 s, and each pose stays fixed over its segment.
- Frames 0-49 hold the standing pose, 50 frames or 1 s.
- Frames 50-149 hold the fallen pose, 100 frames or 2 s.
- Frames 150-549 hold the risen pose, 400 frames or 8 s.
The reference jumps from standing at frame 49 to fallen at frame 50, and from fallen at frame 149 to risen at frame 150, with no transition frames in between.
From eight keyframes to the 50 Hz tracker. The middleware replans every 2 control steps, i.e. every 40 ms. Each plan consists of 8 keyframes spread evenly over the next 0.2 s, and the last keyframe corresponds to the reference at time t + 0.2 s. Joint positions are first clipped to the joint limits, then joint positions and root position are interpolated with cubic splines and root orientation with spherical linear interpolation. The result is a 50 Hz trajectory of 10 frames covering 0.2 s, with joint velocities from finite differences between adjacent frames. From the control step at which a new plan arrives, this trajectory replaces the tracker's reference buffer, and at every step the tracker reads its 10-frame look-ahead window from it to form the reference terms of Sec. 3.3.1.
(3/3)
W3. Ablations of the middleware configuration and of motion sampling
Planning horizon and number of keyframes. We retrained the middleware with different horizons and keyframe counts, each evaluated with the same iFSQ tracker.
On the continuous recovery subset, every horizon between 0.1 and 0.4 s reaches 95.0% to 98.0% CR, and only 0.8 s falls to 64.7%. The discretized subset, with large gaps after each switch, separates the variants clearly. The 0.2 s horizon with K = 8 reaches 92.0%, and every other setting stays at or below 81.1%.
A 0.1 s horizon reaches only 42.9%. The tracker reads a 10-frame look-ahead window at 50 Hz, i.e. 0.2 s, which a 0.2 s plan fills while a 0.1 s plan covers only its first half. Longer horizons lower CR to 61.2% at 0.4 s and 15.5% at 0.8 s, since a longer residual is larger and more varied, which is harder for the same model to fit.
At 0.2 s, recovery CR stays between 97.3% and 98.0% for all keyframe counts, but the discretized subset favors K = 8. K = 6 reaches the highest recovery CR but only 81.1% on discretized references, 10.9 points below K = 8, and K = 4 and K = 10 reach 76.0% and 78.7%. Fewer keyframes describe the transition more coarsely, and with K = 10 the keyframes match the 50 Hz frames in the horizon without improving the plan. Since the discretized subset requires the largest transitions, we choose K = 8.
Adaptive motion sampling. Our sampler, described in App. A, extends the bin-level curriculum of ZEST with continuous bins, kernel-smoothed difficulty propagation and an outlier cap. We retrained the same iFSQ tracker with two alternative samplers, keeping training data, rewards, observations, network and PPO settings unchanged. Uniform sets in Eq. 19, giving every bin equal probability. Raw failure-proportional samples each continuous bin in proportion to its failure count plus a uniform floor, i.e. our sampler without kernel smoothing and outlier cap. All trackers are evaluated on the 1,000-motion benchmark of Table 1.
Uniform sampling matches ours on daily motions at 96.3% CR, but its recovery CR drops from 92.2% to 55.1%, with nearly four times the height error, 0.257 m against 0.068 m, and about 2.5 times the orientation error, so the robot frequently fails to get up. Short, difficult segments such as the contact transitions of a get-up are sampled too rarely when all bins weigh the same.
Raw failure-proportional sampling raises recovery CR from 55.1% to 67.5%, confirming that difficulty-based sampling helps. Without smoothing and the outlier cap, however, the distribution collapses, with a single 1 s bin drawing about 75% of all samples. Daily CR then drops to 94.5%, and joint error rises in every subset, to 1.828 rad overall against 1.109 rad.
Our sampler reaches the highest overall CR, is best on all five metrics overall and on the recovery subset, and has the lowest joint error in every subset. On discretized references all three samplers stay between 21% and 27% CR, and in Heracles the middleware raises CR on this subset to 92.0%.
We appreciate the reviewer's careful examination of the experimental design. We hope the additional experiments and details above address the concerns, and we are glad to provide further information in the discussion.
(1/3)
We thank the reviewer for the detailed reading of the method and the experiments. Each weakness raised in the review is addressed below, together with the additional experiments we ran for it. Our response is split into three comments.
W1. Interpolation baseline on the recovery and discretized subsets
The middleware and the tracker are trained separately and communicate only through the reference. This separation is what allows the same middleware to drive both our tracker and SONIC without retraining. To measure how much the learned model contributes beyond supplying intermediate poses, we built an interpolation planner that takes the place of the flow matching network and leaves every other component of Heracles in place.
At each replanning cycle, the interpolation planner places the eight keyframes on a straight path from the current state to the target. Joint angles and root position change linearly along this path, and root orientation changes by spherical linear interpolation. The first keyframe equals the current state, and the root horizontal position is expressed relative to the current root, both as in Heracles. The state representation, the 0.2 s horizon with K = 8, replanning at 25 Hz, the target given to the planner, the conversion of keyframes into the 50 Hz reference, the tracker checkpoint, the evaluation code and all metric definitions are identical to Heracles.
Table R1. Recovery and discretized subsets of Table 1, same iFSQ tracker, CR in percentSubsetRecoveryDiscretePlannerNoneInterpolationFlow matching, HeraclesNoneInterpolationFlow matching, HeraclesCR↑92.291.197.826.315.192.0Joint (rad)↓1.4411.2681.1002.2082.7781.181Height (m)↓0.0680.0680.0250.4510.5180.055Ori (rad)↓0.3230.2810.1431.3441.4530.159LinVel (m/s)↓0.1750.1510.1730.2580.2010.314
Table R2. Autonomous recovery from the 40 fallen configurations of Table 10, same iFSQ trackerPlannerNoneInterpolationFlow matching, HeraclesSR↑0/400/4037/40
On the recovery subset the reference is continuous, and the robot mostly stays close to it. There, interpolation performs similarly to the tracker alone, with 91.1% CR against 92.2% and somewhat lower joint and orientation errors. Heracles reaches 97.8% CR with the lowest joint, height and orientation errors. The difference widens when the gap between the robot and the reference is large. On the discretized subset, interpolation lowers CR from 26.3% to 15.1%, below the tracker without a planner, and it recovers none of the 40 fallen configurations, where Heracles recovers 37. Two features of straight-line planning account for this.
The updated supplementary material contains videos of the interpolation planner on 10 fallen configurations randomly chosen from Table 10, and this comparison is reported in App. I of the revised paper.