- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Diffusion-based humanoid control policies struggle to use future-state prediction to guide long-horizon motion objectives unless they rely on a separate low-level motion tracker and an external privileged state estimator. This paper proposes a predictive joint state–action diffusion policy that relies solely on onboard proprioceptive observations. Full-body physical states serve as supervision signals only during training. At inference time, they are reconstructed from noise purely as internal latent variables that serve goal guidance. This eliminates the need for a separate motion-reference tracker and an external privileged state estimator, enabling 50 Hz high-frequency control deployed on onboard edge computing hardware.
Critical strengths
- The proposed method has been deployed fully onboard the robot. A single policy accepts multiple command modalities, such as text instructions and joystick velocity commands, and runs closed-loop inference in real time.
- The work is highly reproducible. The paper gives a thorough account of the technical principles, implementation details and relevant hyperparameters, and the released inference source code should make it easy for the community to reproduce the results.
- The manuscript is well written and reads smoothly, with abundant figures and a high level of completeness.
Critical weaknesses
-
The main baseline, Diffuse-CLoC, predicts future states from the full-body states observed over the past few frames. The paper's claimed novelty, "not relying on past full-body state observations," amounts to predicting the past and future states together. The paper neither ablates this predictive approximation nor offers any physical guarantee for it. Moreover, on a real robot most full-body state components can be measured or computed, such as the relative positions of body parts and the pelvis angular velocity. Only a few global quantities, such as pelvis linear velocity, global position/height, and yaw, are hard to obtain. Yet PredActor uses base linear velocity, which is exactly this kind of "privileged information" (it is part of PredActor's 96-dimensional observation, as shown in Table 6). I therefore believe the core assumption of the method does not hold.
-
According to the paper, action-only diffusion produces no future-state trajectory, so it cannot apply classifier guidance and therefore cannot impose analytic objectives defined on future body states (e.g., a specified root velocity, a body height, or point-goal navigation). In practice, however, tasks such as joystick velocity control, body height control, and point-goal navigation can be handled simply by feeding the commands (e.g., velocity commands) to the network as an additional input. This is standard practice in imitation learning: the condition labels are simply fed to the diffusion network together with the observations.
-
The real-time performance comes mainly from common engineering optimizations and does not justify a separate claim of novelty.
Questions for authors
- The authors should clarify where the velocity sensor readings come from on the real robot.
- What results would be obtained if Diffuse-CLoC used the full-body state computed from the same set of deployable sensors as PredActor?
5 comments
Response to Reviewer uGJ5 (2/2)
W1-3 / Q1 · Base linear velocity
Q: The reviewer argued that PredActor relies on hard-to-obtain base velocity, so the novelty claim and core assumption fail, and asked where velocity readings come from.
A: Both versions are deployable. PredActor-93D and -96D both ran on the G1. The 96-D version uses the vendor's onboard velocity estimate with velocity domain randomisation; all the paper's hardware results use PredActor-93D, which takes no base velocity (joints, IMU, previous action and text only). Without base velocity PredActor drops only slightly (push 0.51 → 0.47, point goals 45/45 → 42/45, text survival 0.89 → 0.80), so the method does not depend on it (Appendix Table A3 and Table A4).
W2 · Command conditioning vs. guidance
Q: The reviewer doubted that test-time (classifier) guidance is needed when the goal can be a command input.
A: Complementary, not competing. PredActor supports both conditioning (a command input or CFG) and guidance (CG), so this is a choice, not a limitation. Conditioning needs labelled data and training per command; CG needs neither but may be less precise. Table R2 compares them with the same 96-D inputs.
| Task: accuracy ↑ · falls ↓ | PredActor + our improved CG | PredActor + command input | PredActor w/o state + command | DiffuseLoco + command |
|---|---|---|---|---|
| Static hand targets: reached within 8 cm · falls (of 486 runs) | 198 · 8 | 21 · 15 | 9 · 57 | 59 · 36 |
| Moving hand targets: mean tracking error in m · falls (of 27 runs) | 0.16 · 0 | 0.12 · 3 | 0.14 · 0 | 0.13 · 4 |
| Forward speed 0.5 / 1.0 / 1.5 m/s: segments held (of 45) · runs with a fall (of 15) | 15 · 0 | 45 · 0 | 44 · 0 | 36 · 5 |
| Lateral ±0.3 m/s and yaw ±0.6 rad/s steps: segments held (of 60) · runs with a fall (of 15) | 8 · 0 | 4 · 0 | 4 · 0 | 3 · 6 |
Table R2. Our improved CG vs. three command-input models, all reading the same 96-D...
Coverage decides. Without retraining, our improved CG is by far the most accurate on static hand targets (198/486 vs. ≤ 59/486 for every command-input model; bars in Figure R4). Fall differences between command-input models come from individual weak training seeds and are only descriptive. We have no matched command-input policy for body height or point goals, so we make no comparison there.
Figure R4. Table R2 as bars, one colour per model
Videos. Video V1, Video V2, Video V3 and Video V4 show the four models reaching 18 static two-hand targets (spheres; one training seed); full results in Appendix Table A5.
Video V1. PredActor + our improved CG reaching 18 static targets with both hands
Video V2. PredActor + command input reaching 18 static targets with both hands
Video V3. PredActor w/o state + command reaching 18 static targets with both hands
Video V4. DiffuseLoco + command reaching 18 static targets with both hands
W3 · Real-time performance
Q: The reviewer noted that the real-time performance comes from common engineering optimisations.
A: A system contribution. We agree the individual techniques are known and present them as a system and infrastructure contribution. Prior diffusion controllers met the latency barrier by giving something up: an add-on or off-board GPU, a lower rate (25 Hz for BeyondMimic), a tracker underneath, or no gradient guidance. PredActor is, to our knowledge, the first joint state–action diffusion policy run entirely on the G1's Jetson Orin NX at 50 Hz, with guidance in the loop, direct joint targets and no tracker. In the paper's onboard timing study, graph, structural and runtime optimisations cut the guided actor from 21.85 to 13.18 ms; the full callback takes 16.79 ms at p50 and 19.38 ms at p95 (7/600 cycles over budget). We release the C++ deployment library. → Appendix, W3
Paper changes
We have uploaded the revised paper; the changes are summarised in our general comment Paper Revision 1.
We hope these new experiments resolve the reviewer's concerns.
Thank you very much for your candor and detailed rebuttal; however, I would like to respectfully maintain my current score. The baseline comparison still seems slightly asymmetric. Also, it remains somewhat unclear where the performance ceiling of predicting states lies relative to observing ground-truth states directly.
Response to Reviewer uGJ5 (1/2)
Response to Reviewer uGJ5
We thank the reviewer for the constructive review and for noting the fully onboard deployment of a single multi-command policy, its reproducibility and the clarity of the manuscript. All new results use 3 training seeds per model. The complete reply, with all figures, tables and videos, is at anonymous link.
W1-1 · Ablating joint state prediction; physical consistency
Q: The reviewer asked for an ablation of joint state prediction and questioned whether predicted states are consistent with executed actions.
A: Why predict states as well as actions? PredActor combines behaviour planning (S1) and execution (S0) in one policy: it jointly generates future states and actions, executes the first action and re-plans from new observations at every control step. The predicted states make the plan explicit and give test-time guidance (point goals, hand targets) variables to act on. We therefore test separately whether joint state prediction improves the policy and whether predicted states agree with execution.
Six-model ablation. We agree that PredActor predicts past and future states; our claim concerns the inputs: no full-body state input, and no base-velocity estimate in the deployed 93-D version. We compare PredActor, PredActor w/o state (same inputs, loss and size, no state stream) and Diffuse-CLoC + FK (observes FK-computed full-body states), each with the deployed 93-D input (no base velocity) and a 96-D input (with it), 3 seeds each, same data and protocol (Table R1).
Table R1. Six-model ablation, mean over 3 training seeds
Joint prediction helps. It raises text-trial survival under both inputs and push survival under 93-D, and only it allows point goals.
Consistency is learned, and we measure it. Training windows pair each observation history with the states a tracking teacher executed on the simulated G1 under noise and pushes, and iterative data aggregation adds teacher continuations from states the policy visits. With no theoretical guarantee, we now measure it. Without guidance, predicted and executed states agree to 1.9 cm (links) and 0.14 m/s (root velocity) 100 ms ahead, vs. 7.6 cm for a constant-state baseline. With vanilla CG, the original study gives a 0.42 m/s gap and 9/120 falls (Figure R1); in a separate paired comparison, keeping the guided target in the training range cuts the gap from 0.41 to 0.24 m/s and falls from 11/120 to 1/120 (Appendix Table A2, Figure A2). The gap and falls grow with guidance gain, and more denoising steps shrink the gap (Figure R2). Here and below, our improved CG is the paper's classifier guidance with rebuttal fixes: feasibility-limited guided targets (consistency), and hand-correction temporal smoothing plus a corrected few-step velocity sampler (command vs. guidance).
Figure R1. Predicted vs. executed state without guidance, with vanilla CG and with our...
Figure R2. Predicted-vs-executed gap and falls by guidance gain and number of denoising steps
W1-2 / Q2 · FK-computed instead of predicted states
Q: The reviewer asked whether predicting full-body states is necessary when most can be computed onboard, and how Diffuse-CLoC does with the same sensors.
A: Computing states by FK is worse. Diffuse-CLoC + FK tests this directly (under 93-D, retrained on the onboard-computable part only) and is worse than PredActor under both inputs (Table R1, Figure R3), so predicting the states is necessary.
A likely reason. We hypothesise that FK turns the measured pelvis angular velocity into every link velocity through the term ω × r, so gyro jitter is amplified by the lever arm and accumulates along the chain. The input would thus be least reliable at the feet and hands, exactly where the robot makes and breaks contact.
Figure R3. Six-model ablation per metric and training seed