acceptodds
Under review as a conference paper at ICLR 2027

Kinematic Transition Model for Training-free Online Human Motion Control

Abstract

Interactive human motion synthesis aims to continually generate plausible motion while responding to changing user inputs and environmental constraints. Recent autoregressive motion models enable efficient online generation, but their real-time control typically relies on predefined conditioning signals, specialized control interfaces, or objective-specific training. In practical scenarios, however, control objectives may be newly introduced, changed, or combined during execution, requiring the motion generator to adapt without retraining while maintaining real-time performance. To address this challenge, we propose Kinematic Transition Model (KTM) for training-free online human motion control. Starting from the observed motion, KTM constructs an analytical kinematic continuation from its boundary dynamics and represents possible futures as acceleration corrections to this continuation. We then model future generation through a small number of latent stochastic transitions in this boundary-attached kinematic space, capturing the one-to-many evolution toward plausible future motions. Importantly, the same transition representation provides an efficient interface for training-free control. At each transition step, the predicted transition is kept fixed as a local motion prior while a future endpoint is optimized under the current inference-time objective. The optimized endpoint analytically defines the next controlled transition, allowing new or changing differentiable objectives to steer motion generation without control-specific training, additional motion-model computations, or backpropagation through the learned generator during optimization. Experiments demonstrate that KTM enables high-quality online motion synthesis with flexible control over unseen objectives while maintaining real-time inference.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Motion Synthesis, Training-Free Online Motion Control

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yNJxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Kinematic Transition Control (KTM) for training-free online human motion control with unseen objectives. KTM predicts residual accelerations from the exponentially damped extrapolated past accelerations. The residuals are estimated with K latent conditioned Gaussian transitions from a zero vector. During control, residuals are optimized by 8-time L-BFGS with regularization. KTM enable online human motion control by bypassing the feedback propagation from the network. KTM also incorporates unseen objectives as far as it is differentiable and within 8-frame horizon. The experimental results show the superiority of KTM under two conditions: 3D end-effector control and waypoints.

Critical strengths
  • The control procedure is simple and computationally attractive. The learned networks are evaluated a fixed times per cycle, independent of the optimization. All objective-dependent computation is confined to a lightweight analytical map.
  • On the authors' benchmark, KTM attains a high success rate while staying close to the uncontrolled model in ARD.
  • The target setting is practically relevant: continual online human motion control and unseen objectives.
Critical weaknesses

Baseline fidelity cannot be verified, and no established protocol is used.

  • PRIMAL-G is constructed by authors, not a part of the original PRIMAL. Its collision rate is essentially identical to uncontrolled reference. It seems that the avoidance term is not effective.
  • DART-RL, retrained for this task, has a collision rate (35.04%) worse than the uncontrolled reference.
  • The experimental setups like guidance strengths, step schedules, gradient paths, RL rewards, observations, and training budgets are not reported.
  • Established protocols with public code exist. DART's waypoint goal reaching and path following, and WANDR's original 3D goal reaching (reported there as SR 32%, DTG 24.8 cm).

Missing comparison with existing training-free online methods

  • A training-free method: DART's latent noise optimization is not evaluated. It is feasible that optimize DART-Noise online since it runs at 300 fps as the DART paper reported.

The "unseen objectives" claim is overstated.

  • Non-differentiable objectives. The method requires a differentiable objective, which is narrower than recent training-free methods that handle criterion-based or black-box constraints[1].
  • Conditioning on distant frames. Only an 8-frame window is optimized per cycle. Constraints beyond the window cannot be expressed.
  • Perturbations. Robustness to external perturbations is not evaluated, although PRIMAL, a purely kinematic model used here as a baseline, demonstrates it.

Physics-based methods are excluded without adequate justification.

  • Diffuse-CLoC[2] and BeyondMimic[3] steer a single pre-trained model toward unseen objectives through inference-time guidance in real time. The paper excludes them because they are physics-based. Being physics-based may justify not running their full systems on the authors' benchmark. It does not justify omitting them from the novelty discussion or from Table 1. The "underexplored setting" claim should be explicitly restricted to kinematic methods.

[1] Hui, Xiaofei, et al. "Training-Free Controllable Human Motion Generation Under Heterogeneous Constraints." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026. [2] Huang, Xiaoyu, et al. "Diffuse-cloc: Guided diffusion for physics-based character look-ahead control." ACM Transactions on Graphics (TOG) 44.4 (2025): 1-12. [3] Liao, Qiayuan, et al. "Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion." Science Robotics 11.117 (2026): eadx8924.

Questions for authors

I am ready to raise my rating based on author-reviewer discussions on these questions.

  • Why was DART's latent noise optimization not evaluated, credited in Table 1 with supporting unseen objectives, and publicly available? Please report them with the iteration count capped to the real-time budget, and plot success rate and collision rate against latency.
  • What motivated constructing PRIMAL-G rather than using existing training-free online controllers? Please provide full configurations for PRIMAL-G and DART-RL. Please also explain PRIMAL-G's ineffective collision avoidance and the speed gap between DART-RL and DART's reported throughput.
  • Please report results on at least one established protocol (DART waypoint goal reaching or WANDR's original 3D reaching), with baseline numbers checked against the published values.
  • How does KTM handle (a) non-differentiable objectives, (b) keyframes beyond the 8-frame window, (c) external perturbations to the history?
  • What is the precise criterion for excluding Diffuse-CLoC and BeyondMimic, given that they provide inference-time guidance toward unseen objectives in real time? If they remain excluded, will the setting claim be restricted to kinematic methods?
10 comments
AuthorsOpenReview ↗

Weakness 1 & Question 2

We sincerely thank Reviewer yNJx for their detailed and constructive feedback, and for recognizing the practical importance of our problem setting. We greatly appreciate your positive assessment of our computationally attractive control procedure and strong navigation performance while preserving motion plausibility. We would like to address the reviewer's questions and provide additional clarifications and experimental evidence below.


W1 & Q2. Baseline fidelity details

We thank the reviewer for noting these important details. Both PRIMAL-G and DART-RL are directly based on their official implementations, with only the adaptations necessary for our navigation task.

Baseline implementation fidelity.

For PRIMAL-G, we directly use PRIMAL's official code and released model with their default configurations, while only plugging our navigation objective into its classifier-based guidance (CBG) interface (the suffix “-G” stands for guidance). For DART-RL, we only adapt its original waypoint-reaching RL policy with an additional obstacle avoidance reward, while fully retaining the original model implementation. The obstacle avoidance reward is calculated as

robstacle=−∑fmax⁡(0,,0.1−dmin⁡(f))2,r_{\mathrm{obstacle}} = -\sum_{f}^{} \max\left(0,\\,0.1-d_{\min}(f)\right)^2,

where dmin⁡(f)d_{\min}(f) denotes the minimum signed distance between the character and nearby obstacles at frame ff with a safety margin of 0.10.1. This encourages obstacle avoidance while preserving DART's original waypoint-reaching and motion-quality rewards.

Interpretation of collision rate.

Regarding collision avoidance, we clarify that collision rate should be interpreted jointly with waypoint success rate, because our navigation benchmark jointly evaluates waypoint reaching and obstacle avoidance. In our benchmark, obstacles are deliberately placed along the required waypoint routes to evaluate both objectives simultaneously. Methods that fail to follow these routes may encounter fewer obstacles, resulting in deceptively low collision rates. For example, the uncontrolled reference achieves only 7.63% waypoint success, so its collision rate does not indicate effective avoidance. Similarly, PRIMAL-G's collision rate alone does not establish that its avoidance guidance is ineffective, particularly given its substantially higher waypoint success (73.01%).

To further account for differences in obstacle exposure, we additionally report the proportion of obstacles approached during execution (App.) and the collision rate among approached obstacles (Coll./App.).

Table R2. Obstacle avoidance under online waypoint navigation. App. denotes the percentage of obstacles approached during execution; Coll./App. denotes the collision rate among approached obstacles.

MethodSR↑Coll.↓App.Coll./App.↓
Uncontrolled7.63%28.02%36.50%77.78%
PRIMAL-G (10)42.28%28.23%60.48%46.02%
PRIMAL-G (50)73.01%25.18%77.76%33.72%
DART-RL61.72%35.04%81.63%43.12%
KTM (ours)89.19%18.06%87.83%20.52%

The results show that PRIMAL-G (50 steps) achieves a substantially lower collision rate among approached obstacles than the uncontrolled reference (33.72% vs. 77.78%), despite their similar original collision rates. This clarifies why PRIMAL-G's original collision rate alone does not establish ineffective avoidance guidance. Furthermore, KTM achieves the highest waypoint success (89.19%) and the lowest collision rate among approached obstacles (20.52%), which further supports the effectiveness of our framework.

Runtime differences.

Regarding DART-RL's speed, the reported throughput is not directly comparable due to differences in evaluation settings. Our benchmark requires dynamic-obstacle scene processing and online control updates in addition to motion generation. Additionally, DART's official implementation enables mixed-precision by default, whereas our evaluation consistently uses full-precision floating point for all models. Our compared speed includes the complete online control cycle under the dynamic-obstacle navigation benchmark, rather than only motion-model generation. However, DART-RL still satisfies the real-time requirement under our benchmark.

AuthorsOpenReview ↗

Weakness 1, 2 & Question 1, 3

W1 & Q3. Evaluation on established protocols

Thank you for suggesting the established benchmarks, which would strengthen our evaluation. To further validate our method, we additionally evaluate KTM on DART's original waypoint-reaching protocol (DART's Table 4) and WANDR's original 3D wrist-reaching protocol (WANDR's Table 1), using their published evaluation settings and metrics.

DART's waypoint-reaching protocol.

As shown in Table R5, KTM achieves 100% success rate, matching DART-RL while achieving lower foot skating (1.98 vs. 2.67 cm/s) and floor distance (2.11 vs. 2.24 cm), despite requiring no waypoint-specific training.

Table R5. Evaluation on DART's original obstacle-free waypoint-reaching protocol.

MethodSR ↑Skate (cm/s) ↓Floor dist. (cm) ↓
GAMMA95%5.145.55
DART-RL100%2.672.24
KTM (ours)100%1.982.11

Importantly, DART already achieves 100% success on this protocol, suggesting a ceiling in waypoint-reaching accuracy. This motivates our more challenging benchmark combining sequential waypoint reaching with dynamic obstacle avoidance, which evaluates the ability to satisfy multiple control objectives simultaneously.

WANDR's 3D reaching protocol.

As shown in Table R3, KTM achieves 82% success, 12.35 cm distance-to-goal, and 9.69% foot skating, substantially outperforming WANDR's published results despite requiring no reaching-specific training.

Table R3. Evaluation on WANDR's original single-target 3D reaching protocol.

MethodSR ↑DTG (cm) ↓Foot skating (FS) ↓
GOAL0%149.2029.00%
WANDR32%24.8016.00%
KTM (ours)82%12.359.69%

Unlike the original single-target evaluation, our main reaching benchmark requires continually adapting to sequentially changing targets under limited time, making accurate control while preserving plausible motion more challenging.

Together, these results validate KTM on established protocols and motivate our more challenging online benchmarks, which evaluate our central contribution of flexible training-free control under changing and combined objectives.


W2 & Q1. Comparison with DART's training-free noise optimization

We thank the reviewer for suggesting this important comparison. We clarify that DART's latent-noise optimization is already credited in Table 1 as supporting unseen objectives. However, its reported throughput of over 300 FPS refers to motion generation, rather than noise optimization. In fact, DART reports approximately 74 seconds for generating 60 frames with 100 optimization steps (Appendix F). So we initially chose DART-RL as the real-time control variant. We are evaluating DART's noise optimization and will provide the results in the coming days as soon as they are ready due to our limited hardware availability.

AuthorsOpenReview ↗

Weakness 3, 4 & Question 4, 5

W3 & Q4. Scope of unseen objective and online control

We thank the reviewer for those insightful questions and clarify the following points:

Non-differentiable objectives. Our problem setting (Sec. 2) explicitly focuses on unseen differentiable objectives, where "unseen" means objectives introduced at inference without control-specific training. We acknowledge that MIC (Hui et al., 2026) supports broader constraints including non-differentiable criteria, but they mainly focus on offline motion generation rather than continual real-time online control. Additionally, our endpoint optimization (Eq. 7) is not inherently restricted to differentiable objectives. In principle, derivative-free optimization could potentially replace L-BFGS without modifying the learned model. However, our stated setting focuses on differentiable objectives to maintain efficient gradient-based control optimization under real-time constraints. Thank you for the thoughtful suggestion, we believe extending KTM to black-box objectives while preserving real-time efficiency would be an interesting future direction.

Short-horizon control. The 8-frame prediction horizon is a deliberate design choice for efficient and responsive online control following DART, rather than a restriction on the spatial range of objectives. For example, our reaching task includes distant targets beyond the prediction horizon, which can be gradually approached through successive control cycles. At each cycle, the current objective is evaluated over the predicted motion window, and the generated motion becomes the history for the next cycle. This enables continual progress toward distant goals while adapting to changing objectives. However, exact constraints specified at future times beyond the current window cannot be directly enforced, as acknowledged in our limitations.

External perturbations. Our KTM reconstructs the kinematic continuation from the updated motion history at each control cycle, allowing subsequent predictions to adapt to changes in the character's observed motion state. This mechanism naturally supports conditioning on the changed history, although it does not by itself guarantee recovery from arbitrary external perturbations. We note that our central contribution is training-free objective control during continual online generation, rather than physical perturbation recovery. Nevertheless, we appreciate that perturbation robustness is relevant to interactive motion synthesis, and we are conducting an additional experiment that applies perturbations during online navigation. We will hopefully provide the results in the next coming days.


W4 & Q5. Distinctions from physics-based control methods

We thank the reviewer for highlighting relevant works on physics-based control. We acknowledge that Diffuse-CLoC and BeyondMimic support inference-time guidance toward unseen objectives, as discussed in Appendix A. However, our work addresses a different setting of directly learning and controlling kinematic motion from motion-capture (mocap) data, rather than physics-based policy training or simulation. For example, Diffuse-CLoC uses a physics-based tracking method PHC+ to track and curate a subset of 54 AMASS motions in simulation with state-action data, whereas KTM learns motion transitions directly from AMASS sequences. This allows kinematic generation to leverage existing high-quality mocap datasets, without requiring extra physics simulation, motion-tracking controller, and data preparation requirements for physics-based policy learning. This distinction motivates our study of a practical reusable motion control prior that can be trained directly from existing large-scale mocap data such as AMASS.

Accordingly, Table 1 explicitly focuses on kinematic motion control methods, which directly generate motion sequences without relying on physics-based controllers during execution. While the physics-based approaches address related control objectives, they operate under different training and execution requirements and are not directly comparable within our kinematic generation benchmark. We will make this scope explicit in the setting discussion to avoid ambiguity. We also recognize the complementary benefits of physics-based methods, and incorporating physics-aware priors into our method is a promising extension, as discussed in our limitations.

Reviewer_6S5fOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses training-free online control for interactive human motion synthesis. In each short control cycle, it predicts future motion as acceleration residuals on top of a kinematic extrapolation of the past motion. These residuals are generated with a few-step diffusion bridge whose steps are CVAE-based. Control is applied by optimizing the predicted future at each step against a differentiable objective, with a term that keeps it close to the network prediction, so no backpropagation through the network is needed.

Contributions: (i) addressing training-free online motion control under real-time constraints, where unseen objectives are introduced only at inference; (ii) formulating online future generation as finite stochastic transitions in a kinematic acceleration-correction space; (iii) training-free transition control in this space, which uses each predicted transition as a fixed local motion prior and optimizes its future endpoint.

Validation: The model is trained on AMASS/BABEL and evaluated on two tasks: Sequential waypoint navigation with dynamic obstacles, against PRIMAL-G and DART-RL. Sequential 3D wrist reaching, against GOAL and WANDR.

Critical strengths

Contributions: (1) a training-free, real-time online control setting where objectives are specified only at inference; (2) a few-step stochastic transition model in an acceleration-residual space; (3) a per-step endpoint optimization that steers generation without task-specific training.

Critical weaknesses
  1. Prior art from the game industry is missing. Real-time, online control toward changing goals is a long-established direction in industry, through motion matching and later learned motion matching (Holden et al., ToG 2020). Neither is discussed as a competing approach or compared against. Motion matching or LMM combined with a path planner, and an uncontrolled generator combined with IK, are natural baselines for the two tasks studied.
  2. The claim of "any differentiable goal" is not supported. Only navigation and wrist reaching are evaluated, and both can be handled by traditional pipelines.
  3. The exposition is hard to follow. The method is described with many non-standard terms ("analytical kinematic continuation", "boundary dynamics", "acceleration-correction state", "kinematic operator") instead of standard ones (kinematic extrapolation, current state, residual acceleration, integration).
  4. There is a trade-off between quality and controllability.
Questions for authors

What is the purpose of the theoretical derivations in Appendix B? Could the authors clarify which claim of the paper they are intended to establish, and what new insight they provide beyond existing results?

5 comments
AuthorsOpenReview ↗

Weakness 1

We sincerely thank Reviewer 6S5f for their feedback on the contributions of our work, including training-free online control, finite stochastic motion transitions in acceleration-correction space, and efficient endpoint optimization without task-specific training. We appreciate the opportunity to further clarify our problem setting, contributions, and distinctions from prior motion control methods in the responses below.


W1. Comparisons with traditional motion control methods

We thank the reviewer for highlighting these relevant prior works. We respectfully clarify that our work addresses a different problem from traditional methods such as motion matching. Our method learns a single reusable motion generator that can support new or combined differentiable objectives at inference, without control-specific training, while maintaining real-time online execution. As discussed in Appendix A, traditional methods achieve real-time interactive animation by retrieving or predicting motion according to predefined query features, such as desired trajectories and motion phase. While effective for their designed tasks, incorporating new control objectives generally requires carefully redesigning the query features or modifying task-specific control mechanisms. In contrast, KTM directly expresses control objective as a general function G(⋅)G(\cdot) of the generated motion and optimizes them through the same inference-time control procedure (Eq. 7). Our model is trained only on motion data without supervision from predefined task-specific control inputs. At inference, this enables different objectives such as waypoint navigation, dynamic obstacle avoidance, and wrist reaching, to control the same pretrained motion model through a unified control interface.

Regarding the suggested alternatives, Motion Matching combined with path planning can provide effective navigation, while an uncontrolled motion generator combined with inverse kinematics (IK) can enforce end-effector constraints. However, these task-specific pipelines do not, by themselves, provide a unified mechanism for introducing and combining new objectives while preserving plausible motion during online generation. Achieving this generally requires additional task-specific planning, control, or post-processing mechanisms. Our KTM instead integrates objective optimization directly into the learned stochastic transition process, using the predicted transition as a local motion prior to regularize control plausibility.

To further evaluate this distinction, we additionally compare our method with motion-matching methods combined with path planning under our waypoint navigation and obstacle avoidance benchmark.

Table R4. Comparison with traditional motion-matching methods on sequential waypoint navigation with dynamic obstacles.

MethodSR ↑Coll. ↓ARD ↓Skate ↓
Motion Matching82.56 %33.02%0.3920.114
Learned Motion Matching84.89 %31.11%0.4680.142
KTM (ours)89.19%18.06%0.3840.129

While traditional methods achieve competitive waypoint success, they exhibit notably higher collision rates. This demonstrates KTM's effectiveness in jointly handling waypoint reaching and dynamic obstacle avoidance objectives while maintaining competitive motion quality. In contrast to traditional methods that rely on predesigned matching features, KTM can incorporate both objectives through the same inference-time control interface without modifying or retraining the motion model, and achieve better overall control performance. Our contribution is not the individual capability of navigation or reaching, but a practical step toward a versatile motion generation framework that can adapt to new control requirements at runtime, rather than requiring a dedicated controller for each task.

AuthorsOpenReview ↗

Question 1 & Weakness 2

Clarification of our problem setting and contribution, and the role of Appendix B (Question 1)

(i) Our problem setting and main contribution. We thank the reviewer for this opportunity to clarify our key motivation and contribution. Our work addresses the joint challenge of real-time online and training-free objective control, which we believe is an important yet underexplored problem. Our main contribution is a unified kinematic transition formulation that enables both efficient stochastic motion generation and training-free online control under newly specified differentiable objectives. The key challenge is to achieve this flexibility within a real-time budget, since repeatedly sampling from or backpropagating through the motion generator during control can be computationally expensive.

To address this challenge, KTM directly learns a small number of finite stochastic transitions in the acceleration-correction (residual) space, starting from a kinematic extrapolation of the observed motion. This design enables efficiency in two complementary ways: (i) directly learning finite transitions allows plausible motion generation in a few steps (e.g., five), and (ii) keeping each predicted transition fixed during endpoint optimization enables flexible control without repeated network evaluations or backpropagation through the generator. The resulting controlled state then guides the next transition, allowing generation and control to proceed progressively within a real-time budget.

(ii) Appendix B establishes how this finite transition process is constructed and learned, which directly supports both aspects of KTM. Unlike conventional diffusion models, we do not learn a continuous-time reverse process. Instead, we use a diffusion bridge to derive analytical reference transition kernels connecting the kinematic extrapolation to the ground-truth future over a small number of finite steps. Since the ground-truth future is unknown at inference, we employ latent variational learning to approximate the corresponding transition distribution over plausible futures. Appendix B derives the necessary transition kernels and explains how our training objective relates to the desired transition distribution. Importantly, the same analytical transition formulation also facilitates our control mechanism, where an optimized future endpoint directly determines the next controlled transition (Eq. 4). Thus, Appendix B establishes the mathematical foundation needed for constructing and learning the finite transition process used by KTM, rather than introducing an independent theoretical contribution.

In summary, KTM provides a principled motion-specific formulation that integrates few-step generation and training-free control through the same kinematic transition process. The practical effectiveness is supported by Table 2, where KTM achieves higher waypoint success (89.19% vs. 73.01%) while running over four times faster (52.18 vs. 12.26 FPS) than PRIMAL with 50 diffusion steps.


W2. Differentiable objective control

Thank you for the opportunity to clarify the scope of our objective control. Our contribution is not solving navigation or wrist reaching individually, but enabling new or combined differentiable objectives to control the same pretrained motion generator through the same procedure. We acknowledge that traditional pipelines can effectively handle these individual tasks, but they typically rely on task-specific mechanisms. Supporting additional objectives would require redesigning or adapting their control procedure. In contrast, our control formulation (Eq. 7) allows differentiable objective G(⋅)G(\cdot) to be incorporated directly through endpoint optimization at inference, rather than relying on predefined task-specific conditioning or control modules.

Our experiments demonstrate this flexibility across different and combined control objectives. In Table 2 (main paper), waypoint reaching, heading, and dynamic obstacle avoidance are jointly combined within a single objective. Table 3 (main paper) further demonstrates that the same pretrained model and control procedure can support a structurally different articulated objective (wrist reaching), without task-specific training. These results support the practical benefit of our generic control interface across representative online tasks, rather than merely demonstrating the ability to solve each task individually. We emphasize that our claim refers to the flexibility of introducing differentiable objectives at inference, rather than guaranteeing satisfaction of every possible objective.

AuthorsOpenReview ↗

Weakness 3, 4

W3. Exposition and terminology

Thank you for the feedback. Our terminology is intended to distinguish specific roles within KTM's finite transition formulation, rather than introduce new kinematic concepts. Specifically, kinematic continuation refers to extrapolation continuing from the observed pose, velocity, and acceleration; boundary dynamics denotes these quantities at the observed motion boundary; acceleration-correction state refers to the corrective acceleration residuals over the entire future window at each generative transition step; and the kinematic operator denotes the mapping function that reconstructs an entire future motion from the observed boundary and future accelerations. These terms distinguish the roles of these concepts in our finite transition formulation, particularly between physical motion time and generative transition steps. We will incorporate more intuitive explanations and simplify the terminology where appropriate.


W4. Trade-off between motion quality and controllability

We acknowledge that balancing motion quality and controllability is an inherent challenge in controllable motion generation. Our formulation explicitly addresses this trade-off by balancing objective satisfaction with a learned transition prior (Eq. 7). As shown in Table 4, removing this prior reduces waypoint success from 89.19% to 78.36% while substantially degrading motion plausibility (ARD increases from 0.384 to 0.741). Furthermore, Table 2 demonstrates that KTM (5 steps) achieves a more favorable trade-off than other methods, outperforming PRIMAL-G (50 steps) in both waypoint success (89.19% vs. 73.01%) and motion plausibility (ARD 0.384 vs. 0.705), while maintaining ARD close to the uncontrolled reference (0.384 vs. 0.379). These results demonstrate the effectiveness of our design in balancing control accuracy and motion plausibility during inference-time control.

Reviewer_atmNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper addresses training-free online human motion control: at inference time the user can introduce new, changing, or combined differentiable objectives G(Y) (waypoint reaching, obstacle avoidance, wrist reaching), and the generator must adapt in real time without control-specific retraining. KTM represents any future as an acceleration correction Z = A − Ā relative to an analytical "kinematic continuation" Ā of the observed boundary dynamics, so Y(Z) = κ(X, Ā + Z) stays boundary-attached to the observed motion. Generation proceeds through K finite stochastic latent-variable transitions, trained with analytic diffusion bridges. Control fixes the predicted transition and MAP-optimizes a single-component endpoint Z_K^G under G(Y) (Eq. 7).

Critical strengths
  1. Well-chosen, under-addressed problem. Jointly supporting (i) continual online generation, (ii) unseen inference-time differentiable objectives, and (iii) real-time execution is a genuine gap; Table 1 correctly identifies it. This is a meaningful and timely target.
  2. Strong navigation result. KTM is the only method that simultaneously achieves high success rate (89.19%) and low collision (18.06%) while remaining training-free, and it does so ~4× faster than the best PRIMAL-G setting (52 vs. 12 FPS). This is the paper's most convincing empirical result.
Critical weaknesses
  1. The theory does not support the control mechanism — the core technical gap. The paper's only theorem (Prop. 1 / Cor. 1) concerns the learning objective and only guarantees that the marginal mixture P_k ≈ Q_k (Eq. 28–29). But the control procedure (Alg. 1 line 5, Eq. 6–7) conditions on a single sampled component φ and performs MAP under a single-Gaussian surrogate p_k(Z_K|Z_k,φ) of the true multimodal endpoint distribution p(Z_K|Z_k) that appears in the
    h-transform (Eq. 5). Nothing establishes that (a) a single component μ(Z_k,φ) is a meaningful "motion mode", (b) p_k(Z_K|Z_k,φ) approximates p(Z_K|Z_k), or (c) the single-component MAP approximates the true controlled distribution p^G. Compounding this, the components are not even discrete modes: φ is continuous (Eq. 29) and the conditional
    variance is fixed to the isotropic bridge variance Σ(Z_k,φ)=σ_k²I with only the mean learned (App. B.3). So the "local motion prior" that regularizes control (Eq. 7, via the second-order penalty in Eq. 47) is a weak, unimodal, single-point anchor. Correctness of the paper's central contribution — the control step — is therefore currently heuristic, not demonstrated.
  2. Missing baselines with public code. Several directly relevant methods have released code but are not evaluated: DNO (korrawe/Diffusion-Noise-Optimization) and MaskControl (exitudio/MaskControl) are the training-free generic-objective baselines; A-MDM (Yi-Shi94/AMDM), MotionBricks (NVlabs/GR00T-WholeBodyControl), and ARDY (nv-tlabs/ardy) are online real-time methods that directly support waypoint navigation on the same hardware class. In particular, DART's own repo(zkf1997/DART) provides a training-free latent-noise-optimization spatial-control interface (and waypoint reaching), which the paper skips in favor of retraining the RL variant. At minimum, the 3.1 navigation task — supported by several of these methods — warrants a comparison.
  3. Collision metric is confounded with success rate. A method that does not move toward waypoints (uncontrolled, 7.63% SR) naturally collides less (28.02%) because it does not traverse the obstacle field. Thus "DART-RL collides more than the uncontrolled baseline" is not evidence that DART-RL is worse, and the cross-method collision comparison is not normalized. Collision should be reported per distance traveled or conditioned on obstacles actually approached, jointly with SR.
  4. The core modeling benefit is asserted rather than demonstrated. The acceleration-correction representation relative to a damped-constant-velocity continuation is essentially "predict acceleration residuals from a damping baseline." The paper argues its value through one ablation (Table 4) and the C.2 temporal-structure analysis, but it is not made crisply clear why this buys more than a better-conditioned pose-space model, and the drift rate λ is an untested hyperparameter.
  5. Presentation. There are several typos ("and and", "trainng-free", "optimizaiton", "progresively").
Questions for authors
  1. Single-component control. Do you have any argument or evidence that a single sampled component μ(Z_k,φ) is a meaningful motion mode?
  2. Missing baselines. Why were DNO and MaskControl (training-free, generic objectives) and A-MDM / MotionBricks / ARDY (online, real-time, waypoint-capable) — all with public code — not compared, at least on the 3.1 navigation task?
  3. End-effector failure mode. What limits the wrist-control accuracy (42% SR, 38.5 cm DTG)?
  4. your collision rate is not comparable across methods: the uncontrolled baseline barely moves (7.63% SR), so it rarely encounters obstacles and its low collision rate is meaningless. Can you report collisions only for obstacles the character actually approached, or per meter traveled?
11 comments
AuthorsOpenReview ↗

Weakness 1 & Question 1

We sincerely thank Reviewer atmN for the careful and constructive feedback, and for recognizing the importance of our problem setting, which jointly addresses continual online generation, unseen inference-time objectives, and real-time execution. We also appreciate the positive feedback of our navigation results, particularly the strong control performance and inference efficiency. We would like to address the reviewer's questions in detail below.


W1 & Q1. Single-component transition control.

(i) Role of single-component control and clarification of Q1

We thank the reviewer for discussing this important insight. We would like to clarify a key distinction in our control formulation. Motivated by our problem setting, our single-component control is deliberately designed to enable training-free online motion control under real-time constraints. As described in Sec. 2.3, Eq. 5 introduces the ideal controlled endpoint distribution as a motivation (but it is intractable), whereas Eq. 6--7 use a single latent-conditioned transition prediction to construct a tractable local endpoint prior. The purpose of this local prior is to regularize each controlled transition, rather than to recover the entire multimodal endpoint distribution with a single Gaussian. This provides a practical mechanism for training-free online control under real-time constraints, since the predicted transition remains fixed during endpoint optimization, avoiding repeated model evaluations or backpropagation through the generator.

Regarding Q1, perhaps there is some misunderstanding as our formulation does not claim that each sampled component represents a distinct semantically meaningful motion mode. Instead, different latent samples produce different plausible transition predictions that collectively capture variation toward plausible future motion. We would kindly request further discussion with the reviewer to clarify this.

(ii) Structured control regularization and progressive transition control.

Our local transition prior provides structured regularization for control. Although the local transition prior uses isotropic variance, its mean is learned from the current transition state and sampled latent and can vary across different transitions. As shown in Appendix D.1 (Eq. 48), endpoint optimization penalizes deviations from the learned transition. Furthermore, Appendix C.2 (Eq. 47) shows that this regularization induces a second-order temporal penalty on motion adjustments through our acceleration-correction representation, rather than penalizing independent pose deviations. Its effectiveness is supported by Table 4, where removing the transition prior decreases waypoint success from 89.19% to 78.36% and worsens ARD from 0.384 to 0.741.

Moreover, single-component control is applied progressively at each finite transition step, rather than relying on one fixed prediction throughout generation. The controlled state conditions the next transition, where a new latent is sampled and the local prediction is updated (Sec. 2.3). Table 4 demonstrates the benefit of this progressive control, improving waypoint success from 58.57% to 89.19% compared with applying control only at the final transition under the same optimization budget. Together, these results support our central contribution: progressively steering learned motion transitions toward inference-time objectives without additional motion-model evaluations during endpoint optimization.

(iii) Comparison of single-component versus multiple-component control.

To further assess the effectiveness of the single-component control, we compare control using 1, 4, and 8 sampled latent components per transition step, evaluating control performance and inference efficiency under the same navigation protocol.

Table R1. Single-component versus multiple-component transition control under the same online navigation protocol.

ComponentsSR ↑Coll. ↓ARD ↓Skate ↓FPS ↑Real-time
N=1N=1 (ours)89.19%18.06%0.3840.12952.18✓
N=4N=490.12%18.01%0.3560.13318.72×
N=8N=891.67%17.84%0.3760.11911.26×

Using multiple components yields only small improvements in control performance but substantially increases computation cost, failing to meet the real-time requirement. This supports our design of single-component control as a practical trade-off between control performance and inference efficiency, rather than an exact approximation of the full controlled distribution.

AuthorsOpenReview ↗

Weakness 2, 3 & Question 2, 4

W2 & Q2. Comparisons with relevant baselines

We thank the reviewer for suggesting the additional baselines. As discussed in Table 1 and Appendix A, the suggested methods address related but different control settings. Our main setting is training-free online control under real-time constraints. DNO and MaskControl support flexible inference-time objectives but are primarily designed for offline motion generation, without demonstrating online control under real-time constraints. In contrast, MotionBricks and ARDY achieve efficient online control through smart primitives or learned kinematic-constraint conditioning, rather than directly optimizing newly specified objective functions. A-MDM supports task-oriented sampling using user-defined scores. DART provides latent-noise optimization for spatial control, but this procedure is computationally expensive in the reported configuration (e.g., approximately 74 seconds for 60 frames with 100 optimization steps, Appendix F of DART). We initially compared against DART-RL real-time control variant due to this reason.

Despite different problem settings, we appreciate that additional comparisons with these models would provide valuable evidence and we thank the reviewer for the suggestion. We are currently working on experimenting with DART noise optimization, MaskControl, and ARDY, which are different representative control paradigms. Given our limited computing hardware, we prioritize these representative comparisons and will provide the results as soon as they are ready.

In the meantime, we kindly invite the reviewer to consider our selection of representative baselines over DNO, A-MDM, and MotionBricks, given their related control capabilities and our limited computational resources. We look forward to further discussion on whether these comparisons sufficiently address the concern.


W3 & Q4. Collision rate and success rate

We thank the reviewer for this valuable observation. Our navigation benchmark evaluates waypoint reaching and obstacle avoidance together, so collision rate should be interpreted jointly with success rate. In our benchmark, obstacles are deliberately placed along the required waypoint routes to evaluate the combined control objective. Methods that fail to follow these routes may encounter fewer obstacles, resulting in deceptively low collision rate. For example, the uncontrolled model may move away from the intended route (evidenced by only 7.63% success rate), while avoiding obstacles along the intended route without actually performing obstacle avoidance. Thus, its collision rate will not correctly indicate obstacle avoidance. The uncontrolled reference is primarily included as a reference to motion quality and the effect of objective-driven control to see if other methods can actually respond to the waypoint target. Similarly, our assessment of DART-RL is based on its overall navigation performance, rather than the collision rate relative to the uncontrolled reference.

Thanks for your thoughtful suggestion, we additionally report collision rate based on approached obstacles to provide a more informative comparison, which measures the proportion of collided obstacles among those approached within a proximity threshold along the generated trajectory. We also report the percentage of obstacles approached to contextualize the results.

Table R2. Obstacle avoidance under online waypoint navigation. Approached (App.) denotes the percentage of obstacles approached during execution; Coll./App. denotes the collision rate among approached obstacles.

MethodSR↑Coll.↓App.Coll./App.↓
Uncontrolled7.63%28.02%36.50%77.78%
PRIMAL-G (10)42.28%28.23%60.48%46.02%
PRIMAL-G (50)73.01%25.18%77.76%33.72%
DART-RL61.72%35.04%81.63%43.12%
KTM (ours)89.19%18.06%87.83%20.52%

Interestingly, KTM achieves the highest waypoint success (89.19%) while approaching the largest proportion of obstacles (87.83%), yet still maintains the lowest collision rate among approached obstacles (20.52%). It also reveals that the uncontrolled reference approaches much fewer obstacles, explaining why its relatively low collision rate does not necessarily indicate effective obstacle avoidance. Thank you for suggesting this additional analysis, which provides further evidence that KTM effectively balances waypoint reaching and obstacle avoidance under online control, rather than achieving a low collision rate by merely avoiding the intended navigation routes.

AuthorsOpenReview ↗

Weakness 4, 5 & Question 3

W4. Insights of the kinematic acceleration-correction representation

We thank the reviewer for raising this thoughtful question. Our kinematic acceleration-correction representation provides two complementary benefits for online motion generation and control: (i) the kinematic continuation provides a meaningful starting future consistent with the observed boundary dynamics, and (ii) acceleration-correction coordinates introduce temporal structure into stochastic transitions and control. As analyzed in Appendix C.2 (Eq. 45--47), acceleration perturbations induce temporally correlated changes in future poses, while the control regularization penalizes second-order temporal deviations rather than independent pose deviations per frame. This temporal structure is incorporated directly into the learned stochastic transition and control process, providing a coherent kinematic representation for generating and controlling future motion.

Additional representation ablations. To further investigate the benefits of our formulation, we are conducting additional ablations with: (i) a pose-residual model initialized from the same kinematic continuation, using standard pose-space L2L_2 control regularization; (ii) the same pose-residual model with second-order temporal regularization, equivalent to our acceleration-space control penalty (Eq. 47); and (iii) our KTM. We are also evaluating sensitivity to the acceleration drift rate η\eta to further assess the robustness of our kinematic formulation. We believe these comparisons will help distinguish the benefits of the kinematic initialization, temporal control regularization, and acceleration-space stochastic transition modeling. These experiments are still running (given our limited hardware) and will be reported in a few days as soon as they are ready.

In the meantime, we kindly invite the reviewer to further discuss whether our analysis has clarified the benefits of the acceleration-correction representation for stochastic generation and control.


Q3. End-effector control challenge

We thank the reviewer for this meaningful question regarding wrist reaching control. Our evaluation in Sec. 3.2 extends the original WANDR reaching protocol to a more challenging online setting, where wrist targets change sequentially during continuous motion generation and must be reached within specified deadline. This setting directly evaluates our central study, which is adapting the same pretrained motion model to changing inference-time objectives without task-specific training. Our intended goal is to demonstrate that the same pretrained motion model can support new and changing control objectives without reaching-specific training, rather than optimizing exclusively for reaching accuracy.

To further contextualize the challenge, we have performed additional evaluations under WANDR's published reaching protocol and report them below. As we can see, KTM achieves 82.00% success rate and 12.35 cm distance-to-goal, substantially outperforming WANDR's results despite requiring no reaching-specific training.

Table R3. Evaluation on WANDR's original single-target 3D reaching protocol.

MethodSR ↑DTG (cm) ↓Foot skating (FS) ↓
GOAL0%149.2029.00%
WANDR32%24.8016.00%
KTM (ours)82%12.359.69%

The main challenge in sequential reaching is satisfying newly introduced wrist targets within limited time while preserving plausible motion. When a new target requires substantial body adjustment, reaching may require multiple control cycles. Consequently, the target may expire before completion since the character needs to preserve plausible movement. More aggressive control could improve accuracy but compromise motion plausibility. This highlights the key challenge between reaching accuracy and motion plausibility under changing online objectives.

Notably, our KTM still outperforms the reaching-specific WANDR model under the same protocol, despite requiring no reaching-specific training. These results support our contribution of adapting the same pretrained motion model to different and changing objectives through a generic training-free control interface, highlighting the open challenge of achieving precise control under continuously changing constraints.


W5. Minor presentation issue

Thank you for identifying these typos! We will fix them and double check the complete paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall rebuttal summary for Reviewers and Area Chairs

Dear Reviewers and Area Chairs,

We sincerely thank our reviewers for their careful and constructive feedback, which has helped us further improve our paper. We greatly appreciate the recognition of the importance of our problem setting (atmN, yNJx), strong navigation results and inference efficiency (atmN), and computationally attractive control procedure and motion plausibility (yNJx). We also thank Reviewer 6S5f for their assessment of our contributions and valuable suggestions regarding prior work and clarity.

We have provided detailed responses to all reviewer questions in our rebuttal discussions. We have updated the manuscript accordingly. Below we summarize the main clarifications and improvements.

1. Clarification of Technical Contributions and Theoretical Foundation

  • We have clarified our main contribution of integrating few-step stochastic motion generation and training-free online control through the same kinematic transition process. In particular, we distinguish multimodal transition learning (Eq. 3) from single-component control (Eq. 7), which uses one sampled conditional transition as an efficient practical approximation for real-time control, rather than recovering the full controlled distribution.
  • We further clarify that the local transition prior provides consistency regularization with learned motion transitions, which are directly supervised by analytical transitions constructed from real motion (Appendix B.4). The resulting controlled state conditions subsequent predictions, allowing the objective to progressively steer the learned transition process.
  • We clarify the purpose of Appendix B as the mathematical foundation for constructing and learning the finite transition process, while Appendices C and D further explain the temporal structure and control regularization.

2. Additional Baseline Comparisons

  • We additionally evaluate DART's official latent diffusion-noise optimization, MaskControl, and ARDY as representative control paradigms under our sequential waypoint navigation and dynamic obstacle avoidance benchmark. The results further demonstrate KTM's favorable trade-off between training-free control, real-time online execution, and motion quality.
  • We also compare with traditional methods Motion Matching and Learned Motion Matching combined with path planning, and evaluate KTM on DART's original waypoint-reaching protocol and WANDR's original 3D reaching protocol. These comparisons provide additional evidence of KTM's effectiveness across established and more challenging online control settings.

3. Additional Ablations and Robustness Analysis

  • We conduct additional experiments comparing single-component versus multiple-component transition control, demonstrating that single-component control provides a practical balance between control performance and real-time efficiency.
  • We provide pose-space model ablations using the same kinematic extrapolation and either standard pose-space regularization or second-order temporal regularization, to further validate the benefits of our acceleration-correction transition representation.
  • We additionally investigate sensitivity to the acceleration drift rate and robustness to external velocity perturbations, providing further evidence of KTM's effectiveness under changing motion states.

4. Evaluation Protocol and Baseline Fidelity

  • To address the concern regarding collision-rate comparability, we additionally report the percentage of approached obstacles (App.) and collision rate among approached obstacles (Coll./App.), interpreted jointly with waypoint success. KTM achieves 89.19% waypoint success while approaching 87.83% of obstacles, with the lowest 20.52% collision rate among approached obstacles.
  • We extend Appendix F with baseline implementation details and online evaluation settings. We clarify that PRIMAL-G uses PRIMAL's official pretrained model and classifier-based guidance interface, while DART-RL receives the same control information as KTM.

5. Presentation and Scope Clarifications

  • Following the reviewers' suggestions, we have refined the Introduction, method exposition and terminology to more clearly explain our problem scope and contributions.
  • We clarify the scope of unseen differentiable objectives, short-horizon online control, and the distinction between our kinematic motion generation setting and physics-based methods such as Diffuse-CLoC and BeyondMimic. We also improve the discussion of related work, limitations, and the trade-off between motion quality and controllability.

We are sincerely grateful for the reviewers' time and their insightful feedback that helped us improve the practical impact of the paper. We believe our detailed responses and revised manuscript have addressed the reviewers' concerns and clarified the contributions of our work.

Best regards,
#1386 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.