acceptodds
Under review as a conference paper at ICLR 2027

Dynamic Neural Koopman Distillation for Fast Robot Control Using Diffusion Models

Abstract

Diffusion models excel at generating diverse, multimodal trajectories for robotic control, yet their iterative denoising process introduces latency that can constrain real-time closed-loop execution. To address this problem, we propose a Dynamic Neural Koopman (DNK) distillation framework, which distills multistep diffusion inference into a single forward pass conditioned on observations and sampled noise. Specifically, we introduce a Factorized Dynamic Koopman (FDK) layer that models the denoising process through a latent linear transition. The noisy trajectory input is lifted into a latent space, where the FDK layer parameterizes a factorized linear operator with condition-dependent eigenvalues that adapt the noisy-to-denoised transition to the current context. We evaluate DNK across robot-control benchmarks that span locomotion, state-based, and image-based manipulation, comparing against accelerated generative-policy baselines. The results demonstrate that DNK achieves competitive or improved closed-loop performance relative to accelerated generative-policy baselines, while substantially reducing inference latency relative to the strongest high-performing one-step baselines. On long-horizon manipulation tasks, DNK achieves competitive performance with substantially fewer inference parameters than the one-step policy baselines. Hardware experiments on a physical Kinova manipulator further demonstrate millisecond-scale inference and reduced task completion time. A project page is available at https://fdkoopman.github.io/.

Primary area:
generative models
Keywords:
Diffusion Models, Generative Policies, Robot Control, Imitation Learning, Policy Distillation, Real-Time Control

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SHaSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Dynamic Neural Koopman (DNK) distillation to accelerate diffusion-based robot control by approximating multistep reverse diffusion with a single latent transition. Its core component, the Factorized Dynamic Koopman (FDK) layer, parameterizes the latent operator as �0��0�, where a shared modal basis is combined with context-dependent modal gains. This introduces observation-dependent adaptation while maintaining a compact operator parameterization. The student is trained using offline teacher-generated trajectory pairs, latent alignment, control-oriented supervision, and spectral regularization. The method is evaluated on D4RL locomotion, state- and image-based Push-T, Franka Kitchen, and a physical Kinova Gen3 obstacle-reaching task. The results show competitive closed-loop performance with substantially lower inference latency than multistep diffusion teachers, favorable performance–latency–capacity trade-offs relative to high-capacity one-step baselines, and substantial improvements over static Koopman distillation baselines.

Critical strengths
  1. The paper is clearly written, well organized, and well motivated. The proposed FDK layer is simple and intuitive, yet effective in adapting latent transitions to different observation contexts. The ablation studies provide good empirical support for the main claim regarding context-dependent modal adaptation. This strengthens my assessment of the paper's technical soundness, as the proposed design is clearly motivated and its central claim is supported by targeted experiments.

  2. The experimental evaluation is reasonably comprehensive, covering locomotion, state-based and image-based contact-rich manipulation, and long-horizon manipulation. The results demonstrate a particularly competitive trade-off among closed-loop performance, inference latency, and model capacity. This is a significant strength because it establishes the practical value of the method across different simulated control settings, particularly for computationally constrained robotic deployment.

Critical weaknesses
  1. The hardware evaluation is limited to a single state/goal/obstacle-conditioned reaching task and compares DNK only against the multistep diffusion teacher, rather than other strong one-step policies. While the results convincingly demonstrate the benefit of low-latency inference in this setting, they provide limited evidence for broader real-world applicability or superiority over existing one-step approaches. This limits my confidence in how well the reported advantages generalize to more diverse real-world robotic tasks and whether they are specific to DNK rather than one-step inference in general.

  2. Although the ablations demonstrate the benefit of context-dependent modal gains over static gains, they do not isolate the contribution of the specific shared-eigenspace constraint imposed by the FDK factorization �0��0�. Comparisons against alternative conditional linear parameterizations would help clarify whether this structural constraint is essential to the observed improvements. Furthermore, the theoretical analysis mainly establishes the advantage of conditional over constant gains under a fixed latent representation and derives spectral properties of the chosen factorization, providing relatively limited justification for the Koopman-specific inductive bias. This leaves some uncertainty about whether the observed improvements arise from the particular FDK structure or from context-dependent latent modulation more generally, which limits my assessment of the distinct methodological contribution.

Questions for authors
  1. On the specific contribution of the FDK factorization: The proposed FDK operator �0��0� constrains the context-dependent operators to share a common eigenspace and to commute with one another. While the current ablations demonstrate the effectiveness of context-dependent modal gains over static gains, they do not isolate the importance of this structural constraint. Have you evaluated or considered a more general two-sided diagonal factorization, such as �1��1�, where �2��2� and �3��3� are independently learned, under comparable parameter and computational budgets? Such a comparison would help clarify whether the observed gains primarily arise from context-dependent modulation or from the shared-eigenspace structure itself.

  2. On the scope of the hardware comparison: The real-world experiment demonstrates a substantial latency benefit on the Kinova obstacle-reaching task, but the hardware comparison is limited to the multistep diffusion teacher. Have you evaluated any strong one-step baseline, such as OneDP-S or Consistency Policy, under the same hardware setup? If not, could you clarify the main practical limitations preventing such a comparison? This would help assess whether the reported real-world advantage is specific to DNK or primarily reflects the general benefit of one-step inference over multistep diffusion sampling.

7 comments
AuthorsOpenReview ↗

Response Part 1

W1: The hardware evaluation is limited to a single state/goal/obstacle-conditioned reaching task and compares DNK only against the multistep diffusion teacher, rather than other strong one-step policies. While the results demonstrate the benefit of low-latency inference in this setting, they provide limited evidence for broader real-world applicability or whether the reported hardware advantages are specific to DNK rather than one-step inference in general.

Response:
We did not evaluate competing one-step baselines (such as OneDP-S or Consistency Policy) on the physical Kinova platform. There is no architectural or hardware barrier preventing their deployment under the same setup; rather, our physical evaluation budget was constrained by robot access time and operational manpower (as each evaluated policy requires dedicated teacher/student training on the custom B-spline scene distribution followed by 5050 physical trials under safety supervision).

Instead, our experimental design separates broad cross-method benchmarking from physical closed-loop validation:

  1. Controlled head-to-head one-step comparisons in simulation: Standard benchmarks (Franka Kitchen, state-based and image-based Push-T, and D4RL locomotion) provide strictly reproducible resets, identical initial-state distributions across 3×503\times 50 rollouts, and diverse contact-rich dynamics. Across these established benchmarks, we compare DNK directly against strong one-step baselines (including OneDP-S, Consistency Policy, RACTD, CD, CT, KDM/KDM-F, and Mean Velocity Policy [MVP] in Appendix D.8, Table 14). For example, on Franka Kitchen (Table 3), DNK matches OneDP-S in task score (0.973±0.0160.973 \pm 0.016 vs. 0.975±0.0130.975 \pm 0.013) while using 28×28\times fewer deployed parameters (2.46M2.46\text{M} vs. 68.92M68.92\text{M}) and achieving 4.03×4.03\times lower per-step latency (1.598 ms1.598\text{ ms} vs. 6.434 ms6.434\text{ ms}).
  2. Closed-loop deployability validation on physical hardware: Our Kinova study verifies that DNK's one-step trajectories remain smooth, safe, and executable under real sensor noise, asynchronous 80 Hz80\text{ Hz} Cartesian tracking, and tight latency budgets relative to the multistep teacher. It is not intended to independently claim hardware superiority over other one-step methods.

We have revised the introduction to Section 5 (Experiments), Section 5.4 (Real-World Experiment), and Appendix C (Real-World Experiments) to explicitly state these complementary roles of simulation benchmarking and physical validation, and to clarify the scope of the hardware conclusions.

AuthorsOpenReview ↗

Response Part 2

W2: Lack of comparisons against alternative conditional linear parameterizations to isolate the shared-eigenspace FDK constraint (K(c)=Vdiag(γ(c))V−1K(c)=V\text{diag}(\gamma(c))V^{-1}) from general context-dependent latent modulation, and limited theoretical justification for the Koopman-specific inductive bias.

Response:
The shared-eigenspace restriction Kω(c)=Vdiag(γω(c))V−1\mathbf{K}_{\omega}(\mathbf{c})=\mathbf{V}\text{diag}(\gamma_{\omega}(\mathbf{c}))\mathbf{V}^{-1} provides three concrete structural benefits over unconstrained conditional maps: spectral interpretability, parameter efficiency, and exact inverse consistency. Because all context-dependent operators share a common eigenbasis V\mathbf{V} with exact inverse V−1\mathbf{V}^{-1}, the entries of γω(c)\gamma_{\omega}(\mathbf{c}) correspond to true modal eigenvalues, which makes our spectral-radius regularizer mathematically well-defined and requires learning only a single basis matrix V\mathbf{V}.

To directly test whether the empirical gains arise from this inverse-linked shared-eigenspace structure rather than from context-dependent latent modulation in general, we added two controlled ablations under the exact same training recipe and formal 3×503\times 50 evaluation protocol, replacing only the latent transition operator (Appendix D.3, Table 10 and Appendix D.4, Table 11):

  1. Unconstrained Two-Sided Diagonal Factorization (Kω(c)=W2diag(γω(c))W1\mathbf{K}_{\omega}(\mathbf{c})=\mathbf{W}_2\text{diag}(\gamma_{\omega}(\mathbf{c}))\mathbf{W}_1, Appendix D.3, Table 10): Here W1\mathbf{W}_1 and W2\mathbf{W}_2 are learned independently without enforcing W2=W1−1\mathbf{W}_2 = \mathbf{W}_1^{-1}.
  2. Capacity-Matched Conditional MLP (z(0)=MLP([z(1);c])\mathbf{z}^{(0)} = \text{MLP}([\mathbf{z}^{(1)}; \mathbf{c}]), Appendix D.4, Table 11): We replace the FDK operator with an unconstrained nonlinear conditional MLP whose parameter count is matched to Exact FDK within 10−3M10^{-3}\text{M} parameters (2.457M2.457\text{M} on Franka Kitchen and 2.367M2.367\text{M} on state-based Push-T).
TaskExact FDK (Vdiag(γ(c))V−1\mathbf{V}\text{diag}(\gamma(\mathbf{c}))\mathbf{V}^{-1})Two-Sided (W2diag(γ(c))W1\mathbf{W}_2\text{diag}(\gamma(\mathbf{c}))\mathbf{W}_1)Capacity-Matched Cond. MLP
Franka Kitchen0.973 ± 0.0160.883 ± 0.0180.910 ± 0.017
Push-T (state)0.931 ± 0.0370.819 ± 0.1290.869 ± 0.048

Across both evaluated benchmarks, Exact FDK consistently outperforms both the unconstrained two-sided factorization (0.973±0.0160.973 \pm 0.016 vs. 0.883±0.0180.883 \pm 0.018 on Franka Kitchen; 0.931±0.0370.931 \pm 0.037 vs. 0.819±0.1290.819 \pm 0.129 on state-based Push-T, with substantially lower rollout variance) and the capacity-matched conditional MLP (0.973±0.0160.973 \pm 0.016 vs. 0.910±0.0170.910 \pm 0.017 on Franka Kitchen; 0.931±0.0370.931 \pm 0.037 vs. 0.869±0.0480.869 \pm 0.048 on state-based Push-T). Together with our identity and static-gain ablations (Appendix D.1, Table 8 and Appendix D.2, Table 9), these results demonstrate that the performance gains do not merely come from "any conditional latent map," but specifically benefit from the inverse-linked shared-basis inductive bias.

Finally, regarding the theoretical analysis (Section 4.2 and Appendix A), we clarify that FDK is Koopman-inspired: Proposition 1 (Appendix A.1) formally establishes the strict population-risk advantage of context-dependent modal gains over static gains under a fixed latent representation and shared basis, while Proposition 2 and Corollary 1 (Appendix A.2) derive the exact spectral structure and fixed-context output sensitivity bounds for Vdiag(γω(c))V−1\mathbf{V}\text{diag}(\gamma_{\omega}(\mathbf{c}))\mathbf{V}^{-1} that justify our spectral regularizer (supported empirically by the basis conditioning diagnostics in Appendix B.2, Table 5), rather than claiming recovery of true underlying diffusion SDE dynamics or a standalone proof of closed-loop stability.

AuthorsOpenReview ↗

Response Part 3

Q1: Whether a more general two-sided diagonal factorization K(c)=W2diag(γ(c))W1K(c)=W_2\text{diag}(\gamma(c))W_1 (with independently learned W1,W2W_1, W_2) has been evaluated under comparable parameter and compute budgets to isolate the shared-eigenspace constraint.

Response:
Following your suggestion, we directly evaluated the unconstrained two-sided factorization Kω(c)=W2diag(γω(c))W1\mathbf{K}_{\omega}(\mathbf{c})=\mathbf{W}_2\text{diag}(\gamma_{\omega}(\mathbf{c}))\mathbf{W}_1 with independently learned W1\mathbf{W}_1 and W2\mathbf{W}_2 under matched experimental settings (Appendix D.3, Table 10). As detailed in our response to Weakness 2 above, the shared-eigenspace restriction Kω(c)=Vdiag(γω(c))V−1\mathbf{K}_{\omega}(\mathbf{c})=\mathbf{V}\text{diag}(\gamma_{\omega}(\mathbf{c}))\mathbf{V}^{-1} provides three structural benefits over W2diag(γω(c))W1\mathbf{W}_2\text{diag}(\gamma_{\omega}(\mathbf{c}))\mathbf{W}_1: spectral interpretability (entries of γω(c)\gamma_{\omega}(\mathbf{c}) are true eigenvalues, enabling direct spectral-radius regularization), parameter efficiency (learning a single basis matrix V\mathbf{V} rather than two unconstrained matrices, 2.457M2.457\text{M} vs. 2.474M2.474\text{M} on Franka Kitchen and 2.367M2.367\text{M} vs. 2.383M2.383\text{M} on Push-T), and exact inverse consistency by construction. Empirically, Exact FDK achieves substantially higher closed-loop scores than W2diag(γω(c))W1\mathbf{W}_2\text{diag}(\gamma_{\omega}(\mathbf{c}))\mathbf{W}_1 on both Franka Kitchen (0.973±0.0160.973 \pm 0.016 vs. 0.883±0.0180.883 \pm 0.018) and state-based Push-T (0.931±0.0370.931 \pm 0.037 vs. 0.819±0.1290.819 \pm 0.129). Together with the capacity-matched conditional MLP ablation in Appendix D.4 (Table 11), which replaces the factorized linear operator with an unconstrained nonlinear network, these results confirm the effectiveness of the shared-eigenbasis FDK structure.

Reviewer_XDg9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DNK distillation to turn a multistep diffusion robot policy into a one-pass student. Its main contribution is a Factorized Dynamic Koopman layer: an encoder lifts a noisy trajectory into a latent space, a shared modal basis and observation-dependent gains transform it, and a decoder produces an action trajectory. The student learns from offline teacher outputs using latent, trajectory, and action-focused losses, plus spectral regularization. The authors evaluate DNK on D4RL locomotion, state- and image-based Push-T, Franka Kitchen, and a Kinova robot. They report closed-loop performance near strong teacher and one-step baselines with lower inference latency and fewer deployed parameters; the Kinova experiment shows a reduction from 151 ms to 4.08 ms mean inference latency.

Critical strengths
  1. Strong performance–efficiency trade-off. DNK substantially reduces model size and inference latency while retaining competitive control performance. On Franka Kitchen, it achieves a score of 0.973 versus 0.975 for OneDP-S, using approximately 28× fewer parameters and 4× lower inference latency. Achieving this compact operating point is practically valuable for deploying generative robot policies.
  2. The evaluation covers locomotion, state- and image-based manipulation, long-horizon task execution, and a physical robot. On Kinova, mean inference latency decreases from 151 ms to 4.08 ms, accompanied by shorter task completion times. This breadth supports the practical usefulness of the approach beyond a single benchmark.
Critical weaknesses
  1. The distinctive contribution of the Koopman parameterization remains insufficiently established. The identity ablation demonstrates the usefulness of a learned latent transition, but the paper lacks comparisons against similarly sized conditional MLPs or alternative structured transformations. The matched conditional-versus-static gain comparison is limited to Kitchen. Moreover, the theoretical analysis concerns optimal latent regression with a fixed representation; it does not establish that the learned mapping captures diffusion dynamics or improves closed-loop control. These gaps make it difficult to determine whether the gains specifically justify the proposed Koopman structure.

  2. The comparisons leave uncertainty about the advantage over other efficient policies. The evaluation omits recent single-step flow policies, while the reduced-capacity OneDP-S experiment changes both model capacity and initialization, limiting its ability to isolate architectural advantages. Hardware evaluation compares only against a relatively slow multistep teacher, whose 0% success rate is tied to a 100-second deadline despite eventual task completion. Comparisons with competitive fast policies under matched deployment conditions are needed to establish the method’s broader practical advantage.

Questions for authors
  1. Given a fixed observation and initial noise, is the teacher target deterministic, or does reverse diffusion introduce additional randomness? If additional randomness is present, how is it represented in the student input? This matters because regression to stochastic endpoints may favor conditional averaging. Beyond the PCA visualization, what evidence supports preservation of distinct action modes and their coverage?
  2. Since the shared basis (V) is otherwise unconstrained, how is nonsingularity maintained during training? Can the authors report its condition numbers and the observed modal-gain magnitudes? The inverse residuals in Table 5 establish numerical consistency, but do not establish limited perturbation amplification. Please clarify which robustness properties are supported by the spectral regularizer and which remain assumptions.
  3. What explains the different DNK latencies reported for the same task—for example, 1.283 ms for state-based Push-T in Table 2 versus (2.2\pm1.0) ms in Table 8, and 1.598 ms for Kitchen in Table 3 versus 1.485 ms in Table 8? Please specify whether each measurement includes observation encoding, device transfers, candidate generation, and candidate scoring.
  4. How much does locomotion performance depend on generating 64 candidates and selecting with the external IQL critic? Are single-candidate results or results at other candidate budgets available? This would clarify whether the reported returns reflect consistently useful student samples or rely on the selector finding a small subset of successful actions.
  5. How were the 50 hardware test configurations chosen relative to the training distribution of starts, goals, and obstacle layouts? Does collision monitoring cover only the end-effector or the entire arm? These details determine the scope of the demonstrated generalization and obstacle-avoidance performance.
11 comments
AuthorsOpenReview ↗

Response Part 1

W1: Insufficiently established contribution of the Koopman parameterization: lack of comparisons against similarly sized conditional MLPs or alternative structured transformations, static-gain ablation limited to Kitchen, and theoretical analysis limited to fixed-representation latent regression rather than diffusion dynamics or closed-loop control.

Response:
We thank the reviewer for this comment. To isolate the contribution of the FDK parameterization beyond the identity ablation, we have added (i) a capacity-matched conditional MLP baseline (Appendix D.4, Table 11), (ii) an alternative structured linear transformation using an unconstrained two-sided diagonal factorization Kω(c)=W2diag(γω(c))W1K_{\omega}(c) = W_{2}\text{diag}(\gamma_{\omega}(c))W_{1} without the shared-eigenbasis constraint W2=W1−1W_{2} = W_{1}^{-1} (Appendix D.3, Table 10), and (iii) an extended conditional-versus-static gain study on both Franka Kitchen and state-based Push-T (Appendix D.2, Table 9).

Under matched capacity, data, and training recipes, Exact FDK outperforms both the conditional MLP on Push-T state (0.931 ± 0.037 vs. 0.869 ± 0.048), Push-T image (0.853 ± 0.041 vs. 0.762 ± 0.056), and Franka Kitchen (0.973 ± 0.016 vs. 0.910 ± 0.017), and the unconstrained two-sided factorization on Push-T state (0.931 ± 0.037 vs. 0.819 ± 0.129) and Franka Kitchen (0.973 ± 0.016 vs. 0.883 ± 0.018). Moreover, conditional modal gains consistently improve over both the static-vector and parameter-matched static-head variants on Franka Kitchen (0.973 ± 0.016 vs. 0.945 ± 0.018 and 0.910 ± 0.005) as well as on state-based Push-T (0.931 ± 0.037 vs. 0.834 ± 0.096 and 0.857 ± 0.088, with tighter variance across evaluation groups), which we now state explicitly. Together with the identity ablation in Appendix D.1 (Table 8), these results indicate that the closed-loop gains are not explained by capacity or observation-conditioning pathways alone and that the inverse-linked factorized Koopman transition is the operative component.

We also clarify that Appendix A analyzes optimal latent regression under a fixed representation and is intended only to motivate the form of Kω(c)K_{\omega}(c); it does not claim that the learned map recovers diffusion dynamics or that theory alone predicts closed-loop improvement. Our empirical claim remains competitive one-step performance at lower latency and parameter count on the three representative benchmarks (Tables 1–3), compared with representative baselines.

AuthorsOpenReview ↗

Response Part 2

W2: Uncertainty about advantages over other efficient policies: omission of recent single-step flow policies, confounding of capacity and initialization in the reduced-capacity OneDP-S study, and hardware evaluation comparing only against a slow multistep teacher with a 100s deadline.

Response:
We thank the reviewer for the constructive feedback. As stated in our evaluation setup (Section 5.1 and Appendix B.1), the primary comparisons in the main text (Tables 1–3) evaluate the full-capacity, teacher-matched OneDP-S [1] (ICML 2025) student distilled from the exact same frozen DDPM teacher (TDT_D) and under the exact same host protocol as DNK; the reduced-width study in Appendix D.7 (Table 13) is included solely as a supplementary capacity stress test of the OneDP-S training protocol.

Regarding flow-based policies, standard flow-matching methods require multi-step ODE integration (NFE>1\text{NFE}>1), whereas recent one-step flow policies such as Mean Velocity Policy (MVP) [2] (ICLR 2026 Oral) rely on direct mean-flow / offline-to-online reinforcement learning rather than pure offline distillation from a frozen diffusion teacher, which is why MVP is reported separately from the teacher-distillation tables in the main text. To further establish DNK's advantage over recent efficient single-step policies, our revision now includes comparisons with two newly released baselines from this year: RACTD [3] (ICLR 2026) and the one-step flow policy MVP [2] (ICLR 2026 Oral). Notably, these baselines represent the strongest and most up-to-date one-step methods across both diffusion-distillation and flow-matching paradigms, with MVP in particular highlighted as an ICLR 2026 Oral for one-step action generation. On the state-based Push-T and Franka Kitchen tasks, DNK achieves higher task scores of 0.931±0.0370.931\pm0.037 and 0.973±0.0160.973\pm0.016, respectively, outperforming RACTD (0.706±0.0320.706\pm0.032 / 0.970±0.0100.970\pm0.010) and MVP (0.758±0.0470.758\pm0.047 / 0.510±0.0170.510\pm0.017) while requiring substantially lower latency (4.1×4.1\times faster) and fewer parameters (27×27\times smaller) than RACTD, and roughly half the deployed parameters of MVP. The corresponding results are reported in Section 5.2 (Tables 1–3) and Appendix D.8 (Table 14).

Furthermore, we clarify that the primary comparisons against competitive one-step policies are the matched simulation evaluations in Tables 1–3 and Table 14, whereas the Kinova hardware study in the main text (Section 5.4) serves as a physical validation of real-time closed-loop feasibility. In the 10-page main text, we report only inference latency (4.08 ms4.08\ \text{ms} vs. 151.00 ms151.00\ \text{ms}) and task completion time (85.43 s85.43\ \text{s} vs. 314.95 s314.95\ \text{s}) relative to the teacher; the 0%0\% teacher success rate under a 100-second100\text{-second} deadline appeared solely in Appendix C as a supplementary deadline-based view of the same runs, and does not imply that the teacher fails to complete the task without that cutoff. To avoid any misunderstanding, we have removed the deadline-based success rate and refer to the updated Appendix C and Table 7.

References:

[1] Zhendong Wang et al. "One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation." ICML, 2025.

[2] Guojian Zhan et al. "Mean Flow Policy with Instantaneous Velocity Constraint for One-Step Action Generation." ICLR, 2026.

[3] Xintong Duan et al. "Accelerating Diffusion Planners in Offline RL via Reward-Aware Consistency Trajectory Distillation." ICLR, 2026.

AuthorsOpenReview ↗

Response Part 3

Q1: Given a fixed observation and initial noise, is the teacher target deterministic, or does reverse diffusion introduce additional randomness—and if so, how is it represented in the student input so that regression to stochastic endpoints does not favor conditional averaging? Beyond the PCA visualization, what evidence supports preservation of distinct action modes and their coverage?

Response:
We thank the reviewer for this insightful question. First, we clarify that strict multimodality preservation is not a claimed contribution of this work and is not discussed in the 10-page main text, where our primary focus remains on closed-loop task score, inference latency, and parameter efficiency. We included the PCA visualization in the appendix (Appendix D.6, Figure 8) simply to share an interesting qualitative side observation that the one-step student can still generate diverse trajectories in the same spirit as the DDPM teacher, which we thought might be of interest to readers. We have added a note in the appendix to clarify this intent and to highlight that rigorously characterizing and preserving distinct action modes is an interesting direction for future exploration.

Regarding the teacher target and student input, we use ancestral DDPM teachers, so each distillation target is a stochastic reverse sample. The corresponding initial noise is provided to the student through the conditioned prior τprior\boldsymbol{\tau}_{\mathrm{prior}}, together with the observation, so the student is trained as a deterministic one-step map on the same observation-conditioned noisy prior used to query the teacher. Because this initial noise is explicitly part of the student input, different noise samples induce different outputs rather than being marginalized out during training.

Reviewer_TSheOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Authors propose a one-step diffusion distillation framework that uses offline noise-context-teacher trajectory tuples. They use a Factorized Dynamic Koopman (FDK) layer to apply a shared modal basis and condition dependent modal gains. with Their training uses teacher action regression, latent alignment, and spectral regularization. Experiments on several simulation and real world tasks are used to evaluate performance of their proposed framework.

Critical strengths
  1. Theoretically grounded approach for faster robot control with diffusion models.
  2. Clear ablative studies.
  3. Method generalizes to real world experiments.
Critical weaknesses
  1. Lacking comparisons against newer one step distillation methods. There are many such methods discussed in the related works section, but not compared against in the experiments section.
  2. The conceptual distinction between proposed method and existing one step distillation methods (e.g. single step TI2I models that are conditional) is unclear and statements such as "Although their training objectives exploit diffusion states or teacher-generated samples, the resulting students are typically parameterized as direct conditional generators without an explicit context-dependent latent transition." introduce additional confusion.
  3. All results comparisons are against older weaker baselines. No comparisons to latest state of the art robot control methods.
Questions for authors

See weaknesses - please address these concerns.

8 comments
AuthorsOpenReview ↗

Response Part1

W1. Lacking comparisons against newer one-step distillation methods.

Response: We thank the reviewer for this comment and stress the correspondence between the literature discussed in Section 2 (Related Work) and the baselines evaluated in Section 5 (Experiments).

In the original submission, we evaluated the representative one-step distillation methods from each category discussed in Section 2, including Consistency Distillation (CD) [1], Consistency Training (CT) [2], Consistency Policy (CP) [3], OneDP-S [4], and KDM / KDM-F [5], as reported in Section 5.2 (Tables 1–3). In particular, OneDP-S [4] (ICML 2025) and KDM [5] (NeurIPS 2025) are recent one-step distillation methods, and OneDP-S is distilled from the exact same DDPM teacher as DNK. The remaining methods cited in Section 2 that we did not compare against, namely DDIM [6], DPM-Solver [7], and Progressive Distillation [8], were discussed to provide broader context on diffusion acceleration rather than as one-step distillation baselines. Specifically, DDIM and DPM-Solver are widely used iterative multi-step ODE samplers (NFE>1\text{NFE} > 1, with EDM [9] already included as our fast-sampled multi-step teacher reference TET_E), while Progressive Distillation [8] (ICLR 2022) is an earlier multi-stage step-halving framework that has largely been superseded in one-step generation by Consistency Models (CD/CT/CP) and score-distillation methods (OneDP-S).

The complete mapping between the methods discussed in Section 2 of the original submission and our experimental evaluation in Tables 1–3 is summarized below:

Category in Related Work (Sec. 2)Method & CitationRole in ManuscriptEvaluated in Original Submission?
Fast Samplers (Iterative Background)DDIM [6] (ICLR 2021); DPM-Solver [7] (NeurIPS 2022); EDM [9] (NeurIPS 2022)Multi-step ODE/SDE fast samplers (NFE>1\text{NFE} > 1) cited for background contextEDM [9] evaluated as Teacher TET_E (Tables 1–3); DDIM/DPM-Solver cited as multi-step background
Early Step-Halving DistillationProgressive Distillation [8] (ICLR 2022)Earlier iterative step-halving distillation frameworkNot evaluated (older 2022 method superseded by consistency and 1-step distillation baselines below)
General One-Step Consistency ModelsCD [1] (ICML 2023); CT [2] (ICLR 2024)Representative 1-step consistency distillation & training frameworksYes (Tables 1–3)
Robotic One-Step Diffusion DistillationConsistency Policy (CP) [3] (RSS 2024); OneDP-S [4] (ICML 2025)Representative 1-step visuomotor consistency & score-difference distillation policiesYes (Tables 1–3, Table 13)
Koopman Diffusion DistillationKDM & KDM-F [5] (NeurIPS 2025)Representative 1-step static Koopman latent distillation modelsYes (Tables 1–3)

For detailed quantitative analyses and discussion of the newly added 2026 one-step methods (RACTD [10] and MVP [11]), please refer to our response to Weakness 3 (W3) below.

References:

[1] Y. Song, P. Dhariwal, M. Chen, I. Sutskever. "Consistency Models." ICML, 2023.

[2] Y. Song, P. Dhariwal. "Improved Techniques for Training Consistency Models." ICLR, 2024.

[3] A. Prasad, K. Lin, J. Wu, L. Zhou, J. Bohg. "Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation." RSS, 2024.

[4] Z. Wang et al. "One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation." ICML, 2025.

[5] N. Berman, I. Naiman, M. Eliasof, H. Zisling, O. Azencot. "One-Step Offline Distillation of Diffusion-Based Models via Koopman Modeling." NeurIPS, 2025.

[6] J. Song, C. Meng, S. Ermon. "Denoising Diffusion Implicit Models." ICLR, 2021.

[7] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, J. Zhu. "DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps." NeurIPS, 2022.

[8] T. Salimans, J. Ho. "Progressive Distillation for Fast Sampling of Diffusion Models." ICLR, 2022.

[9] T. Karras, M. Aittala, T. Aila, S. Laine. "Elucidating the Design Space of Diffusion-Based Generative Models." NeurIPS, 2022.

[10] X. Duan, Y. K. He, F. Tajwar, R. Salakhutdinov, Z. Kolter, J. Schneider. "Accelerating Diffusion Planners in Offline RL via Reward-Aware Consistency Trajectory Distillation." ICLR, 2026.

[11] G. Zhan et al. "Mean Flow Policy with Instantaneous Velocity Constraint for One-Step Action Generation." ICLR, 2026.

AuthorsOpenReview ↗

Response Part2

W2. The conceptual distinction between the proposed method and existing one-step distillation methods (e.g., single-step TI2I models that are conditional) is unclear

Response: We thank the reviewer for this helpful clarification. We agree that the original wording was confusing. Existing one-step methods, including conditional single-step TI2I students, are already conditional; we do not claim otherwise.

Our intended distinction is therefore not conditioning versus no conditioning. Rather, it is architectural:

  • Prior one-step distillers: typically realize generation as a direct conditional map fθ(xT,c)f_\theta(x_T, c) with a large teacher-style backbone, where context enters as an input feature.
  • DNK: represents one-step denoising as a compact latent transition z+=V diag(γ(c)) V−1z,z^{+}=V\ \mathrm{diag}(\gamma(c))\ V^{-1}z, where context cc modulates the transition operator itself through modal gains γ(c)\gamma(c).

In the revision, we have rewritten the unclear sentence and clarified this point in Related Work as follows:

Although their distillation processes are trained with diffusion states or teacher-generated targets, they typically retain large teacher-style backbones and realize one-step generation as a direct noise-to-trajectory map. This reduces the number of denoising evaluations, but leaves substantial deployed capacity and does not represent denoising as a compact latent transition whose dynamics vary with context. Such structure is important for robot control, where the noisy-to-denoised mapping can change with observations, contact conditions, and motion regimes. Our work addresses this gap by introducing context dependence directly into a structured latent transition of the one-step student.

AuthorsOpenReview ↗

Response Part 3

W3. All results comparisons are against older weaker baselines. No comparisons to the latest state-of-the-art robot control methods.

Response: We clarify that the original submission already included OneDP-S [1] (ICML 2025), a recent one-step diffusion-distillation method for robot control trained on the exact same DDPM teacher (TDT_D) as DNK, alongside KDM / KDM-F [2] (NeurIPS 2025).

To further strengthen our evaluation against the latest state-of-the-art methods, the revised manuscript adds two recently released one-step generative policies: RACTD [3] (ICLR 2026) for diffusion distillation and Mean Velocity Policy (MVP) [4] (ICLR 2026 Oral) for flow matching. While standard flow-matching policies require iterative multi-step ODE integration (NFE>1\text{NFE} > 1) and fall outside our one-step scope, MVP enforces an instantaneous velocity constraint on the mean velocity field for true single-step (NFE=1\text{NFE} = 1) action generation. Together with RACTD (reward-aware consistency trajectory distillation), these represent the most up-to-date one-step methods across both diffusion-distillation and flow-matching paradigms.

The table below summarizes the offline teacher-distillation results across all benchmarks (NFE=1\text{NFE}=1; see Section 5.2, Tables 1–3, and Appendix D.8, Table 14 for MVP):

BenchmarkMetricOneDP-S [1] (ICML'25, TDT_D)RACTD [3] (ICLR'26, TET_E)DNK (Ours) (TDT_D)
HalfCheetahScore ↑ / Latency (ms) ↓0.748 ± 0.030 / 2.030.918 ± 0.006 / 1.090.873 ± 0.007 / 0.68
Walker2dScore ↑ / Latency (ms) ↓1.105 ± 0.000 / 2.031.067 ± 0.004 / 1.111.095 ± 0.001 / 0.60
Push-T (State)Score ↑ / Latency (ms) ↓ / Params (M) ↓0.923 ± 0.021 / 6.162 / 68.880.706 ± 0.032 / 6.392 / 64.960.931 ± 0.037 / 1.283 / 2.37
Push-T (Image)Score ↑ / Latency (ms) ↓ / Params (M) ↓0.825 ± 0.013 / 7.053 / 80.390.107 ± 0.010 / 7.579 / 77.910.853 ± 0.041 / 1.780 / 13.93
Franka KitchenScore ↑ / Latency (ms) ↓ / Params (M) ↓0.975 ± 0.013 / 6.434 / 68.920.970 ± 0.010 / 6.569 / 66.550.973 ± 0.016 / 1.598 / 2.46

As detailed in Section 5.2, on Franka Kitchen, DNK achieves 0.973 ± 0.016 vs. 0.970 ± 0.010 for RACTD, while using 27× fewer deployed parameters (2.46M vs. 66.55M) and achieving a 4.11× inference speedup. On locomotion, RACTD uses an auxiliary return-to-go (RTG) reward module while sharing the EDM teacher (TET_E) and backbone of CP/CD, attaining 0.918 ± 0.006 on HalfCheetah and 1.067 ± 0.004 on Walker2d. Unlike our reward-free distillation, this RTG objective biases generation toward high-return modes, boosting RACTD's HalfCheetah return 9.0% above its teacher TET_E (0.842). Even without reward supervision, DNK outperforms RACTD by 2.6% on Walker2d (1.095 vs. 1.067) while reducing latency by 37.6% on HalfCheetah and 45.9% on Walker2d. On Push-T, despite its large backbone, RACTD underperforms in state-based control (0.706 ± 0.032) and struggles in image-based control (0.107 ± 0.010), consistent with a domain mismatch between offline RL return guidance and contact-rich imitation. Finally, because MVP follows an offline-to-online RL protocol rather than pure offline distillation, we report it separately in Appendix D.8 (Table 14) on state-based Push-T and Franka Kitchen: DNK achieves substantially higher scores (0.931 ± 0.037 vs. 0.758 ± 0.047 and 0.973 ± 0.016 vs. 0.510 ± 0.017) with roughly half the parameters (2.37M vs. 4.83M and 2.46M vs. 5.20M), while MVP has lower latency (0.617 ms and 0.765 ms).

References:

[1] Z. Wang et al. "One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation." ICML, 2025.

[2] N. Berman et al. "One-Step Offline Distillation of Diffusion-Based Models via Koopman Modeling." NeurIPS, 2025.

[3] X. Duan et al. "Accelerating Diffusion Planners in Offline RL via Reward-Aware Consistency Trajectory Distillation." ICLR, 2026.

[4] G. Zhan et al. "Mean Flow Policy with Instantaneous Velocity Constraint for One-Step Action Generation." ICLR, 2026.

Comments on the paper

2 comments

We sincerely thank the reviewers for their constructive comments and valuable suggestions. We have revised the manuscript accordingly, and all changes are highlighted in blue in the revised PDF. For further and more specific changes made, please refer to the detailed responses to the reviewers below as well as the duly revised manuscript.

AuthorsOpenReview ↗

General Response and Summary of Revisions

Dear Area Chair and Reviewers,

We thank Reviewers TShe, XDg9, and SHaS for their constructive feedback. To help navigate our responses and revised manuscript (changes in blue), we summarize our main experimental additions and clarifications below.

1. One-Step Policy Coverage & New ICLR 2026 Baselines (TShe-W1/W3, XDg9-W2, SHaS-W1/Q2)

We clarified how baselines from each one-step policy family in Sec. 2 are evaluated, and added two newly released ICLR 2026 baselines covering diffusion distillation (RACTD) and 1-step flow matching (MVP):

Method Family (Sec. 2)Representative BaselinesLocation in Paper
Koopman distillationKDM, KDM-F (NeurIPS'25)Tables 1–3 (Sec. 5.2)
Consistency modelsCD(ICLR'23), CT (ICLR'24), CP (RSS'24)Tables 1–3 (Sec. 5.2)
Score-diff. distillationOneDP-S (ICML'25)Tables 1–3, Table 13
Reward-aware distillationRACTD (ICLR'26) (New)Tables 1–3 (pp. 7–9)
1-step flow matchingMVP (ICLR'26 Oral) (New)App. D.8, Table 14
  • RACTD (ICLR'26, Tables 1–3): Without auxiliary reward losses during distillation, DNK outperforms RACTD on Push-T State (0.931 vs. 0.706), Push-T Image (0.853 vs. 0.107), and Walker2d (1.095 vs. 1.067), and matches it on Kitchen (0.973 vs. 0.970) with 27× fewer parameters (2.46M vs. 66.55M) and 4.11× lower latency (1.60ms vs. 6.57ms).
  • MVP (ICLR'26 Oral, App. D.8, Table 14): Because MVP follows an offline-to-online RL protocol rather than pure offline distillation, we report it separately in App. D.8 (Table 14) on state-based Push-T and Franka Kitchen: DNK achieves substantially higher scores (0.931 ± 0.037 vs. 0.758 ± 0.047 and 0.973 ± 0.016 vs. 0.510 ± 0.017) with roughly half the parameters (2.37M vs. 4.83M and 2.46M vs. 5.20M), while MVP has lower latency (0.617 ms and 0.765 ms vs. 1.283 ms and 1.598 ms).
  • VLA Discussion (TShe Follow-up): Added a discussion in Sec. 6 (p. 10) on extending DNK to VLA models.

2. Isolating the FDK Parameterization (XDg9-W1, SHaS-W2/Q1, TShe-W2)

To isolate the shared-eigenbasis inverse constraint in K(c)=Vdiag(γ(c))V−1\mathbf{K}(\mathbf{c}) = \mathbf{V}\mathrm{diag}(\gamma(\mathbf{c}))\mathbf{V}^{-1} from context modulation or capacity, we added structural ablations under matched capacity and 3×50 evaluations (Exact FDK achieves 0.973 ± 0.016 on Kitchen, 0.931 ± 0.037 on Push-T State, and 0.853 ± 0.041 on Push-T Image):

  1. Unconstrained Two-Sided (App. D.3, Table 10): Using W2diag(γ(c))W1\mathbf{W}_2 \mathrm{diag}(\gamma(\mathbf{c})) \mathbf{W}_1 without W2=W1−1\mathbf{W}_2 = \mathbf{W}_1^{-1} drops scores to 0.883 ± 0.018 on Kitchen (−9.0%) and 0.819 ± 0.129 on Push-T State (−11.2%).
  2. Capacity-Matched Conditional MLP (App. D.4, Table 11): Replacing FDK with a nonlinear conditional MLP of matched parameters drops scores to 0.910 ± 0.017 on Kitchen (−6.3%), 0.869 ± 0.048 on Push-T State (−6.2%), and 0.762 ± 0.056 on Push-T Image (−9.1%).
  3. Static vs. Conditional Gains (App. D.2, Table 9): Removing context conditioning γ(c)\gamma(\mathbf{c}) drops scores to 0.910 ± 0.005 on Kitchen and 0.857 ± 0.088 on Push-T State (and 0.738 / 0.563 for Identity, Table 8).

These results validate the benefit of the inverse-linked eigenbasis parameterization. We also clarified the distinction from direct conditional generators in Sec. 2 (p. 3) (TShe-W2) and the theoretical scope in App. A (pp. 13–15) (XDg9-W1, SHaS-W2).

3. Additional Diagnostics & Protocol Clarifications (XDg9, SHaS)

  • Spectral Diagnostics (XDg9-Q2, App. B.2, Table 5): Reported κ2(V)∈[4.41,19.98]\kappa_2(\mathbf{V}) \in [4.41, 19.98] and modal gains (P95 ∣γω∣∈[0.85,1.18]|\gamma_\omega| \in [0.85, 1.18]) across all tasks, confirming well-conditioned torch.linalg.solve training, and clarified regularized vs. empirically verified factors in Prop. 2.
  • Candidate Budget (XDg9-Q4): Evaluated DNK and the DDPM teacher across Ncand∈{64,32,16,1}N_{\mathrm{cand}} \in \{64, 32, 16, 1\}, showing stability down to Ncand=16N_{\mathrm{cand}}=16 (1.0913 ± 0.0039 Walker2d; 0.9051 ± 0.0307 HalfCheetah) and confirming that bimodal medium-expert datasets require candidate scoring for both teacher and student (while Push-T and Kitchen use Ncand=1N_{\mathrm{cand}}=1).
  • Latency & Reduced-Width OneDP-S (XDg9-Q3/W2, Tables 8 & 13): Unified CUDA-synchronized end-to-end timing across Tables 2–3 and 8 (1.283ms Push-T, 1.598ms Kitchen) and clarified teacher initialization in the reduced-width OneDP-S test.
  • Targets & Hardware Scope (XDg9-Q1/Q5, SHaS-W1/Q2, App. C & D.6): Clarified noise-conditioned student inputs u~0\tilde{\mathbf{u}}_0, noted the qualitative nature of the PCA plot (Fig. 8), clarified the complementary roles of simulation and Kinova hardware validation, removed the 100s metric from Table 7, and detailed the 50 held-out ID test scenes and clearance metrics.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.