- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Dynamic Neural Koopman (DNK) distillation to accelerate diffusion-based robot control by approximating multistep reverse diffusion with a single latent transition. Its core component, the Factorized Dynamic Koopman (FDK) layer, parameterizes the latent operator as , where a shared modal basis is combined with context-dependent modal gains. This introduces observation-dependent adaptation while maintaining a compact operator parameterization. The student is trained using offline teacher-generated trajectory pairs, latent alignment, control-oriented supervision, and spectral regularization. The method is evaluated on D4RL locomotion, state- and image-based Push-T, Franka Kitchen, and a physical Kinova Gen3 obstacle-reaching task. The results show competitive closed-loop performance with substantially lower inference latency than multistep diffusion teachers, favorable performance–latency–capacity trade-offs relative to high-capacity one-step baselines, and substantial improvements over static Koopman distillation baselines.
Critical strengths
-
The paper is clearly written, well organized, and well motivated. The proposed FDK layer is simple and intuitive, yet effective in adapting latent transitions to different observation contexts. The ablation studies provide good empirical support for the main claim regarding context-dependent modal adaptation. This strengthens my assessment of the paper's technical soundness, as the proposed design is clearly motivated and its central claim is supported by targeted experiments.
-
The experimental evaluation is reasonably comprehensive, covering locomotion, state-based and image-based contact-rich manipulation, and long-horizon manipulation. The results demonstrate a particularly competitive trade-off among closed-loop performance, inference latency, and model capacity. This is a significant strength because it establishes the practical value of the method across different simulated control settings, particularly for computationally constrained robotic deployment.
Critical weaknesses
-
The hardware evaluation is limited to a single state/goal/obstacle-conditioned reaching task and compares DNK only against the multistep diffusion teacher, rather than other strong one-step policies. While the results convincingly demonstrate the benefit of low-latency inference in this setting, they provide limited evidence for broader real-world applicability or superiority over existing one-step approaches. This limits my confidence in how well the reported advantages generalize to more diverse real-world robotic tasks and whether they are specific to DNK rather than one-step inference in general.
-
Although the ablations demonstrate the benefit of context-dependent modal gains over static gains, they do not isolate the contribution of the specific shared-eigenspace constraint imposed by the FDK factorization . Comparisons against alternative conditional linear parameterizations would help clarify whether this structural constraint is essential to the observed improvements. Furthermore, the theoretical analysis mainly establishes the advantage of conditional over constant gains under a fixed latent representation and derives spectral properties of the chosen factorization, providing relatively limited justification for the Koopman-specific inductive bias. This leaves some uncertainty about whether the observed improvements arise from the particular FDK structure or from context-dependent latent modulation more generally, which limits my assessment of the distinct methodological contribution.
Questions for authors
-
On the specific contribution of the FDK factorization: The proposed FDK operator constrains the context-dependent operators to share a common eigenspace and to commute with one another. While the current ablations demonstrate the effectiveness of context-dependent modal gains over static gains, they do not isolate the importance of this structural constraint. Have you evaluated or considered a more general two-sided diagonal factorization, such as , where and are independently learned, under comparable parameter and computational budgets? Such a comparison would help clarify whether the observed gains primarily arise from context-dependent modulation or from the shared-eigenspace structure itself.
-
On the scope of the hardware comparison: The real-world experiment demonstrates a substantial latency benefit on the Kinova obstacle-reaching task, but the hardware comparison is limited to the multistep diffusion teacher. Have you evaluated any strong one-step baseline, such as OneDP-S or Consistency Policy, under the same hardware setup? If not, could you clarify the main practical limitations preventing such a comparison? This would help assess whether the reported real-world advantage is specific to DNK or primarily reflects the general benefit of one-step inference over multistep diffusion sampling.
7 comments
Response Part 2
W2: Lack of comparisons against alternative conditional linear parameterizations to isolate the shared-eigenspace FDK constraint () from general context-dependent latent modulation, and limited theoretical justification for the Koopman-specific inductive bias.
Response:
The shared-eigenspace restriction provides three concrete structural benefits over unconstrained conditional maps: spectral interpretability, parameter efficiency, and exact inverse consistency. Because all context-dependent operators share a common eigenbasis with exact inverse , the entries of correspond to true modal eigenvalues, which makes our spectral-radius regularizer mathematically well-defined and requires learning only a single basis matrix .
To directly test whether the empirical gains arise from this inverse-linked shared-eigenspace structure rather than from context-dependent latent modulation in general, we added two controlled ablations under the exact same training recipe and formal evaluation protocol, replacing only the latent transition operator (Appendix D.3, Table 10 and Appendix D.4, Table 11):
- Unconstrained Two-Sided Diagonal Factorization (, Appendix D.3, Table 10): Here and are learned independently without enforcing .
- Capacity-Matched Conditional MLP (, Appendix D.4, Table 11): We replace the FDK operator with an unconstrained nonlinear conditional MLP whose parameter count is matched to Exact FDK within parameters ( on Franka Kitchen and on state-based Push-T).
| Task | Exact FDK () | Two-Sided () | Capacity-Matched Cond. MLP |
|---|---|---|---|
| Franka Kitchen | 0.973 ± 0.016 | 0.883 ± 0.018 | 0.910 ± 0.017 |
| Push-T (state) | 0.931 ± 0.037 | 0.819 ± 0.129 | 0.869 ± 0.048 |
Across both evaluated benchmarks, Exact FDK consistently outperforms both the unconstrained two-sided factorization ( vs. on Franka Kitchen; vs. on state-based Push-T, with substantially lower rollout variance) and the capacity-matched conditional MLP ( vs. on Franka Kitchen; vs. on state-based Push-T). Together with our identity and static-gain ablations (Appendix D.1, Table 8 and Appendix D.2, Table 9), these results demonstrate that the performance gains do not merely come from "any conditional latent map," but specifically benefit from the inverse-linked shared-basis inductive bias.
Finally, regarding the theoretical analysis (Section 4.2 and Appendix A), we clarify that FDK is Koopman-inspired: Proposition 1 (Appendix A.1) formally establishes the strict population-risk advantage of context-dependent modal gains over static gains under a fixed latent representation and shared basis, while Proposition 2 and Corollary 1 (Appendix A.2) derive the exact spectral structure and fixed-context output sensitivity bounds for that justify our spectral regularizer (supported empirically by the basis conditioning diagnostics in Appendix B.2, Table 5), rather than claiming recovery of true underlying diffusion SDE dynamics or a standalone proof of closed-loop stability.
Response Part 3
Q1: Whether a more general two-sided diagonal factorization (with independently learned ) has been evaluated under comparable parameter and compute budgets to isolate the shared-eigenspace constraint.
Response:
Following your suggestion, we directly evaluated the unconstrained two-sided factorization with independently learned and under matched experimental settings (Appendix D.3, Table 10). As detailed in our response to Weakness 2 above, the shared-eigenspace restriction provides three structural benefits over : spectral interpretability (entries of are true eigenvalues, enabling direct spectral-radius regularization), parameter efficiency (learning a single basis matrix rather than two unconstrained matrices, vs. on Franka Kitchen and vs. on Push-T), and exact inverse consistency by construction. Empirically, Exact FDK achieves substantially higher closed-loop scores than on both Franka Kitchen ( vs. ) and state-based Push-T ( vs. ). Together with the capacity-matched conditional MLP ablation in Appendix D.4 (Table 11), which replaces the factorized linear operator with an unconstrained nonlinear network, these results confirm the effectiveness of the shared-eigenbasis FDK structure.
Response Part 1
W1: The hardware evaluation is limited to a single state/goal/obstacle-conditioned reaching task and compares DNK only against the multistep diffusion teacher, rather than other strong one-step policies. While the results demonstrate the benefit of low-latency inference in this setting, they provide limited evidence for broader real-world applicability or whether the reported hardware advantages are specific to DNK rather than one-step inference in general.
Response:
We did not evaluate competing one-step baselines (such as OneDP-S or Consistency Policy) on the physical Kinova platform. There is no architectural or hardware barrier preventing their deployment under the same setup; rather, our physical evaluation budget was constrained by robot access time and operational manpower (as each evaluated policy requires dedicated teacher/student training on the custom B-spline scene distribution followed by 50 physical trials under safety supervision).
Instead, our experimental design separates broad cross-method benchmarking from physical closed-loop validation:
We have revised the introduction to Section 5 (Experiments), Section 5.4 (Real-World Experiment), and Appendix C (Real-World Experiments) to explicitly state these complementary roles of simulation benchmarking and physical validation, and to clarify the scope of the hardware conclusions.