- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a structured latent dynamics learning framework based on differential flatness for prediction and control from incomplete observations. The main motivation is that existing latent dynamics models focus more on prediction accuracy, while the learned representations may not contain enough information for recovering the physical states and control inputs. Accordingly, the authors introduce a flat-jet encoder that extracts flat outputs and their derivatives from observation histories. The latent dynamics are then modeled using a predefined derivative-chain structure, with a neural network learning the unknown highest-order dynamics. Based on this representation, the authors also develop a trajectory planning and feedback control method, where physical control inputs are recovered through an auxiliary network.
Critical strengths
-
I think the main idea of using differential flatness to guide latent representation learning is interesting. Compared with standard prediction-based latent models, the proposed method explicitly connects the latent coordinates with physical states and control inputs. This is particularly useful under incomplete observations, where a good observation prediction does not necessarily mean good control performance.
-
The experimental improvements are quite significant, especially with limited training data. For example, in Table 2, under incomplete observations and 15% training data, the prediction error decreases from 0.988 for Deep Koopman to 0.281 for the proposed method. The image-based experiments also demonstrate better terminal control accuracy. I also appreciate the ablation study in Table 3, which shows that both the flat-jet encoder and structured latent dynamics contribute to the performance improvements.
Critical weaknesses
-
The theoretical guarantee does not fully extend to the actual learned control model. My main concern is that Proposition 1 and Corollary 1 assume the learned representation can exactly recover the true flat output and its derivatives. However, the proposed method only encourages this property through its training objective, without guaranteeing that the learned coordinates satisfy it. More importantly, the control input is recovered through a separately trained neural network , as described in Appendix A.9. There is no explicit guarantee that this network correctly inverts the learned dynamics, especially when the desired latent input is outside the training distribution. Since physical input recovery is an important contribution of this paper, I think the authors should provide more direct validation of its accuracy and reliability, rather than relying mainly on the final control performance.
-
The image-based experiments are conducted on synthetic environments with clean backgrounds and no added observation noise. However, the proposed flat-jet encoder relies on numerical differentiation of observations, including third-order derivatives for the crane system. Such differentiation can be very sensitive to measurement noise, which may affect both state estimation and feedback control. Another concern is the computational time reported in Table 4. The proposed feedback controller requires 130.54 ms for the bicopter and 147.45 ms for the crane, while the control interval is only 50 ms. Appendix A.11.4 mentions that the simulator is paused until computation finishes. Therefore, I am not fully convinced that the reported closed-loop performance can be maintained under actual real-time execution.
-
Although the paper claims to learn differentially flat latent representations from observations, the method assumes that the underlying system is differentially flat and that the differential order is already known. This seems a lot to ask for in reality. For example, the authors explicitly set for the nonlinear system and bicopter, and for the crane. This information is obtained from the analytical system dynamics, rather than learned from observations. Moreover, all evaluated systems are selected such that their flat outputs can be determined from the available observations. I think these assumptions are quite strong, especially for systems whose physical dynamics are unknown. It is unclear how the proposed method would work if the differential order is incorrectly specified, or if the system only approximately satisfies the assumed flatness structure. This limits the general applicability of the proposed framework.
-
The experiments may not fully satisfy the regularity conditions required by the theoretical results. In the three-dimensional nonlinear system, Appendix A.11.1 shows that the state-to-flat-jet mapping is only locally invertible when . More importantly, the state reconstruction in Eq. (66) depends on the branch . However, Appendix A.11.2 states that the initial states are uniformly sampled from , which includes states on both sides of this singularity. The paper does not clearly explain whether the trajectories are restricted to a single regular domain or how the learned representation handles different reconstruction branches. Since the flat-jet representation alone may not uniquely recover the physical state across these branches, I think this is an important issue for the theoretical interpretation of the experimental results. The authors should clarify the data generation and whether the claimed state-recovery property actually holds over the evaluated state space.
Questions for authors
-
In Appendix A.9, the input-recovery network is trained separately through supervised regression. How well does the learned network satisfy the inverse relationship between the highest-order dynamics and physical inputs? Can the authors report input-recovery errors on unseen trajectories, especially for latent commands generated by the feedback controller?
-
How sensitive is the proposed method to observation noise and sampling frequency? Since the flat-jet encoder estimates higher-order derivatives from consecutive observations, I wonder whether relatively small visual noise would significantly affect the learned latent states and control performance.
-
In Appendix A.11.2, the bicopter images use a viewing window that is centered and scaled separately for each trajectory. How is a consistent physical flat output recovered across trajectories when the camera transformation changes? Does the model receive any camera calibration information? This seems relevant to the observation sufficiency condition in Definition 1.
-
Table 4 reports feedback computation times above the 50 ms control interval. Could the authors clarify whether the proposed controller can maintain its performance when computation and simulation are executed in real time, without pausing the simulator?