acceptodds
Under review as a conference paper at ICLR 2027

From Observation Prediction to Control Coordinates: Learning Differentially Flat Latent Representations

Abstract

Latent dynamics models enable control of systems from high-dimensional observations. Existing methods primarily emphasize predictive accuracy, while control depends on the relationships among the learned representation, the underlying physical state, and the control input. Accurate observation-level prediction alone may be insufficient for effective control, particularly when observations do not uniquely determine the physical state. We address this gap by introducing differential flatness as a structural bias. For differentially flat systems, we establish a sufficient observation condition: observations need only distinguish states with distinct flat outputs, whose finite-order derivatives enable state and input recovery. The model combines a flat-jet encoder and structured latent dynamics, trained jointly without supervision on the true flat coordinates. This representation supports direct trajectory planning in learned flat coordinates and translation into control inputs. Experiments on an analytically tractable flat system and image-based differentially flat systems under incomplete observations show that our model learns coordinates with differential flatness properties, improves prediction and control accuracy over prediction-centric baselines, particularly with limited data, and reduces computation for closed-loop control.

Primary area:
learning on time series and dynamical systems
Keywords:
Latent Dynamics Learning, Differential Flatness, Representation Learning for Control, Trajectory Planning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4GC5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a structured latent dynamics learning framework based on differential flatness for prediction and control from incomplete observations. The main motivation is that existing latent dynamics models focus more on prediction accuracy, while the learned representations may not contain enough information for recovering the physical states and control inputs. Accordingly, the authors introduce a flat-jet encoder that extracts flat outputs and their derivatives from observation histories. The latent dynamics are then modeled using a predefined derivative-chain structure, with a neural network learning the unknown highest-order dynamics. Based on this representation, the authors also develop a trajectory planning and feedback control method, where physical control inputs are recovered through an auxiliary network.

Critical strengths
  1. I think the main idea of using differential flatness to guide latent representation learning is interesting. Compared with standard prediction-based latent models, the proposed method explicitly connects the latent coordinates with physical states and control inputs. This is particularly useful under incomplete observations, where a good observation prediction does not necessarily mean good control performance.

  2. The experimental improvements are quite significant, especially with limited training data. For example, in Table 2, under incomplete observations and 15% training data, the prediction error decreases from 0.988 for Deep Koopman to 0.281 for the proposed method. The image-based experiments also demonstrate better terminal control accuracy. I also appreciate the ablation study in Table 3, which shows that both the flat-jet encoder and structured latent dynamics contribute to the performance improvements.

Critical weaknesses
  1. The theoretical guarantee does not fully extend to the actual learned control model. My main concern is that Proposition 1 and Corollary 1 assume the learned representation can exactly recover the true flat output and its derivatives. However, the proposed method only encourages this property through its training objective, without guaranteeing that the learned coordinates satisfy it. More importantly, the control input is recovered through a separately trained neural network RuR_u, as described in Appendix A.9. There is no explicit guarantee that this network correctly inverts the learned dynamics, especially when the desired latent input is outside the training distribution. Since physical input recovery is an important contribution of this paper, I think the authors should provide more direct validation of its accuracy and reliability, rather than relying mainly on the final control performance.

  2. The image-based experiments are conducted on synthetic environments with clean backgrounds and no added observation noise. However, the proposed flat-jet encoder relies on numerical differentiation of observations, including third-order derivatives for the crane system. Such differentiation can be very sensitive to measurement noise, which may affect both state estimation and feedback control. Another concern is the computational time reported in Table 4. The proposed feedback controller requires 130.54 ms for the bicopter and 147.45 ms for the crane, while the control interval is only 50 ms. Appendix A.11.4 mentions that the simulator is paused until computation finishes. Therefore, I am not fully convinced that the reported closed-loop performance can be maintained under actual real-time execution.

  3. Although the paper claims to learn differentially flat latent representations from observations, the method assumes that the underlying system is differentially flat and that the differential order rr is already known. This seems a lot to ask for in reality. For example, the authors explicitly set r=3r=3 for the nonlinear system and bicopter, and r=4r=4 for the crane. This information is obtained from the analytical system dynamics, rather than learned from observations. Moreover, all evaluated systems are selected such that their flat outputs can be determined from the available observations. I think these assumptions are quite strong, especially for systems whose physical dynamics are unknown. It is unclear how the proposed method would work if the differential order is incorrectly specified, or if the system only approximately satisfies the assumed flatness structure. This limits the general applicability of the proposed framework.

  4. The experiments may not fully satisfy the regularity conditions required by the theoretical results. In the three-dimensional nonlinear system, Appendix A.11.1 shows that the state-to-flat-jet mapping is only locally invertible when x2≠−1x_2 \neq -1. More importantly, the state reconstruction in Eq. (66) depends on the branch σ=sign(1+x2)\sigma=\mathrm{sign}(1+x_2). However, Appendix A.11.2 states that the initial states are uniformly sampled from [−2,2]3[-2,2]^3, which includes states on both sides of this singularity. The paper does not clearly explain whether the trajectories are restricted to a single regular domain or how the learned representation handles different reconstruction branches. Since the flat-jet representation alone may not uniquely recover the physical state across these branches, I think this is an important issue for the theoretical interpretation of the experimental results. The authors should clarify the data generation and whether the claimed state-recovery property actually holds over the evaluated state space.

Questions for authors
  1. In Appendix A.9, the input-recovery network is trained separately through supervised regression. How well does the learned network satisfy the inverse relationship between the highest-order dynamics and physical inputs? Can the authors report input-recovery errors on unseen trajectories, especially for latent commands generated by the feedback controller?

  2. How sensitive is the proposed method to observation noise and sampling frequency? Since the flat-jet encoder estimates higher-order derivatives from consecutive observations, I wonder whether relatively small visual noise would significantly affect the learned latent states and control performance.

  3. In Appendix A.11.2, the bicopter images use a viewing window that is centered and scaled separately for each trajectory. How is a consistent physical flat output recovered across trajectories when the camera transformation changes? Does the model receive any camera calibration information? This seems relevant to the observation sufficiency condition in Definition 1.

  4. Table 4 reports feedback computation times above the 50 ms control interval. Could the authors clarify whether the proposed controller can maintain its performance when computation and simulation are executed in real time, without pausing the simulator?

Reviewer_dBd1OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for learning differentially flat latent representations from incomplete observations. It combines a flat-jet encoder with structured latent dynamics, enabling trajectory planning and feedback control. Experiments on a nonlinear system and two image-based control environments demonstrate promising prediction and control performance, particularly with limited training data.

Critical strengths

Strengths

  1. The paper studies an interesting problem: learning latent representations that are not only predictive but also useful for control. Incorporating differential flatness as an inductive bias is a reasonable and potentially useful direction. The proposed architecture is clearly motivated, and the connection between the learned representation and trajectory planning is appealing.

  2. The experimental results are also encouraging, especially under incomplete observations and limited training data. The ablation study provides some evidence that both the flat-jet encoder and the structured latent dynamics contribute to learning the desired representation.

Critical weaknesses

Weaknesses

  1. Although the physical dynamics are unknown, the method requires substantial prior knowledge of the underlying system structure. In particular, Eq. (4) assumes a consecutive-order flatness parameterization, while Eq. (6) requires a known derivative order rr, such that the highest-order derivative satisfies ν=y(r)=G(ξ,u)\nu=y^{(r)}=G(\xi,u), with a suitable inverse mapping to recover the physical input. This is more restrictive than differential flatness in general. While Appendix A.2 discusses dynamic extension, determining the appropriate extension itself requires knowledge of the relevant derivative orders and input structure. In a black-box setting, it is unclear how to determine which inputs should be extended and how many times, or how to select the correct rr. Since these assumptions directly enable the proposed representation and control design, their practical availability and the scope of applicable systems deserve further discussion.

  2. First, the suitability of the Deep Koopman baseline under incomplete observations is unclear. Standard Koopman-based system identification and control typically construct lifted representations from physical states or suitable state estimates. While the comparison under o(1)o^{(1)} is consistent with this setting, o(2)=(x1,x2,x32)o^{(2)}=(x_1,x_2,x_3^2) does not uniquely determine the physical state or its future evolution. The proposed method explicitly exploits the assumed flatness structure to recover missing dynamical information. This is a meaningful advantage, but it is unclear whether the Deep Koopman baseline is appropriately adapted to this observation setting.

  3. The reported computational efficiency is difficult to interpret. In Table 4, 'Ours (MPC)' is approximately an order of magnitude faster than Deep Koopman MPC, despite retaining nonlinear learned dynamics while the latter employs linear latent dynamics. Moreover, 'Ours (MPC)' is considerably faster than 'Ours (feedback)', although the feedback controller avoids repeated online optimization. Further details on the MPC formulations, solver implementations, and computation-time breakdown would help explain these results.

  4. While the authors acknowledge that the theoretical results establish structural sufficiency rather than learnability, the proposed controller relies on the learned input-recovery map accurately realizing desired virtual inputs. Although the reported experiments are encouraging, additional evaluation of the inverse consistency between GθG_\theta and RuR_u, particularly for planned virtual inputs, would strengthen the connection between the learned representation and the control guarantees.

Questions for authors
  1. How can the required derivative order rr and consecutive-order structure be determined when the dynamics are unknown, particularly when dynamic extension is necessary? How sensitive is the method to an incorrect choice of rr?

  2. How is the Deep Koopman baseline constructed under the incomplete observation o(2)o^{(2)}, and how does it address the missing state information?

  3. What explains the substantial computational differences between Deep Koopman MPC, 'Ours (MPC)', and 'Ours (feedback)'?

Reviewer_PZrtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a flatness based VAE model for learning flat coordinates and latent dynamics from observations. They use the differential flatness idea from prior literature as a structural bias worthy of introducing into the task of learning representations. Control is planned directly in the learned flat coordinates. The paper is largely empirical, and it reports experiments on a 3D system with a known flat output, and on a bicopter and an overhead crane learned from images, reporting better prediction than some alternatives, possibly with a better sample complexity (empirical only).

Critical strengths
  • The paper is fairly cleanly written
Critical weaknesses
  • The net novelty seems unclear. For instance, there's this work on "NewtonianVAE" that is missing from the discussion, and it seems to match the current paper in its design, albeit just the r=2 case. Positioning vis-a-vis this work is missing.

  • The references are a bit improper (some of them are repeated twice). Did the authors do a rigorous check to eliminate hallucinations in references too?

Questions for authors
  • What about the current work's relation to observability theory?
  • Is assuming a single global flat encoder practically reasonable?
  • Does the bicopter experiment actually satisfy the paper's assumptions? Seems like not to be the case.
  • In terms of run times (see e.g. Table 4), is the overhead caused by the proposed approach justifiable?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.