When Dynamics Are Suboptimal: Isolating Dynamics Inductive Bias in Representation Learning

In many dynamical models, representations and dynamics are learned jointly, and the chosen dynamics model class and fitting objective thereby impose an inductive bias on representation learning. At a given training iterate, the dynamics parameters can be suboptimal for the representation under the chosen fitting criterion, so the induced representation gradient can differ from the corresponding gradient under a fitted dynamics reference. We theoretically characterize this representation-gradient discrepancy. In the canonical case where the dynamics-induced representation gradient is obtained by differentiating the dynamics-fitting objective with respect to the representation parameters, the discrepancy is exactly the gradient of the dynamics-parameter suboptimality gap. Controlled experiments in a sequential latent-variable model show that dynamics-parameter suboptimality alters representation formation, with consequences for learned dynamical quality. The effect is most pronounced during early representation formation, motivating Asymmetric Coupling Warm-up (ACW), a lightweight intervention that temporarily attenuates dynamics-to-representation coupling early in training. Applying ACW across distinct representation–dynamics paradigms yields consistent changes in optimization trajectories and downstream outcomes, providing cross-paradigm evidence that the identified mechanism has observable consequences beyond the controlled setting.