Lagrangian–Hamiltonian Flows for Video Prediction and Image Generation: A Symplectic Perspective

We introduce LHFM, a geometric framework for learning image dynamics that combines Lagrangian image representations with Hamiltonian dynamics. These structures play a central role in classical mechanics, symplectic geometry, and geometric quantization. LHFM represents images as exact Lagrangian graphs and models their evolution through image-dependent Hamiltonian flows. This construction provides a geometric interpretation of image-space dynamics by connecting them to the Hamiltonian evolution of Lagrangian submanifolds. The framework supports both video prediction, our primary application, and image generation. We propose a novel deterministic video prediction method based on the Hamiltonian evolution of Lagrangian image representations. This video variant, LHFM-V, is a recurrent model that achieves the lowest reported FLOP count among the compared recurrent models with similar prediction accuracy. The image variant, LHFM-I, shows that the same construction is compatible with flow matching: in a matched experiment, it attains a lower FID than the flow-matching baseline.