This is a general formalism that has been tackled before. The paper spends significant time elaborating mathematical concepts that then go unused in the actual experiments in the paper, which reduce, basically, to a form of optical flow. This is disappointing, and makes the paper fairly incoherent. It builds the substantial (known) Jacobi-Hamiltonian mathematical framework. It then dives in with symplectic shorthand and undefined terms that are not helpful for the general reader (graph(dS) is not defined for one of many possible examples - yes I know what it means, but you should not believe that every reader has exactly that background - graph means something completely different in most other uses in ML). The maths and context is very unrelated to the actual target of the paper. For example "Whereas a classical phase-space point specifies position and momentum simultaneously, the Heisenberg uncertainty principle prevents a quantum state from being sharply localized in both variable". How is this relevant? Why did the authors even think to mention that? The maths feels like it is textbook reconstruct that does not really relate to the paper application or the audience. The paper then portrays the image as the graph of pixel values at location and applies the Hamiltonian flow to the pixel space image Lagrangian. Lemma 4.1 is presented as a contribution without reference, but it is a (specialisation?) of a standard result. The only new component of it is adding in the specific generating function. That this is used without reference to the underlying literature worries me. The introduction of the probe is extensively described, but is discarded before any model is built.
More importantly, why are we doing all this? What is the point? No Hamiltonian is ever constructed. The Hamiltonian is effectively time dependent and describes dynamics that fits explicitly to the data? What purpose or gain does all this serve?
At the end of the day is a large, rather disconnected, not always relevant, mathematical construct. Sadly most of that is then unused in the actual paper experiments. The experiments reduce to simple transport in pixel space. Applied to images sequences this is fairly useless (it cannot handle temporary occlusion for example), as the method cannot maintain latent information. The application ends up being a form of restricted optical flow. Sadly the paper feels like it is a large well known geometric construct built into a paper looking for a problem. But this large construct mostly collapses on application. Equation 7 is a brightness change equation. This has been used in optical flow methods for decades, and yet this receives no mention. It also feels like the theory constrains nothing. Proposition A.6 makes every path realizable (the probe drops out, and LHFM-I effectively can learn the same probability path as I-CFM).
Moving on to the practice, the experiments are hard to understand in the context of this paper. There is a huge disconnect and it is not clear exactly what is happening in any of the experiments and how they relate to the theory that is established. In image generation, ias described, we have a simple flow matching scenario. But what is the claim about what this construct provides that distinguishes itself from existing flow approaches? Given the lack of constraint in the overall construct it is not obvious to me it ends up being any different to any other flow matching modelling structure?
The background work in this area is extensive, and yet the background work section of the paper is fleeting. There is huge work on Hamiltonian-learning and motion–content literature that receives no comment or discussion including the following (in the Hamitonian and Lagrangian settings - there is also the flow background and optical flow literature)
Greydanus et Al. (2019) Hamiltonian Neural Networks.
Toth et Al. (2020) Hamiltonian Generative Networks
Cranmer et Al. (2020) Lagrangian Neural Networks.
Khan and Storkey (2022) HALO, Hamiltonian latent operators for content–motion separation in image sequences.
Allen-Blanchette et Al. (2020) LagNetViP, a Lagrangian network for video prediction.
These are not doing exactly what this paper does - they are more restrictive but in that sense make use of the conservation properties of the Hamiltonian construct more strongly than this paper does.
There are references that seem out of place (Kennard (1927) and Robertson (1929)) that I can't think why an author would actively add to such a paper, and others that appear to misunderstand of the content of the citations.
As the rebuttal period is drawing to a close, the authors have yet to address my concerns. After considering the other reviewers' comments, I am inclined to lean toward rejection.