- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies a dynamic neural optimal transport framework that maps a source distribution to a reweighted target distribution. This framework has connections to unbalanced and dynamic optimal transport. In the proposed semi-relaxed method, the authors formulate a Lagrangian that is optimized over two neural network variables. The algorithm then becomes a min-max procedure (gradient descent-ascent) over these two variables. Experiments demonstrate some advantages in practice.
Critical strengths
The formulation yields a novel min-max formulation that is different from existing unbalanced and partial OT methods. The formulation captures both dynamic and static formulations, which appear to be solvable effectively in practice.
Experiments demonstrate increased accuracy in domain translation, unlabeled learning, and image translation datasets.
Critical weaknesses
Not enough time is spent really digging into why, theoretically, this formulation is better than or motivated over existing unbalanced OT and partial OT frameworks, which also relax the marginal constraints. Furthermore, these also have dynamic formulations. While empirically there appear to be differences, this work would be stronger if these differences were based on foundational theoretical results stating the advantages of this work.
To optimize the Lagrangian, the authors formulate a min-max problem with neural network variables. This is reminiscent of GANs and other critic-based methods in ML. One concern is that these types of problems are frequently unstable and hard to optimize in practice. However, the authors do not explore this aspect of the problem. I am curious how hard it is to train these variables and what guarantees can be given that the problem can be solved effectively in general.
Questions for authors
If the authors could comment on the critical weaknesses above, that would be helpful. In other words, what theoretically differentiates the new approach? Examples and simple experiments could help to illustrate this.
Second, is the min-max procedure more stable than other variants that have appeared in the ML literature (e.g., GANs), or does this formulation suffer from the same issues? Is there a reformulation we could use without the min-max?
4 comments
Follow-up to our response to Reviewer 7Eb5
Dear Reviewer 7Eb5, Thank you again for your feedback. With the discussion period ending on November 18, we would appreciate your assessment of our clarifications regarding the theoretical distinction from related transport methods, optimization stability, and alternatives to min–max training. We remain available to address any outstanding questions.
Thank you for taking the time to better explain what sets your method apart, and clearly writing this into text would be helpful. I am inclined to raise my score.
Reply to official comment by 7Eb5
We thank the reviewer 7Eb5 for thoughtful review and reply to our rebuttal. We would like to inform the reviewer 7Eb5 that the updated manuscript has been uploaded and all changes to the initial manuscript (both main body and appendix) are colored purple. Due to page length, we had to make the exposition in the rebuttal more compact, but we made the following changes to the text to better explain our method:
- Lines 119-125 (Section 2, Related Work) specifically discuss a mini-batch alternative for subset alignment and its difference from our approach and cite the related reference.
- For limitations regarding stability associated with mini-max training we updated the text between lines 515-519 (Section 5, Limitations) and connected it to the updated Related Work via a footnote.
Additionally, we would like to mention that we made minor changes to Sections 1 and 2 to better position our work in relation to directly related work. While not updated, detailed literature surveys for both static and dynamic neural optimal transport are given in Appendix C.
Reply to Reviewer 7Eb5
We thank the reviewer for recognizing the formulation and its empirical benefits, and for raising two important questions.
Answer to Question 1
"...What theoretically differentiates the new approach?" We discuss the positioning of our work in Section 1, under “Positioning and contributions,” and in Section 2. The central motivation is to learn transport dynamics when the relevant terminal distribution is unknown. Our framework preserves the full source marginal and conserves mass along the path, while allowing the terminal distribution to vary under the constraint ρ1≤cν. This constraint provides an explicit bound on local reweighting. For any measurable target region A, ρ1(A)≤cν(A). For example, if A contains (10%) of the target mass and c=2, it can receive at most (20%) of the transported mass, regardless of how inexpensive it is to reach. Moreover, any region containing all transported mass must have target mass at least c1. These properties give c a direct interpretation and control concentration on a small part of the target.
Dynamic unbalanced OT also relaxes marginal matching, including through mass growth and destruction. Our formulation combines conservation of the entire source mass with a hard terminal density cap. A finite marginal-divergence penalty instead balances marginal discrepancy against transport cost. The hard cap is therefore motivated by settings where preserving the source population and controlling target reweighting are explicit requirements.
The endpoint constraint is closely related to partial and incomplete transport. Our contribution develops a dynamic neural formulation and solver for this free terminal marginal, including the corresponding boundary term and a learned probability path. The method jointly learns terminal reweighting and transport dynamics from samples without requiring ODE integration during each training step. It also provides intermediate distributions and a velocity field for trajectory generation and reverse integration. These capabilities motivate the dynamic formulation in applications such as population trajectory inference.
Answer to Question 2 (Stability and Alternatives)
Since our method uses neural networks in a saddle-point optimization problem, it inherits the associated optimization challenges. Training can depend on initialization, learning rates, and the relative frequency of potential and interpolant updates. The underlying population problem nevertheless has a convex formulation in density and momentum, with a convex kinetic action, a linear continuity constraint, and a convex terminal density constraint. This provides a principled variational basis, although it does not imply convergence of the nonconvex neural updates.
Our experiments, including repeated runs on 20 PU-learning datasets, demonstrate practical applicability and report variability in task performance. These results support the effectiveness of the training procedure in the evaluated settings, while a general convergence guarantee or stability advantage over GAN-style optimization is not established.
The min–max structure arises from our variational solution method. The underlying problem also admits a constrained primal minimization. For finite empirical distributions, one can solve a linear program with fixed source weights and capped target weights, avoiding alternating neural updates.
Another possibility is to construct minibatch couplings and train a flow from them. UOT-FM [1] follows this general strategy using unbalanced transport couplings. Adapting this strategy to our setting would require capped couplings that preserve the source marginal. Each coupling would then optimize over the target points available within its batch. Consequently, solving these local problems does not by itself guarantee recovery of the globally optimal capped transport.
Our approach directly optimizes the sample-based variational objective using shared potential and interpolant networks, avoiding a discrete transport solution within every batch. This is the computational motivation for the proposed saddle-point formulation. Both approaches involve approximation, but they differ in whether transport is learned through the variational objective or through separately computed minibatch couplings. We would further like to point out sections 1 and 2 from paper discussion positioning our work with respect to literature and the appendix C also discuss related work from both static and dynamic optimal transport.
We hope these clarifications would motivate the reviewer to reconsider their assessment of our work.
[1] Eyring, Luca, et al. “Unbalancedness in Neural Monge Maps Improves Unpaired Domain Translation.” International Conference on Learning Representations, 2024.