- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Neural Homotopy Systems (NHS), a new way of solving homotopy problems. Where in classical homotopy solvers, the predictor and corrector are hand-designed according to task-specific rules and criteria, NHS uses neural networks (a neural ODE and an equilibrium network, respectively), which enables end-to-end optimization. The authors report significant performance gains over several baselines on several optimization tasks.
I am on the fence about this paper - pages 1-9 are exceptionally well written and polished, but several things that are “hidden” in the appendix currently prevent me from giving a higher score. Please see the questions / criticism below.
Critical strengths
The writing is great. As mentioned above, the paper is well written, the motivation is clear, and it is a joy to read along. The concept of homotopies is introduced in a clear and self-contained way, accompanied with illustrative didactic figures (Figs. 1 and 2). As a reader not intimately familiar with the field, this has made the paper much more digestible for me, and I appreciate the effort.
On the technical side, viewing amortized and constrained optimization and annealed sampling in a single predictor-corrector homotopy framework is clean and novel. The authors make an effort to show the efficiency of the framework across 3 separate problems, with varying instances, un-constrained and constrained optimization and particle-based sampling.
Finally, NHS seems to beat all competitors in almost all experiments and almost all settings.
Critical weaknesses
- There seems to be a severe disconnect between the presented method and theory, and the actual implementation. Inspecting the appendix reveals that the Neural ODE from the main paper in reality is only 1-2 Euler steps (C./D./E.2), which is a simple residual update and arguably makes the NODE formulation look a bit cosmetic. Could we not just backprop through these updates instead of requiring complicated continuous adjoint gradients?
- Also, am I missing something, or is the sampling corrector not implemented as a proper Wasserstein minimizer? Simply unrolling 5 fixed steps of deterministic SVGD is not an equilibrium solve, as advertised in Eq.30 and Sec. 5.3 (although a Wasserstein term is added), so gradients can just be backpropped through the unrolled states and don’t require the presented implicit differentiation. While SVGD arguably targets a KL-type objective, I would have liked to see at least an analysis of the residual error of this 5-step truncation.
- In the unconstrained optimization setting (Sec 6.1), am I reading the paper correctly in that NHS is trained with 4,096 labeled global instances and a supervised loss, whereas the other baselines (with the exception of MISO) are unsupervised optimizers searching the space from scratch? In any case, since the context domain is only [-1,1] in 2D, with 4,096 samples densely covering the space this decays to a toy example / easy regression task here. I would have liked to see a simple MLP regressor baseline going from c to x_ref, with a correction step applied afterwards.
- For the constrained optimization setting in Sec. 6.2, it is unclear to me what the “shadow equilibrium with adjoint regularization” and the “surrogate gradient” are (App. D.2, L.1010), and why they are required and used. In fact, I thought the point of NHS was that we can derive backward derivatives through adjoint/implicit differentiation, so having to use a workaround here once again defies the paper’s central story.
- The comparison to the baselines in the sampling section (6.3) seems unfair, since NHS is trained with GT samples (Appendix E1&2: samples generated via exact mixture sampling or numerical inverse-CDF), whereas it seems that the other baselines in Tab.4 only have access to the unnormalized density. So this doesn’t really prove the necessity of the neural homotopy - please include a supervised, conditional generative baseline trained on the same reference samples.
- In the presented ablation study, the system is trained jointly (w/ predictor and corrector), and then those trained components are removed individually (L.1228, Tab.12). Of course the performance is going to suffer when you remove half of a trained system at test time, so what does this ablation actually show? Note that L.475 explicitly promises “Independent-training controls [...] in Appendix G”.
- There’s currently no limitations section. I would encourage the authors to add one, as well as discuss the runtime of their approach more thoroughly in the main document (currently in e.g. Tab.8 of the appendix).
- NPC (Neural Predictor Corrector, Mai et al., ICLR2026) seems to be the most recent relevant baseline. I checked, and they have code available on Github, so I’m wondering why the paper did not compare against this baseline, despite mentioning it in the introduction.
Questions for authors
see above.
4 comments
Response to Reviewer fpAo (2/2)
CW4 & Q Does AC-OPF require a shadow equilibrium and a surrogate gradient?
We updated and retrained the AC-OPF model so that its forward and implicit backward passes use the same predictor-anchored terminal equation. We return the corrected state in the forward pass and evaluate implicit differentiation at that state, allowing the terminal loss to reach the predictor through the anchor. With this formulation, we no longer require a separate shadow equilibrium, adjoint regularization or surrogate gradient. The updated model was feasible throughout the test set and achieved a small objective gap, showing that high accuracy is attainable with this consistent forward/backward formulation. Detailed numerical results are in General Response 1.
CW5 & Q Reference supervision and a conditional generative baseline
To compare methods under shared reference-sample supervision, we trained Direct-MMD, a conditional MLP generator that maps noise z and target context c directly to samples x = z + MLP(z, c). It is trained by minimizing the squared maximum mean discrepancy (MMD) between generated and reference samples.
We used the same reference-sampling procedure and 2,000 training updates for both methods. NHS achieved lower W₂ error on all three tasks, with relative reductions of 12.24%–30.76%. These results show that, under shared reference-sample supervision, our combination of learned prediction and target-score-based correction produces more accurate samples than direct conditional generation.
| Sampling task | Direct-MMD, W₂ ↓ | NHS, W₂ ↓ | Relative reduction with NHS |
|---|---|---|---|
| 2D Gaussian mixture | 0.8669 ± 0.0099 | 0.6002 ± 0.0071 | 30.76% |
| 1D double well | 0.1913 ± 0.0011 | 0.1679 ± 0.0013 | 12.24% |
| 4D double well | 0.8664 ± 0.0132 | 0.6948 ± 0.0025 | 19.81% |
CW6 & Q Inference-time component removal and independent training
We trained the predictor-only and corrector-only configurations independently under the same reference supervision and compared them with test-time component removal on Rastrigin 2D. Independent training substantially improved predictor-only performance, giving a more direct assessment of its standalone solving ability. Full NHS achieved the highest success rate across both controls. These results support the benefit of combining learned prediction with correction.
| Components used at inference | Success with test-time component removal ↑ | Success after independent training ↑ |
|---|---|---|
| Predictor only | 10.63% | 68.13% |
| Corrector only | 4.06% | 2.97% |
| Predictor + corrector | 98.44% | 98.44% |
CW7 & Q Limitations and runtime in the main text
Following the reviewer’s suggestion, we will add a limitations and runtime discussion to the main text. We will describe reference-data requirements, sampling dimensions, correction budgets and the accuracy–runtime trade-off, and present inference timings together with solution accuracy.
CW8 & Q Why is there no comparison with the public NPC implementation?
For our full response, please see General Response 2.
Thank you, authors, for providing this substantial follow-up during the rebuttal phase. I'm pleased to see that several of my original concerns have been addressed, such as the independently trained ablations, the modifications to make the AC-OPF fw and bw passes consistent, and the regressor-MLP-experiments.
Regarding the Wasserstein argument: the small particle displacement is a useful experiment/diagnostic, and empirically demonstrates stationarity (though not strictly establishign it), so I'm willing to accept this clarification at this stage.
Yet, I do not see why NPC focusing on step-size and stopping policy makes a comparison unnecessary. If anything, the authors' "accuracy-vs-speed" argument calls for an accuracy-runtime comparison. The promised "future work" integration does not resolve this question and unfortunately leaves the paper incompletely positioned against closely related work.
Overall, my initial weak rejection recommendation has shifted slightly, although I do see the missing NPC baseline as a key missing ingredient whose absence is not sufficiently justified, which prevents me from giving a higher score currently.
Thank you for suggesting NPC. We compared NPC + GH with NHS on the same 128 held-out Hard Ackley 2D instances. We report mean ± SD over five seeds. Recall uses an L2 error threshold of 0.05, and runtime is the end-to-end batch time per instance on a single CPU thread.
| Method | Mean objective gap ↓ | Mean L2 error ↓ | Recall (%) ↑ | ms/instance ↓ |
|---|---|---|---|---|
| Fixed-schedule momentum GH | 0.2653 ± 0.0168 | 0.0505 ± 0.0045 | 66.41 ± 5.71 | 0.538 ± 0.021 |
| NPC + GH | 0.2890 ± 0.0605 | 0.0534 ± 0.0093 | 60.94 ± 3.87 | 0.466 ± 0.101 |
| NHS (ours) | 0.0649 ± 0.0057 | 0.0175 ± 0.0017 | 95.78 ± 2.38 | 8.297 ± 0.233 |
Mai et al. (Sec. 5.3, Table 3) report that NPC accelerated GH on all three 2D benchmarks with comparable final objective values. In our Hard Ackley control, adding NPC to the same momentum GH solver reduced runtime from 0.538 to 0.466 ms per instance, while recall fell from 66.41% to 60.94%. On this benchmark, NPC + GH was 17.8× faster, while NHS was more accurate, with recall of 95.78% versus 60.94%. This result supports our view that NHS is well suited to settings where solution accuracy is the priority.
Response to Reviewer fpAo (1/2)
Thank you for tracing the implementation details in the appendix and pointing out where the main text needs more precision.
The Questions field refers to CW1–CW8, which are answered jointly with Q below.
CW1 & Q The Neural ODE predictor and adjoint differentiation
We use one or two Euler steps per homotopy interval because we observed that the learned solution paths become relatively straight after training, allowing these few steps to provide accurate predictions and good solver performance. Our predictor is modeled as a continuous velocity field, so we can increase the integration step count substantially when a finer discretization is desired. The current few-step updates admit direct backpropagation. The continuous formulation and its adjoint equations provide a unified framework for prediction and gradient propagation when using many more integration steps.
CW2 & Q The Wasserstein sampling corrector and the five-step residual
Our sampling experiments show that five correction steps already provide good sampling performance. As shown below, the RMS particle displacement from one additional update is small on all three tasks, supporting this choice. Adjacent homotopy levels vary gradually, and the predictor provides an initial particle cloud for the next level, so only modest corrections are needed at each level. We therefore choose five steps to balance sampling accuracy and computational cost, and can increase the number of steps when further refinement is desired. We measured these displacements at step size 0.05, averaging over homotopy levels, 128 test conditions and three particle initializations per condition. In the reported experiments, gradients are backpropagated through the five updates.
CW3 & Q Supervision, context density, and the direct MLP baseline
First, we examined how much NHS depends on the density of labeled training contexts. On Rastrigin 2D, we varied the label budget while fixing the architecture and training-update count, and evaluated 128 held-out contexts over five seeds with success defined by L₂ ≤ 0.01. NHS retained high success rates with 256 and 64 labels, showing that it can generalize well from substantially sparser supervision.
Second, we examined whether direct regression under the same supervision could achieve comparable accuracy. We trained a direct MLP and NHS with the same 4,096 reference pairs and 1,000 updates. NHS achieved smaller objective and solution errors and a higher success rate, supporting the benefit of its predictor–corrector beyond direct regression. In addition, our IEEE14 MLP-plus-Newton comparison showed higher feasibility and lower solution error for NHS, and raising the Newton cap from 40 to 100 recovered no additional feasible cases. Detailed IEEE14 numerical results are in General Response 1.
c ↦ x_ref)