acceptodds
Under review as a conference paper at ICLR 2027

Neural Homotopy System: Differentiable Predictor-Corrector for Homotopy Problems

Abstract

The homotopy paradigm provides a general strategy for solving difficult problems. It connects a hard target problem to a simple source problem through a continuous interpolation, then tracks the resulting solution trajectory, typically with a predictor-corrector algorithm. This paradigm underlies methods across many domains, but the interpolation is usually hand-designed and task-specific. Existing learning-based attempts to replace it remain instance-specific, requiring retraining for every new problem instance. We propose Neural Homotopy System (NHS), a differentiable predictor-corrector framework in which the predictor and corrector are learned neural modules, trained once and amortized over problem instances within a given problem family. NHS parameterizes the predictor with a neural ordinary differential equation that learns the solution trajectory across homotopy levels, while the corrector uses a deep equilibrium model that drives the predicted solution toward a problem-specific equilibrium condition. A predictor-anchored proximal term makes this equilibrium explicitly depend on the prediction, allowing gradients to propagate across homotopy levels and enabling end-to-end training of the complete trajectory. Instantiating this shared architecture separately for unconstrained optimization, constrained optimization, and conditional probabilistic sampling, NHS consistently achieves higher solution accuracy than classical local and hand-designed homotopy methods, as well as existing learning-based baselines, across all three settings.

Primary area:
optimization
Keywords:
Homotopy System, Global Optimization, Probabilistic Sampling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fpAoOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Neural Homotopy Systems (NHS), a new way of solving homotopy problems. Where in classical homotopy solvers, the predictor and corrector are hand-designed according to task-specific rules and criteria, NHS uses neural networks (a neural ODE and an equilibrium network, respectively), which enables end-to-end optimization. The authors report significant performance gains over several baselines on several optimization tasks.

I am on the fence about this paper - pages 1-9 are exceptionally well written and polished, but several things that are “hidden” in the appendix currently prevent me from giving a higher score. Please see the questions / criticism below.

Critical strengths

The writing is great. As mentioned above, the paper is well written, the motivation is clear, and it is a joy to read along. The concept of homotopies is introduced in a clear and self-contained way, accompanied with illustrative didactic figures (Figs. 1 and 2). As a reader not intimately familiar with the field, this has made the paper much more digestible for me, and I appreciate the effort.

On the technical side, viewing amortized and constrained optimization and annealed sampling in a single predictor-corrector homotopy framework is clean and novel. The authors make an effort to show the efficiency of the framework across 3 separate problems, with varying instances, un-constrained and constrained optimization and particle-based sampling.

Finally, NHS seems to beat all competitors in almost all experiments and almost all settings.

Critical weaknesses
  • There seems to be a severe disconnect between the presented method and theory, and the actual implementation. Inspecting the appendix reveals that the Neural ODE from the main paper in reality is only 1-2 Euler steps (C./D./E.2), which is a simple residual update and arguably makes the NODE formulation look a bit cosmetic. Could we not just backprop through these updates instead of requiring complicated continuous adjoint gradients?
  • Also, am I missing something, or is the sampling corrector not implemented as a proper Wasserstein minimizer? Simply unrolling 5 fixed steps of deterministic SVGD is not an equilibrium solve, as advertised in Eq.30 and Sec. 5.3 (although a Wasserstein term is added), so gradients can just be backpropped through the unrolled states and don’t require the presented implicit differentiation. While SVGD arguably targets a KL-type objective, I would have liked to see at least an analysis of the residual error of this 5-step truncation.
  • In the unconstrained optimization setting (Sec 6.1), am I reading the paper correctly in that NHS is trained with 4,096 labeled global instances and a supervised loss, whereas the other baselines (with the exception of MISO) are unsupervised optimizers searching the space from scratch? In any case, since the context domain is only [-1,1] in 2D, with 4,096 samples densely covering the space this decays to a toy example / easy regression task here. I would have liked to see a simple MLP regressor baseline going from c to x_ref, with a correction step applied afterwards.
  • For the constrained optimization setting in Sec. 6.2, it is unclear to me what the “shadow equilibrium with adjoint regularization” and the “surrogate gradient” are (App. D.2, L.1010), and why they are required and used. In fact, I thought the point of NHS was that we can derive backward derivatives through adjoint/implicit differentiation, so having to use a workaround here once again defies the paper’s central story.
  • The comparison to the baselines in the sampling section (6.3) seems unfair, since NHS is trained with GT samples (Appendix E1&2: samples generated via exact mixture sampling or numerical inverse-CDF), whereas it seems that the other baselines in Tab.4 only have access to the unnormalized density. So this doesn’t really prove the necessity of the neural homotopy - please include a supervised, conditional generative baseline trained on the same reference samples.
  • In the presented ablation study, the system is trained jointly (w/ predictor and corrector), and then those trained components are removed individually (L.1228, Tab.12). Of course the performance is going to suffer when you remove half of a trained system at test time, so what does this ablation actually show? Note that L.475 explicitly promises “Independent-training controls [...] in Appendix G”.
  • There’s currently no limitations section. I would encourage the authors to add one, as well as discuss the runtime of their approach more thoroughly in the main document (currently in e.g. Tab.8 of the appendix).
  • NPC (Neural Predictor Corrector, Mai et al., ICLR2026) seems to be the most recent relevant baseline. I checked, and they have code available on Github, so I’m wondering why the paper did not compare against this baseline, despite mentioning it in the introduction.
Questions for authors

see above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer fpAo (1/2)

Thank you for tracing the implementation details in the appendix and pointing out where the main text needs more precision.

The Questions field refers to CW1–CW8, which are answered jointly with Q below.

CW1 & Q The Neural ODE predictor and adjoint differentiation

We use one or two Euler steps per homotopy interval because we observed that the learned solution paths become relatively straight after training, allowing these few steps to provide accurate predictions and good solver performance. Our predictor is modeled as a continuous velocity field, so we can increase the integration step count substantially when a finer discretization is desired. The current few-step updates admit direct backpropagation. The continuous formulation and its adjoint equations provide a unified framework for prediction and gradient propagation when using many more integration steps.

CW2 & Q The Wasserstein sampling corrector and the five-step residual

Our sampling experiments show that five correction steps already provide good sampling performance. As shown below, the RMS particle displacement from one additional update is small on all three tasks, supporting this choice. Adjacent homotopy levels vary gradually, and the predictor provides an initial particle cloud for the next level, so only modest corrections are needed at each level. We therefore choose five steps to balance sampling accuracy and computational cost, and can increase the number of steps when further refinement is desired. We measured these displacements at step size 0.05, averaging over homotopy levels, 128 test conditions and three particle initializations per condition. In the reported experiments, gradients are backpropagated through the five updates.

Sampling taskExtra-step RMS displacement after 5 updates
Gaussian mixture0.01023
1D double well0.00139
4D double well0.00155

CW3 & Q Supervision, context density, and the direct MLP baseline

First, we examined how much NHS depends on the density of labeled training contexts. On Rastrigin 2D, we varied the label budget while fixing the architecture and training-update count, and evaluated 128 held-out contexts over five seeds with success defined by L₂ ≤ 0.01. NHS retained high success rates with 256 and 64 labels, showing that it can generalize well from substantially sparser supervision.

Second, we examined whether direct regression under the same supervision could achieve comparable accuracy. We trained a direct MLP and NHS with the same 4,096 reference pairs and 1,000 updates. NHS achieved smaller objective and solution errors and a higher success rate, supporting the benefit of its predictor–corrector beyond direct regression. In addition, our IEEE14 MLP-plus-Newton comparison showed higher feasibility and lower solution error for NHS, and raising the Newton cap from 40 to 100 recovered no additional feasible cases. Detailed IEEE14 numerical results are in General Response 1.

Labeled training contextsNHS success rate
6490.62% ± 3.27%
25698.91% ± 2.04%
1,024100.00% ± 0.00%
4,09698.44% ± 2.28%
MethodMean objective gap ↓Mean L₂ solution error ↓Success rate ↑
Direct supervised MLP (c ↦ x_ref)0.064520.0121951.72%
Full NHS0.009690.0028298.44%
AuthorsOpenReview ↗

Response to Reviewer fpAo (2/2)

CW4 & Q Does AC-OPF require a shadow equilibrium and a surrogate gradient?

We updated and retrained the AC-OPF model so that its forward and implicit backward passes use the same predictor-anchored terminal equation. We return the corrected state in the forward pass and evaluate implicit differentiation at that state, allowing the terminal loss to reach the predictor through the anchor. With this formulation, we no longer require a separate shadow equilibrium, adjoint regularization or surrogate gradient. The updated model was feasible throughout the test set and achieved a small objective gap, showing that high accuracy is attainable with this consistent forward/backward formulation. Detailed numerical results are in General Response 1.

CW5 & Q Reference supervision and a conditional generative baseline

To compare methods under shared reference-sample supervision, we trained Direct-MMD, a conditional MLP generator that maps noise z and target context c directly to samples x = z + MLP(z, c). It is trained by minimizing the squared maximum mean discrepancy (MMD) between generated and reference samples.

We used the same reference-sampling procedure and 2,000 training updates for both methods. NHS achieved lower W₂ error on all three tasks, with relative reductions of 12.24%–30.76%. These results show that, under shared reference-sample supervision, our combination of learned prediction and target-score-based correction produces more accurate samples than direct conditional generation.

Sampling taskDirect-MMD, W₂ ↓NHS, W₂ ↓Relative reduction with NHS
2D Gaussian mixture0.8669 ± 0.00990.6002 ± 0.007130.76%
1D double well0.1913 ± 0.00110.1679 ± 0.001312.24%
4D double well0.8664 ± 0.01320.6948 ± 0.002519.81%

CW6 & Q Inference-time component removal and independent training

We trained the predictor-only and corrector-only configurations independently under the same reference supervision and compared them with test-time component removal on Rastrigin 2D. Independent training substantially improved predictor-only performance, giving a more direct assessment of its standalone solving ability. Full NHS achieved the highest success rate across both controls. These results support the benefit of combining learned prediction with correction.

Components used at inferenceSuccess with test-time component removal ↑Success after independent training ↑
Predictor only10.63%68.13%
Corrector only4.06%2.97%
Predictor + corrector98.44%98.44%

CW7 & Q Limitations and runtime in the main text

Following the reviewer’s suggestion, we will add a limitations and runtime discussion to the main text. We will describe reference-data requirements, sampling dimensions, correction budgets and the accuracy–runtime trade-off, and present inference timings together with solution accuracy.

CW8 & Q Why is there no comparison with the public NPC implementation?

For our full response, please see General Response 2.

Thank you, authors, for providing this substantial follow-up during the rebuttal phase. I'm pleased to see that several of my original concerns have been addressed, such as the independently trained ablations, the modifications to make the AC-OPF fw and bw passes consistent, and the regressor-MLP-experiments.

Regarding the Wasserstein argument: the small particle displacement is a useful experiment/diagnostic, and empirically demonstrates stationarity (though not strictly establishign it), so I'm willing to accept this clarification at this stage.

Yet, I do not see why NPC focusing on step-size and stopping policy makes a comparison unnecessary. If anything, the authors' "accuracy-vs-speed" argument calls for an accuracy-runtime comparison. The promised "future work" integration does not resolve this question and unfortunately leaves the paper incompletely positioned against closely related work.

Overall, my initial weak rejection recommendation has shifted slightly, although I do see the missing NPC baseline as a key missing ingredient whose absence is not sufficiently justified, which prevents me from giving a higher score currently.

Thank you for suggesting NPC. We compared NPC + GH with NHS on the same 128 held-out Hard Ackley 2D instances. We report mean ± SD over five seeds. Recall uses an L2 error threshold of 0.05, and runtime is the end-to-end batch time per instance on a single CPU thread.

MethodMean objective gap ↓Mean L2 error ↓Recall (%) ↑ms/instance ↓
Fixed-schedule momentum GH0.2653 ± 0.01680.0505 ± 0.004566.41 ± 5.710.538 ± 0.021
NPC + GH0.2890 ± 0.06050.0534 ± 0.009360.94 ± 3.870.466 ± 0.101
NHS (ours)0.0649 ± 0.00570.0175 ± 0.001795.78 ± 2.388.297 ± 0.233

Mai et al. (Sec. 5.3, Table 3) report that NPC accelerated GH on all three 2D benchmarks with comparable final objective values. In our Hard Ackley control, adding NPC to the same momentum GH solver reduced runtime from 0.538 to 0.466 ms per instance, while recall fell from 66.41% to 60.94%. On this benchmark, NPC + GH was 17.8× faster, while NHS was more accurate, with recall of 95.78% versus 60.94%. This result supports our view that NHS is well suited to settings where solution accuracy is the priority.

Reviewer_bAwpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a Neural Homotopy System (NHS), an end-to-end trainable framework that combines a Neural ODE predictor with a Deep Equilibrium Model (DEQ) corrector. The framework jointly learns the homotopy interpolation and solution trajectory, enabling amortized learning across different instances within the same problem family.

The main idea is to replace hand-designed components of traditional homotopy solvers with differentiable neural modules, allowing the predictor and corrector to be trained jointly. Numerical experiments on unconstrained optimization, constrained optimization, and probabilistic sampling demonstrate promising improvements in solution accuracy over the considered baselines.

Critical strengths
  1. The paper is well organized, with clear explanations, intuitive illustrations, and well-presented experimental results.

  2. The proposed method achieves promising solution accuracy across several benchmarks, with substantial improvements over the considered baselines.

  3. The integration of the Neural ODE predictor and proximal corrector provides a reasonable framework for jointly learning the homotopy trajectory.

  4. The method is evaluated across multiple problem classes, including a practical AC-OPF application, and demonstrates generalization to unseen problem instances.

Critical weaknesses
  1. The paper does not clearly demonstrate the benefit of learning the homotopy interpolation itself. A comparison with a fixed homotopy under the same predictor-corrector architecture would be valuable.

  2. The advantage of using a learned Neural ODE predictor over classical numerical predictors is not sufficiently investigated.

  3. Since the method relies on supervised solution data, it remains unclear whether the complex homotopy framework offers substantial advantages over directly learning the solution map followed by numerical correction.

  4. The proposed method incurs substantially higher inference costs than existing learning-based baselines on constrained optimization benchmarks, raising concerns about its practical efficiency.

Questions for authors

My main concerns are already summarized in the Weaknesses section. In particular, I would appreciate clarification on the following points:

  1. Can the authors better isolate the contribution of the learned homotopy through a comparison with a fixed homotopy?

  2. What are the advantages of the Neural ODE predictor over classical numerical predictors?

  3. Can the authors provide a fair accuracy–runtime comparison with simpler alternatives, such as direct solution prediction followed by numerical correction?

3 comments
AuthorsOpenReview ↗

Response to Reviewer bAwp

Thank you for asking which parts of the architecture justify their additional cost.

CW1 & Q1 What does learning the interpolation contribute?

We compared fixed and learned interpolation paths while retaining the same predictor–corrector structure, data and training budget. Both variants retained intermediate correction, and each predictor was trained on its own path. We observed lower objective and solution errors with the learned interpolation. Because correction is present in both variants, these gains support learning the intermediate equations in addition to using a predictor–corrector. Detailed numerical results are in General Response 3.

CW2 & Q2 Why use a learned predictor in place of a classical one?

We would like to clarify that a common explicit tangent predictor extrapolates in a forward-Euler form after computing the local tangent at the current state. In NHS, we also need the backward-Euler-form correction uₗ = ûₗ − αₗ h_θ(uₗ, sₗ). Since the unknown output uₗ appears inside the operator, obtaining it requires an additional equilibrium solve. We learn the proposal ûₗ across instances to provide an initial state for this solve. Here, the backward-Euler form describes the correction, while the predictor itself uses explicit Euler updates.

CW3 & Q3 Direct solution prediction followed by refinement

We compared NHS with direct MLP prediction followed by Newton correction under matched reference supervision. NHS achieved higher feasibility and lower solution error, with runtime between Newton-40 and Newton-100. Raising the Newton cap from 40 to 100 added runtime while leaving feasibility unchanged. Thus, the complete NHS pipeline provided more accurate solutions in less time than the larger-budget endpoint refinement. Detailed numerical results are in General Response 1.

CW4 & Q3 Practical inference cost on constrained tasks

On SOCP and QCQP, NHS has longer inference times than FSNet and lower solution errors in the paper’s accuracy comparison. Correction at successive homotopy levels refines the predicted states and adds inference work. These results show the accuracy–runtime trade-off of the design, which we will discuss explicitly in the main text. The IEEE14 comparison further examines solution quality and inference cost. Detailed IEEE14 numerical results are in General Response 1.

MethodSOCP (ms) ↓QCQP (ms) ↓
FSNet1.221570.96713
NHS1395.39983462.17007

Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the fixed-versus-learned homotopy ablation and the comparisons with direct prediction followed by Newton correction. These results strengthen the experimental justification for the proposed framework.

I still have two remaining concerns:

  1. Necessity of the learned predictor. The response explains the role of the implicit corrector, but does not directly clarify the advantage of a learned Neural ODE predictor over a classical tangent predictor under the same correction scheme.

  2. Accuracy–runtime trade-off. The IEEE14 results are encouraging, but trust-constr and COBYLA also achieve 100% feasibility. Reporting their inference times would help establish the practical advantages of NHS. Additionally, a fixed-versus-learned homotopy comparison on IEEE14 would further strengthen the evidence for the proposed design.

Overall, I acknowledge that the additional experiments have addressed several of my original concerns. However, I would appreciate further clarification on these remaining points before reconsidering my overall assessment.

1. Necessity of the learned predictor

On Hard Rastrigin 2D, the learned predictor achieved higher solution accuracy in less time per batch than the classical tangent predictor. With the same NHS checkpoint and correction budget, the Neural ODE (NODE) predictor achieved 98.44% success in 178.3 ms per batch, compared with 0.47% in 317.1 ms for the classical tangent predictor. Retraining the homotopy field and proximal step with the tangent predictor under the same supervision and training budget yielded 0% success. In 530 of 640 seed–instance pairs, the retrained tangent predictor’s first step landed at a corner of the search box, far from the reference solution, and the subsequent corrections did not reach the target accuracy.

PredictorObjective gap ↓L₂ error ↓Success (%) ↑Time per batch (ms) ↓
NODE (Full NHS)0.00969 ± 0.010270.00282 ± 0.0025998.44 ± 2.28178.3 ± 9.61
Identity (same checkpoint)3.723 ± 0.1151.622 ± 0.0324.06 ± 1.28172.3 ± 8.85
Classical tangent (same checkpoint)34.77 ± 2.894.951 ± 0.1710.47 ± 0.43317.1 ± 27.61
Classical tangent (retrained)71.98 ± 13.977.340 ± 0.5900.00 ± 0.00316.4 ± 22.58

Values are mean ± sample SD across five training seeds on the same 128 test instances. Success means L₂ error ≤ 0.01. Timing uses each seed’s median end-to-end latency for a batch of 128 on the same GPU.

We will report this predictor comparison in Appendix G and revise Sec. 4.1 to explain why we use the learned predictor.

2. Accuracy and runtime on IEEE14

End-to-end timing against trust-constr and COBYLA

We thank the reviewer for suggesting a comparison between Anchored NHS and direct solution prediction followed by numerical correction. All three methods met the 10⁻⁴ feasibility threshold on all 1,024 held-out IEEE14 AC-OPF instances. Although MLP + trust-constr is a strong local solver and was faster than Anchored NHS in this experiment, Anchored NHS averaged only 42.36 internal Newton step attempts per instance across its full homotopy path, fewer than the 52.23 optimizer iterations for trust-constr, and achieved a mean objective gap approximately 1/26 as large. The mean objective gap for NHS was also approximately 1/225 of that for MLP + COBYLA.

MethodFeasible instancesMean count per instance (range)Mean objective gap (%) ↓Mean latency, batch=1 (s) ↓Amortized time, batch=128 (ms/instance) ↓
Anchored NHS1,024/1,02442.36 Newton step attempts (31–74)0.000060117.07246.36
MLP + trust-constr1,024/1,02452.23 optimizer iterations (35–300)0.001560.25134.53
MLP + COBYLA1,024/1,024246.85 objective evaluations (154–300)0.013523.713,013.40

The counts describe solver-specific operations with different costs. All methods were timed end to end using the same GPU, including neural inference, data transfers, and numerical correction.

NHS runtime differed slightly between the two GPU timing setups. The mean objective gaps for trust-constr and COBYLA changed when we moved objective and constraint evaluations from CPU to GPU.

Fixed versus learned homotopy

Thank you for suggesting this comparison. We extended our fixed-versus-learned homotopy experiment from Rastrigin to the IEEE 14-bus AC optimal power flow problem. For the fixed homotopy, we set the intermediate residual field rθr_\theta to zero and retrained the predictor. For the learned homotopy, we trained the residual field jointly with the predictor. Both variants used the same predictor–corrector architecture and training budget and retained intermediate correction.

MetricFixed homotopyLearned homotopyRelative reduction
Converged and feasible instances ↑128/128128/128—
Mean absolute objective error ↓0.0026044400.00208591519.91%
Mean absolute relative objective error (%) ↓0.00009633910.000073004524.22%
Mean L₂ solution error ↓0.0002097940.00009371555.33%

With inference parameters selected separately for each variant, both configurations converged to feasible solutions on all 128 IEEE14 validation instances. The learned configuration achieved lower errors on all three metrics, with the largest reduction in mean L₂ solution error (55.33%). These results suggest that learning the intermediate homotopy with a neural network can improve solution accuracy on IEEE14 while maintaining full feasibility across the validation set.

Reviewer_eTYeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a learning-to-optimize method through the homotropy paradigm called Neural Homotopy System (NHS). The method consists of two differentiable components that are the predictor parametrized as a neural ODE and the corrector parametrized as a deep equilibrium model (DEQ). The framework applies to unconstrained optimization, constrained optimization, conditional probabilistic sampling. The experiments demonstrate advantages against classical optimization, handcrafted homotopy methods and learning-based baselines.

Critical strengths

The main strength of this paper is that it is a technically solid approach, with a clean exposition of the two key components (predictor as neural ODE and corrector as DEQ) and how it applies to multiple applications (unconstrained optimization, constrained optimization, conditional probabilistic sampling).

Critical weaknesses
  1. The first main weakness has to do with the methodological framing of this paper and as well as positioning to closely related work. To begin with, it is unclear to me whether there is really a “homotopy” being learned. On the corrector side, in Eq. (18) the residual rθr_\theta is a free network and then the predictor is not the tangent of the path either since vϕv_\phi is a free vector field trained to match corrector outputs. So, neither the path nor the predictor is tied to the homotopy structure. The only link between the two is the consistency loss, which encourages the predictor trajectory and the corrector's fixed points to agree. Another point that is unclear is that given the stop-gradient operator in the consistency loss, θ\theta is trained only via the terminal loss and ϕ\phi mostly by imitating the corrector which contradicts the “end-to-end learning” framing. Also, the DEQ framing of the corrector appears to be overstated since Eq. (6) is simply one backward Euler step. That said, the framework boils down to LL unrolled iterations where each iteration is learned neural ODE followed by a learned proximal step. There is recent work such "ODE-based Learning to Optimize" (https://link.springer.com/article/10.1007/s10107-025-02303-3 ) and “Learning-to-Optimize via Deep Unfolded Flows” (https://openreview.net/forum?id=ZOtOq7hxJP ) which proposes similar schemes; for example the latter paper also proposes a series of sequential layers/iterations with each iteration governed by a neural ODE and is trained end-to-end, so the authors could consider comparing against it to showcase the advantages of the underlying “homotopy” in their setting.

Also, given how that the NPC paper and this paper share the same framing of homotopy predictor-corrector as a unifying abstraction across optimization and sampling paper (the difference is that NPC used RL to learn specific parts of handcrafted steps, while NHS learns the mappings and fixes parts such as the levels as a specific linear grid, the αℓ\alpha_\ell parameter and the corrector cap), the authors should try to position their work more clearly compared to that paper. The multi-domain packaging is also inherited by that paper and overall I believe it would also make a reasonable baseline. This way, the claim that learned differentiable modules beat a learned policy around a fixed solver would also be tested.

  1. The experiments do not really validate the paper claims since the comparisons confound the learned homotopy with extra supervision, extra compute or weak baselines. For instance, the experiments show that an amortized, supervised solver beats classical and unsupervised methods, but not that the homotopy structure is responsible, because the learned baselines are not matched in supervision. Also as noted earlier, NPC is absent and also the OPF comparison is unmatched in compute. There are also some quite relevant baselines on homotopy-guided (https://arxiv.org/abs/2511.11677 ) and neural ODE AC-OPF (https://arxiv.org/abs/2410.19157 ) that the authors could consider comparing against. Several key ablations are also missing such as for example, retraining with rθ=0r_\theta = 0 to test whether the learned path matters, applying the corrector only at s=1s = 1 to test what is the benefit of the intermediate levels or a supervised MLP from cc to xrefx_ref with the same data as input to separate amortization from the benefits of this particular architecture. Finally, it is worth noting that a lot of the experiments are at quite low scale (for example the unconstrained benchmarks are all 2D and the sampling targets are 1D to 4D).
Questions for authors
  1. Can the authors provide a clearer positioning through both presentation and experiments on what are the benefits of the specific proposed architecture compared to the various prior works mentioned above? That could potentially make the advantages of the specific architecture as well as several claims such as the “homotopy”, “differentiable modules are better”, etc. more convincing.

  2. Can the authors provide the above missing ablations? If the full architecture is not clearly better then it remains unclear how the learned homotopy interpolation and the intermediate levels are contributing.

  3. For the AC-OPF, can the authors report per-instance inference time and also consider a stronger baseline than Newton such as an off-the-shelf NLP solver? A comparison against the above mentioned related L2O baselines on AC-OPF could also be useful to further establish the advantages of the method.

5 comments
AuthorsOpenReview ↗

Response to Reviewer eTYe (1/2)

Thank you for the detailed questions about the homotopy formulation and the comparisons.

CW1.1–CW1.2 & Q1 Homotopy structure and predictor tangent

Although r_θ in Eq. (18) is a free network, its contribution s(1 − s)r_θ vanishes at s = 0 and s = 1, so the equation defines an endpoint-preserving operator homotopy. Along a smooth solution branch, the learned velocity converges to the path tangent as the levels become denser, provided the prediction displacement error is o(Δs).

CW1.3 & Q1 Which gradients pass through the consistency target?

The stop-gradient detaches only the target branch of the consistency loss. The terminal loss still propagates through the predictor-anchored correction to both modules, enabling end-to-end gradient propagation through all components.

CW1.4 & Q1 How does backward Euler relate to the DEQ layer?

We agree that Eq. (6) has a backward-Euler form. We define it as a predictor-anchored proximal correction, uₗ = ûₗ − αₗ h_θ(uₗ, sₗ), whose output is obtained by solving an implicit equilibrium equation. We use the DEQ formulation to obtain gradients by implicitly differentiating this equilibrium, avoiding backpropagation through an unrolling of all inner correction iterations. This avoids retaining the full unrolled computation graph and reduces the memory required for training.

CW1.5–CW1.6 & Q1 Relation to ODE-based learning-to-optimize methods

We share with both works the use of learned dynamics for optimization. Xie et al. learn dynamics coefficients under stability and convergence constraints, while FlowOptimizer learns sequential neural-ODE updates. In NHS, we learn endpoint-preserving intermediate operators and couple prediction with correction of the corresponding homotopy equations.

We compared NHS with our FlowOptimizer reproduction on 128 held-out Rastrigin 2D instances over five seeds. At the matched gradient-evaluation budget, NHS achieved lower objective and solution errors. NHS retained this advantage when the reproduction used the paper’s larger inference configuration. NHS was trained with reference solutions, while the reproduction used objective-based training.

We therefore interpret this experiment as evidence for the accuracy of the complete NHS solver under the reported training conditions. The fixed-path, intermediate-correction and direct-regression controls in CW2.6–CW2.8 further examine which architectural choices contribute to this performance.

MethodCandidates × stepsGradient evaluations per instanceObjective gap ↓L₂ solution error ↓
NHS1 × 10800.000561 ± 0.0002300.00121 ± 0.00017
FlowOptimizer reproduction20 × 4807.685 ± 2.001.660 ± 0.27
FlowOptimizer reproduction8 × 10803.531 ± 0.631.265 ± 0.13
FlowOptimizer reproduction, paper’s inference configuration20 × 102002.801 ± 0.641.113 ± 0.11

CW1.7 & Q1 What does NHS add to NPC?

For our full response, please see General Response 2.

CW2.1 Does supervision explain the gains?

To examine whether reference-solution supervision accounts for the improvement, we trained a direct MLP using the same reference solutions, minibatches and update count as NHS, and applied Newton correction to its predictions. NHS achieved higher feasibility, and increasing the Newton cap did not recover additional feasible cases. Thus, NHS retains a feasibility advantage under the same reference-solution label supervision. Detailed numerical results are in General Response 1.

CW2.3 & Q3.1 Is the OPF computation matched?

We timed prediction and correction together on the same test instances at batch size 128 and report the average time per instance. NHS achieved higher feasibility than Newton-40 with a longer runtime, and higher feasibility than Newton-100 with a shorter runtime. Increasing the Newton cap added runtime while leaving its feasible-case count unchanged. These results show how the complete NHS solver balances solution accuracy and inference cost. Detailed numerical results are in General Response 1.

AuthorsOpenReview ↗

Response to Reviewer eTYe (2/2)

CW2.4–CW2.5 & Q3.3 Homotopy-guided L2O and Neural ODE AC-OPF baselines

Following the reviewer’s suggestion, we evaluated static IEEE14 adaptations of the homotopy-guided and Neural ODE methods. The homotopy-guided source method uses self-supervised training, and NHS uses reference solutions. For the Neural ODE baseline, we adapted the SwingNN architecture to static dispatch prediction. On the same test set, NHS achieved full feasibility and lower mean absolute objective gaps than these learning baselines. These results demonstrate NHS’s accuracy and constraint satisfaction on IEEE14. Detailed numerical results are in General Response 1.

CW2.6 & Q1 & Q2 Retraining with the residual fixed to zero

We fixed r_θ = 0 throughout training and retrained the predictor with the same architecture and training conditions as the learned-path model. Both variants retained intermediate correction and trained their predictors on their respective paths. The learned-path model achieved lower objective and solution errors. This shows that learning the intermediate equations improves accuracy within the same predictor–corrector structure. Detailed numerical results are in General Response 3.

CW2.7 & Q1 & Q2 Correction only at s = 1

On Rastrigin 2D, we compared endpoint-only correction with full NHS under the same supervision, training budget, ten prediction steps and total correction cap of 50. Distributing correction across levels reduced both objective and solution errors and increased success. In full NHS, each corrected state becomes the starting point for the next prediction, coupling correction to the subsequent trajectory. The result supports retaining intermediate correction throughout the homotopy under the same iteration cap.

Correction schemeTest-time correction budgetMean objective gap ↓Mean L₂ solution error ↓Success rate ↑
Endpoint onlyUp to 50 iterations at s = 10.846060.0223878.28%
Full NHSUp to 5 iterations at each of 10 levels0.009690.0028298.44%

CW2.8 & Q1 & Q2 Direct supervised map from context to solution

We trained a direct MLP to map context to the reference solution on Rastrigin 2D using the same reference pairs and training budget as NHS. Full NHS achieved lower objective and solution errors and a higher success rate. Under matched reference supervision, these results support combining learned prediction with iterative correction to improve the final solution.

MethodMean objective gap ↓Mean L₂ solution error ↓Success rate ↑
Direct supervised MLP (c ↦ x_ref)0.064520.0121951.72%
Full NHS0.009690.0028298.44%

CW2.9 Scale of the optimization and sampling benchmarks

We extended Bifurcation Himmelblau to 64 decision variables while retaining a two-dimensional context. NHS achieved a lower mean solution error than all compared baselines, showing that its accuracy advantage extends to this higher-dimensional decision space. Our conditional sampling experiments cover targets in one to four dimensions.

MethodMean L₂ error ↓
L-BFGS-B7.8797
GD5.2182
GH5.2605 ± 0.0099
EPGS5.2099 ± 0.0134
PGS5.1968 ± 0.0039
ZOSLGH-d5.2671 ± 0.0477
ZOSLGH-r13.8046 ± 0.2250
MISO PD (K = 4)2.3356 ± 0.0715
MISO WTA (K = 4)2.8053 ± 0.2878
Conditional CPL (adapted)5.6099 ± 0.0367
NHS (ours)2.0868 ± 0.0942

Q3.2 Off-the-shelf nonlinear solver

We compared off-the-shelf NLP solvers initialized from the same frozen MLP predictions as Newton. Trust-constr, COBYLA and NHS all achieved full feasibility, and NHS had the smallest mean absolute objective gap among these three methods. Thus, the objective-accuracy advantage persists when the solvers are compared at the same feasibility level. Detailed numerical results are in General Response 1.

AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewer eTYe,

Thank you again for your review and for your positive assessment of our work. As the discussion period closes on 18 November (Anywhere on Earth), with three days remaining, we wanted to ask whether our rebuttal answered your points. If anything is still unclear, please tell us and we will respond promptly. If your concerns are now resolved, we would be sincerely grateful for your thorough and valuable feedback.

With sincere appreciation and best regards,

Authors of Submission 10851

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their constructive feedback. We summarize our additional AC-OPF experiments, the relationship to NPC and the fixed-versus-learned homotopy control below.

General Response 1. Additional AC-OPF experiments and comparisons

We evaluated NHS, MLP+Newton, NLP solvers and static learning adaptations on 1,024 held-out IEEE14 instances at a feasibility tolerance of 10⁻⁴. The MLP control used the same reference supervision and training budget as NHS, and all NLP solvers started from the same MLP predictions. NHS, trust-constr and COBYLA achieved full feasibility, with NHS attaining the lowest reported mean absolute objective gap. Raising the Newton cap from 40 to 100 left feasibility unchanged, showing that extending endpoint refinement alone did not improve the feasible-case count. Together, these comparisons show NHS’s ability to combine constraint satisfaction with high objective accuracy. The homotopy-guided source method uses self-supervision, and the NODE entry uses a static SwingNN adaptation. Objective gaps are averaged over all instances, including infeasible outputs. Timings include prediction and correction and are amortized per instance at batch size 128.

MethodFeasible instancesFeasibility (%) ↑Mean absolute objective gap (%) ↓Mean L₂ error ↓Time (ms/instance) ↓
Direct MLP0 / 1,0240.00—0.06198—
MLP + Newton-10347 / 1,02433.89—0.32057—
MLP + Newton-40897 / 1,02487.603.3392940.23134185.8
MLP + Newton-100897 / 1,02487.60—0.23139482.3
MLP + trust-constr1,024 / 1,024100.000.00117417——
MLP + COBYLA1,024 / 1,024100.000.04934350——
MLP + COBYQA772 / 1,02475.390.734552——
SaS+GPen (IEEE14 adaptation)0 / 1,0240.0013.10——
Load-stepping+GPen (IEEE14 adaptation)0 / 1,0240.0013.26——
Static NODE adaptation using SwingNN0 / 1,0240.000.595——
Anchored NHS1,024 / 1,024100.006.006 × 10⁻⁵9.918 × 10⁻⁵224.7

General Response 2. Relationship to NPC

NPC learns how far to advance along a prescribed homotopy and when to stop correction. NHS learns the intermediate homotopy field and the predictor on a fixed grid. We compared fixed-schedule momentum GH, NPC + GH, and NHS on the same 128 held-out Hard Ackley 2D instances across five seeds. On a single CPU thread, adding NPC to the same momentum GH solver reduced mean time per instance from 0.538 to 0.466 ms, while recall decreased from 66.41% to 60.94%. NHS achieved the highest recall (95.78%) and took 8.297 ms per instance. Recall is the percentage of solutions with L₂ error at most 0.05. Our follow-up response to reviewer fpAo reports the full comparison.

General Response 3. Fixed versus learned homotopy

We fixed r_θ = 0 and retrained the predictor under the same architecture and training conditions as the learned-homotopy model. Both variants retained intermediate correction. On Rastrigin 2D, learning the homotopy reduced both objective and solution errors. Because both variants use prediction and correction, these gains show the additional accuracy benefit of learning the intermediate operators.

Metric ↓Fixed homotopy (r_θ = 0)Learned homotopyRelative reduction
Mean objective error0.014850.0072751.0%
Mean L₂ solution error0.003010.0022325.9%

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.