acceptodds
Under review as a conference paper at ICLR 2027

MFPI-Net: Learning to Solve Absolute Value Equations via MFPI Unrolling

Abstract

Absolute value equations (AVEs) offer a unified characterization for optimality conditions across a wide range of problems, including linear and convex quadratic programming, bimatrix games, among others. Due to the non-smooth, non-convex structure and NP-hard nature induced by the absolute-value operator, solving AVEs poses significant challenges, for which many classical numerical algrithms have been proposed, and the modified fixed-point iteration (MFPI) ranks among the most effective. By combining a learnable smooth approximation of the absolute-value operator with paired neural modules regularized to preserve mutual-inverse consistency, we propose MFPI-Net, a neural solver that unrolls MFPI into learnable residual updates. Building on MFPI’s convergence theory, we derive approximation error bounds for the unrolled network and establish layerwise stability against input perturbations. To understand how network depth affects the trade-off between computational cost and approximation accuracy, we analyze MFPI-Net from a continuous-time dynamical systems perspective. This motivates a two-stage framework: MFPI-Net is first leveraged to perform sign estimation to eliminate the absolute-value operator, and conventional solvers are then deployed to obtain high-accuracy solutions. Experiments demonstrate approximately 3 faster high-accuracy AVE solving than MFPI, with successful solution recovery on tested instances outside the standard convergence regime. Beyond AVEs, MFPI-Net provides warm starts for various linear programming solvers through AVE reformulations.

Primary area:
optimization
Keywords:
Learning to Optimize, Deep Unrolling, Absolute Value Equations, Neural Solvers

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_9PEdOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MFPI-Net, a neural solver inspired by the classical Modified Fixed-Point Iteration (MFPI) algorithm for solving Absolute Value Equations (AVEs). Building upon the convergence theory of MFPI, the authors provide theoretical guarantees on approximation accuracy and stability. The proposed framework adopts a two-stage approach: MFPI-Net first generates an approximate solution, which is subsequently refined through numerical linear solves to obtain a high-accuracy solution. Experiments are conducted on synthetically generated AVE instances with problem dimensions ranging from 1,000 to 10,000. The results demonstrate that the proposed method achieves low residual errors with relatively short solving times.

Critical strengths
  1. The paper is clearly written and easy to follow. It addresses a meaningful numerical optimization problem.

  2. The paper provides substantial theoretical and empirical analyses. Unrolling MFPI into a learnable neural solver is also an interesting direction.

Critical weaknesses
  1. Limited evaluation on established benchmarks. The experiments are primarily conducted on synthetically generated instances. Evaluations on established LP or combinatorial optimization benchmarks would better demonstrate the effectiveness and practical applicability of the proposed method.

  2. Limited baseline comparisons. The current baseline comparisons are relatively limited. It would be helpful to include comparisons with widely used optimization solvers, such as Gurobi, COPT, or SCIP, particularly for the LP experiments, to better assess the competitiveness of the proposed framework.

  3. Concerns about scalability. The proposed method relies heavily on computing the inverse of a matrix, which raises concerns about its scalability to larger problems. Moreover, as shown in Table 13, MFPI-Net does not consistently outperform PDHG-Net, particularly on larger instances. These results raise questions about the advantages of the proposed method for large-scale optimization.

Questions for authors
  1. In Table 1, the classical MFPI algorithm already solves the generated instances within a few seconds. Could the authors evaluate MFPI-Net on more challenging benchmarks to better demonstrate its advantages over conventional numerical methods?

  2. The proposed framework still relies on numerical refinement to obtain high-accuracy solutions. Could the authors provide more detailed statistics on the quality of the solutions produced by MFPI-Net before refinement? For example, what are the residual errors and solution accuracies achieved by the first stage alone?

  3. Could the authors compare MFPI-Net followed by numerical refinement against classical MFPI followed by the same refinement procedure under matched computational budgets? Such a comparison would help clarify how much of the improvement comes from the learned initialization rather than the refinement stage.

6 comments

Page 1 of 4

We thank the reviewer for the careful reading and constructive feedback. We address the concerns and questions below. While we believe that these concerns do not diminish the paper's core contributions, we recognize that we may not have communicated their novelty clearly enough in the manuscript. We therefore conclude our response by clarifying the three central contributions of this work.

Responses to the weaknesses

Response to W1. PageRank is an established application benchmark for LP algorithms, used in the PDLP/PDHG literature [1] and PDHG-Net [2]. Our LP evaluation therefore already follows benchmark practice in this field.

We have additionally evaluated the WA benchmark used by PDHG-Net [2], consisting of LP relaxations of combinatorial optimization problems from ML4CO [3]. Following its data-generation procedure, we use two parameter settings, yielding AVE dimensions of 750 and 2700 after reformulation. The results are:

DatasetAVE dimensionMFPI-Net + PDLP time (s)PDLP time (s)Time reductionMFPI-Net + PDLP iterationsPDLP iterationsIteration reduction
WA-small7500.00790.009214.1%530.56770.5631.1%
WA-large27000.04480.051813.5%1015.681272.3220.2%

These results demonstrate that the learned AVE solutions also provide effective initializations for LP relaxations from a distinct combinatorial optimization benchmark, reducing PDLP iterations by 20.2%–31.1%.

Response to W2. This paper studies neural AVE solving; it is not an LP-specific study aimed at developing a solver that outperforms state-of-the-art commercial or other LP solvers. The LP experiments, together with the additional QP experiments, evaluate MFPI-Net on AVEs obtained from concrete optimization applications. Their central purpose is to demonstrate the network's robustness on these application-derived AVEs and its ability to produce sufficiently accurate approximate solutions to accelerate existing optimization solvers.

To assess the practical value of these approximations, the paper already compares cold starts and MFPI-Net warm starts for cuPDLPx [4] and HPR-LP [5], state-of-the-art GPU LP solvers, alongside PDLP and SCS. To our knowledge, no prior work on algorithm unrolling has reported comparisons with these GPU LP solvers. These comparisons directly test whether the learned AVE solutions reduce the work required by established solvers. We now add COPT 8.07, using its CPU-based barrier and primal simplex methods in the same experimental environment as the paper. The following table reports solving times in seconds on the PageRank LPs:

LP sizeMFPI-Net + cuPDLPxcuPDLPxCOPT barrierCOPT primal simplex
2000.0200.0360.0450.056
5000.0290.0450.2290.546
10000.0610.0710.9513.378

COPT is slower than cuPDLPx at every reported size, with the gap increasing substantially as the problem grows. Thus, the commercial-solver comparison confirms the strength of the existing GPU baselines on these instances.

The decisive evidence is therefore whether the network can solve application-derived AVEs well enough to improve an existing solver's initialization. Wall-clock time also reflects hardware, implementation, preprocessing, and solver-specific heuristics. Holding the refinement solver fixed, the reduction in its iteration count directly measures how much numerical work the learned approximation removes. Table 3 demonstrates this benefit across all four LP solvers and all tested sizes, establishing both the quality of the neural AVE approximations and their practical value for LP solving.

Response to W3. Computing A−1A^{-1} remains a computational bottleneck in the current method. Its cost is included in the reported solving times, and reducing this cost is explicitly identified as future work in the paper.

Table 13 establishes a clear empirical comparison: MFPI-Net outperforms PDHG-Net at the three smaller sizes, while PDHG-Net performs better at the three larger sizes. PDHG-Net directly unrolls the LP-specific algorithm underlying PDLP [1], whereas MFPI-Net addresses AVEs and their LP/QP reformulations. Even at the larger sizes, MFPI-Net reduces PDLP iterations by 32.0%–44.5% and total solving time by 26.9%–36.5%. These are substantial gains over cold-start solving. The paper's central advances concern the theoretical basis for learned approximation and refinement, the practical use of AVE reformulations, and the expansion of MFPI's demonstrated solving scope. Architectural complexity or the largest speedup on every LP size is not the criterion that defines these contributions.

Page 2 of 4

Responses to the questions

Response to Q1. Our data-generation procedure follows established practice in AVE algorithm research, including the MFPI [6] and SOR-like [7] studies. Longer wall-clock runtimes alone do not make a benchmark more informative. The most important scientific result of these experiments is whether the network produces good approximate solutions, reduces the numerical work required, and handles instances that challenge the source algorithm. Our experiments establish all three.

The short MFPI runtimes also reflect our comparison protocol: we implemented MFPI in PyTorch on the same GPU and with the same batch sizes as MFPI-Net, giving the classical baseline the same batching advantages. Even with this acceleration, MFPI requires 2.124 s per instance at n=10000n=10000 in Table 1's lower-accuracy setting, compared with 0.409 s for MFPI-Net. The speedup grows from approximately 3.4×3.4\times at n=1000n=1000 to 5.2×5.2\times at n=10000n=10000. More fundamentally, a single forward pass of MFPI-Net reaches the residual accuracy that requires 40.2–41.0 classical MFPI iterations. This directly demonstrates that the network has learned an effective approximate solution map.

The paper also already includes substantially harder instances. Section 4.2 and Appendix E.6 examine AVEs outside ∥A−1∥2<1\|A^{-1}\|_2<1: the prescribed MFPI parameter rules provide no admissible parameters, and both FPI and cold-start generalized Newton fail to converge within 10,000 iterations on the tested instances. MFPI-Net predicts the correct signs and enables high-accuracy recovery. For the LP-derived AVEs, Appendix E.7 further shows that FPI residuals remain around 10−210^{-2} after 30,000 iterations, whereas MFPI-Net achieves residuals around 10−610^{-6}. These results establish an expansion of solving capability as well as acceleration. Together with the additional WA benchmark in W1, they directly address the request for more challenging evaluation.

Response to Q2. The requested first-stage statistics are already reported separately from refinement. Table 1's left panel evaluates MFPI-Net alone, with RES ranging from 7.68×10−57.68\times10^{-5} to 8.11×10−58.11\times10^{-5}. On the harder instances in Table 2, the network alone achieves RES of 1.13×10−41.13\times10^{-4}–1.15×10−41.15\times10^{-4}. It correctly predicts the solution signs in both settings. Here, RES is the mean squared equation residual, ∥Ax−∣x∣−b∥22/n\|Ax-|x|-b\|_2^2/n.

Appendix E.2 further reports 100% component-wise sign accuracy up to dimension 25,000. In the reported experiments, sign errors arise in the near-zero-component setting of Appendix E.5, where even small prediction errors can change a component's sign. In that setting, first-stage RES is 6.08×10−56.08\times10^{-5}–2.65×10−42.65\times10^{-4} and component-wise sign accuracy is 99.8%–99.9%. This percentage measures individual components, rather than exact recovery of an entire sign pattern. The second stage corrects the remaining errors. Thus, both the accuracy of the neural output itself and the effect of refinement are explicitly quantified.

Response to Q3. We have added a direct comparison that fixes the second-stage sign-correction procedure and varies only the initialization: MFPI-Net output versus classical MFPI stopped after 5, 10, ..., 40 iterations. The experiment uses n=1000n=1000 and the same final stopping threshold as the paper, RES<10−10\mathrm{RES}<10^{-10}. All times below include matrix preparation, the first stage, and the second stage.

First-stage initializerComponent-wise sign accuracySecond-stage iterationsTotal time (s)
MFPI-Net100%10.0048
MFPI, 5 iterations98.42%20.0125
MFPI, 10 iterations99.34%20.0130
MFPI, 15 iterations99.76%20.0136
MFPI, 20 iterations99.89%20.0137
MFPI, 25 iterations100%10.00739
MFPI, 30 iterations100%10.00740
MFPI, 35 iterations100%10.00743
MFPI, 40 iterations100%10.00744

MFPI-Net predicts all signs correctly and requires one refinement step. Among the tested MFPI budgets, 25 iterations are first needed to reach the same sign accuracy and one-step refinement.

Page 3 of 4

Classical MFPI and SOR-like methods are conventionally formulated as standalone iterative solvers, without a separate second stage for sign correction [6,7]. Adding such a stage introduces an additional switching decision: how many classical iterations should be performed before refinement begins? The baseline algorithms do not prescribe this transition, and a fixed iteration count does not by itself certify that the signs are correct. MFPI-Net learns to produce an accurate approximation and reliable signs at a fixed network depth, making it an effective first stage. Table 1 demonstrates its solution accuracy relative to approximately 40 MFPI iterations, and the new comparison separately establishes its value when both initializers receive the same refinement.

MFPI-Net thus delivers a core contribution of algorithm unrolling and learning to optimize: a neural network directly produces high-quality approximate solutions, reducing the subsequent numerical work required to reach high accuracy [8].

Clarification of the paper's novel contributions

We thank the reviewer again for the thoughtful and constructive feedback. The concerns raised do not undermine the paper's core innovations. We recognize that the manuscript may not have communicated the strengths and main contributions of the work clearly enough. We would therefore like to summarize the following three key contributions.

First, a continuous-time analysis of learned initialization and numerical refinement. To our knowledge, this is the first dynamical-systems analysis in deep unrolling to connect a finite-depth error bound to the design of a two-stage solver. By interpreting the MFPI-Net updates as a discretization of a continuous-time dynamical system, Section 3.2 bounds the finite-depth solution error in terms of three interpretable sources. The bound motivates using a moderately deep network to generate a useful initialization and utilizing numerical refinement to attain high accuracy. Prior theoretical work in deep unrolling and learning to optimize has largely focused on approximation within a single learned stage. Our analysis fills this gap by providing a basis for designing and analyzing hybrid optimization solvers.

Second, an empirical demonstration of unrolling beyond the base algorithm's prescribed convergence regime. Our LP and convex QP experiments, together with tests on AVEs satisfying ∥A−1∥2≥1\|A^{-1}\|_2 \geq 1, provide, to our knowledge, the first numerical evidence that a trained unrolled network can produce useful approximations beyond the prescribed convergence regime of its underlying algorithm. In the latter tests, MFPI's standard parameter-selection rules yield no admissible parameters, and cold-start generalized Newton fails to converge within 10,000 iterations. MFPI-Net still produces high-quality approximations that enable numerical refinement. This finding broadens the potential use of deep unrolling by showing how learned initialization can extend the range of problems a solver can address.

Third, an AVE-based warm-start strategy for linear and convex quadratic programming. AVEs can encode the optimality conditions of both LPs and convex QPs, but the difficulty of solving the resulting AVEs with conventional methods has limited the practical use of this reformulation. MFPI-Net makes this route viable by approximating the reformulated AVEs even outside the standard convergence regime. We map these approximate solutions back to high-quality warm starts for conventional LP and QP solvers. The resulting two-stage framework provides a common route to accelerating both classes of convex optimization problems through their AVE reformulations.

We acknowledge that the current method has limitations. In particular, the reviewer correctly identifies matrix inversion as a computational bottleneck that constrains scalability, and addressing this bottleneck remains an important direction for future work. These limitations, however, do not undermine the three core contributions established above. We respectfully ask the reviewer to carefully weigh these contributions alongside the acknowledged limitations and reconsider the assessment in light of the theoretical results and experimental evidence.

Reviewer_MM5GOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper addresses the problem of efficiently solving absolute value equations (AVEs) of the form Ax−∣x∣=bAx - |x| = b. The authors propose MFPI-Net, a neural network based on unrolling MFPI, a well-established algorithm for solving AVEs. MFPI-Net is followed by an efficient refinement stage, which allows solving AVEs to high accuracy using only a small number of MFPI-Net layers, followed by a small number of linear-system solves. Alternatively, for LPs, using MFPI-Net’s output as the initialization for LP solvers is demonstrated to reduce iteration counts and solution time.

The paper carefully explains the implementation of the approach, focusing on the unrolling strategy and the architectural choices for representing MFPI iterations as neural network layers. It provides several theoretical analyses demonstrating the expressive power of the proposed architecture and establishing stability under specified conditions. Experiments on synthetic instances show speedups obtained using the proposed approach, as well as its ability to recover solutions on tested instances outside the standard convergence regime of traditional AVE solvers.

Critical strengths
  1. The paper is well written. As a non-expert reading about AVEs for the first time, I found it self-contained and easy to read. It provides a good exposition of AVEs, explaining both the problem and the proposed approach nicely.

  2. The authors mostly balance the different parts of their work, which include presenting MFPI-Net and its architecture, presenting the theoretical analysis, and reporting their experiments. They also make clear the limitations of those analyses, particularly the lack of guarantees for learnability beyond the expressive-power claims.

Critical weaknesses
  1. I find it hard to position this work within the framework of unrolling optimization algorithms. Namely, is MFPI-Net a straightforward (“textbook”) unrolling of MFPI? If not, what are the specific choices that differentiate it from a straightforward unrolled implementation? While the architectural modifications are described, the current exposition does not seem to highlight sufficiently which choices are distinctive and why they are important.

  2. The experimental results seem limited in their support for broader performance claims. The main experimental setup uses a narrow synthetic distribution, and it is not consistently clear whether the distribution-variation experiments reuse the same trained model or retrain for each setting. While training and testing within a specific distribution could be useful for some applications, this provides limited support for a broader efficiency comparison with other solvers.

In particular, the main experimental setup places solution entries far from zero, which may explain how a small number of MFPI-Net layers is sufficient to determine the sign pattern. While this setup might be easier for other solvers as well, it places the proposed approach at a notable advantage, since AVEs reduce to a linear system once the sign pattern is known. Appendix E.5 usefully considers near-zero solution components, but does not compare the resulting two-stage performance against cold-start solvers.

  1. Several important experiments and implementation clarifications are left largely to the appendix, particularly the near-zero-component experiments, smoothing ablations, and cross-dimension testing details. Although the main text references these results, it does not sufficiently summarize their findings or clarify their implications. These details directly address questions about the architectural choices and the scope of the experimental claims, and would benefit from more explicit discussion in the main text.

  2. Section 3 is a little long given that the guarantees presented in it have limited bearing on the experimental results and success, which seem to be the main focus of the paper. In particular, the theorems in Section 3 would benefit from more context and explanation, preferably before each result is presented. The current explanations mostly follow the results, which I find less convenient for reading.

Questions for authors

Overall, I like this work and am open to increasing my rating after reading the rebuttal. I would like the authors to address the following questions to help me finalize my assessment. First, please address the issues above, primarily: (5) what distinguishes MFPI-Net from straightforward neural-network unrolling of MFPI? And (6) does the experimental setup provide sufficient evidence for MFPI-Net’s performance beyond the tested distributions?

In addition:

  • What happens if K MFPI-Net layers fail to establish the correct sign pattern? Appendix E.5 provides a useful example where refinement corrects remaining sign errors. How often does an incorrect full sign pattern occur, and how do performance and accuracy compare with competing cold-start solvers in these cases? Can you identify situations in which refinement fails or requires substantially more iterations?
  • Is smoothing the absolute-value operator important or required? Appendix E.3 reports substantial degradation when using the exact absolute-value operator. Can you explain this degradation, and whether it persists after separately tuning the training setup for that variant?
  • Why not design a neural network to predict the sign pattern directly, rather than unroll MFPI? Would such a baseline help establish the value of the proposed architecture?
  • Will the code for your implementation be released for reproducibility?
  • Accuracy (RES) seems to be measured in terms of MSE, which makes the stated accuracy levels somewhat misleading. For instance, RES = 10−1010^{-10} corresponds to an RMS residual of 10−510^{-5}, rather than 10−1010^{-10}. Please clarify the interpretation of the reported accuracy levels.
  • Line 145: Please clarify what “channel expansion” means here, including how the initial vector representation is lifted into multiple channels.
  • Line 285: Please explain the assumption that the “pre-activation state X^(k)\hat{X}^{(k)} is uniformly bounded,” and how it relates to the states encountered by the trained network.
  • Line 321: Please clarify in the main text that existence and uniqueness of the equilibrium follow from the stated Lipschitz/contraction assumptions, as established in Theorem D.2.
  • Section 4.1: Appendix E.1 indicates that the model is trained at dimension n = 1000 and then evaluated at other dimensions. Please make explicit in the main text whether Table 1 uses the same trained weights for n = 1000, 2000, 5000, and 10000.
  • More generally, is MFPI-Net dimension-independent in its parameterization, so that the same trained weights can be applied to an AVE of any dimension, provided A is nonsingular? Please distinguish architectural compatibility across dimensions from empirical generalization.
  • For the distribution-variation experiments in Appendix E.4 and the near-zero-component experiment in Appendix E.5, are the original trained weights reused, or is the model retrained for each setting?
8 comments

Page 1 of 6

We thank the reviewer for the positive assessment of the paper's clarity, soundness, and balance between theory and experiments, and for the careful reading and constructive suggestions. The review identifies several points where the presentation should be more explicit. We will revise the manuscript accordingly to clarify the architectural choices, experimental protocol, and scope of the theoretical results. We address the concerns below.

Responses to the weaknesses

Response to W3. Thank you for this thoughtful question on positioning MFPI-Net within the deep unrolling literature. To be frank, we had not framed our work from this exact perspective previously. Although our MFPI-Net follows the standard unrolling framework overall, our several distinctive design choices distinguish MFPI-Net from straightforward, textbook unrolling of MFPI.

First of all, MFPI-Net follows the standard unrolling framework and many components of MFPI-Net build upon existing advances in algorithm unrolling such as channel expansion, smooth approximations of non-smooth operators, and two-stage pipelines combining neural prediction with numerical refinement have been adopted in prior unrolled networks [1,2,3]. We will explicitly clarify these connections and properly cite relevant prior works in the revised manuscript. Meanwhile, to balance computational efficiency with theoretical analysis, we introduce several distinctive design choices, and these specific disign choices distinct MFPI-Net from straightforward, textbook unrolling of MFPI.

  1. We adopt PReLU as the activation function, which, to the best of our knowledge, has not been explored in the context of algorithm unrolling. The learnable negative slopes of PReLU bring extra flexibility for practical training. Meanwhile, fixing these slopes to one recovers the identity mapping required for our constructive approximation proofs.
  2. To balance computational overhead and theoretical tractability, we decouple the coupled parameters from the original MFPI update into two separate learnable modules, and further introduce a regularization term in the objective to minimize reconstruction loss and encourage inverse consistency on sampled states at a moderate computational cost. This inverse consistency is critical for our equilibrium residual analysis.
  3. Before unrolling, we reformulate MFPI into a residual form. This enables us to interpret the network updates from the perspective of continuous-time dynamical systems. To our knowledge, such a dynamical-systems-based analysis for unrolled networks is new to this line of research. The dynamical perspective allows us to quantify how approximation accuracy scales with network depth under our assumptions. It further reveals the trade-off between the depth required to satisfy approximation guarantees and the computational cost of deployed networks. This insight directly motivates our two-stage framework: a moderate-depth neural predictor followed by numerical refinement.

We will highlight these unique design decisions and their theoretical roles prominently in the revised manuscript. The architecture of MFPI-Net follows conventional unrolling principles in general and may be not very delicate. We believe that developing more refined designs and incorporating additional structural priors from dynamical systems are important directions for future work.

Page 2 of 6

Response to W4. We appreciate the reviewer's request to make the training and evaluation protocol more explicit. In the experiments reported in the original manuscript, whenever the spectral or solution distribution changes, we retrain the model for that setting and evaluate it on independently generated instances from the corresponding distribution. Those experiments do not evaluate transfer of one trained model across distribution changes. In contrast, the cross-dimension experiments reuse the same trained weights across problem sizes within the corresponding data-generation family. We will state this distinction explicitly in the main text and in Appendices E.4 and E.5.

This protocol follows a common evaluation setting in learning to optimize: learning a solver for a specified problem family and assessing its performance on new instances, with transfer to larger dimensions examined separately. For example, PDHG-Net for solving LPs [2], LISTA [3] and LISTA-CPSS [4] for solving sparse coding, they all evaluate size generalization using models trained on the corresponding problem families. We also find it challenging to design a meaningful cross-distribution evaluation because we are aware of very few closely related L2O studies reporting such experiments and therefore lack a clear reference protocol [5].

In response to the reviewer's suggestion, we add the following cross-distribution tests. We use the model trained on the main experimental distribution: the singular values of AA follow U(1.01,2)U(1.01,2), and the magnitudes of the solution components follow U(4,6)U(4,6), with half of the entries positive and half negative. We evaluate the same trained model under the distribution shifts below without retraining. Given the robustness observed in our existing experiments across problem dimensions and choices of the two orthogonal matrices used to construct AA, we focus here on changes to the singular-value distribution and the distribution of solution magnitudes and signs. Each test changes one factor relative to the training distribution.

MetricIn-distribution testSingular values U(1.01,2.5)U(1.01,2.5)Singular values U(1.01,3)U(1.01,3)Singular values U(0.8,2)U(0.8,2)Solution magnitudes U(3.5,6.5)U(3.5,6.5)Solution magnitudes U(3,7)U(3,7)Solution magnitudes U(2,8)U(2,8)Solution magnitudes U(4.5,5.5)U(4.5,5.5)Signs: 90% positive, 10% negative
RES8.11e-50.5051.6560.2542.19e-30.01730.1475.33e-44.57e-3
Component-wise sign accuracy100%98.5%97.7%99.1%100%99.94%99.32%100%100%

All tested distribution shifts increase RES relative to the in-distribution baseline, with larger shifts generally causing greater degradation. Sign accuracy nevertheless remains high and stays at 100% in several settings. This trend is intuitive, but we do not yet have a rigorous theoretical explanation: our current theory does not address distribution shifts, and, to our knowledge, closely related L2O studies also offer very limited experimental or theoretical guidance on this issue. We believe that investigating how learning-to-optimize methods perform under distribution shifts and understanding the reasons for their behavior is an interesting direction for future work. We thank the reviewer for inspiring this new research direction.

The near-zero experiment directly examines the concern about solution components being far from zero. Appendix E.5 samples component magnitudes from U(0.001,1)U(0.001,1), where sign errors do occur. Despite these errors, the two-stage method reaches mean squared residuals between 2.05×10−112.05\times10^{-11} and 8.47×10−118.47\times10^{-11} in only 1.8–2.0 refinement steps on average. Thus, its effectiveness on these instances does not require the first stage to identify every sign correctly. The supplementary table below compares performance with cold-start MFPI and SOR-like solvers in this setting.

Supplementary comparison for Appendix E.5

MethodMetricn=1000n=1000n=2000n=2000n=5000n=5000n=10000n=10000
MFPI-NetRES6.08e-59.37e-51.22e-42.65e-4
SA0.9990.9980.9980.998
Two-stage MFPI-NetTime (s)0.0260.0410.3861.673
Iters1.81.952.02.0
RES2.05e-118.47e-114.43e-112.93e-11
MFPITime (s)0.03770.07680.6243.139
Iters82.581.381.580.2
RES8.80e-118.66e-118.45e-118.93e-11
SOR-likeTime (s)0.03320.07930.6383.217
Iters127.0129.8127.6127.4
RES9.05e-119.22e-119.77e-119.49e-11

SA denotes component-wise sign accuracy. Iters denotes the average number of iterations; for two-stage MFPI-Net, it counts only the refinement iterations. Time includes both stages for two-stage MFPI-Net.

Page 3 of 6

Response to W5. We thank the reviewer for highlighting these results and agree that they deserve greater visibility. We will move the near-zero-component and cross-dimension evaluations from Appendices E.5 and E.2 into the main experimental section, including the supplementary cold-start comparisons for the near-zero instances. We will explain that refinement can correct imperfect sign predictions on these instances and that the cross-dimension results use unchanged weights trained at n=1000n=1000. We will also summarize the smoothing ablation and its implications in the main text, retaining the detailed ablation results in the appendix. Moving the current Section 3.1 analysis to the appendix, as described below, will create space for these experimental results.

Response to W6. We thank the reviewer for this suggestion and agree that the theory section should be more focused and easier to follow. We will move the approximation and stability analysis in the current Section 3.1 to the appendix, retaining a brief summary in the main text. These results provide supporting guarantees within the established unrolling framework but offer limited direct guidance for practical network design. The main theoretical discussion will focus on a clearer explanation of the dynamical-systems analysis in the current Section 3.2. We regard this perspective, which connects finite-depth approximation with the motivation for numerical refinement, as the paper's main theoretical contribution. Before each theorem, we will explain its motivation, assumptions, and intended implication. In particular, we will clarify how trajectory approximation, convergence to equilibrium, and module consistency jointly inform the depth–accuracy trade-off and the two-stage strategy. We will also distinguish these expressive-capacity guarantees from guarantees about the parameters obtained through training.

Responses to the questions

We appreciate the reviewer identifying these two central questions. As discussed in W3, MFPI-Net is not a trivial textbook unrolling of the MFPI.

Beyond the design choices discussed in W3, MFPI-Net has several distinguishing features. (1) Our continuous-time dynamical-systems analysis provides a theoretical foundation for combining learned initialization with numerical refinement. (2) The trained MFPI-Net can also produce useful approximations beyond MFPI's prescribed convergence regime, extending the practical solving scope of the underlying algorithm. (3) Combining this capability with AVE reformulations of convex optimality conditions, we establish a new route to solving LPs and convex QPs through high-quality learned warm starts for conventional numerical solvers. To our knowledge, these contributions are absent from prior unrolling studies. As clarified in W4, the original manuscript evaluates generalization across dimensions but retrains the model when the data distribution changes. We have now added tests that reuse the same trained model across distribution shifts and will incorporate these results into the revised manuscript.

Response to Q1. We thank the reviewer for raising this important question about the limits of refinement. For instances satisfying ∥A−1∥2<1\|A^{-1}\|_2<1, the second stage is guaranteed to converge when the sufficient condition ∥A−1∥2<1\|A^{-1}\|_2<1 hold [6], while the first-stage prediction can reduce the number of refinement iterations.

For the near-zero instances in Appendix E.5, the network achieves 99.8%–99.9% component-wise sign accuracy, and refinement corrects the remaining errors in 1.8–2.0 iterations on average. The final mean squared residuals are 2.05×10−112.05\times10^{-11}–8.47×10−118.47\times10^{-11}. Component-wise sign accuracy and exact recovery of an entire sign vector are different statistics; a small fraction of incorrect components can still affect many high-dimensional instances. The full-pattern recovery rates are reported separately below:

Dimension nnFraction of instances with every sign correctFraction of instances with at least one sign errorComponent-wise sign accuracy
100017%83%99.83%
20005%95%99.81%
50000%100%99.82%
100000%100%99.82%
Reviewer_4vDuOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes MFPI-Net, a neural surrogate model for solving absolute value equations that unrolls the Modified Fixed-Point Iteration (MFPI) algorithm into learnable residual updates. The algorithm is empirically validated on instances with up to 25,000 dimensions, and also as a warm-start method for LP solvers. Theoretical approximation and stability analyses/guarantees are provided. The method achieves up to 3x inference-time speedups over MFPI while retaining high accuracy and successfully recovering solutions on instances outside the standard convergence conditions of classical AVE solvers.

Critical strengths

The paper introduces a theoretically grounded algorithm for solving AVEs, with extensive ablations and promising results demonstrating scalability and meaningful speedups over the baselines. It is also very well-written and well-motivated.

Critical weaknesses

My main concerns are about novelty and incomplete performance comparisons. It would be good to compare against the following 2 baselines:

  1. “SOR-Unrolled Learning-to-Solve Method for Absolute Value Equations” (https://link.springer.com/article/10.1007/s40305-025-00670-4). The methods share many methodological similarities. Therefore, it would help to benchmark against this paper. It would also help to clearly outline the methodological differences between the two and establish the novelty of MFPI-Net relative to SOR-Net.
  2. Generalized Newton method. While the authors explain why they omitted it in Appendix E.6, including it could provide a more complete picture of the literature landscape and how scalable/fast each method is.

These are the main concerns preventing me from giving the paper a much higher score.

Questions for authors

Could the authors address the concerns outlined in the “Critical Weaknesses” section?

4 comments
AuthorsOpenReview ↗

## Page 1 of 2

Page 1 of 2

We thank the reviewer for the careful reading and constructive feedback. We address the concerns and questions below. While we believe that these concerns do not diminish the paper's core contributions, we recognize that we may not have communicated their novelty clearly enough in the manuscript. We therefore conclude our response by clarifying the three central contributions of this work.

1. Comparison with SOR-Net

Both MFPI and the SOR-like method underlying SOR-Net belong to the broad family of fixed-point algorithms for AVEs. Our MFPI-based neural framework is more general: SOR-Net's updates are recovered by setting the paired modules to identity maps, tying the two relaxation matrices, and choosing ReLU activations. The learned paired operators and independently parameterized residual updates therefore extend the SOR-based construction.

More substantially, our theoretical analysis includes the core universal-approximation perspective underlying SOR-Net and develops it into guarantees for the proposed architecture and its associated dynamics. First, Theorem 2 establishes that the specific MFPI-Net architecture can approximate any finite MFPI trajectory under the stated conditions, connecting the approximation argument to the network actually proposed. Second, Theorem 3 quantifies how input-state perturbations propagate through a complete learned layer. Third, Section 3.2 develops a dynamical-systems analysis that decomposes the finite-depth solution error into three sources: trajectory discretization, distance to equilibrium, and module inconsistency. This analysis also relates inverse-consistency errors to the AVE residual at equilibrium, providing a theoretical justification for the reconstruction regularizer.

These additional results connect architectural design, approximation accuracy, and the motivation for numerical refinement. In particular, the error decomposition clarifies what increasing network depth controls and why subsequent refinement is useful for obtaining high-accuracy solutions. This quantitative connection is absent from SOR-Net's analysis and constitutes a central theoretical contribution of our work, complementing the shared iteration-approximation framework.

We acknowledge that both methods unroll fixed-point iterations and use a two-stage strategy for high accuracy. These shared ingredients alone do not establish that their contributions are substantially the same: fixed-point iteration is a broad algorithmic class that also includes ISTA for LASSO [1] and standard PDHG for linear programming [2], both widely used in unrolling. The key distinctions are the more general learned updates and the substantially broader theoretical analysis.

Our experimental evaluation substantially broadens the scope of SOR-Net's reported study along three dimensions. First, AVEs beyond the standard sufficient condition for unique solvability. SOR-Net's experiments focus on instances satisfying ∥A−1∥2<1\|A^{-1}\|_2<1. We additionally evaluate instances with ∥A−1∥2≥1\|A^{-1}\|_2\geq1, on which MFPI-Net followed by numerical refinement successfully recovers high-accuracy solutions (Section 4.2). This extends the empirical investigation beyond the original MFPI algorithm's prescribed convergence regime.

Second, applications through optimization reformulations. Beyond directly generated synthetic AVEs, we evaluate AVEs derived from LP and convex QP problems and assess whether their learned approximations provide useful warm starts for conventional optimization solvers. In particular, the LP reformulation yields ∥A−1∥2=1\|A^{-1}\|_2=1 (Appendix B), connecting the expanded spectral regime to an optimization application. These experiments investigate the downstream utility of AVE approximations, an aspect not evaluated in SOR-Net's reported study.

Third, more varied spectral properties and solution structures. SOR-Net uses solution components with fixed magnitudes of 5. Our main experiments use magnitudes sampled from U(4,6)U(4,6), and additional evaluations cover alternative singular-value distributions, ill-conditioned matrices, asymmetric sign distributions, and near-zero solution components (Appendices E.4 and E.5). The near-zero experiments further test numerical refinement when the network's initial sign predictions are imperfect, examining a more demanding setting than exact sign recovery for solutions bounded away from zero.

Page 2 of 2

2. Comparison with generalized Newton

The table reports time in seconds and mean squared residual RES=∥Ax−∣x∣−b∥22/n\mathrm{RES}=\|Ax-|x|-b\|_2^2/n. Net denotes MFPI-Net at threshold 10−410^{-4}; Net+ref. denotes two-stage MFPI-Net at 10−1010^{-10}, including prediction and refinement time. Newton returns the same results at both thresholds.

nnNet timeNet RESNet+ref. timeNet+ref. RESNewton timeNewton RES
10000.00378.11e-50.00482.05e-110.00552.04e-11
20000.01097.98e-50.01766.82e-110.01846.82e-11
50000.1057.68e-50.1885.12e-110.2135.12e-11
100000.4097.91e-50.6509.63e-110.8649.63e-11

On these relatively easy instances, Newton typically converges in about three iterations, leaving limited room for acceleration. This explains the modest timing differences at high accuracy.

The more consequential advantage is solving challenging instances on which conventional methods fail. For the AVEs with ∥A−1∥2≥1\|A^{-1}\|_2\geq1 in Section 4.2 and Appendix E.6, basic FPI and cold-start Newton fail within 10,000 iterations, while MFPI's prescribed parameter rules yield no admissible parameters. MFPI-Net predicts the solution sign patterns, enabling one Newton refinement step to attain residuals of order 10−910^{-9}. Learned initialization thus makes high-accuracy refinement effective where cold-start Newton fails.

3. Clarification of the paper's novel contributions

We thank the reviewer again for the thoughtful and constructive feedback. We recognize that the manuscript may not have communicated the strengths and main contributions of the work clearly enough. We would therefore like to summarize the following three key contributions.

First, a continuous-time analysis of learned initialization and numerical refinement. To our knowledge, this is the first dynamical-systems analysis in deep unrolling to connect a finite-depth error bound to the design of a two-stage solver. By interpreting the MFPI-Net updates as a discretization of a continuous-time dynamical system, Section 3.2 bounds the finite-depth solution error in terms of three interpretable sources. The bound motivates using a moderately deep network to generate a useful initialization and utilizing numerical refinement to attain high accuracy. Prior theoretical work in deep unrolling and learning to optimize has largely focused on approximation within a single learned stage. Our analysis fills this gap by providing a basis for designing and analyzing hybrid optimization solvers.

Second, an empirical demonstration of unrolling beyond the base algorithm's prescribed convergence regime. Our LP and convex QP experiments, together with tests on AVEs satisfying ∥A−1∥2≥1\|A^{-1}\|_2 \geq 1, provide, to our knowledge, the first numerical evidence that a trained unrolled network can produce useful approximations beyond the prescribed convergence regime of its underlying algorithm. In the latter tests, MFPI's standard parameter-selection rules yield no admissible parameters, and cold-start generalized Newton fails to converge within 10,000 iterations. MFPI-Net still produces high-quality approximations that enable numerical refinement. This finding broadens the potential use of deep unrolling by showing how learned initialization can extend the range of problems a solver can address.

Third, an AVE-based warm-start strategy for linear and convex quadratic programming. AVEs can encode the optimality conditions of both LPs and convex QPs, but the difficulty of solving the resulting AVEs with conventional methods has limited the practical use of this reformulation. MFPI-Net makes this route viable by approximating the reformulated AVEs even outside the standard convergence regime. We map these approximate solutions back to high-quality warm starts for conventional LP and QP solvers. The resulting two-stage framework provides a common route to accelerating both classes of convex optimization problems through their AVE reformulations.

These theoretical advances and expanded solving capabilities are central to the paper's novelty, alongside the shared unrolling framework. We respectfully ask the reviewer to reconsider the assessment in light of these contributions and the additional comparison.

Reference

[1] Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. Proceedings of the 27th International Conference on International Conference on Machine Learning

[2]Bingheng Li, Linxin Yang, Yupeng Chen, Senmiao Wang, Haitao Mao, Qian Chen, Yao Ma, Akang Wang, Tian Ding, Jiliang Tang, et al. Pdhg-unrolled learning-to-optimize method for large-scale linear programming. Proceedings of the 41st International Conference on Machine Learning,

[3]Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang, and Wotao Yin. Learning to optimize: A primer and a benchmark. Journal of Machine Learning Research

Thank you for addressing my concerns and clarifying the novelty of the work.

Overall, I like this work and appreciate the in-depth analyses, experiments, and clearer differentiation from SOR-Net. As I mentioned in my original review, I think the work is well motivated and well written.

In light of the authors' responses to my comments and concerns, and those raised by other reviewers, I will raise my score to a 3.

Thank you for your positive feedback and support. We sincerely appreciate your thoughtful comments and constructive suggestions, which have broadened our perspective and helped us improve the clarity and quality of the manuscript. We have revised the paper accordingly and are grateful for the time and care you have devoted to reviewing our work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thanks to reviewers and Area Chairs

Dear Area Chairs and Reviewers,

We thank the Area Chairs for overseeing the review process and all the reviewers for the time and care they devoted to reading our paper and discussing our responses. Your questions helped us see where readers needed more context, and your suggestions prompted us to revisit both the presentation and the evaluation of our work. We have revised the manuscript in five main ways.

  1. Introduction and contributions (mainly page 2). We substantially rewrote the latter half of the introduction to clarify how the design of MFPI-Net, its continuous-time analysis, and the two-stage solver fit together. We also revised the contribution summary to state more clearly and precisely what is new in our work.

  2. Network design (Section 2.2 and Appendix D). We clarified how MFPI-Net builds on the original MFPI iteration and why we made its main architectural choices. We added Appendix D to describe the network design in greater detail.

  3. Theoretical analysis (Sections 2.2 and 2.3, and the Appendix E). For the convergence analysis of the unrolled scheme, we retained the original main theorem in the main text, now in Section 2.2, and moved the other results formerly presented in Section 3.1 to the appendix. The dynamical-systems analysis, which is a distinctive feature of our work, remains in the main text and has been moved from the former Section 3.2 to Section 2.3. Accordingly, the standalone “Theoretical Analysis” section has been removed. Following the reviewers’ suggestions, we also added an explanation before each theorem to clarify its motivation, purpose, and role in the overall analysis, making the theoretical discussion easier to follow.

  4. Performance across settings (Section 3.3). We added a Section 3.3 to the main text examining how MFPI-Net performs as the problem dimension or data distribution changes. This subsection also investigates how reliably the network predicts signs when solution components are close to zero.

  5. Additional baselines and benchmarks (Appendices F.2, F.6, and F.8). We added baseline comparisons with the generalized Newton method for absolute value equations in Appendix F.2 and with COPT for PageRank linear programs in Appendix F.6. In Appendix F.8, we also evaluated the learned warm starts on the WA benchmark, using linear programming relaxations of combinatorial optimization problems.

We are grateful for the opportunity to discuss and improve the paper. We hope that the revised manuscript, together with the pointers above, makes the changes easy to locate and is helpful for your assessment.

Best regards,
The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.