acceptodds
Desk rejected at ICLR 2027

FastDFL: Reusable local QP solution maps for decision-focused learning

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Decision-focused learning (DFL) trains predictors through downstream optimization problems, but repeatedly solving the downstream problem and differentiating its solution make training expensive. We study DFL for strongly convex quadratic programs (QPs) with a fixed Hessian and feasible set, where predictions enter the linear term of the objective. During training, evolving predictions can revisit regions where the optimal decision is affine in the prediction and its Jacobian is constant. We propose FastDFL to turn this recurrence into reusable computation. FastDFL constructs a local formula only after the same constraint structure is observed twice, verifies its validity before reuse, and limits the cost of retrieving previously useful formulas. Whenever a formula passes the Karush-Kuhn-Tucker (KKT) validity checks, it returns the same decision and Jacobian as a fresh QP solve and differentiation. Across three benchmarks and two predictor architectures, FastDFL replaces more than half of the QP solves and derivative computations in every setting. It makes end-to-end training 1.35 to 3.47 times as fast as an otherwise identical procedure that solves each QP and differentiates its solution, while leaving decision quality nearly unchanged.

Primary area:
optimization
Keywords:
decision-focused learning, differentiable optimization, quadratic programming, optimization layer acceleration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YBTROpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the cost of training predictors through downstream optimization problems. It focuses on strongly convex QPs with fixed Hessians and constraints, where predictions enter only the linear objective term. Exploiting their piecewise-affine solution maps, FastDFL stores local solution formulas and Jacobians and reuses them when predictions satisfy region-validity checks. Experiments demonstrate reduced computation across three tasks and two predictor architectures.

Critical strengths
  • The paper is well written and structured, with relevant background, clear motivation, and an understandable presentation.

  • The experiments demonstrate consistent acceleration. The matched comparison and construction/retrieval ablations are useful.

  • Bringing regional QP/MPC reuse into differentiable learning is a useful direction for taking advantage of recurring optimization structure.

Critical weaknesses

I generally like this work. My concerns mainly require clarification, and I would consider increasing my score if they are addressed.

  • Novelty relative to regional QP/MPC reuse. The contribution appears to be focused on DFL-specific integration and cache-management policies, rather than the underlying reuse mechanism or constant-Jacobian property. Comparisons against methods that solve every QP establish the benefit of reuse more clearly than the improvement over existing MPC regional-reuse approaches. Please clarify the differences. Can an MPC baseline be added to the experiments?

  • Restricted parameterization. How limiting are the fixed Hessian and feasible-set assumptions for intended applications? Please distinguish extensions preserving exact affine reuse from those requiring different approximations or validity checks.

  • Scalability. Appendix E.3 reports substantial memory costs, but tests only up to 192 decision variables with prediction dimension fixed at 24. Please discuss larger QPs, higher-dimensional predictions, and larger datasets, including per-sample storage costs.

  • Solver choice. The speedups are useful but mostly moderate outside M5. Although several solvers are compared, could comparable gains come from changing the backend of competing differentiable methods? Have alternative supported backends been explored, or the selected configurations shown to be competitive?

  • Failure modes. Please characterize when construction, retrieval, and storage costs outweigh avoided solves. The conditional efficiency analysis offers limited practical guidance on when FastDFL should help.

Questions for authors
  1. Could the benchmark in Appendix D.1 be emphasized? Which modeling choices are necessary to fit the framework, and which define the testbeds? Fama is semi-synthetic, while Battery and M5 use particular downstream QP formulations.

  2. Why is decision quality only “nearly unchanged” despite the exact-equivalence result? Appendix E.2 reports rare but material Jacobian discrepancies. How are these diagnosed, and could checks detect them?

  3. Why use q0+Qaq_0+Qa, rather than predicting the linear objective coefficient directly?

  4. How do construction checks distinguish redundant zero-multiplier constraints from weakly active, nonredundant constraints at nondifferentiable points? What ensures differentiability without strict complementarity, and how is an affine multiplier map selected when dual solutions are nonunique?

  5. A region-specific notation for the constant Jacobians in Eqs. (6),(7), or evaluation at aˉ\bar{a}, might improve clarity. This is a notation suggestion, not a mathematical objection.

  6. Please separate the high-level reuse idea from construction/retrieval details more clearly, especially in Section 4.2. Measuring the quantities in the conditional efficiency analysis would also connect the bounds to observed behavior.

8 comments
AuthorsOpenReview ↗

Response to W1, W2&Q3

Dear Reviewer YBTR,

Thanks for your insightful comments! Below are our point-by-point responses:

To W1:

We agree that local affine solution maps and their constant Jacobians are established foundations. Our proposed FastDFL contributes the decisions governing formula construction and retrieval during DFL training. It constructs maps when constraint structures recur, uses each sample's formula history, and budgets searches of older formulas. We have clarified this contribution in Section 1, lines 58–60 of the revised manuscript.

These decisions exploit repeated visits to individual training samples. A sample can return to an earlier region after visiting other regions, allowing its history to recover a valid stored formula. A structure encountered only once can incur construction cost without subsequent reuse. Waiting for recurrence filters these constructions, while the search budget controls the cost of checking older formulas.

We add a regional MPC method for comparison. Regional MPC with active set updates (König and Mönnigmann, 2017) reuses the current region and updates active constraints between successive predictions. It constructs updated maps through the common KKT implementation and uses DAQP when an update is unavailable. The folllwing table shows the results of Regional MPC compared with FastDFL.

Task / predictorRegional MPC time (s)FastDFL time (s)Speedup (mean ratio)Constructions (Regional / FastDFL)Formula processing (Regional / FastDFL, s)
Fama / Linear36.05 ± 2.1024.4 ± 0.41.56222.6 / 567.219.59 / 2.00
Fama / MLP34.92 ± 1.4822.6 ± 0.81.59386.0 / 55.020.72 / 1.13
Battery / Linear75.64 ± 0.6747.2 ± 0.51.621131.2 / 3406.628.71 / 6.05
Battery / MLP67.77 ± 0.3529.0 ± 0.52.326211.8 / 755.827.98 / 2.08
M5 / Linear21.94 ± 0.359.15 ± 0.052.48152.4 / 172.015.16 / 2.18
M5 / MLP45.00 ± 0.9317.4 ± 0.22.624925.2 / 391.637.37 / 2.53

On Fama and M5, Regional MPC avoids more complete solves, while repeated map construction requires more processing. On the Battery task, unavailable regional updates also lead to direct solves. Actually, the ablation study in our original manuscript complements this comparison by identifying the construction and retrieval policies' effects. We have added Regional MPC to Section 5.2 (Table 1), lines 419–426, and Appendix E.1 (Table 9), lines 1261–1268 of the revised manuscript.


To W2 and Q3:

The fixed Hessian and feasible set define the present framework's scope. FastDFL targets prediction of cost vectors for a fixed decision problem, including returns under a fixed portfolio risk model and prices under fixed storage dynamics and limits. These assumptions fix the local KKT matrix, allowing reuse of the decision map and Jacobian when constraint structures recur.

The form q0+Qaq_0+Qa also includes direct prediction of the complete linear coefficient vector. Setting q0=0q_0=0 and Q=IQ=I (identity matrix) gives q(a)=aq(a)=a. The more general form expresses known task relationships. In the Battery task, aa contains 24 hourly prices, QQ places each price in the power coefficients of both storage units, and q0q_0 supplies fixed coefficients. This representation keeps duplicate coefficients and known constants in the decision model.

The structural requirement for exact affine reuse is a fixed reduced KKT matrix with an affine on the right hand side. Fixed P,A,GP,A,G with affine q(a),b(a),h(a)q(a),b(a),h(a) can retain that structure, subject to corresponding construction and validity checks. General predictions of Hessians or constraint matrices change the KKT matrix and require different local maps, checks or approximations. We identify these extensions as open issues in the Discussion.

AuthorsOpenReview ↗

Response to W3 and W4

To W3:

We separate the effects of prediction dimension, QP size and sample count on training time and storage, and have updated the QP scale and resource results in Appendix E.3, lines 1352–1356 and 1383–1395, and added the prediction dimension study in lines 1396–1403 of the revised manuscript.

For prediction dimension, we vary the cost coordinates while holding the QP structure fixed. The Battery formulation uses the original features and prices, with n=192,m=528,N=1313n=192,m=528,N=1313. Predictions specify coefficients of a fixed basis for costs across storage units and hours:

ppTraining speedupFormula arrays (MiB)
122.10143.0
241.76336.4
361.62482.5
481.53643.2
961.34938.6

Higher prediction dimension increases map storage, while FastDFL retains a time advantage through p=96p=96. At every fixed pp, mean speedup also increases at the recorded checkpoints from epoch 5 to 40:

For QP size, we fix p=24,N=438p=24,N=438 and compare the following scales.

(n,m)(n,m)Speedup, epoch 20Speedup, epoch 40Computation reduction, epoch 40 (%)
(96,288)1.281.3667.7
(144,408)1.251.2968.8
(192,528)1.241.2768.6

The time advantage increases at all three scales as training proceeds. By epoch 40, each scale replaces about 2/3 of solver and derivative computations.

For dataset size, we use subsets of the original 1,313-instance. This comparison measures both complete training time and the storage associated with individual samples.

NNDAQP–KKT time (s)FastDFL time (s)SpeedupFormula arrays (MiB)History bytes/sample
32074.86 ± 2.4653.35 ± 3.341.41 ± 0.0756.1104.3
640124.25 ± 4.9772.02 ± 2.351.73 ± 0.04128.9110.6
1280215.93 ± 2.05122.65 ± 1.031.76 ± 0.02330.3124.7

Storage for an individual sample is distinct from the shared formula library. Histories consume about 104 to 125 bytes per sample for references and containers. Native solver state buffers add about 6 KiB per sample in both methods. Shared formula arrays account for the larger library cost. The sample count experiment also measures the peak memory of the complete process. At N=1280N=1280, its means are 1170.1 MiB for DAQP–KKT and 1516.8 MiB for FastDFL. Together, the measurements expose the training time advantage of FastDFL, shared storage and smaller history cost for each sample, supporting a resource choice within the workload limit.


To W4:

We agree that backend choice can change the cost of differentiable optimization. We examine both solver selection and reuse with the solver held fixed. We evaluate the official dQP with OSQP and DAQP, retaining its original backward implementation, and test FastDFL with OSQP alongside its DAQP backend.

TaskBackenddQP time (s)Backend–KKT time (s)FastDFL time (s)
FamaOSQP139.56±0.56139.56\pm0.5651.70±0.7751.70\pm0.7731.18±0.6931.18\pm0.69
FamaDAQP69.91±0.3469.91\pm0.3435.94±0.2835.94\pm0.2824.4±0.424.4\pm0.4
BatteryOSQP325.32±2.17325.32\pm2.1769.96±1.0269.96\pm1.0255.51±0.7255.51\pm0.72
BatteryDAQP267.58±3.04267.58\pm3.0463.58±0.3763.58\pm0.3747.2±0.547.2\pm0.5
M5OSQP192.93±0.69192.93\pm0.6926.33±0.9526.33\pm0.9510.33±0.2610.33\pm0.26
M5DAQP82.99±0.6682.99\pm0.6631.72±0.1631.72\pm0.169.15±0.059.15\pm0.05

It can be found that switching dQP to DAQP reduces its training time, while FastDFL remains faster with that same backend. To examine reuse directly, the OSQP–KKT comparison holds the forward solver and direct KKT backward computation fixed. Its mean speedups are 1.66, 1.26 and 2.55 times on Fama, Battery and M5. Solver selection and formula reuse address complementary costs. The former lowers the cost of each direct computation, and the latter reduces the number of direct computations required. We have added this comparison in Appendix E.4 (Table 12), lines 1427–1448 of the revised manuscript.

AuthorsOpenReview ↗

Response to W5

To W5:

For FastDFL, formula construction is an upfront investment, and subsequent uses determine its return. Consider a sequence involving 5 complete binding sets \textbf{ABCDE}. Each optimization layer evaluation includes a forward decision and a backward computation. Assume direct computation costs 1 s, lookup and validity checks cost 0.1 s per evaluation, and construction adds 0.5 s. Formula evaluation for the forward and backward computations costs 0.1 s, giving a total of 0.2 s for successful reuse. Then we have the following result:

Successive evaluationsDirect computation (s)FastDFL (s)Stored formulas (used formulas)
AA2.02.71 (0)
AAAA4.00.81 (1)
BBCCDDEE8.010.85 (1)
Total14.014.35 (1)

During AA, both evaluations require direct computation, and the second constructs A's formula. The next AAAA reuse it, bringing cumulative time to 3.5 s against 6 s for direct computation. Each of BCDE then appears twice, triggering construction without subsequent reuse. These evaluations add 2.8 s of overhead and erase the earlier 2.5 s saving. Here, 4/5 stored formulas supply no avoided computation. The second observation triggers construction, while subsequent uses determine its return. Recurrence saves time when retrieval and formula evaluation remain cheaper than direct computation.

Practical guidance is to assess subsequent formula use together with construction time, lookup time and memory growth. The QP formulation and data distribution affect the complete binding sets encountered. The training split, sample order and predictor updates affect recurrence and retrieval order. QP size and backend determine the relative costs of direct computation and reuse, while available memory limits library growth. Existing training records therefore support an assessment for the intended workload. More importantly, dynamically removing formulas with little subsequent use under a memory budget remains a promising future direction. We have added the cost insight to Section 6, lines 493–496, and the practical guidance to Appendix E.3, lines 1420–1424 of the revised manuscript.

Reviewer_nDjCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes an approach to improve the computation cost of decision-focused learning for strongly convex programs by reusing the decision of previous QP solution with the same constraint structure. The main idea is to construct a library of local formulas consisting of the QP solution and its Jacobian for caching constraint structures, i.e., history library, and bypass solving the recurring QP problem in decision-focused learning once its constraint structure represented by KKT matrix matches any entry in the history library. The authors present validity check formulas, which bound numerical error propagation in case of reusing the solution from the history library and characterize the cumulative cost of the framework involving the search cost. In the experiment, the proposed framework is compared with extensive differentiable QP baselines in three different tasks.

Critical strengths
  • The motivation of reducing the computation cost of solving the downstream task is solid.
  • The core idea of skipping the optimization by reusing the previous solution in case of the same constraint structure is simple and valid.
  • The analyses on the cumulative cost and characterization on the precision bound in the forward and backward pass provide a theoretical ground of the proposed framework.
  • The experiments with extensive baselines show the efficacy of the proposed method.
  • The idea of combining piecewise affine laws and caching the KKT factorization is novel.
Critical weaknesses

Scope limitations

  • The proposed method is restricted to the problems in which the Hessian is fixed and the prediction only enters in the linear term of the objective. However, there is no guarantee that actual downstream tasks satisfy these conditions.
  • This paper focuses on reducing the computation cost of training in decision-focused learning. Although improving the efficiency of training is a valid contribution, speed is of more importance in test time than in training. This work does not cover the test time.

Result

  • The paper suggests that the end-to-end training becomes 1.35 to 3.47 times faster by using the proposed method. While this is a modest speed improvement, the actual benefit is uncertain in most of the cases where the training computation time constraint is not tight.
  • Comparison with other QP reuse method is not included in the experiment section. It would be more convincing if the proposed method is compared with at least one of the methods in Appendix A.

Tradeoff

  • In case of large QP problems, formula library F can grow large.
  • In Appendix E.4, the memory cost result is introduced, and almost no speed improvement is achieved when the problem becomes large.
Questions for authors
  • Please see the weaknesses section.
  • Based on which QP algorithm is the proposed method developed?
  • What is the formal definition of di,kd_{i, k} in Section 4.2?
  • Which data structure does the formula library use and how is the history search performed?
7 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Weakness 2

Dear Reviewer nDjC,

Thanks for your detailed feedback! We make the following discussion and revisions:

To Weakness 1 (Scope limitations): We agree that the application conditions deserve a clear explanation. FastDFL accelerates training when objective coefficients are predicted for a fixed quadratic decision model. Specifically, setting q0=0q_0=0 and Q=IQ=I (the identity matrix) allows prediction of the entire linear coefficient vector. Expected returns under a fixed portfolio risk model, electricity prices under storage constraints, and allocation costs provide concrete examples. Appendix D.1 specifies these modeling choices. We have clarified Section 5.1, lines 382–385 of the revised manuscript, to identify the benchmarks as QPs constructed from public data. Predicted Hessians and changing feasible sets are extensions for future work.

As for speed-up on test set, efficient deployment is important, but FastDFL focuses on the repeated optimization work that makes DFL training costly. During training, each sample receives evolving predictions over successive epochs. Its history Hi\mathcal H_i helps recover formulas for regions visited earlier, and each valid reuse replaces both a QP solve and KKT differentiation. While on test set, new test samples arrive without these accumulated histories, so deployment requires a separate policy for retrieving formulas from the shared library.


To Weakness 2 (Result):

We agree that practical value depends on the workload and the resources spent on reuse. FastDFL lowers the cost of training with the existing decision loss. The direct DAQP–KKT comparison holds the forward solve and KKT backward computation fixed and reports mean speedups of 1.35 to 3.47 times with nearly unchanged decision quality. These savings also benefit repeated fitting and model selection when completion time is flexible. The relevant choice combines the saved training time with formula memory and management costs.

Following your suggestion, we add a regional reuse comparison from Appendix A. We independently implement a DFL adaptation of Regional MPC with active set updates by König and Mönnigmann (2017). It reuses the current map while valid, updates active constraints between neighboring regions and uses DAQP when an update is unavailable. Both approaches use the common KKT backward interface. The reported training times are:

Task / predictorRegional MPC time (s)FastDFL time (s)Speedup (mean)
Fama / Linear36.05 ± 2.1024.4 ± 0.41.5
Fama / MLP34.92 ± 1.4822.6 ± 0.81.5
Battery / Linear75.64 ± 0.6747.2 ± 0.51.6
Battery / MLP67.77 ± 0.3529.0 ± 0.52.3
M5 / Linear21.94 ± 0.359.15 ± 0.052.4
M5 / MLP45.00 ± 0.9317.4 ± 0.22.6
FastDFL has a lower reported mean time in every task and predictor combination.

Your suggestion also helps us make the solver-free comparison in Appendix A more precise. The previously cited method of Berden et al. (2025) consider LP specifically, it cannot be applied to QP, so we change a new solver-free method for DFL -- LODL (Table 4, lines 718–719 of the revised manuscript, now identifies LODL by Shah et al. (2022).) LODL perturbs training targets, solves the original QPs for regret labels and fits local losses for individual training instances. Predictor updates then differentiate these learned losses. Its preparation therefore contributes to the cost of obtaining a trained model. FastDFL constructs its formulas during training, with construction, validity checks and retrieval included in its timer. The results of the new LODL baseline are also added to the revised manuscript for your reference (See line 420 in Table 1 for linear predictor and line 1262 in Table 9 for MLP predictor.)

AuthorsOpenReview ↗

Response to Weakness 3 and Questions 2-4

To Weakness 3 (Tradeoff):

We agree with you that formula reuse involves a trade-off between training time and memory. The formula library growth has two sources, more retained formulas and larger maps within each formula. Each map stores decisions, multipliers and constraint slacks. Increasing the prediction dimension, decision dimension or number of constraints can enlarge these maps. Encountering more recurring structures can also increase the retained count. The current management policy prioritizes training time, with a fixed memory budget remaining an extension.

1. The QP scale study shows that time savings continue to accumulate as training proceeds. With p=24,N=438p=24,N=438 fixed, the mean results are below. We have updated these results in Appendix E.3, lines 1352–1356 and 1383–1395, and have added the prediction dimension study in lines 1396–1403 of the revised manuscript.

(n,m)(n,m)Speedup at epoch 20Speedup at epoch 40Computation reduction at epoch 40 (%)
(96,288)1.281.3667.7
(144,408)1.251.2968.8
(192,528)1.241.2768.6

The time advantage increases from epoch 20 to 40 at all three scales. Similar computation reduction yields different speedups because direct computation and formula processing contribute different amounts to total time.

2. The prediction dimension study makes the cost of larger maps explicit. We set n=192,m=528,N=1313n=192,m=528,N=1313 fixed, with results shown in the following table:

ppTraining speedupFormula arrays (MiB)
122.10143.0
241.76336.4
361.62482.5
481.53643.2
961.34938.6

Formula storage increases with prediction dimension, while FastDFL retains a time advantage at p=96p=96, reaching 1.34 times the speed of DAQP–KKT. At every fixed pp, mean speedup increases at the recorded epochs 5, 10, 20 and 40. For p=24p=24, the corresponding values are 1.14, 1.28, 1.50 and 1.76 times.

3. Increasing the sample count exposes the cost of retaining more formulas. A separate experiment fixes p=24p=24 and uses nested subsets of the original 1,313-instance training pool. Its mean resource measurements are:

NNTraining time (DAQP–KKT / FastDFL, s)Retained formulasFormula arrays (MiB)Peak memory (DAQP–KKT / FastDFL, MiB)
32074.86 / 53.35859.656.11077.2 / 1137.8
640124.25 / 72.021971.6128.91149.4 / 1287.6
1280215.93 / 122.655032.6330.31170.1 / 1516.8

The practical choice depends on accumulated time savings and the available memory. These measurements quantify both sides of that choice. Retention under a memory budget, clear of useless formula and map compression remain promising extensions and will be explored in the future work.


To Q2: FastDFL uses local KKT solution maps, with the forward QP solver providing direct evaluations when reuse is unavailable. The main experiments use DAQP, a dual active-set solver, followed by KKT adjoint differentiation.

To Q3: di,kd_{i,k} is the Euclidean distance from the current prediction to the boundary of its formula's numerical validity region. For Φci\Phi_{c_i} valid at sample ii's kk-th prediction, di,k=inf⁡a′∈∂Dci∘∥a^i,k−a′∥2d_{i,k}=\inf_{a'\in\partial\mathcal D_{c_i}^{\circ}}\|\hat a_{i,k}-a'\|_2. This distance gives the prediction's margin before leaving that formula's region. We have clarified the definition in Section 4.2, lines 335–339 of the revised manuscript.

To Q4: F\mathcal F stores shared formulas, while Hi\mathcal H_i stores references ordered by recent use for sample ii. The library is an ordered dictionary keyed by tuples of complete binding constraints. Each history is a list with the most recently used formula first. The algorithm checks that entry, then checks older entries sequentially while the cumulative budget Pt≤(Ht+1)τˉtP_t\leq(H_t+1)\bar\tau_t permits. A successful entry moves to the front. A search miss triggers direct QP solution and differentiation. We have clarified these in Section 3.2.2, lines 206–212.

We hope our response addresses your concerns. Please let us know if any concerns remain unaddressed, we are happy to discuss.

AuthorsOpenReview ↗

A follow-up for response

Dear Reviewer nDjC,

We are writing to ask whether our response and revision have addressed your concerns. We would be happy to discuss any remaining issues.

Reviewer_aoX4OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents FastDFL, a method for accelerating decision-focused learning (DFL) by reusing computations in differentiable quadratic programming (QP) layers. It considers strongly convex QPs with fixed Hessians and constraints, where predictions enter only the linear objective term. FastDFL exploits the piecewise-affine structure of the QP solution map to reuse previously computed decisions and Jacobians. When a stored formula passes the KKT validity checks, the method avoids both the forward QP solve and backward differentiation. The paper also introduces rules for constructing and retrieving these formulas during training.

The method is evaluated on portfolio optimization, battery dispatch, and replenishment problems using linear and MLP predictors. It reduces training time compared with the tested differentiable QP layers while largely preserving decision quality. However, the main idea builds on known properties of parametric QPs and existing regional solution reuse methods. It remains unclear what FastDFL adds beyond these methods or how well its computational benefits extend to other problem settings.

Critical strengths

S1. Clear KKT-based formulation and analysis.

The paper provides a clear derivation of local solution maps and their Jacobians from the KKT conditions. It also establishes conditions for valid reuse and analyzes numerical errors. The analysis helps explain when forward and backward computations can be safely reused, although the underlying QP properties are already well known.


S2. Faster training with supporting empirical results.

FastDFL reduces training time by reusing previously computed decisions and Jacobians. The comparison with DAQP–KKT is particularly useful because both methods use the same solver and differentiation procedure. The reported speedups of 1.35–3.47×, with nearly unchanged decision quality, show that reuse can reduce training costs in the tested settings. The ablation studies and memory measurements also help show the benefits and overhead of formula reuse.

Critical weaknesses

W1. Limited novelty beyond existing QP reuse methods.

The main idea relies on well-known properties of parametric QPs. The piecewise-affine solution map, constant Jacobian within an active-set region, and reuse of local solution formulas have already been studied in multiparametric QP and regional MPC methods. FastDFL brings these ideas into the DFL training loop and adds rules for constructing and retrieving formulas.

While these choices are reasonable, I find it difficult to identify a substantial new contribution. The theoretical results largely follow from the existing KKT formulation and the proposed reuse procedure. The paper does not clearly explain what is gained beyond applying known regional QP reuse techniques to repeated training evaluations.


W2. Unclear advantage over existing alternatives.

FastDFL considers only strongly convex QPs with fixed Hessians and constraints, where predictions affect the linear objective term. This is a much narrower setting than those supported by several differentiable optimization layers used as baselines. The speed comparisons are valid for the QPs considered, but they do not address the limitations that come with this narrower scope.

The evaluation also focuses almost entirely on differentiable QP layers, with little comparison to other efficient DFL approaches on shared problems. Recent solver-free DFL methods offer other ways to reduce training costs without repeatedly solving the downstream optimization problem. These methods differ in their assumptions and guarantees, but the paper does not clearly explain when caching local QP solutions would be a better choice. Given its restricted applicability and reliance on established reuse techniques, I find the practical advantage of FastDFL insufficiently established.


W3. Lack of evaluation in settings with less frequent active-set reuse.

FastDFL's efficiency depends heavily on how often the same active-set regions are revisited during training. However, the experiments do not sufficiently examine what happens when active sets are more diverse or change frequently.

All three benchmarks show substantial formula reuse. Although Appendix E.3 considers larger Battery QPs, the prediction dimension remains fixed at 24. Increasing the decision dimension alone does not show how the method behaves with more diverse prediction trajectories or less frequent active-set recurrence. This is an important limitation, since the computational gains depend directly on these properties. The asymptotic efficiency guarantee in Section 4.2 also depends on cache misses and other overhead growing sublinearly.


W4. Scalability and the cost of formula reuse.

The reported speedups vary considerably across tasks. For Fama and Battery, they range from 1.35× to 1.47×, while the larger Battery experiments in Appendix E.3 show only 1.17–1.19× speedups. Meanwhile, the formula library grows from approximately 394 MB to 733 MB as the QP size increases.

The ablation studies help isolate the effects of the proposed formula management policies, but their benefits vary considerably across tasks and are almost negligible on M5. This raises questions about how much the additional cache management contributes in other settings. Overall, the experiments leave open how well the time–memory trade-off will scale and how broadly the reported speedups can be expected.

Questions for authors

Q1. Novelty relative to existing QP reuse methods.

What is the main algorithmic contribution of FastDFL beyond applying existing regional QP solution reuse to DFL training? In particular, how does it differ from using an on-demand regional MPC or active-set caching method in the same setting?


Q2. Practical advantage over existing methods.

Given its restricted QP assumptions, in which settings would FastDFL be preferable to more general differentiable optimization layers or other efficient DFL methods? How does it compare with these alternatives on shared QP tasks in terms of both decision quality and total training time?


Q3. Active-set recurrence and scalability.

How does FastDFL perform when predictions visit many different active-set regions or switch between them frequently? Since formula reuse is central to the method, how do the speedup, solver-call reduction, and memory requirements change as the prediction dimension and QP size increase?

8 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Question 1

Dear Reviewer aoX4,

Thank you for your detailed feedback! Below are point-by-point responses and revisions:

To W1 and Q1:

We agree with you that local affine solution maps, their Jacobians, and regional reuse are established foundations. Our contribution is a framework that organizes regional reuse around repeated visits to individual samples during DFL training. As the predictor evolves, each sample can return to regions encountered in earlier updates. FastDFL uses these histories to organize reusable computations for both decisions and their derivatives, with construction and retrieval decisions aimed at reducing total training work while preserving the original QP decision loss.

FastDFL changes when formulas are constructed and which formulas are searched during training. We compare with a DFL adaptation of Regional MPC with active set updates (König and Mönnigmann, 2017) (denoted as Regional MPC below and in the revised manuscript). Regional MPC reuses a valid current regional map and updates active constraints across neighboring regions when the parameter leaves that region, while our FastDFL constructs formulas after recurrence and uses each sample's formula history, checking its newest reference first and budgeting older searches. For illustration, when a sample returns to a previously visited region after visiting other regions, a valid historical formula can serve the returning prediction without constructing the intervening regional maps. A structure observed only once also incurs no formula construction in FastDFL. These choices organize regional reuse around the recurrence and computational costs encountered during DFL training. We have clarified this contribution in Section 1, lines 58–60 of the revised manuscript.

Our theoretical analysis separates valid reuse from the conditions needed for computational savings. Proposition 1 in Section 4.1 specifies the KKT consistency and validity conditions under which a formula preserves the decision and its Jacobian. Theorem 2 in Section 4.2 accounts for construction, trajectory exits, additional misses, and work for history search. Together with Corollary 1, it identifies conditions for reduced work for the optimization layer.

We compare the two methods on all 3 benchmarks using 2 predictors. Regional MPC adaptation uses the common KKT map construction for its regional updates and DAQP when an update is unavailable. Results are shown below:

Task / predictorRegional MPC time (s)FastDFL time (s)Speedup (mean ratio)Constructions (Regional / FastDFL)Formula processing (Regional / FastDFL, s)
Fama / Linear36.05 ± 2.1024.4 ± 0.41.56222.6 / 567.219.59 / 2.00
Fama / MLP34.92 ± 1.4822.6 ± 0.81.59386.0 / 55.020.72 / 1.13
Battery / Linear75.64 ± 0.6747.2 ± 0.51.621131.2 / 3406.628.71 / 6.05
Battery / MLP67.77 ± 0.3529.0 ± 0.52.326211.8 / 755.827.98 / 2.08
M5 / Linear21.94 ± 0.359.15 ± 0.052.48152.4 / 172.015.16 / 2.18
M5 / MLP45.00 ± 0.9317.4 ± 0.22.624925.2 / 391.637.37 / 2.53

FastDFL has lower training time in every setting. On Fama and M5 benchmarks, Regional MPC avoids more complete QP solves, yet its repeated map construction and regional processing cost more. The Battery task also incurs direct solves when the regional-update conditions fail. These costs explain the measured implementation differences, while Table 3's component ablations identify the effects of FastDFL's construction and retrieval policies. We have added Regional MPC to Section 5.2 (Table 1), lines 419–426, and Appendix E.1 (Table 9), lines 1261–1268 of the revised manuscript.

AuthorsOpenReview ↗

Response to Weakness 2 and Question 2

To W2 and Q2:

We acknowledge that FastDFL addresses a specialized problem class. This includes prediction of the entire linear objective coefficient vector, since setting q0=0q_0=0 and Q=IQ=I gives q(a)=aq(a)=a. Predicting objective coefficients for a fixed decision model is also the setting of methods such as SPO+ (Smart Predict-then-Optimize). In our QPs, these coefficients can represent quantities such as negative expected asset returns, electricity prices, and price-weighted demand targets. Our goal is to exploit this fixed structure to avoid repeatedly computing local decisions and their derivatives during training.

We compare solver-free methods LODL (Shah et al., 2022), which samples perturbed optimization objectives, solves the original QPs to obtain regret labels, and fits a local loss for each training instance. Predictor updates then differentiate these learned losses without solving QPs. Preparation depends on the task, training targets and fitting configuration. Our FastDFL builds formulas as constraint structures recur and includes construction, validity checks and retrieval in its training process. So it can be summarized that LODL prepares losses offline, while FastDFL constructs formulas online during DFL training.

We compare both methods on all three benchmarks using both predictors. FastDFL times are reproduced from Table 1. LODL trains the predictor faster on each task, while preparation accounts for about 90–98% of its total cost. Its preparation plus training cost exceeds FastDFL's recorded training time on all three tasks (As we mention above, the preparation process depends on specific task and experiment settings, so when compared, total time should be considered). FastDFL is a useful choice when preserving the QP decision loss matters and recurrence covers formula management cost. We have added LODL to Section 5.2 (Table 1), lines 420–427, and Appendix E.1 (Table 9 for MLP predictor), lines 1262–1270 of the revised manuscript.

TaskFastDFL training (s)LODL preparation (s)LODL training (s)LODL total (s)
Fama24.4 ± 0.4106.52 ± 2.2611.49 ± 0.28118.01 ± 2.27
Battery47.2 ± 0.5210.59 ± 2.2913.34 ± 0.54223.93 ± 2.05
M59.15 ± 0.05194.00 ± 11.794.88 ± 0.21198.88 ± 11.99
AuthorsOpenReview ↗

To Weaknesses 3,4 and Question 3

To W3 & W4 & Q3:

We agree that prediction dimension, QP scale and region visits deserve separate examination. We have extended Appendix E.3 (lines 1352–1356 and 1383–1403) with the studies in Figures 4–5 and Table 11. All speedups below are relative to direct DAQP–KKT training.

1. For prediction dimension, we vary pp while keeping the Hessian and feasible set fixed. Using the Battery benchmark, we fix n=192,m=528,N=1313n=192,m=528,N=1313. Predictions specify coefficients of a fixed basis for costs across storage units and hours. Full observed prices define the loss and evaluation at every pp. Means after 40 epochs are:

ppTraining speedupFormula arrays (MiB)
122.10143.0
241.76336.4
361.62482.5
481.53643.2
961.34938.6
Increasing pp raises storage requirements and reduces the final speedup, while FastDFL retains a time advantage throughout. At each fixed pp, the advantage also increases at the recorded checkpoints from epoch 5 to 40. Figure 5 shows both trends.

2. For scalability and the time–memory trade-off, we enlarge the QP while fixing p=24,N=438p=24,N=438. Figure 4 and Table 11 have the following results:

(n,m)(n,m)Speedup at epoch 20Speedup at epoch 40Reduction at epoch 40 (%)Formula arrays (MiB)
(96,288)1.281.3667.7394.1
(144,408)1.251.2968.8564.3
(192,528)1.241.2768.6733.6

The time advantage grows from epoch 20 to 40 at every scale, while computation reduction remains near 2/3. Similar reuse fractions yield different speedups because direct computation and formula processing have different costs. Larger stored maps increase memory use, so these results expose a practical trade-off between training time and storage. On M5, the newest formula already captures nearly all observed reuse, which explains the limited additional benefit of history search in Table 3.

3. For active-set reuse, we distinguish region diversity from switching frequency. Diversity counts distinct complete binding sets. Switching measures changes between successive visits to the same sample. Their implications are:

  • When a structure appears only once, FastDFL uses direct computation and records the observation. Formula construction is considered when the structure recurs.
  • When predictions switch among previously visited regions, a stored formula can serve a return once retrieval finds it and its validity checks pass.
  • When many regions coexist with recurring structures, direct computation handles new structures and formulas serve eligible returns. We examine separate training trajectories with N=1313N=1313, still on the Battery benchmark, preserving the original sample order. Each contains 52520 training evaluations:
ppDistinct complete binding setsSwitching (%)Formula uses (%)
126,10541.375.4
249,28952.767.8
4815,68661.244.3
9626,65973.428.6

Reuse decreases as the observed diversity and switching increase, yet remains possible even in the most diverse trajectory. Frequent switching remains compatible with reuse when earlier structures recur. These diagnostics describe region visits separately from the timing results above. Separate training records also connect recurrence to the conditional analysis. From epochs 10 to 40, cumulative evaluations increase from 13,130 to 52,520, while mean direct solves rise from 7,737.0 to 17,095.4, about 2.2 times. This illustrates accumulating reuse over the measured interval, with Corollary 1 retaining its stated asymptotic conditions.

Together, the results show a time advantage across the tested dimensions and QP scales, with its magnitude governed by recurrence and the cost of formula management. Your questions help us separate these factors and clarify the practical benefits and resource requirements of FastDFL. As the Discussion section says, we will continue to discuss the formula library management to improve the cost-effectiveness of FastDFL in future works. Thanks again for these questions.

Please let us know if any concerns remain unaddressed, we are happy to discuss.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response

We thank all reviewers for their careful reading and constructive discussion. We summarize the main clarifications and additional evidence below.

QuestionReviewersMain clarification and revision
Methodological contributionaoX4, YBTRWe clarified FastDFL’s contribution: accelerating QP-based DFL through reuse while accounting for construction and retrieval costs, supported by consistency and cumulative-cost analysis (Sections 1, 3–4 and our responses).
ApplicabilityaoX4, nDjC, YBTRWe clarified the scope of fixed Hessians and constraints, including prediction of the linear objective coefficient vector (responses)
More baseline comparisonsaoX4, nDjC, YBTRWe added comparisons with a Regional MPC method and solver-free method LODL. FastDFL has lower reported total training time in all settings (Tables 1 and 9).
Scalability and memoryaoX4, nDjC, YBTRWe added experiments with different prediction dimensions and numbers of training samples, analyzed active-set changes and reuse during training, and updated the QP scaling results. At p=96p=96, FastDFL remains 1.34× as fast as DAQP–KKT, with 938.6 MiB of formula arrays, and maintains the decision quality (Appendix E.3 and our responses). We also clarified when construction and retrieval costs can outweigh the savings from avoided solves.
Solver choiceYBTRWe added backend comparisons confirming training acceleration with both DAQP and OSQP when the backend is held fixed, where FastDFL retains its advantage with different backends. (Appendix E.4, Table 12)
Theory and numerical consistencynDjC, YBTRWe clarified the complete KKT relations and used them to trace the eight Jacobian discrepancies to binding constraints omitted in the reference reconstruction (Appendices B.2 and E.2). We also clarified the definitions and efficiency analysis in Section 4.2.
Implementation and notationnDjC, YBTRWe clarified the data structures for formula storage and history search, and revised the notation for stored Jacobians (Sections 3–4).

Details are provided in the revised manuscript (with revised content marked in blue) and the following responses. Looking forward to your replies!

Area_Chair_u5FsOpenReview ↗

Reminder: Reviewer–Author Discussion

Thank you to the reviewers who have already engaged in the discussion and provided valuable feedback. As the discussion deadline is approaching, I would kindly encourage the remaining reviewers to review the authors' responses and share their follow-up comments, particularly on whether their concerns have been adequately addressed.

Authors are also encouraged to respond promptly to any further questions.

Thank you all for your time and contributions to a constructive discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.