acceptodds
Withdrawn from ICLR 2027

GG-DFF: Geometry-Guided Decision-Focused Fine-Tuning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Decision-Focused Learning (DFL) trains predictive models for downstream optimization using decision quality as the learning signal. Existing methods make DFL tractable by differentiating through optimization layers, smoothing the optimizer with perturbations, or replacing the true decision loss with tractable surrogates. However, in linear and mixed-integer linear programs with linear objectives, the convex hull of feasible decisions induces a partition of the objective-parameter space into regions associated with different optimal decisions. The decision consequences of a prediction correction depend on the region it reaches, while equal reductions in surrogate loss can lead to decisions of different quality. To address this issue, we propose Geometry-Guided Decision-Focused Fine-Tuning (GG-DFF), which uses feasible-solution differences to guide prediction corrections. We compare the decision under the original prediction with collected feasible solutions. We then use the resulting difference vectors to construct a convex correction region (CCR) around the original prediction, which guides which decision regions prediction corrections can reach. Specifically, we freeze the pretrained predictor and learn corrections through an almost-everywhere differentiable parameterization of the CCR. This parameterization keeps corrected predictions inside the region and allows SPO+ gradients to propagate through its comparison geometry. Our theoretical analysis connects correction boundaries to decision-region reachability, establishes sufficient conditions for excluding decision regions, and derives conditional regret bounds relative to the pretrained decision. Experimental results show that GG-DFF reduces mean normalized decision regret relative to the reference fine-tuning baseline by up to 17.7%, 1.0%, 2.9%, and 2.2% on shortest path, knapsack, Energy-1, and Energy-2, respectively.

Primary area:
optimization
Keywords:
decision-focused learning, combinatorial optimization, polyhedral geometry, fine-tuning, convex correction region

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_KFhVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Geometry-Guided Decision-Focused Fine-Tuning (GG-DFF) for decision-focused learning (DFL) in linear and mixed-integer linear optimization problems. Its key innovation is to construct a convex correction region (CCR) around a frozen pretrained predictor using differences between feasible solutions, thereby explicitly controlling which decision regions corrected predictions can reach; an almost-everywhere differentiable parameterization enables end-to-end fine-tuning with the standard SPO+ loss. Theoretically, the paper characterizes the relationship between correction boundaries and decision-region reachability, establishes sufficient conditions for excluding certain decision regions, and derives conditional regret bounds relative to the pretrained predictor. Experiments on four benchmarks—shortest path, knapsack, and two energy-scheduling tasks—show that GG-DFF consistently outperforms the DFF baseline in mean normalized decision regret.

Critical strengths
  1. The paper introduces a convex correction region (CCR) that uses feasible-solution differences to constrain prediction corrections according to the optimization problem's geometry, offering a principled alternative to unconstrained or box-constrained corrections.
  2. The authors establish theoretical results on decision-region reachability and conditional regret bounds. Experiments on four benchmarks show consistent improvements over the DFF baseline in mean normalized decision regret.
Critical weaknesses
  1. Limited empirical gains on some benchmarks: Improvements are modest on knapsack and energy-scheduling tasks (approximately 1.0%–2.9%), and the paper does not establish that the gains are statistically significant.

  2. The theoretical guarantees rely on specific conditions, including comparison coverage, while the effectiveness of the correction region may depend on how well the constructed boundaries capture the relevant decision geometry. The generality of the approach beyond the evaluated problem settings remains unclear.

Questions for authors
  1. How sensitive is GG-DFF to the quality and coverage of the collected feasible-solution comparisons? How does its performance change when comparison coverage is incomplete or the number of collected feasible solutions is limited?

  2. Are the improvements over DFF statistically significant? Given the relatively modest gains on knapsack and energy-scheduling benchmarks, could the authors provide statistical tests or confidence intervals across random seeds?

  3. How does GG-DFF scale to larger or more complex optimization problems? In particular, what is the computational cost of constructing and parameterizing the convex correction region as the number of feasible-solution comparisons increases?

Reviewer_4jF7OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper improves Decision-Focused Fine-Tuning (DFF), which trains a correction network on top of a frozen MSE-pretrained predictor. The proposed GG-DFF replaces the coordinate-wise box constraint of DFF with a convex correction region defined by cost comparisons between the anchor decision and alternative solutions, and keeps the corrected prediction inside this region. The paper provides a conditional regret guarantee relative to the anchor and reports lower regret than DFF and other baselines on shortest path, knapsack, and energy scheduling benchmarks.

Critical strengths
  1. The motivation that the coordinate-wise box of DFF does not reflect the decision structure is clear, and defining the correction region through cost comparisons between solutions is intuitive.

  2. The paper gives a closed-form parameterization that guarantees every output of the correction network satisfies the constraints.

Critical weaknesses
  1. The experimental scope is very limited compared to most prior work. In particular, for shortest path, the paper states that it follows the data-generation procedure of Elmachtoub & Grigas, but the submitted code generates costs with a noise-free, deterministic RBF-type function. This differs from the description in the paper, and as a result the degree and noise settings do not exist at all. In addition, the experiments use a 4x4 grid instead of the 5x5 grid that prior work uses as the default size. The results show that on shortest path, many DFL baselines, including SPO+ and PFYL, perform much worse than the model trained only with MSE. This appears to be because the setting is very easy: without noise, an MSE-trained model can make nearly exact predictions. Moreover, this trend is not limited to shortest path. On all benchmarks, including knapsack and energy, the MSE-trained model outperforms most DFL baselines, and SPO+ performs worse than MSE on every benchmark except knapsack. Since knapsack and energy are based on real data, this cannot be explained by an easy setting, which raises the question of whether the DFL baselines were properly tuned. These results contradict most prior work and raise doubts about whether the authors followed a rigorous and standard experimental setup. Results under the standard generation procedure with multiple degree and noise settings are needed. Also, in the DFL literature, the 2-Stage baseline usually refers to the two-stage predict-then-optimize approach in which the predictor is trained only with MSE, whereas this paper uses the method of Hu et al. (2023), which is confusing.

  2. The method directly adopts the structure of DFF, and the only difference is the shape of the region in which corrections are allowed. However, even in the authors' own ablation, a correction with the constraint removed entirely performs better than DFF, and the proposed method shows almost no difference from this unconstrained correction on knapsack and Energy-1. It is therefore unclear whether the performance gain comes from the proposed correction region or simply from relaxing the DFF constraint. The improvements themselves are also small, so I am not convinced that the novelty of this work is sufficient.

  3. The mismatch between surrogate loss reduction and decision quality emphasized in the paper arises because regret changes only when the prediction moves to a different normal cone, and this is already well known in the DFL community. The authors express this through various theoretical results, but many of them follow directly from the definitions, and the motivating example is constructed artificially rather than taken from an actual training process. In addition, the stated performance guarantee does not give a meaningful bound at the hyperparameters actually selected in the experiments, and the setting that is less favorable in the authors' reachability analysis performs better in the experiments. It is therefore unclear whether the theory explains the performance of the proposed method or provides new contributions beyond existing work.

Questions for authors
  1. Why are the shortest path data generated with a noise-free RBF function, unlike the description in the paper? I would also like to see results under the standard Elmachtoub & Grigas setting (5x5 grid with multiple degree and noise levels).

  2. The MSE model outperforms most DFL baselines on all benchmarks. How were the hyperparameters of the baselines chosen?

  3. Appendix D.2 reports how many worsening paths the CCR excludes on shortest path, but only at the smallest budget that keeps all improving paths. How many alternative solutions are excluded at the ρ\rho actually selected in the experiments, and on knapsack and energy?

  4. According to the submitted code, all seeds share a single dataset and a single pretrained model. I would like to see the results and standard deviations when the data split or the pretrained model varies.

Reviewer_mVBJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a variant of decision-focussed fine tuning of a pretrained model, where the trust region becomes a trainable 'correction region', based on feasible (or even all) solutions, and trained with the SPO+ loss function.

Critical strengths

It shows minor improvements on the original DFF

Critical weaknesses

The approach is insufficiently explained in terms of related work. More specifically the 'solution ranking and optimality margins' section explains multiple approaches that pre-compute solutions, and compare decisions to pre-computed solutions; the authors then mention that GG-DFF also uses comparisons, without properly explaining if this convex correction region is similar or different than the other work. The authors do not actually relate the works. Also the LAVA loss in the section above does such comparisons; is the current method just doing a variant of an existing DFL method on top of a pre-trained model?

The writing is low quality, it is very dense and cryptic and choses technical jargon and details over intuitive understanding. There is also an unwieldy amount of appendices sprinkled throughout the paper, starting at the top of page 2. Example such sentences: "In the interior of a vertex normal cone, the unique optimal decision is that vertex." "This matters in shared-network training, where the network Jacobian, minibatch aggregation, and optimizer jointly determine how each prediction moves." "Thus even a common slack produces a geometry-dependent region" "The common certificate gives..."

Also some words are misused, like in the abstract, 3rd sentence: "Existing methods make DFL tractable by..." since DFL is about optimizing, and doing that optimisation for each instance in each epoch, there is absolutely nothing about existing methods making DFL 'tractable', or what the non-tractable counterpart would be.

The value and meaning of sections 5.3 and 5.4 escape me. Alg 1 (in the appendix) gives the procedure of what exactly?

Questions for authors

Please relate your work more precisely to existing works that compare to previously computed solutions. Also why do you need training solutions with 3 different cost functions, what are you really doing here? storing all optimal and near-optimal solutions to then 'correct' wrt this inner approximation? a soft/slack version of Mulamba et al? or perceptron-like?

line 356 mentions '2-stage' with 2 citations, but those 2 works propose very different methods. What do you mean with 2-stage? you mean to first train an MSE predictor and then call the solver, what the simple baseline method is? But then, why does your MSE column for 2-stage mismatch method MSE's MSE?

For multiple of your problems, SPO+ and the other DFL methods other than DFF do not really outperform MSE. These methods exist because they can outperform MSE when trained with downstream loss, that is why these papers got published. That does not look right. What is different here in your setup, that the DFF variants do notably better?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.