acceptodds
Under review as a conference paper at ICLR 2027

Solver-Label-Free Pretraining for In-Context Primal Solution Prediction in ILPs

Abstract

Integer linear programs (ILPs) are a foundational tool for modeling discrete decision-making problems, but solving large-scale ILPs remains computationally challenging due to their NP-hardness. This has motivated machine learning (ML) methods that predict the values of the variables in high-quality solutions, a task known as . These predictions then guide an exact solver, which narrows its search to promising regions of the solution space. However, most of these methods are trained on solver-labeled instances, which requires hundreds of solver runs, each taking up to an hour, for every problem class. We propose , a framework that instead uses in-context learning (ICL), predicting solutions for a new instance from a few solved instances of the same class given as input, without retraining. Following prior-data fitted networks (PFNs), we acquire the ICL capability without any solver labels by pretraining on synthetic tasks, in which instances from standard problem generators are labeled by randomly sampled rules that favor feasible, high-quality assignments. At inference time, an exact solver provides the solutions of the few input instances, so the model predicts the solution that solver would find for the new instance. For each problem class, PrimalPFN requires only two solver-labeled instances rather than hundreds, and shares them across all new instances of that class. Across three ILP problem classes, it finds solutions comparable in quality to those of class-specific supervised predictors on two classes and better ones on the third, with a gap to the best known solutions about 40% smaller. Compared with an unsupervised baseline that likewise requires no solver labels for training, PrimalPFN reduces the gap by 39-79%. Moreover, given the same labeled training instances as the supervised predictors, PrimalPFN attains a lower average gap than them on all three classes.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Integer Linear Programming, Learning to Optimize

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 4 reviews, above 99% of the others.
Reviewer_ok6FOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes PrimalPFN, which applies the prior-data-fitted network (PFN) recipe to primal solution prediction for binary ILPs.

  • Pretraining data. Synthetic episodes are built from Ecole-generated instances of the three target classes. Each episode's instances are labeled by a randomly sampled rule: a random MPNN produces an initial assignment, which simulated annealing then refines on a penalized objective.
  • Architecture. Following NodePFN, each layer has two branches: GINE message passing within each instance, and attention from query tokens to context tokens across instances.
  • Inference. Two Gurobi-solved instances of the target class serve as context. The predicted marginals are passed to Predict-and-Search (PS).
  • Results. On the Distributional MIPLIB set cover (SC), independent set (IS), and combinatorial auctions (CA), PrimalPFN with two labeled instances roughly matches PS trained on 240 labeled instances on SC and CA, and beats it on IS (0.32% vs 0.53% gap). It beats DiffILO on all three classes. A variant that also pretrains on the 240 solver-labeled instances is best overall.
  • Analyses. Interventions on the context labels, prior ablations, label-efficiency curves, and SCIP results are provided.
Critical strengths
  1. An original, well-motivated idea. Using PFN-style in-context learning for primal prediction is new. Building context from other solved instances, rather than from labeled nodes in the same graph, is a sensible adaptation.
  2. Convincing analysis of what the model uses. In the context-label interventions (Table 2, Figure 5), shuffling labels within an instance keeps label frequencies but breaks their correspondence with variables, and it badly hurts performance. This shows the model uses the instance–label correspondence. The prior ablation identifies the optimization-aware refinement step as the key part of the prior.
  3. Useful additional evidence. The label-efficiency curves (Figure 4), robustness to which context instances are drawn (Table 14), the SCIP results, and the per-instance win/draw/loss tables strengthen the empirical picture.
  4. Clear writing. The paper is well organized and easy to follow.
Critical weaknesses
  1. The prior is class-specific, which narrows the main claim. Pretraining uses École generators for exactly the three evaluation classes, and Appendix C.1 says unseen classes are "a different setting". The abstract and introduction ("for each problem class, PrimalPFN requires only two solver-labeled instances") suggest adapting to a new class from a few examples. What is shown is adaptation within classes whose generators were seen during pretraining. To support the in-context-learning framing, please add one of:

    • leave-one-class-out results (pretrain on two classes, test on the third), or
    • results on a binary class outside the prior, such as another Distributional MIPLIB binary class or MIPLIB instances.
  2. It is unclear how much comes from in-context learning versus synthetic labels. In Table 2, the "Prior" context uses synthetic-rule labels and no solver labels at all, yet gives 0.27% (SC) and 0.44% (IS). That is close to the 0.23% / 0.32% obtained with solver-labeled context, and on IS it already beats PS (0.53%). Only CA depends strongly on solver context (3.12% → 0.64%). So on SC and IS, most of the gain seems to come from pretraining on annealing-refined labels for in-class instances. Two baselines are needed to show the PFN machinery is necessary:

    • (a) a standard supervised GNN, either the PS architecture or PrimalPFN without the context branch, trained on the same synthetic labeled data, with and without fine-tuning on the two solver-labeled instances;
    • (b) PrimalPFN fine-tuned on the two context instances instead of using them in context.
  3. The compute comparison is incomplete. Figure 1(d) and the abstract stress a drop from 300 to 2 CPU-hours of labeling. This leaves out the cost of generating synthetic labels (annealing over many large instances), about 24 GPU-hours of pretraining, and the effort required to design the prior. Pretraining is shared across classes, but here the prior is built per class (Weakness 1), which weakens that argument. Please report the total CPU and GPU hours for the prior generation and pretraining next to the labeling cost.

  4. The number of context instances is never varied. It is fixed at N_ctx = 2 in both pretraining and evaluation. A defining property of in-context learning is that predictions improve with more context. Please report results for N_ctx ∈ {1, 2, 4, 8} where memory allows, and discuss how cost grows. Each query attends to roughly 8k–12k context tokens, and test instances have only 4k–6k variables. This also matters for applying the method to larger instances.

  5. Improvements are modest, and the statistics are limited.

    • On SC, PrimalPFN is worse than PS: 0.23% vs 0.18% gap, 12 wins and 19 losses.
    • On CA, it is effectively tied: 0.64% vs 0.67%, 53 wins and 47 losses.
    • Only IS shows a clear win.
    • Results come from a single pretraining run and a single PS training run (Figure 4 says "one run each"), with no confidence intervals or significance tests. Please report variability over pretraining seeds and PS training seeds, along with paired tests.
  6. Details of the downstream search are unclear, and a non-learned baseline is missing. The PS parameters k₀ and k₁ (Table 8) are fixed per benchmark. Were they taken from Han et al. (2023) or tuned, and if tuned, for which predictor? The trust-region radius Δ is chosen per method from only {0, 20, 40}; please show sensitivity over a wider range. A non-learned baseline, such as LP-relaxation values fed into the same PS procedure, would show how much the learned predictions add.

  7. Scope and related work. Evaluation covers only pure-binary, generator-based classes, all of which have an easy feasibility structure. Cross-domain or foundation-style predictors (Li et al., 2025; RoME; Pu et al., 2025) are cited but not compared. Since PrimalPFN is pitched as label-efficient adaptation, a comparison with a cross-domain pretrained predictor adapted to the target class with few labels would be informative.

  8. Minor issues.

    • "Two instances" understates the supervision provided, which is 8k–12k labeled variables.
    • The Bayesian interpretation assumes the solver's labeling lies within the support of the prior. A short discussion of what happens when the prior is misspecified would help.
    • Code is promised only on publication.
Questions for authors
  1. How does PrimalPFN perform when the test class is excluded from pretraining (leave-one-class-out)?
  2. How does a supervised GNN trained on the same synthetic labels perform, with and without fine-tuning on the two solver-labeled instances?
  3. How do accuracy, gap, and inference cost change as N_ctx varies?
  4. What are the total compute costs (CPU and GPU hours) of generating the prior and pretraining?
  5. What is the variability across pretraining seeds and PS training seeds, and are the IS improvements significant under a paired test?
  6. Were k₀ and k₁ tuned, and if so, for which method? How sensitive are the results to Δ beyond {0, 20, 40}?
  7. How does a PS run guided by LP-relaxation values (no learning) compare?
11 comments

We appreciate the reviewer's effort in reviewing our paper. We are happy to see that the reviewer recognized several strengths of this work, including the originality and motivation of the idea, the convincing analysis of what the model uses, and the clarity of the writing. Below, we address each concern with additional results and clarifications.

Note that the additional experiments throughout our responses were conducted on a different hardware setup (AMD EPYC 9354 CPU (32 cores) + NVIDIA RTX A6000 GPU (48GB)) from that of the original manuscript due to hardware availability. Except for results explicitly identified as originating from the original manuscript, all results in the additional tables were obtained under this setup. Unless otherwise noted, Δ\Delta is selected by the protocol in Appendix B.3 over {0,10,20,40,80}\{0, 10, 20, 40, 80\}, which we extended from the original {0,20,40}\{0, 20, 40\} in response to the reviewer's comment on the range of Δ\Delta (see W6/Q6/Q7). For statistical comparisons, we first obtain one gap per test instance for each method by averaging across training seeds when multiple runs are available, or using the single-run gap otherwise. We then perform paired t-tests on the resulting 100 matched instance-level gaps.


[W1/Q1: Generalization to unseen classes]

Following the reviewer's suggestion, we conducted leave-one-class-out experiments, in which PrimalPFN is pretrained on two classes and evaluated on the held-out class (Table 1).

Table 1: Gap (%) on each class when it is included in pretraining (PrimalPFN), held out (Leave-one-out), and added by fine-tuning the leave-one-out model on synthetic episodes (Fine-tuning). PrimalPFN is averaged over three training seeds; the other rows use a single seed.

MethodSCISCA
PrimalPFN0.3380.3210.989
Leave-one-out0.6900.6905.387
Fine-tuning0.2340.4181.107

The leave-one-out models perform worse on the held-out class, which is expected since the class is not covered by the prior. As a remedy, fine-tuning the leave-one-out model for 200 epochs on synthetic episodes from unlabeled instances of the held-out class recovers most of the lost performance (lower than PrimalPFN on SC, not significantly different on CA with p=0.059p = 0.059 in a paired t-test, and higher on IS). This fine-tuning requires no additional solver labels beyond the two context instances and takes about 7 GPU-hours and 10 CPU-hours, compared with up to 300 CPU-hours of solver labeling for supervised predictors (see W3/Q4). We have revised the abstract and introduction to clarify that our claims concern problem classes covered by the pretraining prior (highlighted in blue).

[W2/Q2: Individual contributions of ICL and the randomized rule prior]

In-context learning. Table 2 compares PrimalPFN with a variant that removes the context branch (w/o context) while using the same synthetic training data. We evaluate this variant both before and after fine-tuning on the two solver-labeled instances used as PrimalPFN's context. On SC, PrimalPFN is not significantly different from either variant (paired t-test, p=0.20p = 0.20 and p=0.37p = 0.37, respectively). On IS and CA, PrimalPFN significantly outperforms both (p<0.001p < 0.001). Fine-tuning improves the variant, most visibly on CA, but it still performs significantly worse than PrimalPFN on IS and CA. These results indicate that, on IS and CA, the gain does not come from synthetic pretraining alone, and that using the solver-labeled instances in context is more effective than the evaluated gradient-based fine-tuning alternative.

Table 2: Gap (%) of PrimalPFN and a variant without the context branch, with and without fine-tuning (FT) on the two solver-labeled instances. PrimalPFN is averaged over three training seeds, while the other rows use a single seed.

MethodSCISCA
PrimalPFN0.3380.3210.989
w/o context0.4230.5471.374
w/o context + FT0.3970.5301.136

Regarding (b), directly fine-tuning PrimalPFN would require specifying how its context branch is used during fine-tuning and inference, making it difficult to isolate adaptation through gradient updates from adaptation through context. We therefore evaluate a conventional gradient-based alternative using the variant without the context branch, which shares the backbone and synthetic training data with PrimalPFN and is fine-tuned on the same two solver-labeled instances. While this is not an exact substitute for fine-tuning PrimalPFN itself, it provides a comparison under the same solver-label budget.

Randomized rule prior. Table 3 compares PrimalPFN with a variant pretrained under a single fixed rule ω∗\omega^*. As a strong baseline, we select an oracle ω∗\omega^* for each class, namely the rule whose labels are closest in Hamming distance to the solver labels on 60 solver-labeled training instances. Since this selection uses solver labels, the comparison favors the fixed-rule variant.

Table 3: Gap (%) of PrimalPFN and the fixed-rule variant. All reported gaps are averaged over the same 100 test instances. PrimalPFN is averaged over three training seeds, while the fixed-rule variant uses a single seed.

MethodSCISCA
PrimalPFN0.3380.3210.989
Fixed-rule (oracle)0.2630.3400.889

On SC and CA, the fixed-rule variant achieves lower gaps than PrimalPFN. On IS, PrimalPFN achieves a slightly lower gap, with feasible solutions on all 100 test instances for both methods. Notably, at Δ=0\Delta = 0, the fixed-rule variant yields infeasible subproblems on 30 of the 100 IS instances, whereas PrimalPFN yields none. The same tendency appears on CA at Δ=0\Delta = 0 (4 infeasible instances vs. none). These results suggest that pretraining over diverse rules may improve the robustness of confident predictions under tight trust-region constraints, although it does not consistently yield lower gaps across all classes.

[W3/Q4: Breakdown of computational cost]

Table 4 reports the CPU and GPU hours for synthetic data generation and pretraining, alongside the solver labeling cost.

Table 4: Computational cost of PrimalPFN pretraining and of solver labeling. Pretraining costs are measured over 1,200 episodes (10 epochs) with a single CPU worker and extrapolated to the full pretraining (700 epochs). Solver labeling costs are upper bounds, with up to one hour per instance (2 instances for PrimalPFN and 300 for PS).

StageResource10 epochs (measured)TotalIncurred
Instance generationCPU-h0.4833.4Once (shared)
Synthetic label generationCPU-h0.0151.0Once (shared)
Forward + backwardGPU-h0.3524.6Once (shared)
Solver labeling: PrimalPFNCPU-h-≤ 2Per class
Solver labeling: PSCPU-h-≤ 300Per class

Synthetic data generation runs on CPU workers in parallel with GPU training. Two workers are sufficient to keep pace with the GPU, and we used 12 in practice, so data generation does not increase the wall-clock pretraining time. For this reason, it was not reported separately in the original manuscript; we now report it in Appendix B.2.2 (highlighted in blue). Note that synthetic label generation accounts for only about 3% of the CPU cost, since each annealing step is a single-variable flip with an incremental energy update, implemented with JIT compilation (Numba).

We agree that the pretraining cost is amortized only over the classes included in pretraining. It is shared across these three classes, whereas supervised predictors incur the labeling cost for each of them. Extending PrimalPFN to a new class incurs an additional cost of about 7 GPU-hours and 10 CPU-hours (see W1/Q1).

[W4/Q3: Effect of context size]

Performance. Table 5 reports the gap for Nctx∈{1,2,4,8}N_{ctx} \in \{1, 2, 4, 8\} at inference time, using the same model pretrained with Nctx=2N_{ctx} = 2. The differences across context sizes are not statistically significant on any class (paired t-test). Prediction accuracy is also stable across context sizes, varying by at most 0.026 percentage points (SC: 98.960–98.972%, IS: 80.424–80.450%, CA: 89.249–89.263%). Unlike in tabular PFNs, which commonly use hundreds to thousands of labeled rows as context, each context instance here carries 4,000–6,000 labeled variables, so a single solved instance already provides ample supervision. In tabular PFNs, each context row represents an individual labeled example, consisting of an input feature vector and its target label. On IS and CA, the benefit of the context appears between no context (see W2/Q2) and one instance. We chose Nctx=2N_{ctx} = 2 in advance as a small labeling budget and did not tune it, and Table 5 shows that the results are not sensitive to this choice. Since the model was pretrained with a fixed context size, pretraining with variable context sizes may be needed to benefit from larger contexts.

Table 5: Gap (%) for different context sizes at inference time, using the model pretrained with Nctx=2N_{ctx} = 2. PrimalPFN is averaged over three training seeds; the other rows use a single seed.

Context sizeSCISCA
10.3190.3311.032
2 (PrimalPFN)0.3380.3210.989
40.3100.3320.948
80.3030.3470.967

Cost. Tables 6 and 7 report the pretraining and inference costs for each context size. From Nctx=2N_{ctx} = 2 to 88, the GPU time grows by about 7.5 times in pretraining and 7.3 times in inference, whereas the CPU cost and peak memory grow roughly linearly. We therefore do not adopt a larger context, since it brings no significant gain in downstream performance while the cost increases substantially.

Table 6: Estimated pretraining cost (700 epochs) by context size, extrapolated from a 1-epoch simulation. The values for Nctx=2N_{ctx} = 2 differ from those in Table 4, which are extrapolated from 10 epochs.

Context sizeCPU-hGPU-hPeak GPU Memory (GiB)
123.811.11.28
2 (PrimalPFN)35.021.41.89
458.153.13.09
8104.3160.65.49

Table 7: Inference cost per test instance by context size, averaged over the three classes. Only the GPU forward time is measured, whereas Figure 6 reports the end-to-end prediction time, including file I/O and graph conversion.

Context sizeGPU-sPeak GPU Memory (GiB)
10.340.34
2 (PrimalPFN)0.630.49
41.550.77
84.611.35

[W5/Q5: Magnitude and statistical significance of improvements]

We agree that PrimalPFN does not outperform PS on every class. Our main claim, however, is not to outperform supervised predictors, but to achieve competitive solution quality with two solver-labeled instances instead of 300.

To assess the variability, we trained each learning-based method in Table 8 (see W6/Q6/Q7 below) with three seeds and report the mean and standard deviation. Compared with PS at the best Δ\Delta of each method, PrimalPFN achieves a lower gap on IS (paired t-test, p=1.3×10−39p = 1.3 \times 10^{-39}) and a comparable gap on CA (p=0.19p = 0.19). On SC, PS achieves a lower gap (p=3.9×10−7p = 3.9 \times 10^{-7}), but adding the same solver-labeled training instances used by PS reduces PrimalPFN's gap from 0.338% to 0.212%, narrowing the difference to PS (0.164%, p=0.075p = 0.075). This suggests that the labeling budget contributes to the performance difference on SC.

Reviewer_vFAzOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes PrimalPFN for in-context primal solution prediction in binary ILPs. The model is pretrained on synthetic tasks whose labels are produced by randomized graph-based initialization and optimization-aware local search, without solver-generated training labels. At inference, the frozen model uses two solver-labeled context instances to predict variable-wise probabilities for new instances. These predictions guide a Predict-and-Search procedure. Experiments on set covering, independent set, and combinatorial auctions compare the method with supervised and unsupervised predictors and standalone solvers, with additional analyses of context labels, synthetic label generation, and context selection.

Critical strengths
  1. An interesting approach to reducing solver-label requirements. Combining optimization-aware synthetic tasks with cross-instance in-context prediction provides a useful application of PFN-style learning to ILPs. A shared, frozen predictor can use a small number of solved examples without requiring task-specific parameter updates. This addresses a meaningful bottleneck in supervised primal solution prediction.

  2. The experiments provide evidence for the central mechanism. With two solver-labeled context instances, PrimalPFN achieves competitive solution quality relative to supervised PS on SC and CA and improves the average gap on IS from 0.53% to 0.32%. Context-label interventions and prior ablations support the importance of meaningful context labels and optimization-aware synthetic supervision. Results across five context selections provide additional support for the base model’s stability.

Critical weaknesses
  1. The comparative evaluation and distributional coverage are limited. PS and DiffILO are relevant baselines, but the experiments do not sufficiently isolate the benefit of in-context adaptation over conventional synthetic pretraining followed by few-shot fine-tuning. A fixed-rule pretraining control would also help establish the contribution of changing labeling rules across episodes. Furthermore, evaluation covers only three problem classes, each with one target instance distribution, and all three classes are represented during pretraining. This limits the evidence for broader applicability.

  2. The complete computational cost is difficult to assess. The reported pretraining time of approximately 24 GPU-hours and prediction times below three seconds on average are useful. However, the paper does not provide a complete accounting of synthetic instance and label generation, context labeling, and downstream trust-region tuning. In particular, the number of additional tuning instances and total solver time used to select the radius are unclear. These costs matter for assessing when the reduction in solver-label requirements translates into practical savings.

Questions for authors
  1. How much of the improvement can be attributed specifically to in-context adaptation and rule diversity? Comparisons with synthetic pretraining followed by fine-tuning on the same two solver-labeled instances, and with fixed-rule synthetic pretraining, would help clarify these contributions.

  2. Could you provide a breakdown of the complete preparation cost, including whether synthetic data generation is included in the reported pretraining time, and the number of instances and total solver time used for radius selection? How do these costs amortize across different numbers of target instances?

  3. In Table 9, adding solver-labeled training data worsens the SCIP results on SC and IS, although it improves all three Gurobi results. Is this degradation reproducible across independent runs? Analysis of prediction quality and the selected search radii would help explain whether it arises from the predictor or its interaction with downstream search.

5 comments

We thank the reviewer for the careful reading and insightful comments. We are glad that the reviewer found our approach to reducing solver-label requirements interesting and the experiments supportive of its central mechanism. Our point-by-point responses, together with additional results, are provided below.

Note that the additional experiments throughout our responses were conducted on a different hardware setup (AMD EPYC 9354 CPU (32 cores) + NVIDIA RTX A6000 GPU (48GB)) from that of the original manuscript due to hardware availability. Except for results explicitly identified as originating from the original manuscript, all results in the additional tables were obtained under this setup. Unless otherwise noted, Δ\Delta is selected by the protocol in Appendix B.3 over {0,10,20,40,80}\{0, 10, 20, 40, 80\}, which we extended from the original {0,20,40}\{0, 20, 40\} in response to Reviewer ok6F's comment on the range of Δ\Delta. For statistical comparisons, we first obtain one gap per test instance for each method by averaging across training seeds when multiple runs are available, or using the single-run gap otherwise. We then perform paired t-tests on the resulting 100 matched instance-level gaps.

In the revised manuscript, changes made in response to this review are highlighted in red, and other changes are highlighted in blue.


[W1/Q1: Individual contributions of ICL and the randomized rule prior]

Following the reviewer's suggestion, we evaluated three additional baselines, namely (1) a variant without the context branch pretrained on synthetic labels, (2) the same variant fine-tuned on the two solver-labeled instances, and (3) a variant pretrained using a single fixed labeling rule.

In-context learning. Table 1 compares PrimalPFN with a variant that removes the context branch (w/o context) while using the same synthetic training data. We evaluate this variant both before and after fine-tuning on the two solver-labeled instances used as PrimalPFN's context. On SC, PrimalPFN is not significantly different from either variant (paired t-test, p=0.20p = 0.20 and p=0.37p = 0.37, respectively). On IS and CA, PrimalPFN significantly outperforms both (p<0.001p < 0.001). Fine-tuning improves the variant, most visibly on CA, but it still performs significantly worse than PrimalPFN on IS and CA. These results indicate that, on IS and CA, the gain does not come from synthetic pretraining alone, and that using the solver-labeled instances in context is more effective than the evaluated gradient-based fine-tuning alternative.

Table 1: Gap (%) of PrimalPFN and a variant without the context branch, with and without fine-tuning (FT) on the two solver-labeled instances. PrimalPFN is averaged over three training seeds, while the other rows use a single seed.

MethodSCISCA
PrimalPFN0.3380.3210.989
w/o context0.4230.5471.374
w/o context + FT0.3970.5301.136

Randomized rule prior. Table 2 compares PrimalPFN with a variant pretrained under a single fixed rule ω∗\omega^*. As a strong baseline, we select an oracle ω∗\omega^* for each class, namely the rule whose labels are closest in Hamming distance to the solver labels on 60 solver-labeled training instances. Since this selection uses solver labels, the comparison favors the fixed-rule variant.

Table 2: Gap (%) of PrimalPFN and the fixed-rule variant. All reported gaps are averaged over the same 100 test instances. PrimalPFN is averaged over three training seeds, while the fixed-rule variant uses a single seed.

MethodSCISCA
PrimalPFN0.3380.3210.989
Fixed-rule (oracle)0.2630.3400.889

On SC and CA, the fixed-rule variant achieves lower gaps than PrimalPFN (0.263% vs. 0.338% and 0.889% vs. 0.989%, respectively). On IS, PrimalPFN achieves a slightly lower gap (0.321% vs. 0.340%), with feasible solutions on all 100 test instances for both methods. Notably, at Δ=0\Delta = 0, the fixed-rule variant yields infeasible subproblems on 30 of the 100 IS instances, whereas PrimalPFN yields none. The same tendency appears on CA at Δ=0\Delta = 0 (4 infeasible instances vs. none). These results suggest that pretraining over diverse rules may improve the robustness of confident predictions under tight trust-region constraints, although it does not consistently yield lower gaps across all classes.

[W1 (cont.): Generalization to unseen distributions]

To examine classes that are not represented during pretraining, we conducted leave-one-class-out experiments, in which PrimalPFN is pretrained on two classes and evaluated on the held-out class (Table 3).

Table 3: Gap (%) on each class when it is included in pretraining (PrimalPFN), held out (Leave-one-out), and added by fine-tuning the leave-one-out model on synthetic episodes (Fine-tuning). PrimalPFN is averaged over three training seeds; the other rows use a single seed.

MethodSCISCA
PrimalPFN0.3380.3210.989
Leave-one-out0.6900.6905.387
Fine-tuning0.2340.4181.107

The leave-one-out models perform worse on the held-out class, which is expected since the class is not covered by the prior. As a remedy, fine-tuning the leave-one-out model for 200 epochs on synthetic episodes from unlabeled instances of the held-out class recovers most of the lost performance (lower than PrimalPFN on SC, not significantly different on CA with p=0.059p = 0.059 in a paired t-test, and higher on IS). This fine-tuning requires no additional solver labels beyond the two context instances and takes about 7 GPU-hours and 10 CPU-hours, compared with up to 300 CPU-hours of solver labeling for supervised predictors (see W2/Q2). We have revised the abstract and introduction to clarify that our claims concern problem classes covered by the pretraining prior (highlighted in blue).

Regarding the single target distribution per class, we follow the benchmark setting of prior work [1, 2]. The pretraining prior randomizes the generator parameters, such as the number of variables and the constraint density, across episodes (Section 3.2), so it is not restricted to the evaluated distribution. We will report the results on larger instances as soon as they are available, during the rebuttal period.

[1] Han, Q., et al. A GNN-Guided Predict-and-Search Framework for Mixed-Integer Linear Programming. ICLR, 2023.

[2] Geng, Z., et al. Differentiable Integer Linear Programming. ICLR, 2025.

[W2/Q2: Breakdown of computational cost]

Table 4 reports the complete preparation cost, including synthetic data generation, solver labeling, and the selection of the trust-region radius.

Table 4: Preparation cost of PrimalPFN and solver labeling cost of PS. Pretraining costs are measured over 1,200 episodes (10 epochs) with a single CPU worker and extrapolated to the full pretraining (700 epochs). Solver labeling costs are upper bounds, with up to one hour per instance (2 instances for PrimalPFN and 300 for PS). The Δ\Delta selection cost is also an upper bound, as explained below.

StageResource10 epochs (measured)TotalIncurred
Instance generationCPU-h0.4833.4Once (shared)
Synthetic label generationCPU-h0.0151.0Once (shared)
Forward + backwardGPU-h0.3524.6Once (shared)
Solver labeling: PrimalPFNCPU-h-≤ 2Per class
Solver labeling: PSCPU-h-≤ 300Per class
Δ\Delta selection (original)CPU-h-≤ 16.7Per class
Δ\Delta selection (extended)CPU-h-≤ 27.8Per class

The reported pretraining time of about 24 GPU-hours does not include synthetic data generation, which runs on CPU workers in parallel with GPU training. Two workers are sufficient to keep pace with the GPU, and we used 12 in practice, so data generation does not increase the wall-clock pretraining time. For this reason, it was not reported separately in the original manuscript; we now report it in Appendix B.2.2 (highlighted in blue). Note that synthetic label generation accounts for only about 3% of the CPU cost, since each annealing step is a single-variable flip with an incremental energy update, implemented with JIT compilation (Numba).

For radius selection, each method uses 20 additional instances per class, generated under the same settings as the benchmark, without requiring solver-labeled training data. With a 1,000-second time limit per run, the original protocol with three candidate radii requires up to 16.7 CPU-hours per method and class. The extended protocol with five candidate radii, used in the additional experiments, requires up to 27.8 CPU-hours. These are upper bounds, as solver runs may terminate before reaching the time limit. We now report these costs in Appendix B.3 (highlighted in red).

All preparation costs are independent of the number of target instances, since the same context and radius are reused for every target instance of a class. PrimalPFN incurs the pretraining cost once for the three classes and up to 2 CPU-hours of labeling per class, whereas PS incurs up to 300 CPU-hours of labeling per class. For reference, 300 CPU-hours equal the 1,000-second search budget of 1,080 target instances, whereas 2 CPU-hours equal that of about 7. Extending PrimalPFN to a new class incurs an additional cost of about 7 GPU-hours and 10 CPU-hours (see W1).

[Q3: Effect of solver-labeled training data under SCIP]

Following the reviewer's suggestion, we measured the prediction quality and the selected radii of the two models, together with the gaps over three training seeds (Table 5).

Table 5: Prediction quality, gaps, and selected radii of PrimalPFN and the variant with solver labels. Accuracy and AUROC are measured against the solver labels of the test instances. Accuracy, AUROC, and the Gurobi gap (at the best Δ\Delta) are mean ± standard deviation over three training seeds. The SCIP gap and the radius selected with SCIP are from the single run of the original manuscript.

ClassModelAccuracy (%)AUROCGap, Gurobi (%)Gap, SCIP (%)Δ\Delta, SCIP
SCPrimalPFN98.97 ± 0.020.9959 ± 0.00010.338 ± 0.0020.2320
w/ solver labels99.03 ± 0.010.9965 ± 0.00000.212 ± 0.0110.3220
ISPrimalPFN80.42 ± 0.010.8731 ± 0.00030.321 ± 0.0161.750
w/ solver labels80.58 ± 0.070.8768 ± 0.00030.402 ± 0.0212.060
CAPrimalPFN89.23 ± 0.020.8727 ± 0.00050.989 ± 0.0202.0620
w/ solver labels90.25 ± 0.020.9044 ± 0.00020.671 ± 0.0331.430

Overall prediction accuracy and AUROC do not degrade. Adding solver-labeled training data improves accuracy and AUROC on all three classes, but the gain is marginal on SC and IS (at most 0.16 percentage points in accuracy) and clear only on CA (1.02 points). The radii selected with SCIP are also identical for the two models on SC and IS, so the difference does not come from the radius selection.

Regarding reproducibility, we did not repeat the SCIP runs, but under Gurobi with three seeds, the variant with solver labels again has a higher gap than PrimalPFN on IS, at every tested Δ\Delta (Appendix D.7), so the improvement on IS in the original manuscript was not reproduced. On SC and CA, it has a lower gap. These results suggest that improvements in overall prediction quality do not necessarily translate into better downstream search performance. Predict-and-Search fixes only the most confident predictions, so changes in this subset may affect the final solutions differently from overall prediction metrics. Section 4.2 now notes the three-seed result on IS (highlighted in red).

Reviewer_bSE9OpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PrimalPFN, a framework for primal solution prediction in binary ILPs using in-context learning. The method generates synthetic training labels through randomized graph-based initialization followed by optimization-aware refinement. A PFN-based architecture is pretrained to predict variable assignments for query ILP instances using labeled context instances. At inference, two Gurobi-solved instances serve as context, allowing the pretrained model to predict variable probabilities without further parameter updates. These predictions guide the downstream Predict-and-Search procedure. The main contribution is reducing dependence on solver-generated training labels while maintaining competitive solution quality.

Critical strengths

The paper addresses the expensive generation of solver-labeled data for ILP solution prediction through an interesting combination of optimization-aware synthetic labeling and in-context learning. The approach achieves competitive solution quality with substantially fewer solver-labeled instances, particularly showing promising results on independent set problems. The context-label experiments and prior ablations also provide useful evidence supporting the proposed method.

Critical weaknesses
  1. Limited analysis of context-size sensitivity. Figure 4 compares PrimalPFN using two context instances against PS trained with varying numbers of labeled instances. While this demonstrates label efficiency at the chosen context size, it does not establish how PrimalPFN's performance changes with fewer or additional context instances. Appendix D.5 examines different selections of two context instances, but not the effect of context size itself.
  2. Appendix D.1 evaluates SCIP as the downstream solver while retaining the original predictor labels. Since PrimalPFN adapts through solver-generated context labels, it remains unclear whether comparable performance would be obtained when those context labels are generated by SCIP instead of Gurobi.
Questions for authors
  1. What motivates the choice of two context instances? In particular, how would PrimalPFN perform with one context instance or with more than two?
  2. Have the authors evaluated the use of SCIP-generated context labels with the same pretrained PrimalPFN? Would this change prediction quality or downstream performance compared with Gurobi-generated context labels?
2 comments

We sincerely thank the reviewer for the thoughtful evaluation of our work. It is encouraging that the reviewer considers the combination of optimization-aware synthetic labeling and in-context learning interesting, and the context-label experiments and prior ablations useful evidence for the method. We respond to the two concerns below with additional experiments.

Note that the additional experiments throughout our responses were conducted on a different hardware setup (AMD EPYC 9354 CPU (32 cores) + NVIDIA RTX A6000 GPU (48GB)) from that of the original manuscript due to hardware availability. Except for results explicitly identified as originating from the original manuscript, all results in the additional tables were obtained under this setup. Unless otherwise noted, Δ\Delta is selected by the protocol in Appendix B.3 over {0,10,20,40,80}\{0, 10, 20, 40, 80\}, which we extended from the original {0,20,40}\{0, 20, 40\} in response to Reviewer ok6F's comment on the range of Δ\Delta. For statistical comparisons, we first obtain one gap per test instance for each method by averaging across training seeds when multiple runs are available, or using the single-run gap otherwise. We then perform paired t-tests on the resulting 100 matched instance-level gaps.


[W1/Q1: Effect of context size]

Following the reviewer's suggestion, Table 1 reports the gap for Nctx∈{1,2,4,8}N_{ctx} \in \{1, 2, 4, 8\} at inference time, using the same model pretrained with Nctx=2N_{ctx} = 2. The differences across context sizes are not statistically significant on any class (paired t-test). Prediction accuracy is also stable across context sizes, varying by at most 0.026 percentage points (SC: 98.960–98.972%, IS: 80.424–80.450%, CA: 89.249–89.263%). Unlike in tabular PFNs, which commonly use hundreds to thousands of labeled rows as context, each context instance here carries 4,000–6,000 labeled variables, so a single solved instance already provides ample supervision. In tabular PFNs, each context row represents an individual labeled example, consisting of an input feature vector and its target label. On IS and CA, the benefit of the context appears between no context and one instance, since a variant trained without the context branch has significantly higher gaps (first row of Table 1, p<0.001p < 0.001). We chose Nctx=2N_{ctx} = 2 in advance as a small labeling budget and did not tune it, and Table 1 shows that the results are not sensitive to this choice. Since the model was pretrained with a fixed context size, pretraining with variable context sizes may be needed to benefit from larger contexts.

Table 1: Gap (%) for different context sizes at inference time, using the model pretrained with Nctx=2N_{ctx} = 2. The first row is a variant trained without the context branch. PrimalPFN is averaged over three training seeds; the other rows use a single seed.

Context sizeSCISCA
0 (w/o context branch)0.4230.5471.374
10.3190.3311.032
2 (PrimalPFN)0.3380.3210.989
40.3100.3320.948
80.3030.3470.967

A larger context also increases the cost. From Nctx=2N_{ctx} = 2 to 88, the inference time per instance grows from 0.63 to 4.61 GPU-seconds, and the estimated pretraining time from 21.4 to 160.6 GPU-hours.

[W2/Q2: Performance with SCIP-labeled contexts]

Following the reviewer's suggestion, we relabeled the same two context instances of each class with SCIP and evaluated the same pretrained PrimalPFN, with Gurobi as the downstream solver (Table 2). The gaps are not significantly different from those with Gurobi-labeled contexts on any class (paired t-test), the accuracy and AUROC of the predictions are nearly identical, and the same Δ\Delta is selected in every case. Together with Appendix D.1, where only the downstream solver is replaced by SCIP, this indicates that PrimalPFN does not depend on a specific solver. This is consistent with how PrimalPFN uses the context. It is pretrained without labels from any solver, so it is not tied to the solutions of a particular one, and the context serves to convey how instances of the class are labeled rather than particular solutions. The solutions of two solvers may differ in individual variables but reflect the same relationship, in the same way that the results are stable across different context selections (Appendix D.5).

Table 2: Gap (%) of PrimalPFN with context labels generated by Gurobi and by SCIP. PrimalPFN is averaged over three training seeds; the other row uses a single seed.

Context labelsSCISCA
Gurobi (PrimalPFN)0.3380.3210.989
SCIP0.2980.3210.960
AuthorsOpenReview ↗

Kind reminder about our response

Dear Reviewer bSE9,

As the discussion period ends soon, we would be grateful to know whether our response has addressed your concerns. If any questions remain, we would be glad to answer them. We thank you again for your time.

Best regards, The authors

Reviewer_X23BOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PrimalPFN, which applies the in-context learning framework of prior-data fitted networks (PFNs) to primal solution prediction for binary integer linear programs. During pretraining, it replaces solver labels with synthetic labels generated through random initialization and optimization-aware simulated annealing. At test time, two solver-labeled instances serve as a shared context for each problem class. Without parameter updates, the model predicts variable-wise marginal probabilities for query instances. These predictions guide solver search through Predict-and-Search (PS). Experiments on set covering, independent set, and combinatorial auctions show solution quality comparable to or better than class-specific supervised predictors, with substantially fewer solver labels.

Critical strengths
  1. Supervised methods typically require many solver-labeled training and validation instances for each problem class. PrimalPFN offers a useful way to reduce this cost through synthetic pretraining and adaptation from a few solved examples. The same context can be reused across queries. This makes the approach particularly relevant to repeated solving within the same distribution.
  2. With only two solver-labeled context instances per class, PrimalPFN performs comparably to PS on SC and CA. On IS, it achieves a lower average gap and finds better solutions on 75 of the 100 test instances. Gap-over-time curves, per-instance comparisons, and results across different context selections provide useful evidence beyond average performance.
  3. Removing optimization-aware refinement noticeably degrades downstream performance. This supports its importance in label generation. Shuffling context labels also substantially reduces performance while preserving label frequencies. This suggests that the model uses instance–label correspondences, rather than relying only on graph structure or problem class.
Critical weaknesses
  1. The experiments support the value of optimization-aware labels and correct context. However, they do not clearly isolate the additional benefits of the randomized rule prior and in-context learning. We suggest adding fixed-rule pretraining and synthetic pretraining followed by fine-tuning on two solver-labeled instances as baselines.
  2. The ideal minimizer of the PFN objective matches the posterior predictive distribution under the synthetic task prior. This does not directly guarantee transfer to Gurobi-generated labels. The paper treats solver labeling as another rule but provides limited analysis of its relationship to the synthetic rule family. Reporting synthetic-label feasibility rates, constraint violations, objective quality, and statistical differences from solver labels would better support the prior design and transfer mechanism.
  3. Pretraining and testing cover the same three generator-based problem classes. The results therefore mainly support label-efficient adaptation within known classes. Generalization to unseen classes, larger instances, or shifted distributions remains untested. Experiments across problem sizes or generator settings would be helpful. Broader class coverage would further strengthen the evidence.
  4. The CPU model, available computational resources, and solver thread settings are not sufficiently specified. Appendix B.3 describes how the trust-region radius is selected, but the final values for each method and problem class should also be reported. These details would clarify reproducibility and solver usage beyond the two labeled context instances.
  5. The related work could also cover data acquisition, transferable representations, and learning-guided search. For example, [1–2] provide context for synthetic instance generation and the limitations of class-specific generators. Reference [3] addresses shared representations and transfer across tasks. References [4–6] offer other approaches to guiding primal search with learned models. References [1] Geng, Z., et al. A Deep Instance Generative Framework for MILP Solvers Under Limited Data Availability. NeurIPS, 2023. [2] Liu, H., et al. MILP-StuDio: MILP Instance Generation via Block Structure Decomposition. NeurIPS, 2024. [3] Cai, J., Huang, T., and Dilkina, B. Multi-task Representation Learning for Mixed Integer Linear Programming. arXiv:2412.14409, 2024. [4] Sonnerat, N., Wang, P., Ktena, I., Bartunov, S., and Nair, V. Learning a Large Neighborhood Search Algorithm for Mixed Integer Programs. arXiv:2107.10201, 2021. [5] Huang, T., Ferber, A. M., Tian, Y., Dilkina, B., and Steiner, B. Searching Large Neighborhoods for Integer Linear Programs with Contrastive Learning. ICML, 2023. [6] Huang, T., Ferber, A. M., Zharmagambetov, A., Tian, Y., and Dilkina, B. Contrastive Predict-and-Search for Mixed Integer Linear Programs. ICML, 2024.
Questions for authors

see above

4 comments

We appreciate the reviewer's detailed review and constructive feedback on our paper. We are encouraged by the reviewer's recognition of the extensive empirical analyses and practical value of our work. In the following, we address each of the reviewer's concerns with additional results and clarifications.

Note that the additional experiments throughout our responses were conducted on a different hardware setup (AMD EPYC 9354 CPU (32 cores) + NVIDIA RTX A6000 GPU (48GB)) from that of the original manuscript due to hardware availability. Except for results explicitly identified as originating from the original manuscript, all results in the additional tables were obtained under this setup. Unless otherwise noted, Δ\Delta is selected by the protocol in Appendix B.3 over {0,10,20,40,80}\{0, 10, 20, 40, 80\}, which we extended from the original {0,20,40}\{0, 20, 40\} in response to Reviewer ok6F's comment on the range of Δ\Delta. For statistical comparisons, we first obtain one gap per test instance for each method by averaging across training seeds when multiple runs are available, or using the single-run gap otherwise. We then perform paired t-tests on the resulting 100 matched instance-level gaps.

In the revised manuscript, changes made in response to this review are highlighted in orange, and other changes are highlighted in blue.


[W1: Individual contributions of ICL and the randomized rule prior]

Following the reviewer's suggestion, we evaluated three additional baselines, namely (1) a variant without the context branch pretrained on synthetic labels, (2) the same variant fine-tuned on the two solver-labeled instances, and (3) a variant pretrained using a single fixed labeling rule.

In-context learning. Table 1 compares PrimalPFN with a variant that removes the context branch (w/o context) while using the same synthetic training data. We evaluate this variant both before and after fine-tuning on the two solver-labeled instances used as PrimalPFN's context. On SC, PrimalPFN is not significantly different from either variant (paired t-test, p=0.20p = 0.20 and p=0.37p = 0.37, respectively). On IS and CA, PrimalPFN significantly outperforms both (p<0.001p < 0.001). Fine-tuning improves the variant, most visibly on CA, but it still performs significantly worse than PrimalPFN on IS and CA. These results indicate that, on IS and CA, the gain does not come from synthetic pretraining alone, and that using the solver-labeled instances in context is more effective than the evaluated gradient-based fine-tuning alternative.

Table 1: Gap (%) of PrimalPFN and a variant without the context branch, with and without fine-tuning (FT) on the two solver-labeled instances. PrimalPFN is averaged over three training seeds, while the other rows use a single seed.

MethodSCISCA
PrimalPFN0.3380.3210.989
w/o context0.4230.5471.374
w/o context + FT0.3970.5301.136

Randomized rule prior. Table 2 compares PrimalPFN with a variant pretrained under a single fixed rule ω∗\omega^*. As a strong baseline, we select an oracle ω∗\omega^* for each class, namely the rule whose labels are closest in Hamming distance to the solver labels on 60 solver-labeled training instances. Since this selection uses solver labels, the comparison favors the fixed-rule variant.

Table 2: Gap (%) of PrimalPFN and the fixed-rule variant. All reported gaps are averaged over the same 100 test instances. PrimalPFN is averaged over three training seeds, while the fixed-rule variant uses a single seed.

MethodSCISCA
PrimalPFN0.3380.3210.989
Fixed-rule (oracle)0.2630.3400.889

On SC and CA, the fixed-rule variant achieves lower gaps than PrimalPFN (0.263% vs. 0.338% and 0.889% vs. 0.989%, respectively). On IS, PrimalPFN achieves a slightly lower gap (0.321% vs. 0.340%), with feasible solutions on all 100 test instances for both methods. Notably, at Δ=0\Delta = 0, the fixed-rule variant yields infeasible subproblems on 30 of the 100 IS instances, whereas PrimalPFN yields none. The same tendency appears on CA at Δ=0\Delta = 0 (4 infeasible instances vs. none). These results suggest that pretraining over diverse rules may improve the robustness of confident predictions under tight trust-region constraints, although it does not consistently yield lower gaps across all classes.

[W2: Synthetic-label statistics and transfer to solver labels]

We agree that the PFN objective does not guarantee transfer to solver labels, and we do not claim that the solver's labeling lies within the synthetic rule family. Following the reviewer's suggestion, Table 3 reports the statistics of the synthetic labels.

Table 3: Statistics of the synthetic labels, computed on 4 instances per class with 1,000 synthetic labels per instance. The feasibility rate is the fraction of feasible labels, the constraint violation is the fraction of violated constraints, the objective gap is measured against the solver solution, and the Hamming distance is the fraction of variables that differ from the nearest solver solution.

ClassFeasibility rate (%)Constraint violation (%)Objective gap (%)Hamming distance (%)
SC4.101.67981.921.69
IS27.4317.919.0836.43
CA24.3222.1522.9127.28

The synthetic labels differ substantially from solver labels. Most of them are infeasible, and their objective values differ considerably from those of the solver, especially on SC, where the small Hamming distance largely reflects the sparsity of the solutions. Since most synthetic assignments are infeasible, their objective gaps should be interpreted as descriptive differences in objective values rather than feasible solution quality. These results suggest that successful transfer does not require synthetic labels to closely match solver solutions in feasibility or objective value, although useful structural similarities may still exist.

Rather than fitting a single fixed labeling rule, PrimalPFN learns to infer the relationship between instances and labels from the context and apply it to the query. At test time, the solver-labeled context provides information about the target labeling behavior. This interpretation is supported by two results. Removing the optimization-aware refinement from the prior degrades downstream performance (Appendix D.4), and removing the context branch does so on IS and CA (Table 1). Together, these results provide empirical evidence for transfer without assuming that the solver's labeling belongs to the synthetic rule family.

[W3: Generalization to unseen distributions]

To examine classes that are not represented during pretraining, we conducted leave-one-class-out experiments, in which PrimalPFN is pretrained on two classes and evaluated on the held-out class (Table 4).

Table 4: Gap (%) on each class when it is included in pretraining (PrimalPFN), held out (Leave-one-out), and added by fine-tuning the leave-one-out model on synthetic episodes (Fine-tuning). PrimalPFN is averaged over three training seeds; the other rows use a single seed.

MethodSCISCA
PrimalPFN0.3380.3210.989
Leave-one-out0.6900.6905.387
Fine-tuning0.2340.4181.107

The leave-one-out models perform worse on the held-out class, which is expected since the class is not covered by the prior. As a remedy, fine-tuning the leave-one-out model for 200 epochs on synthetic episodes from unlabeled instances of the held-out class recovers most of the lost performance (lower than PrimalPFN on SC, not significantly different on CA with p=0.059p = 0.059 in a paired t-test, and higher on IS). This fine-tuning requires no additional solver labels beyond the two context instances and takes about 7 GPU-hours and 10 CPU-hours, compared with up to 300 CPU-hours of solver labeling for supervised predictors. We have revised the abstract and introduction to clarify that our claims concern problem classes covered by the pretraining prior (highlighted in blue).

Regarding larger instances and shifted distributions, our evaluation follows the benchmark setting of prior work (Han et al., 2023; Geng et al., 2025), with a single target distribution per class. The pretraining prior randomizes the generator parameters, such as the number of variables and the constraint density, across episodes (Section 3.2), so it is not restricted to the evaluated distribution. We will report the results on larger instances as soon as they are available, during the rebuttal period.

[W4: Reproducibility details]

The experiments in the original manuscript were conducted on an AMD EPYC 9754 CPU (128 cores) and a single NVIDIA GeForce RTX 5090 GPU (32GB). Following Predict-and-Search (Han et al., 2023), each solver run uses a single thread, both for labeling and for evaluation. Table 5 reports the trust-region radius selected for each method and class in the original manuscript. Results for every Δ\Delta over an extended range, averaged over three training seeds, are reported in Appendix D.7.

Table 5: Trust-region radius Δ\Delta selected for each method and class in the original manuscript (Gurobi).

MethodSCISCA
PS204020
DiffILO204040
PrimalPFN20020
w/ solver labels2000

[W5: Expansion of related work]

Thank you for suggesting these relevant references. We have incorporated them into Section 5 of the revised manuscript. We added a paragraph on MILP instance generation for [1–2] and discuss [3–6] together with the other learning-based primal heuristics. The changes are highlighted in orange.

AuthorsOpenReview ↗

Additional results on larger instances

We thank the reviewer again. As promised in our response, we report the results on larger instances below. Note that the three-seed results, previously in Appendix D.7, are now in Appendix E.1 of the revised manuscript.

[Additional Experiment: Generalization to larger instances]

On the hardware setup of Appendix E, we conducted additional experiments on larger instances, namely the very-hard2 instances in D-MIPLIB [1]. On SC, these instances have twice as many variables as the original ones with the same number of constraints, whereas on IS and CA, both the variables and the constraints are doubled. Accordingly, we doubled k0k_0, k1k_1, and Δ\Delta for each class, except for k1k_1 on SC, which we kept at its original value because the number of variables set to one in a solution remains similar to that of the original instances. All models are those trained on the original instances and are applied without retraining. The comparison also includes RoME [2], a supervised cross-domain baseline that we trained with the authors' public implementation on the solver-labeled training instances of the three classes. Table 1 reports the results.

Table 1: Gap (%) on the larger instances. For PrimalPFN and the variant with solver labels, the context consists of two solved instances either from the original distribution (original context) or from the larger one (larger context). Values with ± are mean ± standard deviation over three training seeds.

MethodSCISCA
Gurobi0.98815.7061.496
PS0.356 ± 0.0330.781 ± 0.0141.184 ± 0.023
LP Relax.0.784infeas.1.881
DiffILO0.762 ± 0.1351.247 ± 0.2401.645 ± 0.347
RoME0.4441.2931.256
PrimalPFN (original context)0.456 ± 0.0130.144 ± 0.0190.979 ± 0.016
PrimalPFN (larger context)0.488 ± 0.0530.223 ± 0.0130.980 ± 0.021
w/ solver labels (original context)0.458 ± 0.0220.217 ± 0.0110.445 ± 0.027
w/ solver labels (larger context)0.483 ± 0.0260.252 ± 0.0120.386 ± 0.038

PrimalPFN remains effective on the larger instances. It attains the lowest gap on IS and a lower gap than PS, DiffILO, and RoME on CA. On SC, where PS attains the lowest gap, PrimalPFN remains below the LP-relaxation baseline and DiffILO (0.456% vs. 0.784% and 0.762%). Notably, for PrimalPFN, the context from the original instances works at least as well as the context from the larger instances, so the context need not match the size of the target instances. On IS, the original context even gives a lower gap (0.144% vs. 0.223%). One possible reason is that the solver labels of the larger instances are of lower quality, since standalone Gurobi leaves a much larger gap on the larger IS instances (15.7%) than on the original ones (3.3%).

[1] Huang, W. et al., Distributional MIPLIB: a Multi-Domain Library for Advancing ML-Guided MILP Methods, arXiv:2406.06954

[2] Pu, T., et al. RoME: Domain-Robust Mixture-of-Experts for MILP Solution Prediction across Domains. NeurIPS, 2025.

As the discussion period ends soon, we would be grateful to know whether our responses have addressed your concerns. Thank you again for your time.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Discussion for Reviewers and AC

Dear Reviewers and AC,

We thank the four reviewers for their careful reviews and for acknowledging the following strengths of our work.

  • Offers a label-efficient alternative to supervised primal heuristics. With only two solver-labeled instances per class, it matches PS on CA and outperforms it on IS.
  • Enables a single frozen model to reuse the same context across queries without task-specific updates, which suits repeated solving within a distribution.
  • Provides evidence for the central mechanism through context-label interventions and prior ablations.
  • Provides useful additional evidence that strengthens the empirical picture (e.g., label-efficiency curves, robustness to context selection, per-instance comparisons).
  • Presents an original, well-motivated idea that combines optimization-aware synthetic priors with cross-instance in-context learning for primal prediction.

The table below summarizes the main concerns of the reviewers and our responses. In the revised paper, the changes are highlighted.

ConcernByOur response
Source of gain: ICL and randomized priorX23B (W1, W2), vFAz (W1/Q1), ok6F (W2/Q2; follow-up Q3, Q4)We showed that removing the context branch increases the gap on IS and CA even after fine-tuning on the same solver labels and that the randomized prior avoids the infeasible subproblems of a fixed-rule prior at Δ = 0.
GeneralizationX23B (W3), vFAz (W1/Q1), ok6F (W1/Q1; follow-up Q1)We refined our claims to classes covered by the prior (Abstract, Section 1, and Section 6) and showed that fine-tuning on synthetic episodes recovers most of the performance lost under leave-one-class-out pretraining and that PrimalPFN transfers to larger instances without retraining.
Reproducibility, related work & minor clarificationsX23B (W4, W5), ok6F (W7, W8; follow-up Q8)We clarified the hardware and setup of each experiment and added related work (Section 5 and Appendices B and E).
Role and size of contextbSE9 (W1/Q1, W2/Q2), ok6F (W4/Q3; follow-up Q5)We showed that the gap does not change significantly with one to eight context instances or with SCIP-labeled contexts, which suggests that a single solved instance already conveys the labeling pattern of a class.
Computational costvFAz (W2/Q2), ok6F (W3/Q4; follow-up Q6)We reported the full cost including data generation, labeling, and Δ selection, and PrimalPFN needs about 19–30 CPU-hours per class, plus a one-time pretraining cost shared across classes, against 317–328 CPU-hours per class for PS (Figure 1(d) and Appendices B.2.2 and B.3).
Solver-label variantvFAz (Q3), ok6F (follow-up Q9)We showed that solver labels improve accuracy and AUROC on all classes but improve the gap consistently only on CA and refined the claim to CA (Section 1 and Section 4.2).
Baseline comparisonok6F (W5/Q5, W6/Q6/Q7; follow-up Q2, Q7)We added LP-relaxation and RoME baselines, extended the trust-region radii, and repeated the main comparison over three seeds. PrimalPFN attains a lower gap than PS on IS, a comparable gap on CA, and a higher gap on SC (Section 4.2 and Appendix E.1). The RoME results and the extended radii are reported in our responses and will be added to Appendix E.

Accordingly, we have revised the manuscript, mainly to refine and clarify our claims, based on our new experimental results and analyses. The experiments conducted during the rebuttal period that are not yet in the manuscript will be added in the final version. We are also running the experiments that we committed to include in the final version.

  • Fine-tuning PrimalPFN itself under the protocol suggested by Reviewer ok6F
  • The "w/o context + FT" baseline over multiple seeds and learning rates
  • A fixed-rule variant chosen without solver labels
  • Three-seed results for the experiments currently reported with a single seed

Beyond these experiments, we will describe the role of the context more clearly, as suggested by Reviewer ok6F.

We are sincerely grateful to the reviewers and the AC for their time and thoughtful comments, which have helped us improve our paper.

Best regards,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.