- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes PrimalPFN, which applies the prior-data-fitted network (PFN) recipe to primal solution prediction for binary ILPs.
- Pretraining data. Synthetic episodes are built from Ecole-generated instances of the three target classes. Each episode's instances are labeled by a randomly sampled rule: a random MPNN produces an initial assignment, which simulated annealing then refines on a penalized objective.
- Architecture. Following NodePFN, each layer has two branches: GINE message passing within each instance, and attention from query tokens to context tokens across instances.
- Inference. Two Gurobi-solved instances of the target class serve as context. The predicted marginals are passed to Predict-and-Search (PS).
- Results. On the Distributional MIPLIB set cover (SC), independent set (IS), and combinatorial auctions (CA), PrimalPFN with two labeled instances roughly matches PS trained on 240 labeled instances on SC and CA, and beats it on IS (0.32% vs 0.53% gap). It beats DiffILO on all three classes. A variant that also pretrains on the 240 solver-labeled instances is best overall.
- Analyses. Interventions on the context labels, prior ablations, label-efficiency curves, and SCIP results are provided.
Critical strengths
- An original, well-motivated idea. Using PFN-style in-context learning for primal prediction is new. Building context from other solved instances, rather than from labeled nodes in the same graph, is a sensible adaptation.
- Convincing analysis of what the model uses. In the context-label interventions (Table 2, Figure 5), shuffling labels within an instance keeps label frequencies but breaks their correspondence with variables, and it badly hurts performance. This shows the model uses the instance–label correspondence. The prior ablation identifies the optimization-aware refinement step as the key part of the prior.
- Useful additional evidence. The label-efficiency curves (Figure 4), robustness to which context instances are drawn (Table 14), the SCIP results, and the per-instance win/draw/loss tables strengthen the empirical picture.
- Clear writing. The paper is well organized and easy to follow.
Critical weaknesses
-
The prior is class-specific, which narrows the main claim. Pretraining uses École generators for exactly the three evaluation classes, and Appendix C.1 says unseen classes are "a different setting". The abstract and introduction ("for each problem class, PrimalPFN requires only two solver-labeled instances") suggest adapting to a new class from a few examples. What is shown is adaptation within classes whose generators were seen during pretraining. To support the in-context-learning framing, please add one of:
- leave-one-class-out results (pretrain on two classes, test on the third), or
- results on a binary class outside the prior, such as another Distributional MIPLIB binary class or MIPLIB instances.
-
It is unclear how much comes from in-context learning versus synthetic labels. In Table 2, the "Prior" context uses synthetic-rule labels and no solver labels at all, yet gives 0.27% (SC) and 0.44% (IS). That is close to the 0.23% / 0.32% obtained with solver-labeled context, and on IS it already beats PS (0.53%). Only CA depends strongly on solver context (3.12% → 0.64%). So on SC and IS, most of the gain seems to come from pretraining on annealing-refined labels for in-class instances. Two baselines are needed to show the PFN machinery is necessary:
- (a) a standard supervised GNN, either the PS architecture or PrimalPFN without the context branch, trained on the same synthetic labeled data, with and without fine-tuning on the two solver-labeled instances;
- (b) PrimalPFN fine-tuned on the two context instances instead of using them in context.
-
The compute comparison is incomplete. Figure 1(d) and the abstract stress a drop from 300 to 2 CPU-hours of labeling. This leaves out the cost of generating synthetic labels (annealing over many large instances), about 24 GPU-hours of pretraining, and the effort required to design the prior. Pretraining is shared across classes, but here the prior is built per class (Weakness 1), which weakens that argument. Please report the total CPU and GPU hours for the prior generation and pretraining next to the labeling cost.
-
The number of context instances is never varied. It is fixed at N_ctx = 2 in both pretraining and evaluation. A defining property of in-context learning is that predictions improve with more context. Please report results for N_ctx ∈ {1, 2, 4, 8} where memory allows, and discuss how cost grows. Each query attends to roughly 8k–12k context tokens, and test instances have only 4k–6k variables. This also matters for applying the method to larger instances.
-
Improvements are modest, and the statistics are limited.
- On SC, PrimalPFN is worse than PS: 0.23% vs 0.18% gap, 12 wins and 19 losses.
- On CA, it is effectively tied: 0.64% vs 0.67%, 53 wins and 47 losses.
- Only IS shows a clear win.
- Results come from a single pretraining run and a single PS training run (Figure 4 says "one run each"), with no confidence intervals or significance tests. Please report variability over pretraining seeds and PS training seeds, along with paired tests.
-
Details of the downstream search are unclear, and a non-learned baseline is missing. The PS parameters k₀ and k₁ (Table 8) are fixed per benchmark. Were they taken from Han et al. (2023) or tuned, and if tuned, for which predictor? The trust-region radius Δ is chosen per method from only {0, 20, 40}; please show sensitivity over a wider range. A non-learned baseline, such as LP-relaxation values fed into the same PS procedure, would show how much the learned predictions add.
-
Scope and related work. Evaluation covers only pure-binary, generator-based classes, all of which have an easy feasibility structure. Cross-domain or foundation-style predictors (Li et al., 2025; RoME; Pu et al., 2025) are cited but not compared. Since PrimalPFN is pitched as label-efficient adaptation, a comparison with a cross-domain pretrained predictor adapted to the target class with few labels would be informative.
-
Minor issues.
- "Two instances" understates the supervision provided, which is 8k–12k labeled variables.
- The Bayesian interpretation assumes the solver's labeling lies within the support of the prior. A short discussion of what happens when the prior is misspecified would help.
- Code is promised only on publication.
Questions for authors
- How does PrimalPFN perform when the test class is excluded from pretraining (leave-one-class-out)?
- How does a supervised GNN trained on the same synthetic labels perform, with and without fine-tuning on the two solver-labeled instances?
- How do accuracy, gap, and inference cost change as N_ctx varies?
- What are the total compute costs (CPU and GPU hours) of generating the prior and pretraining?
- What is the variability across pretraining seeds and PS training seeds, and are the IS improvements significant under a paired test?
- Were k₀ and k₁ tuned, and if so, for which method? How sensitive are the results to Δ beyond {0, 20, 40}?
- How does a PS run guided by LP-relaxation values (no learning) compare?
11 comments
[W2/Q2: Individual contributions of ICL and the randomized rule prior]
In-context learning. Table 2 compares PrimalPFN with a variant that removes the context branch (w/o context) while using the same synthetic training data. We evaluate this variant both before and after fine-tuning on the two solver-labeled instances used as PrimalPFN's context. On SC, PrimalPFN is not significantly different from either variant (paired t-test, and , respectively). On IS and CA, PrimalPFN significantly outperforms both (). Fine-tuning improves the variant, most visibly on CA, but it still performs significantly worse than PrimalPFN on IS and CA. These results indicate that, on IS and CA, the gain does not come from synthetic pretraining alone, and that using the solver-labeled instances in context is more effective than the evaluated gradient-based fine-tuning alternative.
Table 2: Gap (%) of PrimalPFN and a variant without the context branch, with and without fine-tuning (FT) on the two solver-labeled instances. PrimalPFN is averaged over three training seeds, while the other rows use a single seed.
| Method | SC | IS | CA |
|---|---|---|---|
| PrimalPFN | 0.338 | 0.321 | 0.989 |
| w/o context | 0.423 | 0.547 | 1.374 |
| w/o context + FT | 0.397 | 0.530 | 1.136 |
Regarding (b), directly fine-tuning PrimalPFN would require specifying how its context branch is used during fine-tuning and inference, making it difficult to isolate adaptation through gradient updates from adaptation through context. We therefore evaluate a conventional gradient-based alternative using the variant without the context branch, which shares the backbone and synthetic training data with PrimalPFN and is fine-tuned on the same two solver-labeled instances. While this is not an exact substitute for fine-tuning PrimalPFN itself, it provides a comparison under the same solver-label budget.
Randomized rule prior. Table 3 compares PrimalPFN with a variant pretrained under a single fixed rule . As a strong baseline, we select an oracle for each class, namely the rule whose labels are closest in Hamming distance to the solver labels on 60 solver-labeled training instances. Since this selection uses solver labels, the comparison favors the fixed-rule variant.
Table 3: Gap (%) of PrimalPFN and the fixed-rule variant. All reported gaps are averaged over the same 100 test instances. PrimalPFN is averaged over three training seeds, while the fixed-rule variant uses a single seed.
| Method | SC | IS | CA |
|---|---|---|---|
| PrimalPFN | 0.338 | 0.321 | 0.989 |
| Fixed-rule (oracle) | 0.263 | 0.340 | 0.889 |
On SC and CA, the fixed-rule variant achieves lower gaps than PrimalPFN. On IS, PrimalPFN achieves a slightly lower gap, with feasible solutions on all 100 test instances for both methods. Notably, at , the fixed-rule variant yields infeasible subproblems on 30 of the 100 IS instances, whereas PrimalPFN yields none. The same tendency appears on CA at (4 infeasible instances vs. none). These results suggest that pretraining over diverse rules may improve the robustness of confident predictions under tight trust-region constraints, although it does not consistently yield lower gaps across all classes.
[W3/Q4: Breakdown of computational cost]
Table 4 reports the CPU and GPU hours for synthetic data generation and pretraining, alongside the solver labeling cost.
Table 4: Computational cost of PrimalPFN pretraining and of solver labeling. Pretraining costs are measured over 1,200 episodes (10 epochs) with a single CPU worker and extrapolated to the full pretraining (700 epochs). Solver labeling costs are upper bounds, with up to one hour per instance (2 instances for PrimalPFN and 300 for PS).
| Stage | Resource | 10 epochs (measured) | Total | Incurred |
|---|---|---|---|---|
| Instance generation | CPU-h | 0.48 | 33.4 | Once (shared) |
| Synthetic label generation | CPU-h | 0.015 | 1.0 | Once (shared) |
| Forward + backward | GPU-h | 0.35 | 24.6 | Once (shared) |
| Solver labeling: PrimalPFN | CPU-h | - | ≤ 2 | Per class |
| Solver labeling: PS | CPU-h | - | ≤ 300 | Per class |
Synthetic data generation runs on CPU workers in parallel with GPU training. Two workers are sufficient to keep pace with the GPU, and we used 12 in practice, so data generation does not increase the wall-clock pretraining time. For this reason, it was not reported separately in the original manuscript; we now report it in Appendix B.2.2 (highlighted in blue). Note that synthetic label generation accounts for only about 3% of the CPU cost, since each annealing step is a single-variable flip with an incremental energy update, implemented with JIT compilation (Numba).
We agree that the pretraining cost is amortized only over the classes included in pretraining. It is shared across these three classes, whereas supervised predictors incur the labeling cost for each of them. Extending PrimalPFN to a new class incurs an additional cost of about 7 GPU-hours and 10 CPU-hours (see W1/Q1).
[W4/Q3: Effect of context size]
Performance. Table 5 reports the gap for at inference time, using the same model pretrained with . The differences across context sizes are not statistically significant on any class (paired t-test). Prediction accuracy is also stable across context sizes, varying by at most 0.026 percentage points (SC: 98.960–98.972%, IS: 80.424–80.450%, CA: 89.249–89.263%). Unlike in tabular PFNs, which commonly use hundreds to thousands of labeled rows as context, each context instance here carries 4,000–6,000 labeled variables, so a single solved instance already provides ample supervision. In tabular PFNs, each context row represents an individual labeled example, consisting of an input feature vector and its target label. On IS and CA, the benefit of the context appears between no context (see W2/Q2) and one instance. We chose in advance as a small labeling budget and did not tune it, and Table 5 shows that the results are not sensitive to this choice. Since the model was pretrained with a fixed context size, pretraining with variable context sizes may be needed to benefit from larger contexts.
Table 5: Gap (%) for different context sizes at inference time, using the model pretrained with . PrimalPFN is averaged over three training seeds; the other rows use a single seed.
| Context size | SC | IS | CA |
|---|---|---|---|
| 1 | 0.319 | 0.331 | 1.032 |
| 2 (PrimalPFN) | 0.338 | 0.321 | 0.989 |
| 4 | 0.310 | 0.332 | 0.948 |
| 8 | 0.303 | 0.347 | 0.967 |
Cost. Tables 6 and 7 report the pretraining and inference costs for each context size. From to , the GPU time grows by about 7.5 times in pretraining and 7.3 times in inference, whereas the CPU cost and peak memory grow roughly linearly. We therefore do not adopt a larger context, since it brings no significant gain in downstream performance while the cost increases substantially.
Table 6: Estimated pretraining cost (700 epochs) by context size, extrapolated from a 1-epoch simulation. The values for differ from those in Table 4, which are extrapolated from 10 epochs.
| Context size | CPU-h | GPU-h | Peak GPU Memory (GiB) |
|---|---|---|---|
| 1 | 23.8 | 11.1 | 1.28 |
| 2 (PrimalPFN) | 35.0 | 21.4 | 1.89 |
| 4 | 58.1 | 53.1 | 3.09 |
| 8 | 104.3 | 160.6 | 5.49 |
Table 7: Inference cost per test instance by context size, averaged over the three classes. Only the GPU forward time is measured, whereas Figure 6 reports the end-to-end prediction time, including file I/O and graph conversion.
| Context size | GPU-s | Peak GPU Memory (GiB) |
|---|---|---|
| 1 | 0.34 | 0.34 |
| 2 (PrimalPFN) | 0.63 | 0.49 |
| 4 | 1.55 | 0.77 |
| 8 | 4.61 | 1.35 |
[W5/Q5: Magnitude and statistical significance of improvements]
We agree that PrimalPFN does not outperform PS on every class. Our main claim, however, is not to outperform supervised predictors, but to achieve competitive solution quality with two solver-labeled instances instead of 300.
To assess the variability, we trained each learning-based method in Table 8 (see W6/Q6/Q7 below) with three seeds and report the mean and standard deviation. Compared with PS at the best of each method, PrimalPFN achieves a lower gap on IS (paired t-test, ) and a comparable gap on CA (). On SC, PS achieves a lower gap (), but adding the same solver-labeled training instances used by PS reduces PrimalPFN's gap from 0.338% to 0.212%, narrowing the difference to PS (0.164%, ). This suggests that the labeling budget contributes to the performance difference on SC.
We appreciate the reviewer's effort in reviewing our paper. We are happy to see that the reviewer recognized several strengths of this work, including the originality and motivation of the idea, the convincing analysis of what the model uses, and the clarity of the writing. Below, we address each concern with additional results and clarifications.
Note that the additional experiments throughout our responses were conducted on a different hardware setup (AMD EPYC 9354 CPU (32 cores) + NVIDIA RTX A6000 GPU (48GB)) from that of the original manuscript due to hardware availability. Except for results explicitly identified as originating from the original manuscript, all results in the additional tables were obtained under this setup. Unless otherwise noted, Δ is selected by the protocol in
Appendix B.3over {0,10,20,40,80}, which we extended from the original {0,20,40} in response to the reviewer's comment on the range of Δ (see W6/Q6/Q7). For statistical comparisons, we first obtain one gap per test instance for each method by averaging across training seeds when multiple runs are available, or using the single-run gap otherwise. We then perform paired t-tests on the resulting 100 matched instance-level gaps.[W1/Q1: Generalization to unseen classes]
Following the reviewer's suggestion, we conducted leave-one-class-out experiments, in which PrimalPFN is pretrained on two classes and evaluated on the held-out class (Table 1).
Table 1: Gap (%) on each class when it is included in pretraining (PrimalPFN), held out (Leave-one-out), and added by fine-tuning the leave-one-out model on synthetic episodes (Fine-tuning). PrimalPFN is averaged over three training seeds; the other rows use a single seed.
The leave-one-out models perform worse on the held-out class, which is expected since the class is not covered by the prior. As a remedy, fine-tuning the leave-one-out model for 200 epochs on synthetic episodes from unlabeled instances of the held-out class recovers most of the lost performance (lower than PrimalPFN on SC, not significantly different on CA with p=0.059 in a paired t-test, and higher on IS). This fine-tuning requires no additional solver labels beyond the two context instances and takes about 7 GPU-hours and 10 CPU-hours, compared with up to 300 CPU-hours of solver labeling for supervised predictors (see W3/Q4). We have revised the abstract and introduction to clarify that our claims concern problem classes covered by the pretraining prior (highlighted in blue).