acceptodds
Under review as a conference paper at ICLR 2027

ORDO: Operation-level Round-aware Dynamic Ordering for MIP Presolve

Abstract

Presolve strongly affects mixed-integer programming (MIP) performance, yet learning-based methods only optimize parameter configurations and cannot ex- press the non-commutative temporal dependencies among actions, whose default order is nearly unique on most domains, yet functionally necessary: artificially shuffling the order of the same sequence inflates the tail of the solve-time distribu- tion by up to several-fold. We recast presolve planning as autoregressive sequence generation over a unified atomic action space, moving the decision object to ac- tion sequences; we call this framework ORDO—Operation-level Round-aware Dy- namic Ordering for MIP Presolve. Its payoff is cross-domain generalization: on multiple unseen domains it attains end-to-end zero-shot speedup—to our knowl- edge the first for presolve action sequences—varying by domain and not explained by corpus richness, the strongest domain reaching the largest speedup once racing is added. Deployment uses sequence racing, in which candidate sequences run concurrently and the winner is kept, enabled by an execution-and-observation fa- cility, added by modifying the SCIP source, that injects sequences along the native path and records which actions actually execute and in which round.

Primary area:
optimization
Keywords:
mixed-integer programming, presolving, sequence generation, autoregressive models, parallel racing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_r2SzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ORDO, a framework that formulates mixed-integer programming (MIP) presolve planning as conditional autoregressive generation of executable action sequences. Its action space includes presolver routines, propagator callbacks, constraint-handler callbacks, and round-boundary and termination tokens. A lightweight decoder-only Transformer is trained by behavior cloning on effective-action trajectories collected from SCIP’s default presolve policy. The authors modify SCIP to support sequence injection and execution monitoring. At deployment, generated sequences are evaluated through concurrent racing among custom-sequence, default-fallback, and no-presolve paths. Experiments on seven D-MIPLIB domains investigate order sensitivity, sequence replay, and cross-domain transfer. The results show improvements on several held-out domains, although the contribution of learned generation relative to replay, reduced presolve effort, and racing remains insufficiently isolated.

Critical strengths
  1. An interesting decision formulation. Treating presolve plans as explicit action sequences provides finer scheduling control than selecting a fixed configuration vector. Including constraint-handler callbacks and round boundaries makes this a useful direction for learned solver control.
  2. A valuable execution and observation facility. The SCIP modifications support externally specified sequences while reusing native presolve operations. The patched-default comparison and objective-value consistency checks provide useful empirical validation of this infrastructure.
  3. Useful empirical analysis beyond headline performance. The paper examines sequence diversity, order perturbations, fixed-sequence replay, winner composition, and cross-domain transfer. Reporting unfavorable domains and the additional computational cost of racing helps identify the framework’s practical limitations.
Critical weaknesses
  1. The learned model may not be necessary. ORDO is trained by imitating presolve trajectories generated by default SCIP. Since the training data do not contain better-than-SCIP solutions, it is unclear why the learned model should systematically outperform SCIP instead of simply reproducing its behavior. This concern is supported by Table 20, where model-generated sequences and direct replay of SCIP sequences perform almost the same on four domains
  2. The gains may come from the execution strategy rather than learned ordering. For example, the no-presolve path wins on 75% of SC instances and 43% of CFLP instances. This suggests that part of the gain may come from skipping presolve or changing how presolve is executed, rather than from learning a better action order. Simpler baselines such as a fixed sequence, sequence retrieval, or a small sequence library are therefore needed.
  3. Some theoretical and experimental claims are not fully supported. Proposition 1 assumes independent runtimes but later informally extends the conclusion to weakly correlated candidates without defining this condition. Moreover, the paper claims “4 accelerated, 2 tied and 1 failed,” while MIS has SGM =1.0140 and 𝑝=0.0352, which should be counted as a significant slowdown under the paper’s own rule. The correct summary appears to be 4 accelerated, 1 tied, and 2 failed.
Questions for authors
  1. Since ORDO is trained only on trajectories generated by default SCIP, why should the learned model be expected to outperform SCIP? Why not directly use SCIP, especially when Table 20 shows that replaying SCIP’s sequence performs similarly to the learned model on several domains?
  2. How does ORDO compare with much simpler alternatives, such as one fixed sequence learned from the training set, nearest-neighbor sequence retrieval, or selecting from a small library of presolve sequences?
  3. Can the authors clarify Proposition 1 and correct the reported transfer outcome? In particular, what assumption supports the “weakly correlated” extension, and why is MIS not counted as a failure despite SGM>1SGM>1 and p<0.05p<0.05?
2 comments

We thank the reviewer. Answered in the order given; ratios are against the same-batch default under solver-internal time, so below 1 is faster.

W1. The learned model may not be necessary.

Response: Thanks. In-domain it is not, and the paper states that as its first result: against direct replay on the same batch, generation reads 1.0077 [0.9867, 1.0292] on SC, 1.0163 [0.9959, 1.0367] on CFLP, 1.0008 on MIS and 0.9979 on MVC (Table 1), so what those domains release is the execution form and a candidate pool (§1, §3.4). The learned component separates on the one corpus-diverse, never-trained domain, NNV, at 0.9308 [0.8909, 0.9712] against replay.

W2. The gains may come from the execution strategy rather than learned ordering.

Response: Thanks. Four arms, same batch and denominator: SC 0.9904 / 0.9822 / 0.7457 / 0.7766 and CFLP 0.9327 / 0.9173 / 0.8760 / 0.7955 for content single-path, replay, np single arm and racing; the np path's share is printed, 75% of SC instances and 43% of CFLP against 1% on MVC. The selection rule costs 5.2 points (0.7251 to 0.7766). Where the model fails the mechanism is explicit: on CA its sequence (1.1761) is indistinguishable from replaying that instance's multiset (1.1730). The two adjustments are independent: a preset changing only plug-in parameters composes with our sequence (1.0001 / 1.0019) and no preset in 24 cells reorders the multiset (B.12.2).

W3. Some theoretical and experimental claims are not fully supported.

Response: Thanks. The assumption that matters is independence of racing candidates; the weakly correlated extension is defined as asymptotic tail independence, Pr(T_j > t | T_i > t) -> 0 as t -> infinity (A.1), and is used only to fix the protocol. On the summary, MIS is a tie after correction (0.0352 to p_adj = 0.070), so the paper reports 4/2/1 and prints the uncorrected 4/1/2 beside it (§3.5); CA's slowdown holds either way (p_adj = 5.2 × 10^-5).

Q1. Since ORDO is trained only on trajectories generated by default SCIP, why should the learned model be expected to outperform SCIP? Why not directly use SCIP, especially when Table 20 shows that replaying SCIP's sequence performs similarly to the learned model on several domains?

Response: Thanks. The claim is not a better order in-domain but what a sequence prior provides when the default schedule is unavailable: the model releases the program without first running the default presolve, and extrapolates to unseen domains (NNV 0.9308 against replay, n = 550). In-domain the paper claims no gain (§4).

Q2. How does ORDO compare with much simpler alternatives, such as one fixed sequence learned from the training set, nearest-neighbor sequence retrieval, or selecting from a small library of presolve sequences?

Response: Thanks. All three are in the paper. Replay: the ratios of W1, a tie on four of five domains. A model-free portfolio of the two most frequent corpus sequences through the same five-path protocol: 0.7745 / 0.7999 / 1.0241 / 0.9563 against our 0.7766 / 0.7955 / 1.0182 / 0.9887, every paired ratio within 1 ± 0.035 (Table 1). Nearest-neighbour retrieval in-domain: 0.9866 with a 65% win rate on SC against 0.773 with 93% (B.9). NNV is the one cell where generation separates from replay, and the only one where any control was run against it.

Q3. Can the authors clarify Proposition 1 and correct the reported transfer outcome? In particular, what assumption supports the "weakly correlated" extension, and why is MIS not counted as a failure despite (SGM > 1) and (p < 0.05)?

Response: Thanks. As in W3: the assumption is independence, the extension is asymptotic tail independence (A.1), and the transfer summary is 4/2/1 under Holm over the seven points, MIS moving from 0.0352 to p_adj = 0.070 with the uncorrected 4/1/2 printed beside it (§3.5); CA's slowdown is not a correction artefact (p_adj = 5.2 × 10^-5).

Reviewer_r2SzOpenReview ↗

Official comment by reviewer r2SZ

Thank you for the detailed response. The additional results clarify several of my concerns, particularly the comparison with simpler baselines and the correction of the MIS significance result. However, my main concern about the necessity of the learned model remains only partially resolved. The response shows that learned generation is largely comparable to replay on the in-domain tasks, and that a simple model-free sequence portfolio can achieve similar performance in several cases. The main evidence for the learned model therefore appears to come from the single unseen NNV domain. Overall, the rebuttal improves the clarity of the paper, but I still find the evidence for the learned component somewhat limited.

Reviewer_9k6nOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper recasts MIP presolve planning as generating a sequence of "atomic" presolve actions: presolver routine and constraint-handler callbacks, plus round-boundary and end tokens.

  • Execution facility. The SCIP 10.0.1 source is modified to inject a given action sequence along the native presolve path and to record which actions take effect.
  • Model. A 1.6M-parameter decoder-only transformer, conditioned on instance features, is trained by behavior cloning on 20,003 sequences recorded from SCIP's default presolve on D-MIPLIB training instances.
  • Deployment. "Sequence racing" runs 2K+1 = 5 paths in parallel: one with presolve disabled, K model sequences, and K model sequences followed by default presolve. The first to finish wins.
  • Results. Sequences are executed correctly (identical optima). On seven cross-domain points, transfer gives 4 speedups, 2 ties, and 1 loss, the best being NNV (SGM ratio 0.718 with racing). The paper compares against reimplemented configuration-space methods (L2P-MIP-, SMAC-, ISAC-, Hydra-MIP-, and Song & Gu-style setups) on SC, CA, and GISP.
Critical strengths
  1. A useful engineering artifact. The source-level facility for injecting and recording presolve action sequences is valuable for studying order effects. The correctness verification is careful: bitwise-identical default behavior on 658/658 instances and identical optimal values across replicas (Appendix B.1).
  2. Measurement discipline. Same-batch denominators, solver-internal clocks, and explicit replica protocols are taken seriously. The paper reports unfavorable cells (e.g., the failure on CA) rather than averaging them away.
  3. Informative controls. The random-content control, the presolve-disabled arm, fixed-sequence replay, and the core-time accounting (Appendix A.3) let the reader trace where the gains come from.
Critical weaknesses
  1. The paper's own controls show the learned sequences add little beyond replaying default. Fixed replay of the default sequence matches the model's single-path output on CFLP, MIS, MVC, and SC (difference ≤ 0.019; Table 20). The paper concludes that the within-domain gain comes "from the execution scheme (single-round replay vs. the default multi-round pipeline)", not from generating better sequences. With the target domain in training, the model ties default (CA 0.980, GISP 1.005; Table 1). Mechanism A (Appendix B.5) attributes much of the speedup to skipping "fruitless trial and error" such as repeated probing. Two consequences follow:

    • The central claim, that learning action order is what helps, is not supported within domains.
    • The obvious non-learned baselines are missing: default SCIP with probing disabled or limited, the fast presolve emphasis setting, and limits on presolve rounds. The paper explicitly leaves "disable only probing" out of scope (Appendix B.12), yet it is the most direct test of the mechanism the paper itself identifies.
  2. The racing comparison is not compute-matched, and the pool includes "no presolve". Racing uses five cores, at a core-time cost of 3.9–5.0× (Appendix A.3). The configuration-space baselines get "best of five replicas of the same configuration", which only exploits timing jitter (about 6% mean bonus, Appendix B.2.2). It does not exploit diversity across strategies. The fair baseline is a classical portfolio race of five different configurations, for example {default, presolve off, fast presolve, aggressive presolve, probing off}, on five cores. Without it, the gains cannot be attributed to learned sequences rather than to racing over diverse strategies. On SC, presolve-off alone (0.751) is slightly faster than racing (0.773; Table 18). The presolve-off arm wins 31.8% of NNV races and about 75% of SC races (Appendix B.5, B.7). Please also report the presolve-off ratio alone on NNV, the headline zero-shot domain.

  3. The zero-shot claims are fragile.

    • The 7 transfer points give 4 speedups, 2 ties, and 1 significant slowdown (CA, 1.112), with win rates from 28% to 95%. No family-wise correction is applied across rows (Appendix A.2).
    • The paper notes that transfer success is "not explained by corpus richness" and correlates with how much of the default presolve is redundant in a domain (Appendix B.7). Racing between "less presolve" and "no presolve" would exploit exactly that, regardless of sequence order.
    • Seed-to-seed fluctuation reaches 0.076 in aggregate ratio (Appendix A.2), comparable to several reported effects.
    • Evaluation uses only the D-MIPLIB easy validation split, where presolve takes at most a few seconds (0.007–4.34 s, Appendix B.7) and instances solve within the 300 s limit. Effects on harder instances are unknown.
  4. The modeling choice limits what can be learned. Behavior cloning of the default trajectory, whose order is "nearly unique" (N_eff ≈ 1) in most domains (Section 3.3), cannot produce sequences better than default except by chance or truncation. The "duality of order" finding rests on shuffling all tokens, including round boundaries. That destructive intervention shows some order matters, not that the model's orderings are better. On GISP, shuffling changes nothing (+0.6%, n.s.; Table 16).

  5. The theory is standard. Proposition 1 (quantiles of a minimum of K i.i.d. heavy-tailed runtimes) is textbook extreme-value reasoning. Proposition 2 (a non-contiguous schedule cannot be expressed by fixed per-plugin windows) is immediate. Neither strengthens the empirical case.

  6. The presentation falls well short of ICLR standards.

    • The main text is not self-contained: Section 3.4 states its findings with no numbers and defers to Appendix B.5. Most evidence is spread across about 45 appendix pages.
    • The writing is very hard to follow. It uses non-standard terminology ("isomorphic default", "caliber", "content single-path", "absorbability") and long hedged sentences.
    • Appendix A.4 restates the contribution list several times.
    • There is broken text ("NNV reaching 0.7181/73.6the scope of this claim").
    • Many numbers come from different batches with different denominators, and the paper warns these must not be compared, which makes cross-checking claims very difficult.

    A substantial rewrite is needed.

  7. Reproducibility. The supplement omits model weights, the training entry point, and result dumps (Reproducibility Statement). The results depend on a source-patched SCIP and on a specific cloud machine with "fair CPU allocation".

Questions for authors
  1. How does default SCIP compare with probing disabled or limited, with presolve rounds capped, or with the fast presolve emphasis, on each domain? Do these reproduce the within-domain replay gains?
  2. How does ORDO's racing compare with a five-core portfolio race over distinct presolve configurations (including presolve-off), under the same protocol?
  3. What is the presolve-off-only ratio on NNV, and what fraction of the NNV racing gain disappears if the presolve-off arm is removed from the pool?
  4. Do the zero-shot verdicts survive a family-wise correction and the three-seed recomputation, on the medium or hard D-MIPLIB splits?
  5. Has any sequence the model generated been shown to beat default because of its order, for example by comparing it with the same multiset of actions in default order?
  6. Will model weights and the training pipeline be released so that the reported ratios can be reproduced?
4 comments

We thank the reviewer. Answered in the order given; ratios are against the same-batch default under solver-internal time, so below 1 is faster.

W1. The paper's own controls show the learned sequences add little beyond replaying default.

Response: Thanks. Table 1, column 1: 1.0077 / 1.0163 / 1.0008 / 0.9979 on SC / CFLP / MIS / MVC and 0.9308 on NNV; a model-free portfolio of the most frequent corpus sequences matches us on four domains (within 1 ± 0.035), one to three distinct sequences each (B.3.1).

W2. The racing comparison is not compute-matched, and the pool includes "no presolve".

Response: Thanks. Both sides run 5 online cores under the same limit and racing is 5 paths, so no width advantage; it costs core-time 3.92× / 4.27× / 4.97× on SC / CFLP / MVC (A.3). np is a pool member reading 0.7457 on SC, faster than racing's 0.7766.

W3. The zero-shot claims are fragile.

Response: Thanks. Holm moves only MIS (0.0352 to p_adj = 0.070), giving 4/2/1 with 4/1/2 beside it (§3.5); no seed flips a verdict (A.2); with np removed racing reads 0.8221 against 0.7405 (B.7).

W4. The modeling choice limits what can be learned.

Response: Thanks. Stated as a bound: a clone cannot beat the order it clones, and most domains have a nearly unique default order (N_eff = 1, §3.3). Tested instead: injection is neutral (658/658, §3.2), cross-round shuffling inflates the tail (§3.3), and transfer works on a corpus-diverse domain (NNV 0.9308).

W5. The theory is standard.

Response: Thanks. Neither is in the contributions nor is theory: Prop 1 (A.1) fixes the min-of-K tail direction the calibration uses; Prop 2 (2.3) records why a non-contiguous schedule cannot be written with per-plugin windows (B.12.2).

W6. The presentation falls well short of ICLR standards.

Response: Thanks. Evidence moved, prose not rewritten: template class with no manual overrides; §3.4 and §3.5 state their readings in place; contributions listed once; A.2 item (8) states when two readings divide.

W7. Reproducibility. The supplement omits model weights, the training entry point, and result dumps (Reproducibility Statement). The results depend on a source-patched SCIP and on a specific cloud machine with "fair CPU allocation".

Response: Thanks. The weights of the seven models behind every number are added (6.5 MB each); facility and drivers were already there. Training code is not released: retraining takes 57 s on a single RTX 5060 Ti. Fair core allocation is a precondition (§3.1); SGM reproduces to 1.0% (A.2).

Q1. How does default SCIP compare with probing disabled or limited, with presolve rounds capped, or with the fast presolve emphasis, on each domain? Do these reproduce the within-domain replay gains?

Response: Thanks. Neither reproduces it (B.12.2): probing disabled 1.1380 on SC, 1.0049 on MIS; capping presolve at one round 1.0324 on SC, 0.9863 on MIS, 1.3873 on CA; at the sequence level probing changes nothing (800 instances, five domains, one sequence).

Q2. How does ORDO's racing compare with a five-core portfolio race over distinct presolve configurations (including presolve-off), under the same protocol?

Response: Thanks. Reported rather than raced (§4): on SC a hand-set preset reads lower than any sequence arm (np alone 0.7457 vs 0.7766) with N_eff = 1, so it selects presolve strength, not an instance-conditioned order; presets changing only plug-in parameters compose with our sequence (1.0001 / 1.0019; B.12.2).

Q3. What is the presolve-off-only ratio on NNV, and what fraction of the NNV racing gain disappears if the presolve-off arm is removed from the pool?

Response: Thanks. Alone it reads 1.0706, slower than the default; removing it from the pool moves racing from 0.7405 to 0.8221 on the same 549 keys, leaving 68.5% of the gain (B.7).

Q4. Do the zero-shot verdicts survive a family-wise correction and the three-seed recomputation, on the medium or hard D-MIPLIB splits?

Response: Thanks. Medium over four decoding seeds: SC 0.9847-0.9894 and CFLP 0.9463-0.9603 keep their sign, CA indistinguishable (1.0144-1.0192, 42-47 wins), MVC, MIS and GISP unreadable (cap hit on 66, 88, 96 of 100); hard is not run on grounds of time cost.

Q5. Has any sequence the model generated been shown to beat default because of its order, for example by comparing it with the same multiset of actions in default order?

Response: Thanks. None, and the paper claims none: injection is neutral (658/658, §3.2); cross-round shuffling inflates the tail, intra-round does not (§3.3); generation is 6.9% faster than replay where the corpus is diverse (0.9308, n = 550).

Q6. Will model weights and the training pipeline be released so that the reported ratios can be reproduced?

Response: Thanks. Weights are released: seven checkpoints, 6.5 MB each. The training pipeline is not: the supplement carries facility, collection code and drivers, and the 1.6M-parameter model retrains in 57 s on a single RTX 5060 Ti.

Reviewer_9k6nOpenReview ↗

Response to Reply

Thank you for the response. The response makes two concessions that bear directly on the paper's claims. Within domains, the learned model gives no gain over replaying the default sequence. And no generated sequence has been shown to beat the default because of its order. My remaining questions, therefore, focus on NNV, the only domain in which generation separates from replay, and on the experiment that was declined.

  1. NNV is now the only evidence that the learned model matters (0.9308 against replay, n = 550), but the controls run on other domains were not run there. Could you report the following arms on NNV in a single batch with the same denominator?

    • default;
    • replay of the default sequence;
    • generation as a single path;
    • the generated multiset executed in default order;
    • the default sequence truncated to the generated length;
    • the model-free static portfolio through the five-path protocol;
    • ORDO racing with and without the presolve-off arm.

    This would show whether the 6.9% over replay comes from order, from dropping actions, or from instance conditioning. Please also say which batch the headline should use, since the NNV race reads 0.7181 in the paper and 0.7405 in this response.

  2. The five-core race over distinct presolve configurations was not run, in reply to me or to Reviewer 6PQD. The argument that such a race "selects presolve strength, not an instance-conditioned order" describes what it would measure, but it is still the baseline the racing gain has to beat.

    • Default SCIP runs single-threaded, so giving both sides five online cores does not mean they use comparable compute. The core-time cost of racing is 3.92× to 4.97×.
    • On SC, presolve-off alone on one core (0.7457) is already faster than the five-core race (0.7766).
    • Please run a race over {default, presolve off, fast, aggressive, probing off} under the same protocol on SC, CFLP, MVC and NNV, and report the fast preset as a single path on each domain, which my first question also asked for.
  3. The response says the selection rule costs 5.2 points on SC (0.7251 to 0.7766). What does 0.7251 correspond to, what exactly is the selection rule, and where does the loss come from (contention among the concurrent paths, or detecting the winner)? Does the same loss apply to the best-of-five replicas used for the configuration baselines?

  4. On CA, every arm other than the default multi-round pipeline is slower than default (replay 1.1730, truncation 1.4282, fixed sequence 1.4348, presolve off 1.5830), and the transfer result is 1.1121. The racing pool has no pure default arm. Why not make the default pipeline one of the five paths? That would limit the loss on domains like CA to the racing overhead, and it would show what the generated paths add when the default is also in the race.

  5. In the reply to Reviewer r2Sz, nearest-neighbour retrieval as a single path (0.9866 on SC) is compared with ORDO's five-path race (0.773). Against generation as a single path on SC (0.9904), retrieval is at least as good. Please compare retrieval and generation under the same execution, run retrieval through the racing protocol, and include NNV, retrieving from training-domain sequences as the static portfolio does.

  6. On the medium split, the effects are much smaller than on easy (SC 0.9847 to 0.9894, CFLP 0.9463 to 0.9603).

    • Are these single-path or racing numbers, and are they significant after correction?
    • For MVC, MIS, and GISP, 66, 88, and 96 of 100 runs hit the time limit, so these domains are not informative at this scale. Could you rerun them with a longer limit, at least on a subset, and add NNV if a medium split exists?
    • The reply to Reviewer 6PQD declines a MIPLIB 2017 subset because a pooled ratio "would be dominated by the largest solves". The shifted geometric mean used in the paper is meant to limit that effect, and per-instance win rates do not need domain labels. Since hard instances were not run, a fixed MIPLIB subset that solves within the limit would be a direct test of the zero-shot claim.
  7. Given the two concessions above, the title ("Dynamic Ordering"), the TL;DR ("MIP presolve is a sequence generation problem, not a hyperparameter tuning problem"), and the contribution list no longer match what the experiments show. The paper currently supports four things:

    • a facility for injecting and recording presolve sequences;
    • a single-round execution form that helps on some domains;
    • racing that includes a presolve-off arm;
    • a learned model whose benefit is so far shown on one domain.
  8. Since the model retrains in 57 s, releasing the training script would incur little cost and allow others to check the released checkpoints against the corpus. Will it be included in the supplement?

We thank the reviewer. Every arm below is one batch, one denominator — 550 NNV instances, solver-internal clock, 300 s cap, default re-measured there. The two statements you quote are the \u00a74 boundaries of the claim as scoped.

Q1 (arms). Same batch, wins/550: replay 0.8712, 287; generation single path 0.8104, 320; generated multiset in default order 0.8199, 316; racing 2K+1 0.7401, 374.

Paired, same keys: generation ÷ replay 0.9308 [0.891, 0.971]; multiset-in-default-order ÷ replay 0.9416; multiset-in-default-order ÷ generation 1.0116. Against your accounts: order alone does not explain the gap — the generated multiset still gains 5.8% over replay in the default order; the residual 5.8% is the instance-conditioned action set the model emits, which replay cannot carry. The order increment is small but separated from one: the sequence matters in both components.

On the four domains the portfolio is matched in information — its candidates come from the same corpus the model trains on — so those readings isolate conditioning. At NNV it would draw on NNV's own sequence corpus, which the model never sees, and measure that gap instead; without target-domain data only replay (0.8712) and retrieval (0.8786) remain, both measured, against generation 0.8104. The truncated arm cuts the default sequence at the generated length, so its cut point is the model's output; where run (CA) it is worse than replay (1.4282 vs 1.1730).

Q1 (headline). 0.7181 is a two-batch value: 360 instances under 300 s read 0.7664, the 190 censored re-timed at 600 s read 0.6346 (§3.5). Single batch is 0.7401 over 550, 0.7405 over the 549 with a presolve-off reading; the earlier reply's 0.7405 should have carried that label. The headline stays 0.7181; the table is single-batch, where the decomposition arms exist.

Q2 (configuration race). Declining it is a distinction of objects, not cost. Presolve configuration and sequence order are orthogonal decisions, across the 24 (domain, preset) cells no preset changes the order of the action multiset, and where a preset touches only plug-in parameters the 2x2 factorial is multiplicative (interaction 1.0001 SC, 1.0019 CFLP), our sequence keeping its gain (0.9369 vs 0.9398). The presets that read lower than our race on SC change the round budget (maxrounds, restarts); a round is the axis on which a schedule is defined, so racing them measures a different object, not a competing one; the interface takes presolve settings as inputs, so the two are combined, not traded. The np path already sits in our pool: on NNV presolve-off alone is 7% slower than default (1.0706) while the pool containing it is 26% faster (0.7401), so the gain there is not strength. On cores, racing spends 5× core time (3.92–4.97×, A.3) to truncate the tail; no per-core efficiency is claimed.

Q3 (selection). 0.7251 is the per-instance minimum over the pool, an oracle bound; 0.7766 is the deployed rule, first process to finish. Both come from the same runs' internal clocks, so the 5.2 points are the price of online selection, not contention; the same criterion applies to the best-of-five replicas behind the config baselines — symmetric.

Q4 (default in pool). A multi-round default cannot run under the single-round contract we measure, and the denominator in the numerator would be circular; the K fallback paths cover that role, and the np path supplies the tail protection on NNV — removing it costs 11% (0.8215 vs 0.7401).

Q5 (retrieval). Same execution, NNV: 0.8786 vs generation 0.8104, paired 1.0834 [1.0391, 1.1297]. Retrieval through the five-path protocol was not run.

Q6 (medium, MIPLIB). Racing: SC 0.9821 (64/100, p ≈ 0.007), CFLP 0.9459 (60/100, p ≈ 0.057), CA 1.5248 (2/100), the three censored domains 0.999–1.000 (14–27/100). Only SC survives Holm; CFLP is directional. The censoring is in our own table and we read them as uninformative; NNV has no medium split. On MIPLIB: the setting does not fit this paper. What we report is a per-domain ratio, and MIPLIB 2017 assigns no domain to an instance, so a subset returns a fleet-wide statistic rather than that quantity.

Q7 (framing). Your list is ours. The title names the object the method produces, an ordering over atomic actions, and does not assert that order alone carries the gain; the TL;DR should read that presolve decisions can be expressed, executed and measured as sequences, and that on a fully held-out domain this gives a reproducible gain over replay, carried by the instance-conditioned action set, with a small order increment.

Q8 (code). Not in this cycle: the release carries the solver patch, the evaluation driver and the seven checkpoints, which reproduce every reported number; training code follows on acceptance.

Reviewer_9k6nOpenReview ↗

Response

Thank you for running the NNV arms. They answer my main question. On NNV, generation beats replay (0.9308), retrieval from the training domains (1.0834 in favor of generation), and the generated multiset executed in default order. With the oracle held fixed, the model's pool is 6.5% better than the retrieval pool. I accept that the learned model contributes to NNV, and that its gain there comes mainly from the action set (about 5.8%) rather than the order (about 1%).

One correction to my earlier point 2: the default denominator is itself the first finisher among replicas, so both sides do use five cores. My concern is only that the replicas share one configuration.

Remaining points:

  1. The 5.8% attributed to the instance-conditioned action set could also come from executing fewer actions. A truncation arm whose length does not come from the model, for example, the default sequence cut to the median generated length on NNV and applied to every instance, would separate "fewer actions" from "the right actions". Please also give an interval for the order increment (1.0116). The response says it is separated from one, but no interval is shown.

  2. The headline 0.7181 combines 360 instances run under a 300 s cap with 190 instances re-timed at 600 s after they timed out. The single-batch value is 0.7401. The re-timed subset is selected by its outcome and has the strongest ratio (0.6346), so mixing the two caps in one SGM makes the headline depend on which instances were re-run. I would report 0.7401, or a full batch at 600 s, as the headline, and the two-batch value as a secondary analysis.

  3. On the medium split, the racing gain largely disappears. SC reads 0.9821 and is the only result that survives Holm; CFLP reads 0.9459, and CA reads 1.5248 with 2 wins in 100. In the earlier reply, the medium CA ranged from 1.0144 to 1.0192, with 42 to 47 wins over four decoding seeds. Which protocol produced each number, and why does CA move from about 1.02 to 1.52? If the earlier numbers were single paths, the race is about 50% slower than its own candidates on CA, which would point to a cost of running five paths concurrently on larger instances.

  4. The race over standard presolve settings and a pool that includes the default path were both declined. I understand the argument that configuration and order are different objects, but the paper's practical result is a racing speedup of about five times the core time. Without a race over existing settings under the same protocol, readers cannot tell whether ORDO's racing performs better than the same five cores with standard settings. On the default path, the fallback paths already run the default multi-round presolve after the sequence, so a path that runs the default alone seems compatible with the pool. Without it, the race is 11% slower than default on easy CA and 52% slower on medium CA.

In view of the NNV results, I am raising my score from 1 to 2. I am not going higher because the learned component is shown to help on one domain, the gains largely vanish on medium instances and turn negative on CA, and the comparison that would establish the practical value of racing was not run.

Reviewer_6PQDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ORDO, a framework that formulates MIP presolve planning as conditional autoregressive generation of operation-level action sequences. The authors modify SCIP to record and execute presolve action sequences and train a lightweight Transformer through behavior cloning of SCIP's default presolve trajectories. At deployment, two generated sequences are combined with different execution modes to construct a five-path racing portfolio.

Experiments on several D-MIPLIB domains evaluate solving-time performance, zero-shot cross-domain transfer, order sensitivity, sequence replay, and the contribution of racing.

Critical strengths

S1. The paper investigates a finer-grained decision space for MIP presolve by directly modeling operation-level execution sequences rather than relying exclusively on solver configuration parameters.

S2. The authors implement a sequence execution interface in SCIP and provide extensive empirical evaluations, including cross-domain transfer, random-sequence controls, direct replay, and racing ablations.

Critical weaknesses

W1. Insufficient evidence for the independent contribution of Transformer-based sequence generation.

The Transformer is trained to imitate SCIP's default presolve trajectories rather than directly optimize solving performance. Although the paper includes random-sequence and direct-replay controls, these experiments do not establish whether instance-conditioned generation significantly outperforms simpler approaches to constructing plausible presolve sequences. In particular, the direct-replay results in Appendix B.6 show comparable performance between learned and replayed default sequences on several domains, leaving the independent benefit of the Transformer insufficiently isolated.

W2. Limited evidence for robust and broad zero-shot generalization.

Although zero-shot cross-domain transfer is a central contribution, only four of the seven evaluated unseen domains achieve statistically significant speedups, while the remaining three show no significant improvement or statistically significant slowdowns under the stated criterion. Moreover, the experiments are restricted to a small number of predefined MILP distributions from the easy validation split of D-MIPLIB. This limits the evidence for generalization to more heterogeneous MILP instances beyond those distributions.

Presentation issues.

Several important ablation results are primarily presented in the appendix, and the discussion of related work is somewhat fragmented. Additionally, the line spacing in the main text appears to deviate from the official ICLR formatting template. These presentation issues should be addressed in a revision.

Questions for authors

Q1. Contribution of instance-conditioned sequence generation.

Could the authors construct a static sequence portfolio by selecting two representative presolve sequences from the existing training corpus (e.g., the two most frequently occurring sequences within each training domain), without using the Transformer, and evaluate it using the same five-path racing protocol? Specifically, both methods should use one no-presolve path, two sequence-only paths, and two sequence-plus-default-presolve paths, with identical test instances and parallel budgets. This would directly assess whether instance-conditioned sequence generation offers a meaningful advantage over a simpler static portfolio.

Q2. Comparison with configuration-based racing.

Could the authors additionally compare ORDO against a diversified portfolio of conventional SCIP presolve configurations under the same parallel budget? This would help determine whether directly generating operation-level sequences provides benefits beyond combining several meaningful presolve configurations.

Q3. Generalization to heterogeneous MILP instances.

Could the authors evaluate the frozen ORDO model on a representative subset of standard MIPLIB 2017 instances, rather than necessarily the entire benchmark? The subset should cover diverse problem structures and solving difficulties, with a clearly specified selection protocol and no additional target-instance training or tuning. Such an evaluation would provide stronger evidence that zero-shot transfer extends beyond the predefined D-MIPLIB domains.

Q4. Explanation of heterogeneous cross-domain results.

Could the authors further analyze why ORDO achieves substantial speedups on some unseen domains (e.g., NNV, SC, and CFLP) but provides little benefit or causes slowdowns on others (e.g., MVC, MIS, and CA)? Are there identifiable instance characteristics that could explain these differences?

Q5. Consistency of the cross-domain statistical summary.

Section 3.5 summarizes the seven cross-domain evaluations as four accelerated, two tied, and one failed. However, Table 1 reports an SGM ratio of 1.0140 and a two-sided sign-test p-value of 0.0352 for MIS, which appears to indicate a significant slowdown under the stated criterion. Could the authors clarify this classification and reconcile it with the overall summary?

4 comments

We thank the reviewer. Answered in the order given; ratios are against the same-batch default under solver-internal time, so below 1 is faster.

W1. Insufficient evidence for the independent contribution of Transformer-based sequence generation.

Response: Thanks. Both arms single-path, same batch: generation against replay of that instance's default sequence is a tie on SC / CFLP / MIS / MVC (1.0077 / 1.0163 / 1.0008 / 0.9979) and separates only on NNV, 0.9308 [0.8909, 0.9712] (Table 1)..

W2. Limited evidence for robust and broad zero-shot generalization.

Response: Thanks. Seven transfer points on the easy split, not heterogeneous MILPs; after Holm 4 accelerated / 2 tied / 1 slower, CA the slower one (§3.5). Medium added over four decoding seeds; hard not run on grounds of time cost.

W3. Presentation issues. Several important ablation results are primarily presented in the appendix, and the discussion of related work is somewhat fragmented. Additionally, the line spacing in the main text appears to deviate from the official ICLR formatting template. These presentation issues should be addressed in a revision.

Response: Thanks. The revision uses the template class with no manual leading or column-stretch; controls are in Table 1 (§3.4), the CA ladder in §3.5, and related work is one section (§3.6).

Q1. Contribution of instance-conditioned sequence generation. Could the authors construct a static sequence portfolio by selecting two representative presolve sequences from the existing training corpus (e.g., the two most frequently occurring sequences within each training domain), without using the Transformer, and evaluate it using the same five-path racing protocol? Specifically, both methods should use one no-presolve path, two sequence-only paths, and two sequence-plus-default-presolve paths, with identical test instances and parallel budgets. This would directly assess whether instance-conditioned sequence generation offers a meaningful advantage over a simpler static portfolio.

Response: Thanks. Run as specified: SC 0.7745 vs 0.7766, CFLP 0.7999 vs 0.7955, MIS 1.0241 vs 1.0182, MVC 0.9563 vs 0.9887, all within 1 ± 0.035 (Table 1); those domains hold one to three distinct sequences, so a static portfolio spans the set (B.3.1).

Q2. Comparison with configuration-based racing. Could the authors additionally compare ORDO against a diversified portfolio of conventional SCIP presolve configurations under the same parallel budget? This would help determine whether directly generating operation-level sequences provides benefits beyond combining several meaningful presolve configurations.

Response: Thanks. §4 reports it rather than racing it: on SC a hand-set preset reads lower than any sequence arm (np alone 0.7457 vs our 0.7766) at N_eff = 1, i.e. presolve strength, not order; a preset changing only plug-in parameters composes with our sequence (1.0001 / 1.0019; B.12.2).

Q3. Generalization to heterogeneous MILP instances. Could the authors evaluate the frozen ORDO model on a representative subset of standard MIPLIB 2017 instances, rather than necessarily the entire benchmark? The subset should cover diverse problem structures and solving difficulties, with a clearly specified selection protocol and no additional target-instance training or tuning. Such an evaluation would provide stronger evidence that zero-shot transfer extends beyond the predefined D-MIPLIB domains.

Response: Thanks. Not added: MIPLIB 2017 assigns no domain to an instance, so the per-domain quantities the claim rests on are undefined and one pooled ratio would be dominated by the largest solves (§4).

Q4. Explanation of heterogeneous cross-domain results. Could the authors further analyze why ORDO achieves substantial speedups on some unseen domains (e.g., NNV, SC, and CFLP) but provides little benefit or causes slowdowns on others (e.g., MVC, MIS, and CA)? Are there identifiable instance characteristics that could explain these differences?

Response: Thanks. CA is the informative failure: replay of that instance's recorded multiset 1.1730 against the model's 1.1761, indistinguishable; truncating 1.4282, a fixed sequence 1.4348, no presolve 1.5830, so the loss is the execution form (§3.5).

Q5. Consistency of the cross-domain statistical summary. Section 3.5 summarizes the seven cross-domain evaluations as four accelerated, two tied, and one failed. However, Table 1 reports an SGM ratio of 1.0140 and a two-sided sign-test p-value of 0.0352 for MIS, which appears to indicate a significant slowdown under the stated criterion. Could the authors clarify this classification and reconcile it with the overall summary?

Response: Thanks. The seven p values recomputed from the win counts reproduce Table 2, and Holm moves MIS from 0.0352 to p_adj = 0.070, a tie, giving 4/2/1 with the uncorrected 4/1/2 beside it (§3.5); CA survives either way (p_adj = 5.2 × 10^-5).

Thank you for the additional experiments and clarifications. However, my main concern regarding the independent contribution of Transformer-based sequence generation remains. The new static portfolio experiment shows that selecting representative sequences without a Transformer achieves performance comparable to ORDO on four domains. Similarly, the direct-replay results show no clear advantage of learned sequences on these domains.

Although NNV provides positive evidence for cross-domain transfer, the independent benefit of the Transformer is demonstrated on only one of the seven transfer domains. This is insufficient to establish robust and generalizable advantages, particularly given the limited or negative results on other domains. Moreover, NNV lacks a corresponding static portfolio comparison, leaving it unclear whether its improvement genuinely requires Transformer-based generation.

Overall, the results suggest that much of the observed improvement comes from the sequence execution mechanism and racing rather than instance-conditioned generation. I therefore remain unconvinced that the necessity and generalizable value of the Transformer have been sufficiently established.

We thank the reviewer. The NNV control is answerable from the measurements we have, and it separates instance conditioning from both the execution form and the selection rule.

Same execution, same candidate role. All three arms below are five replicas of one path, first finisher, on the same 550 instances, solver-internal clock, 300 s cap. Generation reads 0.8104, nearest-neighbour retrieval from the six training domains 0.8786 (paired 1.0834 [1.0391, 1.1297]), replay of the instance's own default trajectory 0.8712. Under one execution form the model is faster than every model-free supply we can construct on this domain.

Same selection advantage. Give two pools the same three roles (np, one content candidate, one fallback) and the same oracle: {np, generation, fallback} reads 0.6653, {np, retrieval, fallback} 0.7090, paired 1.0651 [1.0407, 1.0901]. The two pools agree on 247 of 549 instances; of the 302 that differ, the model pool is better on 192 (0.6196 against 0.6958 there). With the selection rule held fixed the candidates the model emits are 6.5% better, and content is the only thing that differs between the pools.

What a raced corpus portfolio would test on NNV. On the four domains its candidates come from the same corpus the model trains on, so those cells isolate conditioning and come out even. NNV supplies neither side of that: the corpus covers eight domains and NNV is not one of them (Table 6), and its instances are 96% continuous against an all-binary training corpus, so no corpus sequence is drawn from NNV's distribution. A portfolio there would race sequences borrowed from other distributions — which is what the retrieval arm already does, and it reads 0.8786 against the model's 0.8104.

Why the four domains come out even, and why that is not evidence against generation. SC, CFLP, GISP and MIS hold one to three distinct corpus sequences at Neff ≈ 1 (§3.3): any representative portfolio is the default trajectory, and replaying the default trajectory is exactly what the model reproduces there (§4). Those cells cannot separate a learned selector from a corpus lookup, and the abstract claims nothing there. Where conditioning can express itself — the held-out domain — the model is faster than every model-free supply, under the same execution and with selection held fixed.

Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts to clarify the scope of their contributions. However, these clarifications have reinforced my concerns about the paper's overall significance.

The authors now explicitly acknowledge that the independent benefit of Transformer-based generation is demonstrated on only one of the seven transfer domains, NNV, describing it as a "narrow and reproducible extrapolation result." While I acknowledge this positive result, it provides limited empirical support for Contribution 1, "Zero-shot generalization from sequence generation." The remaining domains largely demonstrate benefits from the execution mechanism and racing rather than from learned sequence generation. Explaining why the learning component provides little additional benefit does not substitute for evidence of its broader effectiveness.

This also raises concerns about the remaining contributions. Although the operation-level execution interface and sequence racing may be practically useful, they are largely independent of the learning model. In particular, racing applies the relatively straightforward strategy of running multiple candidates concurrently and selecting the fastest. I am not convinced that these mechanisms alone provide sufficient methodological novelty or significance to compensate for the limited independent contribution of the Transformer.

Overall, while I appreciate the authors' clarifications and additional evidence, I remain unconvinced that the paper establishes a sufficiently significant and well-validated machine learning contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.