- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents SchedulerFM, a neural scheduling framework combining reward-free offline pretraining with test-time latent strategy search. Forward–backward representations learn a family of scheduling policies, and CMA-ES searches their latent space using complete schedule evaluations while keeping the model frozen. Experiments evaluate transfer across JSSP benchmarks, larger instances, flexible JSSP, and alternative scheduling objectives.
Critical strengths
S1. Separating scheduling representation learning from downstream optimization offers a useful approach to reusing learned policies across different scheduling requirements.
S2. The evaluation covers several dimensions of generalization using a single frozen checkpoint, providing encouraging evidence of transferable scheduling behavior.
S3. The optimizer comparisons and rollout-budget experiments demonstrate that adaptive latent search can improve solution quality without updating the pretrained model.
Critical weaknesses
W1. The current ablations do not isolate the contribution of reward-free FB pretraining from those of the architecture, training data, and test-time search.
W2. Makespan-based demonstration filtering and checkpoint selection leave the extent of the framework’s independence from a specific downstream objective unclear.
W3. The FJSSP experiments do not disentangle transferred scheduling knowledge from the contributions of the added machine-assignment rules and their joint optimization.
W4. The normalized successor-measure definition in Equation (1) is inconsistent with the discount factor γ = 1 in Table 5, leaving its connection to the implemented training objective unclear.
Questions for authors
Q1. Could you compare FB pretraining against reward-based RL and imitation learning under matched architecture, training data, and test-time search budgets? The existing replay-diversity and optimizer comparisons do not establish whether reward-free representation learning is responsible for the observed transfer benefits.
Q2. Appendices A.1–A.2 indicate that CP-SAT demonstrations are filtered by makespan improvement and checkpoints are selected using validation makespan; how sensitive are the transfer results to these choices? Please clarify the intended scope of “reward-free” and provide evidence from removing the CP-SAT demonstrations or using a checkpoint-selection criterion independent of makespan.
Q3. Appendix A.5 introduces five machine-assignment coefficients and initializes the joint search using eight handcrafted assignment rules; how much of the FJSSP performance comes from this additional machinery? Comparisons that fix the latent strategy while optimizing assignment coefficients, or combine standard dispatching rules with the same assignment search, would help isolate the contribution of the pretrained model.
Q4. For the finite scheduling episodes considered here, substituting γ = 1 into Equation (1) makes the successor measure zero because of the factor (1 − γ), whereas the training objective models nonzero future occupancy; is the implementation instead based on an unnormalized finite-horizon measure? Please reconcile Equation (1), the value relation in Equation (2), and the training targets, and clarify whether this discrepancy is purely notational.
4 comments
Response to Reviewer 4FeM [2/2]
W3 / Q3. FJSSP decomposition of scheduling and machine assignment
Appendix A.5 introduces five machine-assignment coefficients and initializes the joint search using eight handcrafted assignment rules; how much of the FJSSP performance comes from this additional machinery? Comparisons that fix the latent strategy while optimizing assignment coefficients, or combine standard dispatching rules with the same assignment search, would help isolate the contribution of the pretrained model.
We disentangle the contributions of pretrained scheduling strategies from machine-assignment search using both controls suggested by the reviewer: (1) fixing while searching machine assignment weights, and (2) replacing the network with handcrafted dispatching rules. We also evaluate the reverse control, searching with fixed machine assignment.
Results cover 210 FJSSP instances. All searched variants receive 512 complete-schedule evaluations. The machine-assignment search optimizes five weights initialized from eight handcrafted rules.
| Scheduling | Machine assignment | Gap (%) |
|---|---|---|
| Pretrained policy, searched (ours) | 5 weights searched jointly with | 4.28 |
| Pretrained policy, searched | Fixed earliest-completion rule | 4.52 |
| No network: searched mixture of 15 dispatching rules | 5 weights searched jointly | 4.55 |
| Pretrained policy, fixed at shared makespan anchor | 5 weights searched | 7.95 |
| No network: best of 15 fixed dispatching rules | Best of 8 fixed assignment rules | 8.60 |
Assignment search alone does not recover the full performance, and searching the pretrained scheduling strategy is indeed important:
-
With assignment weights optimized in both settings, searching reduces the gap from 7.95% to 4.28% relative to the fixed makespan anchor.
-
Joint assignment search provides a further 0.24-point improvement (4.52% → 4.28%).
-
A searched mixture of handcrafted dispatching rules is also competitive (4.55%), but SchedulerFM achieves the lowest gap (4.28%).
Summary: These controls show that FJSSP transfer benefits from searchable JSSP-pretrained scheduling strategies, beyond machine-assignment search alone, without FJSSP-specific network training.
W4 / Q4. Successor measure at
For the finite scheduling episodes considered here, substituting γ = 1 into Equation (1) makes the successor measure zero because of the factor (1 − γ), whereas the training objective models nonzero future occupancy; is the implementation instead based on an unnormalized finite-horizon measure? Please reconcile Equation (1), the value relation in Equation (2), and the training targets.
Thank you for pointing out this inconsistency. You are correct that, as written, the factor in Equation (1) makes the measure identically zero at our experimental setting . The intended object is the unnormalized episodic successor measure, consistent with the cumulative-return representation and training objectives already specified in the paper.
To make the definition consistent with the training objective, we correct Equation (1) to
where denotes the number of remaining dispatch decisions, including the final dispatch. This accommodates different training-instance sizes: the remaining horizon is determined by the unscheduled operations already encoded in the state, and each action reduces it by one. No additional state variable or architectural change is required.
At , this measure has finite total mass, and its integration with a reward gives the usual undiscounted cumulative return:
Thus, Equation (2) remains unchanged and supports the intended nonzero value representation without an additional scaling factor.
Crucially, the existing training equations already follow this unnormalized, episodic convention:
- Equation (11) includes immediate occupancy without a multiplier.
- Equation (15) masks the bootstrapped continuation at termination
- Equation (18) uses an unscaled immediate reward with undiscounted continuation. The final dispatch’s immediate outcome remains included through the immediate-occupancy term.
The required revision is therefore limited to correcting Equation (1) and briefly explaining its finite, state-dependent horizon. It does not require changing the stated training objectives, FB factorization, or reward embedding. We appreciate the reviewer for catching this issue, and we have also made the definition consistent with the training formulation in the updated manuscript.
Revision: We have updated Equation (1) and its accompanying explanation in Lines 176–189
Thank you for the detailed responses and additional experiments. The controlled FB–BC comparison, demonstration and checkpoint analyses, and FJSSP decomposition substantially address my concerns about the contribution and reuse of the pretrained representation. The corrected episodic successor-measure definition also resolves my concern regarding γ = 1. While the gains over strong heuristic search remain modest and the computational trade-offs warrant careful qualification, the new evidence strengthens the paper sufficiently for me to raise my recommendation.
Thank You for Your Constructive Feedback
Dear Reviewer 4FeM,
Thank you for your thoughtful feedback and for raising your recommendation for our paper. We are excited that our additional experiments and clarifications substantially address your concerns regarding the contribution and reuse of the pretrained representation.
We sincerely appreciate the time and effort you have devoted to reviewing our work. Your constructive feedback has been invaluable in strengthening the paper.
Thank you again.
Response to Reviewer 4FeM [1/2]
We thank the reviewer for the constructive feedback. We provide the point-by-point response as follows.
W1 / Q1. Contribution of reward-free FB pretraining
We compare FB pretraining with imitation learning and reward-based RL using the same graph backbone, logged transitions, and 512 complete-schedule evaluations per instance (cf. global response [G1]).
The closest control replaces FB with behavior cloning (BC) while retaining latent CMA-ES: FB achieves a JSSP gap of 8.01% vs 8.64% under BC for the submitted seed and 8.04 ± 0.24% vs 8.64 ± 0.11% under BC across four and three training seeds, respectively. Because this comparison fixes the backbone, data, and latent-search procedure, it directly tests the contribution of the pretraining objective.
The reward-based TD alternatives achieve 11.11% with four scheduling-cost rewards and reward-weight CMA-ES, and 17.28% with the L2D reward and rollout sampling. These provide additional comparisons under the same evaluation budget, although their adaptation spaces differ, as detailed in [G1].
Moreover, the advantage also extends to unseen objectives. we compare FB and BC on 58 instances, reporting gap (%) to one-hour CP-SAT with one evaluation seed (%, lower is better):
FB achieves a lower gap on every evaluated objective, with particularly clear improvements in weighted tardiness (38.27% vs 43.79%) and maximum tardiness (18.13% vs 22.27%). Unlike comparison with COMPASS alone, this FB-BC comparison controls the architecture, training data, and latent-search procedure. In summary, with the backbone, data, and search held fixed, FB learns a strategy family that achieves better scheduling performance and transfers more effectively across the evaluated objectives.
W2 / Q2. Makespan-filtered demonstrations, checkpoints, and “reward-free”
“Reward-free” describes the FB pretraining loss, which uses no downstream reward labels. It does not imply that demonstration or checkpoint selection excludes scheduling-quality information. We clarify this scope and test both makespan-dependent choices separately.
1. Removing makespan-filtered demonstrations. We retrain FB without CP-SAT demonstrations, retaining the architecture and 512-evaluation search budget. JSSP gaps are relative to best-known solutions on 170 instances (four/three training seeds, respectively); cross-objective gaps are relative to one-hour CP-SAT on 58 instances (one seed).
Removing demonstrations preserves JSSP performance and improves three of four non-makespan objectives, although maximum tardiness worsens. Thus, the observed transfer does not require makespan-filtered demonstrations.
2. Removing makespan-based checkpoint selection. The submitted model uses the 12k checkpoint selected by validation makespan (cf. Appendix A.2). To assess sensitivity to this choice, we evaluate all saved checkpoints through the final 16k training step, without requiring checkpoint selection:
Simply using the final checkpoint, without validation-based selection, achieves 7.95%, versus 8.01% for the submitted checkpoint. Every checkpoint from 4k onward lies within 7.81–8.28%, showing that JSSP performance is insensitive to the specific checkpoint selected.
In summary, transfer persists without makespan-filtered demonstrations, and strong JSSP performance persists without makespan-based checkpoint selection, supporting the robustness of the pretrained strategy family.