- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SWAGES for cost-aware dynamic workflow scheduling. It combines a system-wide graph attention network (SWAN), which jointly considers active workflows, ready tasks, and available VMs, with a gradient-guided evolution strategy (GUIDES) that uses critic-derived directions for policy search. The authors evaluate the method on workflow scheduling benchmarks and several continuous-control tasks. The reported results show improvements over existing scheduling and evolutionary learning methods.
Critical strengths
The overall idea is reasonable. Considering all active workflows and task–VM assignments under a shared global context is useful for dynamic scheduling. The use of critic information to guide evolutionary search is also interesting, as it makes use of rollout data without directly relying on policy-gradient updates. The experimental results are encouraging, particularly the improvements over existing learning-based scheduling methods and the tests on continuous-control tasks.
Critical weaknesses
-
My main concern is whether the experiments sufficiently support the claimed contributions. Although SWAGES achieves good results, the improvements brought by SWAN and GUIDES are not clearly separated. For example, a comparison between SWAN trained with standard ES and SWAN trained with GUIDES would help establish the contribution of the proposed search strategy. I also noticed that Table 1 and Table 2 report substantially different total costs under seemingly identical settings (70.44 vs. 14.51 for S/1). This difference needs clarification. More generally, while the experiments cover several workflow settings and four continuous-control tasks, I am not fully convinced that they establish the broader applicability and computational efficiency of the method.
-
The novelty and implementation of GUIDES could also be better clarified. Guided evolutionary search using surrogate gradients has been studied before, and the main difference here seems to be how the guidance directions are obtained from replayed transitions and learned critics. However, the relationship between the surrogate objective in Eq. (16) and the centered critic weights in Eq. (63) is not clear to me. This is important for understanding and reproducing the proposed algorithm.
Questions for authors
-
Could the authors explain the difference in total scheduling costs between Table 1 and Table 2 (also Tables 11–12)? Are different evaluation settings or cost calculations used?
-
Could the authors provide a more direct comparison to separate the contributions of SWAN and GUIDES, for example, by training SWAN with standard ES? It would also be helpful to clarify the additional computational cost of GUIDES and whether its advantage remains when training time or computational budget is taken into account.
-
Could the authors clarify how the centered critic weights in Eq. (63) are incorporated into the surrogate objective in Eq. (16)? How is the corresponding guidance gradient computed for the continuous-action tasks?
5 comments
(3) Q2: What Are the Computational Costs?
Inference latency. Table R7 shows that SWAGES requires only 6.68–8.19 ms per scheduling decision, increasing by just 1.51 ms from S to L scale. Although GATES and SPN-CWS have lower inference latency, SWAGES remains faster than DRL-G across all scales.
Table R7. Average inference time per scheduling decision (ms; lower is better).
| Scale | SWAGES | GATES | SPN-CWS | DRL-G |
|---|---|---|---|---|
| S | 6.68 | 6.25 | 1.53 | 7.83 |
| M | 7.26 | 6.89 | 1.70 | 15.03 |
| L | 8.19 | 5.63 | 1.90 | 30.59 |
This latency is sufficiently low for task-level online scheduling at the evaluated scales, where assignments are made at scheduling decisions rather than continuously during task execution. At the measured L-scale latency, 100 assignments would require only approximately 0.82 seconds of policy inference.
Training efficiency. Although GUIDES incurs additional critic-training computations, its search guidance reduces the wall-clock training time needed to reach a target scheduling cost.
Table R8. Wall-clock training time (hours, mean ± std) to reach target scheduling costs; lower is better.
| Method | ≤ 250 | ≤ 200 | ≤ 180 | ≤ 150 |
|---|---|---|---|---|
| SWAGES | 2.94 ± 0.80 | 3.24 ± 0.89 | 3.24 ± 0.89 | 4.48 ± 1.25 |
| GATES | 3.05 ± 0.94 | 5.07 ± 1.45 | 6.34 ± 1.72 | 10.10 ± 2.41 |
| SPN-CWS | 6.74 ± 1.79 | 9.10 ± 2.24 | 10.84 ± 2.54 | 14.87 ± 3.16 |
| DRL-G | 3.69 ± 0.52 | 20.18 ± 2.83 | 33.21 ± 4.65 | — |
SWAGES reaches a total cost of 150 in 4.48 hours, compared with 10.10 hours for GATES and 14.87 hours for SPN-CWS. This represents a 55.6% reduction in training time relative to GATES.
These results show that SWAGES achieves low inference latency and faster convergence to target scheduling costs, despite the additional computations required by GUIDES.
(4) W1: Does SWAGES Generalize Beyond the Original Test Settings?
We conducted two additional experiments using the same trained policy without retraining or fine-tuning. They examine generalization to substantially larger workloads and a previously unseen workflow type.
Cross-scale generalization. Table R9 reports performance on 1,000-task workloads, substantially larger than the workflow scales used during training. SWAGES achieves the lowest mean cost across all five deadline settings.
Table R9. Total scheduling cost on 1,000-task workloads (mean ± std; lower is better).
| SWAGES | GATES | SPN-CWS | |
|---|---|---|---|
| 1.0 | 1,369.7 ± 178.2 | 8,806.0 ± 5,144.4 | 11,829.3 ± 6,377.6 |
| 1.5 | 1,340.2 ± 171.3 | 11,724.3 ± 8,916.2 | 17,549.0 ± 7,401.2 |
| 2.0 | 1,350.7 ± 189.9 | 12,909.1 ± 9,600.6 | 17,565.4 ± 8,162.4 |
| 2.5 | 1,327.6 ± 186.2 | 13,229.5 ± 9,287.7 | 17,997.2 ± 8,223.0 |
| 3.0 | 1,341.9 ± 197.9 | 13,714.3 ± 9,077.0 | 17,454.6 ± 7,474.1 |
Generalization to unseen workflows. We further evaluated SWAGES on Epigenomics, a workflow type excluded from training.
Table R10. Total scheduling cost on the unseen Epigenomics workflow type (mean ± std; lower is better).
| SWAGES | GATES | SPN-CWS | DRL-G | |
|---|---|---|---|---|
| 1.0 | 214.71 ± 6.35 | 633.45 ± 150.23 | 960.50 ± 459.09 | 1174.23 ± 741.27 |
| 1.5 | 190.82 ± 12.72 | 566.47 ± 65.72 | 677.86 ± 234.39 | 1158.68 ± 741.30 |
| 2.0 | 169.37 ± 13.96 | 555.54 ± 83.33 | 687.54 ± 115.02 | 1143.13 ± 741.35 |
| 2.5 | 154.54 ± 12.42 | 593.14 ± 189.45 | 664.35 ± 215.73 | 1127.63 ± 741.36 |
| 3.0 | 144.18 ± 11.69 | 604.92 ± 138.75 | 644.28 ± 178.04 | 1112.23 ± 741.28 |
SWAGES reduces mean cost relative to GATES by 84.4%–90.2% on the 1,000-task benchmark and 66.1%–76.2% on Epigenomics. These results demonstrate strong generalization to larger workloads and unseen workflow structures without retraining.
(5) Q3: How Are the Guidance Gradients Computed?
We agree that the relationship between Eq. (16) and the centered critic weights in Eq. (63) was unclear. The guidance computation should be defined consistently for discrete and continuous actions.
Discrete actions (CADWS). The twin critics evaluate all possible task–VM assignments, giving . For each guidance batch , the surrogate objective is
\sum_{s\in\mathcal B_k} \sum_{a\in\mathcal A(s)} \pi_\theta(a\mid s)\widehat Q(s,a).$$ Its gradient is evaluated at the current center parameters $\theta_g$, with the critics held fixed. This objective directly weights every possible assignment by its critic-estimated value. **The batch-centered replay-action weights in Eq. (63) are not part of this formulation** and will be removed to eliminate the inconsistency. **Continuous actions (robotics).** For a differentiable deterministic actor $\mu_\theta(s)$, the corresponding surrogate objective is $$\mathcal J_k^{\mathrm{cont}}(\theta)= \frac{1}{B_G} \sum_{s\in\mathcal B_k} \widehat Q(s,\mu_\theta(s)).$$ The guidance gradient is computed by backpropagating through the critic with respect to the continuous action and then through the actor. The resulting directions are normalized and orthogonalized to construct the search subspace. **In both cases, these surrogate gradients guide ES population sampling; they do not directly update the policy.** The policy update remains determined by rollout fitness. We will remove the inconsistent Eq. (63) and update the appendix and pseudocode to precisely present the guidance-gradient calculations described above.
W1 and Q1–Q3. Experimental Validation and Implementation of SWAGES
We thank the reviewer for these constructive comments. We clarify the experimental differences, provide additional controlled comparisons of SWAN and GUIDES, evaluate computational efficiency and generalization, and clarify the guidance-gradient computation.
(1) Q1: Why do Tables 1 and 2 report different costs?
The difference comes from the number of workflows per test instance, not from different cost calculations. Table 1 uses 30 workflows per instance, whereas the original Table 2 uses only 5. The smaller workload was chosen to reduce the computational cost of repeated ablation experiments. We acknowledge that this distinction was insufficiently explained.
To address this concern, we repeated the ablations using 30 workflows per test instance, matching the main evaluation workload. The updated SWAN results are shown in Table R3.
Table R3. SWAN architecture ablation at γdl=1. Total cost reported as mean (std); lower is better.
Removing any SWAN component increases mean total cost across all three scales, confirming the importance of the proposed architecture under the main evaluation workload. The full method also achieves nearly identical mean costs in Table 1 and the updated ablations (70.44 vs. 70.43 for S-scale workflows at γdl=1).
We will clarify the original reduced-workload setting and the evaluation protocol in the revised manuscript.
(2) W1 and Q2: What Are the Independent Contributions of SWAN and GUIDES?
We isolate the two contributions by holding the training algorithm or policy architecture fixed.
SWAN's contribution. Under the same OpenAI-ES optimizer, SWAN achieves mean costs of 73.68, 111.22, and 163.78, compared with 139.05, 231.53, and 370.50 for GATES on S-, M-, and L-scale workflows, respectively. These reductions of 47.0%, 52.0%, and 55.8% demonstrate the benefit of SWAN independently of GUIDES.
GUIDES' contribution. Holding SWAN fixed, we compare GUIDES with standard OpenAI-ES, Random Subspace, REINFORCE Guidance, and mixed-search variants.
Table R4. Comparison of policy search strategies using SWAN at γdl=1. Total cost reported as mean (std); lower is better.
GUIDES reduces mean cost relative to OpenAI-ES by 4.4%, 4.0%, and 2.4% across the three scales. Random Subspace performs substantially worse, showing that dimensionality reduction alone is insufficient. GUIDES also outperforms REINFORCE Guidance and all tested mixed-search variants, supporting the effectiveness of critic-derived search directions.
Together, these controlled comparisons establish independent performance gains from SWAN's system-wide scheduling architecture and GUIDES' critic-guided evolutionary search.