- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes an asynchronous, edge-level message scheduling scheme for GNN-based autoregressive dynamics prediction. At each intermediate processor layer, only a quantile-selected subset of edges delivers its messages, ranked by a state-dependent priority score . The increments of inactive edges are partially retained in a persistent per-edge buffer through a component-wise gate and are released at the next activation. A terminal full refresh then flushes all buffers before decoding. The scheduler is combined with a physics-motivated decomposition of the latent vector field into transport, diffusion, reaction, skew-symmetric coupling, and source-sink forcing, followed by a projection block, with transport and diffusion assigned separate priority clocks and activation ratios. The authors establish several structural properties (synchronous consistency, permutation equivariance, bounded buffers, terminal synchronization, and a conditional rollout bound) and report the lowest RMSE among nine baselines on four benchmarks, supported by an extensive set of ablations.
Critical strengths
1. A clear, simple idea that plugs into existing models. The observation that synchronous message passing ties propagation depth to interaction-update frequency is intuitive, and the question of when a message should be delivered has received comparatively little attention in the learned-simulator literature. The scheduler leaves the graph topology and backbone untouched, and the percentile rule itself introduces no learnable parameters. Table 3 shows that changing only the propagation schedule improves both MGN and MP-PDE on all four datasets, which lends credibility to the claim that the method is model-agnostic.
2. Thoughtful controlled comparisons. The authors clearly anticipated the "this is just sparsification" objection. Under a matched activation budget, they compare the proposed priority against random, static-distance, and magnitude-based policies (Table 6). They also compare the masked buffer against no buffer, latest-message-only, and random or shuffled gates (Table 5), and the full scheduler against node-batch updates, delayed aggregation, and multi-rate gating (Table 8). The staleness diagnostics in Table 7 are a welcome addition.
3. Careful formulation and honest scoping of claims. Several technical details are handled with care, including the orientation consistency of the upwind transport flux, the symmetrization of the diffusivity together with zero-sum incidence scattering, and the role of the tie-breaking rule in permutation equivariance (Remark H.4). I also appreciate that the authors are explicit about what they do not claim. The rollout bound does not imply that asynchronous scheduling is more stable than its synchronous counterpart, explicit Euler integration does not preserve the latent norm, and masking alone does not reduce computation.
Critical weaknesses
1. The central hypothesis is not directly tested, and the source of the gains remains unclear. The paper argues that deferring weak or slowly varying interactions reduces redundant transformations and thereby mitigates error accumulation during autoregressive rollout. The experiments, however, report only aggregate RMSE.
- There are no plots of error versus rollout step and no estimate of the effective Lipschitz constant for the trained synchronous and asynchronous models. Corollary 3.5 holds for any one-step map, so it does not distinguish the proposed method from the baselines.
- There is no analysis of which edges are deferred, for example whether they lie outside the wake, in low-velocity regions, or where gradients are small. Without such evidence, the "deferring slow interactions" narrative remains a hypothesis.
- The degree of asynchrony is in fact quite shallow. With and , the mean waiting depth is only 1.1 to 1.5 layers, with a maximum of 3. This resembles a structured perturbation of roughly 20% of the edges per layer and is difficult to distinguish from regularization effects such as DropEdge or stochastic depth. A baseline that drops edges at random during training but propagates synchronously at inference is missing.
- The most direct test of the paper's thesis is absent. If the problem is the coupling of depth and update frequency, the benefit of asynchrony should grow with depth, yet no sweep over is reported.
- Although the title emphasizes "delay-aware" scheduling, none of the benchmarks involves genuinely delayed dependencies, and the authors themselves describe the priority score as a relative ranking rather than a physical delay.
2. Concerns about experimental rigor and fairness.
- Several improvements are small. The gain on Gray-Scott is 0.0089 versus 0.0093, and most single-operator ablations differ by only 1 to 2%. Standard deviations are reported only in Table 1, so the significance of the ablation results cannot be assessed.
- Some numbers are identical across tables, e.g., Delayed Aggregation in Table 8 and Full Accum. in Table 5, and Random Edge Update in Table 8 and Random in Table 6. These may well be reused configurations, but this should be stated explicitly.
- The comparison between the full model and MGN is confounded. The full model includes a self-attention-based global refinement , and removing the projection block causes the largest degradation in the ablation (e.g., from 0.1426 to 0.1579 on CF). A substantial share of the gain may therefore stem from global attention rather than from the physics-motivated decomposition. The parameter count and FLOPs of the full model are not reported, and MGN and CiGNN are not equipped with the same projection block.
3. Limited practical value, with added complexity but no efficiency gains.
- Despite the "asynchronous" framing, the method yields no computational savings. Training is about 6% slower per epoch and inference is also slower, because all edge messages must still be computed before ranking. The computation-sparse variant is discussed but neither implemented nor benchmarked.
- The method introduces many components (five operators, two priority clocks, gates, buffers, and a projection block), and and are set by hand. Table 10 shows that performs worse than the synchronous model on three of the four datasets, indicating considerable sensitivity to the scheduling ratios.
- The main text describes the mechanism in terms of increments , whereas Appendix C uses a node-message formulation in terms of . It is unclear which of the two is actually implemented.
Questions for authors
- Is the delivery mask applied during both training and inference? Since the hard quantile mask is non-differentiable, do gradients flow only through the delivered increments, and does detaching the message norm in the priority score affect the results?
- At a matched average activation rate, how does a model trained with random edge dropping and evaluated with full synchronous propagation compare with the proposed method? This control seems essential for separating the effect of the scheduling mechanism from that of regularization.
- Could you provide direct evidence for the claimed mechanism? In particular, how does the gain over synchronous propagation change for , how does RMSE evolve over the rollout horizon, and do the deferred edges correlate with physical quantities such as local velocity, gradients, or vorticity?
- What are the parameter count and FLOPs of the full model, and how much of the gap remains if MGN or CiGNN is equipped with the same projection block? For the prior asynchronous and multi-rate GNNs in Table 8 (e.g., GwAC, Co-GNN, DRew, and G²), did you run the official implementations or re-implement them under a common backbone?
- Please explain the identical numbers in Tables 5, 6, and 8, and clarify whether the scheduling ratios and retention gates were selected on the validation set separately for each dataset. Could you also report longer rollouts on Black Sea together with physically meaningful metrics such as energy spectra or the drift of conserved quantities?
5 comments
Rebuttal to the comments by Reviewer kCBo (Part 2)
Q3: Please compare gains at L ∈ {4, 8, 16}, rollout error, perturbation test, and deferred edges versus velocity, gradients, or vorticity. Should gains increase with depth? How is “delay-aware” justified without demonstrated physical delays, given that the priority is a relative ranking and the corollary does not distinguish models?
Reply: Table K compares L ∈ {4, 8, 16} for the MGN backbone on Burgers. Asynchronous delivery is better at every depth, with a stable relative reduction of 56.8%, 52.9% and 54.5%. At ρ = 0.8 with terminal refresh, the overall delivery fraction is ρ+(1−ρ)/L, as listed below. Rollout error will be shown in a new figure (Fig. S2) in the revision.
Table K: Depth sensitivity and delivery budgets.
| L | Synchronous RMSE | Asynchronous RMSE | Actual overall delivery fraction |
|---|---|---|---|
| 4 | 0.0118 | 0.0051 | 85.00% |
| 8 | 0.0104 | 0.0049 | 82.50% |
| 16 | 0.0099 | 0.0045 | 81.25% |
Regarding the deferred edges, the priority is defined on the latent edge increments, and we do not claim that deferred edges correspond to identifiable physical regions such as low-velocity or low-vorticity zones. Table 7 reports mean waiting depths of 1.12–1.47, a maximum of 3, and pre-terminal coverage of 91.3%–97.7%. Independent random selection of 80% of edges over three intermediate steps would give about 99.2% expected coverage. The lower observed coverage is consistent with persistent selection across layers, rather than independent random perturbations.
Corollary 3.5 is conditional and does not establish smaller sensitivity constants for asynchronous models (Remark H.10). One-step error under input perturbation is reported in Table L with masks recomputed after perturbation.
“Delay-aware” refers to delivery across processor micro-steps within one forecasting step, not a calibrated physical travel time, as stated in Sec. 3.2 and Appendix C. It does not require explicit delay terms in the data equations. CF, GS and Burgers have no such terms; the daily Black Sea data do not establish physical delay dependencies. We will add a one-sentence definition of this usage in the introduction and at the start of Sec. 3.2.
Table L: One-step error under input perturbation on Burgers.
| Perturbation (standard deviation) | Synchronous one-step RMSE | Asynchronous one-step RMSE |
|---|---|---|
| 0(zero perturbation) | 3.48e-5 | 1.52e-5 |
| 1e-4 | 3.57e-5 | 1.627e-5 |
| 1e-2 | 4.11e-5 | 1.832e-5 |
Q4: Could global attention explain the full model’s advantage over MGN/CiGNN? Please report full-model parameters and compare backbones equipped with the same G_att. Were the purported prior methods in Table 8 (GwAC, Co-GNN, DRew, G²) run officially or reimplemented under a common backbone?
Reply: Global attention does not account for the scheduling gain: in Table 3, MGN and MP-PDE have no G_att and improve on all four datasets with scheduling alone, and in Tables 2 and 10 the synchronous and asynchronous full models share the same operators and G_att.
To separate attention from the full-model advantage over MGN, Table M compares MGN + G_att with synchronous and asynchronous delivery. For MGN + G_att, asynchronous delivery lowers RMSE on all four datasets, and the full model is lower than MGN + G_att in both modes on all four datasets, so attention alone does not account for its advantage. On CF, adding G_att does not improve synchronous MGN (0.2615 vs 0.1892), while on BS, GS and Burgers it does. Parameter counts are listed in Table M.
Table M: Global-attention controls
| Model | Parameters | CF | BS | GS | Burgers |
|---|---|---|---|---|---|
| MGN(4 layers) | 0.57M | 0.1892 | 0.6147 | 0.0193 | 0.0118 |
| MGN + G_att, synchronous | 0.83M | 0.2615 | 0.5345 | 0.0102 | 0.0083 |
| MGN + G_att, asynchronous | 0.85M | 0.1912 | 0.5294 | 0.0098 | 0.0053 |
| Ours (4 layers) | 0.96M | 0.1426 | 0.5191 | 0.0089 | 0.0044 |
Table 8 contains no official implementations of GwAC, Co-GNN, DRew or G²; these methods are discussed in related work only. Table 8 compares update mechanisms implemented by us under a common backbone and a matched update budget (Node-Batch Async and Multi-Rate Gating), together with variants that coincide with rows of Tables 3, 5 and 6 (see Q5).
Rebuttal to the comments by Reviewer kCBo (Part 3)
Q5: Why are results repeated across Tables 5, 6, and 8? Were ratios and retention configurations selected separately on each dataset’s validation set? How should sensitivity to manually set ratios be interpreted when (0.6, 0.6) loses to synchronous delivery on three tasks? Can longer Black Sea rollouts include meaningful physical metrics such as spectra or conservation drift?
Reply: Repeated values are reused results, not independent runs. Table 8’s Delayed Aggregation, Random Edge Update, and MGN + Async. correspond to Full Accum. in Table 5, Random in Table 6, and MGN + Async. in Table 3, respectively; MGN + Async. also matches Masked Accum. in Table 5. We will cross-reference these rows in the captions.
Ratios and gates. Hyperparameters, including the scheduling ratios and the retention-gate configuration, are selected on each dataset’s own validation set (Appendix L.4); Table 10 reports test performance of the candidate settings. More deferral is not always better. Table 4 favors ρ = 0.8 over synchronous MGN on all four tasks, while the reported settings with ρ ≤ 0.4 are worse. In Table 10, (0.8, 0.8) is best among the listed settings; (0.6, 0.6) is worse than synchronous delivery on CF, BS, and Burgers but better on GS (0.0094 versus 0.0098).
Black Sea. Existing results use a common 20-step rollout protocol for all methods. Atmospheric forcing and data assimilation mean that surface kinetic energy is not a conserved quantity here, so we do not report conservation drift for this dataset.
Q6: What practical benefit offsets the added complexity when all edge messages are still computed, training is about 6% slower, inference is slower, and the computation-sparse variant is unimplemented?
Reply: The practical benefit is improved forecasting with modest runtime overhead, not acceleration. Table 9 within more broader and larger architecture reports aggregation reduced from 100% to 82.5%, while training rises from 44.2 to 46.8 seconds/epoch and inference from 0.0079 to 0.0085 seconds/step because all candidate messages are still computed.
Applying the scheduler to an existing backbone does not require the five heads or dual priorities. It adds the retention gate (about 0.025M parameters for the 4-layer configuration and 0.049M for the 8-layer configuration) and the buffering mechanism. Table 3 demonstrates the resulting accuracy benefit on MGN and MP-PDE. Computation-sparse execution remains future work.
Concluding remark: Thank you again for your constructive feedback. We hope these clarifications address your concerns and will incorporate the completed results and implementation details into the revision.
Rebuttal to the comments by Reviewer kCBo (Part 1)
We sincerely thank you for your detailed review and constructive suggestions. We appreciate your recognition of the simple, portable idea, the controlled comparisons, and the careful scope of the theoretical claims. Below, we address the mechanism, experimental controls, implementation, and practical trade-offs point by point.
Q1: Are delivery masks used during both training and inference? How do gradients reach deferred messages under hard selection, and does detaching the ranking norm matter? Which implementation is used: main-text increments Δm or Appendix C messages m?
Reply: The delivery schedule (selection, buffers and terminal refresh) is used in both training and inference. Deferred messages can receive gradients through later delivery. Once a buffered increment enters a subsequent node update, the loss can backpropagate to it and its retention gate, since the buffer computation graph is retained.
Hard quantile selection does not propagate ranking gradients. The message norm is used only for this selection, so detaching it changes neither the forward ranking nor the gradient path.
The two notations describe two uses of the same buffer recurrence in Eqs. (14)/(S25):
Q2: Could the gains from asynchrony reflect DropEdge-like regularization? At a matched activation budget, how does random training-time edge dropping followed by synchronous inference compare?
Reply: Table J includes DropEdge-like random edge dropping during training followed by synchronous inference. It is worse than synchronous propagation on all four datasets. Dropping edges permanently discards their information and, when training with dropping but inferring on the full graph, changes the aggregation statistics between training and inference.
Our mechanism defers rather than discards: deferred increments are kept in the buffer and delivered later, and the terminal refresh ensures that every edge contributes within each forecasting step. Training and inference use the same schedule, and Full Accum. loses no increment; the learned gate in Masked Accum. only filters components of deferred increments, and it further improves over Full Accum. on CF, BS and Burgers (Table 5).
Table J: Scheduling-policy comparison