acceptodds
Withdrawn from ICLR 2027

Physics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Grid foundation models (GridFMs) are graph transformers pretrained on a masked power-flow objective, and once trained they evaluate power flow orders of magnitude faster than an iterative solver. Their size, however, targets data-centre inference, whereas low-inertia operation - grids with little rotating reserve, where disturbances evolve within milliseconds - increasingly asks for inference at the edge, next to the measurement, at batch size one and within a single synchrophasor (PMU) reporting frame. We distil a 20.1M-parameter, 12-layer GridFM teacher into a 0.62M-parameter, 6-layer student, a 32.5× parameter reduction at half the message-passing depth, using an objective that combines the masked powerflow loss with output, physics-residual, latent-state and attention transfer. The physics-residual term, which asks the student to reproduce the teacher’s active- and reactive-power residual profile rather than only to minimise its own, has not previously been applied to typed power-grid graphs. We evaluate on a renewable variant of the IEEE 118-bus benchmark grid in which 26 of the conventional (synchronous) generators are replaced by inverter-based renewables with battery flexibility and standardised reactive-power limits (IEEE 1547), giving 97 529 physically valid (AC-OPF-feasible) operating scenarios, and we ablate all eight combinations of the transfer terms, reporting accuracy against ground truth and fidelity to the teacher separately. Distillation lowers student error on all four predicted quantities, by up to 48% on active-power dispatch, and every distilled student is more accurate than its teacher on reactive-power dispatch. The gain is largely carried by the output matching term. The physics-residual term, in the layer-aggregated form used here, is consistently harmful, which we attribute to the spatial averaging it applies to a bus-level residual field. We study the effect of the parameter reduction on latency on the target device to validate the effectiveness of the reduced layer count.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Optimal Power Flow, Knowledge Distillation, GridFM, Graph Neural Networks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7GwfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

this paper proposes a distillation method to distill large GNNs into small ones (especially with fewer layers) and maintain good performance and reduce the latency.

Critical strengths
  1. This paper’s writing is good and clear
  2. Design of the distillation method is reasonable
  3. It has a detailed introduction on the domain they studied (Section 3.2) which is necessary for broad machine-learning practitioner.
Critical weaknesses
  1. The novelty is somewhat limited, even for a domain application-oriented paper. 3 out of the 4 distillation loss terms are off-the-shelf and the only domain-specific loss term is Eq. 3.
  2. It is a bit surprising the proposed 4 distillation terms somewhat conflicts with each other, because according to Table 4, the best combination is without the L_physKD and L_lat terms.
Questions for authors

Suggestions:

  1. Lots of the \cite shall be \citep
  2. Eq. and Equation are both used. Authors might want to use only one of them.
Reviewer_GCzhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper propose a method that distils a 12-layer grid foundation model with approximately 20 million parameters into a 6-layer student model. The student model has less parameters with 0.62 million parameters and is applied in Jetson Orin Nano. It evaluate output, physics-residual, latent-state, and attention distillation on a modified IEEE 118-bus system. Distillation improves the student over an undistilled model of the same size.

Critical strengths
  1. clear problem definition and specific on-device measurements. The paprer reports batch-one latency and peak memory. The mdeian latency is 6.08ms, compared to teacher’s 24.85, which is a large improvement.

  2. An informative ablation study. Compare different distillation objectives under the same student architecture, and explicitly distinguish between “closeness to ground-truth labels” and “closeness to teacher predictions,” making it clear to readers that the two are not equivalent.

Critical weaknesses
  1. Poor readability and lack of clear explanations. The paper is difficult to read due to unclear writing and insufficient definitions of mathematical notation and formulas. Crucially, the paper lacks illustrative figures or diagrams to help readers understand the proposed methodology.

  2. Limitied generelization. The latency measurements are obtained under a fixed topology with model warm-up and CUDA Graph enabled, while accuracy is primarily evaluated on generated scenarios based on a single IEEE 118-bus topology. These results demonstrate fast model inference under the specific experimental setting but are insufficient to establish the model's performance under varying grid topologies.

  3. Insufficient Baseline Comparisons. The paper mainly compares against the teacher and an undistilled student, lacking comparisons with directly trained lightweight models or other compression methods. Thus, the advantages over existing lightweight approaches remain unclear.

Questions for authors

Could the authors add more figures to better illustrate the proposed method and clarify the mathematical notation? Adding more comparisons with lightweight models and other compression methods may help establish its effectiveness. Experiments on different grid topologies would also strengthen the generalizability claims.

Reviewer_CU4fOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates distillation of grid foundation models, aiming to enable real-time inference on edge devices. The authors compress a 20M-parameter teacher GridFM into a 0.62M-parameter student by combining output, physics-residual, latent-state, and attention distillation. Experiments on a modified IEEE 118-bus system demonstrate improved prediction accuracy over an undistilled student and reduced inference latency on edge hardware. However, most improvements come from standard output distillation, while the proposed physics-aware component provides no consistent benefit.

Critical strengths
  • The problem is with clear practical motivation, particularly for deploying grid models on resource-constrained devices.
Critical weaknesses
  • Limited novelty. The framework largely combines established knowledge distillation objectives. Applying these techniques to power-grid models does not, by itself, constitute a substantial ML contribution.
  • Ineffective main component. Physics-residual distillation consistently degrades average prediction accuracy across all four quantities. This substantially weakens the paper's central methodological claim.
  • Limited topology covering. Experiments are restricted to a modified IEEE 118-bus system, without cross-topology evaluation or comparison against stronger compression alternatives such as quantization and pruning.
Questions for authors

See above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.