- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
LLM-PowerTrace provides time-resolved GPU power traces from 1,788 fine-tuning runs and 401 serving runs, together with workload settings and execution metadata. The paper uses these traces to study power fluctuations and develop models of power and energy use across different workloads and hardware.
Critical strengths
- The contribution is clear and useful: the dataset captures how GPU power changes over time, which aggregate energy measurements cannot show. Covering both fine-tuning and inference makes it useful for a broad range of studies.
- The controlled experiments, execution metadata, and tools for exploring the traces make the dataset valuable for studying and modeling power behavior. The paper also openly discusses measurement and modeling limitations.
Critical weaknesses
- Polling every 100 ms does not necessarily capture power changes at that resolution, since the GPU sensors can average readings over roughly one second. This limits how confidently the traces can describe short spikes and fast ramps.
- Most inference configurations are not deliberately repeated, and the power model misses some important fluctuations, especially under continuous high load.
Questions for authors
-
What time resolution can users reliably study with this dataset? How do the sensor averaging windows differ across GPUs and measurement tools?
-
How well does the model predict changes in power, beyond the average? The appendix notes that it cannot capture ramps when the server stays full.
-
How should users connect GPU traces to datacenter or grid behavior? The dataset measures GPU power rather than total server or facility power. What additional measurements or assumptions are needed, and how far can the observed synchronization patterns be extended beyond one node?
7 comments
(W1, Q1) - Reliable Temporal Resolution and GPU Sensor Averaging Behavior
(W1, Q1) - Sensor averaging windows differ across GPUs and measurement tools
Sensor experiment. We distinguish two NVIDIA Management Library (NVML) power measurement fields: power.draw, the standard reported GPU power reading, which may incorporate temporal averaging, and power.draw.instant, which reports power without the longer averaging window applied to the standard field. We have supplemented these results in Appendix G of the revised manuscript.
1. Experimental setup. We tested three GPUs (H100, H200, and A100) using a controlled matrix-multiplication workload with known ground-truth power transitions, including full-load steps and isolated 200-ms bursts.
We sampled power through NVML at approximately 6-ms intervals. By sampling substantially faster than the sensor update rate, we identified how frequently the reported values actually change.
2. Key findings: Sensor update rate versus effective resolution. The experiments reveal that although power readings update approximately every 100 ms on all three GPUs, their effective temporal resolution differs substantially due to sensor averaging (Table 1).
- H100 and H200 (
power.draw): The averaged power readings (power.draw) used in the original manuscript apply a moving average over approximately 1 s, smoothing short-duration fluctuations. Consequently, a 200-ms burst appears as a ~1-s bump at roughly one-quarter of its actual height. - H100 and H200 (
power.draw.instant): We have updated our experiments to use instantaneous power readings(power.draw.instant), which do not apply the 1-s averaging window. These measurements update approximately every 100 ms and capture the full magnitude of the 200-ms power bursts, providing an effective temporal resolution of approximately 0.2 s. This substantially improves our ability to characterize short-duration power fluctuations and ramp events. - A100: The average reading is effectively instantaneous. A 200-ms burst retains approximately 91% of its actual height, indicating a much shorter effective averaging window (at most ~0.25 s).
- Energy measurement accuracy: Integrating the recorded power measurements agrees with the GPU's built-in energy counter within 0.11% across all traces used in the paper.
Table 1. How often each GPU power reading updates, how much it smooths, and the shortest event it resolves, measured with a load of known on/off times.
| GPU | Power reading | Updates every | Smoothing (effective bandwidth) | A 200-ms burst appears as | Shortest event resolved with accurate magnitude and timing |
|---|---|---|---|---|---|
| H100, H200 | Instantaneous reading (updated experiments, power.draw.instant) | ~100 ms | Snapshot of a short window (~25 ms every 100 ms) | Full height | ~0.2 s |
| H100, H200 | Averaged reading (power.draw) | ~100 ms | Average over the past 1 s | A ~1-s bump at about 1/4 of its height | ~1 s |
| A100 | Instantaneous reading | ~100 ms | Snapshot of a short window (~25 ms every 100 ms) | 91% of its height | ~0.2 s |
3. Sensitivity analysis: Impact on reported metrics and conclusions. To quantify the effect of temporal resolution, we recomputed the power metrics after averaging the traces over 1-s and 10-s windows, compared with the 100-ms resolution (Table 2).
- Mean and P95 power: Mean power remains largely unchanged (approximately 1% or less), while P95 power shows modest reductions with coarser averaging.
- Peak power and ramp rates: Are sensitive to temporal resolution and whether averaged or instantaneous readings are used.
Table 2. How the main metrics change when the instantaneous reading is averaged to 1 s and 10 s (medians over runs, relative to 100 ms).
| Metric (instantaneous reading, from 569 additional H100 runs) | 100 ms | 1 s | 10 s |
|---|---|---|---|
| Mean power | 1.00 | 1.00 | 1.00 |
| 95th-percentile power | 1.00 | 0.98 | 0.93 |
| Peak power | 1.00 | 0.85 | 0.76 |
| 99th-percentile ramp rate | 1.00 | 0.52 | 0.14 |
| Spectral entropy (rank correlation with 100 ms) | 1.00 | 0.86 | 0.53 |
(Q2) - Model Accuracy in Predicting Power Fluctuations and Ramp Rates
(Q2) - Model Accuracy in Predicting Power Fluctuations and Ramp Rates
We thank the reviewer for the important question regarding whether our power scaling law can predict temporal power fluctuations and ramp rates beyond average power To address this concern, we extended our evaluation beyond mean power to include P95 power, P99 1-s ramp rates, and temporal correlation between predicted and measured 1-s power trajectories within each run. We applied the original fitted power law, without refitting, to predict power trajectories using the number of requests in flight at 1-s resolution, reconstructed from per-request timestamps (Table 3).
We consider two serving modes: stochastic-arrival workloads, where requests follow externally specified arrival patterns (Poisson or BurstGPT), and fixed-concurrency workloads, where the client maintains a fixed number of outstanding requests by replacing completed requests.
1. Key findings. Our results show that the model accurately predicts power levels and captures fluctuations driven by changes in request concurrency, but cannot fully capture internal GPU execution dynamics.
- Stochastic-arrival workloads: On 42 H200 runs with varying request arrivals, the model achieves a median mean-power error of 2.5% and P95 error of 2.4%. It also captures temporal power patterns, with a correlation of 0.83 and a P99 ramp error of 31%.
- Fixed GPU clock frequencies: On 21 H100 runs with locked clocks (345–1605 MHz), the model maintains a temporal correlation of 0.81 but has a P95 power error of 16%. This occurs because request concurrency extends beyond the range used to fit the clock-dependent terms.
- Fixed-concurrency workloads: The model accurately predicts mean and P95 power (errors of 2.8% and 3.9%), but fails to capture fluctuations caused by batch turnover and internal scheduling, with a temporal correlation of only 0.08 and a P99 ramp error of 98%. This limitation arises because the request count remains nearly constant despite changes in GPU execution activity.
Table 3. Accuracy of the power scaling law beyond mean power. Errors are reported as median absolute percentage errors across runs. Correlation measures the temporal correlation between predicted and measured 1-s power trajectories within each run.
| Regime (1-s load) | Runs | Mean power error | P95 power error | P99 ramp error | Correlation |
|---|---|---|---|---|---|
| Stochastic-arrival workloads, H200 | 42 | 2.5% | 2.4% | 31% | 0.83 |
| Stochastic-arrival workloads, H100, fixed GPU clock frequencies | 21 | 5.4% | 16.0% | 99% | 0.81 |
| Fixed-concurrency workloads (always full) | 118 | 2.8% | 3.9% | 98% | 0.08 |
2. Implications and limitations. These results clarify that our power law is effective for predicting power levels and workload-driven fluctuations, particularly under varying request arrivals, but cannot reliably predict power ramps caused by internal execution dynamics when concurrency remains constant. Modeling such fluctuations requires additional information about GPU execution phases and LLM serving scheduler states.
Importantly, this predictive model represents one illustrative use case of our dataset, rather than its primary contribution. Our broader goal is to provide the research community with comprehensive, time-resolved GPU power measurements to support the development and evaluation of more sophisticated power prediction models. To facilitate this, our additional H100 experiments include per-step scheduler information, which we plan to release to support modeling of execution-dependent power dynamics.
We will include these expanded evaluations and clarify the model's applicable operating conditions and limitations in the revised manuscript.
(W1, Q1) - Reliable Temporal Resolution and GPU Sensor Averaging Behavior
We thank the reviewer for the important question regarding the temporal resolution and sensor. To address these concerns, we validate instantaneous NVIDIA Management Library (NVML) telemetry against 10-kHz oscilloscope measurements and characterize sensor behavior on A100/H100/H200, demonstrating reliable characterization of major power fluctuations lasting approximately 0.2 s or longer, while identifying limitations in capturing short-duration peaks and ramp rates.
(W1, Q1) - Validation of GPU power telemetry against oscilloscope measurements
To assess the accuracy and temporal resolution of GPU power telemetry, we conducted additional experiments comparing instantaneous software power measurements against high-frequency oscilloscope measurements (Fig. 43 of the revised manuscript). Our results show that instantaneous NVML telemetry reliably characterizes major power fluctuations lasting approximately 0.2 s or longer. We supplemented details in Appendix F of the revised manuscript. The key setup and results are summarized below.
1. Experimental setup. We tested an RTX 5090 GPU (600-W power limit) serving Qwen3-8B with vLLM 0.28.0 under fixed-concurrency workloads of 64 requests, each lasting 120 s. We simultaneously recorded:
power.draw.instant, polled at 50 Hz.2. Key findings. Our results demonstrate that instantaneous software telemetry accurately captures major power fluctuations at timescales of approximately 200 ms and above.
3. Relevance to power-system applications. These findings help clarify which applications can be supported by the dataset's temporal resolution.
4. Generalizability and limitations. Overall, these results support the use of instantaneous GPU telemetry for characterizing workload-induced power fluctuations over timescales of approximately 0.2 s and longer, while establishing clear limitations for sub-100-ms dynamics.
We used an RTX 5090 GPU because it was the only GPU available to us with an accessible dedicated power cable that allowed direct hardware-level power measurement. Although this experiment does not directly validate the transient measurement accuracy of the A100/H100/H200 GPUs, it provides independent hardware validation of the same NVML instantaneous power field and vLLM serving framework used in our experiments. The sensor characterization experiment in the next post (i.e., (W1, Q1) - Sensor behavior on A100/H100/H200) will further confirm that this field has a similar ~100-ms refresh interval on A100, H100, and H200.
References
[1] K. Chatterjee, J. Follum, A. Varghese, S. Biswas, E. Farantatos, and L. Zhu, "Measurement Adequacy for Monitoring Data Center Oscillations," Pacific Northwest National Laboratory (PNNL), Technical Report PNNL-39191, April 2026. Link.
[2] IEEE Standards Association, "Review of Industry Efforts and Standards of Grid Readiness for Data Center Deployment," IEEE SA White Paper, January 2026. Link.