acceptodds
Under review as a conference paper at ICLR 2027

LLM-PowerTrace: A Dataset of Time-Resolved GPU Power Traces for LLM Fine-Tuning and Inference

Abstract

AI compute is becoming large and fast-changing electrical loads. Its impacts on data center infrastructure and power grids depend not only on total energy consumption, but also on how their power draw moves over sub-second to minute timescales, including peaks, ramps, and synchronized fluctuations across accelerators. Existing energy benchmarks for large language models (LLMs) report aggregate quantities such as joules per token and rarely release the underlying time-series measurements, leaving these power dynamics poorly characterized and difficult to model. We introduce LLM-POWERTRACE, an open dataset of time-resolved, per-GPU power traces for LLM fine-tuning and inference, sampled every 100 ms on varying NVIDIA platforms paired with complete workload and system metadata. For fine-tuning, LLM-POWERTRACE profiles open model families spanning a wide range of model sizes and fine-tuning methods, together with controlled sweeps over batch size, sequence length, precision, parallelism, and checkpointing. For inference, it covers variations in concurrency, GPU frequency, KV-cache and weight quantization, prompt and output lengths, and request-arrival patterns. Built upon power traces, we define power-grid-facing metrics as well as AI compute power&energy scaling law for extrapolating beyond the measured settings. Together, LLM-POWERTRACE provides a dataset foundation for modeling, forecasting, and managing the fast-changing electrical behavior of AI compute.

Primary area:
datasets and benchmarks
Keywords:
AI Workloads, Dataset, Power, Energy

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_5oyJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

LLM-PowerTrace provides time-resolved GPU power traces from 1,788 fine-tuning runs and 401 serving runs, together with workload settings and execution metadata. The paper uses these traces to study power fluctuations and develop models of power and energy use across different workloads and hardware.

Critical strengths
  • The contribution is clear and useful: the dataset captures how GPU power changes over time, which aggregate energy measurements cannot show. Covering both fine-tuning and inference makes it useful for a broad range of studies.
  • The controlled experiments, execution metadata, and tools for exploring the traces make the dataset valuable for studying and modeling power behavior. The paper also openly discusses measurement and modeling limitations.
Critical weaknesses
  • Polling every 100 ms does not necessarily capture power changes at that resolution, since the GPU sensors can average readings over roughly one second. This limits how confidently the traces can describe short spikes and fast ramps.
  • Most inference configurations are not deliberately repeated, and the power model misses some important fluctuations, especially under continuous high load.
Questions for authors
  1. What time resolution can users reliably study with this dataset? How do the sensor averaging windows differ across GPUs and measurement tools?

  2. How well does the model predict changes in power, beyond the average? The appendix notes that it cannot capture ramps when the server stays full.

  3. How should users connect GPU traces to datacenter or grid behavior? The dataset measures GPU power rather than total server or facility power. What additional measurements or assumptions are needed, and how far can the observed synchronization patterns be extended beyond one node?

7 comments
AuthorsOpenReview ↗

(W1, Q1) - Reliable Temporal Resolution and GPU Sensor Averaging Behavior

We thank the reviewer for the important question regarding the temporal resolution and sensor. To address these concerns, we validate instantaneous NVIDIA Management Library (NVML) telemetry against 10-kHz oscilloscope measurements and characterize sensor behavior on A100/H100/H200, demonstrating reliable characterization of major power fluctuations lasting approximately 0.2 s or longer, while identifying limitations in capturing short-duration peaks and ramp rates.

(W1, Q1) - Validation of GPU power telemetry against oscilloscope measurements

To assess the accuracy and temporal resolution of GPU power telemetry, we conducted additional experiments comparing instantaneous software power measurements against high-frequency oscilloscope measurements (Fig. 43 of the revised manuscript). Our results show that instantaneous NVML telemetry reliably characterizes major power fluctuations lasting approximately 0.2 s or longer. We supplemented details in Appendix F of the revised manuscript. The key setup and results are summarized below.

1. Experimental setup. We tested an RTX 5090 GPU (600-W power limit) serving Qwen3-8B with vLLM 0.28.0 under fixed-concurrency workloads of 64 requests, each lasting 120 s. We simultaneously recorded:

  • Software telemetry: NVIDIA NVML power.draw.instant, polled at 50 Hz.
  • Hardware measurements: An in-lab oscilloscope current clamp on the GPU's 12V-2x6 power cable, sampled at 10 kHz.

2. Key findings. Our results demonstrate that instantaneous software telemetry accurately captures major power fluctuations at timescales of approximately 200 ms and above.

  • Frequency-domain agreement (Fig. 43(b) of the revised manuscript): Software readings refresh approximately every 100 ms, corresponding to an effective sampling rate of 10 Hz and a Nyquist frequency of 5 Hz. The software and oscilloscope measurements exhibit strong agreement at 0.05–2 Hz. Agreement decreases at 2–5 Hz, while frequencies above 5 Hz cannot be reliably resolved by software telemetry.
  • Power transitions and ramp rates: For 64 concurrent requests, where prefill plateaus last approximately 1.2 s, the software measurements capture step magnitudes within 8–10% of the oscilloscope reference. Peak ramp rates computed from 0.2-s mean power have a median relative error of approximately 2% across the 24 tested prefill edges. For 0.1-s ramps, software telemetry underestimates peak values by 10–20%. Faster phenomena, including individual decode transitions, ~30-ms load-on bursts, and high-frequency fluctuations, are only visible in oscilloscope measurements.

3. Relevance to power-system applications. These findings help clarify which applications can be supported by the dataset's temporal resolution.

  • Slower power dynamics [1–2]: Electromechanical oscillations (~0.1–2 Hz) and typical AI-training iteration oscillations (~0.1–1 Hz) fall largely within the frequency range accurately captured by software telemetry [1].
  • Power ramp characterization [1–2]: Several grid operators specify ramp constraints over windows of seconds. For example, PNNL-39191 in [1] summarizes an ERCOT limit of 10 MW within a 5-s sliding window. Our measurements support the use of software telemetry for characterizing workload-induced variations at these timescales.

4. Generalizability and limitations. Overall, these results support the use of instantaneous GPU telemetry for characterizing workload-induced power fluctuations over timescales of approximately 0.2 s and longer, while establishing clear limitations for sub-100-ms dynamics.

We used an RTX 5090 GPU because it was the only GPU available to us with an accessible dedicated power cable that allowed direct hardware-level power measurement. Although this experiment does not directly validate the transient measurement accuracy of the A100/H100/H200 GPUs, it provides independent hardware validation of the same NVML instantaneous power field and vLLM serving framework used in our experiments. The sensor characterization experiment in the next post (i.e., (W1, Q1) - Sensor behavior on A100/H100/H200) will further confirm that this field has a similar ~100-ms refresh interval on A100, H100, and H200.

References

[1] K. Chatterjee, J. Follum, A. Varghese, S. Biswas, E. Farantatos, and L. Zhu, "Measurement Adequacy for Monitoring Data Center Oscillations," Pacific Northwest National Laboratory (PNNL), Technical Report PNNL-39191, April 2026. Link.

[2] IEEE Standards Association, "Review of Industry Efforts and Standards of Grid Readiness for Data Center Deployment," IEEE SA White Paper, January 2026. Link.

AuthorsOpenReview ↗

(W1, Q1) - Reliable Temporal Resolution and GPU Sensor Averaging Behavior

(W1, Q1) - Sensor averaging windows differ across GPUs and measurement tools

Sensor experiment. We distinguish two NVIDIA Management Library (NVML) power measurement fields: power.draw, the standard reported GPU power reading, which may incorporate temporal averaging, and power.draw.instant, which reports power without the longer averaging window applied to the standard field. We have supplemented these results in Appendix G of the revised manuscript.

1. Experimental setup. We tested three GPUs (H100, H200, and A100) using a controlled matrix-multiplication workload with known ground-truth power transitions, including full-load steps and isolated 200-ms bursts.

We sampled power through NVML at approximately 6-ms intervals. By sampling substantially faster than the sensor update rate, we identified how frequently the reported values actually change.

2. Key findings: Sensor update rate versus effective resolution. The experiments reveal that although power readings update approximately every 100 ms on all three GPUs, their effective temporal resolution differs substantially due to sensor averaging (Table 1).

  • H100 and H200 (power.draw): The averaged power readings (power.draw) used in the original manuscript apply a moving average over approximately 1 s, smoothing short-duration fluctuations. Consequently, a 200-ms burst appears as a ~1-s bump at roughly one-quarter of its actual height.
  • H100 and H200 (power.draw.instant): We have updated our experiments to use instantaneous power readings(power.draw.instant), which do not apply the 1-s averaging window. These measurements update approximately every 100 ms and capture the full magnitude of the 200-ms power bursts, providing an effective temporal resolution of approximately 0.2 s. This substantially improves our ability to characterize short-duration power fluctuations and ramp events.
  • A100: The average reading is effectively instantaneous. A 200-ms burst retains approximately 91% of its actual height, indicating a much shorter effective averaging window (at most ~0.25 s).
  • Energy measurement accuracy: Integrating the recorded power measurements agrees with the GPU's built-in energy counter within 0.11% across all traces used in the paper.

Table 1. How often each GPU power reading updates, how much it smooths, and the shortest event it resolves, measured with a load of known on/off times.

GPUPower readingUpdates everySmoothing (effective bandwidth)A 200-ms burst appears asShortest event resolved with accurate magnitude and timing
H100, H200Instantaneous reading (updated experiments, power.draw.instant)~100 msSnapshot of a short window (~25 ms every 100 ms)Full height~0.2 s
H100, H200Averaged reading (power.draw)~100 msAverage over the past 1 sA ~1-s bump at about 1/4 of its height~1 s
A100Instantaneous reading~100 msSnapshot of a short window (~25 ms every 100 ms)91% of its height~0.2 s

3. Sensitivity analysis: Impact on reported metrics and conclusions. To quantify the effect of temporal resolution, we recomputed the power metrics after averaging the traces over 1-s and 10-s windows, compared with the 100-ms resolution (Table 2).

  • Mean and P95 power: Mean power remains largely unchanged (approximately 1% or less), while P95 power shows modest reductions with coarser averaging.
  • Peak power and ramp rates: Are sensitive to temporal resolution and whether averaged or instantaneous readings are used.

Table 2. How the main metrics change when the instantaneous reading is averaged to 1 s and 10 s (medians over runs, relative to 100 ms).

Metric (instantaneous reading, from 569 additional H100 runs)100 ms1 s10 s
Mean power1.001.001.00
95th-percentile power1.000.980.93
Peak power1.000.850.76
99th-percentile ramp rate1.000.520.14
Spectral entropy (rank correlation with 100 ms)1.000.860.53
AuthorsOpenReview ↗

(Q2) - Model Accuracy in Predicting Power Fluctuations and Ramp Rates

(Q2) - Model Accuracy in Predicting Power Fluctuations and Ramp Rates

We thank the reviewer for the important question regarding whether our power scaling law can predict temporal power fluctuations and ramp rates beyond average power To address this concern, we extended our evaluation beyond mean power to include P95 power, P99 1-s ramp rates, and temporal correlation between predicted and measured 1-s power trajectories within each run. We applied the original fitted power law, without refitting, to predict power trajectories using the number of requests in flight at 1-s resolution, reconstructed from per-request timestamps (Table 3).

We consider two serving modes: stochastic-arrival workloads, where requests follow externally specified arrival patterns (Poisson or BurstGPT), and fixed-concurrency workloads, where the client maintains a fixed number of outstanding requests by replacing completed requests.

1. Key findings. Our results show that the model accurately predicts power levels and captures fluctuations driven by changes in request concurrency, but cannot fully capture internal GPU execution dynamics.

  • Stochastic-arrival workloads: On 42 H200 runs with varying request arrivals, the model achieves a median mean-power error of 2.5% and P95 error of 2.4%. It also captures temporal power patterns, with a correlation of 0.83 and a P99 ramp error of 31%.
  • Fixed GPU clock frequencies: On 21 H100 runs with locked clocks (345–1605 MHz), the model maintains a temporal correlation of 0.81 but has a P95 power error of 16%. This occurs because request concurrency extends beyond the range used to fit the clock-dependent terms.
  • Fixed-concurrency workloads: The model accurately predicts mean and P95 power (errors of 2.8% and 3.9%), but fails to capture fluctuations caused by batch turnover and internal scheduling, with a temporal correlation of only 0.08 and a P99 ramp error of 98%. This limitation arises because the request count remains nearly constant despite changes in GPU execution activity.

Table 3. Accuracy of the power scaling law beyond mean power. Errors are reported as median absolute percentage errors across runs. Correlation measures the temporal correlation between predicted and measured 1-s power trajectories within each run.

Regime (1-s load)RunsMean power errorP95 power errorP99 ramp errorCorrelation
Stochastic-arrival workloads, H200422.5%2.4%31%0.83
Stochastic-arrival workloads, H100, fixed GPU clock frequencies215.4%16.0%99%0.81
Fixed-concurrency workloads (always full)1182.8%3.9%98%0.08

2. Implications and limitations. These results clarify that our power law is effective for predicting power levels and workload-driven fluctuations, particularly under varying request arrivals, but cannot reliably predict power ramps caused by internal execution dynamics when concurrency remains constant. Modeling such fluctuations requires additional information about GPU execution phases and LLM serving scheduler states.

Importantly, this predictive model represents one illustrative use case of our dataset, rather than its primary contribution. Our broader goal is to provide the research community with comprehensive, time-resolved GPU power measurements to support the development and evaluation of more sophisticated power prediction models. To facilitate this, our additional H100 experiments include per-step scheduler information, which we plan to release to support modeling of execution-dependent power dynamics.

We will include these expanded evaluations and clarify the model's applicable operating conditions and limitations in the revised manuscript.

Reviewer_GCg5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces LLM-PowerTrace, a dataset comprising 1,788 fine-tuning runs and 401 complete inference-serving runs, with per-GPU power telemetry sampled every 100 ms, workload configurations, and execution metadata. It characterizes power levels, ramps, spectral regularity, and inter-GPU synchronization. It drives applications through fine-tuning energy scaling, an empirical inference power model, and calibration of a disaggregated-serving model. The contribution is a potentially useful resource connecting controlled LLM configurations to power trajectories.

Critical strengths
  1. Interesting and potentially useful dataset contribution. The distinction between total energy consumption and temporal power behavior is well motivated. Configuration-labeled traces could support research on workload characterization, power management, and forecasting.

  2. The collection spans multiple GPU platforms, model families, fine-tuning methods, and serving configurations.

  3. The paper illustrates how power saturation, batching, checkpointing, and request arrivals affect energy and power differently. These observations help explain why energy efficiency alone does not characterize a workload's electrical behavior.

Critical weaknesses

There is no doubt this dataset could become a valuable community resource. IMHO, this fact per se is more than enough to justify a proper ICLR acceptance. My reservations concern whether the measurements adequately support the paper's central claims. Below a list of weaknesses

  1. Polling every 100 ms does not establish that the underlying power measurements resolve changes at that timescale. Section 4.2 states that approximately 0.2-second prefill events appear to be spread over roughly one second due to telemetry averaging. These effects directly influence peak power, ramp rates, spectral entropy, and synchronization. The paper should characterize the effective update rate and averaging behavior for each platform and measurement field and show the sensitivity of the principal metrics to temporal aggregation. Comparison with an independent power measurement on a representative subset would substantially strengthen confidence. An agreement between the integrated power and the available cumulative energy counters would also provide a useful consistency check. Overall, although the sensor caveats are acknowledged, their consequences for the reported conclusions are not quantified.

  2. For a dataset contribution, the authors should provide a clearer accounting of the released resource: run counts by configuration, trace durations, coverage gaps, failed or incomplete runs, missing telemetry, event-log availability, schema, license, and fixed evaluation splits. I suggest adding this information to give the reader a clearer idea of the resource released.

Questions for authors

What is the effective sensor bandwidth on each platform, and which conclusions remain stable under alternative temporal aggregation?

3 comments
AuthorsOpenReview ↗

(W1, Q1) - Validation of GPU power telemetry against oscilloscope measurements

We thank the reviewer for the insightful questions and constructive comments regarding GPU sensor update frequency, temporal averaging, and measurement fidelity. To address these concerns, we validate instantaneous NVIDIA Management Library (NVML) telemetry against 10-kHz oscilloscope measurements and characterize sensor behavior on A100/H100/H200, demonstrating reliable characterization of major power fluctuations lasting approximately 0.2 s or longer, while identifying limitations in capturing short-duration peaks and ramp rates.

(W1, Q1) - Validation of GPU power telemetry against oscilloscope measurements

To assess the accuracy and temporal resolution of GPU power telemetry, we conducted additional experiments comparing instantaneous software power measurements against high-frequency oscilloscope measurements (Fig. 43 of the revised manuscript). Our results show that instantaneous NVML telemetry reliably characterizes major power fluctuations lasting approximately 0.2 s or longer. We supplemented details in Appendix F of the revised manuscript. The key setup and results are summarized below.

1. Experimental setup. We tested an RTX 5090 GPU (600-W power limit) serving Qwen3-8B with vLLM 0.28.0 under fixed-concurrency workloads of 64 requests, each lasting 120 s. We simultaneously recorded:

  • Software telemetry: NVIDIA NVML power.draw.instant, polled at 50 Hz.
  • Hardware measurements: An in-lab oscilloscope current clamp on the GPU's 12V-2x6 power cable, sampled at 10 kHz.

2. Key findings. Our results demonstrate that instantaneous software telemetry accurately captures major power fluctuations at timescales of approximately 200 ms and above.

  • Frequency-domain agreement (Fig. 43(b) of the revised manuscript): Software readings refresh approximately every 100 ms, corresponding to an effective sampling rate of 10 Hz and a Nyquist frequency of 5 Hz. The software and oscilloscope measurements exhibit strong agreement at 0.05–2 Hz. Agreement decreases at 2–5 Hz, while frequencies above 5 Hz cannot be reliably resolved by software telemetry.
  • Power transitions and ramp rates: For 64 concurrent requests, where prefill plateaus last approximately 1.2 s, the software measurements capture step magnitudes within 8–10% of the oscilloscope reference. Peak ramp rates computed from 0.2-s mean power have a median relative error of approximately 2% across the 24 tested prefill edges. For 0.1-s ramps, software telemetry underestimates peak values by 10–20%. Faster phenomena, including individual decode transitions, ~30-ms load-on bursts, and high-frequency fluctuations, are only visible in oscilloscope measurements.

3. Relevance to power-system applications. These findings help clarify which applications can be supported by the dataset's temporal resolution.

  • Slower power dynamics [1–2]: Electromechanical oscillations (~0.1–2 Hz) and typical AI-training iteration oscillations (~0.1–1 Hz) fall largely within the frequency range accurately captured by software telemetry [1].
  • Power ramp characterization [1–2]: Several grid operators specify ramp constraints over windows of seconds. For example, PNNL-39191 in [1] summarizes an ERCOT limit of 10 MW within a 5-s sliding window. Our measurements support the use of software telemetry for characterizing workload-induced variations at these timescales.

4. Generalizability and limitations. Overall, these results support the use of instantaneous GPU telemetry for characterizing workload-induced power fluctuations over timescales of approximately 0.2 s and longer, while establishing clear limitations for sub-100-ms dynamics.

We used an RTX 5090 GPU because it was the only GPU available to us with an accessible dedicated power cable that allowed direct hardware-level power measurement. Although this experiment does not directly validate the transient measurement accuracy of the A100/H100/H200 GPUs, it provides independent hardware validation of the same NVML instantaneous power field and vLLM serving framework used in our experiments. The sensor characterization experiment in the next post (i.e., (W1, Q1) - Sensor behavior on A100/H100/H200) will further confirm that this field has a similar ~100-ms refresh interval on A100, H100, and H200.

References

[1] K. Chatterjee, J. Follum, A. Varghese, S. Biswas, E. Farantatos, and L. Zhu, "Measurement Adequacy for Monitoring Data Center Oscillations," Pacific Northwest National Laboratory (PNNL), Technical Report PNNL-39191, April 2026. Link.

[2] IEEE Standards Association, "Review of Industry Efforts and Standards of Grid Readiness for Data Center Deployment," IEEE SA White Paper, January 2026. Link.

AuthorsOpenReview ↗

(W1, Q1) - Sensor behavior on A100/H100/H20

(W1, Q1) - Sensor behavior on A100/H100/H200

Sensor experiment. In the original submission, we used the standard reported GPU power reading (power.draw) from NVML, which applies a moving average over approximately 1 s. Consequently, a 200-ms burst appears as a ~1-s bump at roughly one-quarter of its actual height. To address this limitation, we have updated our experiments to use instantaneous power measurements (power.draw.instant) and conducted additional sensor characterization experiments to quantify the differences between averaged and instantaneous readings. We have supplemented these results in Appendix G of the revised manuscript.

1. Experimental setup. We tested three GPUs (H100, H200, and A100) using a controlled matrix-multiplication workload with known ground-truth power transitions, including full-load steps and isolated 200-ms bursts.

We sampled power through NVML at approximately 6-ms intervals. By sampling substantially faster than the sensor update rate, we identified how frequently the reported values actually change.

2. Key findings: Sensor update rate versus effective resolution. The experiments reveal that although power readings update approximately every 100 ms on all three GPUs, their effective temporal resolution differs substantially due to sensor averaging (Table 1).

  • H100 and H200 (power.draw): The averaged power readings (power.draw) used in the original manuscript apply a moving average over approximately 1 s, smoothing short-duration fluctuations. Consequently, a 200-ms burst appears as a ~1-s bump at roughly one-quarter of its actual height.
  • H100 and H200 (power.draw.instant): We have updated our experiments to use instantaneous power readings(power.draw.instant), which do not apply the 1-s averaging window. These measurements update approximately every 100 ms and capture the full magnitude of the 200-ms power bursts, providing an effective temporal resolution of approximately 0.2 s. This substantially improves our ability to characterize short-duration power fluctuations and ramp events.
  • A100: The average reading is effectively instantaneous. A 200-ms burst retains approximately 91% of its actual height, indicating a much shorter effective averaging window (at most ~0.25 s).
  • Energy measurement accuracy: Integrating the recorded power measurements agrees with the GPU's built-in energy counter within 0.11% across all traces used in the paper.

Table 1. How often each GPU power reading updates, how much it smooths, and the shortest event it resolves, measured with a load of known on/off times.

GPUPower readingUpdates everySmoothing (effective bandwidth)A 200-ms burst appears asShortest event resolved with accurate magnitude and timing
H100, H200Instantaneous reading (updated experiments, power.draw.instant)~100 msSnapshot of a short window (~25 ms every 100 ms)Full height~0.2 s
H100, H200Averaged reading (power.draw)~100 msAverage over the past 1 sA ~1-s bump at about 1/4 of its height~1 s
A100Instantaneous reading~100 msSnapshot of a short window (~25 ms every 100 ms)91% of its height~0.2 s

3. Sensitivity analysis: Impact on reported metrics and conclusions. To quantify the effect of temporal resolution, we recomputed the power metrics after averaging the traces over 1-s and 10-s windows, compared with the 100-ms resolution (Table 2).

  • Mean and P95 power: Mean power remains largely unchanged (approximately 1% or less), while P95 power shows modest reductions with coarser averaging.
  • Peak power and ramp rates: Are sensitive to temporal resolution and whether averaged or instantaneous readings are used.

Main claims remain consistent across both types of readings. Specifically, (i) ramp rates peak at intermediate workloads, and (ii) GPUs executing the same model in parallel exhibit synchronized power ramps. One important qualification concerns the cross-generation comparison: the claim that newer GPUs exhibit smoother power dynamics does not hold when A100 measurements are averaged to the same temporal resolution as H100 and H200. We will revise this claim accordingly.

Table 2. How the main metrics change when the instantaneous reading is averaged to 1 s and 10 s (medians over runs, relative to 100 ms).

Metric (instantaneous reading, from 569 additional H100 runs)100 ms1 s10 s
Mean power1.001.001.00
95th-percentile power1.000.980.93
Peak power1.000.850.76
99th-percentile ramp rate1.000.520.14
Spectral entropy (rank correlation with 100 ms)1.000.860.53
AuthorsOpenReview ↗

(W2) Dataset inventory, completeness, and availability

(W2) Dataset inventory, completeness, and availability

We thank the reviewer for requesting a more comprehensive description of the dataset. We have added a complete dataset inventory in Appendix E, including recordings that failed or were added after submission. Importantly, failed runs are retained in the dataset but excluded from the paper's successful-training statistics. These records provide useful information about hardware memory limits, software compatibility, and configurations that cannot be successfully executed. A summary of the inventory is given below.

1. Dataset coverage

  • Fine-tuning dataset. The complete inventory contains 4,583 distinct recordings, covering:

    • GPU types: A100 (1,390), H100 (1,637), H200 (805), L40S (735), RTX8000 (14), and V100 (2).
    • Fine-tuning methods: Full fine-tuning (1,130), LoRA (1,743), QLoRA (1,709), and unspecified (1).
    • Recording durations: Median 1,800.63 s, with the middle 50% ranging from 1,800.25 to 1,802.09 s. There are 2,291 verified planned 30-minute captures. Completion of the measurement window is tracked separately from completion of training.
  • Inference dataset. The inventory contains 974 serving runs, consisting of 405 original runs and 569 additional H100 runs with instantaneous power measurements. The two collections contain no overlapping run IDs or source folders.

    • GPU types: A100 (86), H100 (709), and H200 (179).
    • Serving configurations: The main experiments use vLLM with tensor parallelism across 1, 2, or 4 GPUs. The original collection covers BF16, FP8, and INT4 weights; the additional H100 collection uses BF16 weights and BF16 KV cache.
    • Workload coverage: The additional H100 collection includes 389 fixed-concurrency runs, 100 recorded-traffic replays, 56 stepped-load ramp runs, and 24 Poisson-arrival runs, covering Qwen3-8B, 14B, 32B, and 30B-A3B.
    • Recording durations: The 401 original runs with defined measurement windows have a median duration of 1,201 s (range 600.5–5,810.7 s). The additional 569 runs have a median duration of 600 s (range 420–1,200 s).

2. Failed and incomplete runs

  • Fine-tuning failures. We identified 897 recordings with failure evidence, including:

    • Out-of-memory (OOM) errors: 538.
    • Software or configuration errors: 286.
    • Invalid training-loss values: 33.
    • Memory-placement errors: 6.
    • Other failures: 34.
  • Inference completion and quality. The original 405-run collection contains 381 completed runs without quality flags, 5 completed runs with flags, 15 runs failing clock/power checks, and 4 incomplete runs. The additional 569-run collection contains 472 completed runs without flags and 97 completed runs with flags, with no final failed or incomplete entries.

3. Data completeness and limitations

  • All 4,583 fine-tuning and 974 inference power traces are available, with missing metadata, event logs, and sampling anomalies explicitly documented. Most runs include supporting logs, although some fine-tuning logs lack verified timestamp alignment and some original inference runs lack idle-power baselines.
  • Experimental coverage varies across GPU–model combinations. Not all workloads are tested on every GPU, and inference experiments are primarily limited to single-node configurations with up to four GPUs. These limitations are documented in the dataset inventory.

4. Data schema and preprocessing. We have documented the dataset schema and preprocessing procedures in Appendix E.

5. Dataset availability and evaluation splits

Availability and licensing. We will release the GPU power traces and associated run metadata under CC BY 4.0, and the accompanying profiling and analysis code under Apache 2.0.

Evaluation splits. We additionally document the evaluation protocols and recommended data splits to support reproducibility.

Overall, the expanded inventory provides a transparent account of the dataset's coverage, failed experiments, measurement completeness, and evaluation protocols. By retaining failed and incomplete cases with explicit quality labels, we aim to make the dataset useful not only for power modeling but also for understanding hardware limitations and supporting reproducible research.

Reviewer_TXPNOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces LLM-PowerTrace, a dataset of time-resolved GPU power measurements for LLM fine-tuning and inference. It contains 1,788 fine-tuning runs and 401 serving runs, with nominal 100-ms per-GPU power traces, workload configurations, and execution metadata. The dataset covers multiple model families, GPU platforms, fine-tuning methods, and serving configurations through controlled experiments. The authors characterize power dynamics using metrics such as energy consumption, ramp rate, spectral entropy, and inter-GPU synchronization. They also develop FLOPs-based and empirical scaling laws for predicting training energy and inference power, and demonstrate model calibration for disaggregated serving.

Critical strengths
  1. The dataset provides a useful and relatively comprehensive measurement resource. The authors cover a broad range of LLM architectures, GPU platforms, and execution configurations, with controlled parameter sweeps and per-GPU power traces. The combination of time-resolved measurements, workload metadata, and execution information distinguishes it from existing energy benchmarks that mainly report aggregate consumption. The dataset could support further research on power modeling, workload characterization, and power-aware system design.
  2. The empirical characterization is systematic and provides useful insights into LLM power behavior. The paper examines how workload characteristics, execution configurations, and hardware affect power levels, temporal patterns, and ramp rates. The observations on workload saturation, request-arrival patterns, and multi-GPU synchronization are particularly informative.
Critical weaknesses
  1. My main concern is that the practical necessity of high-resolution, per-GPU power traces is not sufficiently established. The paper motivates the dataset primarily through the impact of AI workloads on data center infrastructure and power grids. However, power fluctuations observed at the individual GPU level may be substantially reduced when aggregated across many independent workloads. Their persistence at the data center level depends on cross-GPU and cross-workload correlations. Although the paper examines synchronization within small multi-GPU configurations, it does not establish how these effects scale to larger deployments. Furthermore, it remains unclear what additional value the nominal 100-ms resolution provides over coarser measurements for the intended applications, especially given the temporal averaging of onboard power sensors. A multi-scale aggregation or temporal-resolution analysis would strengthen the connection between the dataset design and its practical motivation.

  2. Some conclusions about power fluctuations are stronger than the experimental evidence supports. In particular, the paper highlights an approximately 21× increase in the P99 1-s ramp rate between steady and bursty request arrivals. However, these workloads differ in both arrival variability and average request rate (5.1 versus 1.6 requests/s). Consequently, the comparison does not isolate the contribution of burstiness from the effect of lower utilization and more frequent idle–busy transitions. The authors are encouraged to clarify the causal interpretation of this result or provide additional controlled comparisons at matched average arrival rates and utilization levels to isolate the effect of burstiness.

  3. The inference scaling law is primarily calibrated against mean power consumption, while its ability to reproduce temporal fluctuations is less thoroughly established. In particular, Appendix B.1 acknowledges systematic errors in ramp-rate estimation and the inability to capture batch-turnover fluctuations under continuously loaded servers. Since modeling fast-changing power dynamics is a central motivation of the paper, the authors are encouraged to provide a more comprehensive evaluation of time-series prediction accuracy, including ramp rates, peak power, and temporal patterns, rather than relying primarily on mean-power prediction errors.

Questions for authors
  1. Could the authors clarify which practical applications specifically require 100-ms per-GPU power measurements rather than lower-resolution telemetry? For example, how would the reported power characteristics change after downsampling to 1-s or 10-s intervals? It would also be useful to examine how power variability and ramp rates change when aggregating multiple independent workloads, since this determines whether fine-grained GPU fluctuations remain significant at the data center scale.
  2. In the steady-versus-bursty arrival comparison, the average request rate and arrival variability both change substantially. Could the authors provide experiments that vary burstiness while controlling the average request rate? Additionally, could they report ramp-rate distributions or exceedance probabilities at multiple thresholds, rather than relying primarily on P99? This would help distinguish changes in the frequency of large power transitions from changes in their magnitude.
  3. What downstream tasks do the authors envision for the proposed scaling laws and released traces? In particular, the inference model uses the observed number of active requests, which would not be directly available for future power prediction. Could the authors evaluate a practical forecasting setting using only historical workload information, or demonstrate how the calibrated model improves a concrete power-aware scheduling or provisioning decision?
7 comments
AuthorsOpenReview ↗

(Q1) - Practical applications requiring 100-ms per-GPU power measurements

(Q1) - Practical applications requiring 100-ms per-GPU power measurements

We thank the reviewer for the important question regarding practical applications. We agree that not all applications require such fine-grained measurements. For example, lower-resolution telemetry generally suffices for energy accounting and long-term power-grid capacity planning. However, fine-grained per-GPU measurements are particularly valuable for characterizing short-duration power fluctuations and their impact on power and data center infrastructure. We highlight three practical applications:

  1. Transient power provisioning: Rapid GPU power fluctuations affect transient power headroom and power-buffer requirements. Our sensor experiments show that 1-s averaging can reduce the observed magnitude of a 200-ms power burst to approximately 25% of its actual value, potentially obscuring important power excursions.

  2. Power-aware scheduling and control: Fine-grained measurements help characterize workload-induced power fluctuations and support the design of scheduling and power-capping policies that mitigate rapid power changes.

  3. Modeling aggregate power dynamics: Single-GPU power traces are the building blocks for modeling server-, rack-, and fleet-level power behavior. Per-GPU measurements are particularly useful for identifying correlations and synchronized power ramps across GPUs. Moreover, individual power-electronic devices and their power-delivery components still require characterization under specific workload-induced power profiles.

These applications motivate our work: developing and characterizing a dataset of fine-grained GPU power measurements to support the modeling and analysis of diverse AI workloads and their impacts on power grids. We emphasize that 100-ms measurements are not universally required; their importance depends on the timescale of the application. We will clarify these motivations and measurement-resolution considerations in the revised manuscript.

AuthorsOpenReview ↗

(Q1) - Effect of Temporal Resolution and Practical Use

(Q1) - Effect of Temporal Resolution and Practical Use

A fine-grained trace can always be averaged to 1 s or 10 s, but a coarse trace cannot be refined, so 100-ms traces let each dataset user choose the timescale their application needs. To assess the impact of temporal resolution, we compared GPU power traces at 100-ms, 1-s, and 10-s resolutions for both fine-tuning (Table 1) and inference (Table 2), including both the original averaged readings and additional instantaneous measurements.

1. Impact of temporal aggregation. Our results show that coarser resolution substantially reduces observed power peaks and fluctuations, while mean power remains largely unchanged.

  • Fine-tuning: Across all GPU types, averaging to 10 s reduces peak power by approximately 31–44%. For example, the H200 peak decreases from 1,803 W to 1,007 W, while its P99 10-s ramp decreases from 80.8 to 51.0 W/s. Updated Fig. 23 also illustrates how temporal aggregation affects an H100 fine-tuning power trajectory. At 100-ms resolution, rapid fluctuations and short-duration peaks are clearly visible. At 1 s, major workload transitions remain identifiable, but short-duration spikes are smoothed out. At 10 s, most rapid fluctuations disappear, leaving primarily the overall power trend.
  • Inference: In the additional H100 experiments using instantaneous measurements, peak power decreases from 561 W to 501 W and 423 W at 1-s and 10-s resolutions, respectively. The P99 10-s ramp decreases from 16.8 to 7.6 and 1.8 W/s, corresponding to reductions of 55% and 89%.
  • Sensor averaging: The original H100 and H200 inference traces already incorporate approximately 1-s smoothing, explaining their smaller changes under further aggregation. We will clarify this distinction in the revised manuscript.

2. Practical implications. These results demonstrate that fine-grained power measurements are important for capturing short-duration peaks and rapid fluctuations, which are relevant to transient power headroom, power converter operating limits, energy storage sizing, and grid ramp constraints. Moreover, per-GPU traces provide the building blocks for modeling aggregate power dynamics across servers and racks. While lower-resolution telemetry is sufficient for average power and energy analysis, fine-grained measurements can be aggregated for slower-timescale applications, supporting our dataset's utility across multiple temporal and spatial scales.

Table 1. Temporal averaging of power summed across 4 GPUs during fine-tuning. Entries are medians across runs. Mean power is shown once because it changes little with averaging. Std. denotes standard deviation; ramp denotes the 99th-percentile 10-s ramp. Each cell reads 100 ms → 1 s → 10 s.

GPU (runs)Mean (W)Peak (W)Std. (W)Ramp (W/s)
A100 (104)5381085 → 765 → 665128 → 113 → 10139.8 → 32.2 → 26.2
H100 (103)6301656 → 1048 → 930226 → 200 → 16273.8 → 54.6 → 44.9
H200 (116)6491803 → 1151 → 1007261 → 228 → 18980.8 → 61.7 → 51.0
L40S (57)612939 → 700 → 64977 → 58 → 4825.2 → 20.6 → 16.7

Table 2. Single-GPU inference metrics at three averaging windows. Each entry is a median across runs. Peak denotes the observed maximum power within a run. Each cell reads 100 ms → 1 s → 10 s.

GPU, power field (runs)Mean (W)Peak (W)Std. (W)P99 10-s ramp (W/s)
A100, standard = instantaneous (78)352447 → 389 → 35910.5 → 5.2 → 2.75.4 → 1.8 → 0.5
H100, instantaneous, new runs (457)367561 → 501 → 42337.2 → 20.1 → 5.716.8 → 7.6 → 1.8
AuthorsOpenReview ↗

(Q1) - Aggregation Across Workloads

(Q1) - Aggregation Across Workloads

AI training workloads are typically highly synchronized across GPUs, and their coordinated computation and communication phases are known to produce substantial power fluctuations. When these fluctuations are synchronized, their aggregate magnitude can scale approximately linearly with the number of participating GPUs and the associated computing capacity. We therefore focus our aggregation analysis on inference workloads, where request arrivals and execution patterns can be more heterogeneous, making the scaling of power fluctuations less straightforward.

To investigate this effect, we constructed synthetic fleets of N=1N=1 to N=64N=64 GPUs using the paper's single-GPU inference power traces (Table 26 and Fig. 38 in the revised manuscript). For each workload pool and fleet size, we randomly sampled NN complete 600-s traces with replacement, repeated the experiment 300 times, and reported the median results. We considered both independent and synchronized workload arrivals.

1. Key findings. Our results show that the scaling of power fluctuations strongly depends on workload correlations.

  • Independent workloads: With randomized start times, per-GPU fluctuations decrease approximately as 1/N1/\sqrt{N} due to statistical multiplexing. At N=64N=64, the P99 ramp per GPU decreases to 0.11–0.16 of the single-GPU value for single-workload pools, consistent with the theoretical 1/64=0.1251/\sqrt{64}=0.125 scaling.
  • Synchronized workloads: When GPUs replay the same BurstGPT traffic with aligned arrival times, bursty workloads retain their per-GPU ramp magnitude, resulting in an almost 64-fold increase in aggregate ramp magnitude at N=64N=64.
  • Mixed workloads: Aggregation provides less smoothing when a small number of highly bursty GPUs dominate the total fluctuations.

2. Implications. These findings demonstrate that aggregate power fluctuations depend not only on fleet size but also on workload characteristics and temporal correlations. Consequently, accurately modeling fleet-level power dynamics requires characterizing individual GPU workloads and their correlations.

Single-GPU power traces remain essential building blocks for understanding server-, rack-, and fleet-level power behavior. Moreover, individual power-electronic components must still be characterized against workload-induced power demands. Table 3 summarizes the aggregation behavior of 64-GPU workloads. This further motivates our dataset's emphasis on scalable, fine-grained GPU power measurements to support power modeling across different workloads and deployment scales. We have added this analysis and discussion to Appendices D.5 and G.4 of the revised manuscript.

Table 3. Synthetic aggregation of single-GPU inference traces using the standard power field. Each triplet gives the median for one GPU, 64 GPUs with random starts, and 64 GPUs with aligned starts, in that order. CVP denotes the coefficient of variation of power (standard deviation divided by mean power); ramp denotes the power ramp rate in W/s per GPU; peak/mean denotes the ratio of peak to mean power.

Workload poolCVPRampPeak/mean
Fixed-concurrency, mid load0.024 / 0.003 / 0.00846 / 5.8 / 7.01.07 / 1.01 / 1.02
Steady BurstGPT replay0.036 / 0.005 / 0.02636 / 5.6 / 161.08 / 1.01 / 1.04
Bursty BurstGPT replay0.510 / 0.067 / 0.420116 / 13 / 1191.81 / 1.20 / 1.89
50:50 bursty/fixed-concurrency0.048 / 0.020 / 0.09166 / 10 / 591.16 / 1.06 / 1.19
All workload types0.027 / 0.010 / 0.01537 / 8.2 / 10.81.08 / 1.03 / 1.03

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.