acceptodds
Withdrawn from ICLR 2027

Renewable-Aware Neural Network Training with Phase-Dependent Lyapunov Control

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

In neural network training on GPUs, a larger batch shortens an epoch but draws more power, and its effect on the loss changes as training progresses. When renewable supply varies, the best batch size therefore depends on the available power and the training phase. Since the batch size changes how long each training segment takes, we formulate renewable-aware training as a renewal control problem that minimizes the renewable shortfall per unit time subject to a duration-weighted error constraint. We enforce the error budget with a virtual queue and design a drift-plus-penalty ratio controller that penalizes its backlog with phase-dependent Lyapunov weights. We prove an bound on the shortfall-rate gap with an average virtual backlog and extend it to finite Markov states and recurrent training classes. To run the controller in GPU training, we score each candidate action with a predictor calibrated from shared-checkpoint measurements and update the queue and the phase with the measured loss. We evaluate the controller in 75 training runs with independent supply and a measured photovoltaic (PV) day. Under independent supply, the controller reaches a common quality target in 59.7% less time than batch 16 at a similar shortfall rate, and relative to batch 128 it reduces the shortfall rate by 72.5% with a 13.4% longer target time. With a noon start on the PV day, its shortfall rate is 32.7% lower than that of batch 128 at the same target time, and as the PV capacity grows, its DWAE falls from 0.040 to 0.026 while fixed schedules keep their error. Emphasizing the backlog during rapid descent shortens the mean target time by 7.0% relative to equal weights.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
renewable-aware training, energy-efficient deep learning, Lyapunov optimization, drift-plus-penalty, virtual queue, renewal systems, batch size scheduling, GPU energy, sustainable machine learning, online control

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Z4diOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies neural network training under varying renewable energy supply. It models training segments as renewal frames with action-dependent durations and uses a virtual queue to constrain error averaged over training time. A phase-dependent weight changes the queue penalty, and a calibrated predictor scores candidate batch sizes before measured feedback updates the state. The paper presents long-run guarantees and evaluates shortfall and target attainment using image classification training and a replayed photovoltaic trace.

Critical strengths
  • Jointly considering supply timing, variable action duration, and learning progress is a meaningful problem.
  • The evaluation executes closed-loop GPU training rather than reporting predicted trajectories alone. Shared-checkpoint calibration and measured feedback are sensible design choices.
  • The paper reports common-quality-target metrics, full-run metrics, and unfavorable midnight results.
  • Small-workload scope, time-compressed PV replay, slack constraints, and overlapping phase-weight outcomes are explicitly acknowledged.
Critical weaknesses
  • The central ratio guarantee has a counterexample. Equation 7 minimizes a separate conditional ratio after observing each state, whereas Theorem 4.1 compares a ratio of expected totals. Let independent states A/B each occur with probability one half. All errors are zero and sigma=1, keeping the queue zero with strict feasibility. Actions a/b have (shortfall,duration)=(0,1)/(1,100) in A and identical (100,1) outcomes in B. The stated rule selects a in A, yielding long-run rate 50. The fixed policy always selecting b yields rate 1. This satisfies bounded costs and positive bounded durations, and the nonvanishing gap contradicts the claimed O(1/V) result. The comparator need not even depend on the state. Global fractional optimization requires a shared ratio parameter in the additive objective; independently minimizing state-specific ratios is different. Appendix J does not justify that substitution.
  • The Markov proof has an additional distribution mismatch. Equation 89 substitutes stationary costs under another policy for a current-state one-step comparison. Uniform mixing of each fixed policy does not make an adaptive policy occupy the comparator's distribution or incorporate future state values. Policy-dependent dynamics generally require a suitable bias/occupancy or multistep argument, not an appended vanishing term.
  • The phase-dependent proof omits the switching term. Equation 37 freezes the current phase, but the actual Lyapunov increment includes (w_next-w_current)Q_next²/2. Such frozen-phase increments do not telescope along changing phases. Positive switches are possible under the stated detector, and their quadratic contribution is not bounded by a fixed U_max. Theorem 5.1 consequently lacks its central justification. The finite-state Markov description also omits the queue on which action selection depends.
  • Average predictor RMSE does not establish the uniform per-frame C-additive oracle accuracy assumed by the theory, particularly when errors are multiplied by backlog and phase weights.
  • All experimental error constraints are slack, on one small MNIST workload. Thus the constraint/backlog tradeoff is untested. Reported training time excludes validation/controller costs despite subsecond target times, and calibration cost also matters. Compressed PV replay supports a trace experiment, not a deployment over a physical day.
  • Frame-indexed independent supply is not the same external wall-clock trace across policies with different durations. Endpoint-loss weighting is also a defined proxy rather than the actual within-frame loss integral.
Questions for authors

Q1. How do Theorem 4.1 and Algorithm 1 address the two-state counterexample in W1, where the stated rule has rate 50 but fixed action b has rate 1? Which precise assumption excludes it, or must the controller instead use a shared global ratio parameter?

Q2. What justifies replacing a current-state one-step comparison with another policy's stationary costs in Appendix L, Equation 89? Where are differing occupancy distributions and future-state values handled under controlled transitions?

Q3. How does the telescoping argument account for (w_next-w_current)Q_next²/2? What bounds or assumptions control this term when phases can switch in both directions?

Q4. Since decisions depend on Q, why is the process containing only training state and phase a finite time-homogeneous Markov chain? Must the state or policy assumptions change?

Q5. How does average prediction RMSE imply the required per-frame C-additive accuracy when errors are multiplied by Q or w_phi Q? If it does not, which guarantees apply to the implemented controller?

Q6. Do the energy and target-time advantages persist with binding error budgets, a common wall-clock supply trace, and validation/controller/calibration costs included?

Reviewer_E8aXOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the neural network training scheduling problem under dynamically changing renewable energy supply conditions, and proposes a dynamic batch size scheduling method based on phase-dependent Lyapunov control. The authors model the training process as a renewal control problem with variable execution duration, constrain the training error through a virtual queue, and adjust the control weights according to different training phases, so as to achieve a dynamic trade-off between energy shortfall and training progress. In addition, the paper provides theoretical analysis for different state dynamics, and conducts experimental verification under a GPU training environment combined with random energy supply and real photovoltaic data.

Overall, the research problem of this paper has certain practical significance, the method design is relatively reasonable, and the theoretical analysis and experimental organization are also relatively complete, reflecting relatively solid research work. However, the following important limitations still exist:

  1. The impact of dynamic batch size on training quality has not been fully verified. Batch size not only affects training efficiency and energy consumption, but may also significantly affect the optimization stability and final convergence quality of the model, especially in tasks that are sensitive to batch size, such as reinforcement learning and contrastive learning. Although this paper introduces training error prediction and a duration-weighted average error constraint, these mechanisms cannot directly guarantee the final model performance. In addition, the error constraint in the experiments never becomes a tight constraint, so the effectiveness of the controller when training quality and energy optimization clearly conflict has not yet been fully verified.

  2. The experimental scale and task diversity are limited. The core scheduling experiments are only carried out on a small CNN and the MNIST dataset, and the BERT experiment is also only used for preliminary analysis of the impact of batch size on power and training time, lacking verification on larger-scale models and complex training tasks, which limits the persuasiveness of the method's generalization ability and practical application value.

  3. There is a difference between the experimental time scale and real energy scheduling scenarios. The authors map the second-level training process to a 12-hour photovoltaic supply trajectory. Although this can simulate the scheduling behavior under different energy supply conditions, it still cannot fully prove the actual benefit of the method in a real time-synchronized renewable energy environment.

Critical strengths

This paper addresses the matching problem between dynamically changing renewable energy supply and neural network training demand, and incorporates the impact of batch size on power consumption, execution time, and training progress into a unified optimization framework. Through renewal control modeling, it handles the variable execution duration corresponding to different training configurations, and introduces a phase-dependent Lyapunov control mechanism to dynamically adjust the weight of the error constraint according to the training phase. This design, which combines external energy supply with the model's internal training state, has certain innovativeness and provides a meaningful solution idea for energy-aware neural network training scheduling.

Then it proposes a dynamic scheduling algorithm, and also provides theoretical analysis for the cases of independent states, Markov states, and training entering a low-loss recurrent class, giving the performance trade-off between the optimality gap of energy shortfall and the virtual queue backlog. In addition, the authors conduct actual GPU closed-loop training, compare with various fixed and dynamic batch size scheduling strategies under different energy supply conditions, and analyze the impact of training phase weights and energy supply changes on the results. Although the experimental model and task scale are limited, these theoretical analyses and experimental results still provide preliminary support for the effectiveness of the proposed control framework.

Critical weaknesses

Batch size not only affects training efficiency and energy consumption, but may also significantly affect the model's optimization trajectory, convergence stability, and final performance, especially in tasks that are sensitive to batch size, such as reinforcement learning and contrastive learning. Although this paper considers training quality through short-term loss prediction and a duration-weighted average error (DWAE) constraint, these mechanisms cannot directly guarantee the long-term convergence performance of the model under dynamic scheduling. In addition, the error constraint in the experiments never becomes a tight constraint, so it has not yet been fully verified whether the proposed controller can effectively maintain the trade-off between energy optimization and training quality when there is a clear conflict between them. This problem limits the applicability and reliability of the method in complex training scenarios.

The core experiments of this paper are only based on a small CNN on the MNIST dataset, and the BERT experiment is mainly used to preliminarily demonstrate the impact of batch size on power and training time, lacking complete verification on large-scale neural networks and more complex training tasks. Considering that the research motivation of this paper mainly comes from the energy demand of large-scale AI training, the current experiments are not yet sufficient to fully support the effectiveness of the method in actual high-energy-consumption training scenarios. In addition, the authors map the second-level training process to a 12-hour photovoltaic supply trajectory, rather than conducting experiments under real time-synchronized energy supply conditions. Although this time-scale mapping facilitates the analysis of scheduling behavior, it may not accurately reflect the dynamic matching relationship between computational load and renewable energy supply in the actual training process. Therefore, the support of the existing experiments for the practical deployment value and generalization ability of the method is still limited.

Questions for authors
  1. In tasks such as reinforcement learning and contrastive learning, batch size often has an important impact on the optimization process, training stability, and final model performance. This paper mainly verifies the proposed method based on supervised classification tasks. Can the authors further explain whether the current framework based on short-term loss prediction and dynamic batch size scheduling is still applicable when batch size has a significant impact on model convergence behavior? In particular, how can adverse effects of dynamically adjusting batch size on long-term training stability and final model quality be avoided?

  2. This paper constrains training quality through a virtual queue and Duration-Weighted Average Error (DWAE), but the error budget in the existing experiments always has a margin. Can the authors provide experimental results under stricter error constraints to verify whether the controller can effectively balance the two objectives when there is a clear conflict between reducing the energy shortfall and maintaining training quality? In addition, to what extent can the DWAE constraint reflect or guarantee the final convergence quality of the model?

  3. The core scheduling experiments of this paper are mainly based on a small CNN on the MNIST dataset, while the BERT experiment is only used to demonstrate the impact of batch size on training power and execution time. Considering that the research motivation of this paper mainly comes from the energy demand of large-scale neural network training, can the authors explain why the complete control algorithm was not verified on larger-scale models or more complex datasets? Are there additional experimental results that can demonstrate the effectiveness of this method in actual training tasks with high computational load, as well as the applicability of the predictor under different models?

  4. In the photovoltaic energy experiments, the authors map the second-level neural network training process to a 12-hour photovoltaic supply trajectory, while actual renewable energy supply usually changes on the scale of minutes or hours. Can the authors further explain the rationality of this time-scale mapping, and whether it may affect the performance comparison among different scheduling strategies? If this time-scale mapping is not performed and the training task is run under real time-synchronized energy supply conditions, can the proposed method still obtain significant energy optimization benefits?

Reviewer_uxkhOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a Lyapunov-based scheduler that adjusts neural network batch sizes based on renewable energy availability and training progress. It aims to reduce grid-energy dependence while maintaining training quality.

Critical strengths

(+) The paper addresses a relevant problem. Improving renewable energy utilization in AI computing systems is an important research direction.

(+) Jointly considering power consumption, training duration, and learning progress is reasonable.

(+) The authors conduct closed-loop GPU experiments with measured training feedback, rather than relying entirely on simulations.

Critical weaknesses

(-) It is unclear how significant this problem is in real AI training infrastructure. Other solutions, such as energy storage, power capping, and workload scheduling, already address renewable energy variability. The paper does not establish why batch-size adjustment is necessary or preferable.

(-) The novelty and significance are limited. Most of the mathematical techniques, including Lyapunov optimization, virtual queues, and renewal control, are already well known. The paper contains pages of mathematical derivations that largely repeat established results. Much of this could be explained more simply, and it is difficult to identify what fundamentally new insight is gained from the extensive mathematical treatment.

(-) The evaluation is far from representative of the claimed application. The main experiments use CNN training on MNIST, with training lasting only about 10 seconds. Mapping this short run onto a 12-hour residential solar trace does not demonstrate how the approach would perform for real data-center workloads or long-running neural network training.

(-) The baselines are weak. Most comparisons use fixed or predetermined batch-size schedules. There is no meaningful comparison against a simpler renewable-aware scheduler or other existing power-management approaches. It is therefore unclear whether the proposed controller is actually needed.

(-) The reported results are not entirely convincing. Some baselines perform better on individual metrics, and the claimed 7% improvement from phase weighting is based on only three initialization seeds. More importantly, the error constraint never becomes binding, so the central optimization tradeoff is not convincingly demonstrated.

Questions for authors

What practical advantage does this approach offer over existing power-management and renewable-aware workload scheduling methods?

Can the authors demonstrate the benefits using realistic, long-running AI training workloads and stronger adaptive baselines?

How does the phase-dependent theoretical analysis account for changes in the Lyapunov weight between consecutive frames?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.