acceptodds
Under review as a conference paper at ICLR 2027

Agreement by Construction: Multi-Agent RL for Datacenter Network Power Management

Abstract

Modern datacenters (DC) face escalating energy demands, with power consumption becoming a critical bottleneck for computational scaling. To mitigate this, we introduce LinkRL, a decentralized multi-agent reinforcement learning framework for managing power in DC networking. Our agent learns the optimal timing windows to toggle network links between active and low-power states. Specifically, we target the substantial energy these links consume while idle during computation phases. Networking infrastructure accounts for approximately 20% of the total DC power, and the inefficiency of idle links presents a significant optimization opportunity. In LinkRL, we train agents to predict traffic patterns, enabling preemptive sleep and wake-up. We guarantee agreement between agents managing a shared link, by construction and without communication, by restricting each agent's input to features both ends observe identically. We train our framework on real-world LLM training workloads using a high-fidelity networking simulator that serves as NVIDIA's primary platform for design, verification, and benchmarking of its networking products. LinkRL achieves 10.9% datacenter power savings while maintaining latency, outperforming baselines. The same policy transfers to other distributed-compute workloads without retraining. Reallocating these savings to compute yields an 8.9% GPU performance gain. LinkRL is integrated into the design of NVIDIA's next-generation networking hardware.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
datacenter networks, power management, energy efficiency, multi-agent reinforcement learning, decentralized control, hardware deployment, real-time inference, network link power states, PPO

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_zz9QOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work addresses the problem of optimal active and low power state control in data center networks, It presents LinkRL where decentralized agents are trained for preempt the traffic behaviors. LinkRL claims significant power savings and latency improvement over baselines on network simulations.

Critical strengths

The paper’s suite of experiments and ablations appear to be thorough. Except for the significance concerns regarding Table 2 (See critical weaknesses), the experimental results appear to be compelling.

Critical weaknesses

In Table 2, the out of domain results (HPC and generic ML) have smaller gaps with or beat LinkRL on the NTG metric, especially within the reported standard deviation margin, so it is not clear whether LinkRL’s improvement is significant as the policy might be hyper-optimized for the in-domain 6 LLM training workloads. The training data detailed in Section 5.2 covers a wide range of model size, architectures, and schedules and is used as a one-size fit all policy for evaluations. A related concern is the impact of the size and quality of the training set, i.e. are the results replicable with just a smaller and less diverse subset of the training workloads?

Additionally, there is a major lack of documentation on training and test datasets and Section 5.2 and Appendix K are insufficient, which hinders replicability. Key missing details include the source of the data, training-holdout set split breakdown, and in the case of the LLM workloads breakdown of the number datapoints from each LLM model.

Questions for authors

1.The reward weight β\beta is swept across {40, 80, 120} and is noted to show monotonically decreasing performance across all evaluated metrics. Then why are reward weights smaller than 40 not tested given the monotonicity observation?

2.Why are Power Reduction and JTO metrics not reported in Table 2 for HPC and Generic ML workloads?

  1. Do the authors have results on the individual held-out LLM workload? In other words, does the trained policy work similarly well across all workloads?

  2. Section 5.2 mentions that there are 14.4 million LLM workload training points. What is the breakdown of these data points by LLM model?

  3. How were the model sizes for the CNN and MLP layers (Appendix A) selected? Does the parameter count, as noted in Table 1, impact the performance given the same architecture?

  4. Several tables in the appendix are missing titles and table numbers.

  5. Details about the “Generic ML” traces appear to be missing from Appendix K.

1 comment

We thank the reviewer for recognizing the experimental suite and for the concrete suggestions on generalization and reproducibility.

  • W1 (Out-of-domain gains and possible over-fitting): Table 2 now reports power reduction (PR), JTO and NTG for all workload groups. On HPC, LinkRL reaches 12.14% NTG versus 12.96% for Threshold, consistent with long, regular idle gaps favoring heuristics. On Generic ML, LinkRL reaches 7.56% versus 5.95% for the strongest baseline. On additional LLM workloads, it reaches 7.12% versus 5.06%. These results support transfer within the tested domains, rather than uniform superiority across domains. W2 directly tests training-workload diversity.

  • W2 (Smaller, less diverse training set): We ran an additional three- versus six-workload training ablation with three seeds. The retained workloads were GPT-4 MoE, GPT-3 175B dense and Llama 3 with SHARP. Mean NTG on their test segments was similar: 7.38% versus 7.68%. On omitted workloads, NTG was 2.37% versus 9.36%; on the ablation's held-out set, 2.54% versus 5.34%. Broader training therefore improves generalization. Appendix D, Table 8 reports the full matched comparison, including PR, JTO and standard deviations.

  • W3 (Dataset documentation): The LLM traces were recorded with NVIDIA Nsight at 100 μs intervals. We will release the full trace data. DNetSim replays interleaved compute and network operations. Each training recording is split chronologically into contiguous segments: approximately 80% training, 10% validation and 10% testing. Hyperparameters and checkpoints are selected by mean validation NTG; testing is reserved for final evaluation. We will expand Section 5.2 and Appendix K with this protocol and the breakdown below.

  • Q1 (Reward weights below 40): We ran additional tests at β = 10 and 20. Mean PR / JTO / NTG (%) was 13.85 / 4.02 / 8.44 and 12.49 / 2.99 / 8.95, respectively. The published β = 40 result is 10.87 / 0.90 / 8.88. Lower β increases power savings but also JTO. We prefer β = 40 because it retains similar NTG with lower JTO. Appendix D, Table 12 reports the lower-weight results.

  • Q2 (Missing HPC and Generic ML PR and JTO): These metrics are now included in manuscript Table 2. For LinkRL, PR / JTO / NTG (%) is 14.89 / 1.26 / 12.14 on HPC and 10.35 / 1.75 / 7.56 on Generic ML.

  • Q3 (Individual held-out LLM results): The breakdown below uses the original Table 2 evaluations. All metrics are percentages. NTG is positive on each workload, although its magnitude varies. Appendix K, Table 15 now reports this breakdown.

WorkloadPRJTONTG
GPT-3 40B14.360.8512.07
GPT-3 43B12.891.5710.04
Llama 3 8B11.241.338.79
GPT-3 175B MoE12.810.5510.98
Llama 3 70B2.041.230.61
Llama 2 70B0.810.520.21
Mean9.031.017.12
  • Q4 (Per-model training-data breakdown): The table gives each workload's share of recorded source-trace samples, collected at 100 μs intervals. Appendix K, Table 16 now reports these shares.
WorkloadShare
GPT-3 175B dense15.04%
GPT-3 175B MoE24.70%
GPT-3 40B2.01%
GPT-4 MoE16.62%
Llama 3 without SHARP20.84%
Llama 3 with SHARP20.79%
Total100%
  • Q5 (Architecture and layer-size selection): We swept MLP, MLP-MLP and CNN-MLP architectures and layer sizes using validation performance. Larger models generally performed better in the size sweeps. All policy weights must fit in on-chip SRAM under a strict area budget, so we focused on compact models. Section 4.1, Table 1 and Figure 3 compare the architecture families; Appendix A lists the selected configuration. We will clarify this selection rationale.

  • Q6 (Missing appendix titles and numbering): The threshold, β and reallocation-factor sweeps now have numbered captions as Tables 5–7.

  • Q7 (Generic ML trace details): The three traces use uneven-cycle ring all-reduce workloads, alternating six-phase compute profiles with 1 MB collectives over 25 iterations. The profiles are balanced, bursty or ramped, providing controlled variation in compute and communication timing. We will add these construction details to Appendix K.

Reviewer_ze19OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents LinkRL that learns when to put datacenter network links to sleep and when to wake them before traffic returns. A shared policy runs independently at each network endpoint. The central claim is that restricting both endpoints to identical link-level observations guarantees that they choose the same action without exchanging messages. The authors evaluate the policy in a packet-level simulator using workload traces, then project link-level savings into system power savings and additional compute throughput.

The simulation results are encouraging, but I do not think the current evidence supports the strongest claims. The agreement guarantee depends on a physical observability assumption that is not established. The headline power and throughput improvements are projections with unresolved accounting and feedback assumptions. Several inconsistencies in the appendices also make it difficult to determine exactly which configuration and evaluation split support the reported results.

Critical strengths
  1. The paper addresses an interesting problem of datacenter traffic management using RL. I found the problem formulation interesting to read, and problem well motivated.

  2. The paper reports power reduction and job-time overhead separately, making the trade-off visible. It includes comparisons with deployed heuristics, three training seeds, quantization results, and a separate unseen-workload evaluation.

  3. The conditional agreement argument is mathematically sound: identical inputs to the same deterministic policy produce identical actions. The issue is whether the hardware satisfies those premises.

  4. The authors report the HPC case where a threshold heuristic performs better, which usefully limits the generalization claim.

Critical weaknesses
  1. The soundness of the method needs to be improved/clarified
  • E.g. In section 3.2, Identical observations and shared parameters do not guarantee identical actions under independently sampled stochastic PPO policies. If both agents select sleep with probability p their disagreement probability is 2p(1-p). The deterministic deployment argument remains valid, but exploration and the handling of conflicting training actions need to be explained.

  • Section 5.1 defines power reduction over a compute-node power envelope, while the abstract calls it total datacenter power savings. Appendix F describes system-wide measurements over a particular 576-GPU fabric.

  • The single-link ablation retains the same coupled simulation but discards experience from all except one designated agent. This changes the diversity and potentially the quantity of training data. It does not remove interactions between links. The improvement from 5.77% to 8.88% NTG therefore supports the benefit of training on broader experience. It does not, by itself, establish that the policy learned cross-link coordination. A comparison with matched transition counts and workload coverage is needed to separate these explanations.

  • Nit: Reward units: Define whether TimeInLowPowerState means time spent asleep during the current step or cumulative residence time. Please clarify

Questions for authors
  1. Can you provide the exact endpoint feature transformations and show that equality survives propagation delay, clock skew, counter resets, and asynchronous forced wake-ups?

  2. Which low-power timing and energy parameters were measured on hardware, and which were assumed? Does trace replay propagate application dependencies after policy-induced delays?

  3. Can you reconcile the power denominators, the 0.9 conversion factor, and the effect of accelerated compute on subsequent idle windows?

  4. Which headline results use INT8 inference and genuinely unseen workloads?

  5. Please address weakness as noted in review.

1 comment

We thank the reviewer for the careful and constructive review. We address each point below.

  • W1 (Stochastic exploration and conflicting training actions): The reviewer is correct that independent stochastic samples can disagree with probability 2p(1 − p). During training, LinkRL samples one action per logical link and applies it to both ports. Exploration occurs at link level, without conflicting endpoint actions. Each link contributes one transition per step. Deployment uses deterministic argmax. We will clarify this in Section 3.2 and Appendix G.

  • W2 (Inconsistent power denominators): The intended denominator is baseline deployment-fabric power, including GPUs, switches and links (Appendices C and F). We will use “fabric power” consistently in place of the ambiguous “datacenter power” and “compute-node” terminology.

  • W3 (What does the single-link ablation establish?): Both settings retain interactions between links. The improvement from 5.8% to 8.9% NTG is consistent with learning these coupled dynamics, but the ablation does not separate this effect from the benefit of broader link experience. We will describe cross-link coordination as a possible explanation supported by the result, rather than an established conclusion.

  • W4 (Reward units): TimeInLowPowerState is time spent in the low-power state during the current step, measured in microseconds, including fractional values. It is not cumulative across the episode. ForcedWakeUpEvent is a binary indicator of a packet arriving while the link is in the low-power state during that step. We will define both explicitly in Section 3.2.

  • W5 (Configuration and evaluation provenance): We will distinguish FP32 evaluation results from INT8 deployment settings in Appendix A and the relevant captions. Each training recording is split chronologically into approximately 80% training, 10% validation and 10% testing. Hyperparameters and checkpoints are selected using mean validation NTG; test segments are used only for final evaluation. Section 5.2 will state this protocol. Q4 identifies the precision and data behind the reported results.

  • Q1 (Endpoint features, timing differences, counter resets and asynchronous forced wake-ups): The reported results account for propagation delay, which DNetSim models natively. We thank the reviewer for highlighting clock skew, which was not explicitly modeled. We evaluated each endpoint using its own observations with 25 ns propagation delay on the six test segments. Compared with applying one shared action to both endpoints, mean NTG changed by less than 0.01 percentage points. The results suggest that small timing differences leave the recent idle-window history largely unchanged. Low-power entry requires both endpoints to agree, while either endpoint can keep the link active or initiate wake-up. We do not anticipate counter resets during normal operation. During warm-up with zero or incomplete history, we observed that links remained active until the history buffer was filled. The endpoint comparison is reported in Appendix D, Table 13, and referenced in Section 5.5.

  • Q2 (Hardware parameter sources and replay dependencies): Absolute power requirements are projections for future hardware. Low-power-state effects, power deltas and the clock-speedup response to increased compute power are based on measurements from existing hardware. The 100 μs entry and exit latency comes from the current generation's specification. We will clarify these sources in Appendix C. Trace replay interleaves compute and network operations in DNetSim. Communication delays postpone subsequent dependent operations, and JTO captures their effect on completion time.

  • Q3 (Power projection, the 0.9 factor and accelerated compute): Power reduction (%) = 100 × ρ × (P_idle − P_lp) / P_base, where ρ is the fraction of time in low power and all power terms use the same fabric envelope. The 0.9 factor comes from laboratory measurements: in the measured regime, 1% additional compute power yields approximately 0.9% higher throughput. Table 7 varies this factor from 0.85 to 1.0, giving NTG from 8.34% to 9.97%. NTG is a first-order projection with fixed compute durations. It does not simulate the feedback of faster compute on later idle windows. We will state this assumption explicitly.

  • Q4 (Which results use INT8 and unseen workloads?): Tables 1–3 report FP32 results. Table 1 uses withheld test segments of training recordings; Table 2 evaluates additional workloads. Figure 3 compares FP32 and INT8 on the Table 1 test segments. We ran an additional test to measure the effect of policy quantization on the held-out dataset. INT8 reduced mean NTG by approximately 0.6% relative to FP32. Appendix D, Table 9 reports paired power reduction, JTO and NTG.

Reviewer_GXq2OpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes LinkRL, a multi-agent RL method where each network link in a datacenter has a small agent that decides when to put the link to sleep and when to wake it up. To make the two ends of a link always make the same choice without talking to each other, the agents only use features of the link itself that both ends see the same way, and all agents share one policy trained with PPO. Trained and tested in NVIDIA's DNetSim simulator on real LLM training traces, LinkRL reports 10.9% power savings and an 8.9% throughput gain over three baselines, and its INT8 version is small enough to run on switch hardware.

Critical strengths
  1. The problem is real and important: idle network links waste a lot of power, and the paper works with a real hardware feature and real LLM training traces.

  2. The idea of using only features that both ends see is simple and fits the hardware limits well, and the ablations (vanilla IPPO, MAPPO, single-link training) clearly show why it matters.

  3. The paper takes deployment seriously: the INT8 model fits a 175 ns per-port time budget, the chip area and power cost are reported, the baselines include methods already used in real hardware, and the policy is also tested on unseen workloads, HPC traces, and generic ML traces.

Critical weaknesses
  1. Abstract: The abstract says the agent "learns the optimal timing windows". If you say optimal, you need to prove it mathematically. I am curious how you prove it is optimal.

  2. Introduction, paragraph 2: The paper says deciding when to sleep and wake "requires anticipating traffic". How? How does LinkRL actually anticipate traffic?

  3. Figure 1: The policy is quantized to INT8. If the policy is not quantized to INT8, what is the original performance? Readers may want to know the performance gap after quantization. Also, if the original performance of the model is better, why do we need the quantized model?

  4. Introduction, paragraph 4: The paper says the symmetric state construction "guarantees endpoint agreement without communication". Why can LinkRL guarantee this? If you say it is guaranteed, you need to prove it mathematically.

  5. Section 3.1: DNetSim is NVIDIA's internal packet-level network simulator. If DNetSim is an internal simulator, how can readers believe the results reported in this paper?

  6. Section 3.2: The paper uses γ\gamma= 0.8. Why? Are there any ablations?

  7. Appendix A: The hyperparameters were selected on a held-out validation set, but there are no details about this validation set. How can readers understand and believe your results?

  8. Appendix A, Table 4: The total parameter count is 9,324. Only these parameters? Given that the CNN and MLP have multiple layers, how do you count the parameters?

  9. Overall: The algorithm itself has little novelty: the learning method is standard IPPO with shared weights, and the "agreement by construction" that the paper lists as its first contribution comes down to giving both ends the same input and the same deterministic policy. The main value of the paper is applying MARL to a real NVIDIA problem and showing energy savings in simulation, which is useful but closer to an application paper than a valuable MARL method.

Questions for authors

Please answer the questions listed in the weaknesses above.

In addition, could the authors share more details about LinkRL and release the code, so that both academia and industry can reproduce your results? If the authors can answer all of these questions, I will consider raising my score.

1 comment

We thank the reviewer for the detailed feedback and willingness to continue the discussion. We have corrected the missing titles and numbering in Appendix D and address the technical questions below.

  • W1 (Claim of optimal timing): We will replace “optimal timing windows” with “learns when to toggle network links between active and low-power states.” We claim no global optimality guarantee. Appendix C derives an idealized reachable ceiling of 14.7% power reduction under representative LLM-training conditions; LinkRL's 10.9% captures approximately 74% of this ceiling.

  • W2 (How does LinkRL anticipate traffic?): Repeated compute and collective phases make recent link activity informative about future traffic. The policy uses a 32-entry idle-window history. We ran additional feature-permutation tests: permuting this history caused the largest NTG reduction, 9.62 ± 0.41 percentage points over three seeds. This supports the interpretation that the policy uses history to identify traffic patterns. The reward penalizes traffic arriving while the link is asleep, encouraging preemptive wake-up. Figure 5 shows sleep entry during a 2.7 ms compute phase, rejection of a short idle gap, and wake-up 200 μs before a burst.

  • W3 (Performance before quantization, and why INT8?): Table 1 reports FP32 results. Figure 3 shows CNN-MLP NTG of 8.9% before and 8.0% after INT8 quantization. We ran an additional test to measure the effect of policy quantization on the held-out dataset. INT8 reduced mean NTG by approximately 0.6% relative to FP32. Appendix D, Table 9 reports the paired power reduction, JTO and NTG results. INT8 reduces weight storage fourfold to meet the on-chip SRAM and area budget (Appendix I).

  • W4 (Why is endpoint agreement guaranteed?): Identical observations passed to the same deterministic policy produce identical actions. This is the condition underlying the agreement argument. Observations can occasionally differ in edge cases. In an additional test with separate endpoint decisions, mean NTG changed by less than 0.01 percentage points. Low-power entry requires both endpoints to agree; either endpoint can keep the link active or initiate wake-up. We will clarify the condition and the hardware handling of disagreements.

  • W5 (Trust in an internal simulator): DNetSim is used to design, verify and benchmark NVIDIA networking products and is built on OMNeT++. Figure 2 and Appendix E compare it with deployed hardware. Across 75 aligned measurements in four collective configurations, median absolute percentage error is 3.7% overall and 2.7% for large messages. All policies are evaluated on the same traces in the same simulator, including baselines deployed in hardware.

  • W6 (Why γ = 0.8?): We ran additional tests at γ ∈ {0.5, 0.8, 0.9, 0.95}, with three seeds per setting. Mean NTG was 7.63 ± 1.10%, 8.52 ± 0.28%, 7.79 ± 0.84% and 8.00 ± 0.79%, respectively. γ = 0.8 achieved the highest mean NTG among the tested settings. Appendix D, Table 11 reports these results.

  • W7 (Validation-set details): Each of the six training traces is split chronologically into contiguous segments: approximately 80% training, 10% validation and 10% testing. Hyperparameters and checkpoints are selected using mean validation NTG. Test segments are used only for final evaluation. Table 1 reports these test segments; Table 2 reports additional workload evaluations. We will document the protocol in Section 5.2.

  • W8 (Counting the 9,324 parameters): We count the weights and biases of one copy of the shared deployed policy. A 1D convolution contributes C_out × (k × C_in + 1) parameters; a dense layer contributes d_out × (d_in + 1). We sum over policy layers. Sharing across agents does not multiply the count. The training-only critic is excluded. Appendix A now states this convention.

  • W9 (Novelty beyond shared-weight IPPO): Our contribution is the link-level controller and observation design for execution under strict switch-hardware constraints. We use standard IPPO with shared weights. Table 3 supports the design: independent per-port agents reach 1.6% NTG and a centralized critic reaches 5.3%, versus 8.9% for LinkRL. The resulting INT8 policy meets the 175 ns per-port budget. We will make this systems contribution clearer.

  • Q1 (Implementation details and code release): We will add the formulation, algorithm box and validation protocol. Code release is pending internal approval. DNetSim is proprietary; LinkRL uses a Gym interface that permits integration with another simulator modeling link low-power states.

Reviewer_dN3fOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a decentralized multi-agent reinforcement learning framework called LinkRL. The framwework is designed to manage power consumption in modern datacenters, addressing the critical bottleneck posed by power demands. The main objective is to optimize the idle state of network links, which account for 20% of total datacenter power. The RL algorithm trains agents to anticipate traffic patterns for preemptive management of link states, guaranteeing coordination without communication, by restricting each agent’s input to features both ends observe identically. The proposed approach is validated on realistic LLM workloads using NVIDIA’s networking simulator; hence LinkRL achieved a 10.9% reduction in datacenter power consumption while preserving latency, and the same policy showed effectiveness in other workloads, ultimately translating to an 8.9% performance boost for GPUs.

Critical strengths

The main strengths of the paper are: (i)The paper presents clearly the objective, the method, and the framework, although it lacks mathematical formulation. (ii) The problem is well posed by limiting the communication among agents using the same observed partial features.

Critical weaknesses

The main weaknesses of the paper are (i)The lack of mathematical formulation of the algorithm, how it is trained and inferred, etc. Although it is well-written, the paper will improve if more details about the training and the full formulation are included. (ii) The validation should be improved because it will be nice to see why the paper needs two agents per link, what happens if there is the same features, how the training curves are, how many iterations you need to train, etc

Questions for authors

The main questions are related to the weaknesses. (Q1) What happens if we have one agent per link? or the performance of the algorithm based on the features? (Q2) What are the features that make the inference better? (Q3) Why is there not any RNN or LSTM as a comparison? (Q4) Why do you not use a supervised method as a comparison? (Q5) Please add learning curves to see how they improve. How long does it take? How many runs do you need to find a good set of agents to justify the 20% of usage? What happens if the LLMs are improved and their behavior changes?

1 comment

We thank the reviewer for the careful reading and for recognizing the clarity of our objective and presentation. We address the questions on the formulation, feature choices and training process below.

  • W1 (Mathematical formulation, training and inference): We will consolidate the formulation in Section 3.2 and add an algorithm box. The problem is a Dec-POMDP ⟨N, S, {Aᵢ}, T, {Oᵢ}, r, γ⟩. N is the set of logical link agents; S contains traffic, queues and link power states; and DNetSim supplies the transition kernel T. Agent i observes its link features and chooses active or low-power. Its reward is r(i,t) = α × TimeInLowPowerState(i,t) − β × ForcedWakeUp(i,t). All agents share a policy trained with PPO to maximize expected discounted return. Trajectories from all links and parallel environments are aggregated for the shared update (Appendix G). At deployment, each endpoint deterministically selects the highest-probability action, using the INT8 policy on the switch's 25 μs control cycle.

  • W2 (Agent and feature choices, learning curves and iterations): We address the agent design in Q1, feature contributions in Q2, and learning curves, training time and iterations in Q5 below.

  • Q1 (One agent per link and feature dependence): LinkRL acts logically as one agent per link. A switch-to-switch link has two physical policy executors, one at each endpoint, sharing the same policy. Each switch controls its own port. We will clarify this distinction in Section 3.2. Table 3 compares symmetric link-level observations with independent per-port agents using port-level observations. The latter reduces NTG from 8.9% to 1.6%. Q2 addresses individual feature contributions.

  • Q2 (Which features improve inference?): We ran additional tests to measure feature importance by permuting one input group at a time across time steps within the same trace. On the six test segments, unperturbed NTG was 9.32 ± 0.07%. Permuting the 32-entry idle-window history reduced NTG to -0.30 ± 0.38%; time since last transmission to 2.28 ± 1.03%; low-power residence time to 6.88 ± 0.06%; and the performance-hit flag to 9.22 ± 0.05%. Values are mean ± sample standard deviation over three seeds. History and recent activity have the largest effects. This suggests that the policy uses idle-window history to identify traffic patterns. This comparison is now in Appendix D, Table 10.

  • Q3 (Why no RNN or LSTM comparison?): We explored RNNs in preliminary tests but did not observe enough benefit to justify pursuing them further. RNN and LSTM inference would also require a substantially more complex hardware implementation on the target switch. We therefore focused on feed-forward architectures and will clarify this design choice.

  • Q4 (Why not compare with a supervised method?): We include a Supervised + Bayesian baseline (Section 5.2, Appendix B). It predicts idle duration and applies a Bayesian sleep decision rule. It achieves 2.20% NTG on the test segments (Table 1) and 1.97% on the held-out workloads (Table 2), compared with 8.88% and 7.12% for LinkRL. RL directly optimizes the sleep/wake decisions under asymmetric error costs and action-dependent changes to future traffic. We will make this comparison more visible in the main text.

  • Q5 (Learning curves, training cost, search size and changing LLM behavior): Training takes approximately four days on CPU, using six parallel DNetSim environments and one aggregation core (Section 4.3, Appendix A). The selected checkpoints were reached after approximately 2,000 policy updates. Figure 6 in Appendix A shows validation power reduction, JTO and NTG against policy updates, with the selected-checkpoint range marked. We trained three seeds for the final configuration. We swept CNN/MLP architecture, learning rate and β: 74 main-method configurations and 285 including the three ablation variants produced validation results. The deployed policy remains fixed. Training on diverse LLM traces supports generalization across the workload behaviors tested so far (Table 2). Substantial future changes outside the behaviors represented by our LLM, HPC and synthetic traces may require offline retraining.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.