acceptodds
Under review as a conference paper at ICLR 2027

Energy-Optimal Test-Time Scaling: Latency and Power Tradeoffs in Sequential and Parallel Scaling

Abstract

Scaling of test-time-computation via both longer model outputs for reasoning chains and repeated parallel samples from a model are frequently leveraged to improve task performance. While both sequential and parallel scaling increase task performance, the real-world costs of this additional test-time computation are poorly quantified with existing test-time scaling laws relying on efficiency proxies such as FLOPS or GPU-hours. In this work, we develop an analytic task performance model for parallel and sequential test-time scaling and conduct an empirical efficiency benchmark of efficiency costs for a wide range of test-time computing configurations. We demonstrate that energy use observes distinct scaling behaviors from latency and FLOPS-optimal scaling, with a preference towards longer reasoning chains and fewer repeated samples. Accordingly, we identify energy-optimal test-time configurations for the Qwen3 family of models and find that energy-optimal selections reduce inference energy use by 22% and 242% savings as compared to latency-optimal and FLOPS-optimal selections, respectively.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Energy, Efficiency, Test-Time Scaling, Environmental Impact

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hfHXOpenReview ↗
Rating
2
Confidence
1
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies energy-optimal test-time scaling for LLM reasoning. It argues that the common proxies for test-time compute allocation—FLOPs and latency—can diverge from energy-optimal allocation because GPU power draw is not constant: increasing parallel samples raises instantaneous power, whereas increasing reasoning length has a comparatively small effect on power. The paper proposes a joint sequential–parallel scaling model for task accuracy, combines it with measured latency, steady-state power, and energy across Qwen3 models and several GPUs, and characterizes how energy-optimal configurations tend to favor longer reasoning chains at lower parallelism, smaller dense models, and sparse mixture-of-experts models. The claimed contributions are the joint accuracy scaling model, an accuracy–latency–energy benchmark, and a characterization of energy-optimal test-time scaling across models, tasks, and hardware platforms.

Critical strengths

Strength 1: The problem is practically important and the central observation is consequential. Test-time scaling is increasingly used in reasoning deployments, yet FLOPs and latency are often treated as sufficient proxies for cost. The paper’s argument that transient power behavior can make energy-optimal allocation systematically different from latency-optimal allocation is directly relevant to serving and infrastructure decisions. This is the strongest reason to consider the paper: it identifies a real mismatch between commonly used efficiency proxies and actual energy use.

Strength 2: The joint sequential–parallel accuracy model provides a useful planning abstraction. Modeling pass@k coverage as a function of both reasoning length and number of samples allows iso-accuracy comparisons across test-time configurations, which is more informative than comparing methods at a single fixed budget. Even if the model is empirical, it gives a concrete way to reason about trade-offs among accuracy, latency, and energy.

Critical weaknesses

Weakness 1: The provided manuscript is materially corrupted, and key sections or figures are missing, which prevents proper evaluation of the method and experimental setup. Page 1 contains only line numbers, Page 3 and Page 5 are largely repeated indices, and the paper’s opening framing, part of Section 2, and at least some figures or surrounding text are not readable in the submitted file. This is not a cosmetic issue: the missing material likely contains the formal problem setup, measurement protocol details, and possibly Figure 2 or Figure 4, which are needed to assess how power and energy are measured and how the scaling model is fitted. For a paper whose central claims depend on empirical power measurements and configuration comparisons, this is a serious evaluability failure and supports rejection in the current form.

Weakness 2: The core claims rest on empirical power measurements and an empirically fitted accuracy model over a limited set of models, tasks, GPUs, and serving configurations, without sufficient evidence that the conclusions generalize or are causally attributable to the stated mechanisms. The experiments center on the Qwen3 family, vLLM serving, a maximum of 16 parallel samples, and a few reasoning/code tasks. The joint scaling model is a curve fit with R2>0.95R^2>0.95, but fit quality does not establish that the model extrapolates to other (k,L)(k,L) regions, other model families, longer contexts, or multi-turn agent workloads. Power draw is measured with DCGM under a specific serving stack; the paper does not adequately separate the effects of model architecture, batch scheduling, KV-cache behavior, and implementation details from the claimed transient-power mechanism. The headline energy savings (up to 60.1% vs latency-optimal and 55.8% vs FLOPs-optimal) are therefore suggestive but not yet decisive. For a paper whose contribution is primarily empirical characterization, this limits the strength of the conclusions.

Questions for authors
  1. The submitted manuscript has corrupted pages: Page 1 is only line numbers, and Pages 3 and 5 are largely repeated indices. Can you provide a complete, readable version? Without the missing setup and figure material, the measurement protocol and several central claims cannot be assessed.

  2. How many repeated measurements were used for steady-state power, latency, and energy for each configuration? Can you report variance across runs and clarify whether the energy-optimal rankings are stable under run-to-run variation?

  3. The paper limits parallel samples to 16 because of KV-cache throttling. How would the energy-optimal versus latency-optimal divergence change at higher concurrency, where the transient power regime may shrink or disappear?

  4. The joint scaling model is an empirical fit. How well does it extrapolate beyond the measured grid of kk and LL, and how sensitive are the iso-accuracy contours and optimal configurations to model misspecification?

  5. The power measurements depend on vLLM and DCGM. How much of the observed power scaling is attributable to the serving stack, batching policy, and KV-cache implementation rather than to the model or test-time configuration itself?

2 comments

We would like to thank the reviewer for their feedback. We appreciate that they found our work practical, consequential, and useful for understanding tradeoffs in test-time computing. Below we address their concerns:

Submission Formatting Issues. We have checked that the pdf for our submission renders properly and no issues with the file format are reported by any of our other reviewers. Please let us know if we can be of any assistance.

Measurement Methodology. As stated in Appendix A, batches of k requests are issued to the inference engine and repeated 8 times (or 4 times for the larger 14B and 32B dense models). Over the duration of the run and across all models, the median coefficient of variation in steady state power draw is 6.3%, and the P90 variation is 12.5%. Additionally, we find there is minimal drift in the power draw over the duration of the run, with only an increase of 0.5% in steady state power draw for the median run, and +1.4% for P90.

In regards to variation in latency per wave, we find minimal variation. The coefficient of variations in wave processing time is Qwen3-32B 0.9%, 14B 1.8%, 8B 3.4%, 4B 4.6%, 30B-A3B 5.6%, 1.7B 6.5%.

Evaluations at Higher Concurrency. In additional experiments on the H100 NVL, we consider a setting in which multiple parallel requests are processed concurrently in addition to pass@k parallel sampling. While increased concurrency reduces the difference between latency and energy-optimal scaling, we find that even when serving up to 8 batched requests with up to 128 concurrent sequences, energy-optimal configurations differ from latency-optimal configurations at 29% of iso-accuracy levels.

Note: In these experiments, we conduct our evaluations on H100 NVLs due to limited compute availability. However, as we note in Section 3.4, power proportional scaling is limited on this platform due to power capping. We expect that the difference between energy-optimal and latency-optimal serving would remain more pronounced on the uncapped platform.

Cpicks differextra energy, latency pick (mean / max)extra latency, energy pick (mean / max)energy pick at k=1latency pick at k=16
154%7.6% / 54%11.4% / 45%59%62%
252%4.8% / 35%5.2% / 36%50%16%
439%1.1% / 10%0.9% / 9%53%0%
829%0.9% / 6%0.6% / 9%92%0%

Generalizing Performance of the Joint Scaling Model. Below we provide the held-out accuracy of the joint scaling model (pooled over models) and find that with the exception of AIME leave-budget-out fitting, the RMSE error of predicted pass@k is less than 0.05:

taskin-sampleleave-one-budget-outleave-one-k-out
AIME260.0350.0650.048
Math5000.0250.0420.031
LiveCodeBench0.0080.0130.014

Limited Model Families. We would like to highlight that while our paper primarily focuses on Qwen3 for its main analysis. We find that the distinction between energy-optimal and latency-optimal scaling is pronounced for other model architectures as well. Concretely, on Qwen 3.5 which utilizes a different Gated Delta Net based architecture as opposed to Group Query Attention similarly observes distinct energy-optimal scaling.

basismetric paidusing the other pickpicks differmeanmaxat pass@k
pass@kenergylatency-optimal54%6.0%35%0.82
pass@klatencyenergy-optimal54%13.2%56%0.64
majority@kenergylatency-optimal88%10.8%27%0.75
majority@klatencyenergy-optimal88%28.7%44%0.48

Variability Under Different Measurement and Serving Configurations. We note that DCGM and vLLM are standard tools for inference energy measurement and serving, used across recent benchmarks and characterization studies (Chung et al., 2026; Patel et al., 2024; Jin et al., 2025); DCGM reads the same board-power sensor as NVML/nvidia-smi while additionally exposing the hardware counters we report in Appendix F.

To validate the effectiveness of our evaluations, we also consider SGLang as another commonly used inference serving engine. Across all Qwen3 model sizes and sequence lengths, we find that the difference in median energy consumption was within 1%, and the difference in median latency differs by no more than 5%.

Additionally, we compare the efficiency of joint scaling using the default bf16 precision with reduced precision fp8 KV Cache. We find that distinct energy-optimal scaling persists.

basismetric paidusing the other pickpicks differmeanmaxat
pass@kenergylatency-optimal49%6.4%54%0.76
pass@klatencyenergy-optimal49%8.6%46%0.56
majority@kenergylatency-optimal57%8.1%50%0.53
majority@klatencyenergy-optimal57%7.8%37%0.31

Thank you again for your feedback! As the discussion period ends today, we would be grateful to know whether our responses address your concerns, and we are happy to clarify any remaining point.

Reviewer_G3P7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work presents an empirical study of how energy-optimal test-time scaling behaves on the qwen3 model family compared to latency and efficiency behaviors. The authors find that energy-optimal scaling differs from the latter as GPU draw grows with parallelism but not with sequence length, and that it favors longer reasoning, fewer parallel samples, and smaller or MoE models. They evaluate on several math and coding reasoning benchmarks (AIME26, MATH500, LiveCodeBench).

Critical strengths

The idea of studying the test-time scaling from a energy-optimal consumption angle is a very interesting one.

Critical weaknesses

The work is empirical and limited to one model family. I would expect a more thorough setup with additional families to generalise the results, otherwise the findings should be confined to the specific architecture used (Qwen).

The results appear varied: for instance, parallel or sequential scaling is favoured depending on whether the model is dense or MoE (section 3.2), on the task (section 3.3), and on model size (section 3.2), which makes challenging to derive a general rule similar to the task performance ones.

Questions for authors
  • In line 239-240, you mention that the reported mean generated tokens is the actual number of tokens generated by the model. Is this averaged across examples? And, is it including the reasoning tokens too, or is it the actual number of tokens generated only for the response?
4 comments

We thank the reviewer for their feedback and appreciate that they found our approach to test-time scaling interesting. To address the reviewer’s concerns, we clarify our methodology and findings below.

Results Across Model Families.

We note that in Appendix D, we demonstrate that our results are consistent across both Qwen3 and the Qwen3.5 family of models. Notably, Qwen3.5 does not share the same model architecture as Qwen3 and uses a hybrid attention architecture based on Gated-Delta Network which reduces the costs of attention as sequence length scales. Concretely, we find that selecting latency-optimal test time configurations yield up to a 35% penalty in energy, and that energy-optimal configurations can result in up to a 56% penalty in latency.

Additionally, we conduct additional experiments on the Gemma 4 family of models (2B, 4B, 12B and 26BA4B) and find that the Gemma family of models likewise exhibits distinct energy-optimal configurations despite containing a smaller range of model sizes than either Qwen family; with up to 26% excess energy used by latency-optimal configurations, and up to 29% additional latency for energy-optimal configurations. We will gladly incorporate these results upon revision.

Effects of Model Size, Sparsity, and Task.

As we highlight in our takeaways, we agree the optimal configuration varies across models, tasks, and hardware. However, these factors consistently predict that energy-optimal configurations select towards greater use of sequential test-time computing. We will revise the paper to state the cases where energy-optimal scaling emerges explicitly in the introduction and as a consolidated takeaway at the end of Section 3.

Energy-optimal scaling emerges for inference in the transient power regime where a model does not saturate a hardware platform’s power cap and increasing sequence length improves tasks performance. Decreasing model size or utilizing sparse MoE models both reduce hardware utilization which makes the bias towards longer sequences more pronounced, and increases the divergence between energy- and latency-optimal scaling. Likewise, reasoning heavy tasks (e.g. AIME and MATH) benefit more from sequential scaling which yields more viable energy-optimal configurations that utilize longer generations.

Number of Generated Tokens.

Mean generated tokens is the average, over all examples in the task and all k samples per example, of the total number of tokens the model emits, including reasoning and the final response. For sequences that reach the cap, the count includes the tokens generated up to the forced termination plus the answer that follows. To fit the joint scaling model (Eq. 3) and compute iso-accuracy contours, we use the mean generated tokens as L. We will add this definition to Section 2.3.

Thank you again for your feedback! As the discussion period ends today, we would be grateful to know whether our responses address your concerns, and we are happy to clarify any remaining point.

Thank you for the detailed responses and the additional Gemma 4 experiments, which address my concern about a single model family. However, the contribution remains largely empirical, and the optimal configurations still vary considerably across model size, sparsity, and task, which limits how general the conclusions are. I will therefore keep my score.

Thank you for the follow-up, and for noting that our Gemma 4 experiments address your concerns regarding a single model-family.

On the empirical nature of our work, we agree that while it is primarily empirical we additionally introduce an analytic model for joint scaling alongside empirical efficiency benchmarking. We note that existing test-time scaling laws for accuracy are empirical as well [1,2]. Among our contributions is extending them to reflect task performance scaling as a function of energy and latency, which is not captured by existing proxies (FLOPs, token budgets).

References

  1. Brown, Bradley, et al. "Large language monkeys: Scaling inference compute with repeated sampling." arXiv preprint arXiv:2407.21787 (2024).
  2. Muennighoff, Niklas, et al. "s1: Simple test-time scaling." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.
Reviewer_kykwOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the energy and latency tradeoffs between sequential test-time compute (longer reasoning) and parallel compute (so more samples). It fits a joint accuracy scaling model and measures GPU latency, power, and energy across six Qwen3 models and three GPU platforms. The main finding is that energy-optimal configurations favor longer reasoning with fewer parallel samples, while latency-optimal configurations favor more parallelism. On AIME 2026, the paper reports up to 60.1% energy savings over latency-optimal configurations and 55.8% over FLOPs-optimal configurations.

Critical strengths
  1. The paper directly measures GPU power and latency across dense/MoE models, reasoning budgets, sample counts, and three GPU platforms, rather than relying on FLOPs or TDP estimates, this definitely provides useful empirical evidence of underutilization at low concurrency.

  2. The authors show that energy- and latency-optimal configurations diverge on AIME and MATH-500 but not on coding tasks or GPQA. The difference is also larger on high-power-cap GPUs. These observations are also useful for low-concurrency deployments.

Critical weaknesses
  1. The claimed savings depend on an underutilized GPU serving one query's samples as its entire batch. In realistic multi-user serving, batching typically keeps GPUs highly utilized, and the paper itself shows that energy and latency then become nearly proportional. The central finding thus seems to be a consequence of the chosen operating regime.

    More importantly, energy is estimated from steady-state GPU power, excluding full-request idle/transient periods and system-level power. These costs penalize the longer-running configurations favored by the method, which incur up to 40% higher latency. So, these savings of up to 60.1% could shrink substantially under complete energy accounting. Reporting only maximum savings further obscures typical benefits.

  2. The joint scaling model is fitted separately to each model and task. The reported R2>0.95R^2 > 0.95 measures in-sample fit rather than predictive accuracy. No held-out budget predictions or uncertainty estimates are provided.

    AIME contains only about 30 problems, with no confidence intervals for accuracy or energy savings. The fitted A_max = 1.0 ±\pm 0.0 also suggests a constrained parameter rather than a reliably estimated asymptote.

    And lastly, optimization uses pass@k, which assumes a perfect verifier. Realistic answer selection through majority voting or reward-model reranking could change the optimal allocation. The non-standard 5-shot prompting introduces another uncontrolled variable. Without validating the accuracy model and accounting for its uncertainty, the claimed energy-optimal configurations are not convincing in the present context.

Questions for authors
  1. Do the energy savings persist under realistic multi-user batching (e.g., 32–256 concurrent sequences)? How common is the low-utilization regime in practical deployments?

  2. How accurately does the scaling model predict held-out token budgets and sample counts? What are the confidence intervals for the estimated savings, and why is A_max exactly 1.0 for AIME?

  3. Do the optimal configurations change under majority voting or reward-model reranking instead of pass@k?

  4. Why use 5-shot prompting for reasoning models? Do the conclusions hold under standard zero-shot prompting?

3 comments

We thank the reviewer for their thoughtful comments. We appreciate that they find our work informative on the energy impacts under variable hardware utilization. Below we clarify methodological details and address the stated questions and concerns:

Energy Accounting

We clarify that our energy estimates use measured GPU power, which includes the GPU’s static (idle) power; longer-running configurations are therefore already charged for static GPU power over their full duration. While we acknowledge that host machine energy consumption introduces additional overhead for long running jobs, we note that GPU power is the dominant component driving energy consumption and we exclude CPU and RAM energy measurements as done in prior work [1,2]. For the camera-ready revision, we will include additional measurements for CPU energy use.

By contrast, data center facility overhead is often measured using Power Usage Effectiveness which is a constant multiplied on top of the IT energy consumption, so it would not change the ordering of configurations.

Reporting of Results.

As the reviewer notes, maximum savings does not capture energy savings across all iso-accuracy levels. However, we find that there remains a distinction between energy and latency-optimal configurations when evaluated by mean difference as well.

We find that on average latency-optimal configurations utilize 9.3% more energy than the energy-optimal baseline, and energy-optimal configurations incur an average 9.6% more latency than the latency-optimal baseline. We will clarify this result in the final manuscript.

Additionally, the reviewer mentions that the distinction between latency and energy-optimal configuration is only present for math tasks. We would like to clarify that the LiveCodeBench code task exhibits the distinction between latency and energy-optimal TTC configurations as well. Latency-optimal configurations consume an average of 6.9% more energy than the energy-optimal configurations for LCB.

Deployment Settings.

Recent cluster-scale telemetry [3] finds that in production online code and chat serving based on traces from industry providers — GPUs spend 61% of in-execution time and 48% of energy in a loaded-but-inactive state driven by bursty arrivals, with some traces as high as 65% of energy. Furthermore, large batch sizes are known to increase per-request latency [4], which can limit feasible utilization when serving under strict latency requirements. Additionally, the KV cache required by long sequences can limit the supported batch size preventing higher utilization.

Additionally, low utilization is especially common for systems deployed on robotics, edge, or mobile platforms where inference is performed on demand in single example batches [5, 6]. Extending our findings to edge deployment settings, we investigate the efficiency of parallel and sequential scaling of the Qwen3 1.7B model on the Nvidia DGX Spark edge accelerator where we find that energy and latency-optimal configurations differ at 42% of iso-accuracy configurations.

Concurrent Requests.

In additional experiments on the H100 NVL, we consider a setting in which multiple parallel requests are processed concurrently in addition to pass@k parallel sampling. Although the gap between energy and latency optimal configurations closes at higher utilizations, our work characterizes regimes in which these approaches diverge, situations which can commonly occur as stated above.

Concretely, we find that even when serving up to 8 batched requests with up to 128 concurrent sequences, energy-optimal configurations differ from latency-optimal configurations at 29% of iso-accuracy levels.

Concurrent requestsPicks differExtra energy using latency pick (mean)(max)Extra latency using energy pick (mean)(max)
154%7.6%54%11.4%45%
252%4.8%35%5.2%36%
439%1.1%10%0.9%9%
829%0.9%6%0.6%9%

Note: In these experiments, we conduct our evaluations on H100 NVLs due to limited compute availability. However, as we note in Section 3.4, power proportional scaling is limited on this platform due to power capping. We expect that the difference between energy-optimal and latency-optimal serving would remain more pronounced on the uncapped platform.

Generalizing Capabilities of the Scaling Model.

Below we provide the held-out accuracy of the joint scaling model and find that with the exception of AIME leave-budget-out fitting, the RMSE error of predicted pass@k is less than 0.05:

TaskIn-sampleLeave-one-budget-outLeave-one-k-out
AIME 20260.0350.0650.048
MATH-5000.0250.0420.031
LiveCodeBench0.0080.0130.014

Bootstrapping confidence intervals, we find that latency-optimal configurations utilize an average excess energy of 9.1% with a confidence interval of [1.7%, 13.5%]; and a maximum energy in excess of 57.7% with a confidence interval of [25.9%, 63.4%]. Conversely, energy-optimal configurations incur an average of 9.7% additional latency with a confidence interval of [2.7%, 16.2%]; and up to 39.8% latency with a confidence interval of [25.8%, 62.8%]. We note slight differences in the means from Figure 1 due to resampling from the bootstrapping.

The value of AMaxA_Max converges to 1.0 as we fit the joint model parameters using a bounded regression where A_max is constrained to [0,1.0] to constrain to practical values of accuracy. Relaxing the constraint on Amax, we find that energy- and latency-optimal configurations exhibit larger divergence.

A_max fitExtra energy using latency pick (mean)(max)Extra latency using energy pick (mean)(max)
Bounded9.3%60%9.6%40%
Free17.6%55%15.3%45%

Pass@K vs Majority Vote.

Pass@k is Realistic for Verifiable Tasks. We acknowledge that utilizing a pass@k coverage metric requires a verifiable task. However, such examples are common in coding tasks such as in LiveCodeBench, a task in which we demonstrate significant differences in energy-optimal and latency-optimal configurations above.

Majority@k Results. We extend our results to evaluate maj@k joint scaling and find that the optimal configuration for majority vote differ at 43% of iso-accuracy levels from pass@k coverage; predominantly at lower-accuracies (<0.50). Optimal maj@k configurations utilize fewer parallel samples compared to pass@k.

However, we find that with H100-SXMs on AIME26, energy-optimal and latency-optimal configurations remain distinct with penalties up to 33% in energy and 11% in latency.

SelectionMetric paidUsing the other pickPicks differMeanMean where differMaxAccuracy at max
pass@kEnergyLatency-optimal57%9.3%16.3%60%0.68
pass@kLatencyEnergy-optimal57%9.6%16.8%40%0.41
majority@kEnergyLatency-optimal34%4.8%13.9%33%0.38
majority@kLatencyEnergy-optimal34%0.6%1.9%11%0.25

Five Shot Prompting.

We utilize 5-shot prompting as is common in evaluation suites for these tasks [7,8]. For comparison we run evaluation with zero-shot evaluations and find that results are largely comparable. Across all models, sequence lengths, and parallel samples (k), we find that utilizing 0-shot prompting improves performance by an average of 0.015 +/ 0.041, or within noise.

We find that the scaling behavior is consistent between 0- and 5-shot prompting with the mean and max savings in energy and latency for 0-shot prompting within the 5-shot confidence intervals.

References

  1. Dodge, J., Prewitt, T., Tachet des Combes, R., Odmark, E., Schwartz, R., Strubell, E., Luccioni, A. S., Smith, N. A., et al. “Measuring the Carbon Intensity of AI in Cloud Instances.” FAccT 2022, pp. 1877–1894.
  2. Chung, Jae-Won, et al. "The ML. ENERGY benchmark: Toward automated inference energy measurement and optimization." NeurIPS (2026).
  3. Lei, Y., Fernandez, J., Kypriotis, V., Skarlatos, D., Strubell, E., Sherry, J., & Vosler, D. (2026). The energy cost of execution-idle in gpu clusters. arXiv preprint arXiv:2604.04745.
  4. Zhong, Yinmin, et al. "{DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving." OSDI 2024.
  5. Alizadeh, K., et al. (2024). LLM in a flash: Efficient Large Language Model Inference with Limited Memory. ACL 2024.
  6. Black, K., Brown, N., Driess, D., Esmail, A., Equi, M. R., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Shi, L. X., ... Zhilinsky, U. (2025). π₀: A vision-language-action flow model for general robot control. Proceedings of Robotics: Science and Systems XXI.
  7. Gu, Yuling, et al. "Olmes: A standard for language model evaluations." Findings of the Association for Computational Linguistics: NAACL 2025. 2025.
  8. Liang, Percy, et al. "Holistic evaluation of language models." arXiv preprint arXiv:2211.09110 (2022).

Thanks for the detailed response. I find the clarifications sufficient, and will increase my score from 2 to 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Discussion

As the discussion period closes, we would like to summarize our contributions, discussion, and responses. We thank the reviewers again for their engagement, and Reviewer kykw for revising their assessment following our response.

To our knowledge, our work is the first to jointly model the effects of scaling test-time compute across task accuracy, energy, and latency. We appreciate that the reviewers found our work identifying the distinction between energy-optimal, latency- and FLOPS-optimal TTC scaling to be "consequential", "practically important", and "useful", as it "identifies a real mismatch between commonly used efficiency proxies and actual energy use."

As we point out in our response to Reviewer kykw, settings which observe distinct energy-optimal scaling are common in both cluster-scale deployments as well as on new edge accelerator platforms. We believe that our analysis will enable researchers and developers to evaluate the energy requirements and to determine the more efficient deployment choices for energy-intensive reasoning workloads.

In our discussion responses, we have addressed all of the reviewers' stated questions and concerns:

Effects of Model Families (Reviewer G3P7). We show that energy-optimal scaling exists across model architectures.

  • Energy-optimal scaling remains distinct from latency-optimal scaling on Qwen3.5, a hybrid Gated DeltaNet architecture already covered in Appendix D of the original submission.
  • In supplementary experiments, Gemma 4 likewise exhibits distinct energy-optimal scaling, with up to 26% more energy for latency-optimal picks and up to 29% more latency for energy-optimal picks.

Effects of Hardware and Serving Stack (Reviewers kykw, hfHX). We show that distinct energy-optimal scaling is not limited to our hardware or software serving stack.

  • In multi-user settings with up to 128 concurrent sequences, energy-optimal and latency-optimal configurations differ at 29% of iso-accuracy levels, converging as the GPU saturates, as the transient-power mechanism predicts.
  • On the DGX Spark edge accelerator, we observe energy-optimal configurations distinct from latency-optimal configurations at 42% of iso-accuracy levels.
  • Our results hold when utilizing the SGLang inference framework (median energy within 1%) or KV caches of differing precision.

Effects of Accuracy Metric (Reviewer kykw). We show that energy-optimal scaling still exists when using verifier-free evaluation metrics.

  • Majority Vote vs. Pass@k. We evaluate majority vote as an alternative to pass@k and demonstrate that energy-optimal scaling is once again present and distinct from latency-optimal scaling, with penalties up to 33% in energy and 11% in latency on AIME.

Reporting of Statistical Significance (Reviewers kykw, hfHX). We provide additional validation for our direct measurements and scaling models.

  • Held-Out Performance of the Joint Scaling Model. Leave-one-budget-out and leave-one-k-out RMSE is below 0.05 in all settings except AIME leave-one-budget-out (0.065).
  • Mean vs. Max Overhead. We now report mean overhead alongside maxima: latency-optimal configurations use 9.3% more energy on average and up to 60.1% more energy in worst-case; energy-optimal configurations incur 9.6% more latency on average.
  • Variation in Latency and Power Draw. Across inference batches there is minimal variation in measured power draw, with a median coefficient of variation of 6.3% for steady-state power and 0.9–6.5% for latency.

Clarifications on Experimental Methodology (Reviewers kykw, hfHX, G3P7).

  • Token and Energy Measurements. We clarify our energy and generated-token measurement definitions and will include them in revision.
  • Effects of Prompting Strategy. Compared to our five-shot baseline, zero-shot prompting changes accuracy by 0.015 ± 0.041, within noise, and leaves energy-optimal scaling behavior unchanged.

Manuscript Formatting Issues (Reviewer hfHX). Reviewer hfHX's primary concern is that the submitted PDF is corrupted. We have verified that the PDF renders properly and no other reviewer reports an issue.

Across all supplementary settings requested by the reviewers, our findings remain consistent: energy-optimal TTC configurations exist distinct from FLOPS- and latency-optimal configurations, selecting for longer reasoning chains and reduced concurrency. We will gladly incorporate all of these additional settings in our revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.