acceptodds
Under review as a conference paper at ICLR 2027

Piggybacking Training on Inference Systems

Abstract

Training a model on the inference system that serves it would enable private on-device fine-tuning and turn idle inference fleet capacity into training capacity without separate training infrastructure. Inference hardware, however, often lacks the memory and interconnect needed for backpropagation. Zeroth-order optimization, such as evolution strategies (ES), uses only forward passes, but evaluating candidate-specific perturbations adds weight-update traffic or separate correction costs to batched inference. We address this problem with NoisyGEMM, a matrix-multiplication kernel that evaluates perturbed models without materializing their weight perturbations. Candidates use compact coefficient vectors over shared Hadamard bases, whose corrections are fused into the base GEMM. We integrate NoisyGEMM into vLLM's linear layers and run training through its existing scheduler, KV-cache manager, and sampler. On an A100, perturbed linear layers run at 1.1×–1.3× the latency of unperturbed cuBLAS GEMM. Using a separable CMA-ES variant, reinforcement learning of Qwen3 models from 1.7B to 30B-A3B improves on the base model on all six reasoning benchmarks and outperforms GRPO on 17 of 18 model–benchmark pairs with the same maximum completion length of 16k tokens. Compared to the state-of-the-art ES framework EGGROLL, NoisyGEMM hosting CMA-ES achieves higher accuracy on AIME 2025 and lower latency when scaling the population size and adapter rank. Finally, we demonstrate piggybacking CMA-ES training on a serving engine that replays one day of BurstGPT live traffic. We will open-source our code upon publication.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Evolution Strategy, Reinforcement Learning, Reasoning, Math, Coding, Zeroth-order Optimization, NoisyGEMM

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_pbiUOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents NoisyGEMM, which enables efficient rollout generation using rank-R perturbed noise added to the weight of the model during its forward pass, added by using randomized basis coefficients per candidate in a shared rank-R Hadamard transformed space with a CMA-ES optimizer, which effectively skips the materialization of perturbed weights on memory. This enables efficient zeroth-order RL training using inference systems such as vLLM, without the need for costly backpropagation.

Critical strengths

The paper provides a novel and effective way of inference-time weight perturbation and batched computation on top of an existing production serving system. Combining such a method with the important problem of efficient RL rollout generation and enabling zeroth-order training of RL problems is interesting and highly valuable.

Critical weaknesses
  1. Zeroth-order methods are often compute and time inefficient due to their random nature. There are no analyses of GPU-hour or per-sample training efficiency results provided by the authors. This raises questions about whether the suggested method is a practical alternative to the traditional backpropagation-based training methods.

  2. It is unclear which training scenario this method is targeted at. The introduction says it is for inference systems without the resources required to support costly backpropagation, such as on local devices fine-tuning on private data. However, the method requires generating thousands (lambda=512 or 2048, 8192 rollouts) of traces and scoring them to provide feedback to the weight update. It is unclear if such a scale of rollout generation and scoring infrastructure would be feasible on local devices and private data. For idling inference infrastructures on the cloud, traditional RL frameworks such as verl can still utilize the infrastructure for their compute-intensive rollouts. The paper's method has real benefits in that it has smaller resource requirements, but it is hard to know if that would lead to real end-to-end benefits in time or compute use without the missing compute and sample-efficiency analysis.

Questions for authors
  1. Can you please provide the total number of candidate traces and the number of generated tokens used to achieve the results in Table 2 for both GRPO (verl) and NoisyGEMM?

  2. Can you please provide the total FLOPs and GPU time spent to achieve the results in Table 2, for both GRPO (verl) and NoisyGEMM?

  3. In Figure 3, what were GRPO (verl)’s group size, prompts per step, step count, and learning rate, and how were they tuned? Are the number of generated tokens for GRPO (verl) and NoisyGEMM per step equivalent to each other?

  4. What kind of training and validation dataset did you use for the experiments? Does the training set contain content from the evaluation benchmarks used in Table 2 in any way?

  5. What kind of use case is this method targeting? Is it for replacing the general RL training step during post-training? Is it for user-specific model updates in batched user serving in large-scale inference providers? Or is it for personalized finetuning of on-device models using private data?

2 comments
AuthorsOpenReview ↗

Response to Reviewer pbiU

Thank you for the incisive questions on sample/FLOP accounting, baseline setup, overlap checks, and deployment scope.

1. Traces, tokens, FLOPs, and GPU time (W1, Q1, Q2; §5.4, Table 3). Node-hours and generated tokens are in the General Response table. To reach GRPO's converged accuracy (2,048 rollouts/step for GRPO vs. 8,192 for NoisyGEMM), it takes GRPO and NoisyGEMM 160 vs. 100 steps (0.33M vs. 0.82M traces) on 1.7B, 180 vs. 80 steps (0.37M vs. 0.66M traces) on 4B, and 300 vs. 120 steps (0.61M vs. 0.98M traces) on 30B-A3B, while NoisyGEMM reaches its higher Table 2 peak at 140 / 140 / 160 steps (1.15M–1.31M traces). Therefore, although GRPO only generates 1/4 rollouts per step (2,048 vs 8,192), it requires more steps to reach the same accuracy as CMA-ES. We also tuned GRPO's batch size and rollouts per question independently to have the total rollouts per step to match NoisyGEMM, but in each configuration, CMA-ES (NoisyGEMM) requires less GPU hours to reach the same performance as GRPO on Qwen3-4B/Qwen3-30B-A3B.

At ≈2P_active forward FLOPs/token (3.4, 8.0, 6.1 GFLOPs/tok on 1.7B, 4B, 30B-A3B), one 16k math step costs 76M tokens / 2.6×10¹⁷ FLOPs on 1.7B and 71M tokens / 5.7×10¹⁷ FLOPs on 4B. Because NoisyGEMM runs forward passes only (1×) while GRPO adds policy log-prob and backward passes (≈5× FLOPs/token; §5.4), multiplying by Table 3's token totals shows NoisyGEMM (@ GRPO) uses fewer total FLOPs at every scale (2.4×10¹⁹ vs. 5.6×10¹⁹ on 1.7B; 4.2×10¹⁹ vs. 1.2×10²⁰ on 4B; 4.6×10¹⁹ vs. 1.8×10²⁰ on 30B-A3B).

2. Fig. 3a GRPO (verl) configuration (Q3). In Fig. 3a (Qwen3-4B, 8k cap, 100 steps), NoisyGEMM uses 1024 × 8 = 8,192 rollouts/step (56.6M tokens/step). GRPO uses 128 prompts × 16 rollouts = 2,048 rollouts/step (14.8M tokens/step), learning rate 3e-5, and no KL penalty, tuned over batch sizes {128, 256} × group sizes {8, 16, 32}. Per-step tokens in Fig. 3a are therefore 4× smaller for GRPO, whereas we give GRPO up to 4× the step budget to match the total tokens used in CMA-ES (NoisyGEMM). Table 2 and 3 reports the accuracy and node-h when GRPO and CMA-ES reach their best validation performance.

3. Datasets and overlap check (Q4; Appendix H). We train on OpenR1-Math-220k, DAPO-Math-17k, and NuminaMath-CoT for math, and Nemotron and Eurus-2-RL for coding, holding out 10k math and 2k coding questions for validation and checkpoint selection. An exact-match and 8-gram check across the 93 math test problems did find one overlapping problem (AIME26 Problem 1; none on AIME25 or HMMT26). Since all runs, including GRPO, share the same training pool, this single overlap should not materially affect our conclusions.

4. Practical deployment regimes vs. verl (W2, Q5). Indeed, 8,192 × 16k math/coding RL is a datacenter workload; however, as a proof of concept, the efficiency advantage of NoisyGEMM on training large models using only decodings opens up the potential of small-population on-device personalization, and we leave this to future work (§6). In datacenters, NoisyGEMM addresses three regimes where verl struggles:

  • Live-serving co-location and batched user-specific updates (Appendix K, Table 5). verl requires activation/optimizer memory and weight broadcasts that interrupt serving; NoisyGEMM keeps weights frozen, adds ≤0.25 GB, and co-runs 16k rollouts alongside live traffic—while each user/task state is just D scalars (86–147 KB), allowing one serving batch to mix multiple user-specific states.
  • Quantized / Ethernet serving fleets (Appendix B, Fig. 8). NoisyGEMM can train directly on quantized weights and exchanges kilobytes (344 KB vs. 3.4 GB gradients), scaling 25.7× across 32 Ethernet nodes where verl degrades past 8 nodes.
  • Standalone efficiency at larger scale (Table 3). Once backward passes dominate at 4B and 30B-A3B, NoisyGEMM reaches GRPO's accuracy in 90% and 36% of the node-hours, respectively.

To summarize, idle inference servers can generate rollouts, but not complete a training step (W2). We agree with the reviewer that rollout generation is just inference and can run on idle inference GPUs, for any RL method including verl. The distinction is the update. verl's rollouts are only useful once shipped to a separate trainer that runs the backward pass and optimizer, after which the new policy must be broadcast back to every rollout worker every step (the policy changes each step). So "idle inference infrastructure" buys verl only the rollout phase; it still needs dedicated training hardware and a per-step weight-sync channel, and on one 80 GB A100 that update does not fit beside an active engine.

AuthorsOpenReview ↗

Thank you for your feedback

Dear Reviewer,

Thank you again for your appreciation of our work and your valuable feedback. We have provided a comprehensive response to your questions, especially with a detailed GPU-hr comparison between GRPO and our NoisyGEMM. We'd very appreciate it if you could acknowledge our response if it has fully addressed your concerns. If you have further questions, feel free to follow up and we will do our best to respond/address them. Thank you!

Reviewer_GGd4OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This submission addresses the challenge of performing RL post-training on LLMs, in a setting where inference and finetuning are constrained to the same serving system. Taking into account hardware (memory) limitations, backprop-free zeroth-order optimisation methods are considered (e.g. MeZO, Evolution Streategies etc); however, these come with their own system bottlenecks, such as the identified noise materialisation problem, as such methods perturb weights multiple times per candidate, to evaluate numerous variants sequentially, sacrificing batching-based efficiency. To solve this, the proposed NoisyGEMM introduces a fused MM kernel that evaluates candiate-specific perturbations on the fly, without materialising weight perturbations on DRAM memory. More specifically, by adopting blockwise Fast Walsh–Hadamard Transforms and reusing transformed vectors across output blocks, NoisyGEMM computes the perturbation correction in registers/shared memory and fuses it into the base GEMM epilogue; paired with a tailored separable optimiser augmented with group-relative fitness normalization and score-weighted momentum for Reinforcement Learning with Verifiable Rewards.

Critical strengths
  • The proposed method approaches the target problem for a synergistic perspective, considering both the hardware constraints of GPU MM kernels and the mathematical structure of the optimiser. The proposed solution is insightful and introduces tailored optimisations, such as the shared randomised Sylvester–Hadamard bases simultaneously eliminatinh DRAM materialisation, slashing PRNG overhead via single-bit signs, reducing projection complexity via FWHT + blockwise output reuse, and creating a shared coordinate frame that makes diagonal covariance adaptation tractable.
  • The effectiveness of the proposed approach is backed by compelling empirical evidence, demonstrating that a zeroth-order method can improve competitive reasoning models on difficult, long-horizon benchmarks, significantly advancing the SoTA for gradient-free LMM post-training.
  • By "piggybacking" on VLLM's existing PagedAttention KV-Cache manager, continuous batching scheduler, and per-request adapter tagging, the system avoids requiring customised training solutions, resulting to highly-scalable performance and applicability.
Critical weaknesses
  • The costs associated to the required approach (compared to baselines) are not adequately quantified. More specifically, while NoisyGEMM outperforms GRPO in final accuracy, the manuscript does not report total (wall-clock) training time or total generated tokens required to reach those numbers for NoisyGEMM versus the adopted baseline (GRPO).
  • The provided live-serving scenario experiments are not convincing, as the training completion is capped at just 64 output tokens, while most reasoning settings used a much larger completion length. This is particularly impactful in the context of serving systems (such as the adopted VLLM inference engine) where longer sequencies can result into vast KV-Cache expansion, shifting the inference speed/memory bottlenecks.
Questions for authors

Please, consider replying to the concerns raised in the Weaknesses section.

4 comments
AuthorsOpenReview ↗

Response to Reviewer GGd4

Thank you for focusing on end-to-end training cost and KV-cache behavior under 16k live serving.

1. End-to-end training hours and tokens vs. GRPO (§5.4, Table 3). Please see the General Response table for full numbers across 8×A100 and 8×B200. In summary, GRPO generates fewer total tokens in each model size (3.1B–5.9B vs. 5.0B–7.5B at @ GRPO), but backward passes and weight sync make NoisyGEMM faster in wall-clock time on 4B (127 vs. 141 node-h) and 30B-A3B (211 vs. 589 node-h, 36%).

2. Co-locating 16k-token rollouts with live traffic (§5.6, Appendix K / Table 5). On Qwen3-4B (one 80 GB A100), we replayed 15% of BurstGPT day-9 at 24× compression (1 h, 22,537 priority live requests with length-matched synthetic tokens) alongside Table 2's 16k math rollouts (1024 × 8). To isolate kernel cost from continuous-batching KV-cache sharing on the same engine, each NoisyGEMM run is paired with an unperturbed plain-inference control running the identical background requests:

Condition (Qwen3-4B, 16k bg. rollouts)Live TPOT P50P99Live TTFT P50P99Live tok/sBg. tok/sCompl.GPU util.
Live traffic only12.7 ms71.9 ms43 ms294 ms4010—55%
Adaptive (≤32), run 1 / 2: plain control22.8 / 22.578.0 / 80.487 / 85396 / 400400 / 400833 / 837213 / 21499%
Adaptive (≤32), run 1 / 2: NoisyGEMM22.1 / 21.682.2 / 83.181 / 74397 / 400400 / 400848 / 843208 / 20199%
Fixed 8 rollouts: plain control / NoisyGEMM37.8 / 38.5115.8 / 115.9111 / 113611 / 611398 / 398398 / 400169 / 16899%
  • Perturbations add no serving latency. NoisyGEMM matches its control within 1 ms P50 TPOT, 5 ms P99 TPOT, and 2% throughput, holding only 0.25 GB of candidate state—whereas 16 GB of materialized rank-16 adapters displace KV cache at 16k and cut batch concurrency from 89 to 67 sequences (Table 4). (A second-half arrival burst pushes live-only P99 TPOT to 71.9 ms above the 50 ms controller target; MeZO-style mutates weights in place and cannot serve live traffic concurrently.)
  • Shedding trade-off. Adaptive shedding harvests 843–848 bg tok/s (72% of a dedicated GPU), with 55% (460 tok/s) in completed rollouts due to burst aborts (830 in run 1). Fixed-8 avoids aborts (400 tok/s) at higher burst P99 TPOT (115.9 ms); KV-cache checkpointing would preserve partial rollouts.
AuthorsOpenReview ↗

Thank you for your feedback

Dear Reviewer,

Thank you again for your appreciation of our work and your valuable feedback. We have provided a comprehensive response to your questions, especially with a detailed GPU-hr comparison between GRPO and our NoisyGEMM. We'd very appreciate it if you could acknowledge our response if it has fully addressed your concerns. If you have further questions, feel free to follow up and we will do our best to respond/address them. Thank you!

Thank you for the thorough responses to all raised concerns, and additional results reported in the rebuttal. The findings of the end-to-end training time comparison between the proposed approach's variants and GRPO strengthen the contribution of the work, and completeness of the manuscript. I am happy to maintain my WA score.

Thank you for acknowledging our rebuttal and we are happy to hear that we addressed all raised concerns!

Reviewer_jwvqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, authors proposed a novel method to finetune LLMs on inference systems, NOISYGEMM, a gemm kernel that add structured Hadamard-basis perturbations inside the kernel from a few coefficients per candidate. The method is to leverage the kernel for training on vLLM frameowrk with CMAES.

They experimented with Qwen famlily models on six math and coding benchmarks and compared to GRPO. Result shows NOISYGEMM beats GRPO on multiple benchmarks, specificly, Perturbed layers run at 1.1-1.3x cuBLAS latency, and training is 72-73% of vLLM inference throughput.

Critical strengths
  • The major contribution of the paper is that it offers a path to solve real system problem that enables training on serving hardware at a reasonable performance and accuracy. The kernel is well designed that reduced the memory and memory traffic overhead of candidate perturbations. It evaluates perturbations "without ever materializing them in GPU" memory (Sec. 1), and perturbed layers run at "1.1×–1.3×the latency of unperturbed cuBLAS GEMM" (Abstract). Training "reaches 72–73% of unperturbed vLLM throughput" (Sec. 5.5).

  • If the experiments are valid, then the method outperforms other baselines in 18 model benchmark pairs as authors stated, and it beats GRPO "with average gains of 5.7 and 1.9 percentage points" (Sec. 5.2). The most advantage compared to Qwen3-30B-A3B is that the proposed model improves LiveCodeBench v5 and v6 by 4.1 and 4.0 points, while GRPO changes them by 0.7 and minus 0.3 (Sec. 5.2)

Critical weaknesses
  • The proposed experiment setting diverge between main and appendix. Section 5.1 says the runs use "R= 8, and λ= 512 candidates", but Appendix H says the main quality setup uses "λ= 2048 and R= 512". And section 5.2 mentioned "the 2048 candidates of each step" and "43,008 trainable coefficients for Qwen3-1.7B". The coefficient count is consistent with R=512 but does not resolve which configuration actually produced Table 2. It would be better to explicitly claim the details so readers can match the setting to the Table 2 results.

  • It is shown apple to apple comparison between GRPO and the proposed model for Table2. Table 2 is based on "averaged over three runs", where does not show variance, total rollouts, # steps, GPU hours. Another comparison is in Figure 3a, Qwen3-4B with 8192 rollouts per step, and the gap is not large "46.3% against 45.4%" (Sec. 5.4). The main throughput measurement is from another set up for Qwen3-1.7B , with at most 512 tokens per candidate in Appendix H. The paper claimed "Our efficiency results measure per-step cost, not time-to-accuracy" in Section 6. So it is hard to exclude noise from the reported wins due to settings are not statistical robust, and the cost of each experiement is not reported. From the writing side, it helps marking config and setting more clearly.

Questions for authors

Clarifications:

  • For table2, could you please confirm the parameters R and λ for table 2? Reader may be confused by section 5.1 and appendix H
  • It will strengthen the conclusion if report std over 3 runs in table 2, and total rollouts, gpu hours used. Was GRPO trained to convergence under the same budget?
2 comments
AuthorsOpenReview ↗

Response to Reviewer jwvq

Thank you for catching the parameter typos and pushing for multi-run variance and end-to-end cost.

1. R and λ in Table 2 (§3.1, §5.1, §5.2, Appendix H). All Table 2 runs use R = 512 and λ = 1024 × 8 = 8,192 rollouts/step (43,008 coefficients on 1.7B, 55,296 on 4B, 73,728 on 30B-A3B). Earlier "R = 8" and "2048 candidates" typos are fixed in the revised paper.

2. Three-run standard deviations (Table 2). NoisyGEMM leads GRPO on 17 of 18 pairs; variance is larger on 30-question AIME sets than on coding benchmarks:

Qwen3MethodAIME25AIME26HMMT26LCB v5LCB v6HumanEval+
1.7BGRPO (verl)35.2 ± 1.235.5 ± 1.624.2 ± 0.859.9 ± 0.655.9 ± 0.582.0 ± 0.4
NoisyGEMM38.1 ± 1.337.5 ± 1.027.7 ± 0.661.4 ± 0.357.8 ± 0.482.9 ± 0.2
4BGRPO (verl)56.3 ± 1.358.8 ± 1.533.7 ± 0.876.5 ± 0.772.6 ± 0.690.8 ± 0.5
NoisyGEMM59.2 ± 1.659.8 ± 1.435.2 ± 0.980.1 ± 0.874.6 ± 0.689.6 ± 0.4
30B-A3BGRPO (verl)62.6 ± 0.869.3 ± 0.739.8 ± 0.679.8 ± 0.474.6 ± 0.489.2 ± 0.3
NoisyGEMM63.5 ± 0.570.6 ± 0.640.3 ± 0.583.2 ± 0.478.9 ± 0.390.2 ± 0.3

3. Total compute, steps, rollouts, and GRPO convergence (§5.4, Table 3; Appendix H). Yes, GRPO was trained to convergence. We tuned it independently to search the best batch size (64, 128, 256), rollouts per question (8, 16, 32), and learning rate. We ultimately adopt batch size of 128 with 16 rollouts per question (2,048 rollouts/step). We gave GRPO up to 4× the step budget to match rollout capacity of our CMA-ES (8,192 rollouts/step). We select both methods' best checkpoints on a held-out split (10k math + 2k coding). The table below gives the steps (and total rollouts) for NoisyGEMM & GRPO to reach GRPO's converged accuracy, and the steps at which NoisyGEMM reaches its own Table 2 peak; full node-hours and token counts are in the General Response table.

Qwen3GRPO (to its best)NoisyGEMM (to match GRPO)NoisyGEMM (to its own Table 2 peak)
1.7B160 steps (0.33M rollouts)100 steps (0.82M)140 steps (1.15M)
4B180 steps (0.37M)80 steps (0.66M)140 steps (1.15M)
30B-A3B300 steps (0.61M)120 steps (0.98M)160 steps (1.31M)

4. Fig. 3a gap and 16k throughput (§5.5, Appendix J / Table 4). Fig. 3a evaluates with an 8k token cap (57% truncation, dropping base AIME25 from 53.5% to 35.8%); at the full 16k limit (Table 2), NoisyGEMM beats GRPO by +2.9 points (59.2 vs. 56.3). Extending Fig. 3b's 512-token microbenchmark to Table 2's 16k math workload on one A100 shows overhead shrinking with length (10% → 3% on 1.7B), which we attribute to KV-cache attention growing while the Hadamard epilogue does fixed work per token:

ModelLimitPlain vLLMNoisyGEMMEGGROLL-style (16 GB)MeZO-styleNoisyGEMM / vLLM
Qwen3-1.7B1k / 4k / 16k13,729 / 5,441 / 1,76512,313 / 5,217 / 1,7165,075 / 2,841 / 9211,805 / 1,426 / 79190% / 96% / 97%
Qwen3-4B16k1,2671,172——93%
AuthorsOpenReview ↗

Thank you for your feedback

Dear Reviewer,

Thank you again for your appreciation of our work and your valuable feedback. We have provided a comprehensive response to your questions, especially with a detailed GPU-hr comparison between GRPO and our NoisyGEMM. We'd very appreciate it if you could acknowledge our response if it has fully addressed your concerns. If you have further questions, feel free to follow up and we will do our best to respond/address them. Thank you!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Official Comment to All Reviewers and Area Chair

We thank all reviewers for their constructive feedback. Every reviewer asked for end-to-end training cost against GRPO, alongside 16k-token throughput and 16k live serving. We added all three to the revised paper:

1. Time-to-Accuracy and Token Cost (§5.4, Table 3; Appendix H). Wall-clock node-hours (×8 = GPU-hours) and generated tokens (10⁹) to reach each method's best validation checkpoint (10k math + 2k coding split), plus NoisyGEMM stopped upon matching GRPO (@ GRPO). We report the results measured on 8 x A100 below:

Qwen3MethodGenerationUpdatesOthersTotal (node-h)Token (10⁹)
1.7BGRPO (verl)561115823.3
NoisyGEMM (@ GRPO)128<0.1<0.11287.1
NoisyGEMM (max)180<0.1<0.118010.0
4BGRPO (verl)8426311413.1
NoisyGEMM (@ GRPO)127<0.1<0.11275.3
NoisyGEMM (max)222<0.1<0.12229.3
30B-A3BGRPO (verl)1642831425895.8
NoisyGEMM (@ GRPO)211<0.1<0.1211 (36%)7.5
NoisyGEMM (max)281<0.1<0.1281 (48%)10.0
  • 1.7B favors GRPO (82 vs. 128 node-h). Rollout generation dominates at 1.7B, and GRPO generates 4× fewer rollouts per step (2,048 vs. 8,192).
  • 4B and 30B-A3B favor NoisyGEMM. GRPO's gradient update grows from 13% of total time at 1.7B to 48% at 30B-A3B (72% including sync/other), whereas NoisyGEMM spends <0.1 node-h outside generation. At 30B-A3B, NoisyGEMM matches GRPO in 36% of the hours (211 vs. 589) and reaches higher peak accuracy in 48% (281 vs. 589).

2. 1k–16k System Throughput (§5.5, Appendix J / Table 4). On Table 2's math prompts on one A100, NoisyGEMM sustains 90–97% of plain vLLM throughput (1.8×–6.8× faster than materialized baselines). Therefore, using NoisyGEMM, serving a perturb-able model that can be trained with evolution strategy only adds very small overhead to the plain vLLM serving a frozen model using the default kernel.

3. 16k Live Serving (§5.6, Appendix K / Table 5). Co-locating 16k math rollouts with BurstGPT traffic on Qwen3-4B adds ≤1 ms P50 and ≤5 ms P99 TPOT over an unperturbed background-inference control while lifting GPU utilization from 55% to 99%.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.