acceptodds
Under review as a conference paper at ICLR 2027

Jumping the Line: Exploiting Length Predictions in LLM Scheduling

Abstract

With the proliferation of large language model (LLM) deployments efficient request scheduling has become increasingly important for reducing completion time and improving user experience. Size-based scheduling policies, such as Shortest Job First, reduce average completion time by prioritizing shorter requests, but require knowledge or estimates of request sizes. In LLM inference, output lengths, and therefore request sizes, are unknown in advance. As a result, LLM schedulers rely on predicted output lengths to guide scheduling decisions. This work proposes JIL, an attack that exposes a vulnerability in such predictionbased LLM schedulers. By manipulating the scheduler’s length prediction, an adversary can cause its requests to receive higher scheduling priority and reduce their completion time. As a case study, we consider TRAIL, a prediction-based LLM scheduling system that uses a lightweight probe to estimate output length. JIL exploits this mechanism by optimizing adversarial suffixes that, when appended to a request, cause the probe to underestimate its output length, thereby increasing the request’s scheduling priority. We evaluate JIL on two datasets and four LLMs across varied request profiles and deployment configurations. Our results show that JIL successfullymanipulates the scheduling signal, reducing predicted output lengths by up to 83.4%. In end-toend serving experiments, adversarial requests complete up to 1.53× faster on average. The reduction in predicted length is substantially larger than the change in actual output length, demonstrating a mismatch between the scheduler’s estimate and the request’s realized size. Response utility varies across models and tasks, sometimes revealing a trade-off between scheduling advantage and response quality. Finally, we evaluate scheduler-side defenses and show that grouping length predictions into coarse intervals reduces JIL’s scheduling advantage and mitigates delays to benign requests.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM inference, LLM serving, adversarial attacks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_junEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies an attack on prediction-based LLM schedulers. These schedulers try to estimate the output length of a request and then prioritize requests that are predicted to be shorter. The paper points out that since this prediction is computed from the user controlled prompt, it can itself become an attack surface. The authors propose JIL, which appends an adversarial suffix to a request and optimizes it using GCG in order to reduce the scheduler's predicted output length. They evaluate the attack on several models and datasets and show that it can substantially reduce the predicted length. They then evaluate the effect in a TRAIL/vLLM serving setup and show that attacked requests move earlier in the completion order and can complete faster. The paper also studies response quality, transfer between models, a better probe trained on generated response lengths, several control experiments, and defenses based on prediction floors and coarse grouping.

Critical strengths

1. Interesting and relevant problem setting. The paper identifies a meaningful vulnerability in prediction-based LLM scheduling: the scheduler relies on a signal derived from user-controlled input, which can therefore be manipulated.

2. Good experimental analysis and controls. I liked that the paper goes beyond showing that the predictor can be fooled, and studies scheduling effects, response quality, transfer, and defenses. The additional controls separating the effects of the suffix, the prediction, and the generated work make the main finding more convincing.

3. The attack remains effective against a stronger predictor. The experiment with a probe trained on model-generated lengths is particularly useful, since it shows the effect is not simply due to a weak or poorly calibrated length predictor.

Critical weaknesses

1. Limited novelty in the attack method. I find the problem setting interesting, but the attack itself is fairly straightforward: JIL applies GCG-style suffix optimization to minimize the scheduler’s length prediction. The main novelty is therefore the attack surface rather than the optimization technique itself.

2. The latency gain is partly confounded by changes in the generated output. The suffix can also change response length and quality, so part of the reported speedup may come from the request doing less work rather than only from higher scheduling priority. In the fixed-output experiments, the attack still changes scheduling order, but the completion-time gain is much smaller.

3. The threat model and serving evaluation are somewhat limited. The attacker assumes white-box access to the model and length probe and optimizes a separate suffix for each request. The end-to-end serving evaluation is also relatively small, using one serving model/GPU and five workloads, so I am not fully convinced how well the practical impact generalizes to realistic deployments.

Questions for authors

1. Novelty: Could the authors clarify what they view as the main technical novelty beyond applying GCG-style optimization to a scheduler-facing prediction? To me, the main novelty seems to be identifying the attack surface rather than the attack method itself.

2. Latency attribution: Can the authors better quantify how much of the reported completion-time speedup comes from changed scheduling priority versus changes in the actual generated output? The fixed-output experiments suggest the scheduling effect is real, but the latency gain becomes much smaller.

3. Threat model: How much of the attack survives under a weaker threat model, where the attacker does not know the exact probe or cannot optimize a separate suffix per request (universal suffix)? Also, do the authors expect the serving results to generalize beyond the relatively small set of workloads and single serving setup evaluated here?

Reviewer_JVJROpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates a security vulnerability in prediction-based LLM request scheduling. Many modern LLM serving systems rely on predicted output lengths to prioritize requests, but the predictions are vulnerable to adversarial manipulation. The authors propose an adversarial attack, JIL, that manipulates the predicted output lengths without reducing the actual output lengths. The authors also investigate response quality degradation, cross-model attack transferability, and scheduler-side defenses based on prediction floors and coarse-grained length grouping.

Critical strengths
  1. The authors bring about an interesting security problem.

  2. The paper is clearly organized and quite easy to follow.

  3. The experiments are interesting and comprehensive.

Critical weaknesses
  1. It is assumed that the attacker has white-box access to both the target LLM and its trained length prediction probe, including the gradients needed for suffix optimization. This assumption is overly strong and substantially limits the practical applicability of the proposed attack. Although the cross-model transfer experiments partially address this concern, the paper evaluates transfer only between Llama-3-8B and Mistral-7B. Moreover, the transferred suffixes are evaluated primarily through prediction manipulation rather than a complete scheduling attack on the receiving system.

  2. I don't see much technical content in this paper. The central technical approach is an application of an established adversarial optimization algorithm to an existing length predictor. The main contribution is therefore the identification and empirical demonstration of a vulnerability rather than a novel technical method.

  3. The paper evaluates prediction floors and coarse-grained grouping as defenses against JIL. These mechanisms are simple and appear to reduce the attack's scheduling advantage in some configurations. However, the experiments do not fully characterize their costs. For example, coarse grouping sacrifices prediction granularity, which may reduce the scheduling efficiency that length-based scheduling was originally designed to provide. An important missing comparison is the performance of these defenses under benign workloads, including average completion time, tail latency, throughput, and fairness.

Questions for authors

See weaknesses.

Reviewer_dG4COpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates security vulnerabilities in request scheduling systems for large language models (LLMs) that rely on output length prediction. Existing scheduling strategies—such as Shortest Job First (SJF) and Shortest Remaining Processing Time (SRPT)—typically use predicted output token lengths to estimate the execution cost of a request. The authors introduce the "Jumping the Line" (JIL) attack, which involves appending an adversarial suffix to a standard request. This causes the scheduler’s length predictor to underestimate the actual number of tokens to be generated, thereby artificially boosting the scheduling priority of the attacker’s request. Focusing primarily on the TRAIL scheduling framework, the study employs a gradient-guided discrete token optimization method to generate these adversarial suffixes. The authors evaluate the attack across four different language models and a variety of request distributions.

Critical strengths
  1. Technical novelty and innovation The paper shifts the focus of attacks from model outputs or resource consumption to the scheduler’s prediction signals, representing a novel research approach.

  2. Experimental rigor and validation Beyond evaluating length prediction errors, the paper also assesses end-to-end completion time, request completion order, latency for legitimate requests, response quality, and the effectiveness of defense mechanisms, providing a comprehensive set of experimental metrics.

  3. Clarity of presentation The threat model is clearly defined, and the relationship between predicted and actual output lengths is easy to grasp. The experiments are organized around specific research questions, ensuring a coherent overall logical flow.

Critical weaknesses
  1. Technical limitations or concerns First, JIL employs gradient-guided suffix optimization; in practical applications, this may require the attacker to have access to the parameters or gradient information of the length predictor—a strong assumption for commercial, closed-source services. Second, the optimization objective focuses primarily on reducing the predicted length without directly constraining the semantic quality of the response; consequently, the adversarial suffix may inadvertently compromise the validity of the original request. Furthermore, the paper’s experiments focus on the prediction mechanism used in TRAIL; it remains unproven whether these attacks can be transferred to systems that utilize independent predictors, dynamic length updates, or alternative scheduling strategies. Finally, a reduction in predicted length does not automatically translate into a scheduling advantage. The actual impact depends on factors such as queue composition, request contention, available concurrent slots, preemption mechanisms, and overall system load.

  2. Experimental gaps or methodological issues Although the paper evaluates prediction manipulation across four models, the end-to-end service experiments are conducted exclusively using Llama-3-8B-Instruct on a single NVIDIA B200 GPU. The service-level testing is limited to five sets of workloads—each comprising 20 benign requests and 5 target requests—which is insufficient to comprehensively represent real-world, high-concurrency service environments. The attack utilizes a fixed 20-token suffix optimized via 30 steps of a GCG-style approach; however, the paper lacks a thorough analysis of the trade-off between the computational cost of generating the attack and the resulting gains in request latency.

  3. Clarity or presentation issues The causal link between changes in predicted length and actual improvements in completion time requires more rigorous analysis. The paper should also clearly distinguish between white-box attackers and black-box attackers who are limited to accessing the service via an API. Given the relatively small scale of the end-to-end experiments, more detailed statistical confidence metrics should be provided.

  4. Missing related work or comparisons

It is recommended to include discussions on adversarial queue scheduling, fair resource allocation, and scheduling methods that are robust to prediction errors. In particular, the study should provide head-to-head comparisons with scheduling approaches that do not rely on length prediction, those that employ independent predictors, and those that dynamically update remaining length estimates.

Questions for authors

Does JIL require white-box access to the length predictor—including its parameters and gradients? Is this assumption realistic for an external attacker?

Why are constraints on response quality and actual output length not explicitly incorporated into the optimization objective?

Does JIL demonstrate transferability across length predictors with different architectures?

How does the scheduling advantage change when an attacker controls only 1%, 5%, or 10% of the requests?

Can the results obtained from the five sets of workloads be consistently replicated across a larger number of independently, randomly generated workloads?

How does the computational cost required to generate adversarial suffixes compare to the actual latency gains achieved?

Can schedulers that incorporate mechanisms such as priority aging, maximum wait-time limits, or online remaining-length correction effectively mitigate these attacks?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.