- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies an attack on prediction-based LLM schedulers. These schedulers try to estimate the output length of a request and then prioritize requests that are predicted to be shorter. The paper points out that since this prediction is computed from the user controlled prompt, it can itself become an attack surface. The authors propose JIL, which appends an adversarial suffix to a request and optimizes it using GCG in order to reduce the scheduler's predicted output length. They evaluate the attack on several models and datasets and show that it can substantially reduce the predicted length. They then evaluate the effect in a TRAIL/vLLM serving setup and show that attacked requests move earlier in the completion order and can complete faster. The paper also studies response quality, transfer between models, a better probe trained on generated response lengths, several control experiments, and defenses based on prediction floors and coarse grouping.
Critical strengths
1. Interesting and relevant problem setting. The paper identifies a meaningful vulnerability in prediction-based LLM scheduling: the scheduler relies on a signal derived from user-controlled input, which can therefore be manipulated.
2. Good experimental analysis and controls. I liked that the paper goes beyond showing that the predictor can be fooled, and studies scheduling effects, response quality, transfer, and defenses. The additional controls separating the effects of the suffix, the prediction, and the generated work make the main finding more convincing.
3. The attack remains effective against a stronger predictor. The experiment with a probe trained on model-generated lengths is particularly useful, since it shows the effect is not simply due to a weak or poorly calibrated length predictor.
Critical weaknesses
1. Limited novelty in the attack method. I find the problem setting interesting, but the attack itself is fairly straightforward: JIL applies GCG-style suffix optimization to minimize the scheduler’s length prediction. The main novelty is therefore the attack surface rather than the optimization technique itself.
2. The latency gain is partly confounded by changes in the generated output. The suffix can also change response length and quality, so part of the reported speedup may come from the request doing less work rather than only from higher scheduling priority. In the fixed-output experiments, the attack still changes scheduling order, but the completion-time gain is much smaller.
3. The threat model and serving evaluation are somewhat limited. The attacker assumes white-box access to the model and length probe and optimizes a separate suffix for each request. The end-to-end serving evaluation is also relatively small, using one serving model/GPU and five workloads, so I am not fully convinced how well the practical impact generalizes to realistic deployments.
Questions for authors
1. Novelty: Could the authors clarify what they view as the main technical novelty beyond applying GCG-style optimization to a scheduler-facing prediction? To me, the main novelty seems to be identifying the attack surface rather than the attack method itself.
2. Latency attribution: Can the authors better quantify how much of the reported completion-time speedup comes from changed scheduling priority versus changes in the actual generated output? The fixed-output experiments suggest the scheduling effect is real, but the latency gain becomes much smaller.
3. Threat model: How much of the attack survives under a weaker threat model, where the attacker does not know the exact probe or cannot optimize a separate suffix per request (universal suffix)? Also, do the authors expect the serving results to generalize beyond the relatively small set of workloads and single serving setup evaluated here?