My biggest concern is the novelty of the queueing-theoretic part (§3). Once the encoder and PD service rates are known, the allocation problem is basically just balancing the throughput of two pipeline stages. The optimal split comes from maximizing the minimum of their capacities, which is a pretty straightforward result. The integer rounding and decision stability intervals also follow from this. I don't see much new queueing theory here. The math looks fine, but I feel the paper spends a lot of effort formalizing something that is already quite intuitive.
Another major issue is that the strongest experiment doesn't actually evaluate the complete proposed method. The paper spends most of §4 developing a component-wise model to estimate encoder service demand. But in the main H800 experiment (§5.3–5.4), the authors directly measure the encoder and PD rates, 57 and 14.3 requests/s/GPU, and then plug them into the allocation formula. So the headline 43.6% throughput improvement mainly shows that balancing two measured stage capacities works. It doesn't really validate the profiling method developed earlier. The actual end-to-end profiling-to-allocation experiment is only done on four RTX 4090D GPUs with a fixed visual token count, and even there the predicted capacity is 11.6% higher than what they measure. I find this gap between the proposed method and the main experiment pretty important.
I'm also not fully convinced that the detailed encoder profiling is necessary. In Appendix D, using GPU execution time alone actually gives the same correct 2:2 allocation on the RTX 4090D setup. The capacity prediction error is 14.0%, versus 11.6% for the proposed model. So after all the decomposition and regression, the allocation decision is still the same. The authors do show that using full serial wall time gives a wrong allocation, which is a good observation. But showing that wall time is a bad proxy is not the same as showing that their complicated profiling method is needed. I would really like to see a case where simpler GPU-based profiling gives a wrong allocation and their method fixes it.
The baseline comparisons are also quite weak. The 43.6% improvement is over an equal 4:4 split, but the encoder is roughly four times faster than PD on the H800 setup. Why would we expect an equal split to perform well in this case? Even a very simple rate-based allocator should pick something close to 2:6. I think the paper needs to compare against that, or against existing workload-aware provisioning methods, rather than mainly showing that equal allocation is inefficient.
The ModServe comparison in §5.1 is also a bit strange. ModServe's profiler measures GPU encoder forward time, while the authors compare its prediction against full CPU-inclusive encoder wall time. These are not the same quantities. Of course a GPU-only measurement will underestimate the full request latency. The large MAPE difference (67.54% vs. 11.76%) therefore doesn't tell us much about whether the proposed method is better at estimating serving capacity. A comparison where both methods target the same quantity would be much more useful.
I also expected more from the resource bottleneck analysis in §4. The authors divide the encoder pipeline into six components, but then find that the worker–GPU cycle is the bottleneck across the entire tested range. So in the end, the encoder demand just becomes one quadratic function of visual token count. It would be much more interesting if they showed how bottlenecks change under different workloads, for example when CPU preprocessing, GPU computation, or data transfer becomes dominant. Right now I'm not sure what we gain from modeling all these components separately. And using only visual token count may be too simple, since image format, aspect ratio, number of images, and preprocessing costs can differ even for requests with similar token counts.
There is also a gap between the queueing model and how vLLM actually works. The model assumes fixed per-instance service rates, uniform routing, FIFO servers, and no additional transfer bottlenecks. But real serving has batching, scheduling, CPU contention, cache behavior, and other interactions. The paper itself shows that per-PD throughput changes with the number of PD instances on H800. On the RTX 4090D platform, the PD rate estimate is also off by 21%. The authors acknowledge that the resource-load bound is not necessarily achievable capacity, but that makes me wonder how reliable the allocation formula would be when the workload or resource contention changes. The theory is correct under its assumptions, but these assumptions are doing quite a lot of work.
I'm less convinced by the SLO analysis (§3.4 and Appendix A.3). The TTFT guarantee assumes uniform tail bounds across the different allocations and arrival processes. But these bounds are exactly the difficult part in a real serving system, and the paper doesn't actually establish them for the H800 experiments. So the theorem feels quite conditional. The experimental SLO results are based on 120-second load waves rather than longer sustained runs. At the reported 85.8 requests/s operating point, the system completes 83.8 requests/s and 1.9% of requests are incomplete or failed at the end of the wave. The authors count these as violations, which is fair, but I would still like to know whether this allocation can sustain the same SLO over a longer period. Also, the 50% SLO improvement is based on the highest tested compliant rates, not the actual SLO capacity boundaries.
The workloads are somewhat limited too. The 4090D experiment uses a synthetic fixed-token workload, while the H800 experiment uses a repeated MMMU trace where around 95% of requests only have one image. Both use fixed 128-token outputs and Poisson arrivals, and encoder-cache misses are forced. I understand why the authors make these choices for controlled experiments, but this is a multimodal serving paper. What happens with video inputs, multiple images, variable output lengths, cache hits, or bursty traffic? These could change the encoder-to-PD demand ratio quite a lot. The paper claims the allocation can be recomputed when workload distributions change, but doesn't really test that claim. The H800 evaluation also only covers five of the seven possible nontrivial allocations, and the 1:7 configuration is not pushed into deep overload, so its saturation capacity is not fully measured. More repeats and error bars for the main throughput results would help as well. The second encoder experiment in Appendix G only tests serial latency prediction, rather than whether the provisioning decision is still correct.
Finally, the practical deployment story is not fully demonstrated. Algorithm 1 can recompute an allocation target from request logs, but there is no real runtime adaptation experiment. We don't know how often the allocation should change, what the switching overhead is, or whether the model remains accurate when traffic shifts. Appendix F argues that calibration is much cheaper than configuration search, but this is mostly an estimated cost comparison against a full allocation sweep. A smarter search method might need much less work. So I think the paper has a reasonable argument for reducing profiling cost, but it hasn't really shown the advantage in a changing serving environment.
Overall, I think the paper has some useful systems measurements, and distinguishing encoder wall time from actual resource demand is a good point. The experiments also show that a bad GPU allocation can waste a lot of throughput. But I keep coming back to the same question: what does this method do that a much simpler stage-throughput profiler and capacity-balancing rule cannot do? The paper gives quite a lot of theory and modeling, but the experiments don't clearly answer that question. This is my main reservation about the contribution.