acceptodds
Withdrawn from ICLR 2027

Critical Points for LLM Admission

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Shared large language model (LLM) services must use limited capacity efficiently without repeatedly postponing certain workloads. Admission controllers can balance request value against service debt, the service owed but not yet delivered, by ranking requests with a weighted score and allocating resources in that order. However, fixed weight grids can repeat allocations while missing useful request orders in narrow weight intervals. We present TERRA-CP (Temporal Entitlement Ranking with Critical Points), an admission controller that searches queue-specific score crossings, where request rankings change. It combines explicit crossing enumeration for small queues with implicit search for large queues, constructs resource-feasible candidates, and selects the one with the smallest predicted maximum service debt. A welfare reserve constrains cumulative estimated utility relative to greedy allocations on each current queue. We prove coverage of the ranking-and-packing family under uncapped exact-arithmetic search and characterize fixed-grid omissions and finite-budget loss. Extensive experiments demonstrate that TERRA-CP reduces maximum cumulative deficit by 14.4% relative to a runtime-matched grid and by 29.0% relative to independently tuned, reserve-adapted drift-plus-penalty scheduling. In six-tenant vLLM experiments, it reduces completion deficit by 11.6% at moderate load and mean response time by 4.85% at high load relative to welfare-constrained token-counter scheduling.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
LLM serving, admission control, resource allocation, scheduling, parametric optimization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_yRT9OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies admission control for shared services with different request values, resource requirements, and tenant service targets. It proposes an algorithm, TERRA-CP, whose algorithm intuition makes sense, by ranking requests using an affine combination of debt-repayment density and value density. Then, they run some numerical experiments to support their algorithm.

Critical strengths

The presentation is clear.

The theoretical math model tries to describe single-instance LLM scheduling with an analytical perspective.

Critical weaknesses
  1. The model is quite general, which does not capture any LLM serving unique pattern. Therefore, if I replace LLM serving by general cloud computing, the model does not change. However, this is wrong since LLM serving has many unique patterns such as KV usage, which is not described in this paper. This makes a big gap between model and real.

  2. The admission policy is similar to single-instance scheduling. I think the authors do not compare with methods from this field. Moreover, a very strange thing is that: the authors abstractly define a term as "value", and use this value to make decisions. However, in real LLM serving, finding this kind of "value" is the hardest thing. For example, shortest first scheduling makes the predicted output length as the "value", but this may not be good. Leaving the definition as abstract and using an abstract term to do admission control, from a practitioner's perspective, I really cannot see any insight from the model and algorithm.

Questions for authors
  1. How should we define "value" in real LLM serving?

  2. What is the specific model pattern related to LLM serving?

Reviewer_ALh4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TERRA-CP is an admission controller that searches the crossings of affine request-priority scores instead of evaluating a fixed numerical grid. It packs the resulting orders into feasible allocations, uses a welfare reserve to restrict cumulative reported-utility loss, and selects candidates by projected service debt. The paper proves coverage of the uncapped ranking family and bounds finite-budget loss. Experiments compare search quality in fractional allocation and evaluate whole-request admission in vLLM.

Critical strengths
  • Many weights repeat the same allocation, while a narrow useful interval can be missed. Searching the queue's score crossings is a reasonable alternative, and it fits naturally into the debt and welfare accounting.

  • The runtime-matched comparison is well designed. Including an independently calibrated adaptive grid makes the fractional-allocation result more persuasive, since the improvement cannot simply be explained by choosing a wider numerical search range.

  • The finite-budget counterexample is useful, as is the distinction between searching the ranking family and finding the best feasible allocation. The oracle comparison makes that distinction concrete rather than leaving it only in the theorem assumptions.

Critical weaknesses
  • The main enumeration observation is already explicit in Holzhauser and Krumke's parametric knapsack work: greedy orders stay fixed between pairwise score crossings. I see the novelty mainly in the budgeted admission controller and debt objective. The related-work discussion should acknowledge this closer overlap when explaining the theoretical contribution.

  • The fractional-allocation gains do not carry over consistently to whole requests. On the native queue states, TERRA-CP and Dense64 choose the same request sets in 629 of 640 cases. In the larger tenant study, guarded VTC is cheaper and has better service point estimates, although the service differences remain uncertain.

  • The controller repairs admission debt, while the primary serving metric credits completed responses on a different clock. Request lengths can therefore change how admission progress translates into delivered service. The paper explains the distinction, but does not establish when optimizing the former is an effective surrogate for the latter.

Questions for authors
  1. What queue conditions leave enough reserve-safe alternatives for critical-point search to improve completed service? Relating the existing window-level gains to those conditions would make the intended operating regime clearer.

  2. How are admission targets mapped to the external completion-reference rates?

  3. At larger queue sizes, how frequently must the controller run and how many requests does each decision admit?

Reviewer_ibH7OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents an admission controller for shared LLM services that limits the "service debt" of each workload. The admission controller searches queue-specific score crossings, based on which decisions are made. There are some theoretical results showing cumulative welfare preservation and some guarantees of the proposed algorithm. Experiments are presented, which show the advantage of the proposed algorithm as seen in the performance metrics.

Critical strengths
  • The problem of serving LLMs and deciding which workloads to admit is generally an important problem in practice.
  • The paper presents some theoretical guarantees and some experimental results showing the performance of the proposed algorithm.
Critical weaknesses
  • The paper is very hard to read and understand.
  • Based on what I could understand, the problem studied seems to be a generic scheduling / resource allocation problem. Although the experiments run on LLM inference traces, important characteristics of LLM serving such as dynamic batching, KV caching, are not considered, which significantly limits the practicality of this work.
  • Several mathematical notations such as git,kit,ℓitg^t_i , k^t_i , \ell^t_i at the beginning of Section 3 are not explained when they first appear, making it difficult to understand what they mean.
  • The experiments seem to be limited in terms of comparison with baselines and some important details are missing. For example, it is unclear what Dense136 means. Among the baselines that have origins like DPP, DRF, and VTC, they are classical scheduling algorithms. More modern algorithms used in LLM serving systems should be compared against but the current work has not done so.
  • Even if the proposed algorithm has merits from the perspective of the job scheduling field, its relevance to LLMs and AI/ML in general is quite weak, so this paper may not be a good fit for ICLR.
Questions for authors
  • Are important characteristics of LLM serving such as dynamic batching, KV caching considered in this work?
  • Can you explain all the mathematical notations and terms like Dense136?
  • Did you run experiments with real LLMs or just with data traces of LLM inference published by others?
  • How is the proposed algorithm specific to AI/ML besides being a generic scheduling algorithm?
2 comments

Thanks for the responses. I still don't see how this problem is different from a generic scheduling problem. The answer given in the authors' rebuttal seems to be about implementation details, while the overall problem framework and solution are still the same as any scheduling problem that has been widely studied.

AuthorsOpenReview ↗

The algorithmic contribution to ML inference infrastructure

Thank you for clarifying. We agree that the problem admits a general scheduling formulation. We would distinguish the generality of that formulation from the contribution of the proposed solution: our work develops and analyzes a candidate-search algorithm, rather than merely implementing an existing scheduling rule in an LLM engine.

Specifically, the controller searches for admissions that repair tenant debt while satisfying resource constraints and a cumulative reported-welfare requirement. A fixed weight grid can repeatedly generate the same request order while missing useful orders in narrow weight intervals. TERRA-CP replaces fixed numerical sampling with queue-dependent critical-point search. We prove that every finite grid fixed in advance misses a strictly better reserve-feasible debt-repair decision in some reachable state, and establish a tight one-step loss bound for limiting the search budget. Uncapped exact-arithmetic search covers the allocation family induced by the specified ranking and packing rules. These results characterize what the search can recover and what a finite budget can lose.

The inference-service objective is to honor application priorities while repairing accumulated service shortfalls among tenants sharing a generative model. Our matched-time comparison isolates candidate search: TERRA-CP reduces deficit by 14.4% relative to the grid baseline under identical packing, selection, and welfare rules. Separate live-generation experiments evaluate the resulting controller and show lower completion deficit at moderate load and lower mean response time at high load relative to guarded VTC under matched admission rules.

ICLR’s scope includes ML infrastructure. We submitted this work in that context: an algorithmic improvement to admission search, with theoretical characterization and demonstrated service benefits in shared LLM execution. The formulation’s applicability beyond LLMs does not reduce the contribution to an implementation detail.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.