acceptodds
Under review as a conference paper at ICLR 2027

Allocate Beyond What You See: Calibrated Resource Allocation Under Endogenous Censoring

Abstract

A deployed resource allocator censors its own training data: a job given too little memory or wall-clock is killed before finishing, so its true requirement is right-censored at the allocator’s own decision, and calibrated prediction silently breaks under this feedback. We give three results. First, because the censoring level is a known decision, an exact finite-sample, distribution-free safe-allocation rule follows from a censored-implies-exceedance completion of split conformal, valid under exchangeability alone whenever the censored fraction is below the risk level. Second, deploying such a rule closes a loop that erases the upper tail. We prove that every support-confined estimator collapses to sub-nominal coverage, including a valid one-sided safe Kaplan–Meier band, whereas a completion that allocates beyond the observed support covers every round: conservatism inside the support cannot hold coverage, and what beyond-support reveals buy back is not coverage (the completion always covers) but non-vacuity. Third, a control arm restores identifiability and, via performative prediction, contracts retraining above f* = 1−1/κ, which we measure on both a convex surrogate and the deployed gradient boosted model. On a real 336K-job high-performance-computing (HPC) trace the method restores 95% per-partition coverage where naive conformal reaches 59–84% (a 16–41% out-of-memory rate, 3–8×the 5% budget; as low as 38% per partition), and an online adaptive-conformal-inference (ACI) margin reaches eaches 95–97% on the deployed model. Deployed to general availability (80% treated, a permanent control arm above the required f∗), it holds treated out-of-memory at 0.06–0.09% across accrual windows (signature cluster-robust 95% CI far under the 5% guardrail ceiling; an absolute before/after safety result, the treated-versus-control gap unidentified on a quiet cluster), reclaiming terabytes of over-reserved memory, and reproduces a failure the theory predicts: a coverage-only margin recalibrates into the trivial allocate-the-request policy unless floored at an operating point.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Conformal prediction, endogenous censoring, distribution-free calibration, resource allocation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ubKXOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I read the paper as studying resource allocation when the allocation itself limits what demand can be observed. The authors propose calibration methods for static and adaptive settings. Beyond that, I could not work out the central problem or contribution.

Critical strengths

None that I could identify.

Critical weaknesses

The writing has every hallmark of generated text, and the result is a paper I could not engage with. Its motivation, contribution, and scope stayed out of reach, so I had no way to weigh its significance. Setting the writing aside, the formal results have substantive gaps that would justify rejection on their own. The main problems are:

  • W1. I could not identify the motivating application, the central problem, or the scope of the claims.
  • W2. The contribution list is too long and detailed, and the main contribution is lost in it.
  • W3. The abstract is far too long, and its sentences presume context the reader does not have.
  • W4. 8 formal results have substantive gaps, and 1 more needs a smaller fix.
  • W5. Several references still carry LLM drafting notes such as “confirmed,” “closest concurrent work,” and “alternative canonical reference.”
Questions for authors
  • Q1. Describe one concrete, real-world problem that this paper solves.
  • Q2. In that example, what is observed, what is censored, and what decision must be made?
  • Q3. In plain terms, what is the problem, and what is your solution?
  • Q4. Who should use your method, and why?
  • Q5. How does this work advance science?
  • Q6. How would you resolve the gaps in Theorems 1, 2, 4, and 5 and Propositions 3, 4, 5, and 6?
Reviewer_Xk5kOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies resource allocation under endogenous censoring, where a job given too little memory or runtime is killed, so its true requirement is censored at the allocator’s own decision.. The authors give a simple distribution-free safety rule. They then show that any method that only ever allocates within what it has already seen will eventually lose its guarantees, and only methods that occasionally allocate beyond observed experience remain reliable. They back this with extensive simulations, and a production deployment. The core insight is clean and the empirical work is thorough and candid.

Critical strengths
  • The problem setting is well explained and seems very practical.
  • The dichotomy is presented clearly where support-confined rules collapse while completed rule covers. The point that a more conservative within-support estimator, KM band, still collapses is a very interesting result and takeaway.
  • The computational experiments are very thorough and solid with strong baselines. The 336K-job HPC trace serves as strong empirical evidences.
  • The authors present candid limitations. The paper has four very clearly discussed limitations.
  • The paper includes useful deployment lessons and discussions that could be useful for practitioners in the field.
Critical weaknesses
  • Most formal results (e.g., Prop 3, Thms 3 & 5) are short arguments built on known conformal and performative-prediction results. The new contribution rests mainly on the framing and the impossibility result. As a result, I feel the distinction from prior work is overstated.
  • Thm 2’s almost-sure failure needs an event whose probability is astronomically small at the window sizes used, so it can’t account for the fast collapse seen in the experiments. That collapse likely comes from drift or rule-specific dynamics the theory doesn’t model.
  • The statement that coverage is maintained “up to a slack that vanishes” under retraining is asserted without definition or proof. It’s also unclear what a stable point trained on censored data actually guarantees.
  • Overall presentation is dense where many key proofs, results, and baselines sit in a long appendix. Quite a few notations in the main text are not introduced, making the main paper hard to evaluate on its own.
Questions for authors

I think it will be helpful to consider clarifying on the distinction from prior work, given that many results read like mostly a thin wrappers around known results. I believe some weaknesses could also be partially addressed, leaving out the ones that require restructuring or rewriting.

Reviewer_ETBvOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies resource allocation when the allocator censors its own training data: a job given too little memory or time is killed, so its true requirement is only known to exceed the allocation. The authors propose a conformal allocation rule that counts every killed job as an exceedance, which gives a finite-sample, distribution-free safety guarantee that is non-trivial when few jobs are killed. They then show that, in a closed loop, any rule that never allocates beyond the largest observed value gradually loses coverage, and that occasional allocations beyond this range, such as a small control group served at the user’s request, restore coverage and stabilize retraining. Experiments on a private HPC trace, two public traces, a simulator, and a live deployment support these findings.

Critical strengths
  1. The paper identifies that an allocator which kills under-provisioned jobs censors its own training data and shows the effect for both memory and runtime. It proves that rules which never allocate beyond the largest observed value lose coverage in a closed loop and that occasional allocations beyond this range restore it. Both proposed fixes (counting killed jobs as exceedances and a small control group) operate on top of an existing predictor.

  2. Experiments are comprehensive and broad. The collapse and its fix are shown on a private HPC trace, two public traces, and a simulator with four residual distributions, and the method is deployed on a production cluster in a randomized A/B test. The paper also reports the failures found during deployment and how they were fixed.

Critical weaknesses
  1. The link between theory and experiments is unclear. Experiments need not match the theory exactly, but here the proven rule is not the one evaluated. Theorem 3 guarantees coverage at every round, while the main experiments use a variant with adaptive conformal inference that guarantees coverage only on average over time. In deployment, the rule fell back to allocating the user’s request, so the authors capped its margin, giving up part of the guarantee. The paper notes these differences only briefly, so it is hard to tell which results support which theorem. Stating which guarantee each result relies on would resolve this.

  2. Statements are inconsistent or incompletely proven, which makes the theory hard to check. Proposition 3 is stated with a condition in the main text but without it in the appendix, and an appendix note refers to “correcting” its phrasing. The text claims 95% coverage on all eight partition-target cells, but the per-partition table shows six. Theorem 1 has only a one-line justification, and Proposition 6 only a proof sketch. Each issue is fixable, but together they make it unclear which results are proved as stated.

Questions for authors

Question 1: Please address the weaknesses mentioned above.

  1. Which statement of Proposition 3 is intended: the main-text version with the condition, or the appendix version without it? Please also give a full proof of Theorem 1.

Minor: 1. From my perspective, the abstract is too long; it will be better to have a concise summary as an abstract and talk about details later in the main text. Several names (dfree, Tier A/B, SC/BS) are used without definition.

  1. Leftover notes: “(confirmed)” in references [3, 4], “To be completed at submission”.

  2. I recommend the author polish the paper; I can see the current contents are not well presented. All figures are set at 60% of the text width, and their labels are hard to read; enlarging them to the full width would improve readability. Moreover, several paragraphs end with a single word on its own line, including the abstract.

Overall, I consider the paper borderline. My score of 3 reflects a slight lean towards acceptance, held with low confidence. I ask the authors to address the questions above in the rebuttal, in particular the link between the theorems and the experiments and the inconsistent or incomplete statements. I will revise my score, up or down, based on their response.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.