acceptodds
Under review as a conference paper at ICLR 2027

The Infinite-Record Limit: A Retrieval Budget That Stays Constant for the Hardest Items and Vanishes for the Median

Abstract

Long-context and memory benchmarks routinely claim that a task requires reaching deep into a record, but that claim is about a retrieval budget, and whether a budget counts as small depends on how long the record is. No evaluation measures how the required budget scales as the record grows, so a claim established at one length is silently carried to another; and because no fixed-corpus lever moves record length alone, even a measured scaling may belong to the lever rather than to the task. That confound is not hypothetical: it produced the field's most expensive scaling disagreement, between two well-known compute-optimal scaling laws for the same nominal resource. We make the scaling the estimand. Pinning the retriever, the scorer and the task and sending record length to infinity exposes a boundary function — the budget at which the unreachable share of items falls below threshold — and its exponent alpha in the unit interval: an exponent below 1 makes "a small budget" meaningful, an exponent of 1 makes it empty. On deletion-built nested records the pooled exponent is 1.011 over a 62x ladder, with the required share stuck at 0.400 to 0.417, refuting the sublinear scaling we had frozen in advance; the exponent differs at every quantile, and that distribution is the finding — the hardest few per cent of items demand a constant share of the record while the median item's demand vanishes. The shape transports to a second corpus as unlike the first as our data allows, but the constant does not, and an undeleted 10x lever reproduces neither the exponents nor their ordering, with neither version wrong: an exponent is identified only as a (task, lever, retriever) triple, and telling the two levers apart is not decidable at this benchmark's item counts, needing 12 to 54 times the items. The reporting rule with a proof behind it is to publish an exponent with the lever and the retriever that produced it; the one unconditional result is that the required share does not vanish as the record grows.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.