acceptodds
Under review as a conference paper at ICLR 2027

The Precision Cliff: An Information-Theoretic Lower Bound on Quantization for Iterative Compositional Retrieval

Abstract

Low-bit quantization degrades multi-hop reasoning faster than single-hop recall, yet no theory has said why deeper reasoning needs more bits or how many more. We answer both with a single closed-form information-theoretic floor. For any post-training compression scheme whose error placement is independent of the stored content and whose distortion is deterministic—conditions met by GPTQ, AQLM, AWQ, and QuIP#—the minimum bits per entry for reliable -hop retrieval is with , and an explicit construction saturates this floor to within bits. A direct corollary is the separation law: deep reasoning demands twice the bit-rate of single-hop recall at capacity saturation (predicted for ; measured ). Capacity utilization transfers this bits-per-entry floor to bits-per-parameter: a synthetic sweep traces a monotone depth penalty in (Spearman , ), and four LLM families (Llama-2-7B, Qwen3-8B, Phi-4-mini, InternLM3-8B) sit on the same -curve at production scale with wall premiums of –—already forcing a – upward correction to any quantization floor calibrated at . A chain-given control (accuracy read-aloud vs. sequential lookup at 3 bits) isolates iterative retrieval as the causal mechanism, ruling out attention-routing and generation-noise alternatives.

Primary area:
learning theory
Keywords:
quantization, multi-hop reasoning, information theory, rate-distortion, precision cliff, large language models, compositional retrieval, error amplification

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 2 reviews, above 0% of the others.
Reviewer_pqoDOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies quantization on multi-hop QA and proposes an information-theoretic limit of how many bits are needed for deeper reasoning. The limit is tested across 4 models. Further summarization of the paper is limited by the incomprehensible nature of the writing.

Critical strengths
  • Providing a lower bound on quantization w.r.t. reasoning depth could potentially help inform quantization methods (though the paper does not take this second step)
Critical weaknesses
  • The paper is unreadably AI generated. Starting in the abstract, it's completely incomprehensible, to the point where it is impossible to understand or assess the method. The level of incomprehensibility makes me doubt that the authors have meaningfully evaluated the contents of the paper.
  • Leaving comprehensibility aside, the paper doesn't take the next step of proposing any improvements or motivating why the lower bound they claim to introduce is valuable. The main claim seems to be: deep reasoning requires more bit-rate than recall. This was known empirically before: what does the lower bound introduced here buy us?
Questions for authors

None

Reviewer_VpzzOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

In one sentence: every hop in a chain of reasoning is one more chance for a quantization error to slip in, so deeper reasoning needs more bits. The paper tries to derive a formula for how many.

Problem:

Quantization makes an LLM smaller by storing each number in it with fewer bits, for example 3 instead of 16. A 3-bit model can often still answer a one-step question like "Who directed Inception?". It fails more often on a question that chains two facts, like "Who is the spouse of the director of Inception?". The paper asks why, and how many more bits longer chains need.

Running example: think of the model's knowledge as a list of N people in which each person points to exactly one other person and each person is pointed to by exactly one, such as Alice → Bob, Bob → Carol, Carol → Dave. A T-hop question says "start at Alice and follow T arrows". Say the 3-bit model gets 10% of the arrows wrong. One hop is then right 90% of the time. Five hops need five right arrows in a row, which happens only about 0.9 × 0.9 × 0.9 × 0.9 × 0.9 ≈ 59% of the time (assuming each arrow is wrong independently of the others).

Goal:

In the arrow example, the question is: how many bits per arrow, on average, are needed to store the list so that a T-hop question is still answered correctly at least half the time? The paper wants one formula that gives this minimum for any number of hops T and list size N. The formula should hold for any compression method, including common quantizers such as GPTQ, as long as two conditions hold: (1) which arrows go wrong does not depend on what the list says, and (2) the number of wrong arrows is fixed in advance rather than random. The paper then wants to carry this per-arrow formula over to the bits per weight of real LLMs, and to check that they behave as predicted.

Method:

Theory. Each extra hop is one more chance to hit a wrong arrow, so the longer the chain, the fewer wrong arrows the list can afford: to stay right at least half the time, up to 50% of arrows may be wrong for 1 hop, but only 13% for 5 hops. Storing a list with fewer mistakes takes more bits, and the paper uses a formula from information theory (rate-distortion theory) to turn "how many mistakes are allowed" into "how many bits are needed". For a list of 4096 people, this gives about 5 bits per arrow for 1-hop questions and nearly 12 for very long chains, a gap of about 2× that the paper calls the "2× separation law".

Experiments. First, the paper trains small neural networks to memorize an arrow list, quantizes them, and checks how quickly their answers break as the chain gets longer. Second, it quantizes four real LLMs to several bit-widths, gives each a 16- or 32-line arrow list in the prompt, and finds the fewest bits at which the model still answers correctly half the time, once for 1-hop and once for 5-hop questions. It then asks whether the gap between those two numbers depends on how "full" the model is. The paper measures this with η: the amount of knowledge stored, divided by the model's total storage space (number of weights × bits per weight at full precision). η = 1 means every bit is used to store knowledge; a small η means lots of spare room, which the paper argues lets the model absorb errors and shrinks the gap.

Results:

What they found. In both the small networks and the four LLMs, longer chains need more bits: the fewest bits that keep 5-hop questions right half the time is about 1.05–1.56× what 1-hop questions need.

Does it match the theory? Mostly not: the theory gets the direction right but overpredicts how many extra bits longer chains need. For 16-line lists it predicts that 5-hop questions need 2.81× the bits of 1-hop questions, yet the measured ratio is only 1.14× for Qwen3, 1.06× for Phi-4-mini, and at most 1.56× in the small networks.

Difference between this paper and the closest baseline:

Closest work: Chakrabarti, Pitassi & Alman, "Every Bit Counts: A Theoretical Study of Precision-Expressivity Tradeoffs in Quantized Transformers" (arXiv:2602.02707). It proves that, at every precision level, a one-layer transformer can solve a certain equality-checking task with p bits but not with p − 1 bits. This paper instead asks how the number of bits needed grows with the number of reasoning hops, for any compression method rather than one architecture.

Critical strengths

S1: The pointer-chasing setup is easy to understand. Each entity points to exactly one other, and a T-hop question follows T pointers. This simple, intuitive model of multi-hop reasoning makes it easy to see why errors compound across hops.

Why it matters: A clean toy problem like this makes the question of how bit-width interacts with reasoning depth concrete and testable.

Critical weaknesses

W1: The main formula (Theorem 2) is mathematically wrong. It uses a formula for lists where each arrow can point to anyone, but in the paper's list each person is pointed to exactly once, which needs fewer bits. So simply storing the list exactly beats the paper's "minimum": for 4096 people this takes 10.56 bits per arrow, while the paper says long chains need up to 11.98.

Why it matters: The paper's two main claims, the minimum-bits formula and the predicted 2.40× gap, both rest on this formula.

W2: Rate-distortion theory cannot tell us how often a quantized model gets a fact wrong. Rate-distortion theory answers one question: if you store the list itself with a given number of bits per entry, how many entries must come out wrong? A quantized model does not store the list itself, so knowing its bits per weight says nothing direct about how many facts it gets wrong. The two are likely correlated, but using rate-distortion theory to precisely bound the error rate is impractical.

Why it matters: This link is the foundation of the paper. Without it, the rest of the paper does not hold.

W3: The proofs are incomplete and not reliable as written. The paper states that AI assistants "contributed to deriving and proving the paper's mathematical claims", and several of these proofs have gaps:

  • Theorem 3 (the matching upper bound) has no proof. Its only support is a "measurement" in §6.1 that appears to evaluate the same formula numerically.
  • Theorem 2's proof is two lines. It drops the error terms of Lemma 1, and the theorem itself states its key quantity only approximately ("δ_T ≈ 1 − 2^(−1/T)").
  • In Lemma 1 (Appendix B), the key monotonicity step is argued informally, and a constant claimed to be below 2 is described as "empirically ≤ 2.4".
  • Appendix D's argument that GPTQ, AQLM and AWQ satisfy conditions (C1)–(C2) consists of three informal bullet points.

No machine-checked proof (e.g., in Lean) or other independent verification is provided.

Why it matters: The contribution is mainly theoretical. W1 already shows a wrong result, so the remaining proofs need to be complete and checkable before they can be relied on.

W4: The experimental results are far from what the theory predicts. The theory says 5-hop questions should need about 2.8 times as many bits as 1-hop questions. In the experiments, the real models need only about 1.1 times as many, and even the best case reaches only about 1.5 times.

Why it matters: The experiments are meant to confirm the theory, but none of them comes close to its predictions.

W5: There are no results on real multi-hop reasoning benchmarks. The only LLM task is a self-made pointer-chasing prompt with 16 or 32 lines and at most 5 hops. No numbers are reported on standard multi-hop benchmarks, such as:

  • HotpotQA
  • 2WikiMultiHopQA
  • BrowseComp-Plus

Why it matters: Without these, there is no evidence that the measured depth penalty carries over to how quantized models actually perform on multi-hop reasoning. Results on a small self-made task are not enough to support claims about real multi-hop reasoning.

Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.