- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper introduces predictive memory complexity, the minimum persistent state bits needed to achieve a target prediction risk. It relates finite-memory prediction to state quantization and analyzes how memory requirements depend on dynamical stability, partial observability, and non-Gaussian distributions. It further claims a compression threshold for a stylized KV-cache model, supported by numerical illustrations and temporal-correlation measurements from GPT-2 key representations.
Critical strengths
-
The paper studies a meaningful resource question and clearly distinguishes persistent mutable memory from communication rate and fixed model parameters. The reduction to state quantization provides an interpretable connection between memory capacity and prediction accuracy.
-
The treatment of partial observability highlights a useful distinction between retaining observations and recursively representing a predictive belief.
Critical weaknesses
-
The exact KV-cache threshold is not established by the presented proof. Equation (20) gives a distortion lower bound, but Equation (21) treats the corresponding normalized gap as an equality without matching achievability. This supports, at most, a necessary condition for recovering half of the predictive improvement. The abstract and Theorem 5 also give different distortion bounds and threshold formulas. Additionally, Appendix E, Step 2 assumes that coordinate independence makes separate scalar encoding optimal without loss of generality, which is not justified. These issues affect a prominently advertised application result.
-
The empirical evidence does not directly validate the proposed cache threshold, and the presentation makes the results difficult to assess. The GPT-2 experiment only measures correlations in key representations; it does not vary cache precision or measure the resulting prediction distortion or language-model performance. Evaluation on GPT-2 124M alone also limits the scope of the empirical conclusions. Figures 2 and 3 are particularly small, with legends, axis labels, and annotations that are difficult to read at normal viewing size. Figure 3 compresses several central results into three undersized panels, substantially reducing readability.
Questions for authors
-
How is the equality in Equation (21) obtained from the lower bound in Equation (20)? For the scalar case with and , optimal Gaussian quantization gives a normalized remaining prediction gap of , whereas Equation (21) predicts . Can the authors reconcile this discrepancy and provide matching achievability?
-
Which KV-cache threshold is intended? The abstract gives , while Theorem 5 gives , independent of . Do these formulas correspond to different assumptions or operational models?
-
Does Theorem 5 apply to arbitrary joint encoders under a total memory budget, or only to coordinatewise encoders? What justifies the claim in Appendix E, Step 2 that separate encoding is optimal without loss of generality? Please also clarify whether the threshold is an exact finite-state result or an asymptotic approximation.
Overall, the inconsistencies between the formulas in the abstract, main text, and appendix made the paper difficult to follow and the validity of the claimed threshold difficult to assess. If the authors can reconcile these discrepancies, address the proof gaps, and clearly revise the corresponding claims, I would be willing to reconsider my assessment and potentially raise my score.