- Rating
- 3(+1)
- Confidence
- 1(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper separates information retained before a deletion request, measured by cache size , from information delivered afterward, measured by communication . It characterizes ambiguity in exact retraining targets when curvature is missing, studies partial-curvature interfaces, and distinguishes the information required for one realized request from the state needed to anticipate multiple possible requests.
Critical strengths
-
The paper is generally well written, with a clear problem formulation and an organized presentation of the theoretical results and empirical evaluations.
-
The problem setup is specified, explicitly distinguishing pre-request storage from post-request communication and clarifying the information available at each stage.
Critical weaknesses
-
Theorem 3.3 and Appendix Theorem C.3 establish matching leading-order bounds at the endpoints, but leave a gap at storage–communication tradeoffs. Thus, the claimed “exact frontier” is not right.
-
Corollary 3.2 assumes block-diagonal curvature and exact partial-curvature information. Its dimensional savings therefore have limited direct applicability to practical curvature caches with coupled directions.
-
The controlled partial-curvature experiment primarily verifies the prescribed achievable construction. Although natural-data witnesses support the ambiguity premise, the experiments provide limited evidence that the worst-case bounds predict practical storage–communication tradeoffs.
Questions for authors
Does the amortization calculation assume independent requests against the same initial model or sequential, cumulative deletions?
2 comments
thanks for your reply. it mainly solve my concern, and i will raise my score
Response to Reviewer tgEc
Thank you for the review and the question about amortization. The table summarizes our answers.
Corollary 3.2 requires block-diagonal curvature and exact partial information. Its exact (d−s) coefficient cannot be transferred to approximate caches for coupled natural Hessians. Appendix D.5 already treats the nonquadratic measurements separately from a direct test of that corollary. The natural-data reweighting witnesses establish local target ambiguity; their candidate-limited packing does not establish a worst-case coefficient for arbitrary neural models.
The amortization study uses independent deletion requests against the same frozen reference model and cache, as specified in Appendix D.3 and the neural restart protocol in D.7. Under a common request interface, M+TB accounts for one-time state placement and repeated additional messages; comparisons across interfaces must also include Q. These results do not model cumulative deletions.
Sequential deletion changes the remaining data, retraining target, and possibly the cache after each request. Its accounting must include state growth, request bytes, additional messages, cache-refresh delivery, and refresh computation. The fixed-reference break-even horizon therefore applies to repeated requests against a frozen reference, not automatically to a cumulative-deletion service.
The revision adds a serializer-level run that charges refresh delivery and refresh computation (Appendix D.9). It uses d=100, r=10, state rate RM=300, and ε=10−3. The serialized cache state is 392 B, and each request costs 86.4 B with the cache and 129.8 B without it. The cache is refreshed every 25 requests, and each refresh takes 0.41 ± 0.03 ms over five traces.
The cache pays for itself after ⌈392/(129.8−86.4)⌉=10 requests. In this run, each request's message has the same size between refreshes. The run therefore prices the refresh schedule but does not model how cumulative deletions change the data between refreshes.
Please let us know whether these answers address your question. We will reply to any further point during the discussion period.