- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes ARCH-CPTQ, a replay-free continual PTQ method that aims to preserve performance on previous domains while adapting to new calibration data. The method combines historical curvature accumulation, Riemannian-guided adaptive weighting, and curvature-based checkpoint anchoring to mitigate forgetting. Experiments demonstrate improved average performance and knowledge retention over GPTQ and CPTQ-AVG baselines.
Critical strengths
- The paper addresses replay-free continual PTQ, aiming to adapt a shared low-bit checkpoint to new domains while preserving performance on previous domains. This is a practically relevant problem that remains relatively underexplored.
- The idea of integrating residual-aware weighting, Riemannian-guided consolidation, and quantized-state anchoring into existing PTQ framework is intersting.
Critical weaknesses
- Although the paper emphasizes replay-free continual PTQ, storing historical curvature incurs substantial memory overhead (4.78 GB even with compression for LLaMA-2-7B), potentially far exceeding the storage required for a small replay buffer. Without storage-matched comparisons against replay-based methods, the practical benefits of the proposed approach remain unconvincing. Moreover, Joint-All even achieves higher average performance.
- The proposed framework mainly combines existing ideas from curvature-based continual learning, Riemannian geometry, and proximal regularization. The paper lacks substantial new optimization insights or theoretical guarantees, weakening its methodological contribution. The main results emphasize comparisons with conventional PTQ methods rather than continual PTQ baselines, making it difficult to isolate the contributions of the proposed mechanisms. The main experiments should be reorganized to prioritize comparisons with continual PTQ methods and clearly demonstrate the incremental benefits of the proposed components.
Questions for authors
- What is the primary motivation for replay-free PTQ: memory efficiency or constraints related to data privacy and retention? The paper should clarify and substantiate the practical motivation for avoiding replay.
- Given that ARCH-CPTQ requires 4.78 GB of stored curvature for LLaMA-2-7B, how does its memory footprint compare with replay-based approaches that retain a small number of calibration sequences? Will this gap widen as model size increases?
- How do the proposed curvature-history consolidation and checkpoint anchoring fundamentally differ from existing curvature-based continual learning methods, such as EWC and online structured Laplace approximations? What are the key new optimization insights beyond adapting these techniques to PTQ?
- In W4A8, ARCH-CPTQ achieves only marginal improvements over CPTQ-AVG, with slightly negative BWT. In W3A16, its current-domain accuracy is even lower than CPTQ-AVG. How do the authors explain these results, and do they indicate that the proposed mechanisms are primarily effective at extremely low bit widths?
3 comments
Thank you again for your review. As the discussion period is nearing its end, we would appreciate your thoughts on our response and the updated manuscript, particularly whether any concerns remain unresolved. We would be happy to provide further clarification. Thank you for your time and consideration.
I appreciate the authors' detailed response. The clarification helps address some of my empirical concerns.
However, my main concern regarding methodological novelty remains. The proposed mechanisms closely resemble existing curvature-based continual learning and proximal regularization techniques. While the ablations demonstrate empirical benefits, the response does not sufficiently clarify the new optimization insights beyond adapting existing ideas to PTQ. Therefore, I will maintain my original score.
Thank you for the careful review.
W1. Replay-free motivation and storage.
The objective is continual low-bit recalibration under a past-sample disposal policy. Once a domain’s calibration is complete, its samples are discarded, and the shared checkpoint must continue serving that domain. ARCH carries its reconstruction geometry forward in curvature (Sec. 2.2; App. A.1).
A small replay buffer can offer a lower storage cost when historical samples may be retained. It addresses that sample-access setting. We clarified the data-access contract and reported the cost of curvature separately so that readers can assess the performance–resource trade-off under the intended condition. Q2 gives the measurements and replay references.
W2. Methodological contribution and continual baselines.
We clarified in Sections 4–5 that solver-matched CPTQ-AVG and component controls are the primary evidence. CPTQ-AVG already preserves historical curvature and shares the GPTQ solver, representation, and fixed reference with ARCH. Their comparison directly evaluates the proposed treatment of history. Three-model AP increased from 0.4755 to 0.5006 (Table 3(b)).
Removal and mean-matched controls tested the added mechanisms. External PTQ and matched-BPW results provide complementary evidence. Q3 explains the solver-compatible objective.
Q1. Is the primary motivation memory efficiency or restricted data access?
Restricted data access is the primary motivation. For example, after BoolQ calibration samples were discarded, the PIQA update still had to preserve support for BoolQ. The checkpoint and reference supplied the model states, and historical curvature retained BoolQ’s reconstruction geometry. We clarified this operational contract in the Introduction and Section 2.2 and reported the storage cost separately in Appendix F.
Q2. How does curvature storage compare with replay, and how does model size affect it?
L2-7B packed curvature occupied 4.78 GB, compared with 28.40 GB for all-FP32 curvature, a reported 83.2% reduction (Table 20). The dense requirement scales with the sum of squared layer-input widths, and its dimensions remain fixed across updates. A small replay buffer can use less storage while maintaining access to its retained samples. Table 3(a) evaluated performance under the following access budgets:
Table 3(a) excerpt. Performance under the stated data-access budgets.
ARCH exceeded both limited-replay references and Joint-Matched using 128 current sequences. Joint-All achieved higher AP, while ARCH achieved lower AF (0.0012 versus 0.0171) and higher BWT (0.0168 versus 0.0150). We revised the interpretation to state this adaptation–retention trade-off alongside the explicit storage and data-access conditions.
Q3. What is new beyond existing curvature-based continual learning?
The contribution combines a historical reconstruction representation, two distinct reference centers, and a GPTQ-compatible update. Reconstruction measures the error W⋆−Q relative to the fixed high-precision model; anchoring measures Q−Qprev relative to the previous low-bit checkpoint. Historical input Grams assign reconstruction sensitivity to these weight changes. This gives a concrete continual-PTQ objective with separate roles for matching the reference and preserving the existing checkpoint (Secs. 2.1, 2.3, 3.4).
We reformulated the combined objective into an effective target–curvature pair consumed by the existing GPTQ column-wise solver (Eqs. 7–8; App. B.5). Residual weighting emphasizes current units with larger reconstruction errors, adaptive mixing adjusts the current–history balance, and anchoring regulates changes along historically sensitive directions.
Each component improved AP in the matched removal comparisons, and adaptive coefficients retained a 0.58-point AP advantage over mean-matched fixed values (Tables 3(b), 14). The derivation supplies the solver connection, and these controls demonstrate the value of the proposed history-guided update.
Q4. Are the benefits confined to extremely low precision?
The benefits extended to W3A16, W4A16, and W4A4, where mean AP improved by 0.66, 0.42, and 0.72 points, respectively. W2A16 achieved the largest gain of 2.16 points. W4A8 gained 0.08 points, while BWT was −0.0003; W3A16 traded 0.13 points of CA for improved AP and BWT (Table 6).
Because effect sizes and acquisition–retention trade-offs varied across precisions, we revised Section 5.2 to emphasize mean sequence-wide improvement across the tested settings, with the strongest benefit at W2A16.