acceptodds
Under review as a conference paper at ICLR 2027

ARCH-CPTQ: Replay-Free Continual Post-Training Quantization with Curvature History and Anchoring

Abstract

Calibration-based post-training quantization (PTQ) can be sensitive to differences between calibration and deployment data, especially at very low precision. Adaptive quantization addresses distribution shifts, but updates based only on current reconstruction objectives may degrade accuracy on earlier domains. We therefore formalize replay-free continual PTQ (CPTQ), which retains reconstruction history while updating a shared low-bit checkpoint using unlabeled calibration data and a fixed high-precision reference. Our derivation shows that weighted quadratic reconstruction losses on fixed inputs can be represented exactly by one fixed-dimensional curvature matrix per layer, but this representation alone does not determine how to guide subsequent updates. We propose anchored reconstruction with curvature history (ARCH)-CPTQ, which emphasizes poorly reconstructed current units, balances current and historical curvature using a Riemannian discrepancy, and penalizes changes from the previous checkpoint along historically sensitive directions. Across six large language models and five precision settings, ARCH-CPTQ improves model-averaged sequence-wide average performance (AP) and retention over equal-domain curvature averaging (CPTQ-AVG) under matched solver and representation settings, with a 2.16-percentage-point AP gain in our main W2A16 (2-bit weights, 16-bit activations) setting. In this setting, all six models achieve higher final accuracy on earlier domains, on average, than their own scores immediately after each domain was first calibrated, without replay.

Primary area:
foundation or frontier models, including LLMs
Keywords:
post-training quantization, large language models, continual learning, low-bit model compression, replay-free adaptation, curvature-aware optimization

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_NdfZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ARCH-CPTQ, a replay-free continual PTQ method that aims to preserve performance on previous domains while adapting to new calibration data. The method combines historical curvature accumulation, Riemannian-guided adaptive weighting, and curvature-based checkpoint anchoring to mitigate forgetting. Experiments demonstrate improved average performance and knowledge retention over GPTQ and CPTQ-AVG baselines.

Critical strengths
  1. The paper addresses replay-free continual PTQ, aiming to adapt a shared low-bit checkpoint to new domains while preserving performance on previous domains. This is a practically relevant problem that remains relatively underexplored.
  2. The idea of integrating residual-aware weighting, Riemannian-guided consolidation, and quantized-state anchoring into existing PTQ framework is intersting.
Critical weaknesses
  1. Although the paper emphasizes replay-free continual PTQ, storing historical curvature incurs substantial memory overhead (4.78 GB even with compression for LLaMA-2-7B), potentially far exceeding the storage required for a small replay buffer. Without storage-matched comparisons against replay-based methods, the practical benefits of the proposed approach remain unconvincing. Moreover, Joint-All even achieves higher average performance.
  2. The proposed framework mainly combines existing ideas from curvature-based continual learning, Riemannian geometry, and proximal regularization. The paper lacks substantial new optimization insights or theoretical guarantees, weakening its methodological contribution. The main results emphasize comparisons with conventional PTQ methods rather than continual PTQ baselines, making it difficult to isolate the contributions of the proposed mechanisms. The main experiments should be reorganized to prioritize comparisons with continual PTQ methods and clearly demonstrate the incremental benefits of the proposed components.
Questions for authors
  1. What is the primary motivation for replay-free PTQ: memory efficiency or constraints related to data privacy and retention? The paper should clarify and substantiate the practical motivation for avoiding replay.
  2. Given that ARCH-CPTQ requires 4.78 GB of stored curvature for LLaMA-2-7B, how does its memory footprint compare with replay-based approaches that retain a small number of calibration sequences? Will this gap widen as model size increases?
  3. How do the proposed curvature-history consolidation and checkpoint anchoring fundamentally differ from existing curvature-based continual learning methods, such as EWC and online structured Laplace approximations? What are the key new optimization insights beyond adapting these techniques to PTQ?
  4. In W4A8, ARCH-CPTQ achieves only marginal improvements over CPTQ-AVG, with slightly negative BWT. In W3A16, its current-domain accuracy is even lower than CPTQ-AVG. How do the authors explain these results, and do they indicate that the proposed mechanisms are primarily effective at extremely low bit widths?
3 comments

Thank you for the careful review.

W1. Replay-free motivation and storage.

The objective is continual low-bit recalibration under a past-sample disposal policy. Once a domain’s calibration is complete, its samples are discarded, and the shared checkpoint must continue serving that domain. ARCH carries its reconstruction geometry forward in curvature (Sec. 2.2; App. A.1).

A small replay buffer can offer a lower storage cost when historical samples may be retained. It addresses that sample-access setting. We clarified the data-access contract and reported the cost of curvature separately so that readers can assess the performance–resource trade-off under the intended condition. Q2 gives the measurements and replay references.

W2. Methodological contribution and continual baselines.

We clarified in Sections 4–5 that solver-matched CPTQ-AVG and component controls are the primary evidence. CPTQ-AVG already preserves historical curvature and shares the GPTQ solver, representation, and fixed reference with ARCH. Their comparison directly evaluates the proposed treatment of history. Three-model AP increased from 0.4755 to 0.5006 (Table 3(b)).

Removal and mean-matched controls tested the added mechanisms. External PTQ and matched-BPW results provide complementary evidence. Q3 explains the solver-compatible objective.

Q1. Is the primary motivation memory efficiency or restricted data access?

Restricted data access is the primary motivation. For example, after BoolQ calibration samples were discarded, the PIQA update still had to preserve support for BoolQ. The checkpoint and reference supplied the model states, and historical curvature retained BoolQ’s reconstruction geometry. We clarified this operational contract in the Introduction and Section 2.2 and reported the storage cost separately in Appendix F.

Q2. How does curvature storage compare with replay, and how does model size affect it?

L2-7B packed curvature occupied 4.78 GB, compared with 28.40 GB for all-FP32 curvature, a reported 83.2% reduction (Table 20). The dense requirement scales with the sum of squared layer-input widths, and its dimensions remain fixed across updates. A small replay buffer can use less storage while maintaining access to its retained samples. Table 3(a) evaluated performance under the following access budgets:

MethodCalibration accessAP
Replay-16 / 64128 current + 16 / 64 past0.4879 / 0.4923
Joint-Matched128 joint0.4701
Joint-All1,024 joint sequences at final update0.5112
ARCH-CPTQ128 current only0.5006

Table 3(a) excerpt. Performance under the stated data-access budgets.

ARCH exceeded both limited-replay references and Joint-Matched using 128 current sequences. Joint-All achieved higher AP, while ARCH achieved lower AF (0.0012 versus 0.0171) and higher BWT (0.0168 versus 0.0150). We revised the interpretation to state this adaptation–retention trade-off alongside the explicit storage and data-access conditions.

Q3. What is new beyond existing curvature-based continual learning?

The contribution combines a historical reconstruction representation, two distinct reference centers, and a GPTQ-compatible update. Reconstruction measures the error W⋆−QW^{\star}-Q relative to the fixed high-precision model; anchoring measures Q−QprevQ-Q_{\mathrm{prev}} relative to the previous low-bit checkpoint. Historical input Grams assign reconstruction sensitivity to these weight changes. This gives a concrete continual-PTQ objective with separate roles for matching the reference and preserving the existing checkpoint (Secs. 2.1, 2.3, 3.4).

We reformulated the combined objective into an effective target–curvature pair consumed by the existing GPTQ column-wise solver (Eqs. 7–8; App. B.5). Residual weighting emphasizes current units with larger reconstruction errors, adaptive mixing adjusts the current–history balance, and anchoring regulates changes along historically sensitive directions.

Each component improved AP in the matched removal comparisons, and adaptive coefficients retained a 0.58-point AP advantage over mean-matched fixed values (Tables 3(b), 14). The derivation supplies the solver connection, and these controls demonstrate the value of the proposed history-guided update.

Q4. Are the benefits confined to extremely low precision?

The benefits extended to W3A16, W4A16, and W4A4, where mean AP improved by 0.66, 0.42, and 0.72 points, respectively. W2A16 achieved the largest gain of 2.16 points. W4A8 gained 0.08 points, while BWT was −0.0003; W3A16 traded 0.13 points of CA for improved AP and BWT (Table 6).

Because effect sizes and acquisition–retention trade-offs varied across precisions, we revised Section 5.2 to emphasize mean sequence-wide improvement across the tested settings, with the strongest benefit at W2A16.

Thank you again for your review. As the discussion period is nearing its end, we would appreciate your thoughts on our response and the updated manuscript, particularly whether any concerns remain unresolved. We would be happy to provide further clarification. Thank you for your time and consideration.

I appreciate the authors' detailed response. The clarification helps address some of my empirical concerns.

However, my main concern regarding methodological novelty remains. The proposed mechanisms closely resemble existing curvature-based continual learning and proximal regularization techniques. While the ablations demonstrate empirical benefits, the response does not sufficiently clarify the new optimization insights beyond adapting existing ideas to PTQ. Therefore, I will maintain my original score.

Reviewer_2gx7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses sequential, replay-free post-training quantization for LLMs: calibration domains arrive one at a time, past calibration data must be discarded, but the same low-bit checkpoint should continue to support earlier domains. The authors propose ARCH-CPTQ, which stores a per-layer accumulated input-curvature matrix as persistent state. The method (i) weights current calibration units by the reconstruction residual of the previous checkpoint, (ii) adaptively mixes current and historical curvature using a Block-AIRM Riemannian discrepancy, and (iii) adds a quantized-state anchoring penalty along historically sensitive directions. The anchored objective is reformulated into an effective target/curvature pair solvable by the shared GPTQ solver.

Critical strengths
  1. The derivation that a weighted sum of historical reconstruction losses collapses to ∥W∗−Q∥Hˉt2\|W^*-Q\|_{\bar H_t}^2, with Hˉt\bar H_t of fixed size per layer, is the conceptual backbone of the paper. It turns “replay-free continual PTQ” from a vague desideratum into a concrete, solver-compatible state representation.

  2. The authors match the GPTQ solver and representation across GPTQ, CPTQ-AVG, and ARCH-CPTQ; report six models and five precisions; include component and distance ablations; compare against limited-replay and joint references; test domain-order and longer-stream robustness; and report storage/update costs.

Critical weaknesses
  1. At W2A16 the AP gain over CPTQ-AVG is relatively large. At W4A8, however, the gain is very small, and BWT remains slightly negative. At W3A16, CA also decreases slightly. The paper’s central claim of broad superiority would be stronger if it demonstrated statistically and practically meaningful gains in these regimes.

  2. Packed curvature matches CPTQ-AVG’s persistent storage, but ARCH-CPTQ increases per-domain update time by about 17.5–21.1% and raises peak update GPU memory substantially.

Questions for authors
  1. For W4A8 and W4A4, are the AP improvements over CPTQ-AVG statistically significant under paired model-seed comparisons? Table 6 reports SDs, but the AP difference at W4A8 is very small; a paired bootstrap interval would materially affect how much weight I place on the “all five precisions” claim.

  2. How were the precision-specific coefficient ranges selected? Were they chosen on a separate development set, and if so, does that development set overlap with any reported evaluation domains? This is important for assessing test-set leakage and reproducibility.

  3. Can the authors disentangle the contributions of “using historical curvature” versus “anchoring to Qt−1Q_{t-1}” more directly? The ablation table shows that removing anchoring or adaptive mixing hurts, but CPTQ-AVG already uses historical curvature. A fuller factorial ablation would clarify whether the gains come mainly from the anchoring term, the residual weighting, or the AIRM-based mixing.

  4. The curvature state is per-layer and grows quadratically with layer input width. Can the authors provide a scaling estimate for larger models (e.g., 70B) or many more calibration updates, and discuss whether cheaper approximations (diagonal, low-rank, or fewer blocks) could preserve most of the benefit while reducing calibration memory and update time?

3 comments

Thank you for the careful review.

W1. Precision-dependent gains.

With the solver and representation matched, ARCH improved mean AP over CPTQ-AVG at all five tested precisions (Table 6):

PrecisionW2A16W3A16W4A16W4A8W4A4
AP gain (pp)+2.16+0.66+0.42+0.08+0.72

Table 6 differences: ARCH-CPTQ − CPTQ-AVG.

At W3A16, AP increased from 0.6502 to 0.6568 and BWT from −0.0037 to 0.0055, while CA decreased from 0.6534 to 0.6521. At W4A8, AP increased by 0.08 points, and BWT was −0.0003. Because the effect sizes and adaptation–retention trade-offs varied, we revised Section 5.2 to emphasize improved mean sequence-wide performance, with the strongest gain at W2A16.

W2. Calibration cost.

We separated persistent storage, peak update memory, and inference memory. ARCH and CPTQ-AVG used the same packed-curvature storage on both measured models. ARCH required 17.5–21.1% longer updates; on L2-7B, update time increased from 902.6 to 1093.2 s/domain, and peak GPU memory increased from 5.49 to 8.06 GiB (Table 20). We retained these measured costs explicitly in the revision.

The extra work occurred during recalibration. Inference used the resulting checkpoint, and the measured dense-inference GPU memory on L2-7B remained 12.94 GiB. Additional calibration computation supported stronger continual performance at the same persistent storage and measured inference footprint as CPTQ-AVG (Apps. F.1–F.2).

Q1. Were the W4A8/W4A4 gains significant under paired comparisons?

Mean AP gains were 0.08 and 0.72 percentage points at W4A8 and W4A4. Table 6 provides per-method sample SDs, while paired inference requires the variability of matched ARCH–AVG differences, which depends on the covariance between the two methods. We therefore revised the interpretation to descriptive mean improvements with precision-dependent effect sizes (Sec. 5.2; App. C).

We attached each reported paired interval to its tested comparison: Table 10 covers W2A16 mixing/anchoring removal, and Table 14 covers W2A16 operating-mean-matched fixed coefficients.

Q2. How were coefficient ranges selected, and did development and evaluation overlap?

We separated the adaptive settings and two fixed controls by purpose. Appendix A.3 documents the precision-specific adaptive ranges; W4A8 and W4A4 reused the W4A16 settings. Table 14 used empirical operating means from the adaptive runs to test whether varying the coefficients was beneficial at matched average strength.

For the fixed-EMA development search, we evaluated nine pairs on L2-7B, seed 0, using final accuracy on OpenBookQA, CommonsenseQA, and SciQ. We selected (α, β) = (0.75, 1×10−51\times10^{-5}) and transferred that pair unchanged to L3.1-8B and Q-7B. These tasks were absent from the canonical K=8 stream but included in the K=16 stream (App. C.3). This specifies the fixed-EMA selection metric, development tasks, transfer rule, and overlap. The revision presents the implemented adaptive settings and the fixed-control selection protocols separately (App. A.3; App. C.3).

Q3. How were historical state and the three component contributions separated?

We used a staged comparison: current-only GPTQ, equal-history CPTQ-AVG, and ARCH with adaptive history use. To measure each component’s contribution within ARCH, we removed that component while retaining the others and the same solver (Table 3(b)).

SettingAP
CPTQ-AVG0.4755
ARCH without residual weighting0.4859
ARCH without adaptive mixing0.4875
ARCH without anchoring0.4894
Full ARCH0.5006

Table 3(b) excerpt. Three-model W2A16 means.

Residual weighting, adaptive mixing, and anchoring contributed 1.47, 1.31, and 1.12 AP points, respectively, in these matched removal comparisons. Table 10 provided positive paired AP intervals for mixing and anchoring. Table 11 demonstrated anchoring’s quantizer effect, and Table 14 showed a 0.58-point adaptive gain at matched mean strength. Together, these controls isolate each component’s contribution within the complete method and connect the mechanism to its observed benefit.

Q4. How does the method scale to larger models and longer streams?

For uniquely stored layer matrices, dense FP32 history requires 4∑u∈Udin,u24\sum_{u\in U}d_{\mathrm{in},u}^{2} bytes (App. F.1). Doubling a layer’s input width quadruples its storage, while subsequent updates reuse the same dimensions. The state budget is therefore fixed over time for a given architecture, with computation performed at each new update.

Packed history occupied 4.78 GB on L2-7B and 7.16 GB on L3.1-8B. These measurements anchor the scaling discussion, while Table 20 reports the corresponding update costs. For a 70B model, unique layer widths determine dense-history storage; packed size and runtime also depend on the codec and execution. Diagonal and low-rank history approximations remain follow-up accuracy–cost comparisons.

Thank you again for your review. As the discussion period is nearing its end, we would appreciate your thoughts on our response and the updated manuscript, particularly whether any concerns remain unresolved. We would be happy to provide further clarification. Thank you for your time and consideration.

Thanks the author for the detailed rebuttal. While I appreciate the clarifications and the toned-down claims, the marginal benefit at W4A8 and the noticeable increase in update memory remain notable limitations. Therefore, I will maintain my current rating.

Reviewer_yznkOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies replay-free continual post-training quantization, where a shared low-bit model is recalibrated on successive unlabeled domains while retaining a fixed high-precision reference. It represents historical fixed-input quadratic reconstruction losses using accumulated layer-wise curvature matrices. The proposed ARCH-CPTQ combines reconstruction-error-based sample weighting, curvature-distance-driven mixing of current and historical information, and anchoring to the previous quantized checkpoint. The resulting objective is reformulated for a GPTQ solver.

Validation covers six LLMs and five precision settings. Ablations examine individual components, distance metrics, and coefficient schedules. The main W2A16 results report a 2.16 improvement in sequence-wide average performance over equal curvature averaging, alongside improved retention metrics.

Critical strengths
  1. The paper addresses a practically meaningful setting in which deployment workloads expand over time: new domains become relevant while previously served domains must remain supported.
  2. Accumulated layer-wise curvature provides a principled representation of historical quadratic reconstruction objectives under the fixed-input, fixed-reference assumptions, with state dimensions independent of the number of updates. The reformulation of the anchored objective into a GPTQ-compatible reconstruction problem also provides a useful connection between the proposed formulation and an established quantization solver.
Critical weaknesses
  1. AP averages performance over both previously encountered and not-yet-encountered tasks, so its improvement cannot be attributed specifically to retention. BWT also depends on arrival-time performance: Table 19 shows that part of its improvement comes from lower arrival accuracy. The predominantly aggregated results make it difficult to establish how consistently individual domains benefit and whether retention gains involve reduced initial adaptation.
  2. The exposition repeatedly describes the framework and summarizes aggregate improvements, while important implementation choices and qualifications are dispersed across the appendices. The distinction between the exact mathematical results, heuristic design choices, and empirical conclusions is harder to follow than it should be.
Questions for authors
  1. The reported median AIRM distance implies an anchoring coefficient of approximately 6.72e-5, whereas the fixed-coefficient search stops at 5e-5. Does the adaptive method retain its advantage over fixed coefficients matched to its actual operating range, particularly its average mixing and anchoring strengths?
  2. Could you provide the per-task arrival and final accuracies underlying the main W2A16 comparisons against CPTQ-AVG, together with final-checkpoint average accuracy over earlier tasks? Table 19 shows that lower arrival accuracy can contribute substantially to improved BWT.
  3. Appendix F.3 acknowledges that lossy curvature storage may violate positive semidefiniteness, which the objective reformulation assumes. How are indefinite or nearly singular restored matrices handled during effective-target construction and GPTQ optimization? Are any repairs or damping adjustments applied to ARCH-CPTQ and CPTQ-AVG?
5 comments

Thank you for the careful review.

W1. Retention versus aggregate performance.

We clarified the role of each metric in Appendix A.4. AP measures sequence-wide performance; arrival-old and final-old measure the same six earlier tasks at arrival and at the final checkpoint. CA averages all seven arrival scores. We evaluated retention using final-old accuracy together with BWT=Final-old−Arrival-old\mathrm{BWT}=\text{Final-old}-\text{Arrival-old}, giving both the achieved final level and the change from arrival.

In the three-model comparison, ARCH achieved a final-old accuracy of 0.5178 versus 0.4957 for CPTQ-AVG. This directly establishes higher final performance on the earlier tasks. Q2 decomposes its contribution to the BWT gain (Table 14).

W2. Exact results and design choices.

We organized the explanation around three roles: Eq. (2) exactly represents weighted reconstruction history on fixed inputs; Eqs. (7)–(8) convert the anchored objective into a GPTQ-compatible target–curvature pair under the stated PSD conditions; residual weighting and discrepancy-dependent schedules determine how that representation guides each update. GPTQ approximately optimizes the resulting low-bit objective (Secs. 2.3, 3.1–3.4).

The algebra establishes the representation-solver connection; the experiments establish the value of the update design. Component removals measured contributions within ARCH, Table 11 measured quantized-output sensitivity, and Table 14 demonstrated an adaptive advantage over the control with coefficients fixed at their operating means. We revised the exposition to connect each claim to its corresponding derivation or experiment.

Q1. Does the adaptive advantage remain when fixed coefficients match operating means?

Yes. ARCH retained an advantage after mean matching. We fixed α and β at each model’s empirical operating means, retained residual weighting and the remaining ARCH settings, and evaluated L2-7B, L3.1-8B, and Q-7B at W2A16 with K=8, a group size of 32, and seeds 0/1/2 (Table 14).

MethodAPBWTFinal-old
CPTQ-AVG0.47550.00230.4957
Mean-matched fixed0.49480.01120.4780
ARCH-CPTQ0.50060.01680.5178

Table 14(a), selected means. Final-old: final accuracy on six earlier tasks.

The fixed (α, β) pairs were (0.3849, 3.0682×10−53.0682\times10^{-5}), (0.4162, 3.4052×10−53.4052\times10^{-5}), and (0.3984, 3.2135×10−53.2135\times10^{-5}), respectively. ARCH improved AP by 0.58 percentage points (95% paired interval [0.22, 0.94]; nine model–seed pairs) and final-old accuracy by 3.98 points. This directly supports adapting the coefficients during the stream at matched average strength. Tables 12–13 additionally evaluated a fixed-EMA pair selected using development-task accuracy, so the two controls address both selection criteria.

Q2. Do the arrival and final scores show genuine final-performance gains?

We decomposed BWT using the same six earlier tasks. From the displayed three-model means in Table 14, Arrival-old=Final-old−BWT\text{Arrival-old}=\text{Final-old}-\mathrm{BWT} gave:

MethodArrival-oldFinal-oldBWT
CPTQ-AVG0.49340.49570.0023
ARCH-CPTQ0.50100.51780.0168
ARCH − AVG+0.0076+0.0221+0.0145

Derived from the displayed means in Table 14. Values use the 0–1 metric scale.

Of the 1.45-point BWT gain, 2.21 points came from higher final accuracy and −0.76 points from higher arrival accuracy. The dominant contribution was therefore better final performance on earlier tasks. This decomposition appears next to Table 14, with the arrival-old and final-old definitions in Appendix A.4.

Q3. How were restored curvature and damping handled?

History was decompressed and restored to FP32 before consolidation and effective-target construction; GPTQ used a damping coefficient of 0.01. Appendix A.2 specifies a direct solve for positive-definite G and a minimum-norm solution for singular PSD G. Distance preprocessing used symmetric, trace-normalized temporary blocks with eigenvalues floored at 10−410^{-4} (App. A.3). These are the respective target-solve and distance-estimation procedures.

Lossy conversion can change matrix definiteness, so we revised Appendices A.3 and F.3 to state the target-solve conditions and attach the PSD result to the exact recurrence. Table 21 assessed packing through the observed AP, AF, and BWT changes; all six model–metric comparisons met the stated 0.002 mean-degradation tolerance. The supplementary CPU reference implementation (arch_cptq.py, anchored_target) uses Cholesky for SPD G, a pseudoinverse for singular PSD G, and an explicit error for negative eigenvalues beyond numerical tolerance.

Thank you for the clarification and additional experiments. The operating-mean-matched control in Table 14 substantially addresses my previous concern about differences in average coefficient strength.

I have one remaining question about what this comparison isolates. In ARCH-CPTQ, α and β vary across both layers and updates, whereas Table 14 uses a single fixed coefficient pair for each model. Consequently, the comparison changes both layer-specific coefficient selection and temporal adaptation. Could the authors compare against a control that fixes each layer’s coefficients to that layer’s respective temporal means? This would help distinguish the benefit of adapting coefficients over time from that of assigning different coefficients to different layers.

Thank you again for your detailed responses and additional experiments.

Thank you for this suggestion. To distinguish layer-specific coefficient assignment from temporal adaptation, we added a layer-wise temporal-mean fixed control to Table 14(a,c).

For each model, seed, and layer, we fixed α and β to their respective means over the adaptive Q1–Q7 trajectory; bootstrap Q0 was unchanged. Calibration inputs, domain order, residual weighting, GPTQ settings, curvature packing, and evaluation were matched. Current curvature, accumulated history, and quantized checkpoints still updated at each step; only α and β were fixed over time.

For three models (L2-7B, L3.1-8B, Q-7B), three seeds, K=8, W2A16, and group size 32, we obtained (mean ± sample SD across seed-level model averages):

MethodAP ↑AF ↓BWT ↑CA ↑Final-old ↑
Model-wide mean-matched fixed0.4948 ± 0.00160.0046 ± 0.00110.0112 ± 0.00130.4827 ± 0.00210.4780 ± 0.0017
Layer-wise temporal-mean fixed0.4962 ± 0.00150.0035 ± 0.00100.0140 ± 0.00180.4819 ± 0.00280.4806 ± 0.0031
ARCH-CPTQ0.5006 ± 0.00120.0012 ± 0.00090.0168 ± 0.00150.4876 ± 0.00310.5178 ± 0.0033

Layer-wise fixed improved AP by +0.14 percentage points over model-wide fixed, while ARCH-CPTQ exceeded layer-wise fixed by +0.44 points (95% paired interval [+0.12, +0.76]; nine model–seed pairs). ARCH-CPTQ also had lower AF and higher BWT, CA, and Final-old accuracy. Thus, update-dependent mixing and anchoring contribute beyond fixed layer-specific strengths. The comparison is reflected in Section 5.3 and Appendix C.3.

Reviewer_Jw2vOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper formulates Replay-Free Continual Post-Training Quantization (CPTQ), aiming to sequentially recalibrate a single shared low-bit LLM checkpoint across arriving domains using unlabeled calibration data, a fixed high-precision reference model W∗W^*, the previous quantized checkpoint Qt−1Q_{t-1}, and layer-wise accumulated input Gram curvature matrices Hˉt−1\bar{H}_{t-1}. The proposed ARCH-CPTQ combines three components: (1) block-output MSE residual weighting for current calibration sequences, (2) dynamic history-current curvature mixing via a 16×1616\times16 block-diagonal Affine-Invariant Riemannian Metric (Block-AIRM) discrepancy, and (3) proximal quantized-state anchoring toward Qt−1Q_{t-1} weighted by Hˉt−1\bar{H}_{t-1}, which is algebraically reformulated into an effective target-curvature pair (W~t,Gt)(\tilde{W}_t, G_t) for standard GPTQ quantization.

Critical strengths
  1. Section 3.4 and Appendix B.5 rewrite the proximal anchoring objective with historical curvature penalty into a standard quadratic reconstruction problem with W~t=(W∗Hˉt+βtQt−1Hˉt−1)Gt†\tilde{W}_t=(W^*\bar{H}_t+\beta_tQ_{t-1}\bar{H}_{t-1})G_t^\dagger and Gt=Hˉt+βtHˉt−1G_t=\bar{H}_t+\beta_t\bar{H}_{t-1}. The algebra is concise and requires no change to GPTQ’s column-wise update logic.
  2. The paper targets old-task degradation caused by cross-domain calibration at very low bit-widths, which has practical relevance.
Critical weaknesses
  1. Motivation and Problem Setting (1) The resource accounting is inverted, and the “replay-free” setting lacks engineering justification. To avoid storing and replaying historical calibration samples, the paper persists layer-wise curvature matrices St={Hˉt(l)}l=1L\mathcal{S}_t=\{\bar{H}_t^{(l)}\}_{l=1}^L. However, each update still keeps the full FP16 reference model W∗W^*: 13.479 GB for LLaMA-2-7B and 16.070 GB for LLaMA-3.1-8B. Each domain uses only 128 unlabeled sequences of length 512, and the raw token IDs for 8 domains total less than 2 MB. In contrast, the curvature matrices require 28.40 GB and 39.19 GB in FP32, and still require 4.78 GB and 7.16 GB after lossy block-wise FP16 conversion and Zstd-9 compression, while increasing calibration update time by 17.5%∼21.1%17.5\%\sim21.1\%. Given an already resident 16 GB reference model, maintaining several GB of curvature to save less than 2 MB of calibration text is difficult to justify in practice. (2) Under a strict data-destruction assumption, the necessity of the complex modules is insufficiently supported. By Section 2.3 Eq. (2) and Appendix B.2, the baseline CPTQ-AVG, which equally accumulates input-activation Gram matrices across domains, is already equivalent to joint reconstruction over all historical data under the fixed-input assumption. Moreover, Joint-All, which directly uses historical samples, still achieves higher AP than ARCH-CPTQ in Table 3(a): 0.5112 versus 0.5006.

  2. Theoretical Derivation and Algorithm Design (1) The “fixed layer input activation” assumption ignores cumulative propagation of cross-layer quantization error. The claim that a single accumulated curvature matrix Hˉt=∑s=0tρt,sHs\bar{H}_t=\sum_{s=0}^t\rho_{t,s}H_s exactly represents the historical weighted reconstruction loss assumes that the input activation XsX_s of layer ll on old domains remains fixed in later updates. In sequential layer-wise quantization, however, after the preceding layers are updated from Qt−1(1:l−1)Q_{t-1}^{(1:l-1)} to Qt(1:l−1)Q_t^{(1:l-1)}, the true activations of old-domain data entering layer ll have shifted. The statically stored Hˉt−1(l)\bar{H}_{t-1}^{(l)} cannot reflect this cross-layer reconstruction deviation. (2) The 16×1616\times16 diagonal block truncation reduces AIRM to a heuristic approximation. Section 3.3 presents AIRM on the SPD manifold as a core theoretical contribution, but Appendix A.3 only extracts 16×1616\times16 diagonal sub-blocks, discarding over 99.6%99.6\% of off-diagonal cross-channel correlations for a 4096-dimensional hidden layer, and applies trace normalization and a 10−410^{-4} eigenvalue lower-bound truncation. The appendix acknowledges that, after preprocessing, block partitioning, and truncation, the affine invariance of AIRM no longer holds. (3) Lossy compression of curvature matrices breaks the PSD property and the pseudoinverse premise. The conclusions in Appendices B.4 and B.5 that Hˉt\bar{H}_t and Gt=Hˉt+βtHˉt−1G_t=\bar{H}_t+\beta_t\bar{H}_{t-1} remain PSD and that the pseudoinverse solution W~t\tilde{W}_t is strictly equivalent to the original anchoring objective rely on lossless exact recursion. The default configuration stores off-diagonal entries in lossy block-scaled FP16, see Appendix F.1、F.3. The decompressed symmetric matrix is not guaranteed to be PSD, which undermines the above theoretical premise. (4) The numerical scale of βmax⁡=7×10−5\beta_{\max}=7\times10^{-5} is difficult to reconcile with the significant gains on a 2-bit discrete grid.In W~t=(W∗Hˉt+βtQt−1Hˉt−1)(Hˉt+βtHˉt−1)†\tilde{W}_t=(W^{\ast}\bar{H}_t+\beta_tQ_{t-1}\bar{H}_{t-1})(\bar{H}_t+\beta_t\bar{H}_{t-1})^\dagger, the pull from Qt−1Q_{t-1} is at most about 0.0070.007\\%, while W∗W^* accounts for over 99.99399.993\\%. Under W2A16, each group of 32 weights is mapped to only 4 discrete integer codes, and Scale and Zero-point are recomputed at each update; a perturbation of this magnitude is unlikely to change rounding results. Yet Table 3(b) reports that anchoring improves AP from 0.4894 to 0.5006 and reduces AF from 0.0086 to 0.0012. The authors should explain this numerical sensitivity.

  3. Experimental Design and Data Credibility (1) AP, CA, and BWT in multiple core tables satisfy a closed-form linear identity, contradicting the full-matrix definition. The main text and Appendix A.4 define

\mathrm{CA}^{(8)}=\frac{1}{7}\sum_{k=1}^{7}R_{k,k},\quad \mathrm{BWT}^{(8)}=\frac{1}{6}\sum_{k=1}^{6}(R_{7,k}-R_{k,k}),$$ and state that CA cannot be recovered from AP and BWT. However, $$\mathrm{CA}^{(8)}+\frac{6}{7}\mathrm{BWT}^{(8)} =\frac{1}{7}\sum_{k=1}^{7}R_{7,k},$$ which is only the average accuracy of the final checkpoint $Q_7$. In tables reporting AP, BWT, and CA, including Table 1b, Table 2, Table 3, Table 6, Table 9, Table 12, Table 13, and Table 15, every row precisely satisfies $\mathrm{AP}=\mathrm{CA}+\frac{6}{7}\mathrm{BWT}$. For example, ARCH-CPTQ in Table 1b gives $0.4979+\frac{6}{7}(0.0256)=0.5198$. This suggests that the tables may report only the final-checkpoint average accuracy or that the metrics are generated by formula conversion. (2) Table 15(b) shows a contradiction between AutoRound single-model metrics and the five-model mean. The five-model mean satisfies the identity, but the deviations of AP from $\mathrm{CA}+\frac{6}{7}\mathrm{BWT}$ in the five single-model rows are nonzero, and the negative deviations of the first four models are canceled by the positive deviation of Q-14B. For Q-14B, despite $\mathrm{BWT}<0$ and $\mathrm{AF}>0$, AP is 2.01 percentage points higher than CA. This is highly unusual under continual-learning evaluation and suggests that the single-model decomposition may be assembled. (3) Random-seed standard deviations in multiple tables exhibit an unnatural deterministic relation. In Table 17, the three-seed sample standard deviations of all 36 W2A16 PPL results approximately satisfy $\mathrm{SD}\approx10^{-4}\times\mathrm{PPL}$, lacking random sampling fluctuation; Table 15 and Table 6 also contain standard deviations that decrease by fixed constants. (4) The main text contradicts Table 18(b), and key baselines are missing. The appendix states that curvature compression increases peak CPU memory and update time but leaves peak GPU memory unchanged; in Table 18(b), packed and all-FP32 for L2-7B have identical peak CPU memory, while peak GPU memory changes from 3.21 GiB to 3.72 GiB. The experiments also omit GPTAQ, Shin et al. (2025), QuaRot, SpinQuant, QuIP#, and AQLM.
Questions for authors
  1. Please provide the raw 7×77\times7 checkpoint-task accuracy matrices for the relevant core tables and explain why every row precisely satisfies AP=CA+67BWT\mathrm{AP}=\mathrm{CA}+\frac{6}{7}\mathrm{BWT}.
  2. Please explain why the single-model deviations in Table 15(b) are exactly canceled by Q-14B, and how Q-14B obtains AP 2.01 percentage points higher than CA when BWT<0\mathrm{BWT}<0 and AF>0\mathrm{AF}>0.
  3. Please provide the raw evaluation logs for the three random seeds in Table 17, Table 6, and Table 15, and explain the proportional relation between SD and PPL in Table 17.
  4. Under W2A16 with βmax⁡=7×10−5\beta_{\max}=7\times10^{-5}, please report what fraction of 2-bit integer codes is actually changed by the anchoring term, thereby producing the +1.12%+1.12\% AP improvement in Table 3(b).
5 comments

Thank you for the careful review. We added numerical verification materials to the 01_original_results_audit/ folder in the supplementary ZIP.

W1. Motivation and problem setting.

Our setting addresses continual recalibration after past calibration samples have been discarded. Each new domain extends the workloads served by one shared low-bit checkpoint. The update therefore uses current samples together with the fixed reference, the previous checkpoint, and retained curvature (Sec. 2.2; App. A.1). Replay methods address the complementary setting in which historical samples remain available.

For example, the PIQA update followed BoolQ calibration in our stream. At that point, the retained BoolQ curvature provided the directional reconstruction costs for its recorded inputs, while PIQA provided the current samples. This explains the role of curvature: the reference weights define the reconstruction target, and the retained input Gram matrix supplies the historical input geometry. Even a small replay buffer would require continued access to BoolQ samples.

Eq. (2) provides an exact representation for any specified admissible set of reconstruction weights. Equal-domain averaging is one choice within that family. ARCH uses current reconstruction errors and current–history discrepancy to choose how history guides the next update, then anchors changes to the previous low-bit checkpoint (Secs. 2.3–3.4). Thus, the identity supplies the state representation, while the controls evaluate the update strategy.

With the GPTQ solver, representation, and reference matched, three-model AP increased from 0.4755 with CPTQ-AVG to 0.5006 with ARCH. ARCH also outperformed the control using operating-mean-matched fixed coefficients by 0.58 percentage points in AP (Tables 3(b), 14). These results demonstrate the additional value of error-aware, adaptive history use. Joint-All achieved an AP of 0.5112 with access to past samples, providing the full-data-access reference.

W2. Theoretical derivation and algorithm design.

The fixed-input condition applies during candidate-weight optimization. We collected layer inputs and previous-checkpoint residual weights and held both fixed while optimizing the candidate layer weights. Eq. (2) then exactly represents reconstruction on those recorded inputs. We made this optimization scope explicit in Sections 2.3 and 3.2 and Appendix A.3, so the historical loss is clearly defined by the inputs we collected.

The stored and solver curvatures retained off-block entries. The 16×16 partition, trace normalization, and eigenvalue flooring were applied to temporary copies used for Block-AIRM distance estimation. Full curvature at the reconstruction scale was used for consolidation and the solver objective (Sec. 3.3; App. A.3). This resolves the concern that distance blocking discarded cross-block information from the reconstruction objective. Q4 reports the direct anchoring diagnostic.

W3. Experimental design and numerical verification.

Q1–Q3 provide the metric calculations, model-wise aggregation, and seed-level checks. We also verified the memory columns: L2-7B packed/all-FP32 history used 3.72/3.21 GiB peak CPU memory and 8.06/8.06 GiB peak GPU memory. The table therefore agrees with the reported CPU increase and unchanged GPU peak (Table 20(b), originally Table 18(b)).

We made solver-matched continual comparisons the primary evidence in Sections 4–5: GPTQ used current data, CPTQ-AVG accumulated equal-weight history, and ARCH changed how that history guided updates. Tables 1 and 17 report external-PTQ comparisons, and Table 4 compares native methods at matched model BPW. The revised comparison claims follow these evaluated protocols; the additional requested methods remain a further benchmarking direction.

Q1. Provide the checkpoint–task matrices and explain the AP–CA–BWT relationship.

We supplied the matrices and evaluation outputs in matrices/, evaluation_outputs/, and reports/J6_metrics.md. For K=8, AP averaged all 49 checkpoint–task entries, CA averaged seven arrival entries, and BWT averaged six final-minus-arrival differences. The seventh task arrived at the final checkpoint. Hence, Afinal=CA+67BWTA_{\mathrm{final}}=\mathrm{CA}+\frac{6}{7}\mathrm{BWT} is the final-row identity, while AP=149∑t,kRt,k\mathrm{AP}=\frac{1}{49}\sum_{t,k}R_{t,k} averages the full matrix (App. A.4).

Revised Table 1(b) reports CA of 0.4921 for CPTQ-AVG and 0.4979 for ARCH, with AP and BWT retained. Using its displayed ARCH row, 0.4979+67×0.0256≈0.5198430.4979+\frac{6}{7}\times0.0256\approx0.519843, whereas AP was 0.5198. This illustrates the distinction between the final-row mean and full-matrix AP.

Q2. Explain the AutoRound model-wise deviations and Q-14B’s AP above CA.

Table 17(b) and reports/J7_autoround.md provide the model-wise results. Q-14B had AP 0.5802, CA 0.5601, and BWT −0.0085, with a final-row mean of approximately 0.5528. AP includes intermediate checkpoints and pre-arrival scores, whereas BWT measures the change from arrival to the final checkpoint on earlier tasks. Their different averaging ranges explain why AP can be above CA while BWT is negative.

We calculated AP−[CA+67BWT]\mathrm{AP}-[\mathrm{CA}+\frac{6}{7}\mathrm{BWT}] from the displayed model means:

ModelL2-7BL2-13BL3.1-8BQ-7BQ-14B
AP − Final mean+0.00380−0.00214−0.00674−0.02216+0.02739

Derived from Table 17(b). Values use the 0–1 metric scale.

The mean offset was approximately +0.00003. The positive offsets of L2-7B and Q-14B balanced the negative offsets of the other three models, explaining the near-cancellation in the five-model aggregate. In particular, L2-7B’s +0.00380 corrects the premise that the first four offsets were all negative. The per-model matrices expose each contribution directly.

Q3. Provide three-seed results to verify the reported SDs and PPL.

We supplied the seed-level records and checks in reports/J8_seed_ppl.md and the seed CSV. Table 6 used a two-stage aggregation: equal-weight averaging of six models within each seed, followed by the mean and sample SD of the three seed averages. Model-specific tables used each model’s three seed values. This specifies the unit of variation for each reported SD.

For ARCH on L3.1-8B, the WikiText-2 PPL values were 40.5055, 40.5122, and 40.5120. These values gave a mean of 40.5099 and a sample SD of ∑s=02(xs−xˉ)2/2≈0.0038\sqrt{\sum_{s=0}^{2}(x_s-\bar{x})^2/2}\approx0.0038. Recomputation of all 36 supplied PPL rows reproduced the reported means and SDs at the displayed precision. The records and calculation example provide a direct path from individual seed values to the published summaries.

Q4. What fraction of integer codes changed when anchoring was enabled?

A small perturbation can move a weight across a rounding threshold, and the quantizer recomputes group scales and zero points. We tested this sensitivity directly: the inputs, previous checkpoint, current/history curvature, α, and solver settings were fixed, and each β condition rebuilt its effective target and solver curvature. OFF/OFF repeated the same input state (Table 11).

ModelON vs. OFF (%)OFF vs. OFF (%)
L2-7B0.07540.0021
L3.1-8B0.08290.0024
Q-7B0.06920.0018

Table 11 excerpt. Changed integer codes (%).

Anchoring changed integer codes beyond repeated-run variation in all three models. The stream-level removal control provided complementary performance evidence: enabling anchoring raised AP from 0.4894 to 0.5006, a paired gain of 1.12 points with a 95% interval of [0.94, 1.30] (Tables 9–10). Together, the matched-state diagnostic and removal control demonstrate actual quantizer operation and its contribution within the complete method.

Thank you for the detailed response and the additional materials.

The clarification of the metric definitions resolves my original concern about the AP–CA–BWT relationship. I agree that, for K=8, AP averages the full 7×7 checkpoint–task matrix, while CA uses seven arrival scores and BWT uses the six earlier tasks. Thus, CA+67BWT\mathrm{CA}+\frac{6}{7}\mathrm{BWT} gives the final-checkpoint mean, which is distinct from AP. I also appreciate the clarification that the 16×16 blocks are used only for distance estimation, and the additional integer-code sensitivity results in Table 11.

I have two remaining questions concerning the numerical records provided in response to my original Q1 and Q3.

1. CA in Table 3(b). I recomputed the metrics from the nine supplied ARCH-CPTQ checkpoint–task matrices for L2-7B, L3.1-8B, and Q-7B across seeds 0–2. The averages are AP = 0.5005667, BWT = 0.0167580, and CA = 0.4876392. AP and BWT match Table 3(b) at the reported precision, but CA rounds to 0.4876 rather than the reported 0.4862. I used the seven diagonal entries for CA and the six earlier tasks for BWT, following Appendix A.4. Were the results in Table 3(b) obtained from a different set of runs? If not, could you clarify this discrepancy?

2. PPL seed records. The supplied CSV reproduces the reported means and standard deviations. However, all 36 three-seed means agree with the published four-decimal means within 3.34×10−93.34\times10^{-9}, which is considerably more precise than required by rounding. Could you confirm whether Table17_PPL_seed_values_example.csv contains values directly exported from the original evaluation runs, and whether any processing was applied when preparing the file? The corresponding original PPL evaluation logs or export script would help clarify this.

Thank you again for addressing the earlier questions. If you can provide the original evaluation logs or other verifiable records that resolve these remaining questions, I would be willing to consider raising my rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the discussion and revisions

We sincerely thank all reviewers. We appreciate Jw2v for recognizing the clarifications and corrections, yznk for confirming that the added control answered their question, and NdfZ for acknowledging the helpful clarifications and demonstrated empirical benefits.

Core contribution and motivation. A shared low-bit model must serve earlier and new tasks after past calibration samples are discarded. ARCH-CPTQ retains curvature describing how weight changes affect reconstruction on past inputs. Our contribution connects this information directly to the next quantization objective: under the stated assumptions, we combine matching the fixed high-precision reference with preserving the previous low-bit checkpoint, deriving the target and curvature for GPTQ (Secs. 2.3, 3.4).

The main discussion points and revisions are:

  • Mixing ratio and anchoring strength. Following yznk’s question, we added a control that fixed each layer’s history–current mixing ratio (α) and anchoring strength to the previous low-bit checkpoint (β) to their respective temporal means. ARCH exceeded this control by 0.44 percentage points in AP at W2A16 (95% paired interval: [+0.12, +0.76]; nine model–seed pairs; Table 14).

  • BWT and final performance on earlier tasks. We clarified the BWT improvement by separating arrival and final scores. Across three models, final accuracy on earlier tasks (Final-old) was 0.4957 for CPTQ-AVG versus 0.5178 for ARCH, showing that the BWT gain accompanied higher achieved final accuracy (Table 14).

  • Numerical verification. For Jw2v, we supplied per-run results, representative original logs, and recomputation code; clarified metric definitions; and corrected CA aggregation and affected standard deviations. Anchoring also changed quantized outputs beyond repeated-run variation (Table 11).

  • Design benefits and costs. For 2gx7 and NdfZ, we explained component contributions through comparisons removing one component at a time, and clarified data access, precision-dependent gains, and storage and update costs (Tables 3(b), 6, 10, 20).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.