acceptodds
Under review as a conference paper at ICLR 2027

PrismQ: Noise-Aware Resource Allocation for Elastic Post-Training Quantization

25 views

Abstract

Elastic post-training quantization (PTQ) jointly calibrates multiple precisions within a single model, enabling flexible precision adaptation under different deployment budgets. However, extending reconstruction from a single precision to multiple precisions introduces two key challenges. First, quantization errors vary across linear layers and precisions, creating a trade-off between cross-precision parameter sharing and precision-specific reconstruction capacity within the reconstruction space. Second, calibration states should reflect precision-dependent error propagation, while preserving complete quantized states for every precision incurs substantial calibration memory overhead. To address these challenges, we propose PrismQ, a noise-aware elastic PTQ framework that adapts both reconstruction space and calibration states according to precision-dependent quantization errors. Specifically, PrismRank estimates the demand for reconstruction capacity at each linear layer and precision from the singular-value characteristics of clipping-induced errors, and PrismLoRA realizes the resulting capacity allocation through shared and precision-specific low-rank components. Moreover, Calibration-only Sparse Token selects quantized token states according to token-wise quantization error under a fixed cache budget and uses them to construct calibration inputs for the next block, allowing these inputs to reflect precision-dependent error propagation. Extensive experiments on language models, vision transformers, and vision-language models show that PrismQ improves quantization performance across multiple precisions, with particularly clear gains at low bit widths.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Elastic Post-Training Quantization, Multi-Bit Quantization, Weight-Activation Quantization, Low-Rank Adaptation, Noise-Aware Rank Allocation, Calibration-State Propagation, Large Language Model Quantization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_j1jFOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

PrismQ introduces noise-aware (sensitivity-aware) resource allocation for elastic PTQ. PrismRank allocates reconstruction ranks based on SVD of clipping errors, PrismLoRA combines shared and precision-specific low-rank components, and Sparse Token selectively propagates high-error states during calibration. The method is evaluated across LLMs, ViTs, and VLMs.

Critical strengths
  1. Noise-aware allocation of reconstruction capacity and calibration states provides an interesting perspective on elastic PTQ.

  2. Extensive evaluations across model architectures and precision configurations support the generality of the approach.

Critical weaknesses
  1. Limited scalability analysis of SVD-based rank allocation. PrismRank requires SVD across layers and precision configurations, potentially incurring substantial computational and memory overhead. However, evaluations are limited to models up to 13B, with no analysis of SVD overhead or comparison of total quantization time against existing methods.

  2. Insufficient evidence of practical serving efficiency. Since LLM decoding is often memory-bandwidth-bound, the practical advantages of elastic activation quantization over weight-only quantization remain unclear. Comparisons with recent OFA methods, such as AnyBCQ and MoBiQuant, are missing. Furthermore, the inference analysis lacks prefill latency measurements and runtime precision-switching overhead.

Questions for authors
  1. Latest LLMs. Could the authors evaluate PrismQ on a recent large-scale LLM, such as Qwen3.8-2.4T-A95B or Qwen3-235B-A22B if architectural compatibility is an issue?
  2. SVD scalability. How does SVD overhead scale with model size? Please report SVD latency, peak memory usage, and total quantization time relative to existing PTQ methods.
  3. Serving efficiency. Could the authors compare PrismQ with AnyBCQ and MoBiQuant under weight-only quantization, reporting accuracy, prefill latency, decode throughput, and memory usage? A controlled comparison with AnyBCQ on a single model would suffice. Please also clarify the overhead of runtime precision switching, including compilation, weight repacking, and CUDA Graph management.

I would consider raising my recommendation to acceptance if the authors demonstrate scalability to recent large-scale LLMs, quantify SVD overhead, and establish competitive serving efficiency against recent OFA methods.

7 comments
AuthorsOpenReview ↗

Response to Weakness 1 & Question 2 (Part I)

Dear Reviewer j1jF,

Thank you for your careful review. Manuscript references refer to the revised paper; supplementary paths refer to files in the uploaded supplementary materials or the anonymous archive. Bold marks the best compared quantized results, including displayed ties; reference rows are unranked.

Weakness 1 & Question 2:Scalability and quantization cost

  • All reported experiments disable budget redistribution across transformer blocks. Analysis and joint reconstruction therefore proceed consecutively within the currently loaded transformer block, avoiding two separate full-model loading passes.
  • Analysis computes only the singular values of the clipping-induced error, saving the computation and storage of singular-vector matrices. These matrices are distinct from PrismLoRA's trainable low-rank factors.

Table R10. Qwen3 calibration cost and monitor-reported memory peaks.

ModelStage1 (h)All (h)Stage1 / All (%)Peak RSS (GiB)Peak allocated (GiB)Peak reserved (GiB)
Qwen3-0.6B0.1000.60416.636.204.154.31
Qwen3-1.7B0.1391.00113.8410.298.208.64
Qwen3-4B0.2282.4039.5012.6513.7114.58
Qwen3-8B0.3043.9767.6518.2120.8521.57
Qwen3-14B0.4697.0746.6322.3931.4033.27
Qwen3-32B1.03616.5576.2622.6344.2346.31
Qwen3-30B-A3B0.72815.4704.7112.734.054.44
Qwen3-235B-A22B1.53634.2424.4922.0911.1212.09

Stage1 includes clipping pretraining, clipping-induced error analysis, singular-value computation and allocation. All spans the first log record through PTQ completion, including pre-rotation, initialization and logging; evaluation is excluded.

RSS is resident host memory; allocated and reserved denote GPU allocator usage and reservation. The memory columns report monitoring peaks.

Current Dense/MoE script settings are compared in Question 1 (Table R15). Timing logs for the completed Dense runs are in the supplementary PrismQ_exp/qwen3_*_seed42_epochs12/ directories.

Table R11. Matched calibration comparison, seed 42, 12 epochs, W4A4.

ModelMethodAll (h)WikiText-2 PPLC4 PPLFive-task Avg. (%)
LLaMA-2-7BQuEPT3.6346.017.7259.51
LLaMA-2-7BPrismQ3.5215.867.5466.31
Qwen3-0.6BQuEPT0.73931.3838.1945.33
Qwen3-0.6BPrismQ0.60428.1233.8346.64
Qwen3-1.7BQuEPT1.16921.0424.5452.92
Qwen3-1.7BPrismQ1.00119.9023.4554.58
Qwen3-4BQuEPT2.66517.2820.6661.32
Qwen3-4BPrismQ2.40315.4618.9662.48

All uses the boundary above. The runs match seed, corpus, sample counts, grouping and learning rates, while retaining native capacity and calibration-state designs. At W4A4, PrismQ takes less time and gives lower PPL and higher five-task means on all four models. Configurations and per-block timing records are in the first entries and PTQ sections of the supplementary PrismQ_exp/ and QuEPT_exp/WA_Epochs/ logs.

AuthorsOpenReview ↗

Response to Weakness 2 & Question 3 (Part II)

Weakness 2 & Question 3:Serving efficiency and precision switching

Weight-only. Completed LLaMA-3-8B measurements use packed W4A16/W8A16 exports from one calibrated superset, with the low-rank correction merged before packing and BF16 activations/KV cache. Both run on one RTX 6000 Ada with vLLM 0.15.1 and verified Marlin kernels.

Table R12. Calibrated-superset PPL and packed weight-only serving.

FormatWikiText-2 PPLC4 PPLPrefill (ms)Decode (tok/s)Weight files (GiB)Sampled GPU peak (GiB)
W4A166.559.5448.67150.315.3141.63
W8A166.158.9164.4693.818.5641.65

PPL is from fake-quantized calibrated-superset evaluation; latency and memory are from packed serving. Prefill is mean engine time for 512 input/1 output token; decode is median client-observed throughput for 128 input/256 output tokens, concurrency 1. GPU peak includes the reserved KV-cache pool (memory utilization 0.85). The supplementary deploy/README_deploy.md describes the protocol; deploy/vllm_w4_*/ and deploy/vllm_w8_*/ contain the packed-serving run.json, performance.json, memory.json and kernels.json records. PPL is logged in deploy/ptq_20261110_135709_2720530/log_2026_11_10_14_29_Llama-3-8b.txt.

Table R13. Additional LLaMA-3-8B W4 comparison on one RTX 6000 Ada.

MethodWikiText-2 PPLMMLU Acc. (%)Decode (tok/s)
FP166.1265.02—
AnyPrecisionLLM6.7064.0487
AnyBCQ6.8463.1590
MoBiQuant7.3163.29129 ∗{}^{*}
PrismQ6.5564.10150

∗{}^{*} MoBiQuant throughput is estimated from its reported A100 40 GB result by memory-bandwidth scaling to RTX 6000 Ada; the other quantized throughput entries are measured. PrismQ's 150 tok/s rounds the 150.31 tok/s above. A dash denotes an unreported measurement.

Weight + activation. Manuscript Table 16 uses LLaMA-2-7B with ABQ-LLM kernels. At batch 1, 512 input and 128 generated tokens, W8A8 reduces sampled peak GPU memory from 13,820 to 7,654 MiB (44.6%) while decode falls from 58.491 to 53.418 tok/s (8.7%), providing a different speed/memory operating point. These are total serving-memory measurements relative to FP16 on this model/backend; the weight-only results above use LLaMA-3-8B and vLLM.

Each weight-only format uses a separately initialized engine. Dynamic precision switching and its isolated overhead, including repacking, compilation and CUDA Graph handling, remain future work.

AuthorsOpenReview ↗

Response to Question 1 (Part III)

Question 1:Evaluation on recent Qwen3 models

Quality evaluation covers Qwen3-0.6B–14B. Qwen3-32B and Qwen3-235B-A22B have calibration-cost records but no reported PPL or task accuracy under our single 48 GB GPU budget.

Table R14. Qwen3 quality and calibration cost. PPL: WikiText-2/C4; Avg.: five tasks (%).

ModelTypeTotal trainable params (M)Stage1 (h)All (h)W4A4-PPLW4A4-Avg.W6A6-PPLW6A6-Avg.W8A8-PPLW8A8-Avg.
Qwen3-0.6BDense34.860.1000.60428.12/33.8346.6421.73/26.2251.6721.15/25.6052.23
Qwen3-1.7BDense70.220.1391.00119.90/23.4554.5817.41/19.7960.5816.79/19.4361.21
Qwen3-4BDense148.220.2282.40315.46/18.9662.4813.96/16.9067.2213.70/16.6768.60
Qwen3-8BDense228.880.3043.97610.99/14.8568.859.94/13.4371.359.72/13.3171.65
Qwen3-14BDense383.700.4697.0749.67/13.3472.418.74/12.0874.548.67/12.0174.93
Qwen3-32BDense859.941.03616.557——————
Qwen3-30B-A3BMoE650.250.72815.470——————
Qwen3-235B-A22BMoE2547.091.53634.242——————

Quality rows use seed 42 and 12 epochs. Tables R14/R11 use normalized accuracy for PIQA, ARC-E/C and HellaSwag, and accuracy for WinoGrande.

Table R15. Comparable calibration settings for Dense and MoE models.

Model typeCalibration targetsEpochsReference rankMinimum rank per precisionPrecision-specific rank
Dense5124022
MoE3101011

These comparable settings follow the supplied Dense and MoE launch scripts.

Thank you for highlighting recent-model coverage, analysis cost, and practical serving performance.

Reviewer_FouSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the problem of elastic post-training quantization, allowing one to jointly calibrate the quantization process for multiple bit-widths. This is achieved by introducing 3 components: PrismRank to estimate the demand of Ean linear layer based on quantization error, PrismLoRA to then fulfill this demand, and finally Sparse Token to select and propagate the quantized tokens under a given cache budget. Experiments are performed on LLMs, vision models, and VLMs.

Critical strengths

The general idea behind the paper is well motivated, although several aspects of this motivation should be clarified.

The experiments evidence the effectiveness of the proposed method.

Critical weaknesses
  1. Clarity

1.1: The discussion of the most relevant methods is quite short. I would appreciate more detail on the statement at Lines 121-124.

1.2: I found it at first difficult to follow the flow of the proposed approach, in particular the connection between PrismRank, PrismLoRA, and ST. The figures (Figs 1 and 2) do not truly help as they are quite cluttered and difficult to parse. I feel that some higher-level summary of the flow would help.

1.3: Equation 2 would deserve some explanations, and potentially a numerical example. The statement related to this equation below Equation 3 is also not clear to me.

1.4: How realistic are the assumptions (Laplace distribution, common clipping center) listed at Line 197?

1.5: Where is the statement at Lines 203-205 (third point) shown?

1.6: At Line 218, can you elaborate on the notion of rank-analysis samples?

1.7: At Lines 249-250, how is this initialization of U and V justified?

  1. Experiments:

2.1: When it comes to the most relevant methods, there are only comparisons with QuEPT. What about CoA-LoRA, MBQuant, PTMQ, and MatQuant, which are mentioned in the related work section?

2.2: Based on the different tables, GPTAQ seems to perform quite well and to be quite cheap in terms of calibration time (Table 2). As such, is it really important to have a single calibration process for multiple bit-widths?

Questions for authors

See Weaknesses above

2 comments
AuthorsOpenReview ↗

Response to Weaknesses (Part I)

Dear Reviewer FouS,

Thank you for your careful review. Manuscript references refer to the revised paper; supplementary paths refer to files in the uploaded supplementary materials or the anonymous archive. Bold marks the best compared quantized results, including displayed ties; reference rows are unranked.

Weakness 1 (1.1):Related-work discussion

Section 2 now gives the relevant mechanisms and learning targets explicitly: QuEPT's cascaded adapters and token merging, CoA-LoRA's configuration-conditioned adaptation, MBQuant's branch reuse, PTMQ's feature mixing, and MatQuant's nested representations. Appendix J, Table 24 further compares quantization scope, data, capacity sharing, and calibration-state budgets. It separates MatQuant's PTQ and QAT settings. This expanded comparison replaces the earlier broad description at Lines 121–124.

Weakness 2 (1.2):Calibration flow and module connections

Section 3 now opens with the three-step calibration flow (Figure 2):

Table R6. Three-step calibration flow.

ModuleDecisionOutput
PrismRankAllocate the parameter budget from singular values of the clipping-induced errorRank plan per linear layer and precision
PrismLoRAReconstruct with shared and precision-specific low-rank componentsCandidate-specific quantized token states
Sparse Token (ST)Select quantized token states by token-wise quantization error under a cache budgetNext-block calibration inputs

PrismRank analyses errors with rounding disabled before joint reconstruction. ST compares PrismLoRA's outputs with full-precision token states, then replaces selected positions with quantized token states to construct candidate-specific calibration inputs for the next block. This calibration-state propagation is used only during calibration (§3.4).

Weakness 3 (1.3):Equations and numerical example

Equation (2) describes nested clipping intervals for the same tensor group with weights and inputs fixed. Section 3.1 distinguishes these continuous intervals from finite quantization grids: interval inclusion does not imply grid inclusion.

The added example uses T=3T=3, a lower-precision interval [−1,1][-1,1], and a higher-precision interval [−2,2][-2,2]. Clipped values are 11 and 22; clipping errors are 22 and 11; the adjacent increment is 11. Equation (3) therefore reads 2=1+12=1+1 before rounding.

Equation (3) follows by adding and subtracting the higher-precision clipped tensor. It holds for arbitrary clipping intervals, independently of Laplace or nesting assumptions; nesting interprets the adjacent increment as additional lower-precision clipping (Section 3.1, Appendix H.1).

Weakness 4 (1.4):Laplace and common-center assumptions

The Laplace distribution and common center define the analytical model in Appendix H.2: for a fixed group and scale, its approximate quantization-MSE minimizer has a clipping radius that increases with bit width. Actual bounds are learned independently. Empirical support comes from Table 22, where 76.29%–88.99% of adjacent weight-precision intervals are nested after clipping pretraining in the dedicated LLaMA-2-7B per-output-channel diagnostic. Section 3.1 distinguishes these model assumptions from measured interval behavior; precision-specific low-rank components accommodate deviations.

Weakness 5 (1.5):Conditions for nested activation intervals

Appendix H.3 gives the condition directly. For a shared activation input/group with extrema a≤0≤ca\le0\le c and ratios 0<ρb≤ρb′≤10<\rho_b\le\rho_{b'}\le1, we have ρb′a≤ρba\rho_{b'}a\le\rho_ba and ρbc≤ρb′c\rho_bc\le\rho_{b'}c. Hence [ρba,ρbc]⊆[ρb′a,ρb′c][\rho_ba,\rho_bc]\subseteq[\rho_{b'}a,\rho_{b'}c]. The configured A4–A8 ratios increase with precision (Appendix G). Section 3.1 states the shared-input, common-grouping, and zero-containing-range conditions alongside the nesting claim.

Weakness 6 (1.6):Rank-analysis samples

A rank-analysis sample is an input sequence used to measure a linear layer's clipping-induced error before joint reconstruction. In the LLM experiments, analysis uses the first two cached 2048-token sequences; clipping pretraining uses the first four, and joint reconstruction uses all 128 training sequences. Candidate precisions reuse the same analysis sequences and full-precision inputs. Each error matrix has tokens as rows and output channels as columns. Section 3.2 and Appendix G now specify this definition and sampling procedure.

Weakness 7 (1.7):Low-rank initialization

With U=0U=0 and Xavier-initialized VV, the initial correction ΔWb=UbVb\Delta W_b=U_bV_b is zero, so the weight before quantization remains WW. For effective correction gradient GbG_b, ∇UbL=GbVb⊤\nabla_{U_b}L=G_bV_b^\top can be nonzero immediately; ∇VbL=Ub⊤Gb\nabla_{V_b}L=U_b^\top G_b starts at zero and can become nonzero after UbU_b updates. This asymmetric initialization preserves the base weight while enabling learning. Section 3.3 now explains this gradient behavior explicitly.

AuthorsOpenReview ↗

Response to Weaknesses (Part II)

Weakness 8 (2.1):Comparisons with related methods

We compare reported LLaMA-2-7B C4 PPL from manuscript Table 14 with CoA-LoRA Table C.5, rank 64, alongside their settings:

Table R7. Reported weight-only settings and C4 PPL.

Setting / resultPrismQCoA-LoRA (Table C.5)
Quantization levelsUniform integerNon-uniform NF
Weight-bit allocationUniform 3/4-bitLayer-wise mixed; average 3/4 bits
RankReference rank 60Fixed rank 64
scale1FP16b1∈{2,3,4,8}b_1\in\{2,3,4,8\} bits (search)
scale2Not usedBF16 / FP16 / FP32 (search)
Group sizePer output channelB0∈{16,32,64}B_0\in\{16,32,64\}; B1∈{16,64,256}B_1\in\{16,64,256\} (search)
Data / objectiveWikiText-2; block MAEC4; next-token fine-tuning
C4 PPL, 3-bit setting7.847.91
C4 PPL, 4-bit setting7.177.23

Reference rank specifies PrismQ's parameter budget. CoA-LoRA's scale/group entries marked "search" are configuration ranges, not the final per-layer choices in Table C.5. Its average-bit budget includes scale metadata; PrismQ reports uniform weight bits. Calibration data and objectives also differ. Other reported bit widths are in manuscript Table 14 and CoA-LoRA Table C.5.

MBQuant uses independent 2-bit weight branches with image-label/distillation QAT. MatQuant mainly evaluates FFN quantization and also reports FFN+attention extensions. Their architecture, training or quantization scope requires alignment for controlled LLM comparisons.

PTMQ's released code supports CNN/ViT calibration. LLM adaptation requires autoregressive transformer-block interfaces, sequence calibration, and memory for weight-level rounding parameters and multi-precision calibration states. Existing vision comparisons are shown below.

Table R8. ImageNet-1K top-1 accuracy (%), excerpted from manuscript Table 3.

ModelMethodW6A6W7A7W8A8
ViT-SPTMQ76.177.178.2
ViT-SPrismQ81.081.281.3
ViT-BPTMQ77.778.679.1
ViT-BPrismQ84.084.484.5
DeiT-SPTMQ78.779.379.5
DeiT-SPrismQ79.379.779.7

Weakness 9 (2.2):Value and cost of joint multi-precision calibration

PrismQ's calibration cost is adjustable through the reconstruction-epoch budget. GPTAQ's advantage is its single-target calibration efficiency; our measured totals are 1.098/3.305 h for five/fifteen independent targets. On LLaMA-2-7B, our 2-epoch, five-format run takes 0.898 h (All), below GPTAQ's five-target total, with W4A4 C4 PPL 7.62 and five-task Avg. 66.53% versus GPTAQ's reported 7.72 and 65.03%.

Table R9. LLaMA-2-7B C4 PPL and five-task Avg. (%). PrismQ uses seed 42 and 2 epochs; GPTAQ quality is from manuscript Table 1. Times cover W4A4–W8A8.

MethodW4A4-PPLW4A4-Avg.W6A6-PPLW6A6-Avg.W8A8-PPLW8A8-Avg.Five-format calibration (h)
GPTAQ7.7265.037.0366.236.9866.521.098 (measured)
PrismQ (2 epochs)7.6266.537.0268.446.9868.860.898 (All)

The 2-epoch run evaluates W4A4/W6A6/W8A8; its complete supplementary log is PrismQ_exp/llama2_7B_seed42_epochs2/log_2026_11_13_10_43_Llama-2-7b.txt.

Appendix I (Table 23) separately evaluates all 25 combinations of W4–W8 and A4–A8 after one 12-epoch joint calibration of the five diagonal formats. Format WiAjW_iA_j reuses the adapter calibrated for WjAjW_jA_j without recalibration; 15 of these pairs satisfy 4≤bw≤ba≤84\le b_w\le b_a\le8.

Thank you for the suggestions on presentation, method comparisons, and the calibration trade-off.

Reviewer_kQSEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PrismQ, a noise-aware framework for elastic post-training quantization. It introduces PrismRank to allocate low-rank reconstruction capacity based on clipping-induced quantization errors, PrismLoRA to combine shared and precision-specific low-rank components, and Sparse Token to preserve important quantized calibration states under a limited memory budget. Experiments on LLMs, ViTs, and VLMs show improvements over existing elastic PTQ methods, particularly at low bit widths.

Critical strengths
  1. Adaptive reconstruction capacity. Allocating low-rank capacity based on layer- and precision-specific quantization errors is a reasonable approach. The combination of shared and precision-specific components provides a flexible way to balance parameter efficiency and reconstruction quality.

  2. Practical improvements across multiple settings. The experiments cover LLMs, ViTs, and VLMs, with improvements particularly at low bit widths. The Sparse Token strategy is also useful for incorporating quantization errors into block-wise calibration under a limited memory budget.

Critical weaknesses
  1. Limited justification for the rank allocation strategy. The proposed Power×Rank score is largely heuristic. It is unclear how well the singular-value structure of clipping-induced errors reflects the actual reconstruction capacity needed after quantization, especially when rounding errors are also present.

  2. Experimental validation could be stronger. The LLM experiments are limited to relatively small and older models, and the improvements are less consistent on LLaMA-3 and Qwen2. In addition, PrismQ and the reproduced QuEPT baseline use different calibration settings, making it difficult to attribute the improvements entirely to the proposed method.

  3. The core design largely combines existing allocation and sharing ideas. Prior work includes multi-precision calibration, cascaded low-rank sharing, and token-state selection or merging in QuEPT, and quantization-error spectral rank allocation in RA-LoRA, albeit for fine-tuning. PrismQ adds clipping-output Power×Rank scores, shared and precision-specific factors, and fixed-cache selection of high-error states. These are useful engineering designs, but do not clearly move beyond existing error-aware allocation and sharing principles. Capacity and states remain separately allocated, so the unified view also does not introduce a new joint optimization mechanism. I find the methodological advance incremental.

Questions for authors
  1. Why is Power×Rank chosen as the allocation score? Could the authors provide stronger evidence that it correlates with the reconstruction capacity actually needed by each layer and precision?

  2. Could the authors provide a comparison with QuEPT under matched calibration settings, particularly the number of reconstruction epochs and random seeds? This would help clarify the source of the reported accuracy and calibration-time improvements.

  3. With the same target formats, data, and hardware, please report quality and runtime as calibration epochs or optimization steps increase. Include pretraining, SVD analysis, and state selection. This would clarify whether the speedup comes from the method or from different training settings.

6 comments
AuthorsOpenReview ↗

Response to Weakness 1 & Question 1 (Part I)

Dear Reviewer kQSE,

Thank you for your careful review. Manuscript references refer to the revised paper; supplementary paths refer to files in the uploaded supplementary materials or the anonymous archive. Bold marks the best compared quantized results, including displayed ties; reference rows are unranked.

Weakness 1 & Question 1:Rank-allocation score and reconstruction capacity

The allocation score Power×Rank\mathrm{Power}\times\mathrm{Rank} balances error magnitude (Power: singular-value mass) and complexity (Rank: minimum count retaining 90% of that mass). Retained rank differs from the budgeted adapter rank. Power prioritizes larger low-precision errors; minimum rank per precision protects higher-precision allocations.

(1) Clipping-induced error analysis. We analyse clipping-induced error—the linear-layer output difference from full precision with rounding disabled—then jointly reconstruct with rounding enabled and shared and precision-specific low-rank components. Table 5 gives W4A4 WikiText-2/C4 PPL of 5.85/7.55 when analysing clipping-induced error versus 5.90/7.61 when rounding is included in the analysis.

(2) Independent capacity diagnostic. We fitted all 28 Qwen3-0.6B up_proj linear layers at W4A4/W6A6/W8A8 using seeds 42/420/4200, 512-token sequences and 200 steps per nonzero rank, with rounding enabled. Reconstruction demand (the demand for reconstruction capacity) is measured by the empirical required rank r∗=min⁡{r:Lval(r)≤min⁡r′Lval(r′)+ϵ}r^*=\min\{r:L_{\mathrm{val}}(r)\le\min_{r'}L_{\mathrm{val}}(r')+\epsilon\} over r∈{0,2,4,8,16,32,64}r\in\{0,2,4,8,16,32,64\}; each seed's tolerance is fixed by its independent pilot.

Table R1. Spearman correlation with empirical required rank; cells show Power×Rank / Power-only.

SeedW4A4W6A6W8A8
420.911 / 0.9180.600 / 0.6460.581 / 0.581
4200.941 / 0.9410.676 / 0.7060.536 / 0.536
42000.943 / 0.9460.614 / 0.6660.536 / 0.536

Table R2. Uniform–oracle gap closed (%) by seed. Cells: mean rank 8 / mean rank 16.

AllocationSeed 42Seed 420Seed 4200
Uniform / Rank-only0.00 / 0.000.00 / 0.000.00 / 0.00
Power-only84.44 / 84.3884.74 / 86.1885.16 / 87.44
Power × Rank85.29 / 87.1487.85 / 91.8190.07 / 89.89
Validation-selected oracle100.00 / 100.00100.00 / 100.00100.00 / 100.00

Gap closed is 100(LUniform−Lmethod)/(LUniform−Loracle)100(L_{\mathrm{Uniform}}-L_{\mathrm{method}})/(L_{\mathrm{Uniform}}-L_{\mathrm{oracle}}), where LL is test MAE across the 84 linear-layer/precision pairs. The parameter budgets are 2,752,512/5,505,024, with minimum rank 2 for every allocation. The validation-selected oracle is evaluated on held-out test data.

Power-only has higher or equal Spearman correlation, while Power×Rank closes more of the gap in all six tested seed–budget settings (85.29%–91.81%). Spearman measures ordering; budget allocation also depends on score ratios and reconstruction curves' marginal gains. These results support the Rank factor's empirical allocation benefit in the tested geometry.

Raw results and configurations are in PrismQ_exp/prismrank_capacity_core_seed{42,420,4200}/, especially metrics.json, allocation_summary.csv and manifest.json.

AuthorsOpenReview ↗

Response to Weakness 2 & Question 2 (Part II)

Weakness 2 & Question 2:Matched calibration settings and model coverage

We added matched comparisons with QuEPT on recent Qwen3 models and LLaMA-2-7B. Each pair matches reconstruction epochs, seed (42), WikiText-2 calibration data (128 training sequences, length 2048), weight grouping (group size 128 at W4; per output channel at W5–W8), and active clipping/low-rank learning rates (0.1/7×10−57\times10^{-5}). For the LLaMA-2-7B comparison, both methods enable online q/k Hadamard to match QuEPT's configuration, unlike the manuscript's main LLM configuration; it remains disabled for the Qwen3 comparisons. Configurations are recorded in the first log entries under the supplementary PrismQ_exp/ and QuEPT_exp/WA_Epochs/ directories.

Each method retains its native capacity design: PrismQ uses reference rank 40; QuEPT's released cascade selects rank-192/128/64 factor prefixes for W4/W5–W6/W7–W8. The manuscript reproduction and supplementary runs use this code-defined cascade; 15 epochs follows QuEPT's paper.

Table R3. Calibration comparison, seed 42 and 12 epochs. PPL: WikiText-2/C4; Avg.: five tasks (%).

ModelMethodW4A4-PPLW4A4-Avg.W6A6-PPLW6A6-Avg.W8A8-PPLW8A8-Avg.All (h)Total trainable params (M)
LLaMA-2-7BQuEPT6.01/7.7259.515.52/7.0362.515.49/6.9862.423.634794.17
LLaMA-2-7BPrismQ5.86/7.5466.315.50/7.0168.525.47/6.9868.933.521211.98
Qwen3-0.6BQuEPT31.38/38.1945.3321.81/26.5351.3821.50/25.8751.460.739125.93
Qwen3-0.6BPrismQ28.12/33.8346.6421.73/26.2251.6721.15/25.6052.230.60434.86
Qwen3-1.7BQuEPT21.04/24.5452.9217.71/20.3360.1617.42/19.7061.001.169217.22
Qwen3-1.7BPrismQ19.90/23.4554.5817.41/19.7960.5816.79/19.4361.211.00170.22
Qwen3-4BQuEPT17.28/20.6661.3213.99/16.9467.0513.77/16.7868.042.665411.84
Qwen3-4BPrismQ15.46/18.9662.4813.96/16.9067.2213.70/16.6768.602.403148.22
Qwen3-8BPrismQ10.99/14.8568.859.94/13.4371.359.72/13.3171.653.976228.88
Qwen3-14BPrismQ9.67/13.3472.418.74/12.0874.548.67/12.0174.937.074383.70

All includes logged preparation through PTQ completion. At 5 epochs (All 1.679 h), PrismQ reaches W4A4 PPL 5.91/7.57 and 66.16% five-task Avg.; both PPL values are lower and accuracy is higher than QuEPT at 5/12/15 epochs (Table R5).

In manuscript Table 1, PrismQ ranks best in five-task Avg. at 17/20 model–precision operating points among all reported quantized baselines, including ties.

AuthorsOpenReview ↗

Response to Weakness 3 (Part III)

Weakness 3:Contribution relative to QuEPT and RA-LoRA

PrismQ derives an exact algebraic clipping-induced error decomposition for elastic PTQ (§3.1): activation- and weight-clipping terms, with each clipped tensor's error expressed as a highest-precision base plus adjacent-precision increments. Interval nesting motivates shared low-rank components; imperfect nesting and rounding motivate precision-specific low-rank components. Building on joint calibration (PTMQ/QuEPT) and adaptive ranks, the contribution is analysis-guided reconstruction sharing and calibration-state design.

Analysis-guided reconstruction sharing: QuEPT uses identical reconstruction spaces within its mid/high groups. PrismQ allocates shared and precision-specific capacity by linear layer and precision, addressing potential competition within fixed groups. Table 7 reports W4A4 PPL of 5.85/7.55 versus Cascade's 5.95/7.62 at the same parameter budget. RA-LoRA analyses W−Q(W)W-Q(W) for single-precision fine-tuning; PrismRank analyses activation-dependent clipping-induced error for cross-precision sharing. Table 5 gives 5.85/7.55 versus 5.95/7.64 without precision-specific rank.

Calibration-state design: Sparse Token (ST) constructs candidate-specific calibration inputs using selected quantized token states under a fixed cache budget; QuEPT supplies the same fused calibration input to all precisions. At matched cache budgets, Table 8 gives 5.85/7.55 versus Fusion Token's 5.99/7.66.

PrismRank and PrismLoRA follow the clipping-induced error decomposition; ST follows the observed mismatch between calibration and inference inputs.

Comments on the paper

2 comments

As only a few days remain in the author-reviewer discussion period, I encourage reviewers to review the authors' rebuttal and actively participate in the discussion. In particular, please clarify which concerns have been addressed and identify any remaining concerns or questions that may require further discussion.

AuthorsOpenReview ↗

Summary of Our Responses to Reviewers' Concerns

We thank the reviewers for their detailed feedback. The revised manuscript clarifies the method and its validation scope; our reviewer-specific replies provide configurations, comparisons, and supplementary source paths.

  1. Method and allocation evidence. Section 3 and Appendix H distinguish the exact clipping-induced error identity from the assumptions used to interpret interval nesting. We clarify the PrismRank → PrismLoRA → Sparse Token (ST) flow, sequence counts, and low-rank initialization. The independent Qwen3-0.6B diagnostic covers 28 linear layers, three precisions, and three seeds. Power-only has higher or equal rank correlation, while Power×Rank closes more of the uniform–oracle gap in all six tested seed–budget settings.

  2. Matched comparisons and model coverage. Additional QuEPT comparisons match epochs, seed, calibration data, grouping, and active learning rates while retaining each method's native capacity design. They cover LLaMA-2-7B and Qwen3-0.6B/1.7B/4B; PrismQ quality is also evaluated at Qwen3-8B/14B. The replies and Appendix K report PPL, task accuracy, trainable parameters, and epoch-budget results. Calibration-resource records additionally cover Qwen3-32B, 30B-A3B, and 235B-A22B.

  3. Calibration cost and related-method scope. Stage timings and monitored memory peaks are reported with explicit metric definitions. All includes logged preparation through PTQ completion and excludes evaluation. The two-epoch LLaMA-2-7B run takes 0.898 h, versus GPTAQ's measured 1.098 h for five independent targets. Dense and MoE results retain their different calibration workloads. The FouS reply also provides vision-model comparisons with PTMQ and a published CoA-LoRA comparison that identifies differences in quantization format, bit-budget accounting, data, and objective. Appendix J, Table 24 summarizes related-method settings.

  4. Serving measurements and switching limits. Section 4.3 and Appendix D distinguish joint calibration from deployment in separately initialized fixed-format engines. We report packed W4A16/W8A16 serving from one LLaMA-3-8B calibrated superset and the LLaMA-2-7B weight-plus-activation memory/speed trade-off. Dynamic precision switching remains future work.

  5. New evidence in the follow-up reply. PrismQ does not require nested integer weights. The follow-up reports post-hoc Matryoshka-style adaptation for 4/6/8 and 3/4/6/8 using pure MSB truncation of a shared parent code, with FP16 and non-nested references, matched additional calibration budgets, and WikiText-2 PPL. PrismQ initialization improves over HF initialization under the tested additional optimization budgets, but nesting can degrade quality relative to native formats.

  6. Activation-error compensation and calibration sensitivity. The follow-up adds Qwen3-0.6B evaluation-time low-rank correction on/off comparisons for weight-only and weight–activation quantization. Calibration on WikiText-2 and C4 with seeds 42/420/4200 reports per-format ranks, total trainable rank by linear layer, PPL, and PIQA normalized accuracy as three-seed means and sample standard deviations. These results provide empirical evidence on combined-error reconstruction and calibration sensitivity; broader calibration distributions may require more rank-analysis sequences.

Supplementary paths refer to the uploaded materials or the anonymous archive. The new adaptation protocol and run locations are documented in PrismQ_MatQuant_Results/pipeline.md.

We thank all reviewers for their time and constructive feedback, which have helped improve the clarity and evaluation of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.