- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
HiCAP-Q proposes capability-aware mixed-precision post-training quantization. It constructs capability-specific probes, measures quantization sensitivity at the layer and module levels, and uses these scores to allocate FP16 channels within otherwise NVFP4-quantized modules. Experiments across five models evaluate capability preservation under a stated average-bit budget.
Critical strengths
S1: The cross-domain experiment supports the central motivation. Allocations derived from domain-matched probes perform best in their corresponding evaluation domains (Table 4), suggesting that capability-specific information can guide precision allocation.
S2: The evaluation covers several capabilities and model families. Results include knowledge, mathematics, science, coding, and safety. The ablations also examine the contributions of sensitivity estimation, probe selection, and metric fusion (Tables 2–5).
Critical weaknesses
W1: The effective-bit accounting needs clarification. Standard NVFP4 requires approximately 4.5 bits per value when block scales are included, before adding FP16 channels. The reported 4.4 “effective bits” therefore needs reconciliation. Without consistent accounting of scales and other overhead, the claimed equal-budget comparison is difficult to assess.
W2: Important implementation details are missing. The paper does not fully specify probe sizes, screening settings, the FP16 allocation schedule, or which channels are preserved within each module. These choices directly affect reproducibility.
W3: The ablations do not clearly isolate the hierarchical design’s contribution. Table 5 is described as cumulative, but the “linear-only” and “layer-only” rows leave unclear whether components are replaced or added. A comparison using identical probes and storage budgets would better establish the benefit of combining both granularities.
Questions for authors
Q1: Does the reported 4.4-bit budget include NVFP4 scales, FP16-channel indices, and other storage overhead? Please clarify the accounting and provide actual model storage sizes to establish whether the comparisons are budget-matched.
Q2: Could you specify the probe sizes, screening settings, FP16 allocation schedule, and within-module channel-selection rule? These details would help reproduce the reported results.
Q3: Are the “linear-only” and “layer-only” rows in Table 5 independent alternatives or cumulative additions? How does their combination compare with each alone using identical probes and storage budgets?
3 comments
Response to Reviewer jPzf W2/Q2
We sincerely appreciate your time and effort in reviewing our manuscript. Below are our responses to your valuable comments.
Q2/W2 (implementation details): We provide the missing details below and in the revised Appendix C–E.
(i) Probe and screening settings. Each capability uses 512 screened probes (Section 4.1). Screening retains prompts passing the capability-relevance and quality thresholds described in Section 3.3, with the target model itself serving as the screening model (no external judge is used). The self-confidence score q_self is obtained via verbalized self-evaluation: the target model generates an explicit confidence judgment for its own response, which is parsed into a score used by the screening stage. These details are added in revised Appendix D.
(ii) FP16 allocation schedule. The schedule is defined by the percentile threshold sets K = {0.15, 0.30} and R = {0.10, 0.30}: each module falls into one of L = 5 hierarchical sensitivity levels h(m) ∈ {0,…,4} (Section 3.5), and each module at level h retains a fixed quota of g·h FP16 channels with granularity g = 128 — i.e., 0 / 128 / 256 / 384 / 512 FP16 channels for levels 0–4; modules at level 0 are fully quantized to NVFP4. The complete quota table is now given in Appendix C (Table 7). We emphasize that the schedule contains no free ratio hyperparameters beyond the thresholds (K, R) and the granularity g: the per-module level map {h(m)} is derived per model from its own capability-specific sensitivity estimates, and the quota table is fixed. Consequently, the achieved parameter-weighted budget is an empirical outcome of the estimated sensitivities rather than a directly tuned target; we now report the actual achieved effective bit-width of every configuration in Appendix C (Table 6), including NVFP4 scales and FP16-channel overhead, and all budget comparisons in Tables 2–3 use these values (see our response to Q1). The per-module level maps for all five models will be released with the code.
(iii) Within-module channel selection. Once the per-module FP16 budget k_m is determined by the hierarchical level h(m), the retained channels are selected by activation magnitude. For each linear module m, we run the full-precision model on the capability probe set X_c and record, for each input channel j, the maximum absolute activation aggregated as a running maximum across probes. Channels are ranked in descending order of s_j, and the top-k_m channels are kept in FP16 while the remaining C_m − k_m channels are quantized to NVFP4. Since LLM activations are heavily skewed, with a small subset of channels carrying disproportionately large magnitudes, this criterion preserves the channels that dominate the module’s computation for the target capability. Crucially, the statistics are computed on the probes of the target capability, so channel selection is itself capability-aware: an allocation driven by math probes preserves the channels that dominate computation on math inputs. The statistics are collected in a single forward pass over the probe set (512 samples × 512 tokens, seed 42) and are shared with the sensitivity-estimation stage, adding no extra calibration cost. Ties are broken by channel index, and selection is performed along the input dimension, consistent with the NVFP4 quantization grouping. This rule is detailed in revised Appendix E.
Response to Reviewer jPzf Q3/Q3
We sincerely appreciate your time and effort in reviewing our manuscript. Below are our responses to your valuable comments.
Q3/W3 (ablation clarity): The “+ linear-only” and “+ block-only” rows are independent alternatives, not cumulative additions: each replaces the sensitivity signal while keeping everything else identical—the same 512 probes, the same NVFP4 quantizer, the same FP16 quota schedule (Eq. 1, g = 128), and the same parameter-weighted bit budget. Concretely, the three sensitivity rows differ only in which output the capability loss is measured on:
- “+ linear-only” measures the behavioral deviation of the module’s own output when that module is quantized—a purely local signal (0.308);
- “+ block-only” measures the deviation of the final output of the module’s parent Transformer block, ignoring the module’s intermediate hidden output—a global signal that captures how quantization errors propagate through the block’s attention–MLP computation (0.319);
- “+ multi-level” combines both signals hierarchically: modules and blocks are ranked separately, fused into levels h(m) = v_lin(m) + v_block(m), and FP16 channels are allocated by the level quota with within-level R(m) refinement (0.334).
This is exactly the controlled comparison requested: under identical probes and identical storage budgets, the hierarchical combination outperforms each single granularity alone (0.334 vs. 0.308 / 0.319), and both single signals outperform random mixed-precision allocation (0.265). The pattern supports the central design rationale: a module that is benign in isolation can still be critical in context, and vice versa—local optimality does not imply global optimality, and neither granularity alone identifies the capability-critical modules. The two signals are complementary, and only their hierarchical combination selects the right modules to protect. Rows 6–7 then add capability-aware probe curation and rank-normalized multi-metric fusion on top of the multi-level configuration, completing the full method (0.386).
We will rewrite the Table 5 caption and the corresponding paragraph in Section 4.4 in the revision to make the alternative-vs-combination structure explicit, and rename “layer-only” to “block-only” for consistency with the terminology defined in Section 3.2.
Response to Reviewer jPzf Q1/W1 (effective-bit accounting)
We thank the reviewer — the original manuscript omitted the bit accounting, which was an oversight. We clarify it here. NVFP4 uses FP8 scales over groups of 16, i.e., 0.5 bits/weight of scale overhead (4.5 bits effective). Our 4-bit baselines (AWQ, GPTQ, OmniQuant, SliM-LLM, QuIP#) use group size 128 with FP16 scales, i.e., 0.125 bits/weight (4.125 bits). To equalize budgets, we additionally assign a random set of 256 FP16 channels to every linear module of each 4-bit baseline. The resulting total effective bit-widths are:
The budgets are matched on Qwen3-8B (within 0.03 bits), and the baselines receive more effective bits than HiCAP-Q on the remaining models — on Qwen3.5-A3B by 0.48 bits, since its many MoE expert modules make the random FP16-channel injection substantially more expensive. We will include this table, the per-component accounting (weights + scales + FP16 channels + occupancy indices), and actual storage sizes in GB in the revised manuscript.