acceptodds
Under review as a conference paper at ICLR 2027

Does Quantization Preserve How Language Models Answer? Causal-Equivariant Quantization for Symbol-Binding Circuits

Abstract

Post-training quantization (PTQ) is typically optimized to preserve weights and activations while maintaining perplexity, or task accuracy. However, these objectives do not reveal whether quantization preserves the causal computation underlying model decisions. Recent studies have localized answer selection and symbol binding in formatted multiple-choice question answering (MCQA) to a sparse set of attention components, making it possible to directly examine their preservation under quantization. To preserve the causal mechanisms underlying formatted MCQA after quantization, we propose Causal-Equivariant Quantization (CEQ), a structured mixed-precision framework consisting of three components: counterfactual causal probing, causal-regret-based precision allocation, and causal-equivariant calibration. First, counterfactual causal probing identifies model components whose causal behavior is most affected by low-bit quantization across different answer formats. CEQ measures these changes using causal-effect distortion (CED) and format-equivariance error (FEE). Second, causal-regret-based precision allocation uses this information to assign higher precision to causally important and quantization-sensitive components under a global bit budget. Third, causal-equivariant calibration (CEQ-Cal) further reduces the discrepancy between the quantized and full-precision models while maintaining consistent behavior across answer formats by preserving output distributions, answer margins, causal effects, and format equivariance. We evaluated CEQ on a novel controlled symbol-binding dataset constructed specifically for this work and on five public MCQA datasets. On the controlled symbol-binding benchmark, CEQ achieves 83.40 ± 14.21% worst-format accuracy at 2.72 effective matrix bits versus 68.00 ± 20.64% for uniform 2-bit RTN. Across the five public MCQA datasets, CEQ consistently improves the worst-format accuracy over the matched TaCQ baseline, achieving an average gain of 9.86% points. The effectiveness of CEQ further generalizes across model scales and architecture families, achieving 76.55% MMLU accuracy on Qwen2.5-32B-Instruct at 2.12 bits and a 67.95% zero-shot average on Llama-3-8B at 4.08 bits. These results show that preserving prediction quality under quantization does not necessarily preserve the underlying causal computation, and motivate mechanism preservation as a distinct objective for low-bit language-model compression.

Primary area:
interpretability and explainable AI
Keywords:
Large Language Model Quantization, Mechanistic Interpretability, Post-Training Quantization, Mixed-Precision Quantization, Model Compression

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_tyFDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates an important yet largely overlooked question in post-training quantization: while existing quantization methods typically optimize local reconstruction error or preserve downstream task accuracy and perplexity, do they actually preserve the underlying causal computation mechanisms driving model decisions? Focusing on symbol-binding circuits in formatted multiple-choice question answering, the authors propose Causal-Equivariant Quantization, a structured mixed-precision framework comprising three stages.

Critical strengths

1.Bridging mechanistic interpretability and model compressionto explicitly distinguish between output preservation and mechanism preservation is highly original and well-motivated. It introduces a valuable conceptual angle for evaluating low-bit compression.

2.The proposed Format-Equivariance Error and Causal-Effect Distortion are mathematically clean and effectively capture both behavioral vulnerability under semantics-preserving format shifts and internal circuit degradation. Section 5.6 convincingly demonstrates strong empirical correlations between these metrics and worst-format accuracy.

3.In the controlled stress tests (Table 2 and Section 5.5), the authors transparently report that while CEQ allocation outperforms simple salience heuristics, it is statistically indistinguishable from strong task-gradient (Gradient-2/8) and Fisher (Fisher-2/8) criteria (with Gradient-2/8 achieving a slightly higher mean worst-format accuracy of 83.88% vs. 83.40%). This candid boundary on their empirical claims is commendable.

Critical weaknesses
  1. In the controlled experiments, CEQ's causal-regret allocation is statistically indistinguishable from standard task-gradient (Gradient-2/8, p=0.403p=0.403) and Fisher (Fisher-2/8, p=0.216p=0.216) criteria, with Gradient-2/8 achieving a slightly higher mean worst-format accuracy (83.88% vs. 83.40%). Since small activation interventions are closely approximated by first-order Taylor expansions (gradients/Fisher), this raises a critical attribution question for the pretrained LLM results (Tables 3 and 4). An ablation comparing Gradient/Fisher allocation + CEQ-CAL on pretrained LLMs is missing.

  2. Computing causal signatures dgfd_g^f and dgbld_g^{b_l} requires running intervention forward passes for every candidate structured group gg across 16 (or 25) counterfactual prompt variants. For large models like Qwen2.5-32B, this exhaustive probing appears computationally prohibitive, yet wall-clock overheads and the exact granularity of group gg in pretrained LLMs are omitted from the main text. Furthermore, the framework relies heavily on explicit option-permutation symmetries (PτP_\tau) specific to formatted MCQA. It remains unclear how Format-Equivariance Error (FEE) and centered intervention signatures can be generalized to open-ended generation or reasoning tasks where such discrete output symmetries do not exist.

  3. The manuscript contains several severe data and formatting errors that hinder evaluation: (a) In Table 4 (MMLU row for CEQ), the standard accuracy (Acc.) is reported as 55.26% while the minimum accuracy across formats (Worst) is reported as 63.40%—by definition, Worst cannot exceed Acc. (and Table 3 reports 63.40% as the standard MMLU accuracy); (b) Table 2 and Line 353 contain highly suspicious leading-digit typos (3.482 for Fisher-2/8 CED instead of 0.482, 3.605 for RTN-2 CED instead of 0.605, and 3.0629 for Magnitude-2/8 FEE instead of 0.0629); (c) Table 5 is severely corrupted in layout with garbled headers ("FITT") and missing method names in the first column; and (d) the mathematical definition of dg(M)d_g(M) in Eq. 4 and the construction of the controlled dataset are omitted from the main text, alongside broken cross-references (e.g., Line 298 citing Table 2 for dataset splits, and Line 313 citing Eq. 1 for the RTN operator).

Questions for authors

1.To isolate the contribution of counterfactual causal probing on large models (e.g., Qwen2.5-7B in Tables 3 and 4), how does a baseline combining standard task-gradient/Fisher precision allocation (or TaCQ's allocation) with your CEQ-CAL calibration perform compared to full CEQ?

2.Could you explain why tuning a single scalar parameter yields such a dramatic improvement, and clarify the practical benefit of the causal/FEE calibration terms given that KL-only achieves slightly higher worst-format accuracy?

3.Could you please verify and correct the swapped Acc. (55.26%) and Worst (63.40%) numbers for MMLU in Table 4, the suspected typos (3.482, 3.605, 3.0629) in Table 2 and Line 353, and the corrupted layout of Table 5? Additionally, please provide the explicit mathematical definition and dimensionality of the intervention signature dg(M)d_g(M) in Eq. 4.

1 comment
AuthorsOpenReview ↗

Additional Allocation Experiments, Preparation Costs, Calibration Benefits, and Clarifications of Results and Definitions

W1/Q1: Allocation benefit

Controlled Base-3L, Deep-4L, and Wide-3L results show competitiveness with Gradient/Fisher. Tables 3-4 compare methods that use different quantizers, so CEQ’s better results may come from both the quantizer and the causal-regret score.

We added a Qwen2.5-7B-Instruct/MMLU comparison fixing checkpoint, candidate groups, symmetric row-wise 2/8-bit quantizer, 2.12-bit weight budget, calibration examples, and evaluation protocol. All use the same CEQ-Cal objective and clipping search, selecting clipping factors independently with frozen weights.

AllocationCalibrationWorst (%)FEECEDCorr.
GradientNone48.60.0620.4020.7540
FisherNone49.00.0600.3950.7610
CEQNone50.20.0550.3700.7850
GradientCEQ-Cal52.40.0490.3500.7920
FisherCEQ-Cal52.70.0480.3460.7980
CEQCEQ-Cal54.00.0430.3220.8200

CEQ exceeds Gradient/Fisher by 1.60/1.20 percentage points before calibration and 1.60/1.30 afterward. Calibration adds 3.80/3.70/3.80 points for Gradient/Fisher/CEQ. CEQ performs better than Gradient/Fisher with the same CEQ-Cal in this comparison.

Manuscript Changes: Sections 5.4/5.7 distinguish allocation and calibration gains; Section 4.1 explains the gradient comparison.

W2: Cost and scope

Each candidate is one query head. We evaluate 784 query heads in Qwen2.5-7B, 2,560 in Qwen2.5-32B, and 1,024 in Llama-3-8B. Shared GQA K/V weights are protected jointly and counted once. Probing uses 128 calibration examples with 16 four-choice variants, or 25 five-choice variants. Preparation time and peak memory usage on an NVIDIA A100 80GB GPU:

ModelProbing (min)Quant./cal. (min)Total (min)Peak GB
Qwen2.5-7B11.67.819.426.2
Qwen2.5-32B46.731.978.674.6
Llama-3-8B9.78.518.226.7

Probing is offline and is not repeated during inference.

FEE requires an output correspondence, supplied exactly by answer permutations in MCQA. For reasoning, mapping final answers to a common representation could measure answer consistency, not preservation of reasoning. Centering $dg(t)−∣T∣−1∑t′dg(t′)d_g(t)-|\mathcal T|^{-1}\sum_{t'}d_g(t') $requires comparable conditions, not discrete permutations. Meaning-preserving prompts and shared reference-continuation log-likelihood offer a possible formulation for generation.

Changes: Section 5.4 adds group counts and costs. Discussion clarifies alignment, centering, and generalization.

Q2: Why can one clipping scalar help?

The scalar changes clipping thresholds and quantization scales across many weight rows. At low precision, it balances clipping outliers against resolving smaller weights. It can therefore change many predictions without training base-model weights.

In Supplementary Section 6.3, Table 7, KL-only achieves 95.44% worst-format accuracy versus 95.16% for CEQ-Cal. CEQ-Cal changes FEE from .0159 to .0156, CED from .369 to .365, and correlation from .884 to .895. The paired seed-wise results show that four seeds select identical clipping factors, while one accounts for the diagnostic improvement and accuracy decrease. Under the evaluation protocol, CED/correlation here are calibration-set diagnostics, whereas accuracy/FEE use held-out examples. Most accuracy recovery comes from clipping calibration. Additional terms offer modest, seed-dependent diagnostic benefits, without establishing necessity.

Manuscript Changes: Section 5.7 and Supplementary Section 6.3 explain clipping, paired results, and diagnostic scope.

W3/Q3: Corrections and definitions

We verified that the manuscript already contains the correct values: MMLU Acc. 63.40%, Worst 55.26%, Fisher CED �2��2�, RTN-2 CED �3��3�, and Magnitude FEE �4��4�. Table 5 correctly displays all method names and evaluation headers. These entries required no numerical corrections.

For condition (c), the signature entry is $

dg,c(M)=∣Dc∣−1∑x∈Dc[mM(x)−mM∖g(x)],d_{g,c}(M)=|\mathcal D_c|^{-1}\sum_{x\in\mathcal D_c}[m_M(x)-m_{M_{\setminus g}}(x)],

$ where �5��5� is the correct-answer logit minus the largest incorrect-answer logit. Thus �6��6�. Controlled probing combines four alphabet averages and four position averages into eight entries. The alphabet term subtracts the mean of the four alphabet effects from each effect.

Each controlled specification samples a target position, four distinct content identifiers, and a label permutation. The target is the symbol at that position. Base-3L uses 60 calibration specifications, four additional position-controlled subsets of 15, and 500 held-out specifications across four alphabets.

Manuscript Changes: Section 4.1 defines the signature and experimental setup explains dataset construction. References identify Supplementary Section 2/Table 2 and Section 4.3/Eq. (1). Results remain unchanged.

Reviewer_ghU5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether post-training quantization (PTQ) can preserve the internal causal computation of language models while maintaining their prediction performance, mainly focusing on answer selection and symbol binding in multiple-choice question answering tasks.

The authors propose a framework called Causal-Equivariant Quantization (CEQ), which consists of three stages. First, it measures the changes in causal effects of attention components before and after quantization through semantically equivalent answer formats and causal interventions. Second, it uses a causal-regret score to assign higher precision to important and quantization-sensitive components under a limited bit budget. Finally, Causal-Equivariant Calibration (CEQ-Cal) is introduced to further preserve output distributions, answer margins, causal effects, and consistency across different answer formats.

Experiments on several small Transformer models and pretrained language models show that CEQ can improve prediction performance and format robustness under low-bit quantization, while reducing causal-effect distortion. Based on these results, the authors suggest that preserving causal mechanisms should be considered as an independent optimization objective for language model quantization.

Critical strengths

1. Good research problem.

This paper considers whether quantization preserves the internal computational mechanisms of language models as an evaluation aspect different from traditional prediction accuracy. This is a meaningful research direction. Preserving the internal computational mechanisms may be related to the robustness and generalization ability of quantized models, so this problem is worth further investigation.

2. The experiments cover a wide range of settings and show some practical effectiveness.

The authors conduct experiments on several small Transformer architectures and pretrained language models, covering different quantization precisions and multiple public question-answering datasets. On pretrained models, CEQ achieves better prediction accuracy and format robustness than some existing quantization methods. These results suggest that the proposed method has some practical potential for low-bit quantization.

Critical weaknesses

W1. Insufficient experimental evidence to demonstrate the independent advantage of the proposed method.

The main contribution of this paper is to use causal intervention information to guide mixed-precision quantization. However, in Table 2, where the quantization operator, bit budget, and evaluation settings are strictly controlled, CEQ achieves lower worst-format accuracy than Gradient, higher FEE than both Gradient and Fisher, and almost the same CED as Gradient. The cross-architecture experiments in the appendix also show no statistically significant advantage. In contrast, the larger improvements on pretrained models come from comparisons between different quantization pipelines, which the authors acknowledge do not use identical quantization operators. Therefore, it is difficult to attribute these improvements specifically to the causal-regret score, which weakens the paper's main methodological contribution.

W2. Some experimental results contradict the claimed contributions, and important implementation details are unclear.

The paper emphasizes causal mechanism preservation and format equivariance, but some ablation results raise doubts about their necessity. Appendix Section 11.2 shows that ABCD-only causal probing achieves almost the same results as using all formats. Appendix Section 6.3 also shows that accuracy recovery mainly comes from ordinary clipping calibration, with KL-only calibration achieving slightly higher worst-format accuracy than CEQ-Cal, while the additional improvements in causal metrics are small.

Furthermore, the paper does not clearly explain how attention groups are defined in pretrained models, especially how shared K/V heads are handled in Grouped-Query Attention (GQA) and how the corresponding bit budget is calculated. The quantization operator and its configuration used to construct the candidate low-bit model in Stage 1 are also unclear. These missing details make it difficult to understand and reproduce the method.

Overall, these issues make it difficult to clearly assess the independent contributions and effectiveness of the proposed method.

Questions for authors

Q1. About the independent advantage of the causal-regret score and the fairness of experimental comparisons.

In Table 2 and the cross-architecture experiments in the appendix, CEQ achieves similar performance to Gradient/Fisher-based methods, but shows larger improvements over TaCQ on pretrained models. However, in Section 5.4, the authors explicitly state that the comparisons on pretrained models "should not be interpreted as implying an identical quantization alphabet or quantization operator across methods." Since the authors have already acknowledged these differences, why are these results still used to support the performance advantage of CEQ? How do the authors ensure the fairness of these comparisons and exclude the influence of other factors, such as different quantization operators? Could the authors provide additional experiments on pretrained models using the same quantization operator, bit budget, and calibration method to compare CEQ with Gradient/Fisher-based precision allocation strategies and verify the independent contribution of the causal-regret score?

Q2. About the necessity of causal mechanism preservation and format equivariance.

Appendix Section 11.2 shows that causal probing using only the ABCD format achieves almost the same results as using all formats. In addition, Appendix Section 6.3 shows that KL-only calibration achieves slightly higher worst-format accuracy than the full CEQ-Cal. Could the authors further explain what independent benefits are provided by cross-format causal probing and causal-equivariant calibration? Is there more direct experimental evidence showing that these designs are necessary for improving model robustness and preserving causal mechanisms?

Q3. About the implementation details on pretrained models.

Could the authors clarify how candidate attention groups are defined in pretrained models? In particular, for models using Grouped-Query Attention (GQA), how is precision independently allocated when multiple query heads share the same K/V heads, and how is the corresponding bit budget calculated? In addition, what specific quantization operator and configuration are used to construct the candidate low-bit model in Stage 1? Are they consistent with those used in the final quantized model?

I am willing to increase my rating if the authors can provide stronger evidence and satisfactory answers to these questions.

2 comments
AuthorsOpenReview ↗

Additional Experiments and Clarifications on CEQ’s Benefits, Calibration, and Pretrained Quantization Setup

W1/Q1:

Pipeline performance and the allocation score require different comparisons. Tables 3 and 4 compare complete pipelines at approximately matched precision; differences in quantizers and calibration prevent attributing their full improvement over TaCQ exclusively to causal-regret scoring. Controlled results on Base-3L, Deep-4L, and Wide-3L establish competitiveness with Gradient/Fisher, rather than consistent superiority.

We have added a Qwen2.5-7B-Instruct/MMLU comparison using the same checkpoint, candidate groups, symmetric row-wise 2/8-bit quantizer, 2.12-bit effective weight budget, calibration examples, and evaluation protocol. All allocations use the same CEQ-Cal objective and clipping search. The clipping factor is selected separately for each allocation, while base-model weights remain frozen.

AllocationCalibrationWorst acc. (%)FEECEDCorr.
GradientNone48.600.06200.40200.7540
FisherNone49.000.06000.39500.7610
CEQNone50.200.05500.37000.7850
GradientCEQ-Cal52.400.04900.35000.7920
FisherCEQ-Cal52.700.04800.34600.7980
CEQCEQ-Cal54.000.04300.32200.8200

CEQ exceeds Gradient/Fisher by 1.60/1.20 percentage points before calibration and 1.60/1.30 afterward, with better diagnostics. Calibration improves accuracy by 3.80, 3.70, and 3.80 points for Gradient, Fisher, and CEQ. Thus, CEQ’s numerical advantage precedes calibration and persists when competing allocations also receive CEQ-Cal.

Manuscript changes: Section 5.4 separates this comparison from the original pipeline comparisons. Section 5.7 explains the allocation/calibration distinction. Discussion retains the controlled results’ competitive interpretation.

W2/Q2:

ABCD-only discovery still uses causal interventions. Its comparable performance across all alphabets demonstrates transfer beyond the discovery alphabet, it does not remove intervention-based allocation. This supports transfer, but does not establish that all-format discovery is necessary. Cross-format evaluation measures robustness across presentations.

CEQ-Cal targets measured intervention-response preservation alongside predictive performance. With the same models, allocation, quantizer, and calibration examples:

MetricKL-onlyCEQ-Cal
Worst-format accuracy (%)95.4495.16
FEE0.01590.0156
CED0.3690.365
Causal-effect correlation0.8840.895

The higher-accuracy solution need not have the strongest diagnostic agreement. Four seeds select identical clipping factors, one accounts for the diagnostic improvement and an accuracy decrease. CED and correlation here are calibration-set diagnostics, whereas, accuracy and FEE use held-out examples.

The evidence supports CEQ-Cal as a multi-objective option, not a necessary source of additional accuracy. Neither this ablation nor the new allocation comparison separately establishes the necessity of each causal/equivariance term.

Manuscript changes: Appendix Sections 11.2 and 6.3 clarify format transfer, seed-wise results, and evaluation splits. Section 5.7 and Discussion distinguish accuracy recovery from diagnostic preservation and restrict preservation claims to measured intervention responses.

W2/Q3:

Each candidate is a query head identified by layer and head index. Its effect is measured by ablation before the attention output projection. Selecting it protects its Q-projection rows, output-projection input columns, and associated K/V rows.

In GQA, query heads are scored individually, but shared K/V precision is coupled. Selecting any associated query head protects those shared parameters. Let �2��2� be the union of protected weight elements and �3��3� all weights in the quantized-weight budget: $

beff=bh∣P(S)∣+bl(N−∣P(S)∣)N.b_{\mathrm{eff}}=\frac{b_h|P(S)|+b_l(N-|P(S)|)}{N}.

$ Shared K/V elements are counted once. Equal head counts need not imply equal budgets. Scales, metadata, packing overhead, and retained full-precision tensors require separate storage accounting.

Stage 1 uses symmetric row-wise round-to-nearest quantization: $

qmax⁡=2b−1−1,s=max⁡(max⁡i∣wi∣,10−8)qmax⁡,w^=s clip(round(w/s),−qmax⁡,qmax⁡).q_{\max}=2^{b-1}-1,\quad s=\frac{\max(\max_i|w_i|,10^{-8})}{q_{\max}}, \quad\widehat w=s\,\mathrm{clip}(\mathrm{round}(w/s),-q_{\max},q_{\max}).

$ Stage 1 does not use a zero-point or search for a clipping factor. Under this quantization convention, nominal 2-bit quantization uses three levels. The final simulation uses the same quantizer family, requantizing protected slices from original weights at higher precision. Scale-sharing boundaries can change, however, CEQ-Cal selects clipping parameters with fixed allocation.

Manuscript changes: Sections 4.1-4.2 define candidates, GQA coupling, and unique-parameter budgets. Pretrained experimental details specify the operator and distinguish the shared quantizer family from parameters selected after allocation and calibration.

Thank you for the clarifications and additional experiments. Single-format probing performed comparably to all-format probing, and it remains unclear whether CEQ-Cal preserves intervention responses better than KL-only calibration on held-out examples. I therefore find the evidence for the added value of these components limited and will maintain my current rating.

Reviewer_2DHnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CEQ, a mixed-precision quantization framework for formatted MCQA. The key idea is to identify components whose causal behavior is most affected by quantization and protect them with higher precision. To do so, the authors construct semantically equivalent MCQA formats by modifying answer positions and answer symbols, compare intervention effects between full-precision and quantized models, and define metrics such as Causal-Effect Distortion (CED) and Format-Equivariance Error (FEE) to characterize these differences. These signals are combined into a causal-regret score, which is used to estimate component importance and perform bit allocation under a global precision budget. A subsequent calibration stage further optimizes quantizer parameters to better preserve the behavior of the full-precision model.

Critical strengths
  1. The paper explores a novel perspective on quantization by examining changes in component-level intervention effects rather than relying solely on output accuracy. The empirical evaluation is reasonably comprehensive, with controlled comparisons, ablation studies, and analyses across multiple model settings.
Critical weaknesses
  1. The paper focuses on a highly specialized MCQA setting.

Most of the proposed concepts and objectives are tightly coupled to answer-symbol binding in multiple-choice problems, making it difficult to assess their relevance beyond this particular scenario. While MCQA provides a clean and controllable environment for analysis, it is often used as an evaluation protocol rather than a primary deployment setting for modern LLMs. More importantly, many key components of the proposed framework rely on the existence of discrete answer options and controlled answer-format modifications, which do not naturally exist in general autoregressive generation tasks. As a result, the broader significance and applicability of the proposed framework remain unclear.

  1. The presentation could be improved in terms of clarity and simplicity.

The paper introduces many causal and equivariance related concepts, metrics, and objectives, which makes the method appear more complicated than necessary. Much of the framework could potentially be described more directly in terms of component importance estimation and bit allocation. The current presentation creates substantial conceptual overhead and hurts readability.

Questions for authors

I do not have any major clarification questions. My concerns are primarily about the scope and significance of the studied problem rather than missing experimental details.

Reviewer_qZDyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Causal-Equivariant Quantization (CEQ), which allocates mixed precision using quantization-induced changes in component-level intervention effects. CEQ-CAL then calibrates quantizer parameters to preserve output distributions, intervention responses, and consistency across answer formats. Experiments combine controlled symbol-binding tasks with pretrained Qwen2.5 and Llama models. The results support useful robustness gains and improved preservation of measured intervention responses, particularly under aggressive low-bit quantization.

Critical strengths
  1. The controlled evaluation is thorough and informative. Matched-budget comparisons span three architectures and five seeds, with paired statistics, ablations, and explicit interaction analyses. Exhaustive enumeration places CEQ near the best three-head allocations in the controlled setting. Reporting both competitive gradient/Fisher baselines and negative interaction-aware results makes the empirical conclusions more credible.

  2. The pretrained results demonstrate practical value beyond the synthetic task. Across five public MCQA datasets, CEQ improves worst-format accuracy over TaCQ by 9.86 percentage points on average, while also improving FEE, CED, and causal-effect correlation. The 32B results provide further performance evidence, and measured preparation costs and decoding throughput support the method’s practical feasibility.

Critical weaknesses
  1. The additional benefit of the causal objective remains only partly isolated. The paired controlled results establish competitiveness with gradient/Fisher selection, rather than a distinct advantage. CEQ-CAL also achieves similar accuracy to KL-only, while its diagnostic gains are small (Supplementary Table 7). Paired uncertainty for these calibration gains and a pretrained ablation using the same quantizer and calibration procedure would clarify what the additional causal terms contribute.

  2. The mechanism-preservation interpretation should be narrower. CED depends on raw logit-margin scale, while FEE measures output consistency; neither alone establishes preservation of the full computation. The reported gains in causal-effect correlation provide useful complementary evidence, but scale-normalized or behavior-matched checks would strengthen the interpretation. The broad claims about preserving how answers are computed should therefore be limited to the intervention responses actually tested.

Questions for authors
  1. How stable are the CEQ-CAL versus KL-only diagnostic gains across seeds?

  2. Do the CED improvements persist after accounting for logit scale or matching output confidence? How does this affect the scope of the mechanism-preservation claim?

  3. Could you specify the pretrained quantization groups, protected-group configurations, metadata accounting, CEQ-CAL settings, and decoding setup needed to reproduce the reported results?

1 comment
AuthorsOpenReview ↗

Seed-Wise Calibration Results, Pretrained Allocation Comparison, and Clarifications on Intervention Preservation and Reproducibility.

W1/Q1:

We compared KL-only and CEQ-Cal using the same five Base-3L models, CEQ allocations, 2/8-bit quantizer, and calibration examples. Supplementary Table 7 reports worst-format accuracy of 95.44% versus 95.16%, FEE .0159 versus .0156, CED .369 versus .365, and causal-effect correlation .884 versus .895.

Differences are CEQ-Cal minus KL-only. SD is the sample SD across five seeds.

SeedsΔ accuracy (pp)Δ FEEΔ CEDΔ correlation
0, 2, 3, 4 (each)0000
1−1.40−.001265−.017998+.057569
Mean ± SD−.28 ± .63−.000253 ± .000566−.003600 ± .008049+.011514 ± .025746

Four seeds select identical clipping factors and models. Seed 1 changes from .80 to .65, improving all three diagnostics while reducing accuracy. Thus, the diagnostic comparison shows one improvement and four ties, with no diagnostic regressions. Accuracy/FEE use 500 held-out specifications per seed, whereas, CED/correlation use 60 calibration specifications. These results demonstrate comparable predictive performance with modest, seed-dependent diagnostic gains.

We also added a Qwen2.5-7B-Instruct/MMLU allocation comparison fixing the checkpoint, candidate groups, symmetric row-wise 2/8-bit quantizer, 2.12-bit weight budget, calibration examples, and evaluation protocol. All allocations use the same CEQ-Cal objective/grid, selecting clipping independently with frozen weights.

AllocationCalibrationWorst (%)FEECEDCorr.
GradientNone48.60.0620.4020.7540
FisherNone49.00.0600.3950.7610
CEQNone50.20.0550.3700.7850
GradientCEQ-Cal52.40.0490.3500.7920
FisherCEQ-Cal52.70.0480.3460.7980
CEQCEQ-Cal54.00.0430.3220.8200

CEQ exceeds Gradient/Fisher by 1.60/1.20 percentage points before calibration and 1.60/1.30 afterward, with better diagnostics. Its numerical advantage persists when competing allocations also receive CEQ-Cal, supporting the benefit of its allocation strategy in this setting. These results complement the competitiveness established on Base-3L, Deep-4L, and Wide-3L.

Manuscript changes: Sections 5.4/5.7 and Supplementary Section 6.3 distinguish allocation gains, calibration effects, paired uncertainty, and evaluation splits.

W2/Q2: Logit scale and scope of preservation

Higher Pearson correlation provides complementary evidence of intervention-response preservation because positive global rescaling cannot explain its improvement. For Table 7 comparison, this supports agreement beyond uniform rescaling of intervention effects.

We distinguish two CED calculations. Eq. 3 measures discrepancies in raw logit-margin units, whereas Supplementary Table 7 uses the reference-normalized distance between mean intervention-signature vectors: $

∥dQ−df∥2/(∥df∥2+ϵ).\|\mathbf d^Q-\mathbf d^f\|_2/(\|\mathbf d^f\|_2+\epsilon).

$ Reference normalization expresses distortion relative to the full-precision signature, but retains sensitivity to relative logit-scale differences. We therefore interpret CED alongside correlation rather than treating reference normalization as scale matching.

Our claim concerns preservation of measured intervention responses of probed answer-selection components under tested transformations and interventions. FEE measures behavioral consistency across formats, while CED and correlation assess intervention-response agreement. This operational definition connects the preservation claim directly to the evidence and distinguishes it from preservation of the complete computation.

Manuscript changes: The CED discussion identifies both calculations and their interpretation. Discussion and Conclusion define preservation through the tested intervention responses.

Q3:

Candidates are query heads indexed by layer and head: 784 for Qwen2.5-7B, 2,560 for Qwen2.5-32B, and 1,024 for Llama-3-8B. Selection protects Q rows, output-projection input columns, and associated K/V rows. In GQA, shared K/V precision is coupled and these weights are counted once.

For unique protected elements �2��2� among �3��3� quantized-weight elements, $

beff=[bh∣P(S)∣+bl(N−∣P(S)∣)]/N.b_{\mathrm{eff}}=[b_h|P(S)|+b_l(N-|P(S)|)]/N.

$ This accounting avoids double-counting shared K/V parameters and expresses precision in terms of protected weights.

The configurations are Qwen7B 2/8-bit (16 protected heads) and 3/8-bit (19), Qwen32B 2/8-bit (51), and Llama8B 4/8-bit (20). Quantization is symmetric, row-wise, and weight-only. CEQ-Cal uses 128 examples, clipping grid .65:.05:1.05, and median-normalized KL/margin/causal/equiv-ref/FEE weights 1/.25/1/.50/.125. Allocation and base weights remain fixed.

We measure decoding on an NVIDIA A100 80GB GPU with BF16 activations/KV cache, batch size 1, 512 input tokens and 128 output tokens, and greedy decoding. We perform 10 warm-up runs, followed by 50 timed runs. Probing is completed beforehand.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.