acceptodds
Under review as a conference paper at ICLR 2027

Where Compression Hurts: Function-Space Sensitivity for Mixed-Precision Neural Network Quantization

Abstract

Selecting layer-wise precision is a central combinatorial problem in mixed-precision post-training quantization and requires a reliable surrogate for the damage induced by each compression action. Prior work has studied this problem from several perspectives, focusing on how compression-induced perturbations affect model parameters, optimization objectives, or model outputs. We study a complementary question: how should a quantization direction be measured in output space, and what additional information is preserved by retaining its first-order tangent response rather than reducing it to a scalar sensitivity score? For a compression action inducing , we analyze the first-order logit perturbation and the associated quadratic form. The resulting sensitivity is specific to the chosen compression direction, is label-independent once the calibration inputs are fixed, and can be reused across different first-order analyses. Across ResNet-18, MobileNetV3-Small, and MobileNetV2, this function-space sensitivity provides an informative ranking of realized compression damage. Its local linear response closely tracks directly measured output distortion under isolated compression actions, while severity-controlled analyses show that the ranking cannot be explained solely by compression magnitude. We further find that the benefit of modeling tangent interference is network-dependent rather than universal. Under budgeted precision allocation, improved prediction of function-space distortion can also lead to better allocation decisions when compression actions are sufficiently distinguishable. These results identify Euclidean logit geometry as a reusable function-space complement to parameter-based, loss-based, and directly measured output-space sensitivity.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Compression, Functional Sensitivity, Mixed-Precision Neural Network, Quantization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tVSSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a function-space sensitivity measure for mixed-precision neural network quantization. Instead of measuring quantization damage only through parameter changes or loss curvature, the method uses JVPs to estimate how specific quantization perturbations affect the final model outputs. It also considers cross-layer tangent interference and uses the resulting sensitivity scores for budget-constrained bit allocation. Experiments on ResNet-18, MobileNetV3-Small, and MobileNetV2 compare different sensitivity measures and evaluate their prediction accuracy and allocation performance.

Critical strengths
  1. The paper provides a useful comparison of parameter-space, loss-space, and function-space sensitivity. The proposed measure achieves strong correlations with actual quantization damage, especially on ResNet-18, and the experiments include several meaningful baselines.
  2. The analysis of cross-layer interference is interesting. The authors also evaluate whether better sensitivity prediction actually leads to better bit allocation, rather than relying only on correlation metrics. These results help clarify both the usefulness and limitations of the proposed approach.
Critical weaknesses
  1. My biggest concern is whether the proposed JVP-based sensitivity provides sufficient practical advantages over directly measuring isolated output distortion. In Table 2, the direct KL baseline achieves a correlation of 0.805 on MobileNetV3-Small, compared with 0.705 for additive functional sensitivity and 0.776 with tangent interference. More importantly, Appendix Table 16 shows that JVP profiling is actually slower than direct isolated profiling (93.81s vs. 16.35s on MobileNetV3-Small). This makes the practical motivation less convincing.
  2. The novelty compared with existing output-space methods is somewhat limited. The paper acknowledges that Jacobian-projected quantization perturbations have already been studied in BLOB-Q. The main distinction seems to be the choice of Euclidean logit geometry and the retention of tangent responses. However, the results do not clearly demonstrate that this choice is consistently better than other output-space metrics, especially CE-GGN.
  3. The experimental scope is relatively narrow. All experiments use Imagenette and three CNN architectures, with full allocation experiments on only two of them. The improvement from tangent interference is also not consistent across architectures. It would be useful to see whether the proposed sensitivity remains effective on larger models or more challenging quantization settings.
Questions for authors
  1. Given the higher profiling cost and mixed prediction results, what is the main practical advantage of JVP sensitivity over direct isolated-output measurements?
  2. Can the authors provide stronger evidence that the proposed output geometry offers advantages over existing Jacobian-based sensitivity methods?
  3. Does the improvement in final bit allocation generalize beyond the currently tested CNN architectures and Imagenette?
2 comments
AuthorsOpenReview ↗

Response to the reviewe

We thank the reviewer for the constructive assessment. We agree that the key issues are the practical role of JVPs, the distinction from prior Jacobian/output-space methods, and the experimental scope. We revised the paper and added direct-vector, low-bit, and exact-budget controls to sharpen these claims.

1. What is the practical advantage over direct isolated-output measurements?

We agree that JVP profiling is not cheaper in our current implementation. Direct isolated profiling takes 56.89 s vs. 194.07 s on ResNet-18 and 16.35 s vs. 93.81 s on MobileNetV3-Small. We have removed any implication of a profiling-speed advantage.

We also added the stronger control suggested by the reviews: cache the exact isolated output vectors

dl,a(x)=fθ+Δθl,a(x)−fθ(x)d_{l,a}(x)=f_{\theta+\Delta\theta_{l,a}}(x)-f_\theta(x)

and compose them directly. On MobileNetV3-Small, this gives

\rho=0.846\;(\mathrm{logit\ MSE}),$$ compared with $0.776/0.802$ for JVP tangent interaction. Thus, exact direct vectors can indeed outperform the first-order representation. This changes our claim. The practical value of JVPs in this study is **analytical and reusable rather than computational**: the same cached compression-direction responses support additive scoring, alternative output geometries such as CE-GGN, and explicit cross-layer interference analysis without further quantized-model evaluations. We no longer claim that vector preservation itself is unique to JVPs. ### 2. Novelty relative to BLOB-Q and other Jacobian-based methods We agree that neither Jacobians nor final-output sensitivity are new, and we now state this explicitly. BLOB-Q already uses Jacobian-projected quantization perturbations for output-distortion-based allocation, and Chen et al. derive related Jacobian-mediated loss geometry. Our contribution is therefore narrower: we use the common form $$S_M(\Delta w)=\Delta w^\top J^\top M J\Delta w$$ to isolate what changes when the **same quantization perturbation** is measured under different output geometries and when its vector response is retained rather than immediately reduced to a scalar. This produces several distinctions not available from a single scalar sensitivity score: - Euclidean logit geometry ($M=I$) vs. CE-GGN loss geometry; - first-order JVP responses vs. exact nonlinear isolated responses; - scalar direct costs vs. vector-preserving direct composition; - additive scoring vs. explicit first-order cross-layer interference. The results do **not** show that Euclidean geometry universally dominates. On ResNet-18 it is significantly stronger than additive CE-GGN for KL and logit MSE, while CE-GGN is stronger for Top-1; on MobileNetV3-Small, all paired functional-vs-GGN confidence intervals cross zero. We therefore revised the paper to present the contribution as a controlled geometric/diagnostic framework rather than a universally superior metric. ### 3. Generalization beyond the current CNN/Imagenette setting We agree that the experimental scope remains limited and do not claim generalization to transformers or full ImageNet-scale evaluation. We did, however, add two checks that broaden the original evidence. First, MobileNetV2 provides an external architecture check: additive functional sensitivity remains strong, with $$\rho=0.935\;(\mathrm{KL}),\qquad \rho=0.904\;(\mathrm{logit\ MSE}),$$ while tangent interaction does not improve ranking. This reinforces that interaction benefits are architecture-dependent rather than universal. Second, we added controlled INT4/INT3/INT2 experiments on ResNet-18. Local JVP-vs-exact correlation decreases from $0.990$ at INT4 to $0.887$ at INT3 and $0.687$ at INT2. Nevertheless, viable INT3 configurations remain strongly ranked: $$\rho_{\mathrm{KL}}=0.829,\qquad \rho_{\mathrm{logit}}=0.936.$$ INT2 then exposes the expected breakdown of the local first-order approximation. Finally, all allocation comparisons now use the same exact parameter-bit constraint and full 3,925-image validation set. These experiments show useful allocations in some regimes, but also confirm that stronger global ranking does not guarantee better allocation at every budget. ### Summary We have therefore narrowed the paper's claim. JVP sensitivity is not presented as universally faster or superior to direct output measurements, nor as a new use of Jacobians in quantization. Its contribution is a reusable first-order representation that makes output geometry, local linearization error, vector retention, and cross-layer composition separately testable. We also now state clearly that validation on larger datasets and transformer architectures remains future work.
AuthorsOpenReview ↗

Additional Details on Transformers

Following the reviewer's concern about evaluation only on CNN architectures, we added an ImageNet-pretrained DeiT-Tiny transformer as a cross-family architecture check, using the same weight-only FP32/INT8/INT4 protocol.

Across 125 heterogeneous configurations, additive functional sensitivity remains strong:

\rho_{\mathrm{logit}}=0.965.$$ This also provides stronger evidence concerning output geometry. CE-GGN obtains $0.920/0.886$, and the functional-minus-GGN improvements are statistically significant for both KL and logit MSE: $$+0.0256\;[0.0015,0.0542], \qquad +0.0798\;[0.0490,0.1230].$$ The transformer produces a different interaction pattern from MobileNetV3-Small: adding tangent-interference terms decreases correlation to $0.921/0.937$, with both paired decreases statistically significant. This strengthens the conclusion that the value of cross-layer tangent composition is architecture-dependent. We additionally performed exact-budget allocation at $B=8,9,10,11$ and re-evaluated selected configurations on all 3,925 Imagenette validation images. All objectives coincide at $B=8$; at higher budgets, functional, direct-output, and CE-GGN allocations remain closely competitive with small target-dependent differences. We therefore no longer restrict the empirical evidence to CNNs. We still avoid claiming broad transformer generalization from a single DeiT-Tiny model and explicitly retain larger datasets and larger/diverse transformer families as limitations.
Reviewer_7TuBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper study how to estimate the error induced by quantizing the different layers of a network to assign them different bit-width, under memory constraints.

For each candidate quantization, the authors approximate its effect on the network's output, using a Jcobian Vector Product (JVP). They use the norm of the displacement as a sensitivity score. By keeping track of the vectors, rather than juste their norms, they can also study how perturbations in different layers can increase or balance one other perturbation.

The experiments focus on three pre-trained CNNs, evaluated on Imagenette. They compare several sensitivity metrics, the classification quality of the different configurations and evaluate accuracy of allocations across two architectures. The results demonstrate good classification performances. However the significance of the interactions depends of the architecture and the metric which is evaluated.

Critical strengths

1. A clear and interpretable mathematical framework

The paper allows the comparaison of several criteria applied to the same quantization perturbation. It explicitly tells the difference between weights changes, shift outputs and loss variations. Keeping the vector responses makes possible to separate individual contributions and their reinforcing or compensating effects. The authors correctly specify the limitations of this analysis, explicitly writing that is a first order approximation and does not account for all interactions within the quantized network.

This formulation is a strength because it makes comparisons understandable and allows for the interpretation of results, even if, it does not constitute a new theoretical guarantee.

2. Informative empirical validations and limitations

The functional score provides a highly accurate ranking on ResNet18 and good results of MobileNetV2. The experiments demonstrate that a limited number of calibration images can be enough to achieve stable rankings under the conditions studied.

The interaction analysis provides a useful insight into : retaining the vectors which improves predictions on MobileNetV3-Small, even if this benefit is not consistent across other architectures. The allocation experiments also show that a better overall ranking does not guarantee a better decision at every budget.

The scientific strength lies if these results and distinctions. The explicit presentation of adverse cases reinforces their credibility.

Critical weaknesses

The added value of the JVP representation compared to the other metrics compared remains insufficiently established

Direct measurements of the degradation induced by a single quantized layer are often just as good or better at classifying configurations. For exemple on MobileNet-V3-Small, the LK ranking achieves a correlation of 0.805 with direct costs compared to 0.705 for the additive functional and 0.776 with interactions.

Direct profiling is also less computationally expensive in the evaluated implementation with 16.35 seconds compared 93.81 on this architecture.

The paper retains diagnostic value and does not need to outperform all the methods compared to be useful. However, since vector preservation is a central contribution, it would be important to better isolate exactly what the tangential response contribute. The current comparison uses only scalar costs sums for the direct measurements. This limitation reduces my confidence in the significance of the additional contribution rather Thant in the validity of the method itself.

Questions for authors

1. What is the specific benefit of the JVP representation over direct output measurements ?

Table 12 and 16 show that direct isolated output tracking is competitive and cheaper in the reported implementation. What happens if the direct output difference vectors are also cached combined using the norm objective in Equation 11 ?

2. How do allocations based on functional, direct output, dans CE-GGN scores compare at the same bit budgets and on the same validation set ?

This would clarify whether differences in prediction quality lead to better allocation decisions.

3. How were the 125 heterogeneous configurations sampled ?

Does the ranking remain informative among configurations with similar average bit widths, particularly around the budgets used is the allocation experiments ?

3 comments
AuthorsOpenReview ↗

Responses to the reviewer

We thank the reviewer for the careful assessment. We agree that the key question is not whether direct profiling can be competitive; it is what additional information the JVP representation provides. We added the requested direct-vector control, same-budget allocation comparisons, and explicit sampling/budget-local analyses.

1. What does JVP add beyond direct output measurements?

We now cache the exact isolated output difference

dl,a(x)=fθ+Δθl,a(x)−fθ(x)d_{l,a}(x)=f_{\theta+\Delta\theta_{l,a}}(x)-f_\theta(x)

and compose these vectors with the same norm objective used for tangent responses.

On MobileNetV3-Small, direct-vector composition reaches

\rho=0.846\;(\mathrm{logit\ MSE}),$$ versus $0.796/0.799$ for scalar direct-logit summation and $0.776/0.802$ for JVP tangent interaction. The direct-vector gain over scalar direct-logit is $+0.0508$ for KL (95% CI $[0.0238,0.0839]$) and $+0.0473$ for logit MSE ($[0.0194,0.0804]$). This clarifies the contribution: **vector preservation is not unique to JVPs**, and exact nonlinear vectors can be more predictive. We therefore no longer claim that JVP is a cheaper or universally better estimator. In our implementation, direct profiling is indeed faster (16.35 s vs. 93.81 s on MobileNetV3-Small). The specific value of JVP is analytical and reusable: the same cached first-order responses provide an explicit compression-direction decomposition and support additive scoring, cross-layer interference, and alternative output geometries such as CE-GGN without re-evaluating compressed models. ### 2. Same-budget allocation comparison We reran functional, direct-output, and CE-GGN allocation under the same **exact parameter-bit constraint** and evaluated selected configurations on all 3,925 validation images. For ResNet-18, Functional, Direct-logit, Direct-KL, and CE-GGN select the same configuration at $B=8$ and $B=9$. At $B=8$, this is uniform INT8: $$\mathrm{KL}=2.762\times10^{-3},\qquad \mathrm{logit\ MSE}=9.466\times10^{-3}.$$ At $B=10$ and $B=11$, the objectives differ slightly; for example, at $B=11$, Functional gives KL $3.48\times10^{-4}$ at 10.789 average bits, while Direct-KL/CE-GGN gives $3.54\times10^{-4}$ at 10.822 bits. For MobileNetV3-Small, the decision depends on both budget and target. At $B=8$, Functional gives lower KL/logit distortion than the common Direct/GGN/vector solution and also improves over uniform INT8 by approximately $77\%/78\%$. At $B=10$, tangent interaction gives the lowest KL ($6.32\times10^{-4}$), whereas Direct-KL/CE-GGN gives the lowest logit MSE ($4.49\times10^{-3}$). Thus, stronger global ranking does not necessarily imply a uniformly better constrained allocation. ### 3. How were the 125 configurations sampled? For each architecture, we retain the three uniform assignments separately. The 125 heterogeneous configurations are generated using a fixed pseudorandom seed by choosing one action independently and uniformly from $$\{\mathrm{FP32},\mathrm{INT8},\mathrm{INT4}\}$$ for every quantized tensor. Duplicate full-network assignments are rejected, and the uniform anchors are excluded from rank-correlation calculations. We also added **budget-local ranking** around 8, 9, 10, and 11 bits. On ResNet-18, functional KL correlations remain $0.978,0.979,0.964,0.988$ across these windows, with logit-MSE correlations $0.974,0.982,0.964,0.976$. MobileNetV3 remains harder, but tangent interaction improves the KL correlations from $$0.812/0.726/0.414/0.742$$ to $$0.854/0.815/0.679/0.819$$ for the 8/9/10/11-bit windows. ### Summary The new controls sharpen our claim. JVP is not the fastest profiler, and exact direct vectors can outperform it. Its contribution is a reusable first-order vector representation that makes output geometry and cross-layer composition explicit. We have revised the manuscript accordingly and added the requested direct-vector, same-budget, and budget-local analyses.

Thank you for this detailed response and for the additional experiments. They answer my questions, and I appreciate the clarifications about the role of the JVP. Nevertheless, I keep my rating. Indeed, the new results show that direct vectors remain a bit more predictive and faster, and that the differences in allocation given an equal budget remain small. The JVP’s contribution therefore seems to me to be analytical in nature, which is of course of interest, but remains my main reservation regarding the scope of the contribution.

AuthorsOpenReview ↗

Additional Details Regarding Transormers

Thank you for the clarification. We agree with the central characterization: the contribution of the JVP formulation is primarily analytical and diagnostic, rather than a claim of uniformly better prediction speed or allocation quality. We have revised the manuscript to state this explicitly.

The point we aim to establish is that a fixed quantization perturbation can be decomposed into four separately testable factors: output geometry, first-order linearization fidelity, preservation of response direction, and cross-layer composition. The direct-vector control is important precisely because it shows that vector preservation is not unique to JVPs, while the low-bit experiments identify where first-order linearization ceases to be reliable.

To address the remaining concern about scope, we additionally evaluated an ImageNet-pretrained DeiT-Tiny transformer under the same 125 heterogeneous FP32/INT8/INT4 configurations. Additive functional sensitivity remains strongly predictive, with

\rho=0.965\;(\mathrm{logit\ MSE}).$$ The transformer also produces a qualitatively different composition result: adding tangent-interference terms decreases the correlations to $0.921/0.937$, with both paired decreases statistically significant. This strengthens the conclusion that cross-layer first-order composition is architecture-dependent rather than a CNN-specific artifact. We also repeated exact-budget allocation at $B=8,9,10,11$ on all 3,925 validation images. Functional, direct-output, and CE-GGN objectives are closely competitive, with identical decisions at $B=8$ and only small target-dependent differences at higher budgets. We therefore do not revise the paper toward a claim of JVP practical dominance. Instead, the revised manuscript positions the contribution more narrowly as a controlled function-space analysis that makes the geometry, validity regime, and composition behavior of mixed-precision quantization explicit, now supported across both CNN and ViT-family architectures.
Reviewer_K3kPOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

In this paper the authors study the problem of measuring post-training quantization error. They explore the efficacy of using the per-layer (ll) first-order change in logits under quantization (aa) given input xx (Jl(x)⋅ΔWl,aJ_l(x)\cdot\Delta W_{l,a}) as a metric, arguing that this approach separates error sources that are conflated by typical metrics/analysis: error in weights, error in the represented function (as Euclidean distance in logits), and error in task loss. The approach requires calibration inputs, but doesn’t need labels (in contrast with second order methods), and the authors claim that the sum over per-layer tangent vectors can be used to approximate the jointly quantized network without quantizing layers pairwise.

In evaluation on a few CNN architectures (ResNet-18, MobileNetV3-Small, MobileNetV2 on the small Imagenette dataset) using int8/int4 per-channel post-training quantization of weights only w/ FP32 activations, they show that the proposed metric ranks function space error well, clearly better than parameter-space and first-order loss-space metrics, a little better than block-Hessian in ResNet and comparable to GGN, and not as well as directly measuring layer-wise error, and that Jl(x)⋅ΔWl,aJ_l(x)\cdot\Delta W_{l,a} is locally (per layer) accurate. With respect to composition, they find that the additive approximation of the jointly quantized network is sufficient on ResNet-18 and MobileNetV2, while including first-order cross-layer interference terms improves ranking on MobileNetV3-Small. Providing the proposed scores as input to a budget-constrained knapsack allocation out-performs Hessian-based scores in some cases (e.g. ResNet-18 at 11 bits) but not others, but all scoring methods seem to perform poorly for selecting allocations at the most interesting budget (8 bits), i.e. selecting mixed-precision allocations that perform worse than the uniform int8 baseline for ResNet-18.

Critical strengths
  • Post-training quantization is widely used to improve computational efficiency of deployed models, metrics/approaches for characterizing quantization error is useful for understanding and improving PTQ.
  • The paper provides a careful, well-controlled set of experiments for the selected models and quantization setting.
Critical weaknesses
  • Pretty narrow contribution with somewhat limited experimental results with respect to real-world settings. The problem setup is not very realistic, in terms of both amount of quantization (and therefore what we can actually learn from the characterization in the paper), and model scale. Weights-only int8 quantization w/ FP32 activations is basically lossless on ResNet-18, so what error is actually being measured here and is it representative of error that would be expected in more realistic settings? It would be good to see how the approach does in more challenging quantization settings, e.g. int3/2 weights, and/or a transformer architecture. The proposed metric seems to work the worst on the network with the most quantized layers (MobileNetV3), and there isn’t a compelling explanation given in the paper for why. The paper doesn’t include the strong Hessian baselines for MobileNetV3, the hardest setting. Ultimately, my main takeaway from the current experiments is that the approach is accurate for small perturbations in this limited setting, which does not seem very impactful.
Questions for authors
  • The proposed metric performs the most poorly on MobileNetV3-Small but the paper doesn’t include an explanation. Do the authors have a hypothesis on what’s going on here? Reporting relative quantization error per architecture might help to control for the effect of perturbation magnitude.
  • On ResNet-18 at 8 bits, all methods pick the same mixed allocation, which is ~3x worse than uniform int8. At this budget the allocator has to actually make trade offs, and I’d be interested to better understand why this happened: Which layers were allocated to int4, whether the direct isolated scores show the same result (Table 13 only goes to 9 bits for ResNet-18), and how the interference score compares to the additive score for this setting. What were the predicted scores for uniform int8 vs the chosen configuration?
  • Would summing cached per-layer logit deltas from the direct isolated baseline recover the interference gain on MobileNetV3-Small? I think this would help isolate the impact of JVP linearization from composition.
4 comments
AuthorsOpenReview ↗

We thank the reviewer for the careful assessment and for identifying several points that required stronger evidence. We have substantially revised the paper and added targeted experiments addressing the low-bit regime, the weaker MobileNetV3-Small behavior, the direct-vector composition question, and the ResNet-18 \(B=8\) allocation result.

1. More challenging INT3/INT2 quantization

We agree that FP32/INT8/INT4 alone did not sufficiently test larger perturbations. We therefore added a controlled ResNet-18 study: starting from all-INT8, we downgrade the same 45 matched layer subsets to INT4, INT3, or INT2 across mild/moderate/strong parameter-mass bands.

Local JVP fidelity decreases systematically: ρJVP,exact=0.990(INT4)  0.887(INT3)  0.687(INT2),\rho_{\mathrm{JVP,exact}}=0.990 (\mathrm{INT4}) \;0.887 (\mathrm{INT3}) \;0.687 (\mathrm{INT2}), with median cosine alignment 0.927→0.739→0.3330.927\rightarrow0.739\rightarrow0.333.

INT3 remains operational: configuration-level correlations are ρKL=0.829\rho_{\mathrm{KL}}=0.829 and ρlogit=0.936\rho_{\mathrm{logit}}=0.936, while 93.3%93.3\% of mild and 73.3%73.3\% of moderate INT3 models retain at least half of FP32 Top-1. INT2 frequently leaves the local regime and first-order accuracy degrades sharply. We therefore revised the claim: the method is highly accurate at INT4, remains useful for viable INT3 perturbations, but is not expected to remain accurate under extreme low-bit perturbations. Transformer validation is now stated explicitly as future work.

2. Why is MobileNetV3-Small harder?

The new evidence suggests that the gap is not simply larger quantization magnitude. At the isolated layer-action level, JVP remains extremely accurate on MobileNetV3-Small: ρ=0.993\rho=0.993 for isolated KL and 0.9990.999 for isolated logit MSE. Severity-controlled correlations also remain substantial.

Instead, MobileNetV3 shows stronger composition effects: median departure from additive tangent scoring is Rabs=0.140R_{\mathrm{abs}}=0.140, versus 0.0690.069 for ResNet-18 and 0.0960.096 for MobileNetV2. Tangent interference improves MobileNetV3 KL correlation from 0.7050.705 to 0.7760.776.

Exact isolated output-vector composition improves further to ρ=0.847\rho=0.847 (KL) and 0.8460.846 (logit MSE), showing that nonlinear within-layer corrections also matter. We therefore revised the interpretation: MobileNetV3 is harder because of both stronger non-additive composition and nonlinear effects beyond JlΔwlJ_l\Delta w_l.

3. Hessian baseline on MobileNetV3-Small

We attempted the same reverse-over-reverse block-Hessian used on ResNet-18, but the required second derivative through Hardsigmoid is unavailable in our PyTorch environment. A central finite-difference alternative was unstable across step sizes (Spearman 0.684/0.5650.684/0.565 for INT8 and 0.846/0.6280.846/0.628 for INT4), so we do not use it as evidence.

We report this limitation explicitly and use CE-GGN as the architecture-compatible curvature comparator. On MobileNetV3-Small, all functional-vs-GGN bootstrap intervals cross zero, so we do not claim superiority there.

4. ResNet-18 at B=8B=8

We thank the reviewer for flagging this. Rechecking exposed a numerical artifact in the original allocator: discretizing normalized bit contributions at 10−310^{-3} resolution could make the exactly feasible uniform-INT8 solution appear infeasible at the tight B=8B=8 boundary.

We replaced this with the exact constraint ∑lnlb(al)≤B∑lnl.\sum_l n_l b(a_l)\le B\sum_l n_l.

The corrected result is that Functional, Hessian, Direct-logit, Direct-KL, and CE-GGN all select uniform INT8 at B=8B=8, with

\mathrm{logit\ MSE}=9.466\times10^{-3}.$$ Thus, the previously reported $\sim3\times$ degradation was an allocator-discretization artifact, not a failure of the functional score. At $B=11$, functional still improves over Hessian while using fewer average bits (KL $-22.3\%$, logit MSE $-23.5\%$). ### 5. Summing cached direct isolated logit vectors Yes. We added exactly this control: $$d_{l,a}(x)=f_{\theta+\Delta\theta_{l,a}}(x)-f_\theta(x),$$ and score configurations using the norm of $\sum_l d_{l,a_l}(x)$. On MobileNetV3-Small, direct-vector composition gives $\rho=0.847$ for KL and $0.846$ for logit MSE, versus $0.796/0.799$ for scalar direct-logit summation and $0.776/0.802$ for tangent interaction. The gains over scalar direct-logit are significant: $+0.0508$ for KL and $+0.0473$ for logit MSE. This changes our interpretation: vector composition is **not unique to JVPs**. Exact isolated vectors also contain within-layer nonlinear corrections, while the jointly quantized model contains additional mixed nonlinear terms. ### Summary The revised paper now makes a narrower but better-supported claim: JVP sensitivity is a reusable first-order function-space diagnostic with a measurable validity regime, not a universally superior or cheaper proxy. The new results establish useful INT3 behavior, identify breakdown at INT2, explain the MobileNetV3 gap more precisely, add the requested exact-vector control, and correct the $B=8$ allocation artifact.

Thanks for the thorough response and additional experiments. I appreciate your candor, and the revised claims are appropriately calibrated.

As far as I can tell, directly quantizing each layer in isolation and caching the output delta matches or beats the JVP approach pretty much across the board. The ability to re-use cached responses also applies to direct vectors. The scope of the contribution is even narrower now, focused on the characterization: importance (or, lack thereof) of output geometry, first-order fidelity holds for INT3 but breaks for INT2, and composition effects are architecture-dependent, with exact vectors performing better than linear. These are potentially interesting findings, but on three small CNNs and Imagenette I don't think they meet the bar in terms of informativeness for this venue. So I'll keep my score.

AuthorsOpenReview ↗

Follow up response

Thank you for the follow-up. We appreciate that the remaining concern is now primarily about informativeness/scope rather than correctness, and we agree that the architectural scope remains limited.

We would only like to clarify two points in the current interpretation.

First, the new direct-vector control does not dominate JVP-based scoring across the board. It is the strongest global predictor on MobileNetV3-Small, but this does not consistently translate into the best constrained decision. For example, at B=10B=10, functional-interaction achieves lower KL than direct-vector:

\quad\text{vs.}\quad 6.63\times10^{-4},$$ while Direct-KL/CE-GGN gives the lowest logit MSE. On ResNet-18, functional and direct-logit objectives select identical configurations across the reported $B=8\!-\!11$ budgets. This is one of the findings we consider important: **global predictor quality and constrained allocation quality are not equivalent.** Second, our conclusion regarding output geometry is not that it is unimportant. Rather, its effect is **target dependent**. On ResNet-18, Euclidean functional sensitivity is significantly stronger than CE-GGN for KL and logit MSE, whereas CE-GGN is stronger for Top-1 degradation; on MobileNetV3-Small the differences are not significant. Thus, the same quantization perturbation can be judged differently depending on whether the objective is function fidelity or task-loss geometry. We agree that the paper is now best viewed as a characterization study. However, we believe the characterization is more than the statement that first-order approximations work for small perturbations. The experiments separate four factors that are usually conflated: 1. output geometry; 2. first-order linearization fidelity; 3. scalar vs. vector preservation; and 4. cross-layer composition. The resulting conclusions are nontrivial: exact vectors can outperform linearized ones, but not necessarily produce better constrained allocations; first-order fidelity remains strong into viable INT3 but breaks sharply toward INT2; and tangent interactions are useful on MobileNetV3-Small but not on ResNet-18 or MobileNetV2. We fully acknowledge that validating these conclusions on transformers and larger-scale datasets would strengthen the generality. Our claim is therefore intentionally restricted to the tested setting. We hope the reviewer might consider whether the controlled separation of these effects, and particularly the negative/conditional findings that prevent overinterpreting any single sensitivity proxy, provides sufficient informativeness despite the limited architectural scope.
AuthorsOpenReview ↗

Additional Experiment Details on Transformer

Thank you again for the follow-up. Your remaining concern was primarily the informativeness and architectural scope of the characterization. Motivated directly by this concern, we added a cross-family experiment with an ImageNet-pretrained DeiT-Tiny transformer, evaluated on Imagenette under the same protocol: 32 calibration images, 512 disjoint predictor-evaluation images, and 125 heterogeneous FP32/INT8/INT4 configurations. We also performed exact-budget allocation and re-evaluated the selected configurations on all 3,925 validation images.

The transformer result is strong:

0.965\;(\mathrm{logit\ MSE}).$$ Importantly, the choice of output geometry is informative here. Additive CE-GGN obtains $0.920/0.886$, so functional-minus-GGN is $$+0.0256\quad (95\%~\mathrm{CI}\,[0.0015,0.0542])$$ for KL and $$+0.0798\quad (95\%~\mathrm{CI}\,[0.0490,0.1230])$$ for logit MSE. The transformer also strengthens the architecture-dependence conclusion. Unlike MobileNetV3-Small, tangent interference **reduces** DeiT-Tiny correlation from $0.946/0.965$ to $0.921/0.937$, with both paired confidence intervals excluding zero. We retain our previous conclusion regarding exact vectors: direct-vector composition is strongest for DeiT-Tiny KL ($\rho=0.981$), so we do not claim JVP superiority. However, it does not uniformly dominate task-level rankings or constrained allocations. At $B=8$, all functional, direct, and CE-GGN objectives select the same uniform-INT8 solution; at $B=9$--$11$, only small target-dependent differences emerge. Thus, the revised evidence is no longer restricted to CNN architectures: the same controlled decomposition of **output geometry, linearization, vector preservation, and composition** now yields informative and nontrivial behavior on a ViT-family transformer. We still explicitly retain Imagenette and single-transformer-scale as limitations. We hope this additional cross-family evidence addresses the principal scope concern underlying the remaining rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.