acceptodds
Under review as a conference paper at ICLR 2027

Repurposing Influence Diagnostics for Low-Bit 3D Object Detection Models

Abstract

3D object detection is essential for instance-level scene understanding, yet deploying computationally intensive detectors on resource-constrained devices often requires low-bit quantization. However, reducing bit width remains challenging because 3D detectors exhibit activation distributions in which high-magnitude outliers are not uniformly informative. While some outliers carry task-relevant information, others arise from incidental or nuisance patterns. Such uninformative outliers can unnecessarily broaden the activation range, leading to coarse discretization and feature distortion under low-bit quantization. To address this issue, we repurpose influence diagnostics for quantization, where influence reflects how strongly an individual activation affects task-relevance. Our approach builds on the statistical perspective that informative outliers are crucial for preserving the properties of the data distribution. This perspective aligns with the classical notion of influential observations, motivating us to identify informative outliers by assessing their impact on task-relevant information. To make this influence task-aware, we use activation gradients to capture the relevance of individual activations to the task objective. Because these gradients contain both dominant task-relevant patterns and incidental variations, we model their shared structure with a low-rank representation that retains the principal patterns to be preserved. We then assess activation influence by measuring how strongly each activation affects the resulting gradient structure. Experiments on the nuScenes benchmark demonstrate improvements across W8A8, W4A16, W4A8, and W4A4 settings, with larger gains at lower bit widths. These results suggest that activation influence provides an effective criterion for identifying informative outliers in low-bit quantization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Quantization, 3D Object Detection, Efficient AI, Statistical Signal Processing, Regression Diagnostics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_V5ftOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes an influence-based activation selection strategy for post-training quantization of 3D object detectors. It constructs a low-rank representation of activation gradients obtained from a heatmap-based objective, then combines leverage and normalized reconstruction residuals into a Cook’s-distance-inspired score. Selected activations guide SmoothQuant’s smoothing scales and AWQ’s scaling and calibration objective. Experiments on nuScenes with BEVDet and CenterPoint show benefits across several precision settings, with the largest improvements under W4A4.

Critical strengths
  1. The method integrates with both SmoothQuant and AWQ across camera and LiDAR detectors without additional inference computation. The reported calibration overhead is modest: Ours-Smooth adds 3.5 minutes over InlierQ on CenterPoint.
  2. The W4A4 improvements are notable. On CenterPoint, the method achieves 52.2 mAP, compared with 46.6 for InlierQ and 41.6 for SmoothQuant.
Critical weaknesses
  1. The added value of the influence formulation needs stronger justification. Gradient-based activation selection already exists in InlierQ. Compared with Fisher Diagonal, the proposed score improves mAP by 0.2–0.4 at W4A16 and 1.0–1.6 at W4A4. Residual-only scoring is also within 0.4 mAP in both W4A16 settings. Although Figure 4 demonstrates structural sensitivity, the connection between this sensitivity and quantization-induced task degradation remains insufficiently validated.
  2. The evaluation is limited, and the stability of the gains is unclear. Experiments cover only nuScenes and two detectors, without transformer-based architectures. Calibration uses 64 samples from one driving sequence, but results across multiple calibration sets are not reported. This matters particularly for the smaller margins over competing selection criteria.
  3. The calibration changes are not fully disentangled. Ours-AWQ changes the activation statistic from a mean to a maximum while also selecting activations and restricting the search objective. It is unclear whether Table 3’s union-of-sets baseline retains the maximum statistic. Controlled comparisons with simple percentile or MSE-based clipping would also help determine whether influence scoring offers benefits beyond range reduction.
Questions for authors
  1. Could you separate the effects of mean-versus-max statistics, activation selection, and restricting AWQ’s search objective, and compare against simple clipping under matched settings?
  2. Could you report mean and standard deviation over multiple calibration sets and include a transformer-based 3D detector?
  3. Does the proposed influence score predict actual task degradation under quantization perturbations better than Fisher or residual-only scoring?
5 comments
AuthorsOpenReview ↗

Response to Reviewer V5ft (1/3)

We appreciate the reviewer's acknowledgment of our method's practicality and its improvements in low-bit 3D detection. We are also grateful for the thoughtful feedback regarding the connection between structural sensitivity and quantization-induced task degradation, as well as a more comprehensive experimental evaluation. In the following, we would like to address each of these concerns one-by-one.

W1-1: The added value of the influence formulation needs stronger justification. Gradient-based activation selection already exists in InlierQ. Compared with Fisher Diagonal, the proposed score improves mAP by 0.2–0.4 at W4A16 and 1.0–1.6 at W4A4. Residual-only scoring is also within 0.4 mAP in both W4A16 settings.

Response. Our method aims to identify informative activation outliers, so its effectiveness is most pronounced under activation quantization, where filtering uninformative outliers narrows the quantization range. Under W4A16, activations remain in FP16, so small margins among gradient-based scores are expected. Under W4A4, residual-only scoring underperforms our full formulation by 1.0 mAP on BEVDet and 1.5 mAP on CenterPoint (Table 3b). Furthermore, when varying only the selection score across 10 random calibration subsets on CenterPoint under W4A4, our influence score outperforms Fisher scoring and InlierQ's score by 1.9 and 1.7 mAP, respectively, while reducing the standard deviation from 1.2–1.4 to 0.6.

Method (W4A4, 10 seeds)mAPNDS
Ours w/ Diagonal Fisher49.5 ± 1.460.3 ± 1.1
Ours w/ InlierQ score49.7 ± 1.260.4 ± 0.8
Ours w/ influence51.4 ± 0.661.4 ± 0.3

W1-2: Although Figure 4 demonstrates structural sensitivity, the connection between this sensitivity and quantization-induced task degradation remains insufficiently validated.

Q3: Does the proposed influence score predict actual task degradation under quantization perturbations better than Fisher or residual-only scoring?

Response. Our formulation measures influence on the model prediction, quantifying how much the gradients’ low-rank structure is perturbed when an activation vector is removed. Thus, corrupting a high-influence activation vector perturbs the model prediction more and degrades task performance. The diagonal Fisher considers gradient magnitude but does not sufficiently measure the low-rank structure. Residual-only scoring also lacks the quantification of the low-rank structure’s perturbation, leading to insufficient reflection of model prediction degradation.

To show this, we conduct a quantitative analysis under controlled perturbation with 10 random seeds in the table below. We apply 50% magnitude amplification to the CenterPoint encoder’s top 5% scored output activations and measure the task loss (Bounding Box Loss and Classification Loss) perturbation of selection methods InlierQ’s score, diagonal Fisher information matrix, and our proposed influence. “Low influence” indicates lowest 5% scored selection, and “High influence” indicates our influential activation selection. Our selection with high influence scores produces approximately a 1.9× larger increase in task loss than perturbing those selected by InlierQ and Fisher scores. In contrast, perturbing activations assigned low influence scores has a negligible effect on task loss. These results suggest that our influence score measures actual task degradation under perturbations.

PerturbationInlierQFisherLow InfluenceHigh Influence
Bounding Box Loss (x1e-2)0.3020.2950.000 ± 0.0020.620
Classification Loss0.2650.2630.000 ± 0.0000.479
AuthorsOpenReview ↗

Response to Reviewer V5ft (2/3)

W2: The evaluation is limited, and the stability of the gains is unclear. Experiments cover only nuScenes and two detectors, without transformer-based architectures. Calibration uses 64 samples from one driving sequence, but results across multiple calibration sets are not reported. This matters particularly for the smaller margins over competing selection criteria.

Q2: Could you report mean and standard deviation over multiple calibration sets and include a transformer-based 3D detector?

Response. We investigate the benefit of our work across 10 random calibration seeds. We randomly collect 64 calibration samples by selecting each driving scene sample-by-sample from the whole driving sequences in the training split. As shown in the table below, the improvement achieved by Ours-AWQ and Ours-Smooth still holds under W4A16 and W4A4 quantization, demonstrating calibration stability.

10 Random SeedsMeanStdMinMax
W4A16 AWQ54.9mAP/63.6NDS0.2mAP/0.1NDS54.7mAP/63.5NDS55.3mAP/63.8NDS
W4A16 Ours-AWQ55.5mAP/64.2NDS0.2mAP/0.3NDS55.1mAP/63.6NDS55.7mAP/64.3NDS
W4A4 SmoothQuant37.4mAP/52.2NDS0.9mAP/0.6NDS36.0mAP/51.5NDS38.7mAP/52.9NDS
W4A4 Ours-Smooth51.4mAP/61.4NDS0.6mAP/0.3NDS50.4mAP/61.1NDS52.2mAP/61.9NDS

Here, we additionally report the quantization results of the transformer-based 3D detector BEVFormer-Tiny. For this experiment, we use the same hyperparameters as those for the camera-based 3D detector BEVDet, as reported in our manuscript (Ours-AWQ: τ\tau=0.7, Ours-Smooth: τ\tau=0.9 with SVD rank 16). We use W4A16 and W4A8 bit settings, as W4A4 quantization results in severe performance degradation (0mAP / 0NDS) across all approaches, consistent with the intolerable low-bit degradation of BEVFormer reported in QD-BEV (Zhang et al., 2023). Under the W4A16 bit setting, Ours-AWQ demonstrates average gains of +0.5 mAP and +0.1 NDS across 10 random seeds. Under the W4A8 bit setting, Ours-Smooth shows average gains of +3.7 mAP and +6.4 NDS across 10 random seeds.

PTQ MethodFP32W4A16W4A8
SmoothQuant25.2mAP/35.5NDS-3.0±0.1mAP / 10.9±0.3NDS
Ours-Smooth25.2mAP/35.5NDS-6.7±0.5mAP / 17.3±0.5NDS
AWQ25.2mAP/35.5NDS20.1±0.2mAP / 32.7±0.2NDS-
Ours-AWQ25.2mAP/35.5NDS20.6±0.4mAP / 32.8±0.4NDS-
AuthorsOpenReview ↗

Response to Reviewer V5ft (3/3)

W3: The calibration changes are not fully disentangled. Ours-AWQ changes the activation statistic from a mean to a maximum while also selecting activations and restricting the search objective. It is unclear whether Table 3’s union-of-sets baseline retains the maximum statistic. Controlled comparisons with simple percentile or MSE-based clipping would also help determine whether influence scoring offers benefits beyond range reduction.

Q1: Could you separate the effects of mean-versus-max statistics, activation selection, and restricting AWQ’s search objective, and compare against simple clipping under matched settings?

Response. We thank the reviewer for the careful comment. The union-of-sets baseline in Table 3 retains the maximum statistics, ensuring a matched comparison. To isolate individual contributions, we conduct an additional ablation study of Ours-AWQ, presented below. We report mean mAP and NDS over 10 independently sampled calibration subsets from the training split. Given the marginal differences under W4A16, we adopt the more challenging W3A16 setting to better distinguish component effects. For each configuration, we replace only the specified component, keeping all other settings unchanged.

As shown in the table, our influential activation selection and MSE clipping on the selected set improve mAP by +0.9 and +1.0 over random selection and simple MSE clipping, respectively. In contrast, using mean instead of maximum statistics or AWQ's original search objective yields negligible performance differences. These findings demonstrate that the primary gains stem from identifying and preserving influential activations using our proposed influence score, rather than the choice of activation statistics or search objective.

Ours-AWQW3A16 mAPW3A16 NDS
Max statistics (Ours)51.2±0.5mAP61.2±0.4NDS
Mean statistics51.2±0.3mAP61.3±0.4NDS
Random activation selection50.3±0.3mAP60.5±0.1NDS
AWQ’s search objective51.1±0.5mAP61.4±0.3NDS
Simple MSE clipping50.2±0.4mAP60.4±0.4NDS
Reviewer_P7qMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces an influence-based calibration method for post-training quantization of 3D object detectors. Its central motivation is that large-magnitude activation outliers are not uniformly informative: some preserve task-relevant information, whereas others unnecessarily expand the quantization range. To distinguish them, the method applies regression influence diagnostics to layer-wise task-loss gradients. It constructs a low-rank gradient representation and combines leverage and residual scores to identify activations that strongly influence the dominant task-relevant structure. The selected activations are then used to calibrate existing quantization methods, producing influence-guided variants of SmoothQuant and AWQ without additional inference-time overhead. The approach is evaluated on nuScenes using the camera-based BEVDet and LiDAR-based CenterPoint detectors under W8A8, W4A16, W4A8, and W4A4 settings. Comparisons with SmoothQuant, AWQ, LiDAR-PTQ, and InlierQ show the largest improvements under aggressive 4-bit quantization. Additional experiments compare alternative activation-selection criteria, ablate the leverage and residual components, analyze sensitivity to the influence threshold and SVD rank, and report the one-time calibration cost.

Critical strengths
  1. The paper presents a well-motivated and technically coherent criterion for identifying informative activation outliers. Rather than treating all large-magnitude activations uniformly, the method adapts leverage and residual diagnostics to a task-aware low-rank gradient representation. This is consequential because it provides a structurally informed calibration signal beyond activation magnitude or local gradient importance, while remaining compatible with both SmoothQuant and AWQ and introducing no additional inference-time overhead.

  2. The method achieves meaningful improvements under challenging low-bit settings and is supported by targeted ablations. The gains are particularly substantial for CenterPoint under W4A4, where the proposed method achieves 52.2 mAP, compared with 46.6 for InlierQ and 41.6 for SmoothQuant. Comparisons with random, feature-norm, and Fisher-based selection, together with analyses of the individual score components, influence threshold, SVD rank, and calibration cost, provide evidence that the improvements arise from the proposed influence-based selection rather than solely from the underlying quantization framework.

Critical weaknesses

The experimental scope is too narrow to support the paper's generality claims. All results are reported on nuScenes using one camera-based detector and one LiDAR-based detector, with 64 calibration samples drawn from a single driving sequence. The paper also uses detector-specific SVD ranks and method-specific influence thresholds, but does not report variation across calibration sequences, random subsets, or seeds. This is a significant concern because several improvements are small, and the current results do not establish whether they persist across datasets, architectures, or calibration samples. The proposed influence score is only partially validated as a faithful measure of activation influence. The leverage-residual formulation is motivated by Cook’s distance, and the paper shows that removing high-scoring activation subsets perturbs the low-rank gradient structure more than removing median- or low-scoring subsets. However, it does not compare the proposed surrogate with an actual or approximate leave-one-out influence estimate or quantify how well the score ranks the structural effect of individual activations. The downstream quantization results therefore establish that the selection criterion is useful, but not whether its advantage specifically comes from a more faithful notion of influence rather than another form of task-aware gradient saliency. This distinction is important because gradient-aware activation selection is already used by InlierQ, while the influence-based formulation is the paper’s primary methodological distinction.

Questions for authors
  1. Does the proposed score truly measure activation influence? Can the authors compare it with an explicit leave-one-out influence estimate?

  2. What benefit comes specifically from the leverage-residual formulation over InlierQ? Under a matched calibration setup, how much improvement remains when only the selection rule is changed?

  3. Are the reported gains dependent on the specific calibration samples selected by the authors? Since the experiments use 64 calibration samples from a single driving sequence, do similar improvements hold across different calibration subsets or sequences?

  4. How do individual components, including task gradients, low-rank SVD, and the leverage-residual formulation, contribute separately and jointly? Do simpler combinations achieve comparable performance, or is the full formulation necessary?

5 comments
AuthorsOpenReview ↗

Response to Reviewer P7qM (1/3)

We greatly appreciate the reviewer's positive assessment of our work's motivation, technical coherence, and achievements in low-bit 3D detection. We are also grateful for the constructive feedback regarding the limited scope of our experiments and the need for further validation of our claims. To address these concerns and reinforce our contributions, we offer additional clarifications below.

W1: The experimental scope is too narrow to support the paper's generality claims. All results are reported on nuScenes using one camera-based detector and one LiDAR-based detector, with 64 calibration samples drawn from a single driving sequence.

Response. To broaden our experimental scope, we extend our approach to the transformer-based 3D detector BEVFormer-Tiny. For this investigation, we use the same hyperparameters as those for the camera-based 3D detector BEVDet, as reported in our manuscript (Ours-AWQ: τ\tau=0.7, Ours-Smooth: τ\tau=0.9 with SVD rank 16). We use W4A16 and W4A8 bit settings, as W4A4 quantization results in severe performance degradation (0mAP / 0NDS) across all approaches, consistent with the intolerable low-bit degradation of BEVFormer reported in QD-BEV (Zhang et al., 2023). Under the W4A16 bit setting, Ours-AWQ demonstrates average gains of +0.5 mAP and +0.1 NDS across 10 random seeds. Under the W4A8 bit setting, Ours-Smooth shows average gains of +3.7 mAP and +6.4 NDS across 10 random seeds.

PTQ MethodFP32W4A16W4A8
SmoothQuant25.2mAP/35.5NDS-3.0±0.1mAP / 10.9±0.3NDS
Ours-Smooth25.2mAP/35.5NDS-6.7±0.5mAP / 17.3±0.5NDS
AWQ25.2mAP/35.5NDS20.1±0.2mAP / 32.7±0.2NDS-
Ours-AWQ25.2mAP/35.5NDS20.6±0.4mAP / 32.8±0.4NDS-

W2: The paper also uses detector-specific SVD ranks and method-specific influence thresholds, but does not report variation across calibration sequences, random subsets, or seeds. This is a significant concern because several improvements are small, and the current results do not establish whether they persist across datasets, architectures, or calibration samples.

Q3: Are the reported gains dependent on the specific calibration samples selected by the authors? Since the experiments use 64 calibration samples from a single driving sequence, do similar improvements hold across different calibration subsets or sequences?

Response. We use a randomly selected driving scene and fix it across all the experiments for a matched calibration setup. Further, we investigate the benefit of our influence score across 10 randomly seeded calibration sets using CenterPoint model. We randomly select each calibration sample-by-sample from the whole driving sequences in the training split. As shown in the table below, the gains from our influence-based quantization persist across different calibration subsets, demonstrating +0.6 mAP and +0.6 NDS for W4A16 Ours-AWQ, and +14.0 mAP and +9.2NDS for W4A4 Ours-Smooth.

10 Random SeedsMeanStdMinMax
W4A16 AWQ54.9mAP/63.6NDS0.2mAP/0.1NDS54.7mAP/63.5NDS55.3mAP/63.8NDS
W4A16 Ours-AWQ55.5mAP/64.2NDS0.2mAP/0.3NDS55.1mAP/63.6NDS55.7mAP/64.3NDS
W4A4 SmoothQuant37.4mAP/52.2NDS0.9mAP/0.6NDS36.0mAP/51.5NDS38.7mAP/52.9NDS
W4A4 Ours-Smooth51.4mAP/61.4NDS0.6mAP/0.3NDS50.4mAP/61.1NDS52.2mAP/61.9NDS
AuthorsOpenReview ↗

Response to Reviewer P7qM (2/3)

W3: The proposed influence score is only partially validated as a faithful measure of activation influence. The leverage-residual formulation is motivated by Cook’s distance, and the paper shows that removing high-scoring activation subsets perturbs the low-rank gradient structure more than removing median- or low-scoring subsets. However, it does not compare the proposed surrogate with an actual or approximate leave-one-out influence estimate or quantify how well the score ranks the structural effect of individual activations.

Q1-2: Can the authors compare it with an explicit leave-one-out influence estimate?

Response. Our influence score uses one-shot regression diagnostics to efficiently approximate leave-one-out (LOO) influence. Across 10 random seeds, we compare detection performance (mAP and NDS) using our score, the InlierQ score, and low-influence selection. For one seed, we additionally measure index-set mIoU between the top-5% set selected by LOO and the sets selected by each method from CenterPoint’s encoder output activations. We compute LOO influence as how much the leading gradient directions change after removing an activation, using the normalized distance between their subspace projection matrices. We also report per-sample scoring and selection latency, excluding gradient extraction and data transfer. Due to its substantial computational cost, “Ours w/ Leave-one-out score” is evaluated using one calibration seed; we will include 10-seed results in the revised manuscript.

As shown in this table, our high-influence set exhibits substantial overlap with the actual leave-one-out selection, with about 37% relative mIoU gain over InlierQ’s selection. Our quantization with low influential set results in no index overlap, and shows detection task degradation. These demonstrate that our score measures influence, and we apply this intuition to post-training quantization.

Score MeasurementSelection mIoUPer-sample LatencyW4A4 mAPW4A4 NDS
Ours w/ Leave-one-out score100.0%294828 ms51.761.5
Ours w/ InlierQ score64.4%0.2 ms49.7 ± 1.260.4 ± 0.8
Low Influence0.0%6 ms41.0 ± 1.454.6 ± 0.7
High Influence (Ours)88.3%6 ms51.4 ± 0.661.4 ± 0.3

W4: The downstream quantization results therefore establish that the selection criterion is useful, but not whether its advantage specifically comes from a more faithful notion of influence rather than another form of task-aware gradient saliency. This distinction is important because gradient-aware activation selection is already used by InlierQ, while the influence-based formulation is the paper’s primary methodological distinction.

Q1-1: Does the proposed score truly measure activation influence?

Response. Our score measures activation influence on the model prediction. It thus enables us to preserve the low-rank structure of prediction-derived gradients and thereby maintain the predictive behavior of the full-precision model. As detailed in our response to Q1-2, comparison with actual leave-one-out influence validates our score, showing comparable activation selection and low-bit 3D detection performance. Moreover, as shown in our responses to Q2 and Q4, gradient-based selection without the leverage-residual formulation plateaus at 49.4–49.7 mAP under W4A4, whereas the full formulation reaches 51.4 mAP, indicating that the gain comes from the influence formulation rather than gradient saliency alone. To further verify that our score captures sensitivity to the detection task loss, we conduct controlled perturbation experiments across 10 random seeds, amplifying by 50% the magnitudes of the top 5% encoder output activations selected by our score, the diagonal Fisher information matrix, and InlierQ’s volume saliency score.

As shown in this table, our high-influence activation set exhibits larger loss perturbation than those of the other methods, while the low-influence activation set produces negligible changes in task loss. This indicates that our proposed score measures influence, and our score is better at capturing influence than the other scores.

PerturbationInlierQFisherLow InfluenceHigh Influence
Bounding Box Loss (x1e-2)0.3020.2950.000 ± 0.0020.620
Classification Loss0.2650.2630.000 ± 0.0000.479
AuthorsOpenReview ↗

Response to Reviewer P7qM (3/3)

Q2: What benefit comes specifically from the leverage-residual formulation over InlierQ? Under a matched calibration setup, how much improvement remains when only the selection rule is changed?

Response. Our formulation measures the perturbation of gradients’ low-rank structure when an activation vector is removed, which better reflects activation influence on the model prediction than InlierQ’s score, which correlates with gradient magnitude. This intuition enables us to precisely define informative outliers, whose preservation matters during calibration. To isolate the effect of the selection rule under a matched calibration setup, we replace only our influence score with InlierQ’s saliency score on CenterPoint, across 10 random calibration seeds. As shown in the table, our approach improves over InlierQ’s score under low-bit activation quantization (W4A4: +1.7 mAP with half the standard deviation), while the two scores perform comparably under weight-only quantization (W3A16, W4A16), where activations remain in FP16.

Random 10-SeedW3A16W4A16W4A4
Ours w/ InlierQ score50.9±0.4mAP / 61.1±0.3NDS55.6±0.1mAP / 64.1±0.1NDS49.7±1.2mAP / 60.4±0.8NDS
Ours51.2±0.5mAP / 61.2±0.4NDS55.5±0.2mAP / 64.2±0.3NDS51.4±0.6mAP / 61.4±0.3NDS

Q4: How do individual components, including task gradients, low-rank SVD, and the leverage-residual formulation, contribute separately and jointly? Do simpler combinations achieve comparable performance, or is the full formulation necessary?

Response. Our proposed formulation quantifies activation influence on the low-rank structure, making both low-rank SVD and the leverage-residual formulation essential. Furthermore, gradients are necessary because perturbations to their low-rank structure provide insights into changes in the detection task loss, reflecting model performance degradation. To substantiate this claim, we conduct a factorial ablation study on CenterPoint under W4A4 quantization across 10 random calibration seeds. The first row (w/o all components) uses feature magnitude for activation selection, while the second row uses low-rank-projected feature magnitude. The third row incorporates our influence formulation in the feature domain. From the fourth row onward, we progressively evaluate the corresponding components in the gradient domain. As shown in the table, task gradients provide the largest gain (+4.8 mAP), and the leverage-residual formulation adds a further +1.8 mAP with lower variance. The low-rank projection alone yields little change (+0.2), as it is a prerequisite for defining leverage and residual rather than an independent source of gain.

GradientLow-RankInfluenceW4A4 mAPW4A4 NDS
---44.6 ± 2.356.8 ± 1.7
-O-44.8 ± 2.556.8 ± 1.8
-OO44.5 ± 2.056.6 ± 1.5
O--49.4 ± 1.660.2 ± 1.0
OO-49.6 ± 1.060.2 ± 0.6
OOO51.4 ± 0.661.4 ± 0.3
Reviewer_hEyTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the challenge of preserving task-relevant information during low-bit quantization of 3D object detection models. The authors argue that activation outliers differ in their importance to the detection task, and treating them uniformly can introduce unnecessary quantization errors. Inspired by classical regression influence diagnostics, they propose a method that identifies influential activations using a low-rank approximation of task-related activation gradients. Specifically, the method combines leverage and residual scores to measure the influence of individual activations on the dominant gradient structure. The selected activations are then used to guide calibration in existing quantization methods, including SmoothQuant and AWQ. Experiments on the nuScenes benchmark with camera-based and LiDAR-based detectors show improvements across several quantization settings, with the largest gains under aggressive 4-bit quantization.

Critical strengths
  1. The paper addresses a clearly defined and practically important problem. Distinguishing informative from uninformative activation outliers is a reasonable approach to improving low-bit quantization.

  2. The use of statistical influence diagnostics provides an interesting perspective on activation selection. Combining leverage and residual information from task-related gradients is a potentially useful criterion for quantization calibration.

  3. The proposed method integrates with existing quantization frameworks without changing the model architecture or introducing additional inference costs.

Critical weaknesses
  1. The statistical justification for the proposed influence score is incomplete. Although the formulation resembles Cook's distance, it is constructed from leverage and reconstruction residuals of a truncated SVD of the gradient matrix. Unlike classical regression diagnostics, the paper does not establish how this score relates to changes in a fitted low-rank estimator when an observation is removed. The mathematical similarity to Cook's distance alone does not establish a comparable influence interpretation. So the general impression is there is still a gap between the statistical heuristic and the utility of the proposed approach.

  2. The connection between gradient subspace influence and quantization importance remains unclear. An activation that strongly influences the low-rank gradient structure may not necessarily produce a large detection loss when quantized. The actual effect of quantization also depends on the numerical perturbation introduced. The paper does not directly establish that the proposed score captures this sensitivity. The structural analysis in Figure 4 supports the influence of selected activations on the gradient representation, but does not establish their importance for the final detection objective.

  3. The empirical validation is limited. The experiments use only the nuScenes dataset and two detector architectures. More importantly, calibration relies on 64 samples from a single driving sequence, without evaluating variability across calibration samples or reporting uncertainty across independent runs. This makes it difficult to assess the robustness of the reported improvements, particularly when the differences between methods are small.

Questions for authors
  1. What is the relationship between the proposed influence score and actual quantization-induced detection loss? It would be helpful to examine whether activations with higher influence scores produce larger changes in detection loss when individually perturbed or quantized.

  2. How stable are the results across different calibration samples and driving sequences? Given the small calibration set, reporting variability across multiple independently sampled calibration sets would strengthen the experimental evidence.

  3. How are the SVD rank and influence threshold selected in practice, and how sensitive are the results to these choices? It would also be helpful to clarify whether the task-related gradient computation requires ground-truth labels during calibration.

4 comments
AuthorsOpenReview ↗

Response to Reviewer hEyT (1/2)

We sincerely thank the reviewer for recognizing the motivation behind our work, interesting perspective, and scalability and applicability without incurring additional inference costs. We also appreciate the reviewer's insightful comments regarding the statistical justification of our proposed influence, the connection between the gradient subspace and quantization importance, and our experimental evaluation. In response to these comments and questions, we provide further clarification to strengthen our claims.

W1: The statistical justification for the proposed influence score is incomplete. Although the formulation resembles Cook's distance, it is constructed from leverage and reconstruction residuals of a truncated SVD of the gradient matrix. Unlike classical regression diagnostics, the paper does not establish how this score relates to changes in a fitted low-rank estimator when an observation is removed. The mathematical similarity to Cook's distance alone does not establish a comparable influence interpretation. So the general impression is there is still a gap between the statistical heuristic and the utility of the proposed approach.

Response. Our score estimates how removing an observation affects a fitted low-rank structure. Figure 4(a–b) shows that removing the top 5% of activations ranked by our score substantially perturbs the low-rank structure of the calibration representations. To further assess its fidelity, we compare our score with explicit leave-one-out (LOO) influence, defined as how much the leading gradient directions change after removing an activation, using the normalized distance between their subspace projection matrices. Across 10 random seeds, we compare detection performance (mAP and NDS) using our score, the InlierQ score, and low-influence selection. For one seed, we also measure index-set mIoU between the top-5% set selected by LOO and the sets selected by each method below. Due to its substantial computational cost, “Ours w/ Leave-one-out score” is evaluated using one calibration seed; we will include 10-seed results in the revised manuscript.

As shown in the table, our high-influence set exhibits substantial overlap with the actual leave-one-out selection, achieving about 37% relative mIoU gain over InlierQ’s selection with negligible calibration overhead, while matching the detection performance achieved by leave-one-out-based selection. This suggests our influence score closely aligns with actual leave-one-out influence while incurring only marginal computational overhead.

Score MeasurementSelection mIoUPer-sample LatencyW4A4 mAPW4A4 NDS
Ours w/ Leave-one-out score100.0%294828 ms51.761.5
Ours w/ InlierQ score64.4%0.2 ms49.7 ± 1.260.4 ± 0.8
Low Influence0.0%6 ms41.0 ± 1.454.6 ± 0.7
High Influence (Ours)88.3%6 ms51.4 ± 0.661.4 ± 0.3
AuthorsOpenReview ↗

Response to Reviewer hEyT (2/2)

W2: The connection between gradient subspace influence and quantization importance remains unclear. An activation that strongly influences the low-rank gradient structure may not necessarily produce a large detection loss when quantized. The actual effect of quantization also depends on the numerical perturbation introduced. The paper does not directly establish that the proposed score captures this sensitivity. The structural analysis in Figure 4 supports the influence of selected activations on the gradient representation, but does not establish their importance for the final detection objective.

Q1: What is the relationship between the proposed influence score and actual quantization-induced detection loss? It would be helpful to examine whether activations with higher influence scores produce larger changes in detection loss when individually perturbed or quantized.

Response. Our influence score estimates how removing an activation perturbs the low-rank structure in layer-wise gradients derived from model predictions, serving as a surrogate for prediction sensitivity. Thus, large quantization errors in high-influence outlier activations can severely disrupt model predictions. Using this intuition, we search for informative outliers, whose preservation matters during calibration. Thus, at the calibration stage of our example regimes (i.e., Ours-AWQ and Ours-Smooth), we select influential activation set and consider the outliers within the activation set as informative outliers.

Here, we supplement this with a quantitative analysis under controlled perturbation averaging 10 random calibration set seeds, applying 50% magnitude amplification to the encoder’s top 5% scored output activations. As shown in this table, perturbing low-influence activation vectors demonstrates negligible changes in detection loss, while perturbing high-influence activation vectors shows the largest perturbation compared to the other sensitivity measurements, including the diagonal Fisher information matrix and InlierQ’s volume saliency score.

PerturbationInlierQFisherLow InfluenceHigh Influence
Bounding Box Loss (x1e-2)0.3020.2950.000 ± 0.0020.620
Classification Loss0.2650.2630.000 ± 0.0000.479

W3: The empirical validation is limited. The experiments use only the nuScenes dataset and two detector architectures. More importantly, calibration relies on 64 samples from a single driving sequence, without evaluating variability across calibration samples or reporting uncertainty across independent runs. This makes it difficult to assess the robustness of the reported improvements, particularly when the differences between methods are small.

Q2: How stable are the results across different calibration samples and driving sequences? Given the small calibration set, reporting variability across multiple independently sampled calibration sets would strengthen the experimental evidence.

Response. Thanks for the suggestion. Our proposed quantization is stable across different calibration samples. Here, we report the variability investigation for 10 randomly selected seeds of calibration subsets. We randomly select each driving scene sample-by-sample across the whole driving sequences in the training split. Improvements from our influence score still hold across different calibration samples.

10 Random SeedsMeanStdMinMax
W4A16 AWQ54.9mAP/63.6NDS0.2mAP/0.1NDS54.7mAP/63.5NDS55.3mAP/63.8NDS
W4A16 Ours-AWQ55.5mAP/64.2NDS0.2mAP/0.3NDS55.1mAP/63.6NDS55.7mAP/64.3NDS
W4A4 SmoothQuant37.4mAP/52.2NDS0.9mAP/0.6NDS36.0mAP/51.5NDS38.7mAP/52.9NDS
W4A4 Ours-Smooth51.4mAP/61.4NDS0.6mAP/0.3NDS50.4mAP/61.1NDS52.2mAP/61.9NDS

Q3: How are the SVD rank and influence threshold selected in practice, and how sensitive are the results to these choices? It would also be helpful to clarify whether the task-related gradient computation requires ground-truth labels during calibration.

Response. Using a fixed calibration subset, we empirically search for SVD ranks and influence thresholds by referring to the quantization error in the model prediction. Throughout the experiments, we used fixed influence thresholds of τ = 0.9 for W4A4 and τ = 0.7 for W4A16, selecting the top 10% and 30% influential activations, respectively. Our gradient computation follows the label-free approach of InlierQ and does not require any ground-truth labels during calibration.

Reviewer_hEyTOpenReview ↗

Thank you

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the leave-one-out comparison, perturbation analysis, and evaluation across multiple calibration subsets.

I still have some reservations about its statistical interpretation, as the relationship to classical influence diagnostics remains largely empirical. The perturbation experiments are encouraging, although they do not directly establish sensitivity to actual quantization errors. The generalizability of the method also remains somewhat unclear.

Overall, the additional evidence has strengthened my assessment of the paper. I would like to see how these results and their implications are incorporated into the revised manuscript before deciding whether to adjust my rating.

We thank the reviewer for the invaluable feedback on our work. In particular, the suggestion to examine whether distorting high-influence activations causes larger task-loss changes helped us strengthen the link between our score and the detection objective.

We have revised the manuscript as follows:

  • Sensitivity to actual quantization errors (W2, Q1): We added experiments injecting actual W4A16/W4A4 quantization errors into selected activations in Section 5.3 (Table 5), with the amplification experiment in Appendix E (Table 9).
  • Robustness and hyperparameters (W3, Q2, Q3): We added results over 10 calibration subsets in Section 5.1 (Table 2) and Appendix D, BEVFormer-Tiny results in Appendix B, and the hyperparameter selection protocol in Appendix A.

Regarding the statistical interpretation (W1), we appreciate the reviewer’s follow-up. Since each row of GG is Gi=∂L/∂xiG_i=\partial L/\partial x_i, an activation perturbation δi\delta_i, including quantization error, changes the detection objective by ΔL≈Gi⊤δi\Delta L \approx G_i^\top \delta_i. The dominant low-rank subspace of GG therefore captures the shared directions along which activation perturbations affect the objective. Inspired by Cook’s distance, our score combines the leverage and residual of each gradient observation as a one-shot proxy for how strongly the associated activation supports this fitted structure, and Section 5.3 (Table 4) shows that it closely tracks explicit leave-one-out influence. High-influence activations are thus expected to be more consequential under quantization, as supported by the task-loss changes in Table 5.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.