- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes an influence-based activation selection strategy for post-training quantization of 3D object detectors. It constructs a low-rank representation of activation gradients obtained from a heatmap-based objective, then combines leverage and normalized reconstruction residuals into a Cook’s-distance-inspired score. Selected activations guide SmoothQuant’s smoothing scales and AWQ’s scaling and calibration objective. Experiments on nuScenes with BEVDet and CenterPoint show benefits across several precision settings, with the largest improvements under W4A4.
Critical strengths
- The method integrates with both SmoothQuant and AWQ across camera and LiDAR detectors without additional inference computation. The reported calibration overhead is modest: Ours-Smooth adds 3.5 minutes over InlierQ on CenterPoint.
- The W4A4 improvements are notable. On CenterPoint, the method achieves 52.2 mAP, compared with 46.6 for InlierQ and 41.6 for SmoothQuant.
Critical weaknesses
- The added value of the influence formulation needs stronger justification. Gradient-based activation selection already exists in InlierQ. Compared with Fisher Diagonal, the proposed score improves mAP by 0.2–0.4 at W4A16 and 1.0–1.6 at W4A4. Residual-only scoring is also within 0.4 mAP in both W4A16 settings. Although Figure 4 demonstrates structural sensitivity, the connection between this sensitivity and quantization-induced task degradation remains insufficiently validated.
- The evaluation is limited, and the stability of the gains is unclear. Experiments cover only nuScenes and two detectors, without transformer-based architectures. Calibration uses 64 samples from one driving sequence, but results across multiple calibration sets are not reported. This matters particularly for the smaller margins over competing selection criteria.
- The calibration changes are not fully disentangled. Ours-AWQ changes the activation statistic from a mean to a maximum while also selecting activations and restricting the search objective. It is unclear whether Table 3’s union-of-sets baseline retains the maximum statistic. Controlled comparisons with simple percentile or MSE-based clipping would also help determine whether influence scoring offers benefits beyond range reduction.
Questions for authors
- Could you separate the effects of mean-versus-max statistics, activation selection, and restricting AWQ’s search objective, and compare against simple clipping under matched settings?
- Could you report mean and standard deviation over multiple calibration sets and include a transformer-based 3D detector?
- Does the proposed influence score predict actual task degradation under quantization perturbations better than Fisher or residual-only scoring?
5 comments
Response to Reviewer V5ft (2/3)
W2: The evaluation is limited, and the stability of the gains is unclear. Experiments cover only nuScenes and two detectors, without transformer-based architectures. Calibration uses 64 samples from one driving sequence, but results across multiple calibration sets are not reported. This matters particularly for the smaller margins over competing selection criteria.
Q2: Could you report mean and standard deviation over multiple calibration sets and include a transformer-based 3D detector?
Response. We investigate the benefit of our work across 10 random calibration seeds. We randomly collect 64 calibration samples by selecting each driving scene sample-by-sample from the whole driving sequences in the training split. As shown in the table below, the improvement achieved by Ours-AWQ and Ours-Smooth still holds under W4A16 and W4A4 quantization, demonstrating calibration stability.
| 10 Random Seeds | Mean | Std | Min | Max |
|---|---|---|---|---|
| W4A16 AWQ | 54.9mAP/63.6NDS | 0.2mAP/0.1NDS | 54.7mAP/63.5NDS | 55.3mAP/63.8NDS |
| W4A16 Ours-AWQ | 55.5mAP/64.2NDS | 0.2mAP/0.3NDS | 55.1mAP/63.6NDS | 55.7mAP/64.3NDS |
| W4A4 SmoothQuant | 37.4mAP/52.2NDS | 0.9mAP/0.6NDS | 36.0mAP/51.5NDS | 38.7mAP/52.9NDS |
| W4A4 Ours-Smooth | 51.4mAP/61.4NDS | 0.6mAP/0.3NDS | 50.4mAP/61.1NDS | 52.2mAP/61.9NDS |
Here, we additionally report the quantization results of the transformer-based 3D detector BEVFormer-Tiny. For this experiment, we use the same hyperparameters as those for the camera-based 3D detector BEVDet, as reported in our manuscript (Ours-AWQ: =0.7, Ours-Smooth: =0.9 with SVD rank 16). We use W4A16 and W4A8 bit settings, as W4A4 quantization results in severe performance degradation (0mAP / 0NDS) across all approaches, consistent with the intolerable low-bit degradation of BEVFormer reported in QD-BEV (Zhang et al., 2023). Under the W4A16 bit setting, Ours-AWQ demonstrates average gains of +0.5 mAP and +0.1 NDS across 10 random seeds. Under the W4A8 bit setting, Ours-Smooth shows average gains of +3.7 mAP and +6.4 NDS across 10 random seeds.
| PTQ Method | FP32 | W4A16 | W4A8 |
|---|---|---|---|
| SmoothQuant | 25.2mAP/35.5NDS | - | 3.0±0.1mAP / 10.9±0.3NDS |
| Ours-Smooth | 25.2mAP/35.5NDS | - | 6.7±0.5mAP / 17.3±0.5NDS |
| AWQ | 25.2mAP/35.5NDS | 20.1±0.2mAP / 32.7±0.2NDS | - |
| Ours-AWQ | 25.2mAP/35.5NDS | 20.6±0.4mAP / 32.8±0.4NDS | - |
Response to Reviewer V5ft (3/3)
W3: The calibration changes are not fully disentangled. Ours-AWQ changes the activation statistic from a mean to a maximum while also selecting activations and restricting the search objective. It is unclear whether Table 3’s union-of-sets baseline retains the maximum statistic. Controlled comparisons with simple percentile or MSE-based clipping would also help determine whether influence scoring offers benefits beyond range reduction.
Q1: Could you separate the effects of mean-versus-max statistics, activation selection, and restricting AWQ’s search objective, and compare against simple clipping under matched settings?
Response. We thank the reviewer for the careful comment. The union-of-sets baseline in Table 3 retains the maximum statistics, ensuring a matched comparison. To isolate individual contributions, we conduct an additional ablation study of Ours-AWQ, presented below. We report mean mAP and NDS over 10 independently sampled calibration subsets from the training split. Given the marginal differences under W4A16, we adopt the more challenging W3A16 setting to better distinguish component effects. For each configuration, we replace only the specified component, keeping all other settings unchanged.
As shown in the table, our influential activation selection and MSE clipping on the selected set improve mAP by +0.9 and +1.0 over random selection and simple MSE clipping, respectively. In contrast, using mean instead of maximum statistics or AWQ's original search objective yields negligible performance differences. These findings demonstrate that the primary gains stem from identifying and preserving influential activations using our proposed influence score, rather than the choice of activation statistics or search objective.
| Ours-AWQ | W3A16 mAP | W3A16 NDS |
|---|---|---|
| Max statistics (Ours) | 51.2±0.5mAP | 61.2±0.4NDS |
| Mean statistics | 51.2±0.3mAP | 61.3±0.4NDS |
| Random activation selection | 50.3±0.3mAP | 60.5±0.1NDS |
| AWQ’s search objective | 51.1±0.5mAP | 61.4±0.3NDS |
| Simple MSE clipping | 50.2±0.4mAP | 60.4±0.4NDS |
Response to Reviewer V5ft (1/3)
We appreciate the reviewer's acknowledgment of our method's practicality and its improvements in low-bit 3D detection. We are also grateful for the thoughtful feedback regarding the connection between structural sensitivity and quantization-induced task degradation, as well as a more comprehensive experimental evaluation. In the following, we would like to address each of these concerns one-by-one.
Response. Our method aims to identify informative activation outliers, so its effectiveness is most pronounced under activation quantization, where filtering uninformative outliers narrows the quantization range. Under W4A16, activations remain in FP16, so small margins among gradient-based scores are expected. Under W4A4, residual-only scoring underperforms our full formulation by 1.0 mAP on BEVDet and 1.5 mAP on CenterPoint (Table 3b). Furthermore, when varying only the selection score across 10 random calibration subsets on CenterPoint under W4A4, our influence score outperforms Fisher scoring and InlierQ's score by 1.9 and 1.7 mAP, respectively, while reducing the standard deviation from 1.2–1.4 to 0.6.
Response. Our formulation measures influence on the model prediction, quantifying how much the gradients’ low-rank structure is perturbed when an activation vector is removed. Thus, corrupting a high-influence activation vector perturbs the model prediction more and degrades task performance. The diagonal Fisher considers gradient magnitude but does not sufficiently measure the low-rank structure. Residual-only scoring also lacks the quantification of the low-rank structure’s perturbation, leading to insufficient reflection of model prediction degradation.
To show this, we conduct a quantitative analysis under controlled perturbation with 10 random seeds in the table below. We apply 50% magnitude amplification to the CenterPoint encoder’s top 5% scored output activations and measure the task loss (Bounding Box Loss and Classification Loss) perturbation of selection methods InlierQ’s score, diagonal Fisher information matrix, and our proposed influence. “Low influence” indicates lowest 5% scored selection, and “High influence” indicates our influential activation selection. Our selection with high influence scores produces approximately a 1.9× larger increase in task loss than perturbing those selected by InlierQ and Fisher scores. In contrast, perturbing activations assigned low influence scores has a negligible effect on task loss. These results suggest that our influence score measures actual task degradation under perturbations.