- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes AnyNormal, a training-free method for few-shot 3D anomaly detection. Its two ideas are normal-variation calibration, which calibrates Euclidean, point-to-plane, and normal-angle residuals against the variation among normal objects, and per-reference scoring, which scores each reference before selecting via a pointwise minimum rather than pooling references first. A parameter-free fixed mean of the calibrated features serves as the scorer. Across Real3D-AD, Anomaly-ShapeNet and MulSen-AD it matches or exceeds learned baselines, and a scorer trained on one category transfers across datasets without further updates.
Critical strengths
- The core contribution is a well-supported negative result: for few-shot 3D anomaly detection, a scorer with no learned parameters is enough.
- Table 5 isolates calibration, local context, and per-reference scoring under matched conditions with confidence intervals, and reports cases where a cue is not best.
Critical weaknesses
- Novelty is at the level of insight and recombination, not a new mechanism. The components are individually standard. The contribution is the score-then-select framing plus calibration against normal variation. The isolated gains are modest.
- As the authors acknowledge, appearance-only defects and reference-absent normal variation are out of scope, and misalignment can be mistaken for a defect. Numerous constants are fixed with appendix justification but without sensitivity analysis, leaving robustness partly unverified.
Questions for authors
- On MulSen-AD the main result uses the full normal training set rather than four-shot support. What accuracy does the method reach on MulSen-AD under the same four-shot setting as the other two benchmarks? This directly affects how firmly the few-shot claim can be stated.
- How do the results change if the main constants, such as the percentile levels, the clipping constant, and the orientation threshold, are varied?
- The analysis that a learned scorer collapses to the fixed mean is derived for a monotone positive-weight MLP. Does the same collapse occur with an unconstrained scorer, or is this conclusion specific to the monotonicity constraint?
- Scoring uses a pointwise minimum across references, so a point is flagged only when every reference scores it high. How does accuracy change as the number of references grows, and is there evidence the minimum does not erode recall?
4 comments
Response to Reviewer iQCc (2/2): scorer constraints and recall
iQCc — 2/2
Q3. Removing the positive-weight constraint
No. Under the same normal-only objective, an unconstrained scorer can reverse the anomaly ranking. Both networks use 12→32→16→1 layers, Softplus hidden activations and 120 updates on four normal airplane scans. We test three seeds, including a model initialized with exactly the same weights as the positive-weight network.
Table I3. Point AUROC (%) before/after scorer training. Three-seed means. Table 26, p. 25.
| Scorer | Real3D-AD | Anomaly-ShapeNet | MulSen-AD |
|---|---|---|---|
| Fixed mean | 95.06 | 96.24 | 87.04 |
| Positive-weight MLP, random initialization (untrained) | 95.03 | 96.24 | 87.01 |
| Positive-weight MLP, random initialization (trained) | 95.02 | 96.26 | 86.98 |
| Unconstrained MLP, same initial weights (trained) | 9.38 | 5.78 | 28.79 |
| Unconstrained MLP, random signed weights (trained) | 9.41 | 6.44 | 28.02 |
When the two networks start with identical weights, they give the same ranking, but unconstrained training reverses it. The normal-only loss rewards low scores on normal examples without requiring larger deviations to receive higher scores. Positive weights preserve this increasing score direction. With constant initialization, hidden units remain identical: the network reduces to an increasing function of a positive-weight sum of the features. The near-equality to fixed averaging under random initialization is an experimental result. Fixed averaging retains the intended score direction without this fitting step.
Q4. Reference count and recall
ROC recall at FPR ≤5% improves as the bank grows, but keeping the old threshold causes more misses. This study fixes the original coordinates and calibration, so its four-reference baseline differs from Q1, which recomputes all stages from four scans.
Table I4. MulSen-AD reference-bank growth. Percentages, three nested orders; recall is the maximum empirical ROC recall at FPR ≤5%. FNR/FPR use the four-reference normal-test-score 95th-percentile threshold retrospectively. “All” uses all available normal training scans per category.
| References | Point AUROC | ROC recall@5%FPR | FNR / FPR (fixed threshold) |
|---|---|---|---|
| 4 | 81.44 | 59.39 | 38.49 / 7.05 |
| 8 | 83.40 | 60.98 | 44.54 / 2.06 |
| 16 | 84.96 | 64.51 | 47.86 / 0.75 |
| All | 87.04 | 68.32 | 50.52 / 0.55 |
The larger bank improves ranking and recall at the same FPR limit. At full support, normal/anomalous score ratios to their four-reference values have medians of 0.25/0.89 across category/order pairs. Normal scores fall more, so keeping the old threshold produces fewer false alarms and more misses. The threshold should therefore be checked for the bank used at deployment.
Under the same fixed-coordinate/calibration protocol, increasing from two to four references raises point AUROC from 93.71 to 95.06% on Real3D-AD and 95.45 to 96.24% on Anomaly-ShapeNet.
Follow-up to Reviewer iQCc: novelty and the size of the gains (Weakness 1)
Our first response covered Q1–Q4. This note answers Weakness 1 directly.
We agree with your description: the components are standard, and the contribution is the score-then-select framing, calibration against normal variation, and the result that a parameter-free scorer is then sufficient. We see this as the strength of the paper. It shows that accurate few-shot 3D detection that generalizes does not require complex network design: one fixed rule with the same constants covers 67 categories of three benchmarks without training, and a compressed configuration runs at 20.46 ms per object with its accuracy cost reported beside it (Table 5). Three points support this.
1. The framing isolates a design choice. In memory-bank detectors the matching distance is the anomaly score, so selecting a reference and scoring it are one step. With calibrated point-to-plane and normal-angle residuals they become two steps, and their order matters: among the 64 pooled candidates of a query point, the lowest-scoring one is not the nearest one at 38%/41% of points on Real3D-AD/Anomaly-ShapeNet (pointwise features). All 13 variants in Table 25 that score before selecting outperform selecting by proximity first in point AP, on both datasets.
2. The effects are larger than the matched-budget number suggests. The 0.68/1.50 AP points we quoted is our most conservative control: its baseline already scores four candidates before selecting.
| Comparison | Gain |
|---|---|
| Score-then-select vs. select-then-score (Table 6b) | +2.50 / +2.65 point AP on Real3D-AD / Anomaly-ShapeNet; 12/12 and 36/40 categories improve |
| Calibration, neighborhood averaging fixed (Table 2a) | +5.48 / +1.35 / +4.10 point AUROC on the three datasets |
| Full method vs. Simple3D, identical inputs and references (Sec. 3.2) | +19.7 / +13.1 object/point AUROC on Real3D-AD; +11.2 / +6.6 on Anomaly-ShapeNet |
| Four scans vs. full-support baselines, MulSen-AD (Table 1b) | 81.67 vs. best listed 80.3 point AUROC |
3. The negative result you highlighted now has an explanation. Your Q3 led to the control we consider the most informative addition. From identical initial weights, removing the positive-weight constraint lets normal-only training reverse the ranking (point AUROC 9.4/5.8/28.8%), while positive-weight scorers stay within 0.1 points of the fixed mean (Table 26). What the scorer needs is the direction "larger calibrated deviation, higher score"; the fixed mean supplies it without training.
On Weakness 2 and Q4: Table 17 reports the sensitivity sweeps, Figure 6 (p. 22) plots accuracy, recall and threshold behavior against bank size, and the Limitations paragraph states the alignment, coverage and threshold conditions.
We would be glad to clarify anything that remains open.
Gentle reminder
We would be grateful to know whether the four-shot MulSen-AD results, the parameter sensitivity study and the unconstrained-scorer control address your questions, and we are happy to clarify anything further before the discussion period closes.
Response to Reviewer iQCc (1/2): four-normal support and sensitivity
iQCc — 1/2
Response to Reviewer iQCc
Thank you for highlighting the training-free result. Normal references provide the shape information and the scale for judging deviations; the same fixed scoring rule can then be used across categories. Our contribution is to show which comparisons make this possible: measuring geometric differences against normal variation and scoring each reference before selection.
With neighborhood averaging fixed, calibration adds 5.48/1.35/4.10 point-AUROC points on Real3D-AD/Anomaly-ShapeNet/MulSen-AD, using four references on the first two and full support on MulSen-AD. At four scored matches, per-reference scoring adds 0.68/1.50 point-AP points on the first two datasets at similar CPU latency.
The scorer controls help explain why this works. Starting from the same ranking, unconstrained normal-only training can reverse it, while positive-weight scoring stays close to fixed averaging (Q3). The useful information is already present in the calibrated geometric features. A fixed mean uses it without having to fit scoring weights for each category.
Q1. Strict four-normal MulSen-AD
We evaluated all 15 categories and 644 objects for each of three support selections, using the same four scans to fit alignment, normalization, calibration and the reference bank with one fixed registration profile.
Table I1. MulSen-AD support comparison. Object/point AUROC (%); mean ± support-selection SD. Tables 1 and 16, pp. 5 and 20.
The strict four-normal point AUROC exceeds the published baseline values listed in Table 1(b), including Simple3D (80.3%) and MAD-3D (80.1%), under their respective published protocols. The table reports our two support regimes separately.
Q2. Parameter sensitivity and geometric conditions
We varied each parameter individually and retained the original defaults.
Table I2. Point-AUROC ranges (%). Main support settings; MulSen-AD full support. Table 17, p. 20.
Percentile pairs are (90,99), (95,99), (95,99.5), (95,99.9) and (97.5,99.5). Percentile and clipping effects are small over these ranges. Orientation affects Anomaly-ShapeNet point AP more: 68.11/65.20/61.81% at thresholds 0.25/0.5/0.75.
We also tested the alignment and normal-shape coverage concerns raised in your review (Tables 21 and 23–24). On 12 objects, rerunning registration recovers geometry within 1% of the diagonal of the normal-reference bounding box in 107/108 perturbations, relative to the registration output without the added perturbation. At equal reference counts, geometrically diverse normal supports improve point AUROC by 1.46/0.93/2.91 points on Real3D-AD/Anomaly-ShapeNet/MulSen-AD.