- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
NOVA generates synthetic 3D anomalies by extracting local deformation patterns from differences between aligned normal surfaces and amplifying them beyond an empirical normal-variation reference. The detector first learns local geometry and conditional covariance from normal data. These components are then frozen, and a scoring head is trained to rank deformed points above their normal counterparts. The paper evaluates the method on Real3D-AD and Anomaly-ShapeNet, with comparisons of synthesis methods, normal-only detection, training budgets, and registration sensitivity.
Critical strengths
-
The main contribution, in my view, is a strong baseline with a well-specified geometric model. NOVA explicitly uses aligned normal references to model local variation, then learns a scorer from synthetic pairs. The normal-only model is already competitive. Adding the scoring branch improves all four metrics in Table 3 on both datasets; P-AP rises from 42.58% to 52.18% on Real3D-AD and from 73.73% to 76.82% on Anomaly-ShapeNet. This is a useful result, particularly given the small model and limited synthetic training set.
-
The paper and supplementary code give enough detail to follow the pipeline, including the synthesis parameters, network dimensions, training schedules, calibration, and evaluation protocol. Per-category and per-seed results are also supplied. I appreciate the additional training-budget controls: they show that the effect of using more synthetic pairs cannot be separated from the number of optimizer updates. The documentation makes this a promising baseline for further work, although I have not run an independent reproduction.
Critical weaknesses
-
I find the novelty limited. Learning a discriminator from synthetic anomalies is well established; the more distinctive parts here are the deformation patterns extracted from normal surfaces and the frozen normal-geometry model. The synthesis comparison is informative, but it also shows that the proposed construction is not consistently better. In Table 2, CASL-based synthesis outperforms NOVA synthesis on all four Anomaly-ShapeNet metrics with the detector fixed.
It is also hard to tell which design choices account for the gain. The normal-only/full comparison does not isolate paired ranking from ordinary binary classification, or freezing from joint training. The quantitative parameter studies cover sample count, training duration, and head widths of 64 and 128, but leave several central geometric settings untested: deformation strength, region size, the normal-variation amplitude reference, and the normal-model geometry scale. Figure 4 only shows what different strengths look like. These omissions matter because they determine how normal variation is turned into synthetic anomalies.
-
My main concern is the dependence on alignment and on how well the four normal references cover test-time shape variation. Appendix D.2 reports both average changes and large individual losses: the maximum observed category P-AP drops are 13.51 percentage points for Chicken and 29.99 points for cap4, each for one residual and head-training seed. What is missing is the relationship between error size and performance. At what point does detection start to deteriorate? How common are these large drops? What happens in the cases with residual rotations close to 180 degrees?
This experiment replays transforms estimated from training normals while keeping the synthetic data and trained models fixed. It does not test errors introduced during synthesis, or separately measure the effect of genuine normal-shape differences beyond rigid pose. Cross-instance registration partly captures such differences, but mixes them with alignment effects. The current evidence leaves open how well NOVA handles valid normal shapes outside the variation represented by its reference set.
Despite these concerns, I lean toward acceptance. The performance and the detailed, usable implementation give the paper practical value. I would need stronger evidence for the individual design choices and the limits of its robustness to recommend a clear accept.
Questions for authors
-
Could you plot the change in detection performance against residual rotation and normalized displacement? Please show the individual category/residual results as well as the trend, and report the frequency of P-AP drops above, for example, 5 and 10 percentage points. Please specify what is counted in those frequencies. I would also like to see the detection results and anomaly maps for the near-180-degree cases. Since some objects may be symmetric, a surface-alignment measure alongside the rotation angle would help interpret these cases.
-
How does NOVA behave when a valid normal test shape differs from the four references beyond rigid pose? An analysis of false positives and detection performance as reference-shape coverage changes, with alignment accounted for, would address this concern. Have you also measured whether misalignment between training normals affects the synthetic anomalies and the detector trained on them?
-
How does binary classification compare with paired ranking when the normal features, synthetic pairs, head, and training budget are held fixed? Separately, what is the effect of freezing versus updating the normal branch? These comparisons would help explain the gain beyond showing that the complete scoring branch works.
-
How sensitive are the results to the added deformation amplitudes, the 512-point region size, and the quantile used for the normal-variation amplitude reference? What supports the chosen normal-model geometry scale and refinement settings? A targeted sensitivity study of these settings would help establish how much tuning the method needs.
For future work, I would be particularly interested in how far this baseline can be simplified. The head already has only about 19,000 parameters, and eight pairs per category retain much of the performance in Table 8. But the width study only compares 64 with 128. Could a linear or narrower head, a simpler deformation rule, and fewer than eight pairs achieve similar results under matched training budgets? Showing the trade-off between accuracy, model size, synthesis cost, and sample count would make the practical contribution more compelling. This is a suggestion for extending the work, not a condition for my recommendation.
4 comments
Response to Reviewer roUa (2/4)
Q4. Sensitivity, and the basis for the settings
Table E. One setting changed at a time (P-AP, %).
| Setting | Real3D-AD | ASN |
|---|---|---|
| Submitted setting | 52.18 ± 0.74 | 76.82 ± 0.17 |
| Increments Δ × 0.5 / × 2 | 52.21 ± 1.49 / 52.91 ± 0.46 | 76.94 ± 0.06 / 76.86 ± 0.24 |
| Amplitude-reference quantile 95% / 99.9% | 52.45 ± 1.03 / 51.95 ± 0.83 | 76.83 ± 0.18 / 76.88 ± 0.21 |
| Region size 256 / 1,024 points | 53.52 ± 0.21 / 50.60 ± 0.86 | 76.82 ± 0.38 / 76.84 ± 0.16 |
| Geometry scale σ₀ × 0.5 / × 2 (retrained; matched baseline 52.01 ± 0.45) | 44.09 ± 0.59 / 51.15 ± 1.01 | 76.53 ± 0.41 / 76.37 ± 0.11 |
| Normal-only refinement removed (Real3D-AD) / added (ASN) | 48.48 ± 1.36 | 77.02 ± 0.14 |
- Low sensitivity: the increments and the quantile (at most 0.73 points on either dataset).
- Moderate sensitivity on Real3D-AD: region size. Regions of 256 and 128 points are 1.34 and 1.42 points better than our 512-point default (53.52 and 53.60), while 1,024 points is 1.59 worse and 64 points becomes unstable across seeds (Appendix F, Table K). The submitted 512-point setting is therefore not the most favourable one for NOVA. We keep it as submitted and report the others alongside it.
- High sensitivity on Real3D-AD: the geometry scale when halved (−7.9 points); doubling costs 0.9. σ₀ is set from normal data (the 75th percentile of nearest-reference distances on Real3D-AD) and is reused unchanged on ASN.
- Refinement adds 3.5 points on Real3D-AD with the head (7.7 for the normal branch alone). On ASN it gives +0.20 P-AP and −0.15 O-AUROC; the ASN model in the paper is trained without it.
ASN moves by less than 0.5 points under every change. The practical guidance is that σ₀ and the region size are the two settings to check on a new dataset.
Response to Reviewer roUa (3/4)
Q1. Alignment-error size vs. performance; frequency of large drops; near-180° cases
Figs. F1–F2 plot the P-AP change for every category/residual combination (108 on Real3D-AD, 360 on ASN, each averaged over three head-training seeds) against rotation angle, displacement of corresponding points, and two-way nearest-surface distance, with binned medians. Fig. F3 gives the full category × residual matrix.
When does detection deteriorate? Surface distance is the informative axis, and the pattern is a tail of isolated failures on top of a flat median.
- Below a surface distance of 0.1%D, no case loses more than 0.49 points on Real3D-AD (93 cases) or 1.35 points on ASN (167 cases).
- Between 0.1% and 1%D the median change stays at 0.0 to −0.1 points, and a tail appears: 1 of 14 Real3D-AD cases and 11 of 193 ASN cases lose more than 5 points.
- Above 1%D we have a single case (Chicken), which loses 12.7 points.
Table F. Frequency of large P-AP drops.
| Dataset | Unit counted | Drop > 5 points | Drop > 10 points |
|---|---|---|---|
| Real3D-AD | category × residual, mean of three seeds | 2 / 108 (1.9%) | 1 / 108 (0.9%) |
| Real3D-AD | category × residual × seed | 6 / 324 (1.9%) | 3 / 324 (0.9%) |
| ASN | category × residual, mean of three seeds | 11 / 360 (3.1%) | 5 / 360 (1.4%) |
| ASN | category × residual × seed | 33 / 1,080 (3.1%) | 17 / 1,080 (1.6%) |
Each category contributes nine residual transforms (three source scans × three residual types), estimated on training normals and replayed on all test objects of that category.
Near-180° cases. There are 24 cases with rotation ≥ 170° (5 on Real3D-AD, 19 on ASN). Their surface distance is small, 0.31–0.40%D and 0.25–0.47%D, and the mean P-AP change is −1.40 to +0.60 and −3.29 to +0.91 points. The rotated object lies close to the reference surface, which the angle alone does not show. The largest single-seed losses occur at small angles: 13.51 points for Chicken at 5.2° and 29.99 for cap4 at 3.3°. Fig. F4 lists all 24 cases with angle, surface distance and four metrics; Fig. F5 shows anomaly maps before and after the transform on a shared colour scale for the worst near-180° case of each dataset.
Q2. Normal shapes beyond the reference set; misalignment between training normals
Shape coverage with alignment accounted for. For each normal test object (604 on Real3D-AD, 600 on ASN) we compute the remaining two-way surface distance to its closest reference after local trimmed ICP, group objects into quartiles within each category, and measure the false-positive rate of the unchanged detector at its calibrated threshold (Fig. F6).
| Normal-object FPR (%) | Q1 (closest) | Q2 | Q3 | Q4 (farthest) |
|---|---|---|---|---|
| ASN | 3.33 | 3.12 | 3.33 | 10.28 |
| Real3D-AD | 27.78 | 20.37 | 20.30 | 24.73 |
On ASN the false-positive rate triples for the quartile of normal shapes farthest from the references. On Real3D-AD the rate is 20–28% in every quartile and shows no monotonic relation to this distance. The threshold is calibrated on the four training normals and their scan-perturbed copies; the reported AUROC and AP do not depend on it.
Fewer references. Removing any one of the four references lowers P-AP by 1.2–6.2 points on Real3D-AD and 0.1–0.8 on ASN, and O-AUROC by 0.1–3.5 and 0.2–0.4. At the same time, the normal-object FPR of the three-reference sets is lower: 7.7–9.2% against 23.3% on Real3D-AD, and 1.9–2.9% against 4.7% on ASN. Each reference set has its own recalibrated threshold, so these rates are measured at different operating points; the threshold-free metrics above show that detection is weaker with fewer references.
Misalignment between training normals. We measured this directly (Table R2 in the general response): references are perturbed before synthesis, and patterns, pairs and the head are all regenerated. Up to 1° rotation and 0.5%D translation, P-AP stays within 0.41 points on Real3D-AD and within 0.10 on ASN; a 3° rotation costs 2.10 points on Real3D-AD and leaves ASN unchanged (76.96 against 76.90); all ASN settings stay within 0.10 points.
Response to Reviewer roUa (4/4)
Suggested extension: how far can the baseline be simplified?
We ran this study because it bears directly on the practical value you describe. All configurations receive the same 256 updates per category (2 / 4 / 8 / 32 pairs train for 128 / 64 / 32 / 8 epochs). Appendix F reports all 48 configurations (Figs. F9–F11).
Table G. Head size, pair count and cost (Real3D-AD / ASN).
| Synthesis + head | Pairs per category | Head parameters | P-AP (%) | Head inference per shape (ms) | Generation per category (ms) |
|---|---|---|---|---|---|
| NOVA + MLP-64 (submitted) | 32 | 19,201 / 19,393 | 52.18 / 76.82 | 0.31 / 0.29 | 110.1 / 295.1 |
| NOVA + MLP-64 | 2 | 19,201 / 19,393 | 52.71 / 77.01 | 0.31 / 0.29 | 7.0 / 16.1 |
| NOVA + MLP-16 | 2 | 4,033 / 4,081 | 52.44 / 76.53 | 0.23 / 0.21 | 7.0 / 16.1 |
| NOVA + linear | 2 | 234 / 237 | 49.05 / 75.45 | 0.05 / 0.06 | 7.0 / 16.1 |
| Gaussian + MLP-64 | 2 | 19,201 / 19,393 | 45.01 / 77.35 | 0.31 / 0.29 | 2.2 / 5.8 |
Timings are on one A100 from precomputed features; generation time starts after reference differences are prepared (CPU for Real3D-AD, GPU for ASN).
- Fewer pairs. Two pairs per category match 32 pairs under the same update budget.
- Narrower head. A width-16 MLP with a fifth of the parameters stays within 0.3 points. A linear head loses 3.1 points on Real3D-AD and 1.4 on ASN.
- Simpler deformation rule. Using the mean difference to the references directly gives 16.55 / 70.48 P-AP, and a single-reference difference gives 25.26 / 72.38. Rescaling the mean difference to NOVA's RMS displacement recovers 51.99 / 76.54. This rescaled rule borrows its amplitude from NOVA; with the amplitude controlled, a simple average of normal-surface differences is a workable pattern.
Together, these results show that the baseline remains effective with a head of about 4,000 parameters and two synthetic pairs per category.
Summary against the two points you raised
| Design choice | Evidence |
|---|---|
| Normal-derived pattern | +4.5 to +7.4 P-AP over Gaussian on Real3D-AD; no advantage on ASN |
| Amplitude reference from normal data | +1.3 on Real3D-AD; a dataset-level reference suffices |
| Frozen normal branch | +11.2 on Real3D-AD, +0.6 on ASN |
| Loss on paired samples | Ranking and BCE perform comparably |
| Region size, σ₀ | The two sensitive settings on Real3D-AD |
| Robustness limit | Measured boundary |
|---|---|
| Test-time misalignment | No loss above 1.4 points below 0.1%D surface distance; isolated large losses above it (2–3% of cases over 5 points) |
| Misalignment during synthesis | Stable to 1° + 0.5%D on Real3D-AD and at every tested level on ASN; −2.1 points at 3° on Real3D-AD |
| Reference coverage | ASN false positives rise from 3% to 10% for the farthest quartile of normal shapes; one fewer reference costs up to 6.2 P-AP points |
Thank you again for the constructive questions and for the simplification idea, which turned out to be one of the most useful additions to the paper. We would be glad to provide further analyses during the discussion period.
Response to Reviewer roUa (1/4)
Thank you for the detailed and thoughtful review, and for stating so clearly what evidence would support a clearer recommendation. We ran every analysis in your four questions, and also the simplification study you suggested. The main outcomes are:
Tables R1 and R2 are in the general response; all figures are in Appendix F of the revised PDF.
Novelty, and CASL on Anomaly-ShapeNet
We agree that training a discriminator on synthetic anomalies is well established. Our claims concern the two parts you identify, and the new controls show where each one holds.
Patterns from normal surfaces. On Real3D-AD they exceed a Gaussian pattern by 4.5–7.4 P-AP points under all three strength rules, consistently across seeds and in 11–12 of 12 categories (Table R1). They also exceed a Perlin pattern placed in the same regions by 0.4–1.3 points (see our reply to yby6 for the comparison with Perlin-noise synthesis). On ASN, where normal shapes differ much less, they give no advantage: CASL and the procedural patterns are 0.5–1.3 points higher on average. For the procedural patterns, most of this difference comes from three categories (bucket0, bottle1, bottle3; Fig. F7). This agrees with the statement in the submission that no strategy is best on both datasets, and we will state the scope explicitly: the pattern helps on real scans.
Frozen normal geometry with paired supervision. Freezing is worth 11.2 P-AP points on Real3D-AD, and BCE matches the ranking loss (Q3 below). These controls support the second contribution: paired synthetic supervision applied to preserved normal-geometry features. The result does not hinge on the specific loss.
Q3. BCE vs. paired ranking; frozen vs. joint training
Table D. Loss and freezing controls (%). Normal features, synthetic pairs, head, displacement weights and budget are identical within each comparison.