acceptodds
Under review as a conference paper at ICLR 2027

PHYSDEFECT: A PHYSICS-BASED BENCHMARK FOR 3D ANOMALY DETECTION

Abstract

3D anomaly detection is central to industrial inspection, yet progress is constrained by scarce real defects and confidentiality restrictions. Existing synthesis pipelines often apply analytic displacements directly to point clouds. While they can create plausible local anomalies, they struggle to represent global shape deviations, material-dependent responses, and the scanning process used in practice. We introduce , a physics-based benchmark for 3D anomaly detection. Its synthesizer organizes seventeen mechanisms into material deformation, removal, and addition, modeling elastoplastic response, transport-driven interface recession, and continuous deposition and growth. Geometry and material state pass between successive events. A triangulation-based simulated scanner explicitly models occlusion, artifacts, and measurement noise, producing incomplete observations. Across 100 industrial geometries, contains 3,213 anomalous and 2,000 normal instances and evaluates eight established methods plus a geometric baseline. Their localization performance is substantially below published results on other benchmarks, while changes in scanning conditions affect methods in both directions. The benchmark and its components are open source to support research on physical defect synthesis, scan simulation, and transfer to real settings.

Primary area:
datasets and benchmarks
Keywords:
ndustrial inspection, physics-based simulation, synthetic defect generation, point-cloud anomaly detection, sensor simulation, domain generalization, benchmark overfitting, self-registration, digital twins

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YwyDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This submission introduces PhysDefect, a benchmark for 3D anomaly localization that combines physics-based defect synthesis with a projector-stereo scanner simulator. The benchmark models seventeen mechanisms spanning deformation, removal, and addition, and evaluates point-cloud methods across three scanning conditions and several normal-reference-bank sizes. The paper also uses a paired simulation of native Anomaly-ShapeNet defects to study the effect of observation changes.

Critical strengths
  1. Well-motivated benchmark direction. The motivation that existing synthetic 3D anomalies are predominantly local and clean is compelling. Treating defect formation and the acquisition process as connected sources of variation is a useful framing.
  2. Broad and coherent synthesis design. The three families and seventeen mechanisms cover a wider range of morphology than standard local displacement pipelines. Carrying geometry and material state between events is an interesting capability, especially for stress-corrosion-type chains.
  3. Useful controlled-observation protocol. The easy, medium, and hard scans and the paired ShapeNet study make it possible to study how acquisition changes interact with method behavior rather than treating the point cloud as a noise-free input.
  4. Promising reproducibility practices. The manuscript specifies random seeds, generation records, scanner parameters, annotations, and several numerical checks.
Critical weaknesses
  1. Physical realism is asserted but not externally validated. Table 7 establishes internal numerical behavior (for example, a finer-grid pitting reference and an analytical capillary corner), not whether the generated morphology, severity, and co-occurrence of mechanisms resemble industrial defects. Similarly, scanner calibration relies on three aggregate statistics from released point clouds. The paper needs evaluation on measured scans with known defects, or at minimum a carefully documented comparison of real and synthetic defect morphology, size, localization, and scanner statistics. A train-on-PhysDefect, test-on-real transfer experiment would directly support the practical premise.
  2. The evaluation metric does not measure the full detection problem. The main P-AUROC is computed only for scans with both point labels, and undefined or failed predictions are omitted. Artifacts, outliers, and mixed pixels remain model inputs but are excluded from scoring, while the 400 normal test scans per level are not used to assess object-level false alarms. This can substantially overstate robustness when defects are occluded or acquisition artifacts are common. Please report object-level AUROC and false-positive metrics on normal objects, a localization metric such as AU-PRO, and per-mechanism/per-level visibility, exclusion, and failure rates.
  3. Baseline comparisons are not controlled tightly enough for a benchmark claim. The compared rows differ in more than the underlying method: point counts, pretraining, modality selection, normalization, and registration procedures vary; Simple3D is only evaluated at K = 1. The appendix explicitly notes that feature and registration effects are entangled. Consequently, Table 4 cannot isolate detector quality, and the strong PTP baseline may partly reflect the chosen alignment protocol. Please provide a standardized input and registration pipeline where possible, retain released method settings where not, and report sensitivity to alignment quality and random seeds.
  4. The paper lacks ablations that attribute gains or difficulty to the proposed physical components. The three scan levels jointly vary coverage, stereo visibility, noise, multipath, and post-processing. Likewise, the paper does not compare its physics-based synthesis with analytic or procedural synthesis while matching geometry, defect area, and severity. Controlled factorial ablations are needed to show which scanner components and which physical mechanisms make the benchmark harder or more realistic, rather than simply different.
  5. Dataset composition and aggregation need clearer reporting. The 100 evaluation geometries are selected from 34,372 eligible CAD meshes, but the manuscript does not provide the exact category split, selection protocol, mechanism-wise sample counts, or severity distributions in the main paper. The metric macro-averages only defined geometry-mechanism cells, and Table 11 shows that effective sample sets differ by method. Please release and summarize these distributions, report uncertainty intervals, and show results under a common valid-scan subset or an explicit treatment of missing and failed cases.
  6. Some quantitative details appear inconsistent or insufficiently explained. Table 2 lists the medium local thickness ratio as 0.028, whereas Table 10 lists 0.057 for the medium configuration. If these values use different geometry sets, point samplings, or processing stages, that distinction should be explicit. The definition and selection of the label threshold tau and the severity controller should also be made reproducible for each mechanism.
Questions for authors
  1. What measured-defect or real-scan evidence supports the claim that the simulator closes a meaningful sim-to-real gap?
  2. How do the conclusions change when all normal test scans and scanner artifacts are included in a false-positive evaluation?
  3. Can the authors provide a severity-matched comparison with at least one existing geometric synthesis method and isolate scanner components one at a time?
  4. Why do the reported medium thickness ratios differ between Tables 2 and 10, and what exact data subset and processing stage does each table use?
3 comments
AuthorsOpenReview ↗

Response to Reviewer YwyD(1)

Point 1: Evidence from measured defects and real-data evaluation

We thank the reviewer for highlighting the need to validate PhysDefect against measured defects and real detection outcomes. We added two experiments: fitting measured defect geometry and evaluating synthetic supervision on real data. First, we impacted a 100 × 100 × 0.1 mm copper foil and a 40 × 40 × 40 mm aluminum industrial heat sink with a 28 mm solid steel ball, then captured the defects using a FreeScan Combo 3D scanner. With fixed defect locations and equal numbers of candidate evaluations, we compared PhysDefect with AF3AD, NormAS, PA3AD, and GLFM, measuring point-to-surface RMSE on held-out scan points excluded from fitting. For the isolated copper-foil dent, RMSEs ranged from 0.024 to 0.038 mm, with PhysDefect achieving 0.0323 mm. For the heat sink, PhysDefect achieved the lowest RMSE of 0.4131 mm, reducing error by 13.5% relative to the best competing method under unconstrained displacement and by 26.1% under matched total vertex displacement. Second, we generated training anomalies from 48 normal templates using four physical material-removal mechanisms and evaluated on 1,206 real Real3D-AD test samples, keeping the DGCNN architecture, training budget, and three random seeds fixed across conditions. Compared with a normal-only replay control, micro P-AUROC increased from 0.5003 ± 0.0245 to 0.5995 ± 0.0149, where ± denotes the standard deviation across seeds. Real anomalous test samples were not used for parameter selection. These experiments provide evidence of fitting measured defect morphology and improving real point-level localization through synthetic supervision, respectively. The supplementary material presents photographs, scan overlays from identical viewpoints, copper-foil cross sections, and complete fitting and transfer results.

Point 2: Detection metrics, coverage, and failures

We thank the reviewer for highlighting the importance of false alarms on normal objects in practical inspection. The original P-AUROC measures defect localization within a scan; distinguishing normal from defective objects requires a separate evaluation. We added O-AUROC using the 400 normal scans in each scanning tier, defining the object score as the 99th percentile of anomaly scores over all input points, including acquisition artifacts. For fixed-threshold evaluation, we set the threshold to the 95th percentile of object scores from 80 normal scans spanning 20 CAD geometries, then evaluated normal and anomalous scans from the remaining 80 geometries. In the medium tier, PTP and Simple3D achieve O-AUROCs of 0.6483 and 0.6689, respectively, whereas their P-AUROCs on the common valid subset are 0.7958 and 0.7252, reversing their relative ranking. At the threshold calibrated on normal scans, PTP has a normal-object false-positive rate of 4.69% and an anomalous-object detection rate of 5.05%. These results demonstrate that localizing observed defects and identifying defective objects are distinct evaluation objectives. We therefore report O-AUROC, normal-object false-positive rates, and the corresponding detection rates alongside P-AUROC. We also added AUPRO with an FPR integration limit of 0.3: PTP and Simple3D achieve 0.4722 and 0.4045, respectively, in the medium tier. Fully unobservable defects account for 0.68%, 0.90%, and 1.40% of anomalous samples in the easy, medium, and hard tiers, respectively. Among the 3,613 test scans in the hard tier, PTP and Simple3D have 38 and 52 inference failures, respectively. The shared supplementary material provides complete results across methods and scanning tiers, statistics on unobservable defects, unscorable cases, and failures by mechanism and tier, and comparisons on the common scorable subset with confidence intervals.

AuthorsOpenReview ↗

Response to Reviewer YwyD(2)

Point 3: Baseline configurations, reference-bank size, and alignment sensitivity

We thank the reviewer for highlighting the need to clarify baseline configurations, reference-bank sizes, and registration conditions. We will report each method’s input point count, pretraining source, input modalities, normalization, and registration settings so that the complete configurations underlying the results can be inspected. For Simple3D, we added evaluations at K=1, 4, and 16, keeping the test scans, registration cache, feature extraction settings, and 5% coreset sampling ratio fixed, with nested sets of normal references. On the common scorable subset of 3,101 anomalous scans spanning 98 CAD geometries in the medium tier, CAD-macro P-AUROC is 0.7187, 0.7456, and 0.7602, respectively. Increasing K from 1 to 16 improves P-AUROC by 4.15 percentage points, with a CAD-clustered bootstrap 95% confidence interval of [3.57, 4.79] percentage points. Regarding registration, the main evaluation applies independent random rotations and translations to normal references and test scans, then estimates alignment using only the observed point clouds; ground-truth transformations are not supplied to the detectors. Measurement noise, acquisition artifacts, and surface-processing errors are introduced during scan generation to simulate practical acquisition conditions. To quantify sensitivity to residual alignment errors, we additionally selected 16 CAD geometries, retained their scanning errors, and used the simulator’s known common coordinate frame as a diagnostic reference. We then applied controlled rotational or translational offsets to the test clouds and scored them without re-registration. On the common scorable subset of 497 anomalous scans in the medium tier, PTP achieves a CAD-macro P-AUROC of 0.8074 after estimated registration and 0.8333 in the known common coordinate frame. Adding rotational offsets of 1° and 5° yields 0.7362 and 0.5657, respectively; a translational offset of 0.5% of the reference cloud’s bounding-box diagonal yields 0.6587. These results quantify the effects of reference-bank size on Simple3D and alignment quality on PTP. The supplementary material will provide complete results across K values, valid-sample and failure counts, and alignment-sensitivity curves.

Point 4: Controlled synthesis and scanner ablations

We thank the reviewer for highlighting the need for controlled ablations of physical synthesis and scanning components. We added two experiments. First, we compared PhysDefect and AF3AD on the same CAD geometries under ideal observation and medium simulated scanning, covering 99 CAD geometries, 17 mechanisms, 3,053 matched pairs, and 11 detector configurations. We matched total vertex displacement for 1,300 topology-preserving pairs and a surface-deviation proxy for 1,753 material-addition or removal pairs, with relative budget differences within 10%. An additional analysis constrained the visible anomalous-point fraction and defect depth. For impact defects, PhysDefect yields P-AUROCs averaging 6.27 and 5.80 percentage points below AF3AD under ideal observation and simulated scanning, respectively. After additionally constraining defect extent and depth, 9/11 configurations retain this direction under both observation conditions. For bending defects, P-AUROC instead increases by an average of 1.55 and 0.71 percentage points, showing that detection difficulty depends on the mechanism. Second, we fixed defect geometry across 8 CAD geometries and 3 mechanisms, separately disabling measurement noise, artifacts, or surface reconstruction, or halving the scanning viewpoints. For example, disabling measurement noise changes the P-AUROCs of PTP, PatchCore-FPFH, and Simple3D by −0.40, +1.53, and +2.68 percentage points, respectively. The CAD-clustered bootstrap 95% confidence intervals for all 12 scanning-component differences include zero, so these results do not establish a consistent directional effect of any component across all algorithms. These controlled comparisons nevertheless quantify sensitivity to individual perturbations, informing algorithm selection and validation under practical acquisition conditions. The shared supplementary material provides examples across the four synthesis-by-observation conditions and individual scanning interventions, together with results by mechanism and confidence intervals.

AuthorsOpenReview ↗

Response to Reviewer YwyD(3)

Point 5: Dataset composition and statistical aggregation

We thank the reviewer for highlighting the need to clarify dataset composition and statistical aggregation. We compiled identifiers, sources, and categories for all 100 CAD geometries, together with sample counts and realized severity distributions for all 17 mechanisms. The geometries comprise 34 models from MCB-A, 33 from MCB-B, and 33 from SimJEB, retaining 39 original category labels. Selection first filters for watertight, SDF-compatible models with at most 200,000 triangular faces, then applies deterministic farthest-point selection over geometric descriptors within each source to cover diverse shapes. The dataset contains 3,213 anomalous and 2,000 normal instances, with 148–198 anomalous instances per mechanism. Severity statistics are computed from the final generated outputs: we report the median and 10th/90th percentiles of maximum geometric deviation for each mechanism, alongside anomalous vertex fractions and, where applicable, anomalous surface-area fractions. For example, the median anomalous vertex fraction is 2.65% for impact defects and 41.74% for vibration-induced deformation, reflecting the distinction between localized defects and extensive deformation. To address differences in valid samples across methods, we added comparisons on common scorable subsets containing 3,111, 3,078, and 3,083 scans in the easy, medium, and hard tiers, respectively, each covering 99 CAD geometries. On the medium common subset, PTP achieves a P-AUROC of 0.7958 with a CAD-clustered bootstrap 95% confidence interval of [0.7781, 0.8135], while Simple3D achieves 0.7252 with an interval of [0.7040, 0.7467]. The shared supplementary material provides category and mechanism distributions, severity distributions, the complete asset list, and valid-sample, exclusion, and failure counts for each method, making dataset composition, aggregation denominators, and uncertainty directly inspectable.

Point 6: Thickness statistics and reproducible annotation

We thank the reviewer for identifying the inconsistency in the thickness statistics and requesting clearer annotation rules. Our audit found that the values of 0.028 in Table 2 and 0.057 in Table 10 lack records that uniquely identify their input samples and processing stages. We will therefore remove these unverifiable values from the revision and explicitly distinguish high-density scan calibration from diagnostics on downsampled detector inputs. For the latter, we added reproducible statistics on the same set of 32 normal scans spanning 8 CAD geometries. Each cloud contains 4,096 points. We estimate the local standard deviation along the surface normal using 24-point neighborhoods, take its median over the flattest 20% of regions, and divide by the cloud’s median nearest-neighbor distance. The median local thickness ratios are 0.193160 for the full scanning pipeline and 0.193223 with surface reconstruction disabled. The input manifest, random seed, and computation code accompany the supplementary material. For annotation, the geometric anomaly score is the absolute signed distance from the generated surface to the normal reference surface in normalized coordinates, with a shared threshold of τ=0.002; a positive binary label additionally requires an attributable defect-event contribution at that location. The nominal severity targets are 0.006–0.06 for maximum deviation and 0.0005–0.30 for anomalous fraction. We record each mechanism’s fraction definition, parameters, retry count, and acceptance outcome for every sample. Controller-stage measurements and final geometric annotation statistics are reported separately so that the target criteria and realized output distributions can be independently verified.

Reviewer_d7s5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a benchmark for point-cloud anomaly detection that connects physical defect formation with simulated 3D scanning. The synthesis pipeline covers seventeen mechanisms grouped into deformation, material removal, and material addition. It combines material-dependent deformation, interface evolution, and deposition models, and can propagate geometry and material state across successive damage events. A triangulation-based scanner then generates partial point clouds with configurable occlusion, noise, and acquisition artifacts. The benchmark evaluates 3,213 anomalous and 2,000 normal instances based on 100 industrial geometries. Eight established point-cloud method configurations and a point-to-plane baseline are tested across three scanning conditions and up to sixteen normal references. Table 4 reports lower localization scores than those commonly reported on other benchmarks, although cross-dataset protocols differ. A separate experiment in Table 6 rescans 79 native Anomaly-ShapeNet defects and shows that the effect of simulated scanning varies across eleven tested methods. The appendix provides numerical checks, scanner calibration details, evaluation denominators, and reproducibility information.

Critical strengths

The benchmark brings together defect formation mechanisms that are rarely assessed within a single point-cloud evaluation framework. Seventeen deformation, removal, and addition mechanisms support both local damage and larger structural changes, with state transfer between successive events. Figures 3 and 5 illustrate the range of generated morphologies. Appendix A.7 provides targeted numerical and mechanism checks, including the pitting interface comparison against a finer solution. This is a substantive engineering and dataset contribution. The experimental design explicitly treats acquisition conditions as a source of variation rather than assuming complete, clean point clouds. The three scanning levels, multiple reference counts, and paired native-defect experiment make it possible to study distinct evaluation regimes. For example, Table 6 reports a 0.058 decrease in P-AUROC for Reg3D and a 0.059 increase for PatchCore-FPFH after scanning the same native defects. Appendix C also documents sample coverage and method-specific configurations, helping readers interpret the findings carefully.

Critical weaknesses

Evidence that the generated defects and scans reproduce real industrial inspection conditions remains limited. The checks in Appendix A.7 establish numerical consistency for selected mechanisms, and Table 2 compares three aggregate point-cloud statistics with released real scans. Neither directly validates the distribution of simulated defect shapes, material responses, or detector behavior against measured defects. Consequently, the paper establishes a plausible and useful simulation benchmark more strongly than it establishes real-world fidelity or transfer. The main metric captures a narrower task than the full industrial anomaly detection setting motivating the benchmark. Section 4.1 and Appendix C.2 compute P-AUROC only for scorable scans with both point classes. Fully unobserved defects and inference failures are omitted from reported means, and artifact points are excluded from evaluation even though they remain in model inputs. Normal test scans are also not assessed for false alarms by this metric. The authors disclose these choices, but method-dependent denominators in Table 11 and the omitted failure modes limit how strongly one can interpret the rankings as measures of practical inspection robustness.

Questions for authors

Is there quantitative evidence comparing PhysDefect outputs with measured industrial defects or real scanner observations beyond the three calibration statistics in Table 2? Even a limited matched case study of defect geometry, material response, or downstream detector behavior would help establish the external validity of the benchmark. How would the main comparisons change if evaluation also reported the frequency of fully unobservable defects, inference failures, false alarms on normal scans, and responses to acquisition artifacts? Since these cases are especially relevant to the proposed scanning component, their inclusion would materially affect interpretation of benchmark difficulty and method rankings. Can the authors isolate the effects of visibility, measurement noise, and surface processing on performance? Table 3 changes several acquisition factors together, so a controlled one-factor-at-a-time result or existing supporting analysis would clarify what drives the method-dependent shifts observed in Tables 4 and 6.

3 comments
AuthorsOpenReview ↗

Response to Reviewer d7s5

Point 1: Evidence from measured defects and real-data evaluation

We thank the reviewer for asking for direct measurements of defect geometry and a downstream evaluation on real scans. We addressed these questions with impact fitting and a separate transfer experiment. We impacted a 100 × 100 × 0.1 mm copper foil sheet and a 40 × 40 × 40 mm industrial aluminum heat sink with a 28 mm diameter solid steel ball and captured their surfaces with a FreeScan Combo 3D scanner. The foil rested on the laboratory floor with its edges free. The heat sink rested on its side with the base vertical and the fins horizontal, and the ball fell approximately 1.5 m onto one corner. We registered the measured points to the reference geometries, fixed the candidate impact locations from the scans, and compared PhysDefect with AF3AD, NormAS, PA3AD, and GLFM. Within each experiment, each method received the same number of candidate evaluations. Parameters were selected on the fitting points, and we report point-to-surface RMSE on spatially held-out scan points. With 64 additional evaluations per method, PhysDefect reduced held-out RMSE across the damaged corner from 0.5813 mm for the undeformed CAD to 0.3075 mm, a 47.1% reduction. The residual was 0.769% of the heat sink’s side length. NormAS gave the closest competing fit in both settings, at 0.4247 mm with unconstrained displacement and 0.5121 mm with total vertex displacement matched to PhysDefect. The photograph and same-view scan overlays show the residual bending and clustering of neighboring fins (S1–S3). For each copper-foil region, we compared the fits obtained after 16 evaluations per method. In two illustrated regions with neighboring dents, PhysDefect achieved 0.07109 and 0.09735 mm, compared with 0.13190 and 0.10775 mm for GLFM, the closest baseline in both. In the first region, PhysDefect reproduced the two minima and the height change between them. In two single-dent examples, its errors were 0.03552 and 0.01838 mm, compared with 0.06953 and 0.02311 mm for PA3AD. F1–F4 show the selected surface fits and error curves, and F5 gives the full regional comparison. For the downstream test, we generated 192 point-labeled training anomalies from 48 normal Real3D-AD templates using four material-removal mechanisms (uniform corrosion, pitting, particle erosion, and wear). The normal-replay control replaced each synthetic sample with its corresponding normal template. We held the DGCNN architecture, training budget, and three random seeds fixed and evaluated on the same 1,206 real scans. Real anomalous test scans were excluded from parameter selection and early stopping. Micro-averaged P-AUROC improved under each seed. Its mean rose from 0.5003 ± 0.0245 to 0.5995 ± 0.0149, a gain of 9.92 percentage points over normal replay (mean ± sample standard deviation across seeds, S4).

Point 2: Coverage, failures, false alarms, artifacts, and rankings

We thank the reviewer for highlighting the need to account for unobservable defects, inference failures, false alarms on normal scans, and acquisition artifacts. We added coverage and failure statistics by mechanism and scanning tier, false-alarm evaluation on normal scans, and localization comparisons on the common scorable subset. Fully unobservable defects account for 22, 29, and 45 cases (0.68%, 0.90%, and 1.40%) among the 3,213 anomalous samples in the easy, medium, and hard tiers, respectively. Among the 3,613 test scans in the hard tier, PTP and Simple3D with one normal reference have 38 and 52 inference failures, respectively. We compute each object score as the 99th percentile of anomaly scores over all input points, incorporating artifacts into alarm evaluation, and separately report artifact scores. For example, in the medium tier, PTP assigns artifacts a mean score of 0.0533, compared with 0.0031 for normal material points in the same scans, showing that artifacts receive elevated anomaly responses. We set the alarm threshold to the 95th percentile of scores from 80 normal scans spanning 20 CAD models and tested on the remaining 80 models. In the medium tier, PTP flagged 15 of 320 normal scans (4.69%) and detected 130 of 2,576 anomalous scans (5.05%). The 0.36-percentage-point difference had a CAD-clustered bootstrap 95% interval of −1.50 to 2.01 percentage points. Method rankings also depend on the evaluation objective: on the 3,078 common scorable anomalous scans in the medium tier, PTP achieves a P-AUROC of 0.7958, exceeding Simple3D’s 0.7252; their O-AUROCs, incorporating normal objects, are 0.6483 and 0.6689, respectively, reversing the ranking. We therefore report visible-defect localization, object-level alarms, observation coverage, and inference success separately to assess both localization quality and practical detection performance. S8–S9 and the accompanying tables provide complete statistics, valid sample counts, and confidence intervals for all methods.

AuthorsOpenReview ↗

Response to Reviewer d7s5 (continue)

Point 3: Individual scanning components

We thank the reviewer for highlighting the need to isolate the effects of individual scanning components. We fixed the defect geometry for 24 samples spanning 8 CAD geometries and 3 defect mechanisms, changing only one scanning setting at a time: disabling measurement noise, disabling acquisition artifacts, disabling surface reconstruction, or halving the viewpoints along the original scanning path. We evaluated PTP, PatchCore-FPFH, and Simple3D. Relative to the full scanning pipeline, disabling measurement noise changes their P-AUROCs by −0.40, +1.53, and +2.68 percentage points, respectively; disabling surface reconstruction changes them by −0.26, +1.65, and +2.18 percentage points. Halving the viewpoints increases PatchCore-FPFH’s P-AUROC by 3.43 percentage points, while PTP changes by +0.03 percentage points; disabling artifacts decreases PatchCore-FPFH’s P-AUROC by 1.77 percentage points. The CAD-clustered bootstrap 95% confidence intervals for all 12 differences include zero, so these results do not establish a consistent directional effect of any component across all algorithms. Nevertheless, this controlled experiment quantifies the sensitivity of different algorithms to individual scanning perturbations, informing algorithm selection and validation according to the noise, artifacts, and observation coverage encountered in industrial settings. S7 provides point-cloud examples of the same defect under each setting and a complete results plot with confidence intervals.

Thank you for responding to my queries. I suggest including the analysis of isolated factors in the main paper and I maintain my original rating of weak accept.

Reviewer_9kaEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces PhysDefect, a physics-driven data synthesis and evaluation framework for industrial 3D anomaly detection.Unlike conventional approaches that directly apply analytical deformations to point clouds, PhysDefect simulates defect formation using the Moving Least Squares Material Point Method (MLS-MPM), level-set methods, transport equations, and contact models. It covers 17 defect mechanisms organized into three categories: material deformation, material removal, and material addition. The framework also supports sequential damage events by transferring geometry and material states between successive processes.Furthermore, the authors develop a simulated scanner based on projector–camera triangulation, explicitly modeling occlusion, missing observations, measurement noise, and scanning artifacts. Three scanning difficulty levels—easy, medium, and hard—are introduced to evaluate detection methods under different observation conditions. The benchmark consists of 100 industrial geometries, 3,213 anomalous instances, and 2,000 normal instances. The authors evaluate eight existing methods and a geometric baseline. Additionally, they conduct a paired scanning experiment on 79 native defects from Anomaly-ShapeNet, showing that simulated scanning can affect different detection methods in opposite directions, improving the performance of some while degrading that of others.

Critical strengths
  1. The paper presents a unified framework covering 17 defect mechanisms across material deformation, removal, and addition. Unlike conventional geometric perturbation methods, it incorporates material-dependent responses, global deformation, and sequential damage with state transfer between events. This is a meaningful contribution because it extends the range of defect types and formation processes represented in existing 3D anomaly detection benchmarks.

2.The paper integrates a simulated scanning pipeline that accounts for occlusion, incomplete observations, measurement noise, and scanning artifacts. The paired experiment on 79 native Anomaly-ShapeNet defects further shows that simulated scanning affects different detection methods in different directions. This is valuable because it highlights the importance of acquisition conditions in benchmark evaluation and provides a useful basis for studying the robustness of 3D anomaly detection methods.

Critical weaknesses
  1. The realism and representativeness of the synthetic defects are not sufficiently validated.Although the paper uses physics-based models to generate different types of defects and provides some numerical validation, there is still a lack of direct comparison with real industrial defects. It is therefore unclear to what extent these synthetic anomalies reflect the actual shapes and distributions of defects found in practice. This makes it difficult to assess how realistic the proposed benchmark is and how well it could generalize to real industrial settings.

  2. The evaluation metrics do not fully reflect practical anomaly detection performance.The paper mainly relies on point-level P-AUROC, which is only computed for scans containing both normal and anomalous points. While this is reasonable for evaluating the localization of visible defects, it does not capture some important aspects of practical anomaly detection. For example, although each scanning level includes 400 normal test scans, these samples are not included in the main P-AUROC evaluation. As a result, a method could achieve a high localization score while producing many false positives on normal objects, which would not be reflected in the reported results.

  3. The ablation study does not sufficiently separate the effects of physics-based defect synthesis and scanning simulation.Although the paper evaluates different scanning difficulty levels and conducts paired experiments on Anomaly-ShapeNet, it is still difficult to tell how much each component contributes to the overall detection difficulty. In particular, a controlled comparison between conventional geometric synthesis and the proposed physics-based synthesis on the same geometries, with similar defect severity, would be helpful. Without such experiments, it remains unclear whether the observed performance drops are mainly caused by more complex defect shapes or by the degradation introduced during simulated scanning.

Questions for authors
  1. How well do the synthesized defects represent real industrial defects? Have the authors conducted any direct comparisons with real defects, either in terms of defect morphology or statistical distributions? The numerical validation is useful, but it would be helpful to understand whether the generated defects are also representative of those encountered in practice.

  2. Could the authors provide additional evaluation results on normal test samples? Since the main P-AUROC metric excludes scans containing only normal points, it would be useful to report false positive rates or image-level anomaly detection performance. This would help assess whether the evaluated methods can reliably distinguish normal objects from defective ones.

3.Could the authors provide a more controlled ablation study to separate the contributions of physics-based synthesis and scanning simulation? For example, comparing conventional geometric synthesis and physics-based synthesis on the same geometries with similar defect severity, under both clean sampling and simulated scanning, would help clarify how much of the detection difficulty comes from defect morphology versus scanning degradation.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 9kaE

Point 1: Validation against real defects

We thank the reviewer for asking how closely the synthetic defect shapes match measured damage. We impacted a 100 × 100 × 0.1 mm copper foil sheet and a 40 × 40 × 40 mm industrial aluminum heat sink with a 28 mm diameter solid steel ball and captured their surfaces with a FreeScan Combo 3D scanner. The foil rested on the laboratory floor with its edges free. The heat sink rested on its side with the base vertical and the fins horizontal, and the ball fell approximately 1.5 m onto one corner. We registered the measured points to the reference geometries, fixed the candidate impact locations from the scans, and compared PhysDefect with AF3AD, NormAS, PA3AD, and GLFM. Within each experiment, each method received the same number of candidate evaluations. Parameters were selected on the fitting points, and we report point-to-surface RMSE on spatially held-out scan points. With 64 additional evaluations per method, PhysDefect reduced held-out RMSE across the damaged corner from 0.5813 mm for the undeformed CAD to 0.3075 mm, a 47.1% reduction. The residual was 0.769% of the heat sink’s side length. NormAS gave the closest competing fit in both settings, at 0.4247 mm with unconstrained displacement and 0.5121 mm with total vertex displacement matched to PhysDefect. The photograph and same-view scan overlays show the residual bending and clustering of neighboring fins (S1–S3). For each copper-foil region, we compared the fits obtained after 16 evaluations per method. In two illustrated regions with neighboring dents, PhysDefect achieved 0.07109 and 0.09735 mm, compared with 0.13190 and 0.10775 mm for GLFM, the closest baseline in both. In the first region, PhysDefect reproduced the two minima and the height change between them. In two single-dent examples, its errors were 0.03552 and 0.01838 mm, compared with 0.06953 and 0.02311 mm for PA3AD. F1–F4 show the selected surface fits and error curves, and F5 gives the full regional comparison. The local foil and bent-fin cases provide direct comparisons of measured morphology across two distinct structural responses to impact.

Point 2: Detection metrics and false alarms on normal objects

We thank the reviewer for highlighting the importance of false alarms on normal objects in practical inspection. The original P-AUROC measures defect localization within a scan; distinguishing normal from defective objects requires a separate evaluation. We added O-AUROC using the 400 normal scans in each scanning tier, defining the object score as the 99th percentile of anomaly scores over all input points, including acquisition artifacts. For fixed-threshold evaluation, we set the threshold to the 95th percentile of object scores from 80 normal scans spanning 20 CAD geometries, then evaluated normal and anomalous scans from the remaining 80 geometries. In the medium tier with one normal reference, PTP and Simple3D achieve O-AUROCs of 0.6483 and 0.6689, respectively, whereas their P-AUROCs on the common subset of 3,078 scorable anomalous scans are 0.7958 and 0.7252, reversing their relative ranking. At the threshold calibrated on normal scans, PTP flagged 15 of 320 normal scans (4.69%) and detected 130 of 2,576 anomalous scans (5.05%). The detection rate exceeded the false-positive rate by 0.36 percentage points at this threshold (CAD-clustered bootstrap 95% interval, −1.50 to 2.01 percentage points). These results demonstrate that localizing observed defects and identifying defective objects are distinct evaluation objectives. We therefore report O-AUROC, normal-object false-positive rates, and the corresponding detection rates alongside P-AUROC. Complete results across methods and scanning tiers are provided in S8–S9 and the accompanying tables.

AuthorsOpenReview ↗

Response to Reviewer 9kaE (continue)

Point 3: Separating synthesis and scanning effects

We thank the reviewer for highlighting the need to separate the effects of defect synthesis and scanning simulation. We added a synthesis-by-observation comparison with four conditions: PhysDefect and AF3AD defects generated on the same CAD geometries, each evaluated under ideal observation and medium simulated scanning. The experiment covers 99 CAD geometries, 17 defect mechanisms, 3,053 matched pairs, and 11 detector configurations. To control severity, we matched total vertex displacement for 1,300 topology-preserving pairs and a surface-deviation proxy for the remaining 1,753 material-addition or removal pairs, with relative budget differences within 10%. For impact defects, PhysDefect yields P-AUROC scores averaging 6.27 and 5.80 percentage points below AF3AD under ideal observation and simulated scanning, respectively, with 10/11 detector configurations showing this direction in both conditions. These averages are computed first across CAD geometries and then across detector configurations. After additionally constraining the visible anomalous-point fraction and defect depth, 9/11 configurations retain this direction under both observation conditions. Thus, impact morphology already introduces additional localization difficulty under ideal observation, and this difference persists after simulated scanning. The effect varies by mechanism: for bending defects, P-AUROC instead increases by an average of 1.55 and 0.71 percentage points under the two observation conditions. This experiment separates the comparison of defect morphology from scanning conditions and shows that the detection challenges introduced by physical synthesis depend on the defect mechanism. S5–S6 and the accompanying tables provide examples from identical viewpoints across the four conditions, results by mechanism, and CAD-clustered confidence intervals.

Reviewer_9kaEOpenReview ↗

Address my concern

Thanks for the response. The additional experiments have addressed my second and third concerns. However, my main concern about the realism of synthetic defects remains, as the physical validation only covers two impact experiments rather than all 17 defect mechanisms. Overall, I think the paper has improved, so I will raise my score from 2 to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.