Beyond Density: Boundary Detection via Local Mean Curvature

Boundary detection in multivariate datasets remains a fundamental challenge in unsupervised learning. Existing methods rely predominantly on density estimation, which conflates flat low-density regions with genuinely curved geometric transitions and degrades on nonlinear manifolds, anisotropic distributions, and high-dimensional sparse data. We propose the mean curvature boundary points (MCBP) method, a geometry-driven framework that identifies boundary samples by estimating pointwise mean curvature via a discrete approximation of the shape operator on local $k$-nearest neighbor patches, without requiring explicit manifold parametrization. We establish three theoretical pillars: (i) a variational characterization showing that high-curvature regions coincide with geometric boundaries via the first variation of the Hausdorff measure; (ii) a connection to the Laplace–Beltrami operator that grounds the MCBP within spectral manifold theory; and (iii) a geometric-statistical duality proving that the curvature score jointly amplifies manifold bending and local sparsity, yielding a richer boundary signal than density alone. We further prove pointwise statistical consistency of the discrete estimator. The curvature signal induces a principled data decomposition into smooth interior and high-curvature boundary subsets, acting as a nonlinear geometric filter. Experiments on more than 40 real-world datasets show consistent improvements in terms of internal clustering metrics, with average silhouette coefficient gains of $49\%$ for $k$-means++ and $58\%$ for HDBSCAN, with curvature-informed centroid initialization outperforming standard $k$-means++ seeding.