acceptodds
Under review as a conference paper at ICLR 2027

Iso-Depth Learning for Anomaly Detection

Abstract

We consider unsupervised anomaly detection for data that lie near a low-dimensional manifold. Our interest is in detecting points that sit far away from the center of the data cloud but remain close to the manifold. Statistical depth in an ambient space may be inadequate for detecting such points, because Euclidean distance can make points that are far apart along the manifold appear close. Local density scores may also fail to detect such points, because they can have sufficiently many nearby neighbors despite being far from the global center. We propose an Iso-depth framework for anomaly detection that first estimates the intrinsic geometry of the data via ISOMAP and then computes statistical depth in the resulting Euclidean embedding. Iso-depth measures global centrality intrinsically. To facilitate efficient prediction, we train a neural network to approximate the depth function, allowing new observations to be scored with minimal computational cost. We derive an error bound with three components: depth estimation, the ISOMAP embedding, and neural approximation. For halfspace depth, we show how root mean square coordinate error transfers to depth error. We evaluate Iso-depth in controlled synthetic experiments that separate depth estimation, embedding, and out-of-sample prediction errors, and in a real-data application that examines the resulting anomaly rankings.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Isomap, Data depth, Deep learning, Anomaly detection, VC dimension, Convergence rate

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jxSLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies on-manifold peripheral anomalies: observations that remain close to a low-dimensional manifold but are peripheral relative to the distribution’s intrinsic geometry. It proposes Iso-depth, a three-stage framework that estimates intrinsic coordinates using ISOMAP, computes statistical depth in the resulting embedding, and trains a neural network to predict these depth values directly from the original observations.

Critical strengths
  • A coherent geometric formulation supported by a useful stability analysis. The paper identifies an interpretable detection objective: global peripherality measured along the manifold, rather than distance from the manifold or local neighborhood isolation.

  • The controlled experiments are informative and address several sources of error separately. The synthetic evaluation is more informative than a single aggregate detection table. Figure 1 on page 8 separates sampling error from geometry-induced depth error and compares neural predictors trained on estimated versus population depth targets.

Critical weaknesses
  • The experiments do not sufficiently isolate the benefit of statistical depth over simpler intrinsic-centrality scores. The paper’s main distinction is between global intrinsic peripherality and local isolation. However, the most informative matched-embedding comparator is LOF, which addresses the latter objective. The reported baseline set does not include a simple global-centrality score on the same recovered coordinates, such as distance from an estimated center, or a score based on average graph distance. Consequently, it remains unclear how much of the gain requires statistical depth specifically, rather than unfolding the data and applying any suitable global-peripherality measure.

  • The real-data evidence does not convincingly validate the proposed intrinsic-geometric mechanism The paper transparently states that Cardio was selected after exploratory comparisons among candidate datasets. It also applies a dataset-specific adjustment to the fetal-movement feature’s scale, and explicitly states that this adjustment is not proposed as a general preprocessing rule. Thus, the only real-data case is an exploratory, specially configured example rather than independent validation of a reusable procedure.

  • The effectiveness of proposed method on real datasets compared with other recent baselines reamins unknown. The authors only select one real dataset from ADBench for case study, but its performance on other 46 datasets reamins unknown.

Questions for authors

See Weaknesses.

1 comment

We appreciate the reviewer for asking us to isolate the contribution of statistical depth, clarify what the Cardio case establishes, and assess Iso-Depth on a broader set of real datasets.

  1. Answer to Q1

Tukey depth is the least probability mass in any halfspace containing a point. A point midway along a long boundary can have low depth even when its distance from the center and average distance to other observations are modest. Distance from an estimated center measures radial separation. Mahalanobis distance adjusts for scatter but retains ellipsoidal contours. Mean graph distance averages shortest-path distances to all observations. Since ISOMAP does not impose elliptical contours, these scores can miss boundary points that depth detects.

Across 100 Swiss-roll seeds, mean AP was 0.3402 for depth, 0.1961 for Mahalanobis distance, and 0.1671 for distance from the sample mean on the same recovered coordinates. Mean graph distance achieved 0.1739. The paired AP gain over Mahalanobis distance was 0.1441 (exploratory, unadjusted 95% interval [0.1303, 0.1579]). Among each method's top 30, pooled recall along the middle of the long edges was 20.7% for depth versus 0.4% for Mahalanobis distance, with fractional weighting of cutoff ties.

  1. Answer to Q2

Prior studies reconstructed phase-space attractors from fetal heart rate recordings [1] and estimated attractor manifolds from cardiotocography signals [2]. This motivates examining geometric structure in Cardio's derived features.

We scaled features to make their different units comparable. Under ordinary IQR scaling, fetal movement (FM) accounted for 94.9% of total pairwise squared distance because its IQR was small relative to its spread. We imposed a lower bound on its scaling denominator so other features could contribute to graph construction. This adjustment used feature values alone, without pathology labels, and retained all observations and values.

At a fixed ISOMAP dimension of two, Iso-Depth achieved AP of 0.538 under standard scaling, 0.558 under ordinary IQR scaling, and 0.596 after the FM adjustment. Under ordinary IQR scaling, PCA-Depth achieved AP of 0.259 and LOF on the same embedding achieved a best AP of 0.304 across four neighborhood sizes. After adjustment, AP was 0.245 for PCA-Depth and 0.167 for the reported embedded LOF baseline. The advantage over these comparators was already present before adjustment. Among the top 176 ranks, Iso-Depth identified 98 pathological recordings before adjustment and 104 afterwards, compared with 93 for ECOD.

Cardio is an exploratory demonstration. Q3 discusses datasets where geometric extremes can be defined independently of the embedding for a more direct assessment.

References

[1] Chaffin, D. G., Goldberg, C. C., and Reed, K. L. (1991). The dimension of chaos in the fetal heart rate. American Journal of Obstetrics and Gynecology, 165(5), 1425–1429. DOI [2] Feng, G., Heiselman, C., Quirk, J. G., and Djurić, P. M. (2023). Cardiotocography analysis by empirical dynamic modeling and Gaussian processes. Frontiers in Bioengineering and Biotechnology, 10, 1057807. DOI

Answer to Q3.

Iso-Depth targets geometric extremes, which can have close neighbors and escape LOF. Its use requires a meaningful reference population, a defensible metric, and evidence linking anomalies to structural extremes.

Across 20 ADBench datasets and 10 seeds per dataset, provisional transductive average AUROC/AP were 0.6734/0.3005 for Iso-Depth, 0.6845/0.2915 for ambient depth, 0.6422/0.2694 for MDS depth, 0.7505/0.4580 for Isolation Forest, 0.6064/0.2794 for ambient LOF, and 0.6077/0.3108 for OCSVM. Iso-Depth exceeded MDS depth and LOF on both average metrics.

We selected COIL-100 [1] because controlled rotation provides a known circular parameter. It has 72 views per object at 5-degree intervals. Removing an arc creates two endpoints. With no original anomaly labels, we marked the three retained views nearest each endpoint as positives using rotation angles, independently of detector scores. Mean AUROC was 0.777 for Iso-Depth and 0.723 for same-embedding LOF. This tests constructed boundary retrieval, whose practical relevance requires separate validation.

On Extended Yale B [2,3], we defined targets by a pose-lighting combination. Iso-Depth had lower AUROC than MDS depth (0.488 versus 0.560). Known acquisition parameters alone therefore do not guarantee better detection.

References [1] Nene, Nayar, Murase. Columbia Object Image Library (COIL-100). CUCS-006-96, Columbia University, 1996. [2] Georghiades, Belhumeur, Kriegman. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE TPAMI 23:643-660, 2001. doi:10.1109/34.927464. [3] Lee, Ho, Kriegman. Acquiring linear subspaces for face recognition under variable lighting. IEEE TPAMI 27:684-698, 2005. doi:10.1109/TPAMI.2005.92.

Reviewer_5RgHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies unsupervised anomaly ranking for observations that remain on or near a low-dimensional manifold but are globally peripheral in its intrinsic geometry. Iso-Depth uses ISOMAP to estimate intrinsic coordinates, computes statistical depth in that representation, and learns a neural approximation from the original features for out-of-sample scoring. The analysis separates empirical-depth estimation, geometry recovery, and neural prediction error, including a transfer bound from RMS coordinate perturbations to halfspace-depth error under stated assumptions. Experiments examine these error sources, detect predefined peripheral anomalies on a gapped circle and Swiss roll, and evaluate rankings and held-out predictions on Cardiotocography.

Critical strengths
  1. A useful geometric perspective on anomaly ranking. Measuring centrality after recovering intrinsic geometry is a coherent way to target peripheral observations that ambient distances or local-density scores may overlook. The distinction between departure from the manifold and peripherality along it is informative, and the comparison with LOF on the same embedding helps isolate the role of the depth score.

  2. Analysis and experiments that separate sources of error. The RMS-coordinate-error to halfspace-depth-error analysis provides a meaningful link between geometry recovery and score accuracy. The controlled experiments distinguish sampling and geometry errors, while paired neural fits using estimated versus population depth targets examine their effect on independent-test prediction. Learning the depth function also offers a practical route to scoring new observations without recomputing the full embedding and depth procedure. These are valuable contributions, subject to the stated geometric and approximation assumptions.

Informativeness rationale: The intrinsic-centrality perspective and the separation of depth-estimation and geometry errors are useful insights. The limited application evidence constrains how much can be concluded about practical anomaly detection.

Critical weaknesses
  1. Limited real-data evidence for the motivating mechanism. The synthetic experiments define the outermost 3% of latent endpoint/boundary regions as anomalies, supporting the intended setting but not independently establishing its relevance to real anomalies. Cardio provides external pathology labels, and Figure 2 places many additional detections near branch ends. However, it does not jointly establish that these observations remain close to the manifold, are locally non-sparse, and have low intrinsic depth. Such diagnostics would strengthen the link between the observed gains and the motivation.

  2. Limited real-application validation. Only one real public dataset is formally reported, selected after exploratory comparisons as acknowledged by the authors. Across 20 held-out Cardio splits, Iso-Depth-NN has lower mean AP/AUROC (0.577/0.916) than ECOD (0.598/0.938) and Isolation Forest (0.589/0.932), but higher top-53 precision (0.566 versus 0.547 and 0.511). These gains are encouraging, yet additional compatible public applications are needed to establish stability and practical utility. Despite the valuable geometric idea and controlled experiments, the current empirical coverage is insufficient to support acceptance.

  3. Scope under multiple clusters and neighborhood scales. An observation can be unusual within a cluster or hierarchy without being globally peripheral. The method does not explicitly condition depth on such structure, and neighborhood scale affects the recovered geometry. Existing radius diagnostics are useful, but further validation is needed to clarify when global depth is appropriate. This concerns the demonstrated scope, not an ill-defined depth function, a specific proof error, or evidence of failure at the stated peripheral-anomaly task.

  4. Presentation clarity (minor). A direct comparison of a flat disk and its bent-manifold counterpart would make the motivation clearer; Figure 1(a) shows only the rolled-disk view. In Figure 1(b), the black total-error curve EDE_D and square markers largely obscure the nearly coincident blue sampling-error curve ET1E_{T_1}. The overlap indicates similar error values but hinders comparison. Different line styles, hollow markers, a revised drawing order, or a zoomed inset would improve readability.

Questions for authors
  1. Could you extend the Figure 2 analysis with joint diagnostics of local density, intrinsic depth, proximity to the estimated manifold, and independent pathology labels? In particular, do the additional true anomalies detected by Iso-Depth relative to local-density baselines exhibit the proposed mechanism, and how do they differ from normal observations?

  2. Is there evidence from additional public datasets with a plausible intrinsic-periphery interpretation that the benefits persist? Please clarify the criteria used to choose applicable scenarios and report both ranking metrics and precision at practically motivated cutoffs, including limitations or negative results.

  3. For data with multiple clusters or hierarchical structure, how should practitioners determine whether a global depth score is appropriate? Could you clarify or diagnose how neighborhood scale affects the ranking of observations that are atypical within a group but not globally peripheral? These questions concern the supported scope of the method and could materially affect my assessment.

1 comment

We appreciate your comments on how to interpret an Iso-Depth alert. Your distinction between an observation at the edge of the full sample and one that is unusual only within its group helped us clarify the scope of the method.

Answer to Q1. With 176 alerts per method, we examined observations that Iso-Depth flagged and LOF missed. For ambient LOF, 58 of 85 pathologic cases met the joint criterion below, versus 25 of 55 normal cases. For embedded LOF, the counts were 53 of 76 and 23 of 53.

We used midrank percentiles relative to all 1,655 normal observations. The criterion required intrinsic depth at or below the 10th percentile and both LOF score and residual at or below the 90th percentile. The residual was the distance to a plane fitted to the 20 nearest other observations using all 21 preprocessed features.

The residual accounted for most of the difference between these groups. All selected observations had low depth, and selecting cases that LOF missed favored lower LOF scores. In the full cohort, the residual comparison reversed: 67.0% of pathologic observations met the cutoff, versus 90.0% of normal observations. These results support the proposed mechanism for these additional detections. The residual measures local plane fit and only indirectly indicates proximity to the manifold.

Answer to Q2. We target anomalies at the extremes of an underlying structure. An application needs evidence for that structure and a reason to associate its extremes with anomalies.

Our search during the response period identified few public datasets with both clearly documented geometry and independent anomaly labels. We therefore used controlled data to evaluate detection of specified geometric extremes. This tests recovery of geometric extremes. Its relevance to practical anomaly detection requires separate validation.

We selected COIL-100 [1] because its acquisition provides a known one-dimensional variable: object rotation angle. It contains 100 objects, each photographed at 72 angles in 5-degree increments with a fixed camera and background. The rotation parameter lies on a circle. Removing a contiguous angular arc leaves an interval with two known endpoints. The endpoints provide a geometric reference independent of detector scores. Image distances, however, need not preserve angular distances.

COIL-100 has no original anomaly labels. We designated the three retained views nearest each endpoint as positives, using their recorded rotation angles. These labels identify the boundary views created by removing the arc, without implying that the objects are abnormal or the views harmful. For retrieving these views, mean AUROC was 0.777 for Iso-Depth and 0.723 for LOF on the same embedding.

On Extended Yale B [2,3], the target was a specific combination of pose and lighting. Iso-Depth had lower AUROC than MDS depth (0.488 versus 0.560), showing that controlled acquisition alone does not ensure that the target is peripheral in the learned geometry.

COIL-100 evaluates recovery of constructed geometric boundaries while Cardio example provides a separate evaluation against clinical anomalies.

Answer to Q3.

Iso-Depth computes halfspace depth in the pooled embedding. A point near the manifold's geometric center can have low depth if a halfspace containing its embedded position holds few observations. A point unusual within its group can be central in the pooled distribution. Conversely, a small normal group at the pooled sample's edge can receive low depth scores.

The ISOMAP radius can change ranks through the embedding, but depth always uses the full sample. Table 8 reports radius sensitivity over 100 seeds. At multipliers 1.00, 1.10, and 1.25, mean average precision was 0.978, 0.982, and 0.983 on the gapped circle, versus 0.290, 0.340, and 0.399 on the Swiss roll. These tests concerned boundary anomalies. How radius affects the rank of a point that is atypical only within its group remains open.

Local depth [4] evaluates centrality within a neighborhood in multimodal or nonconvex distributions. One extension would identify groups by k-means or kernel k-means and apply Iso-Depth within each cluster. Each observation would then be ranked against its estimated group. The usefulness of this ranking would depend on how well the clusters recover the groups of interest.

References [1] Nene, Nayar, and Murase. Columbia Object Image Library (COIL-100). Columbia University, Technical Report CUCS-006-96, 1996. [2] Georghiades, Belhumeur, and Kriegman. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE TPAMI, 23(6):643-660, 2001. doi:10.1109/34.927464. [3] Lee, Ho, and Kriegman. Acquiring linear subspaces for face recognition under variable lighting. IEEE TPAMI, 27(5):684-698, 2005. doi:10.1109/TPAMI.2005.92. [4] Paindaveine and Van Bever. From depth to local depth: A focus on centrality. JASA, 108(503):1105-1119, 2013. doi:10.1080/01621459.2013.813390.

Reviewer_v8LYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Iso-depth, an anomaly detection method for detecting peripheral observations on nonlinear manifolds. It employs ISOMAP to obtain low dimensional coordinates and then computes halfspace depth to assign anomaly scores. A neural network is trained to predict the anomaly scores for new observations without having to re-run the ISOMAP. The paper provides theoretical convergence bounds on the recovered coordinates, the estimated depth values and the out-of-sample predictions under a number of assumptions. The method is evaluated on a number of synthetic manifold datasets as well as on the real world Cardiotocography dataset and is compared against a range of other existing anomaly detection methods.

Critical strengths

The paper derives convergence bounds by separating errors from ISOMAP coordinate recovery, halfspace-depth estimation, and neural-network prediction. The analysis is useful, although its guarantees rely on restrictive geometric assumptions and an unverified neural-network approximation rate.

Critical weaknesses
  1. Insufficient justification for the proposed approach and missing related work

The paper does not adequately justify why ISOMAP followed by halfspace depth is needed for the anomaly detection problem considered. Once a neighborhood graph is constructed, one could directly measure global peripherality using average shortest-path distance (graph distance), without requiring either ISOMAP or statistical depth. Other graph-based embeddings, such as Laplacian Eigenmaps (Belkin and Niyogi, 2003) and Diffusion Maps (Coifman and Lafon, 2006), could also be combined with halfspace depth. Moreover, kernelized spatial depth (Chen et al., 2009) is a closely related approach that uses kernel-based similarities to perform nonlinear depth-based outlier detection.

The paper does not adequately discuss or experimentally compare these alternatives. Without such comparisons, it is difficult to determine whether the proposed combination offers a meaningful advantage over existing approaches. This substantially weakens the paper's methodological contribution.

  1. Limited novelty of the main theoretical contribution

Theorem 3 establishes a bound relating the RMS error of the recovered coordinates to the error in halfspace depth: Op(n−1/2+δISO,n2/3).O_p\left(n^{-1/2}+\delta_{\mathrm{ISO},n}^{2/3}\right).

The proof in Appendix B.4 is based on RMS control of coordinate perturbations, a bounded density assumption to control the probability mass near the halfspace boundaries and uniform convergence of the empirical halfspace probabilities. The resulting bound is a general stability property of halfspace depth and is independent of ISOMAP. The stability and continuity of halfspace depth under distributional perturbations has been extensively studied (Romanazzi 2001, 2002; Mizera and Volauf 2002; Nagy et al. 2019). The paper says absolutely nothing about these prior results and provides no indication as to how Theorem 3 goes beyond them. The fact that Theorem 3 is presented as the paper's principal contribution is an issue with respect to the paper's claim of originality.

References

Questions for authors
  1. The necessity of ISOMAP and a comparison with alternative approaches.

What benefit does computing halfspace depth in an ISOMAP embedding as offer opposed to just measuring peripherality via graph distances (e.g. average shortest path distance)? In the case of the paper's gapped circle example the average geodesic distance and the negative halfspace depth in the embedded space give exactly the same population anomaly rankings (they both rank the observations by their distance from the centre of the intrinsic interval). So in that case a much simpler graph distance score would achieve the same result without the need for ISOMAP or any statistical depth. Other approaches (Laplacian Eigenmaps (Belkin and Niyogi, 2003), Diffusion Maps with halfspace depth (Coifman and Lafon, 2006), or kernelized spatial depth (Chen et al., 2009)) also capture the nonlinear structure of the data. Can the authors be more specific as to what makes their proposed method different from these? Is there any theoretical or empirical justification for the particular choice of ISOMAP followed by halfspace depth?

  1. Novelty of Theorem 3.

The stability bound from Theorem 3 doesn't appear to be specific to ISOMAP at all. How does Theorem 3 differ from the existing work on the stability and continuity of halfspace depth (e.g. Romanazzi (2001, 2002), Mizera and Volauf (2002), Nagy et al. (2019))? What is the new theoretical insight beyond just applying the standard halfspace depth stability arguments to the ISOMAP coordinate recovery error?

  1. Approximation assumption in Theorem 4.

Theorem 4 is predicated on Assumption 1 which requires a neural network approximation rate of O(Wn−1/d)O(W_n^{-1/d}). The authors note that Nakada and Imaizumi (2020) do not in fact provide this approximation rate at the intrinsic dimension dd. Can the authors justify the use of this assumption for the network class under consideration (with the depth and weight bounds specified)? What would the convergence rate be if one were to simply use the approximation guarantee that is actually available from the result of Nakada and Imaizumi (2020)?

1 comment

Thank you for these thoughtful questions. They help us clarify our choice of ISOMAP, the contribution of Theorem 3, and the architecture conditions required for Theorem 4. We address each question below.

Answer to Q1.

We use ISOMAP because it estimates the distances that determine depth in our latent coordinates. Under the assumed isometry to a convex Euclidean domain, manifold geodesic distances equal latent Euclidean distances. Given exact distances and the full intrinsic dimension, classical MDS recovers the latent coordinates up to a rigid transformation, which preserves population and empirical Tukey depth by affine invariance (Zuo and Serfling, 2000). These assumptions connect geodesic representation to latent parametrization, a distinction emphasized by Trosset and Büyükbaş (2021). Finite-sample accuracy also depends on graph-distance accuracy and the stability of coordinate recovery and depth.

This Euclidean structure arises in applications (Tenenbaum et al., 2000), and exploiting it can improve detection when anomalies follow that structure. Laplacian Eigenmaps (LE) preserves local neighborhoods (Belkin and Niyogi, 2003), while Diffusion Maps (DM) represents diffusion distances at a specified time (Coifman and Lafon, 2006).

We compared ISOMAP, LE, DM, and PCA using the same exact empirical Tukey depth on matched observations over 100 seeds per dataset. On the Swiss roll, we retained two dimensions. Mean average precision (AP) was 0.3402 for ISOMAP, 0.1111 for LE, 0.1320 for DM, and 0.2553 for PCA. ISOMAP had the highest depth-rank agreement with the true latent coordinates (mean Spearman correlation 0.9838), although depth on the true coordinates achieved AP 0.5786. On the gapped circle, we retained one dimension. LE and DM slightly outperformed ISOMAP (AP 0.9848, 0.9849, and 0.9815, respectively). These results support ISOMAP for the tested two-dimensional boundary anomalies, while the gapped-circle results indicate that performance depends on the geometry and anomaly structure.

Answer to Q2.

Theorem 3 quantifies how error in the coordinates estimated by ISOMAP propagates to empirical depth. With projection densities bounded by MM, combining the coordinate-to-depth bound with Theorem 2 gives

∥T2∥n,2=Op(n−1/2+(log⁡nn)2/(3d)).\|T_2\|_{n,2} = O_p(n^{-1/2} + (\frac{\log n}{n})^{2/(3d)}).

The cited work addresses influence functions and contamination (Romanazzi, 2001, 2002), continuity of depth contours and maximum-depth estimators (Mizera and Volauf, 2002), and continuity and qualitative robustness under weak convergence (Nagy et al., 2019). Our contribution is an explicit depth-error rate for coordinates estimated from the full sample.

Appendix B proves the required Lipschitz property of population halfspace depth. To handle discrete empirical measures, we track which observations can change halfspace membership under matched coordinate displacements. Bounded latent projection densities and uniform empirical control on the original i.i.d. sample bound the resulting discrepancy. The Lipschitz property then controls depth changes at the displaced evaluation points. This argument permits dependence among recovered coordinates and applies to other embeddings with suitable recovery guarantees. We will clarify this contribution and cite the stability results used in the proof.

Answer to Q3.

The endpoint approximation rate O(Wn−1/d)O(W_n^{-1/d}) requires more specific architecture conditions than we originally stated. Under our existing assumptions, we can establish that the target depth is Lipschitz in ambient Euclidean distance and construct a ReLU approximation achieving this uniform error. A sufficient architecture has at most WnW_n nonzero parameters and satisfies

Ln=2p+3,Bn=CBWn1/d,CB≥max⁡(1,2CW−1/d).L_n=2p+3,\quad B_n=C_BW_n^{1/d},\quad C_B\geq\max(1,2C_W^{-1/d}).

Here LnL_n counts affine layers including the output, and CWC_W is the constant in the interpolation lemma to be included in the appendix. The complexity remains O(Wnlog⁡n)O(W_n\log n). Taking Wn≍(n/log⁡n)d/(d+2)W_n\asymp(n/\log n)^{d/(d+2)} gives

E(f^)=Op((log⁡nn)2/(3d)+(log⁡nn)1/(d+2)).\mathcal{E}(\hat f)=O_p((\frac{\log n}{n})^{2/(3d)}+(\frac{\log n}{n})^{1/(d+2)}).

For fixed 0<ε<p−d0<\varepsilon<p-d, Theorem 5 and Corollary 6 of Nakada and Imaizumi (2020) give approximation error O(Wn−1/(d+ε))O(W_n^{-1/(d+\varepsilon)}) under their architecture. Balancing approximation and estimation gives Wn≍(n/log⁡n)(d+ε)/(d+ε+2)W_n\asymp(n/\log n)^{(d+\varepsilon)/(d+\varepsilon+2)}. The coordinate-error term stays the same, while the exponent in the second term becomes 1/(d+ε+2)1/(d+\varepsilon+2). Because the constants may depend on ε\varepsilon, this bound alone does not establish the endpoint. We will replace Assumption 1 with these sufficient architecture conditions, add the proof to the appendix, and update Theorem 4 accordingly, retaining its 1/(d+2)1/(d+2) exponent and exact-ERM assumption.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.