acceptodds
Under review as a conference paper at ICLR 2027

Nonlinear Label Budgets: How Geometry Determines Prediction Risk

Abstract

Nonlinear models can exploit low-dimensional structure, but this benefit depends on how observations are processed. We ask how many labels need nonlinear processing when an initial sample configures an affine predictor for the remaining labels. For a known compact smooth model manifold, we characterize the minimax risk over this complete class. The leading risk follows a staircase of derivative-space dimensions as the nonlinear budget changes; at the square-root boundary, curvature determines an exact continuous transition. We construct attaining readouts by correcting the mean displacement and shrinking normal uncertainty. A growing-dimensional construction shows that a vanishing pilot fraction can remove a diverging cost of fixed affine prediction. Allowing a quadratic readout removes the leading curvature penalty with a smaller sufficient pilot budget, while minimax laws quantify source uncertainty and prescribe shrinkage under model error. Conditional low-rank readouts turn these principles into a configuration method with an explicit gain criterion. Experiments test the risk laws and learned readouts; on 100 captured objects, configuration reduces MSE by a per-object average of 16.3% over matched fixed readouts.

Primary area:
learning theory
Keywords:
statistical learning theory, minimax estimation, manifold geometry, nonlinear label budgets, two-stage estimation, low-rank prediction

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_NkMFOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies predictors that process an initial sample nonlinearly, then fix an affine map for the remaining observations. For signals lying on a known smooth manifold, it characterizes how prediction risk depends on the initial sample size. For signals on a smooth manifold, it characterizes how prediction risk depends on the initial sample size and the manifold’s geometry.

Critical strengths
  1. The matching upper and lower bounds characterize the statistical cost of nonlinear processing,

  2. The observation that successive derivative directions determine the leading risk is interesting.

Critical weaknesses
  1. The meaning of “nonlinear label budget” is difficult to understand from the introduction and the restriction concerns how observations are processed. The paper introduces complivated terminologies in rapid succession, without any examples. Thus, it is difficult to identify which restriction creates the statistical difficulty.

  2. The main results concern Gaussian observations of a known manifold signal, while the image experiments use sparse pixels and source-fitted conditional moments.

  3. The geometric minimax results appear interesting, but the presentation makes their meaning and relationship to the applications unnecessarily difficult to assess

Questions for authors
  1. Can the authors give one concrete observation-to-prediction example, explicitly stating which operations are permitted before and after the pilot, and use it to explain the main theorems?

  2. Which conclusions from the Gaussian manifold theory apply to the sparse-pixel experiments?

Reviewer_FWmDOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
1
Soundness
3
Read review
Paper summary

At its core, this paper is an investigation of the price of early-commitment to affine-only predictors, with the geometry of the model's manifold determining the prediction error. The paper's setting is that an unknown point v lies on a known smooth manifold and is observed through two gaussian blocks: a k-sized pilot block and a n-k main block. An estimator is used to predict v. The estimator is unrestricted in the choice of functions for processing the pilot block, but must only use affine transformations for the main block, with coefficients fixed before seeing the main block. The authors' main contribution is a sharp characterisation of the minimax risk of this setting. They demonstrate that when the pilot localises v tightly enough that the manifold’s curvature is negligible at that scale, the prediction error reaches its minimum. For this to happen in the paper’s base settings (the affine restriction of the main block predictor), the pilot block needs to be at least of size √n. As the size of the pilot block decreases, the error rises in steps, each tied to a higher-order derivative of the manifold. If quadratic terms are allowed for estimation from the main block, then the pilot needs only be n^(1/3) for the estimator to minimise the prediction error. The authors then address the questions of the additional error incurred from having to learn the shape of the manifold from additional training data (rather than it being given a-priori). Finally they also consider the case, common in the real world, where the point v is not placed exactly on the surface of the manifold. In this case they find that every direction must maintain a minimum weight, which they precisely characterise, alongside the additional error incurred. The authors validate the risk laws on synthetic data and test the framework on sparse-pixel image completion on real-world image datasets: COIL-20, COIL-100, CO3D and MVImgNet RGB.

Critical strengths

The question the paper poses is both novel and relevant and the authors answer it well.

The paper asks the impactful question of what is the cost of fixing linear weights early and answers it exactly through rigorous theoretical explanations. The authors give the best achievable error and proves nothing in the class can do better. Additionally real-data experimental settings and protocols seem rigorous, allowing the paper to clearly isolate exactly the claimed effect across four datasets.

Critical weaknesses

The real-data experiments compare the proposed readout only against variants of itself, even though a natural comparison exists in the form of Conditional Neural Processes (CNP), which are evaluated on the same image task the authors use to test their framework. As a result, whilst the authors validate that pilot-chosen weights beat fixed weights in their framework, there is no contextualisation of the results relative to any existing methods.

Questions for authors

Could the authors add at least one established baseline (such as CNP or gaussian processes) on the same image-completion task ?

Reviewer_Vx1VOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies prediction when a nonlinear pilot configures an affine readout for the remaining observations. It establishes sharp minimax risk laws governed by derivative-space dimensions and curvature, constructs attaining estimators, and extends the analysis to quadratic readouts and uncertain geometry. Simulations test the theoretical laws, while captured-view experiments evaluate conditional configuration against matched fixed readouts.

Critical strengths
  1. Matching upper and lower bounds over the full pilot-affine class provide a substantial theoretical contribution. The risk staircase and exact curvature-dependent transition explain why intrinsic dimension alone does not determine prediction risk under restricted nonlinear access.

  2. The theory yields concrete attaining procedures through mean correction and normal shrinkage. Quadratic readouts further demonstrate how additional computational flexibility changes the sufficient pilot budget. Targeted simulations support these mechanisms.

Critical weaknesses

1, The nonlinear-access budget has an incomplete connection to practical deployment cost. Configuration, feedback, and coefficient transfer introduce overhead, and the reported CPU timings favor the fully nonlinear baseline.

  1. The captured-view experiments primarily establish conditional matrix-adaptation gains. Their connection to the central geometric budget laws is indirect, limiting the demonstrated practical reach of the theory.
Questions for authors

Which concrete deployment constraint would make the reported accumulator savings decisive after including per-task configuration and coefficient transfer? A cost argument based on the existing implementation would help clarify the practical scope.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.