Statistical Inference for Structurally Constrained Nonlinear Regression

A fundamental challenge in using deep neural networks (DNNs) is to obtain valid prediction uncertainty quantification while retaining their flexibility. Two central goals are to construct confidence intervals for the mean regression function and prediction intervals for future observations. The former quantifies uncertainty about the underlying relationship, while the latter accounts for observational noise. Existing approaches address aspects of these goals, but valid uncertainty quantification under structural constraints remains challenging, particularly when the noise distribution is unknown. Building on Extended Fiducial Inference (EFI), we develop a framework for nonlinear regression with deep neural networks, where structural constraints are imposed to restrict the admissible function class while an unknown noise distribution is accommodated. Under the assumption that the structurally constrained function class admits a locally identifiable low-dimensional representation, we establish theoretical guarantees for inference in dense, over-parameterized neural networks without requiring sparse parameterizations. Simulation studies show near-nominal coverage for both confidence and prediction intervals across a range of constrained nonlinear regression settings, including differential equations, monotonicity, and periodicity, under Gaussian and non-Gaussian noise.