Nonparametric Regression, Confidence Regions and Regularization

In this paper we offer a unified approach to the problem of nonparametricregression on the unit interval. It is based on a universal, honest and nonasymptotic confidence region An which is defined by a set of linear inequalities involving the values of the functions at the design points. Interest will typically center on certain simplest functions in An where simplicity can be defined in terms of shape (number of local extremes, intervals of convexity/concavity) or smoothness (bounds on derivatives) or a combination of both. Once some form of regularization has been decided upon the confidence region can be used to provide honest nonasymptotic confidence bounds which are less informative but conceptually much simpler.