The problem of nonparametric estimation of a bounded regression function $f\in L²\left({[a,b]}^d\right)$, [a,b] ⊂ ℝ, d ≥ 1, using an orthonormal system of functions $e_k$, k=1,2,..., is considered in the case when the observations follow the model $Y_i=f\left(X_i\right)+{\eta }_i$, i=1,...,n, where $X_i$ and ${\eta }_i$ are i.i.d. copies of independent random variables X and η, respectively, the distribution of X has density ϱ, and η has mean zero and finite variance. The estimators are constructed by proper truncation of the function $f̂ₙ\left(x\right)={\sum }_{k=1}^{N\left(n\right)}c{̂}_k{e}_k\left(x\right)$, where the coefficients $ĉ₁,...,c{̂}_{N\left(n\right)}$ are determined...

We want to recover a signal based on noisy inhomogeneous data (the amount of data can vary strongly on the estimation domain). We model the data using nonparametric regression with random design, and we focus on the estimation of the regression at a fixed point x0 with little, or much data. We propose a method which adapts both to the local amount of data (the design density is unknown) and to the local smoothness of the regression function. The procedure consists of a local polynomial...

Generalized Methods of Moments (GMM) estimators are a popular tool in econometrics since introduced by Hansen (1982), because this approach provides feasible solutions for many problems present in economic data where least squares or maximum likelihood methods fail when naively applied. These problems may arise in errors-in-variable regression, estimation of labor demand curves, and asset pricing in finance, which are discussed here. In this paper we study a GMM estimator for the rank modelingapproach...

We investigate the estimation of a multidimensional regression function $f$ from $n$ observations of an $\alpha$-mixing process $(Y,X)$, where $Y=f\left(X\right)+\xi$, $X$ represents the design and $\xi$ the noise. We concentrate on wavelet methods. In most papers considering this problem, either the proposed wavelet estimator is not adaptive (i.e., it depends on the knowledge of the smoothness of $f$ in its construction) or it is supposed that $\xi$ is bounded or/and has a known distribution. In this paper, we go far beyond this classical framework....

We study sample-based estimates of the expectation of the function produced by the empirical minimization algorithm. We investigate the extent to which one can estimate the rate of convergence of the empirical minimizer in a data dependent manner. We establish three main results. First, we provide an algorithm that upper bounds the expectation of the empirical minimizer in a completely data-dependent manner. This bound is based on a structural result due to Bartlett and Mendelson, which relates...

We study the performance of empirical risk minimization (ERM), with respect to the quadratic risk, in the context of convex aggregation, in which one wants to construct a procedure whose risk is as close as possible to the best function in the convex hull of an arbitrary finite class $F$. We show that ERM performed in the convex hull of $F$ is an optimal aggregation procedure for the convex aggregation problem. We also show that if this procedure is used for the problem of model selection aggregation,...

We consider, in the framework of multidimensional observations, nonparametric functional estimators, which include, as special cases, the Akaike–Parzen–Rosenblatt kernel density estimators ([1, 18, 20]), and the Nadaraya–Watson kernel regression estimators ([16, 22]). We evaluate the sup-norm, over a given set $𝐈$, of the difference between the estimator and a non-random functional centering factor (which reduces to the estimator mean for kernel density estimation). We show that, under suitable general...

A survey of some recent results on nonparametric on-line estimation is presented. The first result deals with an on-line estimation for a smooth signal S(t) in the classic 'signal plus Gaussian white noise' model. Then an analogous on-line estimator for the regression estimation problem with equidistant design is described and justified. Finally some preliminary results related to the on-line estimation for the diffusion observed process are described.

This paper is a survey of recent results on some problems of supervised learning in the setting formulated by Cucker and Smale. Supervised learning, or learning-from-examples, refers to a process that builds on the base of available data of inputs $x_i$ and outputs $y_i$, i = 1,...,m, a function that best represents the relation between the inputs x ∈ X and the corresponding outputs y ∈ Y. The goal is to find an estimator $f_z$ on the base of given data $z:=((x₁,y₁),...,(x_m,y_m))$ that approximates well the regression function $f_{\rho }$ of...

The problem of nonparametric function fitting using the complete orthogonal system of Whittaker cardinal functions $s_k$, k = 0,±1,..., for the observation model $y_j=f\left(u_j\right)+{\eta }_j$, j = 1,...,n, is considered, where f ∈ L²(ℝ) ∩ BL(Ω) for Ω > 0 is a band-limited function, $u_j$ are independent random variables uniformly distributed in the observation interval [-T,T], ${\eta }_j$ are uncorrelated or correlated random variables with zero mean value and finite variance, independent of the observation points. Conditions for convergence...

This paper is concerned with general conditions for convergence rates of nonparametric orthogonal series estimators of the regression function. The estimators are obtained by the least squares method on the basis of an observation sample $Y_i=f\left(X_i\right)+{\eta }_i$, i=1,...,n, where $X_i\in A\subset {ℝ}^d$ are independently chosen from a distribution with density ϱ ∈ L¹(A) and ${\eta }_i$ are zero mean stationary errors with long-range dependence. Convergence rates of the error $n^{-1}{\sum }_{i=1}^n(f\left(X_i\right)-f{̂}_N\left(X_i\right))²$ for the estimator $f{̂}_N\left(x\right)={\sum }_{k=1}^{N}c{̂}_k{e}_k\left(x\right)$, constructed using an orthonormal system $e_k$, k=1,2,...,...

Nonparametric orthogonal series regression function estimation is investigated in the case of a fixed point design where the observation points are irregularly spaced in a finite interval [a,b]i ⊂ ℝ. Convergence rates for the integrated mean-square error and pointwise mean-square error are obtained in the case of estimators constructed using the Legendre polynomials and Haar functions for regression functions satisfying the Lipschitz condition.