A note on orthogonal series regression function estimators

Waldemar Popiński

Applicationes Mathematicae (1999)

  • Volume: 26, Issue: 3, page 281-291
  • ISSN: 1233-7234

Abstract

top
The problem of nonparametric estimation of the regression function f(x) = E(Y | X=x) using the orthonormal system of trigonometric functions or Legendre polynomials e k , k=0,1,2,..., is considered in the case where a sample of i.i.d. copies ( X i , Y i ) , i=1,...,n, of the random variable (X,Y) is available and the marginal distribution of X has density ϱ ∈ L 1 [a,b]. The constructed estimators are of the form f ^ n ( x ) = ∑ k = 0 N ( n ) c ^ k e k ( x ) , where the coefficients c ^ 0 , c ^ 1 , . . . , c ^ N are determined by minimizing the empirical risk n - 1 ∑ i = 1 n ( Y i - ∑ k = 0 N c k e k ( X i ) ) 2 . Sufficient conditions for consistency of the estimators in the sense of the errors E X | f ( X ) - f ^ n ( X ) | 2 and n - 1 ∑ i = 1 n E ( f ( X i ) - f ^ n ( X i ) ) 2 are obtained.

How to cite

top

Popiński, Waldemar. "A note on orthogonal series regression function estimators." Applicationes Mathematicae 26.3 (1999): 281-291. <http://eudml.org/doc/219239>.

@article{Popiński1999,
abstract = {The problem of nonparametric estimation of the regression function f(x) = E(Y | X=x) using the orthonormal system of trigonometric functions or Legendre polynomials $e_k$, k=0,1,2,..., is considered in the case where a sample of i.i.d. copies $(X_i,Y_i)$, i=1,...,n, of the random variable (X,Y) is available and the marginal distribution of X has density ϱ ∈ $L^1$[a,b]. The constructed estimators are of the form $\widehat\{f\}_n(x) = \sum _\{k=0\}^\{N(n)\}\widehat\{c\}_ke_k(x)$, where the coefficients $\widehat\{c\}_0,\widehat\{c\}_1,...,\widehat\{c\}_N$ are determined by minimizing the empirical risk $n^\{-1\}\sum _\{i=1\}^n(Y_i - \sum _\{k=0\}^Nc_ke_k(X_i))^2$. Sufficient conditions for consistency of the estimators in the sense of the errors $E_X\vert f(X)-\widehat\{f\}_n(X)\vert ^2$ and $n^\{-1\}\sum _\{i=1\}^nE(f(X_i)-\widehat\{f\}_n(X_i))^2$ are obtained.},
author = {Popiński, Waldemar},
journal = {Applicationes Mathematicae},
keywords = {consistent estimator; orthonormal system; empirical risk minimization; nonparametric regression},
language = {eng},
number = {3},
pages = {281-291},
title = {A note on orthogonal series regression function estimators},
url = {http://eudml.org/doc/219239},
volume = {26},
year = {1999},
}

TY - JOUR
AU - Popiński, Waldemar
TI - A note on orthogonal series regression function estimators
JO - Applicationes Mathematicae
PY - 1999
VL - 26
IS - 3
SP - 281
EP - 291
AB - The problem of nonparametric estimation of the regression function f(x) = E(Y | X=x) using the orthonormal system of trigonometric functions or Legendre polynomials $e_k$, k=0,1,2,..., is considered in the case where a sample of i.i.d. copies $(X_i,Y_i)$, i=1,...,n, of the random variable (X,Y) is available and the marginal distribution of X has density ϱ ∈ $L^1$[a,b]. The constructed estimators are of the form $\widehat{f}_n(x) = \sum _{k=0}^{N(n)}\widehat{c}_ke_k(x)$, where the coefficients $\widehat{c}_0,\widehat{c}_1,...,\widehat{c}_N$ are determined by minimizing the empirical risk $n^{-1}\sum _{i=1}^n(Y_i - \sum _{k=0}^Nc_ke_k(X_i))^2$. Sufficient conditions for consistency of the estimators in the sense of the errors $E_X\vert f(X)-\widehat{f}_n(X)\vert ^2$ and $n^{-1}\sum _{i=1}^nE(f(X_i)-\widehat{f}_n(X_i))^2$ are obtained.
LA - eng
KW - consistent estimator; orthonormal system; empirical risk minimization; nonparametric regression
UR - http://eudml.org/doc/219239
ER -

References

top
  1. [1] A. R. Gallant and H. White, There exists a neural network that does not make avoidable mistakes, in: Proc. Second Annual IEEE Conference on Neural Networks, San Diego, California, IEEE Press, New York, 1988, 657-664. 
  2. [2] L. Györfi and H. Walk, On the strong universal consistency of a series type regression estimate, Math. Methods Statist. 5 (1996), 332-342. Zbl0874.62048
  3. [3] G. Lugosi and K. Zeger, Nonparametric estimation via empirical risk minimization, IEEE Trans. Inform. Theory IT-41 (1995), 677-687. Zbl0818.62041
  4. [4] W. Popiński, On least squares estimation of Fourier coefficients and of the regression function, Appl. Math. (Warsaw) 22 (1993), 91-102. Zbl0789.62032
  5. [5] --, Consistency of trigonometric and polynomial regression estimators, ibid. 25 (1998), 73-83. 
  6.  
  7. [7] V. N. Vapnik, Estimation of Dependencies Based on Empirical Data, Springer, New York, 1982. Zbl0499.62005

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.