Nonparametric regression estimation based on spatially inhomogeneous data: minimax global convergence rates and adaptivity

Anestis Antoniadis; Marianna Pensky; Theofanis Sapatinas

ESAIM: Probability and Statistics (2014)

  • Volume: 18, page 1-41
  • ISSN: 1292-8100

Abstract

top
We consider the nonparametric regression estimation problem of recovering an unknown response function f on the basis of spatially inhomogeneous data when the design points follow a known density g with a finite number of well-separated zeros. In particular, we consider two different cases: when g has zeros of a polynomial order and when g has zeros of an exponential order. These two cases correspond to moderate and severe data losses, respectively. We obtain asymptotic (as the sample size increases) minimax lower bounds for the L2-risk when f is assumed to belong to a Besov ball, and construct adaptive wavelet thresholding estimators of f that are asymptotically optimal (in the minimax sense) or near-optimal within a logarithmic factor (in the case of a zero of a polynomial order), over a wide range of Besov balls. The spatially inhomogeneous ill-posed problem that we investigate is inherently more difficult than spatially homogeneous ill-posed problems like, e.g., deconvolution. In particular, due to spatial irregularity, assessment of asymptotic minimax global convergence rates is a much harder task than the derivation of asymptotic minimax local convergence rates studied recently in the literature. Furthermore, the resulting estimators exhibit very different behavior and asymptotic minimax global convergence rates in comparison with the solution of spatially homogeneous ill-posed problems. For example, unlike in the deconvolution problem, the asymptotic minimax global convergence rates are greatly influenced not only by the extent of data loss but also by the degree of spatial homogeneity of f. Specifically, even if 1/g is non-integrable, one can recover f as well as in the case of an equispaced design (in terms of asymptotic minimax global convergence rates) when it is homogeneous enough since the estimator is “borrowing strength” in the areas where f is adequately sampled.

How to cite

top

Antoniadis, Anestis, Pensky, Marianna, and Sapatinas, Theofanis. "Nonparametric regression estimation based on spatially inhomogeneous data: minimax global convergence rates and adaptivity." ESAIM: Probability and Statistics 18 (2014): 1-41. <http://eudml.org/doc/273640>.

@article{Antoniadis2014,
abstract = {We consider the nonparametric regression estimation problem of recovering an unknown response function f on the basis of spatially inhomogeneous data when the design points follow a known density g with a finite number of well-separated zeros. In particular, we consider two different cases: when g has zeros of a polynomial order and when g has zeros of an exponential order. These two cases correspond to moderate and severe data losses, respectively. We obtain asymptotic (as the sample size increases) minimax lower bounds for the L2-risk when f is assumed to belong to a Besov ball, and construct adaptive wavelet thresholding estimators of f that are asymptotically optimal (in the minimax sense) or near-optimal within a logarithmic factor (in the case of a zero of a polynomial order), over a wide range of Besov balls. The spatially inhomogeneous ill-posed problem that we investigate is inherently more difficult than spatially homogeneous ill-posed problems like, e.g., deconvolution. In particular, due to spatial irregularity, assessment of asymptotic minimax global convergence rates is a much harder task than the derivation of asymptotic minimax local convergence rates studied recently in the literature. Furthermore, the resulting estimators exhibit very different behavior and asymptotic minimax global convergence rates in comparison with the solution of spatially homogeneous ill-posed problems. For example, unlike in the deconvolution problem, the asymptotic minimax global convergence rates are greatly influenced not only by the extent of data loss but also by the degree of spatial homogeneity of f. Specifically, even if 1/g is non-integrable, one can recover f as well as in the case of an equispaced design (in terms of asymptotic minimax global convergence rates) when it is homogeneous enough since the estimator is “borrowing strength” in the areas where f is adequately sampled.},
author = {Antoniadis, Anestis, Pensky, Marianna, Sapatinas, Theofanis},
journal = {ESAIM: Probability and Statistics},
keywords = {adaptivity; Besov spaces; inhomogeneous data; minimax estimation; nonparametric regression; thresholding; wavelet estimation},
language = {eng},
pages = {1-41},
publisher = {EDP-Sciences},
title = {Nonparametric regression estimation based on spatially inhomogeneous data: minimax global convergence rates and adaptivity},
url = {http://eudml.org/doc/273640},
volume = {18},
year = {2014},
}

TY - JOUR
AU - Antoniadis, Anestis
AU - Pensky, Marianna
AU - Sapatinas, Theofanis
TI - Nonparametric regression estimation based on spatially inhomogeneous data: minimax global convergence rates and adaptivity
JO - ESAIM: Probability and Statistics
PY - 2014
PB - EDP-Sciences
VL - 18
SP - 1
EP - 41
AB - We consider the nonparametric regression estimation problem of recovering an unknown response function f on the basis of spatially inhomogeneous data when the design points follow a known density g with a finite number of well-separated zeros. In particular, we consider two different cases: when g has zeros of a polynomial order and when g has zeros of an exponential order. These two cases correspond to moderate and severe data losses, respectively. We obtain asymptotic (as the sample size increases) minimax lower bounds for the L2-risk when f is assumed to belong to a Besov ball, and construct adaptive wavelet thresholding estimators of f that are asymptotically optimal (in the minimax sense) or near-optimal within a logarithmic factor (in the case of a zero of a polynomial order), over a wide range of Besov balls. The spatially inhomogeneous ill-posed problem that we investigate is inherently more difficult than spatially homogeneous ill-posed problems like, e.g., deconvolution. In particular, due to spatial irregularity, assessment of asymptotic minimax global convergence rates is a much harder task than the derivation of asymptotic minimax local convergence rates studied recently in the literature. Furthermore, the resulting estimators exhibit very different behavior and asymptotic minimax global convergence rates in comparison with the solution of spatially homogeneous ill-posed problems. For example, unlike in the deconvolution problem, the asymptotic minimax global convergence rates are greatly influenced not only by the extent of data loss but also by the degree of spatial homogeneity of f. Specifically, even if 1/g is non-integrable, one can recover f as well as in the case of an equispaced design (in terms of asymptotic minimax global convergence rates) when it is homogeneous enough since the estimator is “borrowing strength” in the areas where f is adequately sampled.
LA - eng
KW - adaptivity; Besov spaces; inhomogeneous data; minimax estimation; nonparametric regression; thresholding; wavelet estimation
UR - http://eudml.org/doc/273640
ER -

References

top
  1. [1] U. Amato, A. Antoniadis and M. Pensky, Wavelet kernel penalized estimation for non-equispaced design regression. Statist. Comput.16 (2006) 37–55. MR2224188
  2. [2] A. Antoniadis and D.T. Pham, Wavelet regression for random or irregular design. Comput. Statist. Data Anal.28 (1998) 353–369. Zbl1042.62534MR1659207
  3. [3] N. Bissantz, L. Dumbgen, H. Holzmann and A. Munk, Nonparametric confidence bands in deconvolution density estimation. J. Royal Statist. Soc. Series B69 (2007) 483–506. MR2323764
  4. [4] L.D. Brown, T.T. Cai, M.G. Low and C.H. Zhang, Asymptotic equivalence theory for nonparametric regression with random design. Annal. Statist.30 (2002) 688–707. Zbl1029.62044MR1922538
  5. [5] T.T. Cai and L.D. Brown, Wavelet shrinkage for nonequispaced samples. Annal. Statist.26 (1998) 1783–1799. Zbl0929.62047MR1673278
  6. [6] C. Chesneau, Regression in random design: a minimax study. Statist. Probab. Lett.77 (2007) 40–53. Zbl1109.62028MR2339017
  7. [7] I. Daubechies, Ten Lectures on Wavelets. Philadelphia: SIAM (1992). Zbl0776.42018MR1162107
  8. [8] R.B. Dingle, Asymptotic Expansions: Their Derivation and Interpretation. London: Academic Press (1973). Zbl0279.41030MR499926
  9. [9] S. Gaïffas,. Convergence rates for pointwise curve estimation with a degenerate design. Math. Meth. Statist. 14 (2005) 1–27. MR2158069
  10. [10] S. Gaïffas, Sharp estimation in sup norm with random design. Statist. Probab. Lett.77 (2006) 782–794. Zbl1114.62046MR2369683
  11. [11] S. Gaïffas, On pointwise adaptive curve estimation based on inhomogeneous data. ESAIM: Probab. Statist. 11 (2007) 344–364. Zbl1187.62074MR2339297
  12. [12] S. Gaïffas, Uniform estimation of a signal based on inhomogeneous data. Statistica Sinica19 (2009) 427–447. Zbl1168.62036MR2514170
  13. [13] E. Giné and R. Nickl, Confidence bands in density estimation. Annal. Statist.38 (2010) 1122–1170. Zbl1183.62062MR2604707
  14. [14] A. Guillou and N. Klutchnikoff, Minimax pointwise estimation of an anisotropic regression function with unknown density of the design. Math. Methods Statist.20 (2011) 30–57. Zbl1282.62076MR2811030
  15. [15] P. Hall and B.A. Turlach, Interpolation methods for nonlinear wavelet regression with irregularly spaced design. Annal. Statist.25 (1997) 1912–1925. Zbl0881.62044MR1474074
  16. [16] W. Härdle, G. Kerkyacharian, D. Picard and A. Tsybakov, Wavelets, Approximation, and Statistical Applications, vol. 129 of Lect. Notes Statist. Springer-Verlag, New York (1998). Zbl0899.62002MR1618204
  17. [17] M. Hoffmann and M. Reiss, Nonlinear estimation for linear inverse problems with error in the operator. Annal. Statist.36 (2008) 310–336. Zbl1134.65038MR2387973
  18. [18] I.M. Johnstone, Minimax Bayes, asymptotic minimax and sparse wavelet priors, in Statistical Decision Theory and Related Topics, edited by S.S. Gupta and J.O. Berger. Springer-Verlag, New York (1994) 303–326, Zbl0815.62017MR1286310
  19. [19] I.M. Johnstone, Function Estimation and Gaussian Sequence Models. Unpublished Monograph (2002). http://statweb.stanford.edu/~imj/ 
  20. [20] I.M. Johnstone, G. Kerkyacharian, D. Picard and M. Raimondo, Wavelet deconvolution in a periodic setting. J. Royal Statist. Soc. Series B 66 (2004) 547-573 (with discussion, 627--657). Zbl1046.62039MR2088290
  21. [21] G. Kerkyacharian and D. Picard, Regression in random design and warped wavelets. Bernoulli10 (2004) 1053-1105. Zbl1067.62039MR2108043
  22. [22] M. Kohler, Nonlinear orthogonal series estimation for random design regression. J. Statist. Plann. Inference115 (2003) 491-520. Zbl1016.62040MR1985881
  23. [23] A.P. Korostelev and A.B. Tsybakov, Minimax Theory of Image Reconstruction, vol. 82 of Lect. Notes Statist. Springer-Verlag, New York (1993). Zbl0833.62039MR1226450
  24. [24] A. Kovac and B.W. Silverman, Extending the scope of wavelet regression methods by coefficient-dependent thresholding. J. Amer. Statist. Assoc.95 (2000) 172-183. 
  25. [25] R. Kulik and M. Raimondo, Wavelet regression in random design with heteroscedastic dependent errors. Annal. Statist.37 (2009) 3396-3430. Zbl05644284MR2549564
  26. [26] O.V. Lepski, A problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl.35 (1990) 454-466. Zbl0745.62083MR1091202
  27. [27] O.V. Lepski, Asymptotically minimax adaptive estimation I: Upper bounds. Optimally adaptive estimators. Theory Probab. Appl. 36 (1991) 682-697. Zbl0776.62039MR1147167
  28. [28] O.V. Lepski, E. Mammen and V.G. Spokoiny, Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors Annal. Statist.25 (1997) 929-947. Zbl0885.62044MR1447734
  29. [29] O. Lepski and V. Spokoiny, Optimal pointwise adaptive methods in nonparametric estimation. Annal. Statist.25 (1997) 2512-2546. Zbl0894.62041MR1604408
  30. [30] S. Mallat, A Wavelet Tour of Signal Processing. 2nd Edition, Academic Press, San Diego (1999). Zbl1170.94003MR2479996
  31. [31] Y. Meyer, Wavelets and Operators. Cambridge: Cambridge University Press (1992). Zbl0776.42019MR1228209
  32. [32] M. Pensky and T. Sapatinas, Functional deconvolution in a periodic case: uniform case. Annal. Statist.37 (2009) 73-104. Zbl1274.62253MR2488345
  33. [33] M. Pensky and T. Sapatinas, On convergence rates equivalency and sampling strategies in functional deconvolution models. Annal. Statist.38 (2010) 1793-1844. Zbl05712439MR2662360
  34. [34] M. Pensky and B. Vidakovic, On non-equally spaced wavelet regression. Annal. Instit. Statist. Math.53 (2001) 681-690. Zbl1003.62037MR1879604
  35. [35] S. Sardy, D.B. Percival, A.G. Bruce, H.-Y. Gao and W. Stuelzle, Wavelet shrinkage for unequally spaced data. Statist. Comput.9 (1999) 65-75. 
  36. [36] K. Tribouley, Adaptive simultaneous confidence intervals in non-parametric estimation. Statist. Probab. Lett.69 (2004) 37-51. Zbl1116.62346MR2087668
  37. [37] A.B. Tsybakov, Introduction to Nonparametric Estimation. Springer-Verlag, New York (2009). Zbl1029.62034MR2724359
  38. [38] G. Wahba, Spline Models for Observational Data. SIAM, Philadelphia (1990). Zbl0813.62001MR1045442
  39. [39] S. Zhang, M.-Y. Wong and Z. Zheng, Wavelet threshold estimation of a regression function with random design. J. Multivariate Anal.80 (2002) 256-284. Zbl1003.62040MR1889776

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.