Partition-based conditional density estimation

S. X. Cohen; E. Le Pennec

ESAIM: Probability and Statistics (2013)

  • Volume: 17, page 672-697
  • ISSN: 1292-8100

Abstract

top
We propose a general partition-based strategy to estimate conditional density with candidate densities that are piecewise constant with respect to the covariate. Capitalizing on a general penalized maximum likelihood model selection result, we prove, on two specific examples, that the penalty of each model can be chosen roughly proportional to its dimension. We first study a classical strategy in which the densities are chosen piecewise conditional according to the variable. We then consider Gaussian mixture models with mixing proportion that vary according to the covariate but with common mixture components. This model proves to be interesting for an unsupervised segmentation application that was our original motivation for this work.

How to cite

top

Cohen, S. X., and Le Pennec, E.. "Partition-based conditional density estimation." ESAIM: Probability and Statistics 17 (2013): 672-697. <http://eudml.org/doc/274371>.

@article{Cohen2013,
abstract = {We propose a general partition-based strategy to estimate conditional density with candidate densities that are piecewise constant with respect to the covariate. Capitalizing on a general penalized maximum likelihood model selection result, we prove, on two specific examples, that the penalty of each model can be chosen roughly proportional to its dimension. We first study a classical strategy in which the densities are chosen piecewise conditional according to the variable. We then consider Gaussian mixture models with mixing proportion that vary according to the covariate but with common mixture components. This model proves to be interesting for an unsupervised segmentation application that was our original motivation for this work.},
author = {Cohen, S. X., Le Pennec, E.},
journal = {ESAIM: Probability and Statistics},
keywords = {conditional density estimation; partition; penalized likelihood; piecewise polynomial density; gaussian mixture model; piecewise polynomial densities; Gaussian mixture models},
language = {eng},
pages = {672-697},
publisher = {EDP-Sciences},
title = {Partition-based conditional density estimation},
url = {http://eudml.org/doc/274371},
volume = {17},
year = {2013},
}

TY - JOUR
AU - Cohen, S. X.
AU - Le Pennec, E.
TI - Partition-based conditional density estimation
JO - ESAIM: Probability and Statistics
PY - 2013
PB - EDP-Sciences
VL - 17
SP - 672
EP - 697
AB - We propose a general partition-based strategy to estimate conditional density with candidate densities that are piecewise constant with respect to the covariate. Capitalizing on a general penalized maximum likelihood model selection result, we prove, on two specific examples, that the penalty of each model can be chosen roughly proportional to its dimension. We first study a classical strategy in which the densities are chosen piecewise conditional according to the variable. We then consider Gaussian mixture models with mixing proportion that vary according to the covariate but with common mixture components. This model proves to be interesting for an unsupervised segmentation application that was our original motivation for this work.
LA - eng
KW - conditional density estimation; partition; penalized likelihood; piecewise polynomial density; gaussian mixture model; piecewise polynomial densities; Gaussian mixture models
UR - http://eudml.org/doc/274371
ER -

References

top
  1. [1] N. Akakpo, Adaptation to anisotropy and inhomogeneity via dyadic piecewise polynomial selection. Math. Meth. Stat.21 (2012) 1–28. Zbl1308.62070MR2901269
  2. [2] N. Akakpo and C. Lacour, Inhomogeneous and anisotropic conditional density estimation from dependent data. Electon. J. Statist.5 (2011) 1618–1653. Zbl1271.62060MR2870146
  3. [3] A. Antoniadis, J. Bigot and R. von Sachs, A multiscale approach for statistical characterization of functional images. J. Comput. Graph. Stat.18 (2008) 216–237. MR2649646
  4. [4] A. Barron, C. Huang, J. Li and X. Luo, MDL Principle, Penalized Likelihood, and Statistical Risk, in Festschrift in Honor of Jorma Rissanen on the Occasion of his 75th Birthday. Tampere University Press (2008). 
  5. [5] D. Bashtannyk and R. Hyndman, Bandwidth selection for kernel conditional density estimation. Comput. Stat. Data Anal.36 (2001) 279–298. Zbl1038.62034MR1836204
  6. [6] L. Bertrand, M.-A. Languille, S.X. Cohen, L. Robinet, C. Gervais, S. Leroy, D. Bernard, E. Le Pennec, W. Josse, J. Doucet and S. Schöder, European research platform IPANEMA at the SOLEIL synchrotron for ancient and historical materials. J. Synchrotron Radiat.18 (2011) 765–772. 
  7. [7] Ch. Biernacki, G. Celeux, G. Govaert and F. Langrognet, Model-based cluster and discriminant analysis with the MIXMOD software. Comput. Stat. Data Anal.51 (2006) 587–600. Zbl1157.62431MR2297473
  8. [8] L. Birgé and P. Massart, Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli4 (1998) 329–375. Zbl0954.62033MR1653272
  9. [9] L. Birgé and P. Massart, Minimal penalties for gaussian model selection. Probab. Theory Related Fields138 (2007) 33–73. Zbl1112.62082MR2288064
  10. [10] G. Blanchard, C. Schäfer, Y. Rozenholc and K.R. Müller, Optimal dyadic decision trees. Mach. Learn.66 (2007) 209–241. 
  11. [11] E. Brunel, F. Comte and C. Lacour, Adaptive estimation of the conditional density in presence of censoring. Sankhy69 (2007) 734–763. Zbl1193.62055MR2521231
  12. [12] S.X. Cohen and E. Le Pennec, Conditional density estimation by penalized likelihood model selection and applications. Technical report, RR-7596. INRIA (2011). arXiv:1103.2021. 
  13. [13] S.X. Cohen and E. Le Pennec, Conditional density estimation by penalized likelihood model selection. Submitted (2012). 
  14. [14] S.X. Cohen and E. Le Pennec, Unsupervised segmentation of hyperspectral images with spatialized Gaussian mixture model and model selection. Submitted (2012). 
  15. [15] J. de Gooijer and D. Zerom, On conditional density estimation. Stat. Neerlandica57 (2003) 159–176. Zbl1090.62526MR2028911
  16. [16] D. Donoho, CART and best-ortho-basis: a connection. Ann. Stat.25 (1997) 1870–1911. Zbl0942.62044MR1474073
  17. [17] S. Efromovich, Conditional density estimation in a regression setting. Ann. Stat.35 (2007) 2504–2535. Zbl1129.62025MR2382656
  18. [18] S. Efromovich, Oracle inequality for conditional density estimation and an actuarial example. Ann. Inst. Stat. Math.62 (2010) 249–275. Zbl06531036MR2592098
  19. [19] J. Fan, Q. Yao and H. Tong, Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika83 (1996) 189–206. Zbl0865.62026MR1399164
  20. [20] Ch. Genovese and L. Wasserman, Rates of convergence for the Gaussian mixture sieve. Ann. Stat.28 (2000) 1105–1127. Zbl1105.62333MR1810921
  21. [21] L. Györfi and M. Kohler, Nonparametric estimation of conditional distributions. IEEE Trans. Inform. Theory53 (2007) 1872–1879. Zbl1316.62047MR2317148
  22. [22] P. Hall, R. Wolff and Q. Yao, Methods for estimating a conditional distribution function. J. Amer. Stat. Assoc.94 (1999) 154–163. Zbl1072.62558MR1689221
  23. [23] T. Hofmann, Probabilistic latent semantic analysis, in Proc. of Uncertainty in Artificial Intelligence (1999). 
  24. [24] Y. Huang, I. Pollak, M. Do and C. Bouman, Fast search for best representations in multitree dictionaries. IEEE Trans. Image Process.15 (2006) 1779–1793. 
  25. [25] R. Hyndman and Q. Yao, Nonparametric estimation and symmetry tests for conditional density functions. J. Nonparam. Stat.14 (2002) 259–278. Zbl1013.62040MR1905751
  26. [26] R. Hyndman, D. Bashtannyk and G. Grunwald, Estimating and visualizing conditional densities. J. Comput. Graphical Stat.5 (1996) 315–336. MR1422114
  27. [27] B. Karaivanov and P. Petrushev, Nonlinear piecewise polynomial approximation beyond besov spaces. Appl. Comput. Harmonic Anal.15 (2003) 177–223. Zbl1032.41018MR2010943
  28. [28] E. Kolaczyk and R. Nowak, Multiscale generalised linear models for nonparametric function estimation. Biometrika92 (2005) 119–133. Zbl1068.62046MR2158614
  29. [29] E. Kolaczyk, J. Ju and S. Gopal, Multiscale, multigranular statistical image segmentation. J. Amer. Stat. Assoc.100 (2005) 1358–1369. Zbl1117.62371MR2236447
  30. [30] Q. Li and J. Racine, Nonparametric Econometrics: Theory and Practice. Princeton University Press (2007). Zbl1183.62200MR2283034
  31. [31] J. Lin, Divergence measures based on the Shannon entropy. IEEE Trans. Inform. Theory37 (1991) 145–151. Zbl0712.94004MR1087893
  32. [32] P. Massart, Concentration inequalities and model selection, vol. 1896 of Lecture Notes in Mathematics (2007). Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour (2003), With a foreword by Jean Picard. Zbl1170.60006MR2319879
  33. [33] C. Maugis and B. Michel, A non asymptotic penalized criterion for Gaussian mixture model selection. ESAIM: PS 15 (2012) 41–68. Zbl06157507MR2870505
  34. [34] C. Maugis and B. Michel, Data-driven penalty calibration: a case study for Gaussian mixture model selection. ESAIM: PS 15 (2012) 320–339. Zbl06157520MR2870518
  35. [35] M. Rosenblatt, Conditional probability density and regression estimators, in Multivariate Analysis II, Proc. of Second Internat. Sympos., Dayton, Ohio, 1968. Academic Press (1969) 25–31. MR254987
  36. [36] L. Si and R. Jin, Adjusting mixture weights of gaussian mixture model via regularized probabilistic latent semantic analysis, in Advances in Knowledge Discovery and Data Mining (2005) 218–252. 
  37. [37] Ch. Stone, The use of polynomial splines and their tensor products in multivariate function estimation. Ann. Stat.22 (1994) 118–171. Zbl0827.62038MR1272079
  38. [38] S. Szarek, Metric entropy of homogeneous spaces, in Quantum Probability (Gdansk 1997) (1998) 395–410. Zbl0927.46047MR1649741
  39. [39] S. van de Geer, The method of sieves and minimum contrast estimators. Math. Methods Stat.4 (1995) 20–38. Zbl0831.62029MR1324688
  40. [40] A. van der Vaart and J. Wellner, Weak Convergence. Springer (1996). MR1385671
  41. [41] I. van Keilegom and N. Veraverbeke, Density and hazard estimation in censored regression models. Bernoulli8 (2002) 607–625. Zbl1007.62029MR1935649
  42. [42] R. Willett and R. Nowak, Multiscale poisson intensity and density estimation. IEEE Trans. Inform. Theory53 (2007) 3171–3187. Zbl1325.94036MR2417680
  43. [43] D. Young and D. Hunter, Mixtures of regressions with predictor-dependent mixing proportions. Comput. Stat. Data Anal.54 (2010) 2253–2266. Zbl1284.62467MR2720486

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.