The LASSO estimator: Distributional properties

Rakshith Jagannath; Neelesh S. Upadhye

Kybernetika (2018)

  • Volume: 54, Issue: 4, page 778-797
  • ISSN: 0023-5954

Abstract

top
The least absolute shrinkage and selection operator (LASSO) is a popular technique for simultaneous estimation and model selection. There have been a lot of studies on the large sample asymptotic distributional properties of the LASSO estimator, but it is also well-known that the asymptotic results can give a wrong picture of the LASSO estimator's actual finite-sample behaviour. The finite sample distribution of the LASSO estimator has been previously studied for the special case of orthogonal models. The aim in this work is to generalize the finite sample distribution properties of LASSO estimator for a real and linear measurement model in Gaussian noise. In this work, we derive an expression for the finite sample characteristic function of the LASSO estimator, we then use the Fourier slice theorem to obtain an approximate expression for the marginal probability density functions of the one-dimensional components of a linear transformation of the LASSO estimator.

How to cite

top

Jagannath, Rakshith, and S. Upadhye, Neelesh. "The LASSO estimator: Distributional properties." Kybernetika 54.4 (2018): 778-797. <http://eudml.org/doc/294354>.

@article{Jagannath2018,
abstract = {The least absolute shrinkage and selection operator (LASSO) is a popular technique for simultaneous estimation and model selection. There have been a lot of studies on the large sample asymptotic distributional properties of the LASSO estimator, but it is also well-known that the asymptotic results can give a wrong picture of the LASSO estimator's actual finite-sample behaviour. The finite sample distribution of the LASSO estimator has been previously studied for the special case of orthogonal models. The aim in this work is to generalize the finite sample distribution properties of LASSO estimator for a real and linear measurement model in Gaussian noise. In this work, we derive an expression for the finite sample characteristic function of the LASSO estimator, we then use the Fourier slice theorem to obtain an approximate expression for the marginal probability density functions of the one-dimensional components of a linear transformation of the LASSO estimator.},
author = {Jagannath, Rakshith, S. Upadhye, Neelesh},
journal = {Kybernetika},
keywords = {linear regression; LASSO; characteristic function; finite sample probability distribution function; Fourier-Slice theorem; Cramer–Wold theorem},
language = {eng},
number = {4},
pages = {778-797},
publisher = {Institute of Information Theory and Automation AS CR},
title = {The LASSO estimator: Distributional properties},
url = {http://eudml.org/doc/294354},
volume = {54},
year = {2018},
}

TY - JOUR
AU - Jagannath, Rakshith
AU - S. Upadhye, Neelesh
TI - The LASSO estimator: Distributional properties
JO - Kybernetika
PY - 2018
PB - Institute of Information Theory and Automation AS CR
VL - 54
IS - 4
SP - 778
EP - 797
AB - The least absolute shrinkage and selection operator (LASSO) is a popular technique for simultaneous estimation and model selection. There have been a lot of studies on the large sample asymptotic distributional properties of the LASSO estimator, but it is also well-known that the asymptotic results can give a wrong picture of the LASSO estimator's actual finite-sample behaviour. The finite sample distribution of the LASSO estimator has been previously studied for the special case of orthogonal models. The aim in this work is to generalize the finite sample distribution properties of LASSO estimator for a real and linear measurement model in Gaussian noise. In this work, we derive an expression for the finite sample characteristic function of the LASSO estimator, we then use the Fourier slice theorem to obtain an approximate expression for the marginal probability density functions of the one-dimensional components of a linear transformation of the LASSO estimator.
LA - eng
KW - linear regression; LASSO; characteristic function; finite sample probability distribution function; Fourier-Slice theorem; Cramer–Wold theorem
UR - http://eudml.org/doc/294354
ER -

References

top
  1. Austin, C. D., Moses, R., Ash, J., Ertin, E., 10.1109/jstsp.2009.2038313, IEEE J. Selected Topics Signal Process. 4 (2010), 3, 560-570. DOI10.1109/jstsp.2009.2038313
  2. Babacan, S., Molina, R., Katsaggelos, A., 10.1109/tip.2009.2032894, IEEE Trans. Image Process. 19 (2010), 1, 53-63. MR2729957DOI10.1109/tip.2009.2032894
  3. Baraniuk, R., Candes, E., Nowak, R., R., M., Vetterli, 10.1109/msp.2008.915557, IEEE Signal Processing Magazine 25 (2008), 2, 12-13. DOI10.1109/msp.2008.915557
  4. Ben-Tal, A., Nemirovskiaei, A. S., 10.1137/1.9780898718829, Society for Industrial and Applied Mathematics, Philadelphia 2001. MR1857264DOI10.1137/1.9780898718829
  5. Boufounos, P., Duarte, M. F., Baraniuk, R. G., 10.1109/ssp.2007.4301267, In: IEEE/SP 14th Workshop on Statistical Signal Processing, 2007, pp. 299-303. MR2883008DOI10.1109/ssp.2007.4301267
  6. Candes, E., 10.1016/j.crma.2008.03.014, Comptes Rendus Mathematique 346 (2008), 9-10, 589-592. MR2412803DOI10.1016/j.crma.2008.03.014
  7. Chen, S. S., Donoho, D. L., Saunders, M. A., 10.1137/s003614450037906x, SIAM Rev. 43 (2001), 1, 129-159. MR1854649DOI10.1137/s003614450037906x
  8. Cuesta-Albertos, J. A., Fraiman, R., Ransford, T., 10.1007/s10959-007-0060-7, J. Theoret. Probab. 20 (2007), 2, 201-209. MR2324526DOI10.1007/s10959-007-0060-7
  9. Donoho, D., 10.1109/tit.2006.871582, IEEE Trans. Inform. Theory 52 (2006), 4, 1289-1306. MR2241189DOI10.1109/tit.2006.871582
  10. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R., 10.1214/009053604000000067, Ann. Statist. 32 (2004), 407-499. MR2060166DOI10.1214/009053604000000067
  11. Eldar, Y., 10.1109/tsp.2008.2008212, IEEE Trans. Signal Process. 57 (2009), 2, 471-481. MR2603376DOI10.1109/tsp.2008.2008212
  12. Fan, J., Li, R., 10.1198/016214501753382273, J. Amer. Statist. Assoc. 96 (2001), 456, 1348-1360. MR1946581DOI10.1198/016214501753382273
  13. Grant, M., Boyd, S., {CVX}: Matlab software for disciplined convex programming, version 2.1. 
  14. Kabaila, P., 10.1017/s0266466600009403, Econometr. Theory 11 (1995), 537-549. MR1349934DOI10.1017/s0266466600009403
  15. Kay, S. M., Fundamentals of Statistical Signal Processing: Estimation Theory., Prentice-Hall, Inc., Upper Saddle River, NJ 1993. 
  16. Knight, K., Fu, W., 10.1214/aos/1015957397, Ann. Statist. 28 (2000), 5, 1356-1378. MR1805787DOI10.1214/aos/1015957397
  17. Krim, H., Viberg, M., 10.1109/79.526899, IEEE Signal Processing Magazine 13 (1996), 4, 67-94. DOI10.1109/79.526899
  18. Leeb, H., Pötscher, B. M., 10.1017/s0266466605050036, Econometr. Theory 21 (2005), 21-59. MR2153856DOI10.1017/s0266466605050036
  19. Lockhart, R., Taylor, J., Tibshirani, R. J., Tibshirani, R., 10.1214/13-aos1175, Ann. Statist. 42 (2014), 2, 413-468. MR3210970DOI10.1214/13-aos1175
  20. Lopes, M. E., Estimating unknown sparsity in compressed sensing., CoRR 2012, abs/1204.4227. 
  21. Ng, R., 10.1145/1073204.1073256, ACM Trans. Graph. 24 (2005), 3, 735-744. DOI10.1145/1073204.1073256
  22. Panahi, A., Viberg, M., 10.1109/lsp.2011.2179534, IEEE Signal Process. Lett. 19 (2012), 2, 79-82. DOI10.1109/lsp.2011.2179534
  23. Pötscher, B. M., Leeb, H., 10.1016/j.jmva.2009.06.010, J. Multivar. Anal. 100 (2009), 9, 2065-2082. MR2543087DOI10.1016/j.jmva.2009.06.010
  24. Tibshirani, R., Regression shrinkage and selection via the lasso., J. Roy. Statist. Soc., Ser. B 58 (1994), 267-288. MR1379242
  25. Zou, H., 10.1198/016214506000000735, J. Amer. Statist. Assoc. 101 (2006), 476, 1418-1429. MR2279469DOI10.1198/016214506000000735

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.