Notion of information and independent component analysis

Una Radojičić; Klaus Nordhausen; Hannu Oja

Applications of Mathematics (2020)

  • Volume: 65, Issue: 3, page 311-330
  • ISSN: 0862-7940

Abstract

top
Partial orderings and measures of information for continuous univariate random variables with special roles of Gaussian and uniform distributions are discussed. The information measures and measures of non-Gaussianity including the third and fourth cumulants are generally used as projection indices in the projection pursuit approach for the independent component analysis. The connections between information, non-Gaussianity and statistical independence in the context of independent component analysis is discussed in detail.

How to cite

top

Radojičić, Una, Nordhausen, Klaus, and Oja, Hannu. "Notion of information and independent component analysis." Applications of Mathematics 65.3 (2020): 311-330. <http://eudml.org/doc/297066>.

@article{Radojičić2020,
abstract = {Partial orderings and measures of information for continuous univariate random variables with special roles of Gaussian and uniform distributions are discussed. The information measures and measures of non-Gaussianity including the third and fourth cumulants are generally used as projection indices in the projection pursuit approach for the independent component analysis. The connections between information, non-Gaussianity and statistical independence in the context of independent component analysis is discussed in detail.},
author = {Radojičić, Una, Nordhausen, Klaus, Oja, Hannu},
journal = {Applications of Mathematics},
keywords = {dispersion; entropy; kurtosis; partial ordering},
language = {eng},
number = {3},
pages = {311-330},
publisher = {Institute of Mathematics, Academy of Sciences of the Czech Republic},
title = {Notion of information and independent component analysis},
url = {http://eudml.org/doc/297066},
volume = {65},
year = {2020},
}

TY - JOUR
AU - Radojičić, Una
AU - Nordhausen, Klaus
AU - Oja, Hannu
TI - Notion of information and independent component analysis
JO - Applications of Mathematics
PY - 2020
PB - Institute of Mathematics, Academy of Sciences of the Czech Republic
VL - 65
IS - 3
SP - 311
EP - 330
AB - Partial orderings and measures of information for continuous univariate random variables with special roles of Gaussian and uniform distributions are discussed. The information measures and measures of non-Gaussianity including the third and fourth cumulants are generally used as projection indices in the projection pursuit approach for the independent component analysis. The connections between information, non-Gaussianity and statistical independence in the context of independent component analysis is discussed in detail.
LA - eng
KW - dispersion; entropy; kurtosis; partial ordering
UR - http://eudml.org/doc/297066
ER -

References

top
  1. Barron, A. R., 10.1214/aop/1176992632, Ann. Probab. 14 (1986), 336-342. (1986) Zbl0599.60024MR0815975DOI10.1214/aop/1176992632
  2. Bell, A. J., Sejnowski, T. J., 10.1162/neco.1995.7.6.1129, Neural Comput. 7 (1995), 1129-1159. (1995) DOI10.1162/neco.1995.7.6.1129
  3. Bickel, P. J., Lehmann, E. L., 10.1214/aos/1176343240, Ann. Stat. 3 (1975), 1045-1069. (1975) Zbl0321.62055MR0395021DOI10.1214/aos/1176343240
  4. Bickel, P. J., Lehmann, E. L., 10.1214/aos/1176343648, Ann. Stat. 4 (1976), 1139-1158. (1976) Zbl0351.62031MR474620DOI10.1214/aos/1176343648
  5. Bilodeau, M., Brenner, D., 10.1007/b97615, Springer Texts in Statistics, Springer, New York (1999). (1999) Zbl0930.62054MR1705291DOI10.1007/b97615
  6. Chernoff, H., Savage, I. R., 10.1214/aoms/1177706436, Ann. Math. Stat. 29 (1958), 972-994. (1958) Zbl0092.36501MR0100322DOI10.1214/aoms/1177706436
  7. Cover, T. M., Thomas, J. A., 10.1002/0471200611, Wiley Series in Telecommunications, John Wiley & Sons, New York (1991). (1991) Zbl0762.94001MR1122806DOI10.1002/0471200611
  8. Faivishevsky, L., Goldberger, J., ICA based on a smooth estimation of the differential entropy, NIPS'08: Proceedings of the 21st International Conference on Neural Information Processing Systems Curran Associates, New York (2008), 433-440. (2008) 
  9. J. L. Hodges, Jr., E. L. Lehmann, 10.1214/aoms/1177728261, Ann. Math. Stat. 27 (1956), 324-335. (1956) Zbl0075.29206MR0079383DOI10.1214/aoms/1177728261
  10. Huber, P. J., 10.1214/aos/1176349519, Ann. Stat. 13 (1985), 435-475. (1985) Zbl0595.62059MR0790553DOI10.1214/aos/1176349519
  11. Hyvärinen, A., New approximations of differential entropy for independent component analysis and projection pursuit, NIPS '97: Proceedings of the 1997 Conference on Advances in Neural Information Processing Systems 10 MIT Press, Cambridge (1998), 273-279. (1998) 
  12. Hyvärinen, A., Karhunen, J., Oja, E., 10.1002/0471221317, John Wiley & Sons, New York (2001). (2001) DOI10.1002/0471221317
  13. Jones, M. C., Sibson, R., 10.2307/2981662, J. R. Stat. Soc., Ser. A 150 (1987), 1-36. (1987) Zbl0632.62059MR0887823DOI10.2307/2981662
  14. Kim, K., Shevlyakov, G., 10.1109/MSP.2007.913700, IEEE Signal Processing Magazine 25 (2008), 102-113. (2008) DOI10.1109/MSP.2007.913700
  15. Kristiansson, E., Decreasing Rearrangement and Lorentz L ( p , q ) Spaces. Master Thesis, Department of Mathematics, Lulea University of Technology, Lulea (2002), Available at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.111.1244rep=rep1type=pdf. (2002) 
  16. Kullback, S., Information Theory and Statistics, Wiley Publication in Mathematical Statistics, John Wiley & Sons, New York (1959). (1959) Zbl0088.10406MR0103557
  17. Learned-Miller, E. G., III, J. W. Fisher, 10.1162/jmlr.2003.4.7-8.1271, J. Mach. Learn. Res. 4 (2004), 1271-1295. (2004) Zbl1061.62007MR2103630DOI10.1162/jmlr.2003.4.7-8.1271
  18. Lindsay, B. G., Yao, W., 10.1002/cjs.11166, Can. J. Stat. 40 (2012), 712-730. (2012) Zbl1349.62300MR2998858DOI10.1002/cjs.11166
  19. Marshall, A. W., Olkin, I., 10.1016/B978-0-12-473750-1.50001-1, Mathematics in Science and Engineering 143, Academic Press, New York (1979). (1979) Zbl0437.26007MR0552278DOI10.1016/B978-0-12-473750-1.50001-1
  20. Miettinen, J., Nordhausen, K., Oja, H., Taskinen, S., 10.1109/TSP.2014.2356442, IEEE Trans. Signal Process 62 (2014), 5716-5724. (2014) Zbl1394.94394MR3273526DOI10.1109/TSP.2014.2356442
  21. Miettinen, J., Nordhausen, K., Oja, H., Taskinen, S., Virta, J., 10.1016/j.sigpro.2016.08.028, Signal Process. 131 (2017), 402-411. (2017) DOI10.1016/j.sigpro.2016.08.028
  22. Miettinen, J., Taskinen, S., Nordhausen, K., Oja, H., 10.1214/15-STS520, Stat. Sci. 30 (2015), 372-390. (2015) Zbl1332.62196MR3383886DOI10.1214/15-STS520
  23. Nordhausen, K., Oja, H., 10.1002/wics.1440, WIREs Comput. Stat. 10 (2018), Article ID e1440, 23 pages. (2018) MR3850449DOI10.1002/wics.1440
  24. Nordhausen, K., Oja, H., 10.1146/annurev-statistics-031017-100247, Annu. Rev. Stat. Appl. 5 (2018), 473-500. (2018) MR3774756DOI10.1146/annurev-statistics-031017-100247
  25. Oja, H., On location, scale, skewness and kurtosis of univariate distributions, Scand. J. Stat., Theory Appl. 8 (1981), 154-168. (1981) Zbl0525.62020MR0633040
  26. Pečarić, J. E., Proschan, F., Tong, Y. L., 10.1016/S0076-5392(13)60006-5, Mathematics in Science and Engineering 187, Academic Press, Boston (1992). (1992) Zbl0749.26004MR1162312DOI10.1016/S0076-5392(13)60006-5
  27. Ryff, J. V., 10.2140/pjm.1963.13.1379, Pac. J. Math. 13 (1963), 1379-1386. (1963) Zbl0125.08405MR0163171DOI10.2140/pjm.1963.13.1379
  28. Serfling, R., 10.1007/978-3-642-04898-2_126, International Encyclopedia of Statistical Science Springer, Berlin (2011), 68-72. (2011) DOI10.1007/978-3-642-04898-2_126
  29. Shannon, C. E., 10.1002/j.1538-7305.1948.tb01338.x, Bell Syst. Tech. J. 27 (1948), 379-423. (1948) Zbl1154.94303MR0026286DOI10.1002/j.1538-7305.1948.tb01338.x
  30. Staudte, R. G., 10.1080/02331888.2016.1277225, Statistics 51 (2017), 782-800. (2017) Zbl1387.62045MR3669289DOI10.1080/02331888.2016.1277225
  31. Staudte, R. G., Xia, A., 10.3390/e20050317, Entropy 20 (2018), Article ID 317, 10 pages. (2018) MR3862507DOI10.3390/e20050317
  32. Zwet, W. R. van, Convex Transformations of Random Variables, Mathematical Centre Tracts 7, Mathematisch Centrum, Amsterdam (1964). (1964) Zbl0125.37102MR0176511
  33. Vigneron, V., Jutten, C., 10.1007/978-3-540-30110-3_22, International Conference on Independent Component Analysis and Signal Separation Lecture Notes in Computer Science 3195, Springer, Berlin (2004), 168-176. (2004) DOI10.1007/978-3-540-30110-3_22
  34. Virta, J., On characterizations of the covariance matrix, Available at https://arxiv.org/abs/1810.01147 (2018), 11 pages. (2018) 
  35. Virta, J., Nordhausen, K., 10.1007/978-3-319-53547-0_40, Latent Variable Analysis and Signal Separation Theoretical Computer Science and General Issues, Springer, Berlin (2017), 427-437. (2017) DOI10.1007/978-3-319-53547-0_40

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.