| Title:
|
Notion of information and independent component analysis (English) |
| Author:
|
Radojičić, Una |
| Author:
|
Nordhausen, Klaus |
| Author:
|
Oja, Hannu |
| Language:
|
English |
| Journal:
|
Applications of Mathematics |
| ISSN:
|
0862-7940 (print) |
| ISSN:
|
1572-9109 (online) |
| Volume:
|
65 |
| Issue:
|
3 |
| Year:
|
2020 |
| Pages:
|
311-330 |
| Summary lang:
|
English |
| . |
| Category:
|
math |
| . |
| Summary:
|
Partial orderings and measures of information for continuous univariate random variables with special roles of Gaussian and uniform distributions are discussed. The information measures and measures of non-Gaussianity including the third and fourth cumulants are generally used as projection indices in the projection pursuit approach for the independent component analysis. The connections between information, non-Gaussianity and statistical independence in the context of independent component analysis is discussed in detail. (English) |
| Keyword:
|
dispersion |
| Keyword:
|
entropy |
| Keyword:
|
kurtosis |
| Keyword:
|
partial ordering |
| MSC:
|
62B10 |
| MSC:
|
62H99 |
| MSC:
|
94A17 |
| idZBL:
|
07217113 |
| idMR:
|
MR4114255 |
| DOI:
|
10.21136/AM.2020.0326-19 |
| . |
| Date available:
|
2020-06-10T13:12:15Z |
| Last updated:
|
2022-07-04 |
| Stable URL:
|
http://hdl.handle.net/10338.dmlcz/148146 |
| . |
| Reference:
|
[1] Barron, A. R.: Entropy and the central limit theorem.Ann. Probab. 14 (1986), 336-342. Zbl 0599.60024, MR 0815975, 10.1214/aop/1176992632 |
| Reference:
|
[2] Bell, A. J., Sejnowski, T. J.: An information-maximization approach to blind separation and blind deconvolution.Neural Comput. 7 (1995), 1129-1159. 10.1162/neco.1995.7.6.1129 |
| Reference:
|
[3] Bickel, P. J., Lehmann, E. L.: Descriptive statistics for nonparametric models. II: Location.Ann. Stat. 3 (1975), 1045-1069. Zbl 0321.62055, MR 0395021, 10.1214/aos/1176343240 |
| Reference:
|
[4] Bickel, P. J., Lehmann, E. L.: Descriptive statistics for nonparametric models. III: Dispersion.Ann. Stat. 4 (1976), 1139-1158. Zbl 0351.62031, MR 474620, 10.1214/aos/1176343648 |
| Reference:
|
[5] Bilodeau, M., Brenner, D.: Theory of Multivariate Statistics.Springer Texts in Statistics, Springer, New York (1999). Zbl 0930.62054, MR 1705291, 10.1007/b97615 |
| Reference:
|
[6] Chernoff, H., Savage, I. R.: Asymptotic normality and efficiency of certain nonparametric test statistics.Ann. Math. Stat. 29 (1958), 972-994. Zbl 0092.36501, MR 0100322, 10.1214/aoms/1177706436 |
| Reference:
|
[7] Cover, T. M., Thomas, J. A.: Elements of Information Theory.Wiley Series in Telecommunications, John Wiley & Sons, New York (1991). Zbl 0762.94001, MR 1122806, 10.1002/0471200611 |
| Reference:
|
[8] Faivishevsky, L., Goldberger, J.: ICA based on a smooth estimation of the differential entropy.NIPS'08: Proceedings of the 21st International Conference on Neural Information Processing Systems Curran Associates, New York (2008), 433-440. |
| Reference:
|
[9] J. L. Hodges, Jr., E. L. Lehmann: The efficiency of some nonparametric competitors of the $t$-test.Ann. Math. Stat. 27 (1956), 324-335. Zbl 0075.29206, MR 0079383, 10.1214/aoms/1177728261 |
| Reference:
|
[10] Huber, P. J.: Projection pursuit.Ann. Stat. 13 (1985), 435-475. Zbl 0595.62059, MR 0790553, 10.1214/aos/1176349519 |
| Reference:
|
[11] Hyvärinen, A.: New approximations of differential entropy for independent component analysis and projection pursuit.NIPS '97: Proceedings of the 1997 Conference on Advances in Neural Information Processing Systems 10 MIT Press, Cambridge (1998), 273-279. |
| Reference:
|
[12] Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis.John Wiley & Sons, New York (2001). 10.1002/0471221317 |
| Reference:
|
[13] Jones, M. C., Sibson, R.: What is projection pursuit?.J. R. Stat. Soc., Ser. A 150 (1987), 1-36. Zbl 0632.62059, MR 0887823, 10.2307/2981662 |
| Reference:
|
[14] Kim, K., Shevlyakov, G.: Why Gaussianity?.IEEE Signal Processing Magazine 25 (2008), 102-113. 10.1109/MSP.2007.913700 |
| Reference:
|
[15] Kristiansson, E.: Decreasing Rearrangement and Lorentz $L(p,q)$ Spaces. Master Thesis.Department of Mathematics, Lulea University of Technology, Lulea (2002), Available at \brokenlink {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.111.1244&rep=rep1{&type=pdf}}\kern0pt. |
| Reference:
|
[16] Kullback, S.: Information Theory and Statistics.Wiley Publication in Mathematical Statistics, John Wiley & Sons, New York (1959). Zbl 0088.10406, MR 0103557 |
| Reference:
|
[17] Learned-Miller, E. G., III, J. W. Fisher: ICA using spacings estimates of entropy.J. Mach. Learn. Res. 4 (2004), 1271-1295. Zbl 1061.62007, MR 2103630, 10.1162/jmlr.2003.4.7-8.1271 |
| Reference:
|
[18] Lindsay, B. G., Yao, W.: Fisher information matrix: A tool for dimension reduction, projection pursuit, independent component analysis, and more.Can. J. Stat. 40 (2012), 712-730. Zbl 1349.62300, MR 2998858, 10.1002/cjs.11166 |
| Reference:
|
[19] Marshall, A. W., Olkin, I.: Inequalities: Theory of Majorization and Its Applications.Mathematics in Science and Engineering 143, Academic Press, New York (1979). Zbl 0437.26007, MR 0552278, 10.1016/B978-0-12-473750-1.50001-1 |
| Reference:
|
[20] Miettinen, J., Nordhausen, K., Oja, H., Taskinen, S.: Deflation-based FastICA with adaptive choices of nonlinearities.IEEE Trans. Signal Process 62 (2014), 5716-5724. Zbl 1394.94394, MR 3273526, 10.1109/TSP.2014.2356442 |
| Reference:
|
[21] Miettinen, J., Nordhausen, K., Oja, H., Taskinen, S., Virta, J.: The squared symmetric \hbox{FastICA} estimator.Signal Process. 131 (2017), 402-411. 10.1016/j.sigpro.2016.08.028 |
| Reference:
|
[22] Miettinen, J., Taskinen, S., Nordhausen, K., Oja, H.: Fourth moments and independent component analysis.Stat. Sci. 30 (2015), 372-390. Zbl 1332.62196, MR 3383886, 10.1214/15-STS520 |
| Reference:
|
[23] Nordhausen, K., Oja, H.: Independent component analysis: A statistical perspective.WIREs Comput. Stat. 10 (2018), Article ID e1440, 23 pages. MR 3850449, 10.1002/wics.1440 |
| Reference:
|
[24] Nordhausen, K., Oja, H.: Robust nonparametric inference.Annu. Rev. Stat. Appl. 5 (2018), 473-500. MR 3774756, 10.1146/annurev-statistics-031017-100247 |
| Reference:
|
[25] Oja, H.: On location, scale, skewness and kurtosis of univariate distributions.Scand. J. Stat., Theory Appl. 8 (1981), 154-168. Zbl 0525.62020, MR 0633040 |
| Reference:
|
[26] Pečarić, J. E., Proschan, F., Tong, Y. L.: Convex Functions, Partial Orderings, and Statistical Applications.Mathematics in Science and Engineering 187, Academic Press, Boston (1992). Zbl 0749.26004, MR 1162312, 10.1016/S0076-5392(13)60006-5 |
| Reference:
|
[27] Ryff, J. V.: On the representation of doubly stochastic operators.Pac. J. Math. 13 (1963), 1379-1386. Zbl 0125.08405, MR 0163171, 10.2140/pjm.1963.13.1379 |
| Reference:
|
[28] Serfling, R.: Asymptotic relative efficiency in estimation.International Encyclopedia of Statistical Science Springer, Berlin (2011), 68-72. 10.1007/978-3-642-04898-2_126 |
| Reference:
|
[29] Shannon, C. E.: A mathematical theory of communication.Bell Syst. Tech. J. 27 (1948), 379-423. Zbl 1154.94303, MR 0026286, 10.1002/j.1538-7305.1948.tb01338.x |
| Reference:
|
[30] Staudte, R. G.: The shapes of things to come: Probability density quantiles.Statistics 51 (2017), 782-800. Zbl 1387.62045, MR 3669289, 10.1080/02331888.2016.1277225 |
| Reference:
|
[31] Staudte, R. G., Xia, A.: Divergence from, and convergence to, uniformity of probability density quantiles.Entropy 20 (2018), Article ID 317, 10 pages. MR 3862507, 10.3390/e20050317 |
| Reference:
|
[32] Zwet, W. R. van: Convex Transformations of Random Variables.Mathematical Centre Tracts 7, Mathematisch Centrum, Amsterdam (1964). Zbl 0125.37102, MR 0176511 |
| Reference:
|
[33] Vigneron, V., Jutten, C.: Fisher information in source separation problems.International Conference on Independent Component Analysis and Signal Separation Lecture Notes in Computer Science 3195, Springer, Berlin (2004), 168-176. 10.1007/978-3-540-30110-3_22 |
| Reference:
|
[34] Virta, J.: On characterizations of the covariance matrix.Available at https://arxiv.org/abs/1810.01147 (2018), 11 pages. |
| Reference:
|
[35] Virta, J., Nordhausen, K.: On the optimal non-linearities for Gaussian mixtures in \hbox{FastICA}.Latent Variable Analysis and Signal Separation Theoretical Computer Science and General Issues, Springer, Berlin (2017), 427-437. 10.1007/978-3-319-53547-0_40 |
| . |
