Article
Full entry | Fulltext not available
(moving wall
24 months)
Feedback
Keywords:
covariance matrix; ridge estimation; two-class data; contamination
covariance matrix; ridge estimation; two-class data; contamination
Summary:
This paper deals with the problem of estimating a covariance matrix from the data in two classes: (1) good data with the covariance matrix of interest and (2) contamination coming from a Gaussian distribution with a different covariance matrix. The ridge penalty is introduced to address the problem of high-dimensional challenges in estimating the covariance matrix from the two-class data model. A ridge estimator of the covariance matrix has a uniform expression and keeps positive-definite, whether the data size is larger or smaller than the data dimension. Furthermore, the ridge parameter is tuned through a cross-validation procedure. Lastly, the proposed ridge estimator is verified with better performance than the existing estimator from the data in two classes and the traditional ridge estimator only from the good data.
References:
[1] Ahsanullah, M., Nevzorov, V. B.: Generalized spacings of order statistics from extended sample. J. Stat. Plann. Inference 85 (2000), 75-83. DOI 10.1016/S0378-3758(99)00067-1 | MR 1759240 | Zbl 0968.62017
[2] Besson, O.: Maximum likelihood covariance matrix estimation from two possibly mismatched data sets. Signal Process. 167 (2020), Article ID 107285, 9 pages. DOI 10.1016/j.sigpro.2019.107285
[3] Bhatia, R.: Positive Definite Matrices. Princeton Series in Applied Mathematics. Princeton University Press, Princeton (2007). DOI 10.1515/9781400827787 | MR 3443454 | Zbl 1133.15017
[4] Bien, J., Tibshirani, R. J.: Sparse estimation of a covariance matrix. Biometrika 98 (2011), 807-820. DOI 10.1093/biomet/asr054 | MR 2860325 | Zbl 1228.62063
[5] Bodnar, O., Bodnar, T., Parolya, N.: Recent advances in shrinkage-based high-dimensional inference. J. Multivariate Anal. 188 (2022), Article ID 104826, 13 pages. DOI 10.1016/j.jmva.2021.104826 | MR 4353848 | Zbl 1493.62298
[6] Cho, S., Katayama, S., Lim, J., Choi, Y.-G.: Positive-definite modification of a covariance matrix by minimizing the matrix $\ell_{\infty}$ norm with applications to portfolio optimization. AStA, Adv. Stat. Anal. 105 (2021), 601-627. DOI 10.1007/s10182-021-00396-7 | MR 4340896 | Zbl 1478.62118
[7] Danaher, P., Wang, P., Witten, D. M.: The joint graphical lasso for inverse covariance estimation across multiple classes. J. R. Stat. Soc., Ser. B, Stat. Methodol. 76 (2014), 373-397. DOI 10.1111/rssb.12033 | MR 3164871 | Zbl 07555455
[8] Fisher, T. J., Sun, X.: Improved Stein-type shrinkage estimators for the high-dimensional multivariate normal covariance matrix. Comput. Stat. Data Anal. 55 (2011), 1909-1918. DOI 10.1016/j.csda.2010.12.006 | MR 2765053 | Zbl 1328.62336
[9] Götze, F., Tikhomirov, A.: Rate of convergence in probability to the Marchenko-Pastur law. Bernoulli 10 (2004), 503-548. DOI 10.3150/bj/1089206408 | MR 2061442 | Zbl 1049.60018
[10] Hannart, A., Naveau, P.: Estimating high dimensional covariance matrices: A new look at the Gaussian conjugate framework. J. Multivariate Anal. 131 (2014), 149-162. DOI 10.1016/j.jmva.2014.06.001 | MR 3252641 | Zbl 1306.62120
[11] Hoshino, N., Takemura, A.: On reduction of finite-sample variance by extended Latin hypercube sampling. Bernoulli 6 (2000), 1035-1050. DOI 10.2307/3318470 | MR 1809734 | Zbl 0979.65005
[12] Huang, C., Farewell, D., Pan, J.: A calibration method for non-positive definite covariance matrix in multivariate data analysis. J. Multivariate Anal. 157 (2017), 45-52. DOI 10.1016/j.jmva.2017.03.001 | MR 3641735 | Zbl 1362.62136
[13] Huang, J. Z., Liu, N., Pourahmadi, M., Liu, L.: Covariance matrix selection and estimation via penalised normal likelihood. Biometrika 93 (2006), 85-98. DOI 10.1093/biomet/93.1.85 | MR 2277742 | Zbl 1152.62346
[14] Jia, S., Zhang, C., Lu, H.: Covariance function versus covariance matrix estimation in efficient semi-parametric regression for longitudinal data analysis. J. Multivariate Anal. 187 (2022), Article ID 104900, 14 pages. DOI 10.1016/j.jmva.2021.104900 | MR 4339021 | Zbl 1480.62098
[15] Kalina, J., Tebbens, J. D.: Algorithms for regularized linear discriminant analysis. Proceedings of the International Conference on Bioinformatics Models, Methods and Algorithms Scitepress, Setúbal (2015), 128-133. DOI 10.5220/0005234901280133
[16] Kochan, N., Tütüncü, G. Y., Giner, G.: A new local covariance matrix estimation for the classification of gene expression profiles in high dimensional RNA-Seq data. Expert Systems Appl. 167 (2021), Article ID 114200, 5 pages. DOI 10.1016/j.eswa.2020.114200
[17] Le, C. M., Levin, K., Bickel, P. J., Levina, E.: Comment: Ridge regression and regularization of large matrices. Technometrics 62 (2020), 443-446. DOI 10.1080/00401706.2020.1796815 | MR 4165992
[18] Ledoit, O., Wolf, M.: A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 (2004), 365-411. DOI 10.1016/S0047-259X(03)00096-4 | MR 2026339 | Zbl 1032.62050
[19] Li, C.-N., Ren, P.-W., Guo, Y.-R., Ye, Y.-F., Shao, Y.-H.: Regularized linear discriminant analysis based on generalized capped $\ell_{2,q}$-norm. (to appear) in Ann. Oper. Res. DOI 10.1007/s10479-022-04959-y
[20] Lim, L.-H., Sepulchre, R., Ye, K.: Geometric distance between positive definite matrices of different dimensions. IEEE Trans. Inf. Theory 65 (2019), 5401-5405. DOI 10.1109/TIT.2019.2913874 | MR 4009241 | Zbl 1432.15033
[21] Massignan, J. A. D., London, J. B. A., Bessani, M., Maciel, C. D., Fannucchi, R. Z., Miranda, V.: Bayesian inference approach for information fusion in distribution system state estimation. IEEE Trans. Smart Grid 13 (2022), 526-540. DOI 10.1109/TSG.2021.3128053
[22] Mestre, X.: On the asymptotic behavior of the sample estimates of eigenvalues and eigenvectors of covariance matrices. IEEE Trans. Signal Process. 56 (2008), 5353-5368. DOI 10.1109/TSP.2008.929662 | MR 2472837 | Zbl 1391.62092
[23] Raninen, E., Ollila, E.: Coupled regularized sample covariance matrix estimator for multiple classes. IEEE Trans. Signal Process. 69 (2021), 5681-5692. DOI 10.1109/TSP.2021.3118546 | MR 4332948
[24] Raninen, E., Tyler, D. E., Ollila, E.: Linear pooling of sample covariance matrices. IEEE Trans. Signal Process. 70 (2022), 659-672. DOI 10.1109/TSP.2021.3139207 | MR 4381805
[25] Scheidegger, C., Hörrmann, J., Bühlmann, P.: The weighted generalised covariance measure. J. Mach. Learn. Res. 23 (2022), Article ID 273, 68 pages. MR 4577712
[26] Tsukuma, H., Kubokawa, T.: Unified improvements in estimation of a normal covariance matrix in high and low dimensions. J. Multivariate Anal. 143 (2016), 233-248. DOI 10.1016/j.jmva.2015.09.016 | MR 3431430 | Zbl 1328.62348
[27] Wieringen, W. N. van, Peeters, C. F. W.: Ridge estimation of inverse covariance matrices from high-dimensional data. Comput. Stat. Data Anal. 103 (2016), 284-303. DOI 10.1016/j.csda.2016.05.012 | MR 3522633 | Zbl 1466.62204
[28] Vershynin, R.: How close is the sample covariance matrix to the actual covariance matrix?. J. Theor. Probab. 25 (2012), 655-686. DOI 10.1007/s10959-010-0338-z | MR 2956207 | Zbl 1365.62208
[29] Wang, H., Peng, B., Li, D., Leng, C.: Nonparametric estimation of large covariance matrices with conditional sparsity. J. Econom. 223 (2021), 53-72. DOI 10.1016/j.jeconom.2020.09.002 | MR 4252147 | Zbl 1471.62378
[30] Warton, D. I.: Penalized normal likelihood and ridge regularization of correlation and covariance matrices. J. Am. Stat. Assoc. 103 (2008), 340-349. DOI 10.1198/016214508000000021 | MR 2394637 | Zbl 1471.62362
[31] Witten, D. M., Tibshirani, R.: Covariance-regularized regression and classification for high dimensional problems. J. R. Stat. Soc., Ser. B, Stat. Methodol. 71 (2009), 615-636. DOI 10.1111/j.1467-9868.2009.00699.x | MR 2749910 | Zbl 1250.62033
[32] Xi, B., Li, J., Li, Y., Song, R., Hong, D., Chanussot, J.: Few-shot learning with class-co-variance metric for hyperspectral image classification. IEEE Trans. Image Process. 31 (2022), 5079-5092. DOI 10.1109/TIP.2022.3192712
[33] Xue, L., Ma, S., Zou, H.: Positive-definite $\ell_1$-penalized estimation of large covariance matrices. J. Am. Stat. Assoc. 107 (2012), 1480-1491. DOI 10.1080/01621459.2012.725386 | MR 3036409 | Zbl 1258.62063
[34] Yang, Y., Zhou, J., Pan, J.: Estimation and optimal structure selection of high-dimensional Toeplitz covariance matrix. J. Multivariate Anal. 184 (2021), Article ID 104739, 17 pages. DOI 10.1016/j.jmva.2021.104739 | MR 4236460 | Zbl 1467.62095
[35] Yin, Y.: Spectral statistics of high dimensional sample covariance matrix with unbounded population spectral norm. Bernoulli 28 (2022), 1729-1756. DOI 10.3150/21-BEJ1391 | MR 4411509 | Zbl 07526604
[36] Yuasa, R., Kubokawa, T.: Ridge-type linear shrinkage estimation of the mean matrix of a high-dimensional normal distribution. J. Multivariate Anal. 178 (2020), Article ID 104608, 18 pages. DOI 10.1016/j.jmva.2020.104608 | MR 4079038 | Zbl 1440.62036
[37] Zhang, H., Jia, J.: Elastic-net regularized high-dimensional negative binomial regression: Consistency and weak signals detection. Stat. Sin. 32 (2022), 181-207. DOI 10.5705/ss.202019.0315 | MR 4359629 | Zbl 07484115
[38] Zhang, Y., Zhou, Y., Liu, X.: Applications on linear spectral statistics of high-dimensional sample covariance matrix with divergent spectrum. Comput. Stat. Data Anal. 178 (2023), Article ID 107617, 19 pages. DOI 10.1016/j.csda.2022.107617 | MR 4483317 | Zbl 07626679
Search
Partner of
