We estimated convex-structured covariance/correlation matrices by minimizing the entropy loss corresponding to the given matrix. We first considered the estimation of the Weighted sum of known Rank-one matrices with unknown Weights (W-Rank1-W) structural covariance matrices, which appeared commonly in array signal processing tasks, e.g., direction-of-arrival (DOA) estimation. The associated minimization problem is convex and can be solved using the primal-dual interior-point algorithm. However, the objective functions (the entropy loss function) can be bounded above by a sequence of separable functions—we proposed a novel estimation algorithm based on this property under the Majorization-Minimization (MM) algorithmic framework. The proposed MM algorithm exhibited very low computational complexity in each iteration, and its convergence was demonstrated theoretically. Subsequently, we focused on the estimation of Toeplitz autocorrelation matrices, which appeared frequently in time-series analysis. In particular, we considered cases in which the autocorrelation coefficient decreased as the time lag increased. We transformed the Toeplitz structure into a W-Rank1-W structure via special variable substitution, and proposed an MM algorithm similar to that for the W-Rank1-W covariance estimation. However, each MM iteration involved a second-order cone programming SOCP problem that must be resolved. Our numerical experiments demonstrated the high computational efficiency and satisfactory estimation accuracy of the proposed MM algorithms in DOA and autocorrelation matrix estimation.
Citation: Chen Chen, Xiangbing Chen, Yi Ai. Convex-structured covariance estimation via the entropy loss under the majorization-minimization algorithm framework[J]. AIMS Mathematics, 2024, 9(6): 14253-14273. doi: 10.3934/math.2024692
We estimated convex-structured covariance/correlation matrices by minimizing the entropy loss corresponding to the given matrix. We first considered the estimation of the Weighted sum of known Rank-one matrices with unknown Weights (W-Rank1-W) structural covariance matrices, which appeared commonly in array signal processing tasks, e.g., direction-of-arrival (DOA) estimation. The associated minimization problem is convex and can be solved using the primal-dual interior-point algorithm. However, the objective functions (the entropy loss function) can be bounded above by a sequence of separable functions—we proposed a novel estimation algorithm based on this property under the Majorization-Minimization (MM) algorithmic framework. The proposed MM algorithm exhibited very low computational complexity in each iteration, and its convergence was demonstrated theoretically. Subsequently, we focused on the estimation of Toeplitz autocorrelation matrices, which appeared frequently in time-series analysis. In particular, we considered cases in which the autocorrelation coefficient decreased as the time lag increased. We transformed the Toeplitz structure into a W-Rank1-W structure via special variable substitution, and proposed an MM algorithm similar to that for the W-Rank1-W covariance estimation. However, each MM iteration involved a second-order cone programming SOCP problem that must be resolved. Our numerical experiments demonstrated the high computational efficiency and satisfactory estimation accuracy of the proposed MM algorithms in DOA and autocorrelation matrix estimation.
| [1] | S. Haykin, J. Litva, T. Shepherd, Radar array processing, Heidelberg: Springer-Verlag, 1993. https://doi.org/10.1007/978-3-642-77347-1 |
| [2] | Z. Yang, J. Li, P. Stoica, L. Xie, Sparse methods for direction-of-arrival estimation, In: Academic press library in signal processing, volume 7: array, radar and communications engineering, London: Academic Press, 2018,509–581. https://doi.org/10.1016/B978-0-12-811887-0.00011-0 |
| [3] |
X. Wu, X. Yang, X. Jia, F. Tian, A gridless DOA estimation method based on convolutional neural network with Toeplitz prior, IEEE Signal Proc. Lett., 29 (2022), 1247–1251. https://doi.org/10.1109/LSP.2022.3176211 doi: 10.1109/LSP.2022.3176211
|
| [4] | J. Hamilton, Time series analysis, Princeton: Princeton University Press, 1994. |
| [5] |
T. Cai, Z. Ren, H. Zhou, Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation, Electron. J. Stat., 10 (2016), 1–59. https://doi.org/10.1214/15-EJS1081 doi: 10.1214/15-EJS1081
|
| [6] |
Y. Sun, P. Babu, D. Palomar, Robust estimation of structured covariance matrix for heavy-tailed elliptical distributions, IEEE T. Signal Proces., 64 (2016), 3576–3590. https://doi.org/10.1109/TSP.2016.2546222 doi: 10.1109/TSP.2016.2546222
|
| [7] |
A. Mieldzioc, M. Mokrzycka, A. Sawikowska, Covariance regularization for metabolomic data on the drought resistance of barley, Biometrical Letters, 56 (2019), 165–181. https://doi.org/10.2478/bile-2019-0010 doi: 10.2478/bile-2019-0010
|
| [8] |
K. Filipiak, D. Klein, A. Markiewicz, M. Mokrzycka, Approximation with a kronecker product structure with one component as compound symmetry or autoregression via entropy loss function, Linear Algebra Appl., 610 (2021), 625–646. https://doi.org/10.1016/j.laa.2020.10.013 doi: 10.1016/j.laa.2020.10.013
|
| [9] |
D. Ramírez, G. Vazquez-Vilar, R. López-Valcarce, J. Vía, I. Santamaría, Detection of rank-p signals in cognitive radionetworks with uncalibrated multiple antennas, IEEE T. Signal Proces., 59 (2011), 3764–3774. https://doi.org/10.1109/TSP.2011.2146779 doi: 10.1109/TSP.2011.2146779
|
| [10] |
O. Besson, Adaptive detection of gaussian rank-one signalsusing adaptively whitened data and rao, gradient and durbin tests, IEEE Signal Proc. Lett., 30 (2023), 399–402. https://doi.org/10.1109/LSP.2023.3265802 doi: 10.1109/LSP.2023.3265802
|
| [11] |
P. Stoica, P. Babu, J. Li, SPICE: a sparse covariance-based estimation method for array processing, IEEE T. Signal Proces., 59 (2011), 629–638. https://doi.org/10.1109/TSP.2010.2090525 doi: 10.1109/TSP.2010.2090525
|
| [12] |
P. Stoica, P. Babu, SPICE and LIKES: two hyperparameter-free methods for sparse-parameter estimation, Signal Process., 92 (2012), 1580–1590. https://doi.org/10.1016/j.sigpro.2011.11.010 doi: 10.1016/j.sigpro.2011.11.010
|
| [13] |
C. Chen, J. Zhou, M. Tang, Direction of arrival estimation in elliptical models via sparse penalized likelihood approach, Sensors, 19 (2019), 2356. https://doi.org/10.3390/s19102356 doi: 10.3390/s19102356
|
| [14] |
S. Kullback, R. Leibler, On information and sufficiency, Ann. Math. Statist., 22 (1951), 79–86. https://doi.org/10.1214/aoms/1177729694 doi: 10.1214/aoms/1177729694
|
| [15] |
Y. Sun, P. Babu, D. Palomar, Majorization-minimization algorithms in signal processing, communications, and machine learning, IEEE T. Signal Proces., 65 (2017), 794–816. https://doi.org/10.1109/TSP.2016.2601299 doi: 10.1109/TSP.2016.2601299
|
| [16] |
T. Cai, Z. Ren, H. Zhou, Optimal rates of convergence for estimating toeplitz covariance matrices, Probab. Theory Relat. Fields, 156 (2013), 101–143. https://doi.org/10.1007/s00440-012-0422-7 doi: 10.1007/s00440-012-0422-7
|
| [17] |
H. Li, P. Stoica, J. Li, Computationally efficient maximum likelihood estimation of structured covariance matrices, IEEE T. Signal Proces., 47 (1999), 1314–1323. https://doi.org/10.1109/78.757219 doi: 10.1109/78.757219
|
| [18] |
K. Filipiak, A. Markiewicz, A. Mieldzioc, A. Sawikowska, On projection of a positive definite matrix on a cone of nonnegative definite Toeplitz matrices, Electron. J. Linear Al., 33 (2018), 74–82. https://doi.org/10.13001/1081-3810.3750 doi: 10.13001/1081-3810.3750
|
| [19] |
L. Lin, N. Higham, J. Pan, Covariance structure regularization via entropy loss function, Comput. Stat. Data Anal., 72 (2014), 315–327. https://doi.org/10.1016/j.csda.2013.10.004 doi: 10.1016/j.csda.2013.10.004
|
| [20] |
C. Chen, J. Zhou, J. Pan, Correlation structure regularization via entropy loss function for high-dimension and low-sample-size data, Commun. Stat.-Simul. Comput., 50 (2021), 993–1008. https://doi.org/10.1080/03610918.2019.1571607 doi: 10.1080/03610918.2019.1571607
|
| [21] |
Y. Yang, J. Zhou, J. Pan, Estimation and optimal structure selection of high-dimensional toeplitz covariance matrix, J. Multivariate Anal., 184 (2021), 104739. https://doi.org/10.1016/j.jmva.2021.104739 doi: 10.1016/j.jmva.2021.104739
|
| [22] |
C. Stein, Lectures on the theory of estimation of many parameters, Journal of Soviet Mathematics, 34 (1986), 1373–1403. https://doi.org/10.1007/BF01085007 doi: 10.1007/BF01085007
|
| [23] | S. Boyd, L. Vandenberghe, Convex optimization, Cambridge: Cambridge University Press, 2004. |
| [24] |
B. Zhang, S. Yuan, Shrinkage estimators of large covariance matrices with Toeplitz targets in array signal processing, Sci. Rep., 12 (2022), 19032. https://doi.org/10.1038/s41598-022-21889-8 doi: 10.1038/s41598-022-21889-8
|
| [25] |
A. Dembo, C. Mallows, L. Shepp, Embedding nonnegative definite toeplitz matrices in nonnegative definite circulant matrices with application to covariance estimation, IEEE T. Inform. Theory, 35 (1989), 1206–1212. https://doi.org/10.1109/18.45276 doi: 10.1109/18.45276
|
| [26] | F. Zhang, The Schur complement and its applications, New York: Springer, 2005. https://doi.org/10.1007/b105056 |
Figures(3) / Tables(3)
Chen Chen, Xiangbing Chen, Yi Ai. Convex-structured covariance estimation via the entropy loss under the majorization-minimization algorithm framework[J]. AIMS Mathematics, 2024, 9(6): 14253-14273. doi: 10.3934/math.2024692
DownLoad: